Hostname: page-component-cd9895bd7-dk4vv Total loading time: 0 Render date: 2024-12-22T11:22:46.982Z Has data issue: false hasContentIssue false

Protocol for process evaluation of integration of mental health into primary healthcare in two states in Nigeria: the mhSUN programme

Published online by Cambridge University Press:  15 February 2021

Julian Eaton*
Affiliation:
Centre for Global Mental Health, London School of Hygiene and Tropical Medicine, UK
Yusuf Akande
Affiliation:
Research Unit, Department of Clinical Services, Federal Neuropsychiatric Hospital Kaduna, Nigeria
Uchechi Onukogu
Affiliation:
Department of Clinical Psychology, Federal Neuropsychiatric Hospital Calabar, Nigeria
Emeka Nwefoh
Affiliation:
Mental Health Department, CBM Country Office, Nigeria
Taiwo Lateef Sheikh
Affiliation:
Department of Psychiatry, Ahmadu Bello University College of Medical Sciences, Nigeria
Ekpe Essien Ekpe
Affiliation:
Department of Clinical Services, Federal Neuropsychiatric Hospital Calabar, Nigeria
Oye Gureje
Affiliation:
Department of Psychiatry, University of Ibadan College of Medicine, Nigeria
*
Correspondence: Julian Eaton. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Background

Current international recommendations to address the large treatment gap for mental healthcare in low- and middle-income countries are to scale up integration of mental health into primary care. There are good outcome studies to support this, but less robust evidence for effectively carrying out integration and scale-up of such services, or for understanding how to address contextual issues that routinely arise.

Aims

This protocol is for a process evaluation of a programme called Mental Health Scale Up Nigeria. The study aims are to determine the extent to which the intervention was carried out according to the plans developed (fidelity), to examine the effect of postulated moderating factors and local context, and the perception of the programme by primary care staff and implementers.

Method

We use a theoretical framework for process evaluation based on the Medical Research Council's Guidelines on Process Evaluation. A Theory of Change workshop was carried out in programme development, to highlight relevant factors influencing the process, ensure good adaptation of global normative guidelines and gain buy-in from local stakeholders. We will use mixed methods to examine programme implementation and outcomes, and influence of moderating factors.

Results

Data sources will include the routine health information system, facility records (for staff, medication and infrastructure), log books of intervention activities, supervision records, patient questionnaires and qualitative interviews.

Conclusions

Evidence from this process evaluation will help guide implementers aiming to scale up mental health services in primary care in low- and middle-income countries.

Type
Papers
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © The Authors, 2021. Published by Cambridge University Press on behalf of the Royal College of Psychiatrists

Increasing access to evidence-based mental healthcare, or reducing the treatment gap, is a central priority of global mental health.1 In low- and middle-income countries (LMICs), reform of overly centralised services has focused on delivering mental health interventions in community or primary care settings,Reference Eaton, McCay, Semrau, Chatterjee, Baingana and Araya2 making use of existing infrastructure and systems where possible, for efficiency, sustainability, local ownership and equity.Reference Patel, Araya, Chatterjee, Chisholm, Cohen and DeSilva3 In the past 20 years, a substantial body of evidence has been established demonstrating the efficacy in terms of clinical, social and disability outcomes of programmes that use such models.Reference Patel, Araya, Chatterjee, Chisholm, Cohen and DeSilva3,Reference Araya, Rojas, Fritsch, Gaete, Rojas and Simon4

A key concern of any global health field is the degree to which recommended approaches are acceptable, feasible or effective in different settings. The mandate of the World Health Organization (WHO) is, in part, to make available normative guidelines that can be applied in different settings, and the mental health Gap Action Programme (mhGAP) was developed for this purpose.5 Despite the fact that relevance to low- and middle-income settings was a clearly stated consideration of the mhGAP Intervention Guide,Reference Barbui, Dua, van Ommeren, Yasamy, Fleischmann and Clark6 and there is a wide range of reports of mhGAP use in formal and grey literature, including a systematic review,Reference Keynejad, Dua, Barbui and Thornicroft7 there remains a question as to local applicability of the guidelines in the vastly different contexts in which they are to be applied.Reference Gómez-Carrillo, Lencucha, Faregh, Veissière and Kirmayer8,Reference Mills and Hilberg9

Although evidence on outcomes is essential to demonstrate the efficacy of mhGAP-based interventions, most evaluations in global mental health have been limited in terms of fidelity testing or formal analysis of the role of context, despite the complexity of such interventions, and uncertainty in relation to causality.Reference Ling10 Most studies have either been either formative in nature or carried out with tightly managed experimental methods, such as randomised controlled trials. Cost-effectiveness evaluation,Reference Chisholm, James, Sekar, Kumar, Murthy and Saeed11 and some assessment of client acceptability and feasibility,Reference Asher, Fekadu, Hanlon, Mideksa, Eaton and Patel12 have generally been the main methods of additional exploration of these wider questions, usually as a part of larger outcome studies. More recently, there has been a dedicated process evaluation of integration of mental health into primary care in MexicoReference Miguel-Esponda, Bohm-Levine, Rodríguez-Cuevas, Cohen and Kakuma13 and IndiaReference Shidhaye, Murhar, Muke, Shrivastava, Khan and Singh14 (part of the Programme for Improving Mental Health Care (PRIME) programme).

There remains a gap in terms of understanding mechanisms of impact, the role of local context and what factors contribute to effective integration of mental health services into primary care in low-income settings.Reference Hanlon, Luitel, Kathree, Murhar, Shrivasta and Medhin15,Reference Prince, Patel, Saxena, Maj, Maselko and Phillips16 In fact, the WHO mhGAP Operations Manual was not published until 2018,17 a full 8 years after the mhGAP Intervention Guidelines. Although the mhGAP Intervention Guidelines followed detailed methodologies for evidence use,18 this was not the case for the Operations Manual, reflecting the far less robust level of empirical knowledge about effective processes of implementing services, even after such a long period of field experience. This process evaluation aims to contribute to a stronger theoretical basis to guide translation of international evidence-based guidelines and consensus like mhGAP into effective implementation in the field.

The Mental Health Scale Up Nigeria programme

The Mental Health Scale Up Nigeria (mhSUN) programme was developed in collaboration between the University of Ibadan, State Ministries of Health in Kaduna and Cross River States, Federal Ministry of Health (including two Federal Neuropsychiatric Hospitals in the states) and CBM, an international development agency.

The programme used the WHO mhGAP as a basis for integration of mental health into state primary care services, and was aligned to the National Mental Health Policy in Nigeria. In addition to the model of service provision, relevant factors deemed as necessary to maximise impact and sustainability were considered in programme design; for example, governance, population engagement and health system components such as human resource capacity, supervision and referral, health information systems and medication availability. It is described in detail in a previous paper,Reference Eaton, Gureje, De Silva, Sheikh, Ekpe and Abdulaziz19 and its components are listed below (Table 1). See also Supplementary File 1, available at https://doi.org/10.1192/bjo.2021.7, for a summary of the core considerations in designing the intervention.

Table 1 Research questions, and linked data sources and analysis approach

mhSUN, Mental Health Scale Up Nigeria; mhGAP, Mental Health Gap Action Programme; PACIC, Patient Assessment of Chronic Illness Care.

Two sites for the research were chosen to capture the major differences between the north and south of the country (Fig. 1). Cross River State is in the southern Niger Delta, with a predominantly Christian population made up of several small linguistic and ethnic groups. Kaduna, in the north of the country, instead has an influential religious structure among the strongly Muslim Hausa-speaking population. Kaduna State has a lower Human Development Index (0.404 compared with 0.551 for Cross River State), reflecting lower average income, poorer access to healthcare and lower education levels. With very different cultures and traditions, in Nigeria, it is considered essential to demonstrate an understanding of both Northern and Southern contexts, to show potential broad applicability of any proposed system reform (and to meet the political imperative of so-called ‘national character’). Both sites have a Federal Neuropsychiatric Hospital, with a leadership committed to applying the national policy with respect to establishing access to mental healthcare at primary health level, and a basic health system structure that reflected global conventions. However, in both cases, decades of under-resourcing have left a weak primary care system, influenced by vertical programmes funded by international donors, taking priority-setting, at least in part, out of the hands of local authorities.Reference Mounier-Jack, Rudge, Phetsouvanh, Chanthapadith and Coker20 This has tended to undermine mental and other non-priority areas of health, including capacity to deliver at primary care level, whatever policy dictates.Reference Bitton, Ratcliffe, Veillard, Kress, Barkley and Kimball21

Fig. 1 Map of Nigeria showing the research sites Kaduna State and Cross River State. The Gross Domestic Product (GDP) is a monetary value of all goods and services produced in a given period. The Human Development Index (HDI) is a measure combining dimensions of health, education and standard of living.

Method

The aim of the process evaluation is to build a practical evidence base to support the appropriate scale-up of acceptable, effective and accessible mental health services in LMICs, based on learning from the Nigeria context.

Theoretical basis for the process evaluation

We have drawn on a rich literature providing theoretical frameworks for evaluating implementation of health reforms; for example, Linnan and Steckler's work,Reference Steckler and Linnan22 based on large public health trials in the 1990s,Reference Flora23,Reference Baranowski and Stables24 further developed by Carrol et alReference Carroll, Patterson, Wood, Booth, Rick and Balain25 and HassonReference Hasson26 (see Fig. 2 below). Much of this work is synthesised in the Medical Research Council's ‘Guidelines for Process Evaluation of Complex Interventions’.Reference Moore, Audrey, Barker, Bond, Bonell and Hardeman27 Given the poor record of sustainability of externally funded programmes in international development, we also draw upon two other influential frameworks, Reach, Effectiveness, Adoption, Implementation, and MaintenanceReference Glasgow, Vogt and Boles28 and Normalisation Process Theory,Reference May and Finch29,Reference Murray, Treweek, Pope, MacFarlane, Ballini and Dowrick30 as they highlight essential relevant factors to be considered in recognising context, and sustainability and acceptability issues.

Fig. 2 Conceptual framework for evaluation of the Mental Health Scale Up Nigeria (mhSUN) intervention. HMIS, health management information system.

Literature around strengthening primary healthcare, often now linked to achieving universal health coverage, has a long tradition of measuring outcomes and some process factors, including in LMICs.Reference Levy-Bruhl D, Osseni, Ndiaye, Dieng, De Bethune and Diallo31 The WHO and global funders have developed national and global scorecards,Reference Murray and Evans32,Reference Foundation33 typically using the structure-process-outcome model as a framework.Reference Donabedian34 Well-established work in HIV services probably has the most relevance for mental health integration because of parallels in being a stigmatised chronic disease, often requiring long-term care and support. Less work has been done in the non-communicable disease sector, and in the contexts in which we are working, to date, most research has been associated with specific global health priorities such as vaccination,Reference Webster, Kayentao and Bruce35 child and maternal health,Reference McDougall, Campbell and Wendy36 and bed net distribution for malaria control.37 Important work has been carried out to explore the integration of mental health into primary care in LMICs, such as the PRIMEReference Lund, Tomlinson, Silva, Fekadu, Shidhaye and Jordans38 and Emerald projects,39 and this intervention and study makes use of many of the approaches developed in this work.

The basis of the intervention itself was developed using Theory of ChangeReference De Silva, Breuer, Lee, Asher, Chowdhary and Lund40 (ToC), and was documented in a mhSUN Operations Manual (see Supplementary File 2). ToC uses a participatory approach to map the steps by which we hypothesise achieving a desired outcome, documenting the associated assumptions and contextual issues that are relevant for each step. ToC was used in a similar way in the PRIME programme, which highlighted additional advantage of this approach; alongside drawing on their expertise, the participation of key stakeholders in the ToC workshop recognise and value their contribution to the change process, gaining buy-in and commitment.Reference Breuer, De Silva, Shidaye, Petersen, Nakku and Jordans41 This may help achieve project outcomes by gaining future political support and investment from health system leaders, motivation for key staff, or trust from patients and families.

Methods, data sources and analysis

These theoretical frameworks and the ToC map for this intervention provides a basis for identification of key elements of the process of change researchers may wish to investigate further, by asking research questions relevant to the assumptions at key steps. Establishing and documenting such putative mechanisms for change and the effect of contextual factors in advance allows for subsequent hypothesis testing during the process evaluation.

We will use a mixed-methods approach with data from individual, facility and system levels and a variety of sources (below). Analysis and interpretation will include triangulation between quantitative and qualitative data, measuring the degree to which the intervention was implemented as intended, the influence of context and moderating factors, and providing an opportunity to gain insights from participants in the implementation.

Data sources include:

  1. (a) A situation analysis carried out at the programme development stage.

  2. (b) A facility case study comprising quarterly evaluation of facilities, including a facility checklist of human resource presence, medication availability and infrastructure, as well as record of changing context.

  3. (c) Log books of intervention elements such as governance processes, advocacy meetings, awareness campaign, training and supervision.

  4. (d) Routine health information system recording service use statistics, disaggregated by primary diagnosis and gender, and patient clinical records.

  5. (e) Supervision record forms documenting monthly supervision visits, incorporating assessment of quality of care provided.

  6. (f) Cohort of patients comprising a sample of patients with each of the three major diagnoses of epilepsy, psychosis and depression (the priority conditions for the service) will be interviewed on entry to the service and at 6 months. Data includes validated questionnaires on clinical, human rights and disability outcomes, patient satisfaction and service use (including costs). A conservative sample size of 200 participants per diagnosis has been calculated to achieve sufficient power to measure change in outcomes on clinical and disability measures. This was based on outcomes of similar studies exploring integration of treatment of depression, psychosis and epilepsy in primary care in LMIC settings.Reference Murthy, Kumar, Chisholm, Thomas, Sekar and Chandrashekar42,Reference Kwan and Sander43

  7. (g) Qualitative data collected through in-depth key informant interviews, and observation notes kept during site visits. Processes to engage and gain buy-in from relevant actors are built into the governance of mhSUN, so we expect to be able to obtain consent from key actors.

Data collection is carried out by research assistants and managed by a research coordinator in each site. The research team is attached to the Federal Neuropsychiatric Hospital where the mhSUN programme is managed, but functions independently of the implementing teams in terms of roles. Data collection forms were developed and tested for the facility data, supervision records and log books for activities. Mental health elements also had to be incorporated into the routine Primary Health Care data collection forms. as this was not previously included. The cohort questionnaire is programmed into a mobile application (Mobenzi Android v4.15.0; www.mobenzi.com), allowing efficient and secure data collection, storage and transfer. Such applications have the advantage of flexible use of skips and assured completion of questions, making data collection efficient, reducing missing data and increasing data quality.

We now outline background, methods and analysis for each research question in greater detail.

Primary research question 1: intervention, adherence (fidelity, dose) and coverage

The intervention has been described, after development with ToC, including its essential components and the theoretical underpinning of these components.Reference Eaton, Gureje, De Silva, Sheikh, Ekpe and Abdulaziz19 The major components of the model, as described in the Operations Manual, are summarised in Table 2. This is used as a basis for training and is available to implementers, and it is against this that fidelity will be measured.

Table 2 Components of the mhSUN intervention, and data sources for assessment of fidelity

mhSUN, Mental Health Scale Up Nigeria.

Following descriptive analysis of these data, comparing expected with actual intervention components, we will collect qualitative data as a means of documenting the perspectives of the key actors involved (healthcare providers, implementers and leaders in the community and health system), to help interpretation of quantitative results.

The interview topic guide for the qualitative data will be situated within the theoretical models employed, and will be based on examination of the assumptions (leading to points of enquiry for research) made in the ToC. Inductive thematic analysis of interviews will be carried out following transcription, and the results used to triangulate with quantitative results on hypothesised moderators and contextual factors.

The data generated can also be used for subsequent deductive analysis, such as using structured frameworks like the Consolidated Framework for Implementation Research,Reference Damschroder, Aron, Keith, Kirsh, Alexander and Lowery44 allowing for greater comparability across contexts, and application to structured and practical processes of translation of research to practice. Observational data collected throughout the implementation (field notes and notes made by implementers in the log books) will also be used to inform the interpretation of the quantitative data.

It should be noted that dynamic adaptation to context and changing circumstances is essential in successful implementation of any intervention, especially when in a novel environment.Reference Fonagy and Luyten45 This will be recorded throughout the intervention, particularly in the observational notes and qualitative interviews.

Coverage/reach

This refers to the proportion of the target group are affected by the intervention (see Supplementary File 3 for a process flow diagram). This is a key consideration for efficiency and equity, and is of great interest to decision makers in government – the target of our research results. Three levels of coverage will be measured.Reference De Silva, Lee, Fuhr, Rathod, Chisholm and Schellenberg46 The first level is contact coverage. Routinely collected data available at the clinic will be used to measure clinic attendance for mental disorder. Published mental disorder prevalence data, weighted for the demographics of the local community, will then be used to obtain the denominator for calculation.

The second level is adequate coverage. To assess the proportion of the people receiving an adequate intervention, the cohort questionnaire included a measure of patient satisfaction with service (Patient Assessment of Chronic Illness Care (PACIC) questionnaireReference Glasgow, Wagner, Schaefer, Mahoney, Reid and Greene47). We will compare the PACIC results with the supervision report, which scores patient treatment on a scale based on whether assessment, diagnosis, treatment and appropriate referral were carried out by attending clinicians (general Primary Health Care nurses). Although specific tools to assess quality of clinical care in Primary Health Care in LMICs have been developed (e.g. the ENhancing Assessment of Common Therapeutic factors toolReference Kohrt, Ramaiya, Rai, Bhardwaj and Jordans48), this is prone to the Hawthorne effect. We have tried to avoid this by assessing anonymous patient notes for quality of clinical care.

The third level is effective coverage. This is defined as the proportion of patients achieving recovery based on 6-month clinical (symptom reduction) and disability outcome data (WHO Disability Assessment Schedule 2.0, which has been used extensively in similar studies of mental healthcare reform,Reference Patel, Weiss, Chowdhary, Naik, Pednekar and Chatterjee49,Reference Üstün, Chatterji, Kostanjsek, Rehm, Kennedy and Epping-Jordan50 including in NigeriaReference Gureje, Oladeji, Montgomery, Bello, Kola and Ojagbemi51). For depression, we used the Patient Health Questionnaire-9, which has been validated and used widely in Nigeria;Reference Adewuya, Ola and Afolabi52 for epilepsy, we used a count of seizures in the previous month. Clinical Global Impression–Severity score was used for all participants entering the cohort study, and this was used as a proxy for psychosis, where a good correlation has been reported for more complex change measures.Reference Pinna, Deriu, Diana, Perra, Randaccio and Sanna53 In addition, quality of training is assessed with standard mhGAP pre-and post-questionnaires, and competency forms part of supervision reports.

In addition, issues of equity (a component of reach/coverage) will be assessed by routinely collected service use data, disaggregated by age, gender, socioeconomic status, diagnosis and distance from the service.

Research questions 2 and 3: context and potential moderators

Context

A detailed situation analysis at state and local government levels was carried out in each state during design of the intervention. This will be reviewed and summarised with respect to potential contextual factors that could affect the implementation. In addition, each quarter, as part of the facility case study, personnel document any environmental factors that they feel may have influenced implementation or outcomes of the programme. Both the situation analysis and facility case study borrow extensively from the work carried out in the PRIME programme,Reference De Silva, Rathod, Hanlon, Breuer, Chisholm and Fekadu54 which shared many contextual characteristics with mhSUN. The importance of these contextual factors will be further explored by key informant interviews, recognising that this is a dynamic process, with the programme being affected by, and affecting, the local environment.

Moderators

Postulated moderators for effective implementation were derived from theoretical frameworks and the ToC developed for this project. This process drew upon local experience to highlight what factors might influence outcomes. These were documented as preconditions for achieving the different steps of change in the mhSUN ToC (and associated indicators assigned to them) (see previous mhSUN publicationReference Eaton, Gureje, De Silva, Sheikh, Ekpe and Abdulaziz19):

  1. (a) Participant responsiveness/satisfaction: Does quality of treatment or staff clinician behaviour influence service use patterns?

  2. (b) Resources: What was the effect of availability/lack of availability of human resources, health information systems, medication, clinic infrastructure and other resources to implementation of the programme as per time/output expectations?

  3. (c) Recruitment/service uptake: What factors increased service use and the number of appointments patients attended? What was the effect of recognised barriers to access (costs, distance) and of the awareness programme?

  4. (d) Model description and communication: Did staff feel they understood the model and feel involved in its development and adaptation? How do base training and refresher sessions improve staff knowledge and improve treatment?

Information on putative moderating factors influencing implementation have also been included in data to be collected (Table 3).

Table 3 Potential moderating factors and data sources

a. See supplementary material for facility case study and cohort questionnaire details.

PACIC, Patient Assessment of Chronic Illness Care; CSRI, Client Service Receipt Inventory; HES Nigeria, Household Expenditure Survey; HMIS, (routine) health management information system at Primary Health Care.

We will use descriptive analysis of facility-level data and logs (training attendance, medication availability, staff presence, etc.), using data at different time periods to compare outcome with and without the relevant moderating factors of impact of awareness-raising programme on service uptake; availability (or not) of medication, and any correlation with service provision and uptake; and actual rate of programme components bedding down (compared with initial plans and expectations).

In addition, as the programme is being carried out in two states (and within seven and eight districts within each of these), we will be able to examine the effect of differences in implementation across location; for example, the impact of localised policies on drug distribution on availability, the effects of support from local political and health leadership, and the influence of insecurity on implementation in different districts.

For data collected through the cohort questionnaire, we will be able to summarise data on costs, service use and other cross-sectional information, and carry out logistic regression analyses (using Stata for Windows version 9 to generate odds ratios) to identify associations between different factors and outcomes, such as looking at whether a group reporting easy versus barriers to access to a clinic had better outcomes.Reference Bonell, Fletcher, Morton, Lorenc and Moore55 Although such analysis risks being underpowered, we will explore whether the results confirm or refute our hypotheses, and we will be triangulating this with the qualitative work, allowing us to explore potential mechanisms, using the experience and expertise of key informants.

Research question 4: accessibility and acceptability

Ensuring accessibility and acceptability was a consideration throughout the design of the intervention, as demonstrated by the use of ToC and extensive engagement with local authorities. Language and cultural understanding are a consideration throughout in the research and programme elements, as shown by awareness-raising and local concepts of mental illness being incorporated into communication materials. Accessibility and acceptability will be assessed through the cohort questionnaire (PACIC), and in qualitative interviews with patients, carers and healthcare staff, drawing on similar research.Reference Mendenhall, De Silva, Hanlon, Petersen, Shidhaye and Jordans56 Focus group discussions will be used both in qualitative evaluation of acceptability of the intervention, and in triangulating with factors deemed to influence outcomes, as acceptability is a potential moderator (part of Participant Responsiveness (see Fig. 1).

Strengths and limitations of the study design

The study is a pre-planned process evaluation exploring local contextual factors, filling a gap in literature for mental health integration into primary care that is mainly of outcome evaluations to date. There is a strong theoretical basis for the intervention, based on international guidelines (WHO mhGAP) that are widely accepted normative standards. Local experience and context are considered with a ToC approach. The use of mixed methods, with data capture through routinely collected key data and through purposive quantitative and qualitative methods, allows triangulation of results from different sources, testing assumptions from the ToC and allowing reflection by key actors of potential mechanisms. Weaknesses in routine data collection will be managed with provision of additional statistics forms for mental health and related processes.

A key limitation is the many external factors that are likely to influence the implementation of the mental health intervention in a generally weak and fragile system, and a context of political uncertainty with a history of communal violence. However, the research aims to inform practice in just these settings and a real-world evaluation is of more value than a highly controlled experimental design. These factors will be captured and their impact interpreted through the study design used.

Ethics and dissemination

Ethical approval has been given by the ethical review boards of London School of Hygiene and Tropical Medicine (Ref: 11056 /RR/5812), Ibadan University and the two Federal Neuropsychiatric Hospitals in Calabar and Kaduna. Consent will be obtained from all participants interviewed in the quantitative or qualitative components of the research. Extensive engagement was carried out with the local primary care services, including gaining permission from the State Ministries of Health (Commissioner, Primary Care Director), during programme design. At the local government area level, programme staff visited the local government Chairman and discussed the research with the Supervisory Councillor for Health, and each Primary Health Care unit head and relevant local government area monitoring and evaluation leads. A Memorandum of Understanding was signed with each state Ministry of Health and local government area, which gave formal authority for the work. In addition to these local health officials, local traditional leaders, tradition healers and religious leaders were regularly visited throughout the period of the study.

In Kaduna, most data will be collected in Hausa, and surveys questionnaires have been translated and back-translated accordingly. In both sites, research assistants speak the range of local languages. Interviews will be transcribed and de-identified. The number of key informants will be relatively low, but we still do not expect it to be possible to identify individuals. Recognising language and literacy issues, clear participant information sheets have been developed, and research assistants have been taught to share relevant information clearly (verbally) and to respond to queries. Data will be securely stored throughout the process, including through the use of mobile digital data collection, and is only accessible to the research team members who possess a password (primary investigators and research coordinators at sites). All data for the cohort is anonymised at the point of collection. Other data collected at the sites will be stored by the research coordinator in a locked office, and on a password-protected computer. It will be sent at regular intervals to a server established for this purpose at London School of Hygiene and Tropical Medicine.

Safety of research and implementation teams is a major consideration in this project because there is a history of communal violence in both states. Local districts were chosen to avoid this as much as possible, but it is felt important to carry out research in these settings that represent a large proportion of Africa, where mental healthcare is most needed and least available. Procedures will be put in place to minimise risk, guided by site teams and based on local knowledge of the risks at any particular time. This process (used during the research phase) will reflect real decision-making processes used in implementing programmes in practice.

As with other mental health service reform programmes at a pilot scale, the study's aim of influencing service planners is the main justification for a process evaluation study. The study aims to directly inform the investment and policy decisions of political and health leaders, and the Federal Ministry and State Ministries of Health in Nigeria are the primary target for dissemination, but results would be of interest to other governmental and civil society implementers, particularly in wider sub-Saharan Africa and in LMICs more broadly.

Resources have been allocated to share results with the National Mental Health Action Committee of the Federal Ministry of Health and in meetings at state Ministries of Health, as well as directly with those involved in the study in community meetings.

We plan to publish the results of this work in two separate papers (in addition to the development paper already published): research questions 1–3, where analysis will draw together fidelity and outcomes with moderators and contextual factors; and a separate paper on accessibility and acceptability of the intervention.

In addition to peer-reviewed academic literature, we will use networks of global mental health, such as the Mental Health Innovation Network (www.mhinnovation.net), which is widely accessed by implementers and policy makers. We expect to be able to provide the mhSUN programme implementers with learning that they may apply to refinement of the model to facilitate improved outcomes and sustainability, and will hold feedback sessions to local implementers and relevant stakeholders. In this way, we hope that the results of the work will be of value both locally and more broadly.

Supplementary material

Supplementary material is available online at https://doi.org/10.1192/bjo.2021.7

Data availability

As a protocol paper, data availability is not applicable to this article because no new data were created or analysed in this study. However, any information related to the study protocol are available on request from the corresponding author, J.E.

Author contributions

J.E. developed the protocol, based on contribution from the other authors with respect to methodological (O.G.) and contextual knowledge (E.N., T.L.S., E.E.E., Y.A. and U.O.). The other authors were also involved in development of the mhSUN intervention model and Theory of Change workshop.

Funding

This work is funded by the Australian Department of Foreign Affairs and Trade, through CBM Australia, in the form of support for mhSUN programme costs.

Declaration of interest

J.E. works for CBM, which funded the mhSUN programme and its evaluation. Other authors declare that they do not have conflicts of interest. ICMJE forms are in the supplementary material.

References

Lancet Global Mental Health Group. Scale up services for mental disorders: a call for action. Lancet 2007; 370: 1241–52.CrossRefGoogle Scholar
Eaton, J, McCay, L, Semrau, M, Chatterjee, S, Baingana, F, Araya, R, et al. Scale up of services for mental health in low-income and middle-income countries. Lancet 2011; 378(9802): 1592–603.CrossRefGoogle ScholarPubMed
Patel, V, Araya, R, Chatterjee, S, Chisholm, D, Cohen, A, DeSilva, M, et al. Treatment and prevention of mental disorders in low and middle income countries. Lancet 2007; 370: 9911005.CrossRefGoogle Scholar
Araya, R, Rojas, G, Fritsch, R, Gaete, J, Rojas, M, Simon, G, et al. Treating depression in primary care in low-income women in Santiago, Chile: a randomised controlled trial. Lancet 2003; 361(9362): 9951000.CrossRefGoogle ScholarPubMed
World Health Organization. mhGAP Intervention Guide for Mental, Neurological and Substance Use Disorders in Non-Specialized Health Settings: Mental Health Gap Action Programme (mhGAP). World Health Organization, 2010 (https://www.who.int/mental_health/publications/mhGAP_intervention_guide/en/).Google Scholar
Barbui, C, Dua, T, van Ommeren, M, Yasamy, MT, Fleischmann, A, Clark, N, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS Med 2010; 7(8): e1000322.CrossRefGoogle ScholarPubMed
Keynejad, RC, Dua, T, Barbui, C, Thornicroft, G. WHO mental health Gap Action Programme (mhGAP) intervention guide: a systematic review of evidence from low and middle-income countries. Evid Based Ment Health 2018; 21(1): 30–4.CrossRefGoogle ScholarPubMed
Gómez-Carrillo, A, Lencucha, R, Faregh, N, Veissière, S, Kirmayer, LJ. Engaging culture and context in mhGAP implementation: fostering reflexive deliberation in practice. BMJ Global Health 2020; 5(9): e002689.CrossRefGoogle ScholarPubMed
Mills, C, Hilberg, E. ‘Built for expansion’: the ‘social life’ of the WHO's mental health GAP intervention guide. Sociol Health Illn 2019; 41: 162–75.CrossRefGoogle ScholarPubMed
Ling, T. Evaluating complex and unfolding interventions in real time. Evaluation 2012; 18(1): 7991.CrossRefGoogle Scholar
Chisholm, D, James, S, Sekar, K, Kumar, KK, Murthy, RS, Saeed, K, et al. Integration of mental health care into primary care: demonstration cost–outcome study in India and Pakistan. Br J Psychiatry 2000; 176(6): 581–8.CrossRefGoogle ScholarPubMed
Asher, L, Fekadu, A, Hanlon, C, Mideksa, G, Eaton, J, Patel, V, et al. Development of a community-based rehabilitation intervention for people with schizophrenia in Ethiopia. PLoS One 2015; 10(11): e0143572.CrossRefGoogle ScholarPubMed
Miguel-Esponda, G, Bohm-Levine, N, Rodríguez-Cuevas, FG, Cohen, A, Kakuma, R. Implementation process and outcomes of a mental health programme integrated in primary care clinics in rural Mexico: a mixed-methods study. Int J Ment Health Syst 2020; 14(1): 114.CrossRefGoogle ScholarPubMed
Shidhaye, R, Murhar, V, Muke, S, Shrivastava, R, Khan, A, Singh, A, et al. Delivering a complex mental health intervention in low-resource settings: lessons from the implementation of the PRIME mental healthcare plan in primary care in Sehore district, Madhya Pradesh, India. BJPsych Open 2019; 5(5): e63.CrossRefGoogle ScholarPubMed
Hanlon, C, Luitel, NP, Kathree, T, Murhar, V, Shrivasta, S, Medhin, G, et al. Challenges and opportunities for implementing integrated mental health care: a district level situation analysis from five low- and middle-income countries. PLoS One 2014; 9(2): e88437.CrossRefGoogle ScholarPubMed
Prince, M, Patel, V, Saxena, S, Maj, M, Maselko, J, Phillips, MR, et al. No health without mental health. Lancet 2007; 370: 859–77.CrossRefGoogle ScholarPubMed
World Health Organization. mhGAP Operations Manual. World Health Organization, 2018.Google Scholar
World Health Organization. Manual for Guidelines Development. World Health Organization, 2007.Google Scholar
Eaton, J, Gureje, O, De Silva, M, Sheikh, TL, Ekpe, EE, Abdulaziz, M, et al. A structured approach to integrating mental health services into primary care: development of the Mental Health Scale Up Nigeria intervention (mhSUN). Int J Ment Health Syst 2018; 12(1): 11.CrossRefGoogle Scholar
Mounier-Jack, S, Rudge, JW, Phetsouvanh, R, Chanthapadith, C, Coker, R. Critical interactions between Global Fund-supported programmes and health systems: a case study in Lao People's Democratic Republic. Health Policy Plan 2010; 25(suppl 1): i3742.CrossRefGoogle ScholarPubMed
Bitton, A, Ratcliffe, HL, Veillard, JH, Kress, DH, Barkley, S, Kimball, M, et al. Primary health care as a foundation for strengthening health systems in low-and middle-income countries. J Gen Intern Med 2017; 32(5): 566–71.CrossRefGoogle ScholarPubMed
Steckler, A, Linnan, L. Process Evaluation for Public Health Interventions and Research. Jossey-Bass, 2002.Google Scholar
Flora, JA. A community education monitoring system: methods from the Stanford Five-City Project, the Minnesota Heart Health Program, and the Pawtucket Heart Health Program. Health Educ Res 1993; 8: 8195.CrossRefGoogle Scholar
Baranowski, T, Stables, G. Process evaluation of the 5-a-day projects. Health Educ Behav 2000; 27(2): 157–66.CrossRefGoogle ScholarPubMed
Carroll, C, Patterson, M, Wood, S, Booth, A, Rick, J, Balain, S. A conceptual framework for implementation fidelity. Implement Sci 2007; 2(1): 40.CrossRefGoogle ScholarPubMed
Hasson, H. Systematic evaluation of implementation fidelity of complex interventions in health and social care. Implement Sci 2010; 5: 67.CrossRefGoogle ScholarPubMed
Moore, GF, Audrey, S, Barker, M, Bond, L, Bonell, C, Hardeman, W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 2015; 350: h1258.CrossRefGoogle ScholarPubMed
Glasgow, RE, Vogt, TM, Boles, SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999; 89(9): 1322–7.CrossRefGoogle ScholarPubMed
May, C, Finch, T. Implementing, embedding, and integrating practices: an outline of normalization process theory. Sociology 2009; 43(3): 535–54.CrossRefGoogle Scholar
Murray, E, Treweek, S, Pope, C, MacFarlane, A, Ballini, L, Dowrick, C, et al. Normalisation process theory: a framework for developing, evaluating and implementing complex interventions. BMC Med 2010; 8(1): 63.CrossRefGoogle ScholarPubMed
Levy-Bruhl D, SA, Osseni, R, Ndiaye, JM, Dieng, B, De Bethune, X, Diallo, AT, et al. The Bamako initiative in Benin and Guinea: improving the effectiveness of primary health care. Int J Health Plann Manage 1997; 12: S4979.3.0.CO;2-P>CrossRefGoogle ScholarPubMed
Murray, CJL, Evans, DB. Health Systems Performance Assessment: Debates, Methods and Empiricism. World Health Organization, 2003.Google Scholar
Foundation, K. Logic Model Development Guide. W.K Kellogg Foundation, 2004.Google Scholar
Donabedian, A. The quality of care. How can it be assessed? JAMA 1988; 260(12): 1743–8.CrossRefGoogle ScholarPubMed
Webster, J, Kayentao, K, Bruce, J, et al. Prevention of malaria in pregnancy with intermittent preventive treatment and insecticide treated nets in Mali: a quantitative health systems effectiveness analysis. PLoS One 2013; 8(6): e67520.CrossRefGoogle ScholarPubMed
McDougall, L, Campbell, OM, Wendy, CW. Maternal Health An Executive Summary for The Lancet’s Series “Every woman, every newborn, everywhere has the right to good quality care”. Lancet 2016; 388(8).Google Scholar
The Global Fund. The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria Synthesis of Study Areas 1, 2 and 3. The Global Fund, 2009 (https://www.theglobalfund.org/en/technical-evaluation-reference-group/evaluations/2009-five-year-evaluation/).Google Scholar
Lund, C, Tomlinson, M, Silva, M, Fekadu, A, Shidhaye, R, Jordans, M. PRIME: a programme to reduce the treatment gap for mental disorders in five low- and middle-income countries. PLoS Med 2012; 9: e1001359.CrossRefGoogle ScholarPubMed
Centre for Global Mental Health. EMERALD: Emerging Mental Health Systems in Low- and Middle-Income Countries. Centre for Global Mental Health, 2018 (https://www.centreforglobalmentalhealth.org/emerald-emerging-mental-health-systems-in-low-and-middle-income-countries).Google Scholar
De Silva, MJ, Breuer, E, Lee, L, Asher, L, Chowdhary, N, Lund, C, et al. Theory of change: a theory-driven approach to enhance the Medical Research Council's framework for complex interventions. Trials 2014; 15: 267.CrossRefGoogle ScholarPubMed
Breuer, E, De Silva, MJ, Shidaye, R, Petersen, I, Nakku, J, Jordans, MJ, et al. Planning and evaluating mental health services in low-and middle-income countries using theory of change. Br J Psychiatry 2016; 208(s56): s5562.CrossRefGoogle ScholarPubMed
Murthy, RS, Kumar, KK, Chisholm, D, Thomas, T, Sekar, K, Chandrashekar, C. Community outreach for untreated schizophrenia in rural India: a follow-up study of symptoms, disability, family burden and costs. Psychol Med 2005; 35(3): 341–51.CrossRefGoogle Scholar
Kwan, P, Sander, J. The natural history of epilepsy: an epidemiological view. Neurol Neurosurg Psychiatry 2004; 75(10): 1376–81.CrossRefGoogle Scholar
Damschroder, LJ, Aron, DC, Keith, RE, Kirsh, SR, Alexander, JA, Lowery, JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4(1): 50.CrossRefGoogle ScholarPubMed
Fonagy, P, Luyten, P. Fidelity vs. flexibility in the implementation of psychotherapies: time to move on. World Psychiatry 2019; 18(3): 270–1.CrossRefGoogle Scholar
De Silva, M, Lee, L, Fuhr, D, Rathod, S, Chisholm, D, Schellenberg, J, et al. Estimating the coverage of mental health programmes: a systematic review. Int J Epidemiol 2014; 43: 341–53.CrossRefGoogle ScholarPubMed
Glasgow, RE, Wagner, EH, Schaefer, J, Mahoney, LD, Reid, RJ, Greene, SM. Development and validation of the Patient Assessment of Chronic Illness Care (PACIC). Med Care 2005; 43(5): 436–44.CrossRefGoogle Scholar
Kohrt, BA, Ramaiya, MK, Rai, S, Bhardwaj, A, Jordans, MJ. Development of a scoring system for non-specialist ratings of clinical competence in global mental health: a qualitative process evaluation of the Enhancing Assessment of Common Therapeutic Factors (ENACT) scale. Glob Ment Health (Camb) 2015; 2: e23.CrossRefGoogle ScholarPubMed
Patel, V, Weiss, HA, Chowdhary, N, Naik, S, Pednekar, S, Chatterjee, S, et al. Effectiveness of an intervention led by lay health counsellors for depressive and anxiety disorders in primary care in Goa, India (MANAS): a cluster randomised controlled trial. Lancet 2010; 376(9758): 2086–95.CrossRefGoogle ScholarPubMed
Üstün, TB, Chatterji, S, Kostanjsek, N, Rehm, J, Kennedy, C, Epping-Jordan, J, et al. Developing the World Health Organization Disability Assessment Schedule 2.0. Bull World Health Organ 2010; 88: 815–23.CrossRefGoogle ScholarPubMed
Gureje, O, Oladeji, BD, Montgomery, AA, Bello, T, Kola, L, Ojagbemi, A, et al. Effect of a stepped-care intervention delivered by lay health workers on major depressive disorder among primary care patients in Nigeria (STEPCARE): a cluster-randomised controlled trial. Lancet Glob Health 2019; 7(7): e951–60.CrossRefGoogle ScholarPubMed
Adewuya, AO, Ola, BA, Afolabi, OO. Validity of the Patient Health Questionnaire (PHQ-9) as a screening tool for depression amongst Nigerian university students. J Affect Disord 2006; 96(1): 8993.CrossRefGoogle ScholarPubMed
Pinna, F, Deriu, L, Diana, E, Perra, V, Randaccio, RP, Sanna, L, et al. Clinical Global Impression-severity score as a reliable measure for routine evaluation of remission in schizophrenia and schizoaffective disorders. Ann Gen Psychiatry 2015; 14(1): 6.CrossRefGoogle ScholarPubMed
De Silva, MJ, Rathod, SD, Hanlon, C, Breuer, E, Chisholm, D, Fekadu, A, et al. Evaluation of district mental healthcare plans: the PRIME consortium methodology. Br J Psychiatry 2016; 208(s56): s6370.CrossRefGoogle Scholar
Bonell, C, Fletcher, A, Morton, M, Lorenc, T, Moore, L. Realist randomised controlled trials. a new approach to evaluating complex public health interventions. Soc Sci Med 2012; 75(12): 2299–306.CrossRefGoogle ScholarPubMed
Mendenhall, E, De Silva, M, Hanlon, C, Petersen, I, Shidhaye, R, Jordans, M, et al. Acceptability and feasibility of using non-specialist health workers to deliver mental health care: stakeholder perceptions from the PRIME district sites in Ethiopia, India, Nepal, South Africa, and Uganda. Soc Sci Med 2014; 118: 3342.CrossRefGoogle ScholarPubMed
Figure 0

Table 1 Research questions, and linked data sources and analysis approach

Figure 1

Fig. 1 Map of Nigeria showing the research sites Kaduna State and Cross River State. The Gross Domestic Product (GDP) is a monetary value of all goods and services produced in a given period. The Human Development Index (HDI) is a measure combining dimensions of health, education and standard of living.

Figure 2

Fig. 2 Conceptual framework for evaluation of the Mental Health Scale Up Nigeria (mhSUN) intervention. HMIS, health management information system.

Figure 3

Table 2 Components of the mhSUN intervention, and data sources for assessment of fidelity

Figure 4

Table 3 Potential moderating factors and data sources

Supplementary material: File

Eaton et al. supplementary material

Eaton et al. supplementary material 1

Download Eaton et al. supplementary material(File)
File 1.1 MB
Supplementary material: File

Eaton et al. supplementary material

Eaton et al. supplementary material 2

Download Eaton et al. supplementary material(File)
File 7.5 MB
Submit a response

eLetters

No eLetters have been published for this article.