Study design

This study relies on data collected for the pilot evaluation of the mhGAP-based training in the Greater Tunis area of Tunisia. The pilot study design has been described in detail elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} In brief, we conducted a pilot trial between January 2016 and September 2017 to assess the effect of the training on PCPs’ mental health knowledge, attitudes, self-efficacy and self-reported practice, using several designs. First, a pre-test–post-test control group design^{Reference Campbell, Stanley and Gage12} was employed to assess the training's short-term effects (circles 1–4, Fig. 1). In this design, PCPs were randomly assigned to two groups: the intervention group (group 1) and the control group (group 2). Group 1 received the mhGAP-based training between February and March 2016 (circles 1 and 2, Fig. 1), and group 2 received the training between March and April 2016 (circles 4 and 5, Fig. 1). Second, a repeated measures design was employed to assess the training's long-term effects.

Fig. 1 The design of the pilot trial in which this study is inscribed.

Our study is inscribed within a pilot trial that aimed to assess the mental health capacities of primary care physicians (PCPs) after their participation in Mental Health Gap Action Programme (mhGAP)-based training. For the purposes of this study, we rely on a repeated measures design (as shown in Fig. 1) to improve understanding of the influence of mental health knowledge, attitudes and self-efficacy on self-reported referrals to specialised mental health services at three time periods: pre-training (circles 1 and 3), immediately post-training (circles 2 and 5) and 18 months post-training (circles 6 and 7). X represents the implementation of the training. R represents when PCPs were randomized to either group 1 or group 2. The circled areas depict the pooling of group 1 and group 2 over three time periods. This study relies on the secondary data analysis of data collected during the pilot evaluation of the mhGAP-based training in the Greater Tunis area of Tunisia. The pilot study design in which this study is inscribed has been described in detail elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6}

For this study, we employed a repeated measures design to further assess data from our pilot evaluation of the mhGAP-based training.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} Specifically, we assessed the influence of mental health knowledge, attitudes and self-efficacy on self-reported referrals to specialised mental health services over three time periods, as illustrated in Fig. 1: before the training (circles 1 and 3), immediately after the training of both groups (circles 2 and 5) and 18 months after the training of both groups (circles 6 and 7).

Study setting

The evaluation of the mhGAP-based training on PCPs’ mental health capacities was conducted in the Greater Tunis area of Tunisia, located in the north of the country.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} This area includes four governorates: Ariana, Ben Arous, Manouba and Tunis. The Greater Tunis area was chosen as the pilot setting for the first implementation of the mhGAP-based training because it is diverse: it includes governorates that are rural, urban, semi-rural and semi-urban, which reflects Tunisia as a whole.

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008. All procedures involving human subjects/patients were approved by the ethics committees of Université de Montréal (Québec, Canada) (approval number #15-117-CERES-D) and Hôpital Razi (Tunisia).

During the recruitment phase, PCPs were presented the training programme and the study. They were informed that the training and the study were voluntary, and that their decisions related to participation in the training and study would not have any adverse consequences on their employment. Informed consent was obtained from all study participants.

Participants

Participant recruitment for the pilot trial in which this study is inscribed has been described in detail elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} We compiled the names of 345 PCPs who were registered with the PCPs’ professional order in Tunisia, worked in the public and primary care sectors and previously attended continuing medical education training. Continuing medical education is recommended and encouraged in Tunisia for advancing PCPs’ careers. Of the 345 PCPs, the majority (91.30%) worked in primary care in the Greater Tunis area and had 5 or more years of clinical experience, and so were eligible for the study. A total of 132 PCPs (41.90%) agreed to participate in the pilot trial. In January 2016, the first author contacted the 132 PCPs who agreed to participate in the trial, to further explain the study and to obtain consent. A total of 112 (n = 112) PCPs completed baseline questionnaires and were thus included in the trial.

Forty-five PCPs from group 1 and 43 PCPs from group 2 completed the training programme and agreed to complete questionnaires. This resulted in a pooled total of 88 PCPs included in the analyses immediately after the training. Thirty-two PCPs from group 1 and 27 PCPs from group 2 agreed to complete questionnaires at 18 months (September 2017) post-training. This resulted in a pooled total of 59 PCPs included in long-term analysis.

Intervention

The mhGAP-based training (version 1.0)⁵ was offered to PCPs in the Greater Tunis area between February and May 2016. The programme's adaptation to the local context has been described elsewhere.^{Reference Spagnolo, Champagne, Leduc, Melki, Guesmi and Bram13} Members of the Ministry of Health in Tunisia (W.M., F. Charfi) chose specific training modules considered important to Tunisia: general principles of care, depression, psychosis, suicide/self-harm and problems related to substance use. These modules were adapted to meet the Greater Tunis area's primary care realities, with the help of W.M., F. Charfi, three Tunisian psychiatrists (‘trainers’) and seven PCPs responsible for continuing medical education in the Greater Tunis area who had the role of supporting trainees during and post-training (‘tutors’). The training lasted 6 weeks for a total of 19 h, with sessions being offered once a week. The first five sessions comprised general lectures, role-plays and group discussions (17 h). The last session was a 2-h support session offered by the trainers consisting of further role-plays to help PCPs with challenging mental health cases.

Measures

Variables for the pilot trial^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} were chosen according to Kirkpatrick's conceptual model,^{Reference Kirkpatrick and Kirkpatrick14} often used for training programme evaluations.^{Reference Smidt, Balandin, Sigafoos and Reed15}

Independent variables

Mental health knowledge was assessed using the questionnaire developed by the WHO to accompany the mhGAP training programme. The questionnaire contained 16 questions on the modules selected for training. Correct answers were scored as 1 and incorrect answers were scored as 0. A participant's score is the sum of correct answers for individual items. Overall knowledge scores were converted to a score ranging from 0 to 10, where a higher score indicates more knowledge. This questionnaire reported a good degree of reliability in our sample (the average measure Intraclass Correlation Coefficient (ICC) was 0.708, with a 95% confidence interval of 0.478–0.837).^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16} These psychometric analyses have been reported elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16}

The Mental Illness: Clinicians’ Attitudes (MICA) Scale (version 4.0) was used to measure attitudes toward mental illness and the field of mental health.^{Reference Gabbidon, Clement, van Nieuwenhuizen, Kassam, Brohan and Norman17–18} The scale's Cronbach's alpha, tested on our sample, was found to be poor (0.521). To increase the scale's internal consistency, we kept 11 items (i.e. questions 1, 2, 4, 5, 7, 10, 12, 13, 14, 15 and 16) from the MICA-4 (version 4.0), which resulted in an increase in the total Cronbach's alpha (0.608). Cronbach's alpha is a function of scale length, and increased in our case by removing five items from the original scale. Hence, the new Cronbach's alpha was sufficient.^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16} This procedure has been explained in detail elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16} For statements 10, 12 and 16, items were scored on a scale of 1–6, from ‘strongly agree’ to ‘agree’, ‘somewhat agree’, ‘somewhat disagree’, ‘disagree’ and ‘strongly disagree’. All other items were reverse scored. Scores on individual items were summed to obtain each participant's overall score within a range of 11–66 points. A higher global score indicates a more negative perception about MNS disorders and mental healthcare delivery. This modified questionnaire reported a good degree of reliability in our sample (the average measure of the ICC was 0.704, with a 95% confidence interval of 0.468–0.835).^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16}

The self-efficacy questionnaire was developed for the pilot trial and comprised 35 questions about PCPs’ judgement of their capability to detect, treat and manage mental health conditions included in the training programme. The scale's Cronbach's alpha, tested on our sample, was found to be satisfactory (0.937). Each statement was scored on a scale of 0–4, from ‘strongly agree’ to ‘somewhat agree’, ‘neutral’, ‘somewhat disagree’ and ‘strongly disagree’. A participant's overall score is the sum of answers for individual items. Overall scores were converted to a score ranging from 0 to 10, where a higher score indicates more self-efficacy. This questionnaire has shown a good degree of reliability in our sample (the average measure of the ICC was 0.781, with a 95% confidence interval of 0.606–0.878).^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16} These psychometric analyses have been reported elsewhere.^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16}

Dependent variable

Our variable of interest was PCPs’ self-reported referrals to specialised mental health services, which represents the average weekly percentage of mental health clientele that PCPs reported referring to specialised mental health services. Specifically, PCPs were asked the following question related to their self-reported referrals: among patients presenting with mental health problems, what percentage do you refer to specialised mental health services per week? The variable ‘self-reported referrals to specialised mental health services’ is evaluated on a score that ranges between 0 and 100%.

Covariates

Baseline sociodemographic (age, gender) and practice characteristics, such as part-time or full-time work, average number of weekly work hours, average number of weekly hours dedicated to mental health, average number of weekly patient consultations, average number of weekly patient consultations specifically for mental health, average weekly percentage of consultations per type of mental health condition and previous mental health training, were examined as potential covariates.

Data collection

All questionnaires were pre-tested^{Reference Spagnolo, Champagne, Leduc, Rivard, Piat and Laporta16} and administered at four time points: at baseline, before randomisation (January 2016); following group 1's training (March 2016); following group 2's training (April and May 2016); and 18 months after the training (September 2017) (Fig. 1).

Analyses

This study is a secondary analysis of data collected within the context of a pilot trial.^{Reference Spagnolo, Champagne, Leduc, Rivard, Melki and Piat6} To assess whether PCPs’ mental health knowledge, attitudes and self-efficacy predict PCPs’ self-reported referrals to specialised mental health services, we conducted multiple linear regression analyses. Multivariable analyses with linear mixed models, accounting for the correlation among participants, were performed with the SAS version 9.4 PROC MIXED procedure (platform X64_8PRO for Windows). One of the strengths of these models is that they consider an unequal number of measurements per participant (owing to attrition and/or non-response) for a given time period and even if time intervals are not constant. The level of significance was set at α < 0.05.