Background
In recent decades, mental health care in middle- and high-income countries has been shifting toward community care and a combination of treatment interventions and rehabilitation practices toward recovery (Thornicroft and Tansella, Reference Thornicroft and Tansella1999; Piat and Sabetti, Reference Piat and Sabetti2009; Chen et al., Reference Chen, Krupa, Lysaght, Mccay and Piat2013). Thus, mental health services workers are required to update their practice by adopting new professional behaviours and visions of mental health care (Chester et al., Reference Chester, Ehrlich, Warburton, Baker, Kendall and Crompton2016). However, staff expectations, perceptions and attitudes may foster or hinder the implementation of new practices, such as evidence-based interventions (Schoenwald et al., Reference Schoenwald, Kelleher, Hoagwood, Mayberg and Green2008), routine outcome assessment procedures (Trauer et al., Reference Trauer, Callaly and Herrman2009) and goal setting (Clarke et al., Reference Clarke, Crowe, Oades and Deane2009). Professional sensemaking (Bloor and Dawson, Reference Bloor and Dawson1994) is particularly relevant in mental health care, where mental health workers deal with clinical conditions that are socially constructed (Rosenhan, Reference Rosenhan1973; Weiner, Reference Weiner1975; Eisenberg, Reference Eisenberg1988; Brown, Reference Brown1995). Colombo and colleagues (Colombo et al., Reference Colombo, Bendelow, Fulford and Williams2003) assert that six models of mental disorder can be systematically identified via the narratives of mental health services workers: medical (organic), social, cognitive-behavioural, psychotherapeutic, family and conspiratorial. Moreover, each model may be classified on the basis of 12 key dimensions that define what a mental disorder is, what should be done about it and how those involved should behave toward each other. Thus, every mental health worker explicitly defines what kind of person should be defined as ‘mentally ill’, how they should be ‘treated’ and how much autonomy ‘should be given’ to him/her according to a system of beliefs and attitudes (Slade, Reference Slade2009, 8–34).
Consistent with scholars' interest in the dissemination of the recovery model, a number of tools have been developed to assess staff's professional behaviours, attitudes and competencies related to the recovery framework, such as the Recovery Self-Assessment (RSA) and the Recovery Attitude Questionnaire (RAQ, Borkin et al., Reference Borkin, Steffen, Ensfield, Krzton, Wishnick, Wilder and Yangarber2000). However, since these instruments are based on the recovery model, they do not assess other professional behaviours that could be relevant for service evaluation or training purposes; moreover, they rely mostly on the measurement of attitudes and beliefs that may poorly predict real professional behaviours (Ginsburg et al., Reference Ginsburg, Regehr, Hatala, McNaughton, Frohna, Hodges, Lingard and Stern2000).
Professional behaviours may be conceptualised within the construct of ‘professional culture’. The creation of a particular professional group leads to the development of a system of values and knowledge that is shared among members of this group and thus allows for the orientation of professional behaviours (Bloor and Dawson, Reference Bloor and Dawson1994). These elements constitute the ‘professional culture’ of a particular group, influence their decisions and behaviours and thus contribute to the definition of the organisational culture in which those professionals work. Moreover, for healthcare workers who are used to working in multiprofessional teams, as in mental health services, professional culture is also partially shared among team members, since personal endorsement of the shared culture among in-group members fosters identification with that social group (Rapisarda and Miglioretti, Reference Rapisarda and Miglioretti2019) and with the organisational culture (Schein, Reference Schein2004). A recent meta-synthesis (Rapisarda and Miglioretti, Reference Rapisarda and Miglioretti2019) identified three interrelated dimensions that contribute to professional culture for the staff of MHSWs, i.e. ‘interpersonal distance with users’, ‘power games’ and ‘professional identity over uncertainty’, along with specific interprofessional differences.
Currently, to our knowledge, no instrument has been developed to explicitly assess the professional culture of mental health workers interacting with severely mentally ill people in publicly or privately run mental health services. Because of theoretical and methodological concerns, we designed a self-administered tool to assess the professional culture of mental health services workers. The study aims to validate this tool, named the Mental Health Professional Culture Inventory (MHPCI). The questionnaire was created and developed by a conjoint Italian-Canadian research team, and every research step, from item creation to data analyses, was conducted in parallel in Italy and Canada. Since the construct of professional culture has not been evaluated for professional categories of mental health services workers, this study will focus essentially on construct validity using exploratory and confirmatory factor analyses.
Methods
Development phase
The MHPCI was conceived using a multidimensional definition of professional culture and a discrete number of psychometrically derived dimensions related to meaningful professional behaviour (Lesser et al., Reference Lesser, Lucey, Egener, Braddock, Clarence, Linas and Levinson2010). The MHPCI has been developed by a conjoint team of researchers from Milan (Italy) and from Montreal (Quebec, Canada) with a clinical background. Northern Italy and Quebec have a mental health system with similar historical trajectories (Thornicroft and Tansella, Reference Thornicroft and Tansella1999; Lesage, Reference Lesage, Thornicroft and Tansella2006): the two regions have a public system, financed by public welfare, in which mental health care is delivered mostly in community settings, and have sustained a process of deinstitutionalisation during the second half of the 20th century. The entire development phase, from item creation to data analysis, was conducted together by the conjoint team with the purpose of creating an instrument with good ecological and cultural validity (Beaton et al., Reference Beaton, Bombardier, Guillemin and Ferraz2000; Johnson, Reference Johnson2006).
A first draft of the questionnaire was created in 2013 and included 71 items generated through a literature review and suggestions from clinical researchers and professionals in Milan and Montreal. Items were created to describe an array of behaviours, feelings and cognitive assessments that a staff member could experience in his/her work with users, with a preference for items focusing on behaviour, instead of focusing on explicit beliefs. For that reason, most of the items were not suitable for describing the professional culture of staff members who do not work in direct contact with users (e.g., administrative staff). The MHPCI items have been created on the basis of the following issues and constructs:
• personal involvement in the relationship with users (Akerjordet and Severinsson, Reference Akerjordet and Severinsson2004; Burks and Robbins, Reference Burks and Robbins2012; Carpenter-Song and Torrey, Reference Carpenter-Song and Torrey2015);
• collaboration with the social network: working with clients' families and making contact with members of the informal social network (Goodwin and Happell, Reference Goodwin and Happell2007);
• collaboration with the professional network: cooperation among professionals of different services and interorganisational collaboration could be a delicate issue in the MHS practice (Leutz, Reference Leutz1999);
• evidence-based practices and outcome assessment: attitudes and knowledge regarding evidence-based practice (Lavoie-Tremblay et al., Reference Lavoie-Tremblay, Sounan, Lavigne, Bonin, Lesage, Denis, Renaud, Maisy, Farand and Racine2008; Trauer et al., Reference Trauer, Callaly and Herrman2009) and goal-setting methodology (Clarke et al., Reference Clarke, Crowe, Oades and Deane2009);
• users' involvement in the process of setting rules and making decisions;
• management of aggressive behaviour: staff's perception of patient aggression (De Benedictis, et al., Reference De Benedictis, Dumais, Sieu, Mailhot, Létourneau, Tran and Lesage2011) and cognitive assessments of challenging behaviours (Lambrechts et al., Reference Lambrechts, Van, Eeman and Maes2010);
• users' sexuality (US): MHWs' attitudes and professional behaviours toward actively broaching sexual issues with clients (Cort et al., Reference Cort, Attenborough and Watson2001), including homosexuality (Bowers and Bieschke, Reference Bowers and Bieschke2005);
• spirituality and religion: spirituality may be a relevant issue for users (Wilding et al., Reference Wilding, Muir-Cochrane and May2006; Curlin et al., Reference Curlin, Lawrence, Odell, Chin, Lantos, Koenig and Meador2007): MHWs' attitudes and behaviours toward spirituality in MH care;
• professional identity and recognition: the perceived degree of professional recognition by colleagues.
Italian-to-French translation was performed by a bilingual Canadian co-author of this paper (A.L.), and French-to-Italian back-translation was done by an Italian-Canadian co-author (M.V.). Further Canadian-French revisions were also performed by other co-authors (M.C. and A.F.). In 2014, pilot testing was conducted on a small sample of MHWs in Milan (n = 38) and Montreal (n = 31). Researchers from the Italian-Canadian conjoint team (F.R., M.M. and A.L.) examined the items' quality and removed items with poor psychometrical properties (i.e. more than 10% of missing responses and abstentions, standardised skewness or kurtosis >1.96). The outcome of this preliminary process was a new draft of the questionnaire, composed of 49 items and available in Italian and French Canadian. This new version was used in the following validation phase.
Validation phase
The MHPCI was tested using a survey design by enrolling staff members working in public MHSs in Italy and Canada. Participants were enrolled considering the following inclusion criteria: (1) staff working in a unit/service that provides mental health care for people with severe mental illness; staff participating in an internship or a voluntary job; (2) staff who have face-to-face contact with users during working hours and (3) staff from a good variety of mental health settings. To do so, participants were recruited from different services and units, including community mental health centres, residential facilities, day care centres and hospital wards, from public, private and non-profit organisations in Italy and only public MHSs in Canada.
In Italy, participants were enrolled in public mental health departments of northern Italy. Moreover, staff from residential and day care units managed by local non-profit organisations were also included in the survey. In Italy, data collection took place from October 2015 to September 2016. The principal investigator (F.R.) presented the project and confidentiality issues to MHWs during routine staff meetings and allowed them 2 weeks to complete the task. All participants were requested to sign a consent form, approved by the Ethics Committee of the University of Milan – Bicocca. Questionnaires were anonymous, and they were printed and presented to participants separately on different documents. Once completed, participants were asked to place the questionnaires and the consent form into different envelopes.
In Canada, all participants were enrolled from the Centre Intégré Universitaire de Santé et de Services Sociaux (CIUSSS) de l'Est-de-l’Île-de-Montréal – mental health and addiction program in the district of Montreal. In Montreal, the research team (A.L., A.F., L.D.B., J-F.P. and Y.L.), with the help of the human resources staff and the mental health program directory, compiled a list of names and positions of all 1200 staff members and 73 physicians from the mental health and addiction programs. The principal investigator from the Italian team (F.R.) assigned numeric codes to each Canadian staff member. Only investigators knew the participants' codes. The Canadian team (A.F.) explained the project and guaranteed confidentiality to the program managers and unit heads before letters were sent to the selected staff members via random sets of 400 questionnaires. Three sets of 400 questionnaires were sent to the staff, the first in October 2016, the second in December 2016 and the third in February 2017, for a total of 1200 questionnaires.
A research coordinator and a group of researchers managed the mailing of the questionnaires from the list by sending an envelope containing an invitation letter, a consent form and the questionnaire to the randomly selected set of participants. The envelope cover only displayed codes and not staff names. Staff members were free to participate by completing the documents (or not, i.e. refusing to participate) and sent envelopes by internal mail to the attention of the main investigator (A. L.). The research coordinator registered the participants' codes written on the received questionnaire and sent a digital scanned copy of each questionnaire to the Italian principal investigator for data entry.
The study received approval from the Ethics Committees of the University of Milan Bicocca and the CIUSSS de l'Est-de-l'Ile-de-Montréal.
Data analysis
Database preparation involved the overall inspection of data quality, exclusion of incomplete questionnaires, and replacement of missing values. Participants who did not answer five or more MHPCI items were identified as having ‘incomplete questionnaires’ and consequently were excluded from the analysis of validation. Common characteristics and requirements between professional groups in Italy and in Canada were compared to create comparable professional categories for data analysis. The following professional categories are used: registered nurse, counsellor (includes rehabilitation workers with university degree, i.e. professional educator, occupational therapist and rehabilitation technician), psychologist, psychiatrist, social worker and support worker (including mental health workers without a university degree, i.e. paraprofessional nurses).
Data analysis was conducted in different steps designed to maximise the cross-cultural adaptation of the questionnaire through a recursive procedure. The first step of analysis consisted of a principal component analysis (PCA) with Varimax rotation performed on the Italian sample. A factor loading of 0.40 was established as the lower bound for a variable to be included in the respective factor structure (Vallerand, Reference Vallerand1989; Corbière, Reference Corbière, Corbière and Larivière2014); i.e., items that scored a factor loading lower than 0.40 on a single dimension were eliminated from the model. Cronbach's α was also obtained for each identified factor to avoid low internal consistency. Next, confirmatory factor analysis (CFA) was run on the Canadian sample to confirm the factor solution that emerged in the Italian PCA. To obtain a well-fitted CFA model, the fit indices are as follows: the comparative fit index (CFI) and Tucker-Lewis index (TLI) had to be higher than 0.90, and the root mean square error of approximation (RMSEA) had to be less than or equal to 0.08 (Corbière, Reference Corbière, Corbière and Larivière2014).
The factors' internal consistency and reliability were then tested for both the Italian and Canadian samples. Cronbach's α was obtained for each emerging factor. Test-retest reliability was evaluated by computing correlation indices (Pearson's ρ) for scale scores during a 2-week period.
All data analysis was performed using IBM SPSS Statistics v. 23, except for the CFA, which was performed using R software with the Lavaan package retrieved from the project's website (http://lavaan.ugent.be/index.html).
Results
From a large sample (n = 467), 229 questionnaires were completed in Italy, and 238 were completed in Canada. The response rate was approximately 65% in Italy and 20% in Canada (when considering only comparable professionals and full-time present staff). Only nine participants were excluded for missing data (eight in the Italian sample and one in the Canadian sample). All analyses were performed on a sample of 458 participants (i.e., 221 + 237), whose characteristics are described in Table 1.
Table 1. Sample characteristics
Statistically significant differences (χ 2 or T test): *p < 0.05; **p < 0.01.
Participants were mostly female (69.4%), with an average age of 45.2 years (s.d. = 11.3), 66.8% of them had a university degree, and they had worked in the mental health field for an average of 16.4 years (s.d. = 10.4). Among the professional roles, nurses, psychiatrists (including five residents in training) and counsellors were the most represented categories, constituting 64.6% of the sample. Significant differences (p < 0.05) between the Italian and Canadian samples were found in age, education levels, professional roles and weekly working hours (Table 1).
The results of the PCA conducted on the Italian sample are displayed in Table 2.
Table 2. PCA of the BMHPCI (Italian sample)
Factor loadings <0.30 were not displayed in table.
Thirty-four items presented saturation coefficients <0.40 and were removed from the model; the resulting solution, which explained 56.4% of the variance, included 15 items and were grouped in the following four components:
(1) Family involvement (FI), four items that describe MHWs' attitudes toward the involvement of users' families and informal networks in the care process;
(2) US, four items that describe how frequently MHWs deal with US issues in their practice;
(3) Therapeutic framework (TF), four items describing the adoption of theoretical models and protocols in the professional's practice and collaboration with staff from other services;
(4) Management of aggression risk (MAR), three items about the risk of being assaulted by clients.
While some of the removed items dealt with topics of the above components, most of them referred to different topics (such as personal involvement in the relationship with users, users' involvement in the process of rule setting and decision-making, spirituality and religion) that did not emerge with a clear pattern of loadings.
Components that emerged in the PCA were also confirmed in the CFA ran on the Canadian sample (Figure 1): the model had good fit indices (CFI = 0.90; TLI = 0.90; RMSEA = 0.072), and a significant correlation among FI, US and TS was observed.
Fig. 1. Confirmatory factor analysis (Canadian sample).
The scale validity and reliability scores are presented in Table 3. Cronbach's α indicated an acceptable level of internal consistency for all scales in both samples, with a borderline value of 0.59 for the MAR scale in the Canadian sample. The correlation coefficients for test-retest reliability ranged from 0.69 to 0.91. As the satisfaction threshold is generally set at 0.60 (Vallerand, Reference Vallerand1989), the four dimensions of the MHPCI can be regarded as stable measures at a 2-week time interval. Moreover, a statistically significant correlation among FI, US and TF was found in both samples, with only a slightly significant difference in the correlation between FI and MAR in the Italian sample.
Table 3. Scales reliability and internal consistency
α = Chronbach's α values in the Italian (first) and Canadian (latter) sample.
r = test-restest correlation coefficients in the Italian (first) and Canadian (latter) sample.
Below the diagonal, correlation coefficients (Pearson's r) between subscales for the Italian sample are presented, and above the diagonal, correlation coefficients between subscales for the Canadian sample.
**p < 0.01.
Discussion
The study's aim was to develop a new questionnaire, the MHPCI, to assess professional behaviours related to the professional cultures of mental health services workers. The results indicate that the MHPCI has good internal consistency, cross-cultural validity and short time stability. Although the ten clusters were conceived a priori, both the explorative PCA and the CFA obtained four dimensions, including 15 items: three scales (FI; US and MAR) confirmed the clusters hypothesised by the researchers during the development process; one scale entitled TF emerged in the PCA and was confirmed by the CFA. To our knowledge, no existing questionnaire covers all four dimensions and adopts a behavioural approach to assess professional culture.
The dimensions that constitute the final version of the MHPCI include constructs that are well known in research and clinical practice. FI has been progressively better recognised in the psychosocial rehabilitation literature (Hall, Reference Hall1999; Goodwin and Happell, Reference Goodwin and Happell2007). In one survey of psychiatric foster families and residents of group homes, the latter nominated their own family as the one who believed most in their recovery, and the Competency Assessment Instrument (Chinman et al., Reference Chinman, Young, Rowe, Forquer, Knight and Miller2003) considers FI as a core competency of mental health services workers. The US scale is an original scale, since current tools that assess professional behaviour do not include items related to sexuality. However, items about US were included in the Camberwell Assessment of Needs (Phelan et al., Reference Phelan, Slade, Thornicroft, Dunn, Holloway, Wykes, Strathdee, Loftus, McCrone and Hayward1995) but not in previous formal needs assessment questionnaires (Comtois et al., Reference Comtois, Morin, Lesage, Lalonde, Likavcanova and L’Écuyer1998). In the latter study, sexuality emerged as the fifth most important problem reported by patients, and it was not part of the needs assessment questionnaire. The MAR scale refers to an issue that is not included in the most frequently used recovery questionnaires (such as the RSA or the RAQ), reflecting adherence to set values toward the patient, instead of the realities of daily practice, such as those experienced by ward nurses or relative caregivers of patients (Lemelin et al., Reference Lemelin, Bonin and Duquette2009; Edward et al., Reference Edward, Stephenson, Ousey, Lui, Warelow and Giandinoto2016). Even though the internal consistency of this scale was slightly below a satisfactory level, this value could be due to the small number of items (only three items), explaining the low internal consistency (Corbière and Fraccaroli, Reference Corbière, Fraccaroli, Corbière and Larivière2014). Previously a silenced dimension, it may become a point of negotiation between patients, staff and relatives, who are all potential ‘victims’ of violence by one other (Guglielmetti et al., Reference Guglielmetti, Gilardi, Licata and De Luca2016; Onwumere et al., Reference Onwumere, Parkyn, Learmonth and Kuipers2018).
Finally, the TF scale groups together items from two different semantic clusters, i.e., collaboration with formal network and evidence-based practices. Even though this scale combines topics that are already known in the literature and that have been studied with specific scales (Clarke et al., Reference Clarke, Crowe, Oades and Deane2009; Trauer et al., Reference Trauer, Callaly and Herrman2009; Lavoie-Tremblay et al., Reference Lavoie-Tremblay, Sounan, Lavigne, Bonin, Lesage, Denis, Renaud, Maisy, Farand and Racine2008), our findings suggest the presence of a common construct that may refer to professionalism.
Strengths and limitations
The four scales of the MHPCI were validated through a bottom-up approach: 49 items were tested in two samples from different countries, and four factors emerged in both samples. Moreover, participants worked in a wide range of mental health settings, from hospital wards to community teams, with different professional roles and backgrounds. This variety of professionals may ensure that the emerging dimensions have cross-professional and cross-cultural validity that allow the tool to be used across all types of MHSs and diverse professionals, which is consistent with the study's aim.
This study does have some methodological limitations. First, the item generation procedure was not adequately sustained by conceptual mapping. Moreover, since professional culture has not been studied in mental health care, researchers could only hypothesise the constructs by relying on a free interpretation of the existing literature. The lack of conceptual mapping in item generation could also explain the elimination of a large percentage of items during the PCA. Since the elimination involved some specific topics and not others, it could be asserted that the remaining factors are better suited to measure professional culture than other topics that, although they are already present in the literature, do not fit well in a multifactorial solution. Second, response rates were different in the two countries, reflecting differences in administrative procedures: in Italy, researchers presented the questionnaire to the research team during regular staff meetings, whereas in Canada, questionnaires were sent via regular mail. Despite differences in the response rate, the two samples were comparable in terms of participants' characteristics. Third, convergent and discriminant validity were not evaluated (Corbière and Fraccaroli, Reference Corbière, Fraccaroli, Corbière and Larivière2014) since no other instruments were available in Italian and French to perform correlational analysis.
Conclusion and future research
Despite the limitations mentioned above, this study suggests that the MHPCI could be considered a valid and reliable instrument to determine the professional behaviours of mental health services workers. The content of the four scales is consistent with the existing literature on psychosocial rehabilitation, suggesting that the instrument could be used to evaluate staff behaviour regarding four crucial dimensions of mental health care: (1) working with the microsocial environment (families); (2) valuing the principles of psychosocial rehabilitation and evidence-based practices; (3) addressing the intimate life needs of patients and (4) promoting a culture of safety in the work environment in which patients, staff and relatives interact with each other.
Further research is needed to test the differences between specific professional groups (i.e., psychiatrists v. nurses) and to investigate the determinants of professional culture, such as age, experience in mental health settings and training. A further step would be the implementation of the tool to evaluate training programs and user satisfaction.
Availability of data and materials
The data on which this manuscript is based are available upon request. The first author can be contacted with a request within 5 years after the article publication. Researchers are requested to submit a research plan, describing its background, scope and an outline the analyses. Upon approval, a written agreement will be signed stating that the data will only be used for addressing the agreed-upon research scope and not for other purposes.
Acknowledgement
We want to thank Antonio Lora (Dipartimento Salute Mentale ASST Lecco), Wilma Xocco (Dipartimento di Salute Mentale ASL Torino 1), Davide Motto (Cooperativa Sociale Lotta Contro L'Emarginazione) and Marco Meregalli (Cooperativa Sociale Novo Millennio) for promoting this research project among their public and non-profit organisations.
Financial support
The present study was part of a PhD research project; it did not receive any funding and was not used for profit at all.
Conflict of interest
All authors declare that they have no conflicts of interest.
Ethical standards
The authors confirm that all procedures in the study that involved participants comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1964 and its later amendments.
Open access
