The experience of psychiatric in-patient care in the UK has been described as an unpleasant and negative one for many service users. Reference Rose1,Reference Quirk and Lelliott2 Dissatisfaction may be explained by a range of factors, including unavailability of therapeutic interventions, unpleasant built environments, staff being too busy, the stigma of a psychiatric admission, high levels of disturbance on the wards and – for many service users – the experience of being admitted involuntarily. There is some evidence that alternative residential services may be more popular. In north London a crisis house specifically for women experiencing acute mental health problems was highly valued by service users. Reference Killaspy, Dalton, McNicholas and Johnson3,Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington4 Rose et al reported that alternative residential services were preferred by both carers and service users. Reference Rose1 However, there is a lack of quantitative data comparing service user experience in traditional and alternative services.
We aimed to compare satisfaction and experience of admission in four residential alternative services and four paired standard services, as part of a national study of the function and availability of such services in England.
Method
Services were selected from a national survey of residential alternatives in England. Reference Johnson, Gilburt, Lloyd-Evans, Boardman, Leese and Osborn5 The survey identified 131 different services which were categorised into eight main types of residential alternatives, using factor analysis. For the patient experience study, four services were chosen to represent each of the following types of alternative: a clinical crisis house; crisis team beds; a non-clinical (Black minority ethnic) crisis house; and a general therapeutic ward, implementing the Tidal Model of care. Reference Barker6 Johnson et al provide more detailed descriptions of each service and the rationale for their inclusion. Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade7 In contrast to that paper, this experience study did not include the short-stay admission ward (since admissions were too short to allow recruitment of patients) and we only included one (rather than two) non-clinical alternative.
Participants
Potential participants for the patient experience survey were initially identified and approached by staff on each unit. Every person currently receiving in-patient treatment was eligible for inclusion. Staff informed researchers which service users were well enough and willing to discuss the study. We aimed to recruit 40 participants from each of the eight services, close to their point of discharge. Each participant gave informed consent to complete the questionnaires and all were reimbursed for their time.
Measures
Service user satisfaction was assessed with two established user-rated measures. The Client Satisfaction Questionnaire (CSQ) is an eight-item questionnaire measuring patient satisfaction with a mental health service. Reference Attkisson and Zwick8 Scores range from 8 to 32, with higher scores indicating higher satisfaction. The Service Satisfaction Scale – Residential Form (SSS–Res) is a more detailed, 33-item scale measuring aspects of patient satisfaction with residential and in-patient mental health services. Reference Greenfield and Attkisson9 The total SSS–Res score ranges from 33 to 165. The scale produces a total score and four subscales, with ratings for the staff programme (15–75), medication and aftercare (15–55), day and night availability (3–15) and facilities (4–20). Higher scores indicate higher satisfaction.
To assess service user experience we applied the user-rated Admission Experience Survey (AES), a 16-item measure that derives subscales related to perceived coercion at admission (0–5), having a ‘voice’ in decisions (0–3) and negative pressures (0–6). Reference Gardner, Hoge, Bennett, Roth, Lidz and Monahan10 It is designed for assessing the process of admission to hospital or residential acute care and was developed for the MacArthur Coercion Study. Finally, the Ward Atmosphere Scale – Short Form (WAS) is a 40-item user-rated measure of service style, producing ten subscales (0–4), including a unit's practical orientation, levels of anger and aggression, levels of patient autonomy, staff control and support. Reference Moos11
We also recorded age at admission, gender, ethnicity and Mental Health Act status (involuntary or voluntary) during admission, and length of time between admission and completing the questionnaire.
Missing data
When questionnaires were incompletely filled, missing items were pro-rated with the mean of the other scores, up to a predetermined maximum number of replacements (CSQ up to two items, SSS–Res six items, AES subscales one item, WAS subscales one item). Measures with more than the maximum number of missing items were excluded.
Statistical analysis
Data were entered using SPSS version 14.0 for Windows, and then converted to Stata version 10 for further analysis. Satisfaction and experience outcomes were initially compared between each local pair of services and then aggregated according to alternative or traditional service. Data were inspected for normality of distribution and compared using t-tests and chi-squared tests if appropriate. Where distributions deviated from normal, non-parametric tests were employed. Potential confounding variables were also explored with univariate tests. Multiple linear regression was then used to explore associations between alternatives and satisfaction/experience outcomes. All regression models were adjusted for clustering by service, using the Stata cluster option. The proportion of variance in outcomes explained by each multivariate model (R 2 statistic) was noted. When outcome variables were not normally distributed, a sensitivity analysis was performed, using ordinal logistic regression to explore whether significant associations were stable irrespective of method of regression analysis.
Results
A total of 447 patients were asked to participate in completing satisfaction questionnaires (alternative services n = 186, standard services n = 261). Completed forms were obtained from 314 respondents (alternative services n = 142, standard services n = 172), an overall response rate of 70% (alternative services 76%, standard services 66%). Because of its similarities to standard wards in role, patient profile and care provision, Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade7,Reference Lloyd-Evans, Johnson, Morant, Gilburt, Osborn and Jagielska12 results for the Tidal Model ward and its comparison service are excluded from main analyses and presented separately in this paper, and the main comparisons reported and discussed are between the three community alternatives and their three local comparison wards.
Table 1 describes the distribution of participants across the different units, including their sociodemographic features and the proportions admitted compulsorily. Compared with people in the comparison units, participants in the residential alternatives were more likely to be female, less likely to define their ethnicity as White and less likely to have been detained involuntarily. Table 2 examines the main experience outcomes according to whether the participant was admitted to an alternative or a standard in-patient unit. Results for the Tidal Model are presented in online Table DS1.
Table 1 Participant characteristics
| Alternative units n = 102 | Comparison units n = 125 | P | Tidal Model ward n = 40 | Tidal Model comparison n = 47 | P | |
|---|---|---|---|---|---|---|
| Age, years: mean (s.d.) | 40.2 (12.6) | 38.4 (13.2) | 0.289 | 39.6 (11.7) | 40.4 (13.8) | 0.764 |
| Gender, n (%) | ||||||
| Male | 47 (46) | 77 (62) | 0.019 | 16 (40) | 22 (47) | 0.523 |
| Ethnicity, n (%) a | ||||||
| White British | 58 (58) | 88 (72) | 0.001 | 20 (51) | 33 (88) | 0.047 |
| White Irish | 0 (0) | 1 (1) | 0 (0) | 0 (0) | ||
| White other | 1 (1) | 3 (2) | 0 (0) | 2 (5) | ||
| Black Caribbean | 21 (21) | 11 (9) | 5 (13) | 1 (3) | ||
| Black African | 16 (16) | 4 (3) | 3 (8) | 0 (0) | ||
| Black other | 0 (0) | 6 (5) | 1 (3) | 0 (0) | ||
| Pakistani | 0 (0) | 4 (3) | 5 (13) | 0 (0) | ||
| Bangladeshi | 0 (0) | 0 (0) | 1 (3) | 0 (0) | ||
| Asian other | 0 (0) | 1 (1) | 1 (3) | 0 (0) | ||
| Mixed: White/Black Caribbean | 2 (2) | 2 (2) | 1 (3) | 1 (3) | ||
| Mixed: White/Black African | 2 (2) | 0 (0) | 0 (0) | 0 (0) | ||
| Mixed: White/Asian | 0 (0) | 0 (0) | 1 (3) | 0 (0) | ||
| Chinese | 0 (0) | 1 (1) | 0 (0) | 0 (0) | ||
| Other | 0 (0) | 1 (1) | 1 (3) | 0 (0) | ||
| Admission status, n (%) | ||||||
| Involuntary | 5 (5) | 46 (37) | <0.001 | 21 (54) | 6 (14) | <0.001 |
| Patients in each service, n (%) | ||||||
| Crisis beds | 41 (40) | 43 (34) | ||||
| Clinical crisis house | 20 (20) | 40 (32) | ||||
| Non-clinical alternative | 41 (40) | 42 (34) | ||||
| Time to completion of questionnaire, days: mean (s.d.) | 10.7 (13) | 57.3 (152) | 0.027 | 47.6 (73) | 44.8 (60) | 0.854 |
a. Ethnicity data were not completed by some participants
Table 2 Satisfaction and experience outcomes in traditional units (n = 125) and alternative units (n = 102)
| Outcome scores | ||||||
|---|---|---|---|---|---|---|
| Alternative service Mean (s.d.) | Standard service Mean (s.d.) | Difference Mean (95% CI) | P | |||
| Satisfaction | ||||||
| CSQ score (8–32) | 26.4 (4.9) | 23.1 (6.6) | 3.3 (1.8 to 4.9) | <0.001 | ||
| SSS–Res scores | ||||||
| Total SSS–Res score (33–165) | 125.8 (25.8) | 114.5 (21.7) | 11.4 (5.0 to 17.7) | <0.001 | ||
| Staff programme (15–75) | 57.7 (10.4) | 52.5 (12.5) | 5.2 (2.1 to 8.2) | 0.001 | ||
| Medication and aftercare (15–55) | 40.8 (4.0) | 36.8 (9.7) | 4.0 (2.1 to 8.2) | 0.002 | ||
| Day–night availability (3–15) | 11.1 (2.3) | 10.4 (2.9) | 0.7 (0.0 to 1.4) | 0.050 | ||
| Facilities (4–20) | 16.5 (2.4) | 14.6 (3.3) | 1.9 (1.1 to 2.7) | <0.001 | ||
| Admission experience | ||||||
| AES subscales | ||||||
| Perception of coercion (0–5) | 1.3 (1.6) | 2.6 (2.1) | –1.3 (–1.8 to –0.8) | <0.001 | ||
| Negative pressures (0–6) | 0.65 (1.3) | 2.5 (2.3) | –1.8 (–2.3 to –1.3) | <0.001 | ||
| Voice (0–3) | 2.4 (1.0) | 1.5 (1.2) | 0.9 (0.6 to 1.2) | <0.001 | ||
| Ward atmosphere | ||||||
| WAS subscales (0–4) | ||||||
| Involvement | 2.7 (1.2) | 2.5 (1.1) | 0.2 (–0.1 to 0.5) | 0.255 | ||
| Support | 2.7 (1.3) | 1.9 (1.3) | 0.8 (0.4 to 1.2) | <0.001 | ||
| Spontaneity | 1.9 (1.2) | 1.8 (1.1) | 0.1 (–0.2 to 0.5) | 0.354 | ||
| Autonomy | 2.4 (0.9) | 2.1 (1.1) | 0.4 (0.1 to 0.6) | 0.010 | ||
| Practical orientation | 2.4 (1.3) | 2.2 (1.1) | 0.2 (–0.1 to 0.5) | 0.231 | ||
| Personal problems orientation | 1.5 (1.2) | 1.7 (1.1) | –0.2 (–0.5 to 0.1) | 0.175 | ||
| Anger and aggression | 1.1 (1.0) | 2.3 (1.0) | –1.2 (–1.4 to –0.9) | <0.001 | ||
| Order and organisation | 3.3 (0.9) | 2.7 (1.1) | 0.6 (0.3 to 0.8) | <0.001 | ||
| Programme clarity | 2.9 (1.1) | 2.5 (1.3) | 0.4 (0.1 to 0.8) | 0.021 | ||
| Staff control | 2.0 (1.1) | 2.0 (1.0) | 0.0 (–0.2 to 0.3) | 0.782 | ||
AES, Admission Experience Scale; CSQ, Client Satisfaction Questionnaire; SSS–Res, Service Satisfaction Scale – Residential; WAS, Ward Atmosphere Scale
People admitted to residential community alternatives had significantly higher satisfaction scores on both the CSQ and the SSS–Res total scales. Satisfaction was also significantly higher on each subscale on the SSS–Res. Scores for coercion and negative pressures at admission on the AES were significantly lower in alternative units, whereas scores for ‘voice’ were significantly greater in alternatives. The Ward Atmosphere Scale revealed fewer consistent differences between alternatives and in-patient units. However, levels of anger and aggression were rated as significantly higher on the traditional in-patient units, whereas the alternative units scored significantly more highly for programme clarity, autonomy, order and organisation, and support. These associations on the CSQ, SSS–Res, AES and WAS all remained significant (at P<0.05) whether parametric or non-parametric univariate analyses were performed. The other variable most strongly associated with total satisfaction on the CSQ was detention under the Mental Health Act (t-test for detained v. not detained, P<0.001). Gender, age, and time elapsed between admission and completion of the experiences questionnaire were not significantly associated with the total CSQ score.
Table 3 contains results from the multivariate analyses. For each outcome two models are presented: first, a model including only type of service (alternative v. traditional), and second, a model adjusted for age, gender, ethnicity (binary: White v. Black and other) and detention under the Mental Health Act. After full adjustment, alternative units were associated with significantly greater levels of satisfaction on the CSQ, but the association with greater scores on the total SSS–Res lost significance at the P<0.05 level. Associations between each subscale of the SSS–Res and service type also lost significance (P>0.05) after adjustment and these results are not included in the table. On the AES, admission to residential alternatives was still significantly associated with greater levels of ‘voice’ and autonomy as well as significantly lower levels of coercion and negative pressures. From the Ward Atmosphere Scale, the only subscale that remained significantly associated with type of unit was the anger and aggression scale, with greater levels reported in the traditional units.
Table 3 Comparison of satisfaction and experience in traditional and alternative in-patient units: regression analysis
| R 2 | Regression coefficient (95% CI) | P | |
|---|---|---|---|
| Satisfaction | |||
| Total CSQ score | |||
| Model 1 a | 0.07 | 3.34 (0.87 to 5.81) | 0.018 |
| Model 2 b | 0.17 | 2.35 (0.58 to 4.13) | 0.019 |
| Total SSS–Res score | |||
| Model 1 | 0.05 | 11.4 (1.14 to 21.6) | 0.036 |
| Model 2 | 0.11 | 9.44 (–1.69 to 20.6) | 0.081 |
| Perception of coercion | |||
| AES | |||
| Model 1 | 0.11 | –1.34 (–2.08 to –0.060) | 0.006 |
| Model 2 | 0.27 | –0.77 (–1.45 to –0.08) | 0.035 |
| Negative pressures | |||
| AES | |||
| Model 1 | 0.19 | –1.84 (–2.82 to –0.85) | 0.005 |
| Model 2 | 0.36 | –1.38 (–2.44 to –0.32) | 0.020 |
| Voice | |||
| AES | |||
| Model 1 | 0.14 | 0.91 (0.43 to 1.39) | 0.004 |
| Model 2 | 0.32 | 0.53 (0.13 to 0.92) | 0.019 |
| Anger and aggression | |||
| WAS subscale | |||
| Model 1 | 0.24 | –1.13 (–1.31 to –0.96) | < 0.001 |
| Model 2 | 0.28 | –1.18 (–1.35 to –1.01) | < 0.001 |
AES, Admission Experience Scale; CSQ, Client Satisfaction Questionnaire; SSS–Res, Service Satisfaction Scale – Residential; WAS, Ward Atmosphere Scale
a. Model 1: service type (community alternative or standard)
b. Model 2: service type, adjusted for detention under the Mental Health Act, age, gender and ethnicity. All models are adjusted for clustering by service
We performed three additional sensitivity analyses. First, the statistical significance of each multivariate result in Table 3 remained stable when ordinal, rather than linear, regression was applied. Second, the multivariate results were almost identical when geographical area was added to the model. Third, adding duration of time between admission and completion of the questionnaires made no substantial difference. The stability of the sensitivity analyses was reflected in both the proportion of variance explained by the models and the statistical level of significance for each association between type of service (alternative v. traditional) and outcomes.
The experience results for each individual alternative service and its comparison standard service are given in online Table DS1. Numbers are small, but in the main they suggest greater satisfaction with the alternative services than with their comparison; however, the Tidal Model ward and its comparison have similar scores.
Discussion
Participants in the three residential alternative services considered to be the clearest alternatives to traditional in-patient care were more satisfied with their care than participants in traditional units. However, levels of satisfaction in both types of service were at the higher end of the range for both of the satisfaction scales we used. People admitted to residential alternative units also reported less coercion and negative pressures during their admission process and described greater levels of autonomy and voice. Univariate analysis indicated they were more satisfied with facilities, staff programmes, day and night availability of services, and their medication and aftercare. However, there were important differences between participants in the different types of service, particularly the numbers detained under the Mental Health Act during their stay, as well as differences in ethnicity and gender. When these factors were accounted for in the multivariate analysis, satisfaction was still significantly greater in the alternative units on the CSQ, as were levels of autonomy and voice. However, detention under the Mental Health Act attenuated some of the reported differences in descriptions of ward atmosphere, such as greater levels of support, autonomy and order, which were no longer significant. Thus the differences between people detained in each service type explain some, but not all, of the differences in overall patient experience.
Some admission experiences did not differ between the services. Compared with traditional units, people in residential alternatives reported similar levels of staff control, focus on personal problems, practical orientation, involvement and spontaneity.
Strengths of the study
This is the first national in-depth study of patient experiences within residential alternatives to in-patient psychiatric care. Each service was chosen to represent one of the main types of alternative that had been identified in a previous national survey of all identifiable novel residential mental health services. Reference Johnson, Gilburt, Lloyd-Evans, Boardman, Leese and Osborn5
The greater levels of satisfaction within residential alternatives, especially in relation to the CSQ, were generally robust whichever form of analysis was used to explore them, including different statistical methods and adjusting for different possible explanatory variables such as geographical area and duration of admission. This difference in satisfaction is an important service user outcome, given that a comparison of clinical outcomes revealed no clear-cut clinical or economic advantage for either type of service. Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn13
The variability in our quantitative experience findings is consistent with results from qualitative interviews undertaken with people admitted to the same services. Gilburt et al found that although many people preferred admission to residential alternatives, they also identified a number of similarities between the two types of service, including the type of care that they received during their stay. Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14 However, there are some consistent themes between the two methodological approaches that add weight to our findings. In particular, the qualitative work also confirmed that patients favoured alternative services in terms of their atmosphere, with lower levels of disturbance and less experience of negative coercion. Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14
In a separate paper we compared the content of care provided in different service types, Reference Lloyd-Evans, Johnson, Morant, Gilburt, Osborn and Jagielska12 and once again there were similarities between alternative residential services and traditional units, such as access to activities and the quantity of time spent with staff. This is consistent with our findings from the Ward Atmosphere Scale, where after adjustment, the only distinguishing features of alternatives were lower levels of anger and aggression.
Service user satisfaction seems to be one key difference between the services, yet it is not simply explained by the type of care received. Reference Lloyd-Evans, Johnson, Morant, Gilburt, Osborn and Jagielska12 Our findings suggest this satisfaction may hinge on feeling less coerced at admission and having more influence over the type of care received. In conjunction with the findings from Gilburt et al, Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn14 this points to a future research agenda which is more focused on interpersonal and relationships variables rather than concentrating exclusively on interventions and outcomes.
Limitations
This study involved only one exemplar service from four of eight types of residential alternative, so the generalisability of our results to other services cannot be guaranteed. Our observational study was not based on random sampling or on random allocation to services, and therefore sampling bias and residual confounding are important considerations. It is possible that participants in the two types of services were not fully representative of all those in the services. Although we adjusted for demographic variables, including area and length of stay, we did not have further data to explore how differences in clinical presentation in the services (such as severity of illness) might account for the increased satisfaction in alternative services. However, detention under the Mental Health Act is one proxy for severity and although it attenuated some results, satisfaction remained greater in alternatives, with less negative experiences and perception of coercion. Johnson et al reported that although alternative services do cater for similar groups of patients to traditional in-patient units, they admit fewer patients with a diagnosis of psychosis and fewer patients who may pose a risk of violence to others. Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade7 We were unable to control for these variables in this part of the study and it is possible that they explain some of the differences in admission experiences and satisfaction that we report. Similarly, higher levels of anger and aggression have previously been associated with lower levels of satisfaction, Reference Svensson and Hansson15 as have perceived levels of coercion. Reference Iversen, H⊘yer and Sexton16 Since anger and aggression were rated as more prevalent in our traditional in-patient units, this may explain the lower levels of satisfaction we observed in such settings.
We now require further understanding of the causes of satisfaction among in-patient service users. Our final model explained less than 20% of the variance in satisfaction, hence future research may require more in-depth explorations of factors present at the point of admission and then throughout each in-patient stay. Our participants completed their satisfaction ratings near to discharge and this might explain the relatively high levels of satisfaction across all units. Future research might assess satisfaction at multiple time-points during admission, and assess predictors of any changes in satisfaction within individuals, perhaps in relation to quality of care, clinical progression and recovery as well as environmental influences within the unit (such as the quality and nature of relationships with staff, other service users and agencies).
Satisfaction at discharge may be an important clinical outcome with potential impact far beyond the experience of an individual admission. It may shape individuals' perception of their care as a whole and their willingness to engage with a range of mental health services in the future. Since the residential alternative units in this study were not inferior in terms of costs and clinical outcomes in either the short term or the medium term, Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn13,Reference Byford, Sharac, Lloyd-Evans, Gilburt, Osborn and Leese17 greater satisfaction levels are a compelling reason to support further development of such services as one component of acute adult mental health services.
Funding
This project was funded by the National Institute for Health Research Service Delivery and Organisation programme (project number 08/1304/075).
Acknowledgements
This study was undertaken in the context of the NIHR Specialist Mental Health Biomedical Research Centre at the Institute of Psychiatry, King's College London and the South London and Maudsley NHS Foundation Trust. The study was supported by the Mental Health Research Network (MHRN) and associated with the MHRN acute care group (convenor S.J.). The views expressed in this paper are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
eLetters
No eLetters have been published for this article.