Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-08T17:32:27.124Z Has data issue: false hasContentIssue false

Effectiveness and cost-effectiveness of admissions to women's crisis houses compared with traditional psychiatric wards: pilot patient-preference randomised controlled trial

Published online by Cambridge University Press:  02 January 2018

L. Howard*
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
C. Flach
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
M. Leese
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
S. Byford
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
H. Killaspy
Affiliation:
Department of Mental Health Sciences, University College London, London, UK
L. Cole
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
C. Lawlor
Affiliation:
Department of Mental Health Sciences, University College London, London, UK
J. Betts
Affiliation:
Department of Mental Health Sciences, University College London, London, UK
J. Sharac
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
P. Cutting
Affiliation:
South London and Maudsley National Health Service (NHS) Foundation Trust, Acute In-patient and Complex Care Services, Beckenham
S. McNicholas
Affiliation:
Camden and Islington NHS Foundation Trust, London
S. Johnson
Affiliation:
Health Service and Population Research Department, Institute of Psychiatry, King's College London
*
Louise Howard, Health Service and Population Research Department, Institute of Psychiatry, Box PO29, De Crespigny Park, London SE5 8AF, UK. Email: [email protected].
Rights & Permissions [Opens in a new window]

Abstract

Background

Women's crisis houses have been developed in the UK as a less stigmatising and less institutional alternative to traditional psychiatric wards.

Aims

To examine the effectiveness and cost-effectiveness of women's crisis houses by first examining the feasibility of a pilot patient-preference randomised controlled trial (PP–RCT) design (ISRCTN20804014).

Method

We used a PP–RCT study design to investigate women presenting in crisis needing informal admission. The four study arms were the patient preference arms of women's crisis house or hospital admission, and randomised arms of women's crisis house or hospital admission.

Results

Forty-one women entered the randomised arms of the trial (crisis house n = 19, wards n = 22) and 61 entered the patient-preference arms (crisis house n = 37, ward n = 24). There was no significant difference in outcomes (symptoms, functioning, perceived coercion, stigma, unmet needs or quality of life) or costs for any of the groups (randomised or preference arms), but women who obtained their preferred intervention were more satisfied with treatment.

Conclusions

Although the sample sizes were too small to allow definite conclusions, the results suggest that when services are able to provide interventions preferred by patients, those patients are more likely to be satisfied with treatment. This pilot study provides some evidence that women's crisis houses are as effective as traditional psychiatric wards, and may be more cost-effective.

Type
Paper
Copyright
Copyright © Royal College of Psychiatrists, 2010 

In the past 25 years several women's crisis houses have opened around the UK, offering an innovative alternative to psychiatric hospital admission which is potentially less stigmatising, coercive and institutionalised. Reference Johnson, Gilburt, Lloyd-Evans, Osborn, Boardman and Leese1 The development of women's crisis houses was recommended in the UK Department of Health Women's Implementation Guidance. 2 A few descriptions and evaluations of women's crisis houses have been published, Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington3Reference Meiser-Stedman, Howard and Cutting5 but these small uncontrolled studies have substantial methodological limitations. At present there is no clear evidence on whether women's crisis houses are effective or cost-effective, although a qualitative study indicated that a women's crisis house was highly valued and often preferred to conventional in-patient treatment. Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington3 We aimed to evaluate the effectiveness and cost-effectiveness of women's crisis houses and decided to use a patient-preference randomised controlled trial (PP–RCT) study design for two reasons: first, this was acceptable to staff and service users, and second, strong patient preferences can lead to patients refusing consent to enter a trial and undergo randomisation, which can lead to bias and may weaken the external validity of the study results. Reference Torgerson and Sibbald6,Reference King, Nazareth, Lampe, Bower, Chandler and Morou7 The PP–RCT involves a conventional randomised controlled design but data are also collected for those who do not wish to be randomised but consent to data collection, and their preferred treatment is offered. This design has been little used in psychiatry but the current available evidence suggests it may prove to be a useful paradigm when evaluating mental health services. Reference Howard and Thornicroft8 This paper describes the results of a pilot trial which was funded to investigate the feasibility of a future larger PP–RCT; our conclusions on the feasibility of a future trial are discussed elsewhere. Reference Howard, Leese, Byford, Killaspy, Cole and Lawlor9

Method

Our study design (PP–RCT) involved four arms. Women consenting to participate in the study but not to randomisation entered the patient-preference arms of the study and entered either the women's crisis house or the hospital, depending on their preference for treatment location. Women who consented both to participate in the study and to randomisation were randomised to the women's crisis house or the hospital by means of an independent 24 h telephone randomisation service at the time of recruitment. Recruitment was carried out between March and August 2007: a companion paper gives a more detailed account of the procedure. Reference Howard, Leese, Byford, Killaspy, Cole and Lawlor9 The overarching aim of this pilot study was to test the feasibility of a future PP–RCT; we report its results because these are the only available data on the effectiveness and cost-effectiveness of women's crisis houses.

The intervention – admission to a women's crisis house – was available at two London centres where there were also psychiatric wards that admitted women living in the same catchment areas. The women's crisis houses were located in ordinary residential streets, had a domestic atmosphere and were funded by the UK National Health Service (NHS). Staff were available 24 h per day and were either nurses or healthcare workers with a background in mental health. Women's crisis houses cannot admit women who are detained under the Mental Health Act at admission although they can accept them as transfers from hospital on leave. In addition, women who are considered to be at current risk of violent behaviour, who are misusing drugs or alcohol such that they require detoxification under medical supervision or who are unable to engage in a safety plan and therefore need constant supervision are not admitted to women's crisis houses. The two houses included in this study had the following operational characteristics.

Croydon crisis house

The Croydon women's crisis house opened in 1999 and is a nursing-led service with support from a female psychiatrist. It has 8 beds and a mean length of stay of 21 days. Referral is made by telephone, by the woman herself or a concerned other (including carers or health professionals). After telephone assessment, if appropriate the woman is invited to see a staff member at the women's crisis house for further assessment. The Croydon catchment area also has a women-only and a mixed acute admission ward.

North London crisis house

The north London Drayton Park women's crisis house opened in December 1995. This serves the inner-London boroughs of Camden and Islington. It has 12 beds (plus 4 beds for children) and a mean length stay of 19 days. Both self-referrals and referrals from professionals are accepted. After referral, women are invited to attend the house for assessment. Other psychiatric services in the catchment area include women-only in-patient wards and mixed wards.

Study population

The study population consisted of all women requiring voluntary admission who could be admitted to a psychiatric in-patient ward or women's crisis house. Exclusion criteria included women detained under the Mental Health Act, or those who would not be considered clinically appropriate for admission to a women's crisis house, for example because of violence or needing intensive observation.

Procedure

There were a number of potential entry points to the study. First, mental healthcare professionals from home treatment teams, psychiatric liaison teams in accident and emergency departments, and duty workers in community mental health teams were asked to discuss the study with any women who needed to be admitted informally to hospital or a women's crisis house. Second, women admitted to psychiatric wards or the women's crisis house who had not been approached when the admission was being arranged were traced through local bed management teams and approached by a research worker as soon as possible after being admitted. Similarly, self-referrals (to the Croydon women's crisis house) were told about the project and asked if they would agree to participation.

Potential participants were informed that although a place might not be available at the allocated or preferred service they would be transferred there as soon as a place was available and transfer was clinically appropriate (this was similar to normal clinical practice, in which a place might not be available immediately but would be provided as soon as practically possible). As we knew that these women would be acutely distressed and would find it difficult to read a long participant information sheet during the process of admission, we provided them with a brief written explanation of the interventions and randomisation, to allow the women to make a decision regarding randomisation to treatment condition. The women were contacted by the research worker as soon as possible after admission and were then provided with a full participant information sheet detailing the research procedures, including interview measures at baseline and follow-up.

Ethical approval was obtained from the National Hospital for Neurology and Neurosurgery and the Institute of Neurology Joint Research Ethics Committee (06/Q0512/104) and included ethical approval to include women without the capacity to consent to the trial using the provisions of the UK Mental Capacity Act 2005. The trial was registered with the International Standard Randomised Controlled Trials Registry (ISRCTN20804014).

Outcome measures

Measures at baseline included sociodemographic details, past psychiatric history, and preferences for admission to psychiatric hospital or women's crisis house on a five-point Likert scale from ‘definitely crisis house’ to ‘definitely hospital ward’ – this was dichotomised into a preference for either ward or crisis house. The actual treatment that women received was defined as the location of the majority of their treatment (ward or crisis house). Data on the participants' treatment preference and the actual treatment they received were combined to group them into ‘Did not receive preference’ and ‘Received preference’. We used standardised measures of functioning (Global Assessment of Functioning; GAF), symptoms (Brief Psychiatric Rating Scale; BPRS), quality of life (Manchester Short Assessment of Quality of Life; MANSA), health-related quality of life (EQ–5D), unmet needs (Camberwell Assessment of Need; CAN), satisfaction with services (Verona Service Satisfaction Scale; VSSS), coercion (MacArthur Perceived Coercion Scale) and stigma (Ritsher scale). Reference Endicott, Spitzer, Fleiss and Cohen10Reference Ritsher, Otilingam and Grajales17 Patients were followed up 12 weeks later and re-interviewed with these measures.

Our primary outcomes were level of functioning and symptoms, because these were the primary outcomes most important to women patients and staff interviewed in a previous qualitative study. Our secondary outcomes were quality of life, unmet needs, satisfaction with services, coercion and stigma.

Economic measures

The economic evaluation took a broad perspective, including all hospital and community health and social services, productivity losses resulting from time off work due to illness, and criminal justice resources. Economic data were collected in interview at baseline (covering the 12-week period prior to study entry) and at the 12-week follow-up point (covering the period from baseline to follow-up) using the Adult Service Use Schedule (AD–SUS), an instrument designed on the basis of previous studies of adult mental health populations. Reference Byford, Fiander, Torgerson, Barber, Thompson and Burns18Reference Kuyken, Byford, Taylor, Watkins, Holden and White20 The AD–SUS asks recipients for the number and length of contacts with various services and professionals relevant to the population of interest over the period of interest. To ensure that this measure covered all services relevant to the current population, it was adapted following discussions with clinical team members and through pilot testing at the baseline interviews.

All unit costs were for the financial year 2006–7, the most recent financial year over which the trial data were collected. The women's crisis houses were costed directly from annual budgets provided by the crisis house managers. All hospital contacts were costed using the Department of Health's national reference costs for NHS Trusts. 21 National UK unit costs, inflated where necessary, were applied to community health and social services, supported accommodation and criminal justice resources. Reference Finn, Hyslop and Truman2226 Productivity losses were calculated using the human capital approach, which involves multiplying the individual's salary by reported days off work due to illness. Reference Koopmanschap and Rutten27

Statistical analysis

Data were entered into Access and then transferred to Stata version 9 for Windows. Descriptive statistics (means and standard deviations or percentages) were recorded for baseline characteristics of the following groups:

  1. (a) participants randomised to the ward and those randomised to the crisis house;

  2. (b) participants who refused randomisation and were allocated treatment by their own preference to ward compared with those choosing the crisis house;

  3. (c) participants allocated (by randomisation or preference) to the ward compared with those allocated to the crisis house;

  4. (d) all participants whether randomised or not who spent more than half of their in-patient stay on the ward compared with those who spent most of their stay at the crisis house.

Differences were tested using t-tests and chi-squared tests for all except the comparison of the randomised groups (for which such tests would not be meaningful). Differences between patients who were followed up and those who were lost to follow-up were examined in terms of baseline characteristics. We also examined such differences for the randomised participants (comparing those who were followed up with those who were not) and patient-preference participants.

A simple bivariate analysis of outcome measures was completed to compare outcomes at follow-up in all four groups listed above. Differences were assessed using t-tests. Finally, the independent differences in functioning (GAF), symptoms (BPRS), quality of life (MANSA), health-related quality of life (EQ–5D), coercion (MacArthur), stigma (Ritsher), unmet needs (CAN) and satisfaction with services (VSSS) after treatment at the crisis house compared with the ward were assessed using linear regression, controlling for possible confounders. The impact of treatment preference on outcomes was also investigated. Variables associated with loss to follow-up were included in the regression to account for selection bias in the sample results due to missing values. No adjustment for multiple testing was made. Linear regression was used to investigate whether participants who received their preferred treatment tended to experience better outcomes than those who did not. The analysis was repeated for only those who were randomised to a treatment, and then for all participants whether randomised or from the patient-preference group together.

Differences in mean costs between the four groups described above were analysed using standard parametric t-tests with the validity of results confirmed using bias-corrected, non-parametric bootstrapping (repeat resampling). Reference Efron and Tibshirani28 Despite the skewed nature of cost data, this approach is recommended to enable inferences to be made about the arithmetic mean. Reference Thompson and Barber29 Cost-effectiveness was explored in terms of quality-adjusted life-years (QALYs), calculated using the EQ–5D measure of health-related quality of life. Reference Brooks13 Uncertainty around the cost and effect estimates is presented using cost-effectiveness acceptability curves which show the probability that an intervention is cost-effective compared with the alternative for a range of maximum monetary values that a decision-maker might be willing to pay for a particular unit change in outcome, in this case QALYs. Reference Fenwick, Claxton and Sculpher30 All economic analyses were adjusted for baseline costs, baseline EQ–5D score and variables associated with loss to follow-up.

Results

Participants

A total of 103 women were recruited to the study, 42 to the randomised arms and 61 to the patient-preference arms (Fig. 1). One patient was not admitted after recruitment, so no baseline questionnaire was completed for her and she was excluded from the study because she did not receive either intervention; 102 women were therefore included in the baseline analyses. Most women (82%, n = 84) received their allocated or chosen service, although 9 women were not transferred immediately and had to wait 3–27 days; the remaining 18% did not receive their allocated or chosen service. We therefore carried out analyses by service received for more than 50% of the admission, in addition to examining differences for women in the randomised arms and patient-preference arms.

The average age of the recruited participants was 37.5 years (s.d. = 11.1). Sociodemographic and clinical characteristics of the women are presented in Table 1; no significant difference was found between women agreeing to randomisation and those declining randomisation and entering the patient-preference arms, other than that the latter had higher average numbers of unmet needs (P = 0.005).

Table 1 Sociodemographic and clinical characteristics of women agreeing to and declining randomisation

Agreeing to randomisation n = 41 Declining randomisation n = 61 t or χ2 P
Age, years: mean (s.d.) 36.1 (2.0) 37.6 (1.3) 0.67 (98) 0.51
Diagnosis, n (%)
   Schizophrenia 7 (20) 10 (18)
   Mood/anxiety disorder 10 (29) 21 (38)
   Bipolar disorder 10 (29) 11 (20)
   Personality disorder 8 (23) 13 (24) 1.29 (d.f. = 3) 0.73
Ethnicity, n (%)
   White 25 (66) 43 (74) 0.77
   Black or other 13 (34) 15 (26) (d.f. = 1) 0.38
Previous admission, n (%) a 25 (39) 39 (61) 0.44 (d.f. = 1) 0.83
GAF score: mean (s.d.) 50 (16) 46 (14.5) 1.17 0.25
BPRS score: mean (s.d.) a 48 (9.8) 49 (9.7) –0.81 0.42
CAN unmet needs score: mean (s.d.) b 4.5 (3.3) 6.6 (3.3) –2.89 (d.f. = 85) 0.005 **
MANSA score: mean (s.d.) c 3.8 (0.8) 3.4 (1.01) 1.67 (d.f. = 94) 0.10
EQ–5D score mean (s.d.) c 0.49 (0.33) 0.41 (0.34) –1.14 (d.f. = 94) 0.26

BPRS, Brief Psychiatric Rating Scale; CAN, Camberwell Assessment of Need; GAF, Global Assessment of Functioning; MANSA, Manchester Short Assessment of Quality of Life

a. Data available for 93 participants

b. Data available for 87 participants

c. Data available for 96 participants

** P < 0.01

Fig. 1 Study profile.

No substantial difference in baseline characteristics was found between those randomised to the ward and those randomised to the crisis house. There were some differences in baseline characteristics when comparing women in the patient-preference arms, with women who preferred the crisis house being significantly younger and having lower baseline functioning (GAF) than those who preferred the ward (Table 2). Women who spent most of their treatment at the crisis house (either by randomisation or preference) had significantly lower quality of life (MANSA), more perceived stigma (Ritsher total score) and more unmet needs (CAN) but less coercion (MacArthur total score) at baseline, and were more likely to be from the borough of Croydon than women allocated to the ward (Table 2).

Table 2 Comparison of baseline characteristics of women given ward and crisis house treatments categorised by method of analysis

Randomised Patient preference Preference and randomised arms together Treatment received > 50% of time
Ward Crisis house Ward Crisis house P Ward Crisis house P Ward Crisis house P
Participants, n 22 19 24 37 46 56 49 51
Age, years: mean (s.d.) 40 (13.1) 34 (11.0) 41 (11.7) 36 (8.8) 0.04 41 (12.3) 35 (9.6) 0.01 39 (12.3) 36 (9.4) 0.14
Area of service, n (%)
   Camden & Islington 14 (64) 10 (53) 14 (58) 21 (57) 28 (61) 31 (55) 33 (67) 24 (47)
   Croydon 8 (36) 9 (47) 10 (42) 16 (43) 0.90 18 (39) 25 (45) 0.58 16 (33) 27 (53) 0.04
Ethnicity, n (%)
   White 14 (70) 11 (61) 17 (81) 27 (73) 31 (76) 38 (69) 33 (75) 34 (68)
   Black or other 6 (30) 7 (39) 4 (19) 10 (27) 0.50 10 (24) 17 (31) 0.48 11 (25) 16 (32) 0.45
Diagnosis, n (%)
   Schizophrenia and related 3 (16) 4 (25) 5 (22) 5 (16) 8 (19) 9 (19) 11 (24) 5 (12)
   Mood disorder 6 (32) 4 (25) 7 (30) 14 (44) 13 (31) 18 (38) 15 (33) 16 (37)
   Bipolar disorder 5 (26) 5 (31) 6 (26) 5 (16) 11 (26) 10 (21) 12 (27) 8 (19)
   Personality disorder 5 (26) 3 (19) 5 (22) 8 (25) 0.62 a 10 (24) 11 (23) 0.91 7 (16) 14 (33) 0.15
Have a care coordinator, n (%) 8 (42) 10 (59) 12 (75) 17 (49) 0.08 20 (57) 27 (52) 0.63 25 (64) 20 (44) 0.06
Previous in-patient, n (%) 13 (62) 13 (76) 16 (84) 22 (61) 0.08 29 (73) 35 (66) 0.51 31 (74) 31 (63) 0.28
Scores: mean (s.d.)
   GAF 49 (14.3) 51 (17.9) 52 (16.3) 43 (12.7) 0.03 50 (15.2) 46 (14.9) 0.13 50 (17.2) 45 (12.8) 0.11
   BPRS 47 (9.0) 48 (10.9) 48 (12.7) 50 (7.5) 0.35 47 (11.0) 49 (8.7) 0.34 47 (11.2) 49 (7.9) 0.41
   MANSA total 3.7 (0.9) 3.8 (0.6) 3.7 (1.1) 3.3 (0.9) 0.09 3.7 (1.0) 3.4 (0.9) 0.15 3.8 (0.9) 3.4 (0.9) 0.04
   CAN unmet needs 5.3 (3.8) 3.6 (2.4) 6.1 (3.9) 6.9 (3.0) 0.42 5.7 (3.8) 5.8 (3.2) 0.93 4.7 (3.3) 6.5 (3.3) 0.01
   MacArthur total 5.3 (4.9) 5.3 (4.5) 4.5 (4.9) 2.9 (3.1) 0.12 4.9 (4.8) 3.7 (3.8) 0.16 5.4 (4.8) 3.2 (3.6) 0.01
   Ritsher total 68 (18.2) 68 (7.1) 73 (16.7) 76 (9.8) 0.57 71 (17.0) 73 (9.6) 0.40 68 (16.2) 75 (8.8) 0.02

BPRS, Brief Psychiatric Rating Scale; CAN, Camberwell Assessment of Need; GAF, Global Assessment of Functioning; MANSA, Manchester Short Assessment of Quality of Life

a. Fisher's exact test due to small expected values in some cells

Seventy per cent (n = 72) of women interviewed at baseline completed follow-up interviews (68% of those who had agreed to randomisation and 72% of those in the patient-preference arm) (see Fig. 1). Women who were allocated to the crisis house were statistically significantly more likely to participate in the follow-up (P = 0.05). Among the randomised patients, those not seen at follow-up had a higher baseline mean MANSA score than those who completed the study (P = 0.05). In the patient-preference arm no characteristic was significantly different between those followed up and those not followed up.

Outcomes

Bivariate analysis of follow-up data indicated that average quality of life (MANSA score) was significantly higher in those randomised to the crisis house compared with those randomised to the ward (at an unadjusted level of 5%). No other bivariate analysis was significant at that level (Table 3).

Table 3 Outcome measures at follow-up categorised by treatment arm and analysis method

Randomised Patient preference Preference and randomised arms together Treatment received >50% of time
Ward Crisis house P Ward Crisis house P Ward Crisis house P Ward Crisis house P
GAF score
   n 14 13 14 29 28 42 33 37
   Mean (s.d.) 51 (17.6) 51 (16.4) 0.99 51 (13.4) 50 (10.3) 0.69 51 (15.3) 50 (12.3) 0.76 51 (15.2) 50 (11.9) 0.65
BPRS
   n 11 11 11 30 22 41 25 38
   Mean (s.d.) 46 (11.4) 38 (7.0) 0.07 38 (10.2) 44 (9.9) 0.13 42 (11.3) 42 (9.5) 0.95 41 (11.1) 43 (9.4) 0.38
MANSA total
   n 14 14 14 30 28 44 33 39
   Mean (s.d.) 3.7 (0.8) 4.4 (0.7) 0.05* 4.0 (1.0) 3.9 (0.9) 0.64 3.9 (0.9) 4.0 (0.9) 0.49 4.0 (0.8) 3.9 (0.9) 0.64
CAN unmet needs
   n 12 13 12 30 25 42 29 38
   Mean (s.d.) 6.4 (4.5) 4.1 (2.4) 0.12 4.6 (3.8) 6.0 (3.3) 0.23 5.5 (4.2) 5.4 (3.2) 0.95 5.1 (3.6) 5.8 (3.5) 0.43
MacArthur total
   n 11 11 12 29 23 40 27 36
   Mean (s.d.) 5.3 (5.3) 4.2 (3.8) 0.59 3.2 (2.9) 3.8 (3.3) 0.57 4.2 (4.3) 3.9 (3.4) 0.78 4.3 (4.1) 3.8 (3.4) 0.59
Ritsher total
   n 14 11 14 28 28 39 30 37
   Mean (s.d.) 68 (12.1) 69 (10.7) 0.79 69 (14.8) 70 (9.2) 0.75 68 (13.3) 70 (9.5) 0.64 68 (13.8) 70 (8.6) 0.55
VSSS total
   n 14 10 11 29 25 39 28 36
   Mean (s.d.) 3.2 (0.6) 3.4 (0.4) 0.63 3.2 (0.5) 3.4 (0.7) 0.60 3.2 (0.6) 3.4 (0.6) 0.45 3.2 (0.5) 3.4 (0.6) 0.27
EQ–5D
   n 14 12 14 30 28 42 31 39
   Mean (s.d.) 0.51 (0.40) 0.52 (0.44) 0.92 0.56 (0.30) 0.52 (0.30) 0.70 0.53 (0.35) 0.52 (0.34) 0.89 0.54 0.37 0.73
QALY
   n 14 12 14 30 28 42 31 39
   Mean (s.d.) 0.14 (0.12) 0.22 (0.19) 0.21 0.14 (0.07) 0.16 (0.10) 0.50 0.14 (0.10) 0.18 (0.13) 0.21 0.17 (0.14) 0.16 (0.10) 0.51

BPRS, Brief Psychiatric Rating Scale; CAN, Camberwell Assessment of Need; GAF, Global Assessment of Functioning; MANSA, Manchester Short Assessment of Quality of Life; VSSS, Verona Service Satisfaction Scale

Regression adjustments to the differences between arms were made to account for differences in demographic factors and severity of illness found at baseline between the ward and crisis house groups (see Table 2) as well as indicators of missing follow-up (see above). The results are shown in Table 4. Although coercion (MacArthur scale) and stigma (Ritsher scale) were significantly different at baseline between the treatment arms they are measures of the process of admission and were not included in the analysis of outcome. We therefore adjusted only for baseline GAF, MANSA, CAN, age, borough and the baseline score of each outcome measure. There was no significant difference in outcomes between those randomised to the crisis house compared with those randomised to the ward; between those declining randomisation and selecting the crisis house compared with those selecting the ward; or between those allocated (whether by randomisation or not) to the crisis house compared with the ward. However, there was a trend toward significance for BPRS score in favour of crisis houses (adjusted difference –6.47, 95% CI –13.18 to –0.23, P = 0.06). Satisfaction with treatment was also significantly higher in those who spent the majority of their admission in the crisis house (randomised or patient-preference arms) compared with those who spent the majority of time on the ward (adjusted difference 0.30, 95% CI 0.01 to 0.59, P = 0.04) (Table 4).

Table 4 Treatment effects after adjustment for baseline characteristics categorised by method of analysis a

n a Adjusted difference b , c 95% CI P
Randomised
   GAF 25 –3.54 –15.82 to 8.75 0.55
   BPRS 21 –6.47 –13.18 to 0.23 0.06
   MANSA 26 0.41 –0.19 to 1.00 0.17
   CAN 24 –0.93 –3.47 to 1.61 0.45
   MacArthur total 21 –1.11 –3.40 to 1.17 0.31
   Ritsher total 23 –1.33 –7.47 to 4.81 0.65
   VSSS total 23 –0.13 –0.47 to 0.22 0.44
   EQ – 5D 25 –1.06 –0.38 to 0.13 0.30
   QALY 25 0.75 –0.08 to 0.18 0.46
Patient preference
   GAF 37 –0.51 –10.54 to 9.51 0.92
   BPRS 36 4.22 –2.82 to 11.25 0.23
   MANSA 38 –0.10 –0.68 to 0.48 0.72
   CAN 36 0.88 –1.85 to 3.61 0.51
   MacArthur total 36 1.69 –1.69 to 5.08 0.31
   Ritsher total 34 –4.48 –11.70 to 2.74 0.21
   VSSS total 34 0.33 –0.25 to 0.91 0.26
   EQ – 5D 38 –1.68 –0.37 to 0.04 0.10
   QALY 38 0.04 –0.07 to 0.74 0.97
Preference and randomised arms together
   GAF 62 –1.01 –8.10 to 6.09 0.78
   BPRS 57 –1.21 –5.85 to 3.43 0.60
   MANSA 64 0.07 –0.33 to 0.47 0.73
   CAN 60 0.01 –1.64 to 1.65 0.99
   MacArthur total 57 0.18 –1.78 to 2.14 0.86
   Ritsher total 57 –2.20 –6.57 to 2.16 0.32
   VSSS total 57 0.20 –0.11 to 0.51 0.19
   EQ – 5D 63 –1.53 –0.26 to 0.03 0.13
   QALY 63 0.51 –0.05 to 0.08 0.62
Treatment received > 50% of time
   GAF 62 –1.44 –8.29 to 5.42 0.68
   BPRS 57 2.39 –2.03 to 6.81 0.28
   MANSA 64 0.05 –0.34 to 0.43 0.82
   CAN 60 –0.06 –1.65 to 1.53 0.94
   MacArthur total 57 0.31 –1.58 to 2.21 0.74
   Ritsher total 57 –2.66 –6.94 to 1.63 0.22
   VSSS total 57 0.30 0.01 to 0.59 0.04
   EQ – 5D 63 –0.98 –0.21 to 0.07 0.33
   QALY 63 –0.37 –0.07 to 0.05 0.71

BPRS, Brief Psychiatric Rating Scale; CAN, Camberwell Assessment of Need; GAF, Global Assessment of Functioning; MANSA, Manchester Short Assessment of Quality of Life; QALY, quality-adjusted life-year; VSSS, Verona Service Satisfaction Scale

a. Numbers do not always reflect those in Fig. 1 owing to missing data

b. Ward, 0; crisis house, 1

c. Adjusted for baseline GAF, MANSA, unmet needs, age, borough and baseline measure of the outcome analysed

Analysis of satisfaction subscales indicated that satisfaction with information, access and efficacy was significantly higher in those who spent most of their time in the crisis house compared with the ward (Table 5).

Table 5 Treatment effects on satisfaction domain after adjustment for baseline characteristics, categorised by location of majority of treatment

VSSS domain n Adjusted difference a , b 95% CI P
Total score 57 0.30 0.01 to 0.59 0.04
Overall satisfaction 56 0.53 –0.01 to 1.08 0.06
Professional skills behaviour 56 0.30 –0.09 to 0.69 0.13
Information 54 0.69 0.22 to 1.16 0.01
Access 56 0.68 0.15 to 1.21 0.01
Efficacy 56 0.57 0.12 to 1.01 0.01
Types of intervention 57 0.14 –0.17 to 0.45 0.37
Relatives 34 0.43 –0.21 to 1.07 0.18

VSSS, Verona Service Satisfaction Scale

a. Ward, 0; crisis house, 1

b. Adjusted for baseline Global Assessment of Functioning, Manchester Short Assessment of Quality of Life, unmet needs, age, borough and baseline measure of the outcome analysed

Impact of preferences

Only three participants had no preference for treatment in the ward or the crisis house before treatment. Since so few indicated no preference, comparisons could not be made with this group and they were removed from the preference analysis. Fourteen participants did not respond to the preference question and two had no information on the length of time that they spent in treatment on the ward or crisis house, and their data were also excluded from the analysis.

A total of 83 participants in either the randomised or preference group indicated a preference for either the ward or the crisis house before treatment was given, of whom 67 (81%) preferred the crisis house. Fifty-eight per cent (21 of 36) of those receiving most of their treatment on the ward had stated a preference for the crisis house, and 1 person out of 47 (2%) who spent most of her stay in a crisis house had stated a preference for treatment on the ward. Almost three-quarters (73%, n = 61) of participants received their preferred treatment.

Bivariate analysis of outcomes at follow-up indicated that participants who obtained their preferred treatment had significantly higher levels of overall satisfaction, whether we examined data only for randomised patients (P = 0.03) or for all patients (P = 0.05). There was weak evidence of lower levels of perceived coercion for all patients who received their preferred service (P = 0.09), but there was no other difference in outcomes between those who did and those who did not receive their preferred treatment. After adjustment for baseline functioning, unmet needs, quality of life and demographic factors there was no significant difference in outcome when examining the randomised groups only; however, when examining data for the whole group, receiving the preferred treatment was associated with significantly higher levels of total satisfaction (adjusted preference coefficient 0.37, 95% CI 0.05–0.69, P = 0.03).

Economic evaluation

Use of services is reported in online Table DS1. The sample as a whole made the greatest use of hospital services, particularly in-patient stays, with an average length of stay of 33 days for the index admission and 8 days for all subsequent admissions. There was little difference between the randomised groups in terms of the total number of days spent in hospital (mean 36 days for those allocated to the ward and 34 for those allocated to crisis house). Slightly larger differences were observed between the patient-preference arms (mean 44 days and 55 days respectively) and total in-patient days were longer on average for the patient-preference group compared with the randomised group. A wide range of community services were accessed by participants, but the average number of contacts per participant was low. Use of supported accommodation, criminal justice contacts and productivity losses were low in all groups.

Cost results are reported in Table 6. Baseline costs tended to be higher in the crisis house groups, and were significantly so in the combined preference and randomised groups. All subsequent economic analyses were adjusted for baseline costs to compensate for this imbalance. Total follow-up costs were dominated by the cost of in-patient stays, which constituted 85% of total costs on average. Total follow-up costs per participant did not differ significantly in any of the groups compared.

Table 6 Costs for the 12 weeks prior to study entry and over the 12-week follow-up period, categorised by method of analysis

Costs, UK£: mean (s.d.)
Randomised Patient preference Preference and randomised arms Treatment received >50% of the time
Ward (n = 14) Crisis house (n = 12) P Ward (n = 14) Crisis house (n = 30) P Ward (n = 28) Crisis house (n = 42) P Ward (n = 31) Crisis house (n = 39) P
Baseline costs
Total baseline costs 2018 (3453) 5208 (6748) 0.13 1151 (1298) 5047 (7747) 0.07 1584 (2598) 5093 (7394) 0.02 2982 (5333) 4252 (6785) 0.40
Follow-up costs
Hospital services 9623 (12 692) 8944 (6134) 12 363 (12 966) 13 675 (20 874) 10 993 (12 667) 12 323 (17 972) 14 792 (21 414) 9406 (9384)
   Index admission 7286 (5756) 7939 (5208) 10 324 (12 326) 7088 (11 153) 8805 (9565) 7331 (9768) 11 057 (13 465) 5428 (3375)
   Subsequent admissions 2212 (7153) 713 (2290) 1820 (2651) 5782 (12 713) 2016 (5297) 4334 (11 005) 3475 (9398) 3352 (9123)
   Out-patient/day patient 25 (96) 252 (873) 60 (202) 758 (2272) 43 (156) 613 (1977) 133 (557) 585 (2010)
   A&E 99 (125) 40 (86) 159 (464) 48 (114) 129 (335) 45 (106) 126 (321) 41 (102)
Community services 574 (652) 1216 (1372) 336 (323) 668 (599) 455 (519) 824 (907) 719 (1034) 643 (541)
   Medication 191 (228) 354 (421) 310 (505) 296 (583) 250 (398) 313 (537) 322 (440) 260 (516)
   Supported accommodation 905 (1948) 0 (0) 95 (356) 0 (0) 500 (1435) 0 (0) 346 (1268) 84 (521)
   Criminal justice 40 (80) 21 (42) 20 (60) 7 (26) 30 (71) 11 (32) 31 (68) 9 (29)
   Productivity losses 39 (146) 20 (67) 0 (0) 646 (2018) 20 (103) 467 (1721) 25 (105) 497 (1783)
Total follow-up costs a 11 371 (12 541) 10 555 (6570) 0.90 13 123 (13 032) 15 293 (21 137) 0.22 12 247 (12 582) 13 939 (18 229) 0.37 16 235 (21 269) 10 900 (10 109) 0.11

A&E, accident and emergency

a. Adjusted for baseline Global Assessment of Functioning, Manchester Short Assessment of Quality of Life, unmet needs, age, borough and costs

Although there was no statistically significant difference in either costs or outcomes, exploration of cost-effectiveness and the uncertainty surrounding the cost and effect estimates suggests there may be a greater probability of women's crisis houses being more cost-effective than traditional psychiatric wards. In the randomised group there is little to choose between the two arms, with the probability of crisis houses being more cost-effective ranging from 47% to 58% depending on willingness to pay for gains in QALYs (online Fig. DS1). In the patient-preference group there is almost a 90% probability of crisis houses being the more cost-effective option, irrespective of willingness to pay (online Fig. DS2), with similar findings for the combined preference and randomised group and the treatment received group.

Discussion

We found no significant difference in outcomes for women admitted to women's crisis houses or traditional psychiatric wards other than higher levels of satisfaction with services for those admitted to the crisis houses, and no significant difference in costs. This pilot study does not have the statistical power to establish whether admissions to crisis houses are as effective or less effective than traditional wards, but the results from this pilot can be used to inform future hypotheses on the effectiveness and cost-effectiveness of women's crisis houses, and provide some evidence on whether a larger sample is likely to yield differences. The confidence intervals for BPRS score in the randomised comparison indicate that the symptoms are not likely to be much worse on follow-up in crisis houses compared with wards and may be much better. This suggests that women's crisis houses may be as effective as traditional psychiatric in-patient ward admissions in treating women presenting with acute psychiatric problems and do not have an adverse impact on resource use. Exploration of cost-effectiveness suggests there is an equal or greater probability of women's crisis houses being cost-effective compared with traditional psychiatric wards. These findings counter those found in other research into alternative services, where outcomes were better for standard services, Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn31 but these differences may reflect differences in measurement – our results were derived from patient-rated measures, including patient satisfaction, and not staff-rated measures of functioning. The difference in time of follow-up might also be important in accounting for this admission: in our study follow-up assessment took place after 3 months rather than at the time of discharge as in the Alternatives Study. Reference Slade, Byford, Barrett, Lloyd-Evans, Gilburt and Osborn31 A previous larger study of 245 consecutive informal admissions to women's crisis houses and psychiatric wards found that women's crisis houses have a similar case-mix when compared with admissions to catchment area wards, Reference Howard, Rigon, Cole, Lawlor and Johnson32 although an investigation of six residential alternatives to hospital suggested that people admitted to community alternatives are less likely to be assessed as a risk to others, less likely to be behaviourally disturbed, more likely to have referred themselves for help and more likely to be already known to services. Reference Johnson, Lloyd-Evans, Morant, Gilburt, Shepherd and Slade33 These studies therefore suggest that crisis houses are a viable alternative to traditional psychiatric wards for voluntary female admissions where intensive observation is not required.

We also found that receiving preferred treatment was associated with significantly higher levels of total satisfaction in multivariate analyses, although no difference in adjusted outcome was found when examining the randomised groups only. Recent systematic reviews that have examined the relationship between patient preferences and the outcome of interventions have concluded that there is evidence to suggest that patient preferences have a significant, though small, effect on outcome, Reference King, Nazareth, Lampe, Bower, Chandler and Morou7,34 and our failure to detect other differences in outcome may reflect our small sample size. Previous qualitative research has found that women often prefer women's crisis houses to conventional in-patient treatment, as women find the former less stigmatising, less frightening and a better environment in which to discuss their problems, Reference Johnson, Bingham, Billings, Pilling, Morant and Bebbington3 a finding confirmed regarding alternatives in general. Reference Gilburt, Slade, Rose, Lloyd-Evans, Johnson and Osborn35 The offer of admission to a women's crisis house where this is available is therefore likely to lead to greater satisfaction with services; this may be particularly important for women who may have adverse service experiences, such as women from Black and minority ethnic (BME) groups.

Limitations

This was an exploratory study with small numbers of patients in each treatment arm. Our failure to find any significant difference in symptoms, perceived coercion and stigma, unmet needs, level of functioning or costs for women admitted to the different services may reflect the relatively small sample size in this pilot study, and a larger study of outcomes of women in these two groups is needed to investigate such outcomes further. A further difficulty was in facilitating the receipt of the allocated or preferred arm; not all patients received the allocated or preferred service immediately owing to limits by the clinical demands of the services and 18% were never transferred to their allocated or preferred service. However, this reflects clinical practice in which services are not always immediately available even if desirable, and women may have to wait before they are transferred to the more appropriate service or may find that they are ready for discharge before transfer is possible. Where significant findings have been reported, the problems of multiple testing should be noted. No adjustment has been made for this and the results should therefore be regarded as hypothesis-generating only.

Methodological considerations

Recruiting potential participants into a trial investigating services for acutely ill patients is challenging. We tried to overcome this using a number of strategies, including provisions under the Mental Capacity Act, recruitment at different sites on the pathway to admission and use of a brief participant information sheet at the point of recruitment when patients were in crisis, which was supplemented by a longer information sheet a few days later. The success or otherwise of these strategies has been discussed in more detail elsewhere, Reference Howard, Leese, Byford, Killaspy, Cole and Lawlor9 but we found recruitment in the community just before admission and of women without capacity particularly challenging. It is clear that the research community needs to find innovative ways of carrying out evaluations of acute services so that evidence is available on their effectiveness.

Implications

Women's crisis houses have been criticised for being expensive and for admitting less severely ill women than psychiatric wards, but this study suggests that they can admit women who would otherwise be admitted to psychiatric wards, are as effective in improving symptoms and functioning as traditional settings, while providing a service that women prefer and feel more satisfied with, and moreover at no greater expense. This study, together with other qualitative and quantitative research reported in this supplement, Reference Johnson, Lloyd-Evans, Howard, Osborn and Slade36 suggests that health service providers should consider giving women the choice of admission to women's crisis houses.

Footnotes

The study was funded by the Medical Research Council (grant G0401241) and was adopted by the National Institute for Health Research Mental Health Research Network.

Declaration of interest

None.

References

1 Johnson, S, Gilburt, H, Lloyd-Evans, B, Osborn, DPJ, Boardman, J, Leese, M, et al. In-patient and residential alternatives to standard acute psychiatric wards in England. Br J Psychiatry 2009; 194: 456–63.CrossRefGoogle ScholarPubMed
2 Department of Health. Women-Only and Women-Sensitive Mental Health Services. Department of Health, 2003.Google Scholar
3 Johnson, S, Bingham, C, Billings, J, Pilling, S, Morant, N, Bebbington, P, et al. Women's experiences of admission to a crisis house and to acute hospital wards: a qualitative study. J Ment Health 2004; 13: 247–62.CrossRefGoogle Scholar
4 Killaspy, H, Dalton, J, McNicholas, S, Johnson, S. Drayton Park, an alternative to hospital admission for women in acute mental health crisis. Psychiatr Bull 2000; 24: 101–4.CrossRefGoogle Scholar
5 Meiser-Stedman, C, Howard, L, Cutting, P. Evaluating the effectiveness of a women's crisis house: a prospective observational study. Psychiatr Bull 2006; 30: 324–6.CrossRefGoogle Scholar
6 Torgerson, DJ, Sibbald, B. Understanding controlled trials. What is a patient preference trial? BMJ 1998; 31: 360.CrossRefGoogle Scholar
7 King, M, Nazareth, I, Lampe, F, Bower, P, Chandler, M, Morou, M, et al. Impact of participant and physician intervention preferences on randomized trials: a systematic review. JAMA 2005; 293: 1089–99.CrossRefGoogle ScholarPubMed
8 Howard, LM, Thornicroft, G. Patient preference randomised controlled trials in mental health research. Br J Psychiatry 2006; 188: 303–4.CrossRefGoogle ScholarPubMed
9 Howard, LM, Leese, M, Byford, S, Killaspy, H, Cole, L, Lawlor, C, et al. Methodological challenges in evaluating the effectiveness of women's crisis houses compared with psychiatric wards; findings from a pilot patient preference RCT. J Nerv Ment Dis 2009; 197: 722–7.CrossRefGoogle ScholarPubMed
10 Endicott, J, Spitzer, RL, Fleiss, JL, Cohen, J. The Global Assessment Scale. A procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry 1976; 33: 766–71.CrossRefGoogle Scholar
11 Overall, JE, Gorham, DR. BPRS. Brief Psychiatric Rating Scale. In ECDEU Assessment Manual for Psychopharmacology. US Department of Health, Education and Welfare, 1976.Google Scholar
12 Priebe, S, Huxley, P, Knight, S, Evans, S. Application and results of the Manchester Short Assessment of QoL (MANSA). Int J Soc Psychiatry 1999; 45: 712.CrossRefGoogle Scholar
13 Brooks, R, EuroQol Group. EQ–5D: the current state of play. Health Policy 1996; 37: 3–2.CrossRefGoogle Scholar
14 Phelan, M, Slade, M, Thornicroft, G, Dunn, G, Holloway, F, Wykes, T, et al. The Camberwell Assessment of Need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. Br J Psychiatry 1995; 167: 589–95.CrossRefGoogle ScholarPubMed
15 Ruggeri, M, Dall'Agnola, R. The development and use of the Verona Expectations for Care Scale (VECS) and the Verona Service Satisfaction Scale (VSSS) for measuring expectations and satisfaction with community-based psychiatric services in patients, relatives and professionals. Psychol Med 1993; 23: 511–23.CrossRefGoogle ScholarPubMed
16 Gardner, W, Hoge, S, Bennett, N, Roth, L, Lidz, C, Monahan, J, et al. Two scales for measuring patients' performance perceptions of coercion during hospital admission. Behav Sci Law 1993; 20: 307–21.Google Scholar
17 Ritsher, JB, Otilingam, PG, Grajales, M. Internalized stigma of mental illness: psychometric properties of a new measure. Psychiatry Res 2003; 121: 3149.CrossRefGoogle ScholarPubMed
18 Byford, S, Fiander, M, Torgerson, DJ, Barber, JA, Thompson, SG, Burns, T, et al. Cost-effectiveness of intensive v. standard case management for severe psychotic illness: UK700 case management trial. Br J Psychiatry 2000; 176:537–43.Google Scholar
19 Barrett, B, Byford, S, Crawford, M, Patton, R, Drummond, C, Henry, J, et al. Cost effectiveness of referral to an alcohol health worker in patients attending an accident and emergency department: a decision-making approach. Drug Alcohol Depend 2006; 81: 4754.CrossRefGoogle Scholar
20 Kuyken, W, Byford, S, Taylor, RS, Watkins, E, Holden, E, White, K, et al. Mindfulness-based cognitive therapy to prevent relapse in recurrent depression. J Consult Clin Psychol 2008; 76: 966–78.CrossRefGoogle ScholarPubMed
21 Department of Health. National Health Service Reference Costs. Department of Health, 2007.Google Scholar
22 Finn, W, Hyslop, J, Truman, C. Mental Health, Multiple Needs and the Police. Revolving Doors Agency, 2000.Google Scholar
23 Legal Services Commission. Civil Remuneration Rates. Legal Services Commission, 2004.Google Scholar
24 Curtis, L. Unit Costs of Health and Social Care. Personal Social Services Research Unit, 2006.Google Scholar
25 HM Prison Service. Prison Service Annual Report and Accounts. TSO (The Stationery Office), 2007.Google Scholar
26 British Medical Association, Royal Pharmaceutical Society of Great Britain. British National Formulary. BMJ Books/Pharmaceutical Press, 2007.Google Scholar
27 Koopmanschap, MA, Rutten, FFH. A practical guide for calculating indirect costs of disease. Pharmacoeconomics 1996; 10: 460–6.CrossRefGoogle ScholarPubMed
28 Efron, B, Tibshirani, RJ. An Introduction to the Bootstrap. Chapman & Hall, 1993.CrossRefGoogle Scholar
29 Thompson, SG, Barber, JA. How should cost data in randomised controlled trials be analysed? BMJ 2000; 320: 1197–200.CrossRefGoogle ScholarPubMed
30 Fenwick, E, Claxton, K, Sculpher, M. Representing uncertainty: the role of cost-effectiveness acceptability curves. Health Econ 2001; 10: 779–87.CrossRefGoogle ScholarPubMed
31 Slade, M, Byford, S, Barrett, B, Lloyd-Evans, B, Gilburt, H, Osborn, DPJ, et al. Alternatives to standard acute in-patient care in England: short-term clinical outcomes and cost-effectiveness. Br J Psychiatry 2010 (suppl 53): s149.CrossRefGoogle ScholarPubMed
32 Howard, LM, Rigon, E, Cole, L, Lawlor, C, Johnson, S. Admissions to women's crisis houses or to psychiatric wards: women's pathways to admission. Psychiatr Serv 2008; 59: 1443–9.CrossRefGoogle ScholarPubMed
33 Johnson, S, Lloyd-Evans, B, Morant, N, Gilburt, H, Shepherd, G, Slade, M, et al. Alternatives to standard acute in-patient care in England: roles and populations served. Br J Psychiatry 2010 (suppl 53): s613.CrossRefGoogle ScholarPubMed
34 Preference Collaborative Review Group. Patients' preferences within randomised trials: systematic review and patient level meta-analysis. BMJ 2008; 337: 1864.CrossRefGoogle Scholar
35 Gilburt, H, Slade, M, Rose, D, Lloyd-Evans, B, Johnson, S, Osborn, DPJ. Service users' experiences of residential alternatives to standard acute wards: qualitative study of similarities and differences. Br J Psychiatry 2010 (suppl 53): s2631.CrossRefGoogle ScholarPubMed
36 Johnson, S, Lloyd-Evans, B, Howard, L, Osborn, DPJ, Slade, M. Where next with residential alternatives to admission? Br J Psychiatry 2010 (suppl 53): s524.CrossRefGoogle Scholar
Figure 0

Table 1 Sociodemographic and clinical characteristics of women agreeing to and declining randomisation

Figure 1

Fig. 1 Study profile.

Figure 2

Table 2 Comparison of baseline characteristics of women given ward and crisis house treatments categorised by method of analysis

Figure 3

Table 3 Outcome measures at follow-up categorised by treatment arm and analysis method

Figure 4

Table 4 Treatment effects after adjustment for baseline characteristics categorised by method of analysis a

Figure 5

Table 5 Treatment effects on satisfaction domain after adjustment for baseline characteristics, categorised by location of majority of treatment

Figure 6

Table 6 Costs for the 12 weeks prior to study entry and over the 12-week follow-up period, categorised by method of analysis

Supplementary material: PDF

Howard et al.

Data Supplement

Download Howard et al.(PDF)
PDF 62.5 KB
Submit a response

eLetters

No eLetters have been published for this article.