Routine outcome measurement has been increasingly advocated for mental health and psychological services, with the Department of Health committed to the implementation of outcome measures in routine practice in mental health services (National Institute for Mental Health in England, 2002, 2004). This momentum has been evidenced by a report (Reference Fonagy, Matthews and PillingFonagy et al, 2004) building on the work of an expert advisory group on mental health outcomes, which concludes that the Health of the Nation Outcome Scale (HoNOS; Reference Wing, Beevor and CurtisWing et al, 1998) ‘could provide a developmental anchor setting the minimum domains to be covered for the relevant population’ (Reference Fonagy, Matthews and PillingFonagy et al, 2004: p. 6). The report also states that ‘Trust[s] should also explore the use of other clinician rated and self report measures, including quality of life measures, so as to ensure that relevant domains can be appropriately measured across the populations served’ (Reference Fonagy, Matthews and PillingFonagy et al, 2004: p. 6).

These statements reflect a strategy whereby the appropriateness and utility of bonafide outcome measures should be investigated but that they need to be empirically anchored against HoNOS. Although no single measure could possibly capture the whole range of presenting problems and outcomes across mental health, establishing evidence of the relationship between measures and the extent to which they capture common and unique aspects of experience is important. This would add to validation evidence for the measures and clarify the extent to which they are useful for particular populations and in particular service settings. Accordingly, we focused on the HoNOS and a widely used self-report measure, the Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM; Barkham et al, Reference Barkham, Margison and Leach2001, Reference Barkham, Gilbert and Connell2005; Reference Evans, Connell and BarkhamEvans et al, 2002). Interestingly, each has been advocated for use at different ends of the severity range. Both measures can be used to assess various aspects of risk of self-harm or harm to others.

The HoNOS is advocated as suitable for use at the severe end of the spectrum to provide a practitioner-completed assessment across 12 domains, divided into four subgroups. Risk to self and others is assessed by two items in subgroup A (behavioural problems). This measure has derived support from some reviewers (e.g. Reference Rees, Richards and ShapiroRees et al, 2004) and been criticised by others (e.g. Reference Audin, Margison and Mellor ClarkAudin et al, 2001). The empirical relationship between HoNOS and other outcome measures has been investigated and significant associations have been reported. Orrell et al (Reference Orrell, Yard and Handysides1999) established significant correlations between HoNOS and a battery of six outcome measures on a sample of 100 patients: 0.59 for the Social Behaviour Scale (Reference Wykes and SturtWykes & Sturt, 1986), 0.51 for the Location of Community Support Scale (Reference Kazarian and JosephKazarian & Joseph, 1994), 0.40 for the Brief Psychiatric Rating Scale (BPRS; Reference Overall and GorhamOverall & Gorham, 1962), – 0.40 for the Global Assessment Scale (GAS; Reference Endicott, Spitzer and FleissEndicott et al, 1976), 0.36 for the General Health Questionnaire (Reference GoldbergGoldberg, 1978) and – 0.33 for the Medical Outcomes Study Short Form-36 (Reference Ware and SherbourneWare & Sherbourne, 1990). McClelland et al (Reference Mcclelland, Trimble and Fox1998) reported correlations on presentation to the service between HoNOS and GAS of 0.49 and with the BPRS of 0.44.

The CORE-OM is a 34-item self-report measure tapping four domains, including a risk sub-scale assessing risk to others (two items) and risk to self (four items). It has been widely used to measure emotional disturbance in service settings delivering psychological interventions in primary and secondary care (Reference Barkham, Gilbert and ConnellBarkham et al, 2005). Its relationship to other measures, including the Beck Depression Inventory (Reference Beck and SteerBeck & Steer, 1993), has been explored in large-scale studies (e.g. Reference Leach, Lucock and BarkhamLeach et al, 2005).

Our study assesses the empirical relationship between these measures and their potential to complement each other across a range of settings.

Method

Data were collected as part of routine service delivery for clients referred to the Primary Care Liaison, Assessment Treatment and Training (PLATT) team of a large mental health National Health Service (NHS) trust. This multidisciplinary team provides a service within two primary care trusts and serves a population of 330 000. The PLATT team receives 60 adult referrals a week, mainly from general practice, and provides an assessment filter for all mental health services in one locality. All those referred routinely complete the CORE-OM at assessment. For those receiving therapy within the PLATT service, the CORE-OM is also used before therapy and at discharge. For the first 6 months of the service, assessors completed HoNOS ratings of people assessed as requiring help from community mental health or psychiatry services.

Data for this study come from a service evaluation database for 1497 persons referred between October 2001 and March 2003. Of these, there were complete data for 1297 clients on CORE-OM and for 507 clients on HoNOS, with 315 clients having complete data for both measures. For some clients, CORE-OM and HoNOS were completed at different stages of therapy, so the study focused on 297 clients (175 women and 122 men) for whom the measures had been completed at assessment and within 30 days of each other. One client with an extreme outlying HoNOS total score of 37 was excluded from further analysis, leaving 296 clients in the study sample, whose HoNOS scores ranged from 0 to 25 (mean=9.03, s.d.=4.27, median=9.00). The CORE-OM scores ranged from 0.15 to 3.76 (mean=2.24, s.d.=0.69, median=2.28), with 274 clients (93%) scoring above the CORE-OM clinical cut-offs of 1.29 (female) and 1.19 (male), and 109 (37%) scoring above the ‘severe’ cut-off of 2.50 (see Reference Barkham, Margison and LeachBarkham et al, 2001). The excluded client scored 2.93 on the CORE-OM.

Those included in the study sample (n=296) differed from those not included (n=1000) in CORE-OM score (mean=2.24, s.d.=0.69, for sample; mean=2.08, s.d.=0.76, for non-sample; t=3.28, d.f.=1294, P=0.001), but not in age (mean=37.5, s.d.=12.4, for sample; mean=36.9, s.d.=11.8, for non-sample; t=0.71, d.f.=1294, P=0.48) or gender (60% females in sample, 62% in non-sample; χ ²=0.86, d.f.=1, P=0.36). The difference in CORE-OM mean scores reflects the fact that HoNOS was completed only for individuals assessed as suitable for referral to psychiatric or community mental health team services.

The relation between the two measures was explored using Pearson product-moment correlations to compare total and sub-scale scores. At the item level, hierarchical cluster analysis (Ward's method) and maximum likelihood factor analysis (with oblique rotation) were both carried out, all analyses being performed using the Statistical Package for the Social Sciences, version 12.0.

Results

The correlation between CORE-OM overall score and HoNOS total score was 0.50 (P<0.001), the scales thus sharing 50% of the variance (Reference OzerOzer, 1985), with CORE-OM correlating most highly with the HoNOS sub-scales A (behavioural problems; r=0.40) and D (social problems; r=0.39), with a slightly smaller correlation with the C sub-scale (symptomatic; r=0.34) and an effectively zero correlation with the B sub-scale (impairment; r=0.05). The same pattern is apparent for the CORE-OM risk sub-scale, which had correlations of 0.51 with the HoNOS total score, 0.57 with the A sub-scale, 0.39 with the D sub-scale, 0.22 with the C sub-scale and 0.02 with the B sub-scale.

Table 1 summarises the results of the hierarchical cluster analysis and factor analysis of the CORE-OM and HoNOS items. The cluster analysis shows two main clusters (1 and 2), each broken down into two further clusters (a and b). Cluster 1 features all the non-risk CORE-OM items and three of the HoNOS items (‘other mental and behavioural problems’, ‘problems with relationships’ and ‘problems with depressed mood’), whereas cluster 2 brings together the CORE-OM risk items and the remaining HoNOS items. Cluster 1a includes items mainly from the CORE-OM Problems sub-scale together with the three HoNOS items; cluster 1b includes items mainly from the CORE-OM Wellbeing and Functioning sub-scales. Cluster 2a includes the CORE-OM risk items and two HoNOS risk items (‘non-accidental self-injury’ and ‘overactive, aggressive, disruptive or agitated behaviour’), with the self-injury HoNOS item being closely related to the four CORE-OM ‘ risk to self’ items, and the aggression HoNOS item being closely related to the two CORE-OM ‘risk to others’ items. Cluster 2b contains the remaining HoNOS items.

Table 1. Results of hierarchical cluster analysis and maximum likelihood factor analysis for CORE-OM and HoNOS items

Questionnaire	Item	Sub-scale	Content	HCA (Ward's method) cluster	MLFA factor (oblique) ≥ 0.25
CORE—OM	C13	Problems	I have been disturbed by unwanted thoughts and feelings	1a	1
CORE—OM	C11	Problems	Tension and anxiety have prevented me doing important things	1a	1
CORE—OM	C28	Problems	Unwanted images or memories have been distressing me	1a	1
CORE—OM	C15	Problems	I have felt panic or terror	1a	1
CORE—OM	C02	Problems	I have felt tense, anxious or nervous	1a	1
CORE—OM	C08	Problems	I have been troubled by aches, pains or other physical problems	1a	1
CORE—OM	C18	Problems	I have had difficulty getting to sleep or staying asleep	1a	1
HoNOS	HC8	C	Other mental and behavioural problems	1a	1
CORE—OM	C05	Problems	I have felt totally lacking in energy and enthusiasm	1a	3
CORE—OM	C21	Functioning	I have been able to do most things I needed to do	1a	3
HoNOS	HD9	D	Problems with relationships	1a	4
HoNOS	HC7	C	Problems with depressed mood	1a	—
CORE—OM	C33	Functioning	I have felt humiliated or shamed by other people	1b	1
CORE—OM	C29	Functioning	I have been irritable when with other people	1b	1
CORE—OM	C17	Wellbeing	I have felt overwhelmed by my problems	1b	1
CORE—OM	C14	Wellbeing	I have felt like crying	1b	1
CORE—OM	C20	Problems	My problems have been impossible to put to one side	1b	1
CORE—OM	C25	Functioning	I have felt criticised by other people	1b	1
CORE—OM	C30	Problems	I have thought I am to blame for my problems and difficulties	1b	1
CORE—OM	C01	Functioning	I have felt terribly alone and isolated	1b	2
CORE—OM	C23	Problems	I have felt despairing or hopeless	1b	3
CORE—OM	C27	Problems	I have felt unhappy	1b	3
CORE—OM	C07	Functioning	I have felt able to cope when things go wrong	1b	3
CORE—OM	C26	Functioning	I have thought I have no friends	1b	3
CORE—OM	C12	Functioning	I have been happy with the things I have done	1b	3
CORE—OM	C32	Functioning	I have achieved the things I wanted to	1b	3
CORE—OM	C04	Wellbeing	I have felt OK about myself	1b	3
CORE—OM	C31	Wellbeing	I have felt optimistic about my future	1b	3
CORE—OM	C03	Functioning	I have felt I have someone to turn to for support when needed	1b	3
CORE—OM	C19	Functioning	I have felt warmth or affection for someone	1b	3
CORE—OM	C10	Functioning	Talking to people has felt too much for me	1b	3
CORE—OM	RS09	Risk to self	I have thought of hurting myself	2a	2
HoNOS	HA2	A	Non-accidental self-injury	2a	2
CORE—OM	RS16	Risk to self	I made plans to end my life	2a	2
CORE—OM	RS24	Risk to self	I have thought it would be better if I were dead	2a	2
CORE—OM	RS34	Risk to self	I have hurt myself physically or taken dangerous risks with my health	2a	4
HoNOS	HA1	A	Overactive, aggressive, disruptive or agitated behaviour	2a	4
CORE—OM	RO22	Risk to others	I have threatened or intimidated another person	2a	4
CORE—OM	RO06	Risk to others	I have been physically violent to others	2a	4
HoNOS	HA3	A	Problem drinking or drug-taking	2b	4
HoNOS	HD11	D	Problems with living conditions	2b	4
HoNOS	HC6	C	Problems with hallucinations and delusions	2b	4
HoNOS	HB4	B	Cognitive problems	2b	4
HoNOS	HD12	D	Problems with occupation and activities	2b	4
HoNOS	HD10	D	Problems with activities of daily living	2b	4
HoNOS	HB5	B	Physical illness or disability problems	2b	—

The results of the factor analysis echo those of the cluster analysis, with some minor differences. A HoNOS item (‘problems with relationships’) grouped with the CORE-OM non-risk items by the cluster analysis is now grouped closer to the other HoNOS items. The CORE-OM risk to self item 34 (‘I have hurt myself physically…’) is closer to the risk to others items in the factor analysis than to the remaining risk to self items.

Discussion

The findings that the HoNOS and CORE-OM total scores share 50% of the variance and that their respective risk items (CORE-OM risk sub-scale and HoNOS behavioural problems sub-scale) have 57% common variance suggest that both these widely used measures are identifying a common component of clients presenting with severe problems. Interestingly, the risk scale of the CORE-OM, comprising only six items, yielded as good if not better associations with the overall HoNOS and its behavioural scale than the CORE-OM total. Hence, the CORE-OM risk scale might prove a cost-efficient means of quickly obtaining clinically meaningful data.

The cluster 1a grouping of three HoNOS items, ‘other mental and behavioural problems’, ‘problems with relationships’ and ‘ problems with depressed mood’, with nine of the CORE-OM non-risk items reinforces the finding that these three HoNOS items are those that have shown evidence of being sensitive to change following psychological therapies (Reference Audin, Margison and Mellor ClarkAudin et al, 2001).

The HoNOS is the recommended measure for clients with severe and enduring problems (Reference Fonagy, Matthews and PillingFonagy et al, 2004). In circumstances where a self-report measure is appropriate, for example where staff trained in the use of HoNOS are not available, CORE-OM can provide information relating to some of the key domains, particularly risk.

Declaration of interest

M. B. is a member of the Board of Trustees of the CORE System Trust.