As its name implies, the remit for the Health of the Nation Outcome Scales (HoNOS) was to construct an instrument brief enough to be found useful by busy clinicians (chiefly nurses and psychiatrists in everyday National Health Service (NHS) practice) but also robust enough to provide, when aggregated and anonymised and used within the framework of a minimum data set (MDS), an index of progress for local and national public health purposes. The College Research Unit (CRU) of the Royal College of Psychiatrists was asked to provide only the instrument. Separate arrangements were made to construct and test the MDS.

In taking on their part of the remit, the CRU set out a basic strategy. The most important aim was to provide an instrument that most keyworkers would recognise as relevant for their own clinical records. As well as being brief (starting with 20 and ending with 12 items), it should be sensitive to change or the lack of it and have reasonable reliability and relationship to more established scales. The caveats were equally important. Incorporation of aggregated HoNOS information into sector or district registers for clinical or epidemiological purposes, or to measure progress towards a specified target, requires quality assurance. The areas to be covered include confidentiality and security of clinical data, training and ongoing supervision to maintain standards. This is true of all information collected for public health purposes, most of which requires instruments far larger than HoNOS. With these cautions, and after four stages of assessment, HoNOS-4, was recommended for more general testing, both clinical and in the setting of a draft MDS. Apart from one-day training courses for local supervisors, the responsibility of the CRU ceased at this point.

Further work has been uncoordinated and supervision haphazard. This has meant that publications so far have been of variable quality and slow to accumulate, although the pace is now accelerating. We therefore read with interest the papers on the use of HoNOS-3 and HoNOS-4 published in the May 1999 edition of the Journal. We comment briefly on their content before considering the future more broadly. Other members of the ‘HoNOS family’ are not considered since, at present, insufficient data are available to allow an informed assessment.

Comment on the HoNOS papers

Australia has been well ahead of the UK in testing HoNOS and the study by Trauer et al (Reference Trauer, Callaly and Hantz1999), the largest and most detailed so far, provides very useful experience. Three major points stand out for consideration in future modifications. One is that, as would be expected, ‘routine training’ (in this case a rather hurried question-and-answer session across a ‘dodgy’ trans-world video link) needs to be well designed and followed by supervision if results are to be used for public health purposes, as is likely to happen in Victoria. The second point for note is the poor performance of the two items on problems with ‘living conditions’ and ‘activities’. Both require information that raters frequently do not have. The third point is that the outcomes recorded on consecutive sets of HoNOS ratings might not match clinical expectations; a very important contingency. But don't shoot the messenger!

The project by Amin et al (Reference Amin, Singh and Croudace1999) used the final interviews of an outcome study of first-episode psychoses to rate patients on an early version of the 10th edition of the Present State Examination (PSE-10; World Health Organization, 1992) and HoNOS. Many patients were symptom-free at follow-up. A trained researcher obtained good correlations between HoNOS and PSE-10 totals (n=129), while the equivalent for keyworkers given only sketchy training was low but acceptable (n=42). The correlations between researcher and keyworkers (n=46) on HoNOS totals and subtotals varied across a similar range. As was reported in the field trials, social workers scored more highly than clinicians, indicating a possible calibration problem.

The results of the study (n=100) by Orrell et al (Reference Orrell, Yard and Handysides1999) were not dissimilar to those of Amin et al (Reference Amin, Singh and Croudace1999), with reasonably good validity and acceptance but poor reliability on ‘other mental problems’ and ‘problems with relationships’. Missing data made the two social items unusable.

The paper by Sharma et al (Reference Sharma, Wilkinson and Fear1999) (n=156) provides very useful comment and ideas on the clinical uses of HoNOS. Results relative to its remit are broadly positive. However, the HoNOS item profile and scores were also used to test its value for ‘care planning in day-to-day clinical practice’, which unsurprisingly proved negative. Only clinical judgement based on a range of relevant items in a full data set, including HoNOS as one highly relevant contribution, could do that.

The project carried out by Slade et al (Reference Slade, Beck and Bindman1999) (n=337) provided an opportunity to consider the complementary uses of an instrument such as HoNOS within the context of a full clinical data set. Although the authors did not use it in this way, their data and conclusions amply demonstrate the advantages of such a combination. We fully agree.

The work by Bebbington et al (Reference Bebbington, Brugha and Hill1999) was chiefly concerned with the impact of training on the use of HoNOS. The circumstances of the trials are not fully described but patients newly admitted to an acute ward or recently allocated to a community team in a London (n=57) and a Leicester (n=58) district were included. It is odd that, “even in our severely ill group of patients, most HoNOS items were endorsed only at the minor or mild level” out of the five degrees of severity available. The effect of this on the outcomes and on the reliability results is not discussed. However, in terms of reliability there is a clear gradient. The two full-time researchers who received some training, although not the full one-day course, had the best results. The Leicester keyworkers trained by the full-time researchers were intermediate, and the London keyworkers whose training was ‘less intensive’ performed worst. Nevertheless, the Spearman correlations between keyworkers and research workers on totals and sub-scores were adequate and those at follow-up acceptable and satisfactory.

A curious innovation was the creation of a set of complex algorithms derived from an early version of PSE-10, each intended to test the content of one clinical HoNOS item. Several other instruments were used to provide ‘equivalents’ to the non-clinical items. An unpublished 73-page document (available from P. Bebbington, Royal Free and University College London Medical School, Archway Campus, Whittington Hospital, London N19 5NX, upon request) does little to clarify the procedures. Moreover, the ‘equivalents’ thus derived are untested instruments. Far from being a ‘gold standard’ they have no track record. HoNOS could well be testing them.

The overview by Stein (Reference Stein1999) contains no reference to the aims and recommendations of the original authors. Without an exposition of our intentions, specific caveats and suggestions for use, his comments simply summarise the papers and can be compared with our summaries, above. One specific conclusion needs correction. This is the assertion that HoNOS is unlikely ever to be used by clinicians in routine practice. In fact, the Department of Health has now completed the development work on its new MDS (Reference Glover, Knight and MelzerGlover et al, 1997). HoNOS has been formally adopted for use in the context of major Care Programme Approach reviews, to be available to clinical staff on networked information systems. The data set reader program permits easy and flexible analysis of total and sub-scale scores in association with a wide range of socio-demographic, diagnostic and service use data.

More generally, there has already been a significant uptake of HoNOS. A telephone survey of 140 English trusts providing mental health services, conducted between October 1997 and May 1998, showed that one-third were using HoNOS routinely in one or more service setting. Another third had either received training or had adopted plans to use HoNOS as part of a pilot exercise. Current interest remains at least as high. Even without promotion, the CRU has distributed over 10 000 scales and glossaries to some 90 services during the past two years and has coordinated training in 25 trusts. There are translations into many languages. Updated software will be available early in the year 2000.

What next for HoNOS?

The papers reviewed reflect a real interest in the clinical, epidemiological and administrative ideas behind HoNOS. They also reflect the lack until now of a general strategy to take forward its testing and improvement in an orderly manner within an MDS. The instrument has the potential to fulfil three small but essential roles:

1. (a) The first is as a simple tool that large numbers of nurses and psychiatrists have found useful as a ‘present state profile’. A label with the current profile can be stuck into a case record after each contact with a patient. Good software now under development could make such a record even more useful over time and allow the addition of other information. How the clinician uses the items and three types of score is for the clinician to decide.

2. (b) The second kind of use depends in part upon the goodwill generated by the first but it requires a discipline involving training and supervision to achieve a reasonable degree of reliability as a small part of an MDS, where it can fulfil a vital role as a measure (not a predictor) of progress and for making local comparisons.

3. (c) Third, the total score and subsection scores, when aggregated and anonymised and supervised to establish quality and prevent misuse, could be valuable for epidemiological and administrative functions. No other such simple clinical indicator is at present available.

We suggest that these limited aims are worth pursuing in a coordinated way. HoNOS cannot emulate the computer ‘Deep Thought’, which calculated that the answer to life, the universe and everything was the number 42 (in Douglas Adams' Hitchhiker's Guide to the Galaxy). But it is reasonably robust. The weaker items, either attempting to squeeze too much information into one item or concerned with social information often unavailable to the rating clinicians, do need attention, possibly in the context of a slightly larger HoNOS. As with all such public health documents, training and supervision will undoubtedly also be essential. Ideally there would be strong coordination with workers in other countries.