Background. In psychosis, the prime indicator of outcome has been relapse, but hospital readmission can no longer be used for this purpose. Researchers now require methods for assessing relapse that are objective, blind, reliable and valid. We describe the reliability and validity of such a technique using case-notes.

Method. Information from routine clinical notes of participants in the Lambeth Early Onset (LEO) study (less all references that would unblind the assessor) were recorded on a form divided into 1-month sections. Operational definitions of remission and relapse enabled clinicians to identify remissions and relapses blindly from the summary information. We calculated reliability regarding both the fact and the timing of remission and relapse. PANSS ratings at 6 and 18 months provided a measure of validity.

Results. The kappa value for the identification of remission by individuals ranged from 0·64 to 0·82, while that for consensus between paired raters was 0·56. The corresponding values for relapse were 0·57–0·59 and 0·71. Intra-class correlations for time to remission and to relapse were very high. Raters guessed correctly whether the participants came from the intervention or control group on 60–75% of occasions. Independent PANSS ratings were strongly related to the remission/relapse status of participants.

Conclusions. The reliability of the technique described here was moderate to good, its validity was good, and it provides a useful and timely addition to methods of evaluating remission and relapse in psychosis. On the basis of our experience, we recommend consensus rather than individual ratings.