The basic aims of treatments for schizophrenia, and indeed all severe mental disorders, are to reduce symptoms and distress, enhance functioning and prevent relapse. For many decades, individual variability in the nature and severity of symptoms and differing responses to available treatments suggested achieving these aims would remain elusive. The introduction of second-generation atypical antipsychotics followed by developments such as cognitive–behavioural therapy (CBT) for medication-refractory delusions created renewed optimism, especially as initial randomised controlled trials (RCTs) indicated that these new interventions produced significantly better clinical outcomes with fewer side-effects than all the previously established treatments. Presentations at research meetings began to echo the Dodo in Alice's Adventures in Wonderland: ‘Everybody had won and all must have prizes’. ^{Reference Carroll1} Further efficacy RCTs of second-generation antipsychotics and CBT demonstrated benefits across the full range of severe mental disorders – indeed, a cursory review of the research literature at the turn of the century might easily leave the impression that second-generation antipsychotics and CBT were psychiatry's equivalent of steroids in general medicine.

Failures to replicate the initial impressive benefits in later trials together with a series of meta-analyses of antipsychotics and psychological therapies for schizophrenia suggested that effects had been overestimated. Sadly, this information neither received as much attention as the results of earlier studies, nor deterred those advocating for increased availability of these interventions. The National Institute for Health and Clinical Excellence (NICE) ² in the UK, and similar international organisations elsewhere, published treatment guidelines recommending therapeutic doses of antipsychotic medication, ^{Reference Baldessarini, Cohen and Teicher3} with a nod of approval towards the newer second-generation antipsychotics, as the mainstay of clinical management, but also promoted wider access to adjunctive ‘empirically supported’ therapies such as family therapy and CBT. Providing CBT for psychosis across the National Health Service (NHS) was not supported unanimously. ^{Reference McKenna4} However, the guidance was based on the balance of available evidence at that time from RCTs, still the gold standard for judging the relative benefits of treatments. ^{Reference Schulz, Chalmers, Hayes and Altman5} Most of these trials involved selected patient populations and now we have new evidence from larger pragmatic studies that the second-generation antipsychotics have, quite literally, been oversold, ^{Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins, Keefe, Davis, Davis, Lebowitz, Severe and Hsiao6} and the trial of CBT reported in this issue by Garety et al, ^{Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers7} seems to suggest that this treatment too is in a similar position. So the important question is whether we have truly been led astray or whether these larger studies in the real world are defective and have come up with the wrong answers?

Large-scale multicentre efficacy RCTs of (relatively) homogeneous patient populations are critical in establishing the benefits and tolerability of antipsychotic compounds, but most of these have the prime objective of meeting regulatory requirements. Narrow inclusion criteria dictate that only about 10% of individuals receiving treatment for schizophrenia are eligible for recruitment. ^{Reference Leucht, Heres, Hamann and Kane8} The necessary post-marketing, large-scale effectiveness RCTs are seldom undertaken, but those available less commonly indicate clear advantages for new medications. It is this limited focus and lack of generalisability of efficacy trials that has prompted the multicentre CATIE (Clinical Antipsychotic Trials of Intervention Effectiveness) and CUtLASS (Cost Utility of the Latest Antipsychotic Drugs in Schizophrenia Study) trials. ^{Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins, Keefe, Davis, Davis, Lebowitz, Severe and Hsiao6,Reference Jones, Barnes, Davies, Dunn, Lloyd, Hayhurst, Murray, Markwick and Lewis9} The benefits achieved were considerably less than for highly selected patients receiving carefully titrated doses of medication in controlled research environments. ^{Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins, Keefe, Davis, Davis, Lebowitz, Severe and Hsiao6,Reference Leucht, Heres, Hamann and Kane8–Reference Davies, Lewis, Jones, Barnes, Gaughran, Hayhurst and Markwick12} The medication efficacy-effectiveness gap and failure to achieve significant recovery in schizophrenia stimulated the introduction of adjunctive psychological treatment for those with severe mental disorders. Ironically, the data supporting adjunctive therapy has virtually all come from efficacy RCTs, and it is important to acknowledge that it is not just Big Pharma that introduces bias into such studies; ^{Reference Schulz, Chalmers, Hayes and Altman5} it is equally possible with psychological treatments, and the findings are as much at risk of tendentious reporting as medication studies. The consequence of this now seems to be that when independent multicentre RCTs of effectiveness are carried out in ‘real world’ secondary care settings the apparent additional benefits of new treatments disappear. ^{Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins, Keefe, Davis, Davis, Lebowitz, Severe and Hsiao6,Reference Jones, Barnes, Davies, Dunn, Lloyd, Hayhurst, Murray, Markwick and Lewis9,Reference Scott, Paykel, Morriss, Bentall, Kinderman, Hayhurst, Abbot and Johnson13–Reference Tyrer, Thompson, Schmidt, Jones, Knapp, Davidson, Catalan, Airlie, Baxter, Byford, Byrne, Cameron, Caplan, Cooper, Ferguson, Freeman, Frost, Godley, Greenshields, Henderson, Holden, Keech, Kim, Logan, Manley, MacLeod, Murphy, Patience, Ramsay, De Munroz, Scott, Seivewright, Sivakumar, Tata, Thornton, Ukoumunne and Wessely15} It is in this context that the trial of Garety et al ^{Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers7} needs to be judged.

Onken et al ^{Reference Onken, Blaine, Battjes, Hengeler and Amentos16} describe three progressive stages in the evolution of evidence-based therapies: the development phase (feasibility studies, development of therapy manuals), efficacy RCTs and exploratory studies of mechanisms of action, and studies of generalisability and transportability. Cognitive–behavioural therapy for psychosis had a remarkably long gestation, with case reports appearing nearly 40 years before the publication of pilot RCTs and therapy manuals. There are now over 30 published efficacy trials and a recent comprehensive meta-analysis ^{Reference Wykes, Steel, Everitt and Tarrier17} reported that, compared with usual treatment alone, CBT plus usual treatment demonstrates statistically significant effects on depression, anxiety, positive and negative symptoms and functioning, but not relapse rates. It confirmed that individuals who show the most robust benefit from CBT for psychosis have stable, persistent but distressing, medication-refractory positive symptoms. However, the meta-analysis also demonstrated that as sample sizes and methodological rigour increase, effect sizes decrease by 50–100%.

Effectiveness trials and the exploration of generalisability

In light of this comprehensive evaluation of existing research, ^{Reference Wykes, Steel, Everitt and Tarrier17} the findings of Garety et al ^{Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers7} fall within the range of expected outcomes. The design of their trial also offers insights into factors affecting the general application of CBT and family therapy for psychoses. The findings that CBT is not effective in reducing relapses in psychosis, yet has some benefit on depression, delusional distress and social functioning are consistent with the more modest effect sizes reported in larger-scale efficacy RCTs. Also, Garety et al note that in CBT, for people with psychosis, some of the therapy benefits are restricted to those living with a significant other person (a factor known to predict better outcome in CBT for depression). Unfortunately, the equivalence in group outcomes was not a consequence of a good response to standard care in the controls but rather to a disappointing lack of progress across all randomised groups. Although suboptimal prescribing ^{Reference Baldessarini, Cohen and Teicher3} and non-adherence were frequent, the clear message was that adjunctive therapy did not perform well and it is important to consider the reasons for this. First, in a sample predominantly comprised of single males, the acceptability of and adherence with therapy was no greater than for medication; the drop-out rates for CBT and family therapy (25–30%) are equivalent to those reported for medication, while those allocated to therapy usually attended only about 60% of the sessions offered. Furthermore, even though access to NHS therapy is still restricted and waiting times extensive, only 44% of more than 600 potential participants agreed to inclusion. It is highly likely that, among the many reasons for this low level of agreement, some declined because they did not want therapy. It seemed especially hard to get patients and their family to simultaneously consent to participate in family therapy, and there appeared to be under-representation of high ‘expressed emotion’ families (those we might postulate would benefit most from family therapy) among those recruited.

One barrier to the generalisability of CBT may be the level of competence of local therapists and their compliance to the model. Garety et al monitored quality as much as was feasible, but their comment that ‘therapy was competently delivered’, may be an overestimate. The data provided only allow us to conclude that 44% of those in the CBT and 39% in the family therapy groups received some sessions that met appropriate standards. More tellingly, the authors discuss that, in the absence of acute symptoms or distress, CBT therapists often found it difficult to maintain a clear focus and instead covered a wide range of nonspecific problems and ‘adopted a general approach to emotional distress’. The lack of specific active elements of CBT would suggest the approach had relatively few features distinguishing it from generic case management. This may have undermined the effectiveness of CBT in this RCT, but it is a realistic representation of the problems of delivering therapy in general clinical settings.

Every treatment has limits

No single research trial provides the answers to all the questions that arise when we endeavour to employ research-proven treatments in clinical practice. Critics of efficacy RCTs are quick to suggest we cannot use reported outcomes to predict response rates in heterogeneous clinical samples, but critics of effectiveness trials justifiably argue that these RCTs just trade one set of problems for another. Broad-based, pragmatic trials with few exclusion criteria often mean that moderators of group outcome (e.g. duration of illness, social support, comorbidities, quality of prescribing, compliance) produce so much ‘noise’ that the chances of uncovering evidence of differential treatment response are minimised. ^{Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers7,Reference Scott, Paykel, Morriss, Bentall, Kinderman, Hayhurst, Abbot and Johnson13–Reference Tyrer, Thompson, Schmidt, Jones, Knapp, Davidson, Catalan, Airlie, Baxter, Byford, Byrne, Cameron, Caplan, Cooper, Ferguson, Freeman, Frost, Godley, Greenshields, Henderson, Holden, Keech, Kim, Logan, Manley, MacLeod, Murphy, Patience, Ramsay, De Munroz, Scott, Seivewright, Sivakumar, Tata, Thornton, Ukoumunne and Wessely15} However, RCTs report group outcomes while clinicians treat individuals and we will continue to make selective rather than blanket referrals for CBT. Clinically meaningful information about who is more likely to do well or badly with different treatment packages can be provided from secondary and/or post hoc analyses of effectiveness trials. ^{Reference Lieberman, Stroup, McEvoy, Swartz, Rosenheck, Perkins, Keefe, Davis, Davis, Lebowitz, Severe and Hsiao6,Reference Scott, Paykel, Morriss, Bentall, Kinderman, Hayhurst, Abbot and Johnson13} These secondary studies, despite the perils of post hoc interpretation, can also pave the way for the next step in the research process. In Garety et al's study, the standard deviations are very wide for most of the continuous measures, suggesting considerable interindividual variability in outcome. It may be possible to identify signals relating to patient, therapy/therapist, or combinations of factors suggesting under what circumstances adjunctive therapy should be employed or directing us to where further studies are warranted. Garety et al's study should not be interpreted as a setback for CBT research, but it introduces some healthy realism about the limits for the role of adjunctive therapy in severe mental disorders. It is also a timely reminder that, away from the ‘therapy for all’ media hysteria, the world of routine psychiatric practice brings us into contact with some patients who do not want or do not respond optimally to antipsychotic medication, but who also do not always want or benefit from psychological therapies either.