Is CBT effective in high-quality trials?

Figure 1 shows pooled ESs for positive symptoms, the class of symptoms that CBT was originally developed to treat, in meta-analyses carried out since 2001. A fall over time is evident, with meta-analyses carried out in the last 5 years all finding pooled ESs in the small range (0.1–0.3). It seems likely that this reflects the larger sample sizes and increased attention to methodological factors that have tended to characterize more recent trials.

Fig. 1. Pooled effect sizes for CBT against positive schizophrenic symptoms in meta-analyses from 2001 (all – all controls; TAU – treatment as usual; conint – control psychological interventions).

In our 2014 meta-analysis (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014), we found a significant moderating effect of blinding: the pooled ES reduced from 0.33 to 0.15 (95% CI 0.03–0.27) in 20 trials of overall symptoms, and fell from 0.25 to non-significant levels [0.08 (95% CI −0.03–0.18) in 20 trials of positive symptoms (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014)]. Bighelli et al. (Reference Bighelli, Salanti, Huhn, Schneider-Thoma, Krause, Reitmeir, Wallis, Schwermann, Pitschel-Walz, Barbui, Furukawa and Leucht2018) had more nuanced results in their network meta-analysis: they found the small but significant ES for CBT was maintained in blind studies when compared against TAU (0.27, 95% CI 0.13–0.41) but not against inactive control interventions (0.14, 95% CI −0.09–0.37).

Of course, blinding is not the only measure that contributes to trial quality – others include randomization (especially allocation concealment, the taking of steps to ensure that the randomization code cannot be broken, typically by use of remote allocation) (Schulz and Grimes, Reference Schulz and Grimes2002a), and the use of measures to deal with incomplete outcome data (Schulz and Grimes, Reference Schulz and Grimes2002b). In our meta-analysis (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014), we addressed this broader issue by pooling data from studies that were rated as being at low risk of bias from all three of these factors. ESs for overall and positive symptoms were small and non-significant in these studies [overall symptoms: 0.15 (95% CI −0.01–0.32), eight studies; positive symptoms: 0.10 (95% CI −0.09–0.28), nine studies] (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014, p. 22014). However, pooling data only from trials deemed to be at low risk of bias not only severely restricts the study base but also depends on information given by authors about their methodology, which may be scanty, especially in older studies. In these circumstances, findings from a very large, methodologically rigorous, ‘definitive’ trial would be of considerable interest.

One such trial, the POSITIVE trial of Klingberg et al. (Reference Klingberg, Wittorf, Meisner, Wölwer, Wiedemann, Herrlich, Bechdolf, Müller, Sartory, Wagner, Kircher, König, Engel and Buchkremer2010), comes close to meeting these requirements. Begun in 2007, this aimed to recruit 330 people diagnosed with schizophrenia, schizoaffective disorder or delusional disorder, who were randomized to 9 months of CBT or an intervention designed to control for non-specific elements of therapeutic contact, supportive therapy. Randomization and blinding were state-of-the-art and analysis was planned to be by intention-to-treat. Publication of this trial has been delayed [for details see (see Coyne: http://blogs.plos.org/mindthebrain/2016/02/23/effect-of-a-missing-clinical-trial-on-what-we-think-about-cognitive-behavior-therapy)], but is now anticipated to appear in the near future.

Is CBT effective against particular symptoms?

van der Gaag et al. (Reference van der Gaag, Valmaggia and Smit2014) meta-analyzed 11 studies that separately examined outcomes for delusions and hallucinations, and which were also selected on the basis that they used individually tailored, case formulation-based CBT. The findings for delusions were not greatly different from those of our 2014 meta-analysis (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014) for positive symptoms: the pooled ES was 0.36 in nine studies, reducing to 0.24 in six blind studies. However, the ES was noticeably larger for hallucinations, being 0.44 in 11 studies and, at 0.46, this value was maintained in eight blind studies.

The same year saw publication of the COMMAND trial of Birchwood et al. (Reference Birchwood, Michail, Meaden, Tarrier, Lewis, Wykes, Davies, Dunn and Peters2014), a large (n = 197), 9-month trial of hallucination-adapted CBT compared with TAU. This employed randomization by remote allocation, used multiple measures to prevent unblinding of assessors, and analysis was by intention-to-treat. The primary outcome was harmful compliance with voices: on this measure the CBT group showed significant superiority, although this was not seen at the end of treatment [odds ratio (OR) 0·74, 95% CI 0·40–1·39] but became apparent at 18-month follow-up (OR 0·45, 95% CI 0·23–0·88). No significant effect on overall hallucinations severity was seen, however, at either time point, or on individual hallucination variables such as distress or frequency.

It seems increasingly clear that CBT is ineffective against negative symptoms. Velthorst et al. (Reference Velthorst, Koeter, van der Gaag, Nieman, Fett, Smit, Staring, Meijer and de Haan2015) found no significant effect in 28 trials of negative symptoms as a secondary outcome [ES 0.09 (95% CI −0.03 to 0.21)], nor in two trials where negative symptoms were the primary outcome [ES 0.16 (95% CI −0.10–0.41)]. One of these latter two trials (Klingberg et al., Reference Klingberg, Wölwer, Engel, Wittorf, Herrlich, Meisner, Buchkremer and Wiedemann2011) was large (n = 198), employed randomization by remote allocation, blinding, and analysis by intention-to-treat. Importantly, the form of CBT used was also adapted to specifically target negative symptoms (Klingberg et al., Reference Klingberg, Wölwer, Engel, Wittorf, Herrlich, Meisner, Buchkremer and Wiedemann2011).

CBT for relapse

NICE found an effect of CBT on reducing hospitalization compared with TAU in one of the three meta-analyses examining this outcome carried out for the 2009 guideline (see: National Collaborating Centre for Mental Health: www.nccmh.org.uk). On the other hand, there was no evidence of effectiveness against relapse, when compared with either standard care [relative risk (RR) 0.85 (95% CI 0.50–1.41), three trials] or with other active treatments [RR 1.05 (CI 0.85–1.30), four trials].

Neither of these two latter meta-analyses included the large (n = 218) trial of Garety et al. (Reference Garety, Fowler, Freeman, Bebbington, Dunn and Kuipers2008) of CBT v. TAU. This examined relapse rates based on evidence of re-emergence of or significant deterioration in positive psychotic symptoms lasting at least 2 weeks. The trial employed blinding, randomization by remote allocation, and analysis by intention-to-treat. Its results were negative, leading the authors to conclude generic CBT for psychosis was not indicated for relapse prevention in patients recovering from a recent episode of psychosis.

Is CBT more effective in treatment-resistant patients?

Burns et al. (Reference Burns, Erickson and Brenner2014) meta-analyzed randomized trials of CBT in medication-resistant patients, finding apparently substantial end of treatment ESs of 0.52 (95% CI 0.35–0.70) in 12 studies of overall symptoms and 0.47 (95% CI 0.27–0.67) in nine studies of positive symptoms. However, they used an unconventional measure of ES (difference in change scores divided by end of treatment standard deviation) which does not permit their results to be quantified in terms of the widely used rule of thumb of small (0.1–0.3), medium (0.4–0.7), and large (0.8+). Recalculation of the pooled ES for their nine studies of positive symptoms using a conventional metric (Hedges’ g) gives an approximate result (values had to be estimated for one study) of 0.31 (95% CI 0.12–0.50) falling to 0.26 (95% CI 0.04–0.47) in seven blind studies – not markedly different from the values found in our meta-analysis of the same year (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014).

The recently published FOCUS trial (Morrison et al., Reference Morrison, Pyle, Gumley, Schwannauer, Turkington, MacLennan, Norrie, Hudson, Bowe, French, Byrne, Syrett, Dudley, McLeod, Griffiths, Barnes, Davies and Kingdon2018) examined the effectiveness of CBT over a period of 9 months (plus later booster sessions) v. TAU in 487 patients with clozapine-resistant schizophrenia. As well as being very large, this trial was well-conducted, employed randomization by remote allocation and assessor blinding (with measures taken to address accidental breaking), and analysis was by means of a modern equivalent of intention-to-treat (White et al., Reference White, Horton, Carpenter and Pocock2011). No significant effect was seen at the primary endpoint of 21 months (ES 0.06), but a small effect (ES 0.16) was noted at the end of the 9-month treatment period.

CBT: not a ‘quasi-neuroleptic’?

More than a decade ago, Birchwood and Trower (Reference Birchwood and Trower2006) argued that treating CBT as a ‘quasi-neuroleptic’ is inappropriate and that the intervention is more likely to have a distinctive profile of effects that is complementary to drug treatment rather than substituting for it. Such a view finds echoes in UK clinical guidelines: NICE (2009) emphasized not only effects on psychotic symptoms but also reduction of distress associated with symptoms, promotion of social and educational recovery, and reduction of depression and social anxiety. Similarly, SIGN (2013) stated: ‘The aim [of CBT] is to help the individual normalize and make sense of their psychotic experiences, and to reduce the associated distress and impact on functioning’.

Laws et al. (Reference Laws, Darlington, Kondel, McKenna and Jauhar2018) recently meta-analyzed these relatively understudied outcomes. In 27 trials that examined the effects of CBT on functioning, the pooled ES was small at end-of-trial [ES 0.25 (95% CI 0.14–0.33)], although this became non-significant at follow-up [16 trials, ES 0.10 (95% CI −0.17–0.26)]. A small-to-medium benefit on distress was found at end-of-treatment in eight trials [ES 0.37 (95% CI 0.05–0.69)], which became non-significant when adjusted for possible publication bias [ES 0.18 (95% CI −0.12–0.48)]. There was no evidence of an effect on quality of life in 10 trials [ES 0.04 (95% CI −0.12–0.19)].

Freeman et al. (Reference Freeman, Dunn, Startup, Pugh, Cordwell, Mander, Černis, Wingham, Shirvell and Kingdon2015) examined another non-core outcome, worry secondary to delusions, in a large trial. One hundred and fifty patients with schizophrenia, schizoaffective psychosis, or delusional disorder who scored significantly on a worry questionnaire were randomly assigned to 8 weeks of either worry-directed CBT or TAU. At both end of treatment and 24 weeks follow-up the CBT group showed significantly reduced worry scores (ES estimate common to both outcome points 0.47). This trial was generally methodologically robust, but it should be noted that worry was measured on a self-report scale and so could not be considered to have been evaluated under blind conditions.

Can CBT prevent transition to psychosis?

Hutton and Taylor (Reference Hutton and Taylor2014) meta-analyzed six trials using CBT in individuals at risk of developing psychosis, and found evidence that it was effective in reducing the transition rate at 6, 12, and 18 months [6 months: RR 0.47 (95% CI 0.27–0.82), six studies; 12 months: RR 0.45 (95% CI 0.28–0.73), six studies; 18–24 months: RR 0.41 (95% CI 0.23–0.72), four studies]. This meta-analysis included two large, well-conducted multicenter trials (n = 288, Morrison et al., Reference Morrison, French, Stewart, Birchwood, Fowler, Gumley, Jones, Bentall, Lewis, Murray, Patterson, Brunet, Conroy, Parker, Reilly, Byrne, Davies and Dunn2012 and n = 206, van der Gaag et al., Reference van der Gaag, Nieman, Rietdijk, Dragt, Ising, Klaassen, Koeter, Cuijpers, Wunderink and Linszen2012); the former failed to find a significant difference in transition frequency but the latter found a significant (55%) reduction.

Interpretation of the findings in this area is complicated by a subsequent network meta-analysis that examined the effect of CBT and a range of other interventions (Davies et al., Reference Davies, Cipriani, Ioannidis, Radua, Stahl, Provenzani, McGuire and Fusar-Poli2018). This found no significant effect of CBT on transition compared with standard clinical management. This meta-analysis excluded one of the two multicenter trials included by Hutton and Taylor (Reference Hutton and Taylor2014) (van der Gaag et al., Reference van der Gaag, Nieman, Rietdijk, Dragt, Ising, Klaassen, Koeter, Cuijpers, Wunderink and Linszen2012), on the grounds that it used CBT enhanced with psychoeducation and metacognitive strategies. It also included two new studies: one of these was a relatively small trial (n = 57) (Stain et al., Reference Stain, Bucci, Baker, Carr, Emsley, Halpin, Lewin, Schall, Clarke, Crittenden and Startup2016) which found three transitions in the CBT group but none in the control group, and the other was the as-yet unpublished PREVENT trial (Bechdolf et al., Reference Bechdolf, Muller, Stutzer, Wagner, Maier, Lautenschlager, Heinz, de Millas, Janssen, Gaebel, Michel, Schneider, Lambert, Naber, Brune, Kruger-Ozgurdal, Wobrock, Riedel, Klosterkotter and group2011) which randomized 216 individuals to CBT, antipsychotic treatment, or standard clinical management (plus placebo). A preliminary report of this study (Bechdolf et al., Reference Bechdolf, Müller, Stützer, Lambert, Karow, Zink, Lautenschlager, Heinz, de Millas, Janssen, Gaebel, Schneider, Juckel, Krüger-Özgürdal, Wobrock, Wagner, Maier and Klosterkötter2017) found a smaller number of transitions in the CBT group compared with clinical management (19.2% v. 30.0%), but the difference did not reach statistical significance.