Introduction
Before the 1990s, psychological therapies for people with psychosis and schizophrenia were widely held to be ineffective and potentially harmful. Since then, meta-analyses of the many randomized controlled trials have indicated that cognitive behavioural therapy (CBT) delivered in addition to routine care (RC) is more effective in improving symptoms than RC alone (Tarrier & Wykes, Reference Tarrier and Wykes2004; Zimmermann et al. Reference Zimmermann, Favrod, Trieu and Pomini2005; Wykes et al. Reference Wykes, Steel, Everitt and Tarrier2008). However, these trials often report positive results for non-specific psychological therapies (counselling, befriending) used as controls, which also turn out to be better than RC alone (Sensky et al. Reference Sensky, Turkington, Kingdon, Scott, Scot, Siddle, O'Carroll and Barnes2000; Tarrier et al. Reference Tarrier, Kinney, McCarthy, Humphreys, Wittkowski and Morris2000).
Randomized controlled trials comparing psychological therapies often either fail to demonstrate a significant difference between treated groups or only demonstrate a small difference, leading to the claim that all therapies are effective but none are more effective than others, sometimes referred to as the ‘Dodo bird conjecture’ (after a character in Alice Through the Looking Glass; Wampold, Reference Wampold2001). For example, in an influential meta-analysis of seven common psychological treatments for mild to moderate depression, Cuijpers et al. (Reference Cuijpers, van Straten, Andersson and van Oppen2008) found no difference in the efficacy of CBT, behavioural activation treatment, psychodynamic treatment, problem-solving therapy or social-skills training. A small increase in efficacy was found for interpersonal psychotherapy (d = 0.20), and non-directive supportive treatment was less efficacious than the other therapies (d = −0.13). Similarly, in a meta-analysis and meta-regression of brief CBT, counselling or problem-solving therapy for depression or anxiety in primary care, Cape et al. (Reference Cape, Whittington, Buszewicz, Wallace and Underwood2010) found that, when diagnosis was controlled for, there was no difference in the effectiveness of the three types of therapy. In meta-analyses comparing CBT to other psychosocial treatments for schizophrenia, Jones et al. (Reference Jones, Hacker, Cormac, Meaden and Irving2012) found no significant differences between psychosocial interventions on positive or negative symptoms, relapse, re-hospitalization, or global mental state measures. Turner & van der Gaag (Reference Turner and van der Gaag2014) similarly reported that for overall symptoms, after sensitivity analyses, there were no significant differences between the efficacies of CBT, social skills training and cognitive remediation. Only the comparison between CBT and befriending demonstrated a significant difference for overall symptoms. Where small differences in the efficacy could be detected, the pattern of these differences was consistent with specific foci of specific interventions, for example CBT, was significantly more efficacious than supportive counselling (SC) for reducing positive symptoms, but not for reducing overall symptoms.
Hence, although it is clear that psychosocial treatments are effective for psychosis, it is also clear that the bulk of the effect size is common to all psychosocial interventions. The most parsimonious explanation for this finding is that the major causal mechanism for change is common to all therapies. Identifying the components of therapy with a causal effect is a growing and clinically important area of research, which has recently benefitted from more methodologically sophisticated statistical techniques. This type of research has important implications for the development of more effective interventions, for clinical practice, and for the theoretical understanding of the process underlying therapeutic change (Green & Dunn, Reference Green and Dunn2008).
The non-specific factor most commonly claimed to have a causal effect on outcome is the therapeutic alliance (TA), defined as the quality of the relationship between therapist and client, characterized by trust and a sense of common purpose (Wampold, Reference Wampold2001). For example, in a comparison of interpersonal therapy, CBT, imipramine with clinical management, and placebo with clinical management in the treatment of depression, the patient-reported alliance was reported to predict outcome in both psychotherapies, pharmacotherapy and placebo therapy (Krupnick et al. Reference Krupnick, Sotsky, Simmens, Moyer, Elkin, Watkins and Pilkonis1996). Meta-analytical reviews of research with mostly non-psychotic patients, across a range of therapies and diagnoses, consistently claim a good alliance predicts positive outcome across different therapy modalities and different conditions, with moderate effect sizes (Martin et al. Reference Martin, Garske and Davis2000; Horvath & Symonds, Reference Horvath and Symonds2001; McLeod, Reference McLeod2011). However, the question of real interest is whether a good TA causes a better outcome, rather than merely correlates with it, in the sense that patients set to have a good outcome regardless of therapy might be those in whom a good TA occurs. All prior research has used a simple ‘ordinary least squares’ regression or correlation analysis and so is not protected from this possibility. This limitation is addressed here by using instrumental variables in a structural equation model, which allows causal interpretations (Pearl, Reference Pearl1998; Halpern & Pearl, Reference Halpern and Pearl2005).
There is already evidence to suggest a heterogeneous response to therapy in patients with psychosis. Dunn et al. (Reference Dunn, Fowler, Rollinson, Freeman, Kuipers, Smith, Steel, Onuwmere, Jolley, Garety and Bebbington2012) in an analysis of the Psychological Prevention of Relapse in Psychosis (PRP) trial, found that, although receipt of full therapy including specific cognitive and behavioural techniques improved clinical outcomes, delivery of partial therapy involving engagement and assessment was not effective; indeed the analysis indicated that for patients with low levels of engagement who are not receiving full therapy, persistence in trying to deliver full therapy can have a detrimental effect on symptoms. This was in contrast to the overall intention-to-treat result of the trial which showed no or minimal effect of CBT, since full therapy was only delivered to a minority. The authors suggest that problems in establishing the TA may have been one of the reasons why full therapy was not delivered to more individuals. For this reason, the present research also investigates the effect of therapy at different levels of alliance.
The SoCRATES randomized controlled trial of CBT for acutely ill first and second episode patients (Lewis et al. Reference Lewis, Tarrier, Haddock, Bentall, Kinderman, Kingdon, Siddle, Drake, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer, Faragher and Dunn2002; Tarrier et al. Reference Tarrier, Lewis, Haddock, Bentall, Drake, Kinderman, Kingdon, Siddle, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer and Dunn2004) was chosen for the present study as (i) measures of the TA were taken, and (ii) it is of high quality; having been independently assessed as the highest quality of 18 trials of CBT for psychosis in a meta-analytical review (Wykes et al. Reference Wykes, Steel, Everitt and Tarrier2008). Strengths of the trial included a large sample (308 patients aged 21–35 years), rigorously blinded assessments, high follow-up rates, multiple assessment points, and checks for assessor reliability. Participants included both inpatients and outpatients and the study received ethical approval from a National Health Service Research Ethics Committee. All participants gave informed consent. There were three arms: CBT, SC (both delivered in addition to RC over 6 weeks in the acute phase with booster sessions) and RC only. RC comprised in most cases inpatient care and antipsychotic medication. Both CBT and SC were manualized and supervised by expert practitioners. Fidelity to CBT (e.g. Socratic questioning style, completion of homework) was independently rated from taped sessions using the psychosis version of the Cognitive Therapy Scale (Haddock et al. Reference Haddock, Devane, Bradshaw, McGovern, Tarrier, Kinderman, Baguley, Lancashire and Harris2001) and was assessed as high for the CBT arm and low for the SC arm, indicating high-quality therapy. The main trial results are reported elsewhere (Lewis et al. Reference Lewis, Tarrier, Haddock, Bentall, Kinderman, Kingdon, Siddle, Drake, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer, Faragher and Dunn2002; Tarrier et al. Reference Tarrier, Lewis, Haddock, Bentall, Drake, Kinderman, Kingdon, Siddle, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer and Dunn2004); this paper reports a secondary analysis. The 18-month follow-up found that both CBT and SC patients fared better on the primary outcome total Positive and Negative Syndrome Scale score (PANSS; Kay et al. Reference Kay, Fiszbein and Opler1987) than patients receiving RC alone (Tarrier et al. Reference Tarrier, Lewis, Haddock, Bentall, Drake, Kinderman, Kingdon, Siddle, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer and Dunn2004) with an effect estimate of −6.22 (s.e. = 2.17, p = 0.005).
Using instrumental variable methods, as proposed and explained in detail in a methodological paper by Dunn & Bentall (Reference Dunn and Bentall2007), we aimed to evaluate the causal effect of the TA as a modifier of the dose-effect of number of sessions attended on the effect of treatment and, subsequently, (1) evaluate the effect of the number of sessions attended at the maximum alliance; (2) evaluate the effect of the number of sessions attended at the poorest level of the TA and (3) to estimate the causal effect on outcome if therapy had occurred at one unit better alliance.
Method
Stata IC/13.1 (StataCorp, 2013) was used for all statistical analyses. The data used in the analyses is obtainable in Stata format from http://www.population-health.manchester.ac.uk/biostatistics/research/data/.
Patients meeting DSM-IV criteria for schizophrenia or schizophreniform disorder during a first (80%) or a second (20%) episode were randomized within 2 weeks to one of three treatment groups. The two psychological treatment groups received weekly sessions for the first 6 weeks, followed by two boosters.
TA was measured using two measures: a therapist-rated measure [the psychotherapy status report (PSR; Frank & Gunderson, Reference Frank and Gunderson1990), and the self-report version of the California Therapeutic Alliance Scales (CALPAS; Marmar et al. Reference Marmar, Gaston, Gallagher and Thompson1989)]. The CALPAS was used in this research as self-report measures correlate more strongly with outcome than therapist-rated measures (Horvath & Symonds, Reference Horvath and Symonds2001). As already noted, there was no significant difference between the improvement in the CBT and SC groups Tarrier et al. (Reference Tarrier, Lewis, Haddock, Bentall, Drake, Kinderman, Kingdon, Siddle, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer and Dunn2004); the effect estimate for this comparison was −0.85 (s.e. = 2.42, p = 0.725). This similarity of outcomes was likely due to factors common to both therapies and cannot be explained by the SC group inadvertently receiving a version of CBT or vice versa as fidelity to CBT was good (see above).
Alliance measures from the fourth session were used, as this is late enough in the therapy for the alliance to have developed, but not too late for it to be an indirect measure of outcome. The CALPAS scores range from 0 to 7 (calculated as the mean items score). In the treated group, the 4th session CALPAS score had 45% missing data, and the number of sessions attended had 12% missing data. The primary outcome measure was the total score on the PANSS, administered by three research psychiatrists blinded to treatment allocation at 18-month follow-up. Structural equation modelling with full information maximum likelihood (SEM FIML) was used to fit the model. The structural equation model is shown in Fig. 1. The parameters of the SEM model were evaluated simultaneously at the point at which they cross the axes (i.e. the point at which the variable is equal to zero), as is customary. Observing the figure, the parameter for sessions (S) gives the effect of an additional session when alliance is zero and the parameter for the interaction of sessions and alliance (SA) gives the effect of increasing the alliance from zero to one. The interaction of sessions and alliance was used in the absence of a main effect alliance alone, as alliance can only have a moderating effect on the effects of the number of sessions attended, and cannot influence outcome (and is not defined) in the absence of therapy. The parameters for S and SA are shown in the Results section as β S and β SA respectively. The model allows for the possibility that additional sessions may have different effects at different levels of alliance. To explore this possibility the model was fitted twice using first the CALPAS scores as they were originally scaled (with 0 the minimum CALPAS score) and then simply re-scaled (by −7) to make 0 the maximum CALPAS score.
Fig. 1. X 1, X 2 and X 3 are baseline variables: the baseline Positive and Negative Syndrome Scale (PANSS) total score, years of education and log of the duration of untreated psychosis. C 1 and C 2 are different centres. The bottom row includes randomization, Z (coded in binary) and the interaction of randomization and the baseline variables. Outcome, Y is the 18-month PANSS total score. S is sessions and SA is the interaction of sessions and alliance. Measurement errors are labelled ε 1, ε 2 and ε 3 . The bottom row therefore shows the interaction of randomization and variables having a causal effect on the number of sessions attended and the interaction of sessions and alliance (the effect of alliance is modelled in a dose-response manner). In turn, these post-randomization variables have a causal effect on the symptomatic outcome (the PANSS 18-month outcome score). The paths connecting the post-randomization variables to outcome are of primary interest and these are shown in the results section as β S and β SA . The top row of Fig. 1 shows the baseline variables directly affecting the PANSS 18-month score. The strength of these causal relationships is the same for patients randomized to receive a psychological therapy or not. By using the interactions of randomization and baseline variables in the bottom row, and coding randomization to treatment as usual only as 0, patients not receiving a talking therapy are not included in the causal pathway estimates in the bottom part of the diagram.
Instrumental variable methods were used to control for the potential effects of additional variables (hidden common causes, i.e. confounders) affecting the relationship between both the number of sessions attended and level of TA on outcome. The instrumental variables are the interactions between randomization and baseline covariates on the bottom line of Fig. 1. They influence outcome but this influence is fully explained by their effects on the number of sessions attended and the product of sessions and the TA. The variables used were selected for these properties. The baseline variables used to create instruments are log DUP (the log of the duration of untreated psychosis), years of education, baseline PANSS score, and centre (a binary variable; three different centres were used; two of which were included explicitly in the model, leaving the other centre implicitly coded as the comparison centre). No further variables were used as models with fewer variables have better statistical properties and are more parsimonious.
The random disturbances (‘errors’) included in the causal model (ε 1, ε 2, ε 3) show that the symptomatic outcome, dose and the interaction of dose and alliance are all modelled as imperfectly predicted. Importantly, allowing for correlation between these errors (shown with curved bidirectional arrows), allows for hidden confounding, the problem that makes most research investigating the effect of post-randomization variables merely correlational rather than causal. To state this more clearly, not allowing for these correlations between post-randomization variables and outcome would not have excluded the possibility that those who tend to develop good alliance also tend to have a good outcome (i.e. that perhaps the alliance does not cause the good outcome).
Full Information Maximum Likelihood (FIML) estimation was used to fit the instrumental variable model, a procedure that includes cases with missing data, grouping data according to missing data patterns. Probable values for the missing data are implied by the observed values. FIML deals appropriately with data missing at random, and has a superior statistical profile than a complete cases analysis, which only analyses the observed data (referred to as listwise deletion in the SEM literature) (Yuan et al. Reference Yuan, Yang-Wallentin and Bentler2012; Enders & Bandalos, Reference Enders and Bandalos2001).
Ethical standards
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Results
Firstly, summary statistics from the SoCRATES trial are presented in Table 1. The results of the analysis described in this paper are presented in Table 2. The core (causal) parameters of our statistical model are the effects of increasing sessions (when alliance is coded to be zero: β S ) and of increasing the product of sessions and alliance (the interaction: β SA ) on outcome (18-month PANSS score). The strength of the TA (CALPAS score) is coded in two ways: (a) from −7 (lowest alliance) to 0 (maximum alliance), and (b) from 0 (lowest alliance) to +7 (maximum alliance). This makes no difference to the fit of the model (they are mathematically equivalent) but the use of each option aids the interpretation of the parameter β S ). For option (a) β S is the effect of sessions at maximum alliance; for option (b) β S is the effect of sessions at minimum alliance. The interaction effect (β SA ) should be unaffected. We fit the models to two different data sets: (1) including only those subjects for which there is a measure of both sessions attended and the TA available, and (2) using the full dataset, allowing for the gaps (missing values) in the data.
Table 1. Summary statistics from the SoCRATES trial by centre
CBT, Cognitive behavioural therapy; SC, supportive counselling; RC, routine care; PANSS, Positive and Negative Syndrome Scale; DUP, duration of untreated psychosis; CALPAS, California Therapeutic Alliance Scales.
a The missing data for intermediate outcomes refers to the treated group only.
Table 2. Causal estimands for the effect of sessions and alliance
CI, Confidence interval; SEM FIML, structural equation modelling with full information maximum likelihood.
Given the relatively large proportion of trial participants with a missing TA measure, it is important that the causal parameters of the explanatory models are fitted using methods that satisfactorily allow for these missing data (i.e. using FIML SEM). The initial methodological work of Dunn & Bentall (Reference Dunn and Bentall2007) was illustrated using only those SoCRATES participants who had non-missing data for both sessions and alliance. Here we focus on the FIML SEM estimates. However, our findings do not appear to be unduly influenced by the method chosen to deal with missing data.
The key substantive finding is that there is a statistically significant sessions by alliance interaction effect on outcome. That is, the effect of increasing the number of sessions attended is dependent on the participants’ reported TA. At one extreme (a reported CALPAS score of 7 – the best level of alliance) increasing the number of sessions attended improves the effectiveness of the therapy (total 18-month PANSS score reduced, on average, by just under 3 points for every session attended). At the other extreme (a reported CALPAS score of 0 – the worst level of alliance) increasing the number of sessions attended makes things worse – the therapy appears to be detrimental (total 18-month PANSS score increased, on average, by about 7.5 points for every session attended). As the confidence intervals for each estimate both have the same sign as the estimate (e.g. the confidence intervals for positive effects are both positive), there is a high level of certainty that these effects exist.
Discussion
Much empirical evidence suggests that the nature and strength of the TA established between therapist and client explains some of the effect of psychotherapies. This appears to be so across different types of therapy, and across diagnostic boundaries. However, until now, it has not been possible to be sure that this is a truly causal relationship: that a good TA contributes to, rather than simply correlates with, the good outcome. This study used structural equation modelling to deal with unmeasured confounding (correlation between post-randomization variables and outcome) and potential bias due to missing data. We have presented three main findings: in psychological therapies for psychosis, (i) a good TA causes improvement; (ii) a poor TA has a detrimental effect, and (iii) improving the alliance causes a better outcome. Hence, the findings provide strong evidence that the TA was a common causal factor contributing to the outcome of patients receiving either of the therapies, CBT or SC. If this finding proves generalizable to other therapies and conditions, it may tend to support for the Dodo bird conjecture that all therapies are (usually) equally effective (Wampold, Reference Wampold2001) and help to explain why this is the case.
The finding that therapy may have a detrimental effect when the alliance is poor is equally important and perhaps an echo of traditional attitudes about treatment for schizophrenia, which dictated that any discussion of the details of the psychosis or delusional ideas should be avoided, as it would exacerbate the psychosis or constitute ‘inadvertent collusion’ (McCabe & Priebe, Reference McCabe and Priebe2008). Hence, although CBT has been indicated to be effective in many randomized controlled trials, the present findings indicate that therapy should proceed with caution if the TA is poor. A corollary is that CBT is more likely to be an effective treatment for psychosis if only delivered when a good TA is possible. Current debates about the effectiveness of CBT (e.g. Jauhar et al. Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014) that fail to address the heterogeneity of treatment response may underestimate the effects of treatment in optimal circumstances.
Previous studies have reported the TA to be a predictor of other important outcomes in severe mental illness, for example attitudes to medication (Day et al. Reference Day, Bentall, Roberts, Randall, Rogers, Cattell, Healy, Rae and Power2005), adherence to antipsychotic medication (McCabe et al. Reference McCabe, Bullenkamp, Hansson, Lauber, Martinez-Leal, Rössler, Salize, Svensson, Torres-Gonzalez, van den Brink, Wiersama and Priebe2012), the effectiveness of community mental health case management (Howgego et al. Reference Howgego, Yellowlees, Owen, Meldrum and Dark2003), maintenance of contact with services (Johansson & Eklund, Reference Johansson and Eklund2006) and supported workplace performance (Davis & Lysaker, Reference Davis and Lysaker2007), indicating the importance of the quality of therapeutic relationships in many aspects of recovery from psychosis. The statistical methodology used in this paper extends our understanding, showing that for psychological therapies, the relationship is not just correlational, but causal. Hence, an important clinical implication is that psychiatric services may be able to improve outcomes by improving the quality of therapeutic relationships across the range of interactions, and by developing a more personalized approach in which interventions are tailored to patients’ needs.
The limitations of the present paper are that the variation in alliance was constrained by the model to be all due to client variation, i.e. as the assignment of therapist was not randomized, it was not possible to model the variation in TA due to the therapist, a model which would acknowledge that some therapists tend to form better alliances than others (Wampold, Reference Wampold2001). This issue would benefit from further research. Additionally, further research may investigate whether the causal effect of the TA is observed in therapies for other conditions. It might be argued that another limitation is our inclusion of both the CBT- and SC-treated patients in the same analysis, as the two therapies place a different emphasis on the importance of the alliance as a mechanism, at least in theory. For example, Rogers (Reference Rogers1957) perhaps the leading advocate of the SL approach, emphasized the quality of the therapeutic relationship as a necessary and sufficient condition for successful therapy whereas CBT therapists tend to see the alliance as more instrumental in ensuring the patient's adherence to the treatment protocol (e.g. Dunn et al. Reference Dunn, Morrison and Bentall2006). However, separating the groups or studying only one of the conditions would have severely limited the numbers available, and the two interventions did not produce significantly different outcomes (Tarrier et al. Reference Tarrier, Lewis, Haddock, Bentall, Drake, Kinderman, Kingdon, Siddle, Everitt, Leadley, Benn, Grazebrook, Haley, Akhtar, Davies, Palmer and Dunn2004). Moreover, the focus of our approach has been to test the hypothesis that the alliance is a common mechanism affecting outcome, as might be expected according to the Dodo bird conjecture (Wampold, Reference Wampold2001).
Strengths and weaknesses in relation to other studies
A small number of prior studies have examined the relationship between the TA and outcome in psychotic patients receiving psychotherapy. Gunderson et al.'s (Reference Gunderson, Frank, Katz, Vannicelli, Frosch and Knapp1984) clinical trial of psychodynamic psychotherapy reported the alliance, as rated by the therapist, strongly predicted outcome even when other variables, for example, symptom severity at inception, were taken into account. Svensson & Hansson (Reference Svensson and Hansson2007) reported a positive relationship between initial patient-rated alliance scores and outcome in 26 schizophrenia-spectrum patients treated with CBT. Similarly, the meta-analytical reviews, which consider mostly non-psychotic patients, across a range of therapies and diagnoses, find that the alliance predicts outcome moderately well (Martin et al. Reference Martin, Garske and Davis2000; Horvath & Symonds, Reference Horvath and Symonds2001; McLeod, Reference McLeod2011). However, these studies essentially calculated the correlation between alliance and outcome. The prior research concerning TA can be interpreted as indicating that patients who tend to form good alliances also tend to have a better prognosis, or, at least, it is not protected methodologically from such an interpretation. In terms of inferring causality, these studies are methodologically confounded by essentially just correlating the alliance score with outcome.
This research dealt with that problem by using structural equation modelling, and allowing for correlation between post-randomization variables and outcome separately to the causal effect. Missing data, which can bias estimates or suggest inaccurate confidence intervals, was addressed using FIML estimation.
Conclusions
This analysis shows clearly the causal effect of TA on the outcome of psychotherapy for psychosis. The results indicate that at high levels of alliance, therapy is beneficial, but that at low levels of alliance, therapy is detrimental. Clinically, this suggests that establishing a good alliance in psychotherapy for psychosis is essential for a patient to benefit from therapy and that if the alliance is poor, persistence in trying to engage the client in psychotherapy is not appropriate. Future trials of psychological treatments for psychosis should consider methods of maximizing the alliance, or at least employ procedures for discontinuing therapy if the alliance is poor. A wider implication is that psychiatric services should prioritize ensuring that all staff engage effectively with patients.
Acknowledgements
This research received no specific grant from any funding agency, commercial or not-for-profit sectors. The SoCRATES trial was supported by a grant from the UK Medical Research Council.
Declaration of Interest
None.
Open access
