The limitations of research-based inferences of suitability

Psychotherapy research rarely addresses the question of the ‘fit’ between person and treatment. In this issue, Watzke and colleagues ^{Reference Watzke, Rüddel, Jürgensen, Koch, Kriston and Grothgar1} highlight the scarcity of knowledge about the individual characteristics that may make someone suitable for a particular form of therapy. ^{Reference Roth and Fonagy2} Yet as clinicians we spend a considerable amount of time attempting to identify what might work best for whom. We often make research-based inferences from data about someone's ‘suitability’ for an intervention on the basis of (a) effect sizes calculated from meta-analytic sensitivity analyses, (b) post hoc analyses of variables moderating treatment response rates in randomised controlled trials that compare two treatments, and (c) individual differences in response rates in trials that include only ‘treatment/no treatment’ randomisations. Even more problematic are inferences based on response rates from correlations of outcomes observed in cohort studies. In most contexts, statistical power is insufficient for a meaningful examination of moderator variables. Post hoc analyses are treacherous. Trials are costly and pertinent replications are rare. Predictors of good outcome in follow-along studies are not necessarily related to the treatment concerned. Much of what is known about relative treatment effectiveness is focused on major diagnostic conditions. Even gross moderators such as gender, age and comorbidity are rarely the subject of systematic, statistically valid studies. Clinical judgement of suitability therefore continues to have an important role to play in this field.

Challenges of clinical decision-making about suitability

But are such judgements worthy of the name? Mental health professionals tend to assume that they know what works for whom. Given that the vast majority of psychotherapists believe themselves to be above average in terms of therapeutic effectiveness, ^{Reference Sapyta, Riemer and Bickman3} conviction about competence cannot be considered sufficient grounds for accepting such judgements. Personalised medicine is becoming extremely influential as we see treatments interacting with individual differences in tackling a disease process. Psychotherapy may be light years away from such sophistication, given our lack of understanding of the therapeutic mechanisms by which treatments have their effects. ^{Reference Kazdin4} Yet, most therapists would claim to know intuitively what type of treatment is likely to lead to the best outcome, basing their judgements on a constellation of demographic features, psychological capacities, clinical history (including previous treatment response), contextual factors, personal goals in relation to treatment, and implicit naive theories in relation to both treatment and treatment process that a particular individual presents to the referring clinician. Given this range of parameters, let alone the practically infinite number of combinations in which they might occur, the chance of clinical decision-making of this sort having practical value may seem quite small.

Indication for psychodynamic psychotherapy

Notwithstanding the obvious barriers to informed judgement, the paper by Watzke and colleagues appears to find value in giving thought to what might work best for whom. At least in the case of psychodynamic therapy, there appears to be a valid if implicit clinical algorithm that identifies some patients as more suitable than others for this type of treatment. The findings suggest that this goes beyond superficial judgement of demography, although clearly demography does play a part, and points to indicators such as psychological-mindedness, a wish to target the treatment beyond symptom removal and a concern with the antecedents as well as the relational contexts of the presenting problem. These characteristics make psychodynamic psychotherapy an appropriate choice, at least for relatively short-term treatments assessed in terms of symptom distress 6 months after termination of therapy.

The outcome of unselected assignment for psychodynamic psychotherapy

Two further issues cry out for commentary. First, that cognitive–behavioural therapy (CBT) does as well or better when patients are randomised to this treatment arm (random treatment selection, RTS) as when patients are systematically selected for it (systematic treatment selection, STS): pre–post effect size d _(STS) = 0.50, 95% CI 0.20–0.80 and d _(RTS) = 0.54, 95% CI 0.11–0.96. In other words, even when psychodynamic therapy appears particularly appropriate, it is no more effective than CBT, but when individuals are randomly allocated to this treatment, the outcomes can suffer. By contrast, there appears to be no loss of effectiveness in randomly assigning individuals to CBT, even though presumably some of these individuals might initially have preferred another modality of treatment. On the face of it, the best outcomes appear to be associated with random assignment to CBT. This is despite the fact that CBT in this arm loses some individuals who might be particularly suitable for a cognitive–behavioural treatment approach. The relative benefit that CBT patients receive from being randomised as opposed to assigned to that treatment, despite the reduced number of particularly well-suited individuals, would be consistent with systematic treatment selection favouring psychodynamic treatment because the implicit algorithm ‘cherry picks’ for psychodynamic therapy those patients who are more likely to improve in symptom distress in any case.

Second, patients randomly assigned to the psychodynamic arm of the protocol appeared to change little in terms of symptom distress, whereas those assigned to CBT showed substantial reduction of scores. One does not wish to steal the authors' thunder in relation to future publications, yet these findings, if robust, raise significant questions about a ‘Dodo bird’ verdict in relation to CBT and psychodynamic psychotherapy (i.e. a conclusion that both treatments are equally effective; the term was coined by Wampold after the Dodo bird in Alice in Wonderland who said ‘Everybody has won and all must have prizes’). ^{Reference Wampold5} Unselected consecutive admissions appear better suited to a CBT approach, whereas psychodynamic therapy requires cases to be specifically chosen for this type of treatment. Certainly, although there is no difference between pre-treatment means, 6 months after the end of treatment the difference between the means has an effect size of around half a standard deviation (bias corrected d = 0.52, 95% CI 0.22–0.82). This finding comes at a time when the movement behind gathering evidence for psychodynamic psychotherapy is gathering momentum to the point where it is hard to doubt the value of these methods when administered under reasonably controlled conditions. ^{Reference Leichsenring and Rabung6,Reference Abbass, Kisely and Kroenke7}

Pre-selection may protect from negative treatment effects

What makes Watzke et al's observations particularly important is the pragmatic nature of the trial – real clinicians working as they normally would with just a slight modification to their practice. There may be a minor complication to these findings in that the first level of randomisation was only partially successful (for whatever reason, patients in the systematic treatment selection group had significantly higher initial symptom distress scores), but the statistical control leaves only slight doubt about comparability of the extent of change. Unlike the frequently made claim that non-generalisable experimental methodology exaggerates support for CBT, here the naturalistic design evidently identifies weaknesses in the practice of psychodynamic therapy. Previous experimental findings showing comparable effects of psychodynamic therapy and CBT in randomised controlled trials appear not to be generalisable to an unselected sample of patients in a busy out-patient clinic. It is likely that the lacklustre mean treatment response to psychodynamic therapy in the RTS condition was composed of a number of positive treatment responses as well as some individuals whose response to psychodynamic therapy was one of worsening symptom distress. To put it bluntly, unless patients are pre-screened for suitability they may be harmed by psychodynamic therapy as practised, at least by some psychotherapists working at this clinic.

The take-home message: the risks of routine psychotherapy practice

Is the need to be highly selective before referring for psychodynamic therapy the inevitable conclusion? I think not. It is my belief that the issue here is less of patient pre-selection and more of routine psychotherapy practice. The conceptual frameworks of psychodynamic therapy and CBT differ in that direct feedback from patients about symptom change forms the foundation of practice for clinicians practising CBT but tends to be taken less literally by psychodynamic practitioners, who are trained to focus on ‘process outcomes’ (e.g. transferential responses, insight) that are believed to bring about symptomatic benefit. Rather than making ‘palliative’ suggestions about the inappropriateness of the outcome measure used (which has been used in many successful psychodynamic therapy trials ^{Reference Sandell, Blomberg and Lazar8} ) or the lack of appropriate training of the practitioners involved in the psychodynamic therapy arm (they had more intensive training than most of the therapists in our trials – e.g. Bateman & Fonagy ^{Reference Bateman and Fonagy9} ) or insufficiency of treatment duration (average treatment was longer than in most trials of short-term psychotherapy ^{Reference Abbass, Hancock, Henderson and Kisely10} ), psychodynamic therapists should take these findings to heart as pointing to risks associated with the normal protocols for practising this (usually quite effective) therapy.

The study helps us focus on the need to optimise the effectiveness of routine psychodynamic therapy practice by (a) more rigorous specification of therapeutic methods, including loose manualisation of routine psychodynamic treatment procedures, (b) closer attention to symptomatic as well as process aspects of treatment response in the course of a treatment and (c) attention to the overall effectiveness of individual practitioners and the provision of supervision and support for those whose outcomes are regularly below average. One cannot help wondering whether continuous (session by session) outcome monitoring by appropriate measures (such as the Schwartz Outcome Scale ^{Reference Haggerty, Blake, Naraine, Siefert and Blais11} ) might not have helped some of the patient–therapist pairs achieve better outcomes.

Given that patients largely approach therapy with the aim of receiving help in relation to symptom distress, there is little room for special pleading. We know that psychodynamic therapy is a highly efficacious treatment for a range of psychological disorders. However, when applied in the context of modern healthcare, uncritically implemented parameters of therapeutic psychodynamic practice inherited from the past century can yield disappointing results. Psychodynamic psychotherapists need to upgrade psychodynamic treatment protocols to address the needs of individuals who might otherwise appear to benefit only from a largely symptom-focused orientation.