Hostname: page-component-586b7cd67f-g8jcs Total loading time: 0 Render date: 2024-11-25T16:07:24.915Z Has data issue: false hasContentIssue false
  • English
  • Français

Toward a paradigm shift in treatment and research of mental disorders

Published online by Cambridge University Press:  02 September 2019

Falk Leichsenring ,
Christiane Steinert  and
John P.A. Ioannidis
Falk Leichsenring*
Affiliation:
University of Giessen, Department of Psychosomatics and Psychotherapy, 35392 Giessen, Germany
Christiane Steinert
Affiliation:
University of Giessen, Department of Psychosomatics and Psychotherapy, 35392 Giessen, Germany MSB Medical School Berlin, Department of Psychology, 12447 Berlin, Germany
John P.A. Ioannidis
Affiliation:
Stanford University School of Medicine, Department of Medicine, Stanford, CA 94305, USA Stanford University School of Medicine, Department of Health Research and Policy, Stanford, CA 94305, USA Stanford University School of Medicine, Department of Biomedical Data Science, Stanford, CA 94305, USA Stanford University School of Humanities and Sciences, Department of Statistics, Stanford, CA 94305, USA Meta-Research Innovation Center at Stanford (METRICS), Stanford, CA 94305, USA
*
Author for correspondence: Falk Leichsenring, E-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

An abstract is not available for this content. As you have access to this content, full HTML content is provided on this page. A PDF of this content is also available in through the ‘Save PDF’ action button.

Keywords

Evidencemeta-analysispharmacotherapy efficacypsychotherapy
Type
Editorial
Information
Psychological Medicine , Volume 49 , Issue 13 , October 2019 , pp. 2111 - 2117
Copyright
Copyright © Cambridge University Press 2019 

Mental disorders are common and represent a significant and worldwide public health concern (Smith, Reference Smith2011; Vigo et al., Reference Vigo, Thornicroft and Atun2016; Patel et al., Reference Patel, Saxena, Lund, Thornicroft, Baingana, Bolton, Chisholm, Collins, Cooper, Eaton, Herrman, Herzallah, Huang, Jordans, Kleinman, Medina-Mora, Morgan, Niaz, Omigbodun, Prince, Rahman, Saraceno, Sarkar, De Silva, Singh, Stein, Sunkel and UnÜtzer2018). The global burden of disease due to mental illness accounts for 21–32% of years lived with disability and 7–13% of disability-adjusted life-years (Vigo et al., Reference Vigo, Thornicroft and Atun2016). The Lancet commission on global mental health and sustainable development just recently estimated a loss of US$16 trillion to the global economy due to mental disorders in the period 2010–2030 (Patel et al., Reference Patel, Saxena, Lund, Thornicroft, Baingana, Bolton, Chisholm, Collins, Cooper, Eaton, Herrman, Herzallah, Huang, Jordans, Kleinman, Medina-Mora, Morgan, Niaz, Omigbodun, Prince, Rahman, Saraceno, Sarkar, De Silva, Singh, Stein, Sunkel and UnÜtzer2018).

Psychotherapy and pharmacotherapy are the two key available treatments presently offered to millions of subjects with mental disorders around the world. However, recent evidence suggests that their effects are overestimated due to several factors, such as publication bias, researcher allegiance, and other shortcomings in study design (Ioannidis, Reference Ioannidis2005, Reference Ioannidis2008; Driessen et al., Reference Driessen, Hollon, Bockting, Cuijpers and Turner2015; Tajika et al., Reference Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015; Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016; Leichsenring et al., Reference Leichsenring, Abbass, Hilsenroth, Leweke, Luyten, Keefe, Midgley, Rabung, Salzer and Steinert2017; Leucht et al., Reference Leucht, Leucht, Huhn, Chaimani, Mavridis, Helfer, Samara, Rabaioli, Bacher, Cipriani, Geddes, Salanti and Davis2017; Cuijpers et al., Reference Cuijpers, Karyotaki, Reijnders and Ebert2019; van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019). Thus, the true efficacy of psychotherapy and pharmacotherapy remains contested.

Meta-analyses or systematic reviews of randomized controlled trials (RCTs) are considered to provide the highest level of evidence (1a) (Oxford Centre, 2009). Both meta-analyses and RCTs, however, may differ with regard to the strictness of testing treatment efficacy, depending, for example, on the comparator against which the treatment is tested. Whereas comparisons to waiting list or no-treatment can at best show that a treatment is better than doing nothing, comparisons with treatment as usual (TAU) or placebo show whether treatments have an additional gain compared to TAU or placebo. They also provide information on whether the efforts, costs, and possible side effects of specialized treatments pay off from a health-economic perspective. Thus, these comparisons provide better estimates of the true efficacy of a treatment (Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016).

Recent high-ranking meta-analyses suggest that the efficacy of psychotherapy and pharmacotherapy in comparison to placebo or TAU is limited. For key mental disorders such as depressive disorders (Driessen et al., Reference Driessen, Hollon, Bockting, Cuijpers and Turner2015; Cipriani et al., Reference Cipriani, Furukawa, Salanti, Chaimani, Atkinson, Ogawa, Leucht, Ruhe, Turner, Higgins, Egger, Takeshima, Hayasaka, Imai, Shinohara, Tajika, Ioannidis and Geddes2018; Cuijpers et al., Reference Cuijpers, Karyotaki, Reijnders and Ebert2019), anxiety disorders (Heeren et al., Reference Heeren, Mogoase, Philippot and McNally2015; Curtiss et al., Reference Curtiss, Andrews, Davis, Smits and Hofmann2017; Li et al., Reference Li, Zhu, Su and Fang2017; Liu et al., Reference Liu, Li, Han and Liu2017; Carpenter et al., Reference Carpenter, Andrews, Witcraft, Powers, Smits and Hofmann2018; Gomez et al., Reference Gomez, Barthel and Hofmann2018), somatoform disorders (van Dessel et al., Reference van Dessel, den Boeft, van der Wouden, Kleinstauber, Leone, Terluin, Numans, van der Horst and van Marwijk2014), borderline personality disorder (Cristea et al., Reference Cristea, Gentili, Cotet, Palomba, Barbui and Cuijpers2017a), bipolar disorder (Cipriani et al., Reference Cipriani, Reid, Young, Macritchie and Geddes2013), schizophrenia spectrum disorders (Jauhar et al., Reference Jauhar, McKenna, Radua, Fung, Salvador and Laws2014; Leucht et al., Reference Leucht, Leucht, Huhn, Chaimani, Mavridis, Helfer, Samara, Rabaioli, Bacher, Cipriani, Geddes, Salanti and Davis2017), and psychotherapy of children and adolescents (Weisz et al., Reference Weisz, Jensen-Doss and Hawley2006, Reference Weisz, Kuppens, Eckshtain, Ugueto, Hawley and Jensen-Doss2013, Reference Weisz, Kuppens, Ng, Eckshtain, Ugueto, Vaughn-Coaxum, Jensen-Doss, Hawley, Krumholz Marchette, Chu, Weersing and Fordwood2017, Reference Weisz, Kuppens, Ng, Vaughn-Coaxum, Ugueto, Eckshtain and Corteselli2019; Eckshtain et al., Reference Eckshtain, Kuppens, Ugueto, Ng, Vaughn-Coaxum, Corteselli and Weisz2019), psychotherapy and pharmacotherapy yielded effect sizes in terms of standardized mean differences (SMDs) of about 0.30 or below in comparison with TAU or placebo, especially if effect sizes were adjusted for biases (Leucht et al., Reference Leucht, Leucht, Huhn, Chaimani, Mavridis, Helfer, Samara, Rabaioli, Bacher, Cipriani, Geddes, Salanti and Davis2017; Gomez et al., Reference Gomez, Barthel and Hofmann2018; Cuijpers et al., Reference Cuijpers, Karyotaki, Reijnders and Ebert2019). Large effect sizes (⩾0.80) were only achieved in comparison of psychotherapy to weak comparators such as waiting list conditions (Huhn et al., Reference Huhn, Tardy, Spineli, Kissling, Forstl, Pitschel-Walz, Leucht, Samara, Dold, Davis and Leucht2014; Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016; Liu et al., Reference Liu, Li, Han and Liu2017).

Rates for remission and response were found to be limited as well. For depressive and anxiety disorders, meta-analyses reported rates of remission between 37% and 43% (Cuijpers et al., Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014; Li et al., Reference Li, Zhu, Su and Fang2017; Springer et al., Reference Springer, Levy and Tolin2018). For schizophrenia, a recovery rate of 23% was found (Leucht, Reference Leucht2014). Response rates for depressive and anxiety disorders are about 50% (Cuijpers et al., Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014; Loerinc et al., Reference Loerinc, Meuret, Twohig, Rosenfield, Bluett and Craske2015; Barth et al., Reference Barth, Kriston, Klostermann, Barbui, Cipriani and Linde2016; Imai et al., Reference Imai, Tajika, Chen, Pompoli and Furukawa2016; Li et al., Reference Li, Zhu, Su and Fang2017; Williams et al., Reference Williams, Hattingh, Kariuki, Tromp, van Balkom, Ipser and Stein2017) and 23% for schizophrenia spectrum disorders (Leucht et al., Reference Leucht, Leucht, Huhn, Chaimani, Mavridis, Helfer, Samara, Rabaioli, Bacher, Cipriani, Geddes, Salanti and Davis2017), with response usually defined by a 50% reduction of symptoms (Cuijpers et al., Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014). According to these meta-analyses, presently most patients do not remit and about 50% or more do not respond to the available treatments. Furthermore, success rates of treatments need to be compared to those of placebo or TAU. For depressive and anxiety disorders, placebo response rates range between 35% and 40% (Furukawa et al., Reference Furukawa, Cipriani, Atkinson, Leucht, Ogawa, Takeshima, Hayasaka, Chaimani and Salanti2016; Li et al., Reference Li, Zhu, Su and Fang2017; Williams et al., Reference Williams, Hattingh, Kariuki, Tromp, van Balkom, Ipser and Stein2017). Thus, the difference in response rates in comparison to placebo is between 10% and 15%, indicating small effect sizes in terms of success rate differences, corresponding to SMDs between 0.20 and <0.30 (Kraemer and Kupfer, Reference Kraemer and Kupfer2006).

In this context, it is of note that TAU is a heterogeneous condition and effect sizes may depend on the type of TAU (Watts et al., Reference Watts, Turnell, Kladnitski, Newby and Andrews2015). In a meta-analysis testing different forms of TAU psychotherapy (cognitive–behavior therapy) achieved small effect sizes when compared with general practitioner management (0.20) and larger effect sizes (0.71) when compared with minimal contact (Watts et al., Reference Watts, Turnell, Kladnitski, Newby and Andrews2015). Placebo may be a heterogeneous condition as well when used in trials of psychological interventions. If (psychological) placebos were structurally equivalent to active treatments (e.g. in number and duration of sessions, training of therapists, format of therapy), the differences in outcome were significantly smaller than for structurally inequivalent placebos (SMD = 0.15 v. 0.47) (Baskin et al., Reference Baskin, Tierney, Minami and Wampold2003). Thus, TAU and placebo may be more or less strong comparators, with treatments yielding small effect sizes in comparison to treatments that work or to structurally equivalent placebos and larger effect sizes in comparison to weaker forms of TAU or placebo (Baskin et al., Reference Baskin, Tierney, Minami and Wampold2003; Watts et al., Reference Watts, Turnell, Kladnitski, Newby and Andrews2015).

Further concerns

There are several reasons for further concern.

  1. (1) Even for the above presented estimates of efficacy, it cannot be ruled out that at least some of them are inflated by several biases, such as publication bias, selective reporting of outcomes/analyses, insufficient blinding (psychotherapy studies can per se not be fully double-blind), other shortcomings in study design, financial conflicts (e.g. industry funding) and spontaneous remission due to the natural course of mental disorders (Ioannidis, Reference Ioannidis2005, Reference Ioannidis2008; Cuijpers et al., Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014, Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016; Huhn et al., Reference Huhn, Tardy, Spineli, Kissling, Forstl, Pitschel-Walz, Leucht, Samara, Dold, Davis and Leucht2014; Leichsenring et al., Reference Leichsenring, Abbass, Hilsenroth, Leweke, Luyten, Keefe, Midgley, Rabung, Salzer and Steinert2017; Cipriani et al., Reference Cipriani, Furukawa, Salanti, Chaimani, Atkinson, Ogawa, Leucht, Ruhe, Turner, Higgins, Egger, Takeshima, Hayasaka, Imai, Shinohara, Tajika, Ioannidis and Geddes2018).

  2. (2) As another concern which is consistent with the existence of biases, rates of replication among the most highly-cited articles were found to be low for psychotherapy and pharmacotherapy (Tajika et al., Reference Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015; Sakaluk et al., Reference Sakaluk, Kilshaw, Williams and Rhyner2019): when large and or better studies were done, the initial highly-cited study was found to have overestimated the treatment benefit by 132% (Tajika et al., Reference Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015).

  3. (3) The description of interventions in publications is often remarkably poor (Hoffmann et al., Reference Hoffmann, Glasziou, Boutron, Milne, Perera, Moher, Altman, Barbour, Macdonald, Johnston, Lamb, Dixon-Woods, McCulloch, Wyatt, Chan and Michie2014), in both individual trials and in systematic reviews (Glasziou et al., Reference Glasziou, Macleod, Chalmers, Ioannidis, Al-Shahi Salman and Chan2014; Hoffmann et al., Reference Hoffmann, Oxman, Ioannidis, Moher, Lasserson, Tovey, Stein, Sutcliffe, Ravaud, Altman, Perera and Glasziou2017). Incomplete reporting contributes to an avoidable waste in research (Chalmers et al., Reference Chalmers, Bracken, Djulbegovic, Garattini, Grant, Gulmezoglu, Howells, Ioannidis and Oliver2014; Glasziou et al., Reference Glasziou, Macleod, Chalmers, Ioannidis, Al-Shahi Salman and Chan2014). Poor reporting of interventions was found for pharmacological interventions and even more so for non-pharmacological interventions (Glasziou et al., Reference Glasziou, Meats, Heneghan and Shepperd2008; Schroter et al., Reference Schroter, Glasziou and Heneghan2012; Hoffmann et al., Reference Hoffmann, Erueti and Glasziou2013, Reference Hoffmann, Glasziou, Boutron, Milne, Perera, Moher, Altman, Barbour, Macdonald, Johnston, Lamb, Dixon-Woods, McCulloch, Wyatt, Chan and Michie2014, Reference Hoffmann, Walker, Langhorne, Eames, Thomas and Glasziou2015). In a consecutive sample of RCTs testing non-pharmacological interventions published in six leading general medical journals, only 39% of interventions were found to be adequately described (Hoffmann et al., Reference Hoffmann, Erueti and Glasziou2013). For psychotherapy, treatment integrity (i.e. the degree to which an intervention is delivered as intended) was only adequately reported in 11% of the analyzed studies published in six high-impact-factor journals (Cox et al., Reference Cox, Martinez and Southam-Gerow2019).

  4. (4) Reported effect sizes of psychotherapy for anxiety and depressive disorders seem to have stagnated or even decreased during recent decades (Öst, Reference Öst2008; Johnsen and Friborg, Reference Johnsen and Friborg2015; Friborg and Johnsen, Reference Friborg and Johnsen2017; Cristea et al., Reference Cristea, Stefan, Karyotaki, David, Hollon and Cuijpers2017b; Weisz et al., Reference Weisz, Kuppens, Ng, Vaughn-Coaxum, Ugueto, Eckshtain and Corteselli2019). This is also true for antidepressants in depressive and anxiety disorders and may apply to antipsychotic drugs, too (Schalkwijk et al., Reference Schalkwijk, Undurraga, Tondo and Baldessarini2014; Leucht et al., Reference Leucht, Leucht, Huhn, Chaimani, Mavridis, Helfer, Samara, Rabaioli, Bacher, Cipriani, Geddes, Salanti and Davis2017; Gomez et al., Reference Gomez, Barthel and Hofmann2018). In the latest meta-analysis of 522 trials on antidepressants, the best efficacy estimates were obtained for an old drug, amitriptyline (Cipriani et al., Reference Cipriani, Furukawa, Salanti, Chaimani, Atkinson, Ogawa, Leucht, Ruhe, Turner, Higgins, Egger, Takeshima, Hayasaka, Imai, Shinohara, Tajika, Ioannidis and Geddes2018).

  5. (5) Long-term treatment effects (which may be even smaller than short-term effects) are under-studied (Ioannidis, Reference Ioannidis2008; Huhn et al., Reference Huhn, Tardy, Spineli, Kissling, Forstl, Pitschel-Walz, Leucht, Samara, Dold, Davis and Leucht2014; Steinert et al., Reference Steinert, Kruse and Leichsenring2016; Leichsenring and Leweke, Reference Leichsenring and Leweke2017). Especially for pharmacotherapy, only 5% of studies reported more than just short-term follow-up data (compared to 55% of psychotherapy trials) (Huhn et al., Reference Huhn, Tardy, Spineli, Kissling, Forstl, Pitschel-Walz, Leucht, Samara, Dold, Davis and Leucht2014).

  6. (6) About 20% of patients drop out of psychotherapy, even more of pharmacotherapy (Swift et al., Reference Swift, Greenberg, Tompkins and Parkin2017), with patients apparently experiencing the treatments as not acceptable.

  7. (7) Data on side effects of psychotherapy are scarce (Linden and Schermuly-Haupt, Reference Linden and Schermuly-Haupt2014).

  8. (8) It is unclear whether the effect sizes from randomized clinical trials approximate real-world effectiveness (Sherman et al., Reference Sherman, Anderson, Dal Pan, Gray, Gross, Hunter, LaVange, Marinac-Dabic, Marks, Robb, Shuren, Temple, Woodcock, Yue and Califf2016). Patients seen in clinical practice usually show concomitant disorders but are often excluded from efficacy studies and these patients are more difficult to treat successfully. A large-scale (real-world) effectiveness study, however, recently reported recovery rates of 50% for depressive and anxiety disorders (Clark, Reference Clark2018). These rates are based on self-report measures (Clark, Reference Clark2018), whereas in the meta-analyses cited above, remission rates were based on observer-rated measures.

  9. (9) Finally, despite earlier hopes, research on neuroscience and genetics of mental disorders has not been very successful to identify better treatments or useful biomarkers of treatment effects (Insel, Reference Insel2017). While in daily practice, some patients do respond well and others totally fail, there are no clinically validated biomarkers or other tools to individualize the treatment and to know precisely in advance who will respond best to what treatment (van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019).

Overall, while a certain proportion of patients (who cannot be identified in advance) does benefit from available treatments, most patients do not remit and at least half of the patients do not respond to the available treatments (Cuijpers et al., Reference Cuijpers, Karyotaki, Weitz, Andersson, Hollon and van Straten2014; Leucht, Reference Leucht2014; Li et al., Reference Li, Zhu, Su and Fang2017; Springer et al., Reference Springer, Levy and Tolin2018). Thus, results for the efficacy of psychotherapy and pharmacotherapy are sobering, indicating only a small incremental gain over TAU or placebo and limited rates for remission and response. As noted above, this (limited) incremental gain needs to be balanced against the efforts, costs, and side effects associated with psychotherapy and pharmacotherapy. The situation is aggravated by the numerous concerns mentioned above (e.g. biases, inflated effect sizes, low rates of replication, lack of long-term studies, stagnating or decreasing effect sizes) raising serious doubts about the available evidence.

A dead end?

Each mental disorder raises its own host of issues. However, recent evidence across multiple meta-analyses on key mental disorders provides an overarching picture of limited benefits for both psychotherapy and pharmacotherapy. Some differences for specific disorders are not strong enough to weaken the overall impression that a dead end has been reached in the treatment of mental disorders. For this reason, a paradigm shift seems to be required, fostering a new research agenda which has a clearly different orientation, with more appropriate study design features, outcomes, processes, and funding mechanisms.

Suggestions for a research agenda that makes a difference

To overcome this situation, a research agenda is suggested here which encompasses methodological improvements and strategies to discover new treatments, to identify and evaluate new settings for interventions, and to improve available treatments. In addition, a change in funding policy seems to be required. The community of mental health specialists is already becoming receptive to the possibility of major changes in mindset and strategy, as exemplified in the recent deliberations of the Lancet commission on global mental health and sustainable development (Patel et al., Reference Patel, Saxena, Lund, Thornicroft, Baingana, Bolton, Chisholm, Collins, Cooper, Eaton, Herrman, Herzallah, Huang, Jordans, Kleinman, Medina-Mora, Morgan, Niaz, Omigbodun, Prince, Rahman, Saraceno, Sarkar, De Silva, Singh, Stein, Sunkel and UnÜtzer2018). Mental health is seen as a global challenge in a rapidly changing world and with many unmet needs. While many of these needs reflect policy, public health, and social structures, the ability to meet them will require more effective interventions. For developing and implementing more effective interventions, a paradigm shift with improvements on many different fronts is needed, as we discuss below.

Methodological improvements

As an important first step for further progress, improving study quality is required. The field of mental health interventions needs more reproducible research practices (Tajika et al., Reference Tajika, Ogawa, Takeshima, Hayasaka and Furukawa2015; Sakaluk et al., Reference Sakaluk, Kilshaw, Williams and Rhyner2019). Independent methodological support with larger studies run without industry control, expansion of team science efforts, adversarial collaboration, study pre-registration, adequate reporting, and data sharing may help avoid biases which often lead to overestimation of effect sizes (Open Science Collaboration, 2015; Leichsenring et al., Reference Leichsenring, Abbass, Hilsenroth, Leweke, Luyten, Keefe, Midgley, Rabung, Salzer and Steinert2017; Munafò et al., Reference Munafò, Nosek, Bishop, Button, Chambers, Percie du Sert, Simonsohn, Wagenmakers, Ware and Ioannidis2017). Furthermore, an adequate description of interventions is required for researchers to build on findings or replicate results and for clinicians and patients to reliably implement interventions (Boutron et al., Reference Boutron, Moher, Altman, Schulz and Ravaud2008; Hoffmann et al., Reference Hoffmann, Glasziou, Boutron, Milne, Perera, Moher, Altman, Barbour, Macdonald, Johnston, Lamb, Dixon-Woods, McCulloch, Wyatt, Chan and Michie2014). Both the experimental and the control conditions need to be adequately described (Guidi et al., Reference Guidi, Brakemeier, Bockting, Cosci, Cuijpers, Jarrett, Linden, Marks, Peretti, Rafanelli, Rief, Schneider, Schnyder, Sensky, Tomba, Vazquez, Vieta, Zipfel, Wright and Fava2017) and researcher allegiance needs to be controlled for (Leichsenring et al., Reference Leichsenring, Abbass, Hilsenroth, Leweke, Luyten, Keefe, Midgley, Rabung, Salzer and Steinert2017). To improve reporting of interventions, the template for intervention description and replication checklist and guide (TIDieR) was developed (Hoffmann et al., Reference Hoffmann, Glasziou, Boutron, Milne, Perera, Moher, Altman, Barbour, Macdonald, Johnston, Lamb, Dixon-Woods, McCulloch, Wyatt, Chan and Michie2014). Whether the quality of reporting has improved needs to be examined over time. Furthermore, active comparators need to be included since waiting list or no-treatment conditions are likely to overestimate effect sizes (Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016; Guidi et al., Reference Guidi, Brakemeier, Bockting, Cosci, Cuijpers, Jarrett, Linden, Marks, Peretti, Rafanelli, Rief, Schneider, Schnyder, Sensky, Tomba, Vazquez, Vieta, Zipfel, Wright and Fava2017). While waiting list conditions may be acceptable for a first test of efficacy, active comparators provide more rigorous tests in further steps of research. Long-term follow-ups of RCTs are required capturing major outcomes, including suicide attempts, completed suicides, loss of job, days spent in hospital or on sick leave, overall clinical and social disability, quality of life, side effects, costs, and utilities (Ioannidis, Reference Ioannidis2008). In addition, trials under real-world conditions are needed to also evaluate pragmatic effectiveness (Sherman et al., Reference Sherman, Anderson, Dal Pan, Gray, Gross, Hunter, LaVange, Marinac-Dabic, Marks, Robb, Shuren, Temple, Woodcock, Yue and Califf2016).

Improving available treatments: tailoring the treatment more specifically to the patient

For improving available treatments, a primary focus on the large proportion of patients who do not benefit sufficiently from available treatments or who drop out prematurely is promising (non-responders and drop-outs). Examining, for example, the reasons for prematurely dropping-out allows to identify the limitations of existing treatments (Leichsenring et al., Reference Leichsenring, Sarrar and Steinert2019). This type of research will provide important information about patients’ needs and for improving treatments. Identifying characteristics of drop-outs and non-responders may allow for both differential and adaptive indication, that is, offering alternative treatments or tailoring a treatment more specifically to the patient, in both psychotherapy and pharmacotherapy. Taking into account relevant factors besides a patient's present state such as response to previous treatments (staging) may be helpful (Fava et al., Reference Fava, Tomba and Sonino2012; Steinert et al., Reference Steinert, Kruse and Leichsenring2016).

Related to non-response and dropping-out, there is a perceived need to apply a more flexible psychotherapeutic approach tailoring the treatment more specifically to the patient – one treatment does not fit all (Cloitre, Reference Cloitre2015). This applies to pharmacotherapy as well. Furthermore, since there is evidence to suggest that differences between therapists seem to explain more variance in outcome than differences between treatments, not only in psychotherapy but also in pharmacotherapy (McKay et al., Reference McKay, Imel and Wampold2006; Wampold and Imel, Reference Wampold and Imel2015; van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019), examining patient-treatment matching represents another promising approach (van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019). Focusing on those interactional skills related to better outcome may be helpful in both training and research (van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019). Furthermore, including patients in the evaluation of treatments may help to enhance efficacy and to identify what is helpful, less helpful, or even harmful (Dakin and Arean, Reference Dakin and Arean2013). In this way, treatment manuals may be improved on the basis of systematic patient feedback. Similarly, including patient representatives in discussing study design and results may help to build a new generation of pragmatic trials with patient-centered interventions and outcomes.

This kind of patient-centered research needs to take into account what really matters most to patients, which does not only include improvements in specific symptoms but also in trans-syndromal dimensions, social participation, and existential integration (e.g. well-being, social connectedness, occupational integration) (Tolin et al., Reference Tolin, McKay, Forman, Klonsky and Thombs2015; van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019). For patients who do not achieve response or remission, strengthening resilience in these social and existential domains may be especially helpful (van Os et al., Reference van Os, Guloksuz, Vijn, Hafkenscheid and Delespaul2019).

Quality of treatment implementation and delivery may be a crucial issue. New developments in technology-assisted supervision and training are available that need to be systematically studied (Rousmaniere et al., Reference Rousmaniere, Abbass and Frederickson2014). As a somehow puzzling result, some preliminary data suggest that neither measures of adherence to treatment manuals nor of competence in delivering interventions were associated with outcome (Webb et al., Reference Webb, deRubeis and Barber2010). In routine clinical practice, however, organizational factors of treatment implementation such as problem description, number of treatment sessions, or waiting time before treatment were found to be related to outcome (Clark et al., Reference Clark, Canvin, Green, Layard, Pilling and Janecka2018).

There is evidence that providing feedback on the individual patient's progress may improve the outcome of psychotherapy in patients at risk of non-response (Shimokawa et al., Reference Shimokawa, Lambert and Smart2010). Feedback may include recommendations to alter the treatment plan, shift intervention strategies, or intensify treatment (Shimokawa et al., Reference Shimokawa, Lambert and Smart2010). This approach may be applied to pharmacotherapy as well.

In psychotherapy questions of optimal dosing remain open. While some patients benefit from short-term treatments, long-term treatments may be required for others. Most treatments included in the meta-analyses mentioned above were short-term, encompassing, for example, 1–28 treatment sessions (Loerinc et al., Reference Loerinc, Meuret, Twohig, Rosenfield, Bluett and Craske2015; Cuijpers et al., Reference Cuijpers, Cristea, Karyotaki, Reijnders and Huibers2016). Short-term therapy may be adequate for patients with acute distress (Kopta et al., Reference Kopta, Howard, Lowry and Beutler1994; Lambert, Reference Lambert and Lambert2013). For patients with chronic disorders or personality problems, short-term treatment fails most patients (Kopta et al., Reference Kopta, Howard, Lowry and Beutler1994; Lambert, Reference Lambert and Lambert2013). It is of note that longer-term treatments do not necessarily imply higher health-care costs. In clinical practice in Germany, for example, therapies of an average of 48 sessions are carried out (Albani et al., Reference Albani, Blaser, Geyer, Schmutzer and Brähler2010) which were shown to save health-care costs (Altmann et al., Reference Altmann, Thielemann, Zimmermann, Steffanowski, Bruckmeier, Pfaffinger, Fembacher and Strauss2018). These data also reflect the gap between efficacy research and clinical practice with regard to treatment duration. For longer-term psychotherapy benefits, costs and harms need to be assessed – the assumption that long-term psychotherapy is safe by default is naïve. Rigorous data are needed to test the effectiveness, acceptability, and harms of longer-term psychotherapy as well as its combination and/or alteration with drug treatments.

A patient-centered approach also needs to include adaptive strategies of switching from one treatment to another in case of non-response or augmenting one treatment by another, including augmenting psychotherapy by pharmacotherapy or vice versa (Thase, Reference Thase2014; Markowitz and Milrod, Reference Markowitz and Milrod2015). Switching or augmenting is common in pharmacotherapy research (Rush et al., Reference Rush, Trivedi, Wisniewski, Nierenberg, Stewart, Warden, Niederehe, Thase, Lavori, Lebowitz, McGrath, Rosenbaum, Sackeim, Kupfer, Luther and Fava2006) but such strategies are practically non-existing in psychotherapy research (Markowitz and Milrod, Reference Markowitz and Milrod2015). For psychotherapy, no evidence-based treatment sequence algorithms exist how to proceed if a treatment fails (Markowitz and Milrod, Reference Markowitz and Milrod2015), while designs for such trials are available (Nahum-Shani et al., Reference Nahum-Shani, Qian, Almirall, Pelham, Gnagy, Fabiano, Waxmonsky, Yu and Murphy2012; Steinert et al., Reference Steinert, Kruse and Leichsenring2016). Switching from one form of psychotherapy to another requires that sufficiently different forms of evidence-based treatments are available, that is, a diversity of treatments. For all these approaches, rigorous trials are required.

A focus on prevention: identifying (and evaluating) new opportunities and settings for interventions

Considering different approaches to treatment may offer added value, for example, developing interventions for therapy and prevention at the society, community or workplace level to prevent and/or treat mental disorders. Mental problems such as the ‘burn-out syndrome’ may need interventions in occupational and educational or training settings. Some approaches have been shown to be potentially cost-effective (McDaid and Park, Reference McDaid and Park2011) and health care systems are called for to provide effective interventions (Herpertz et al., Reference Herpertz, Herzog and Taubner2016). Training trainers in the field of health or education in stress prevention, for example, midwives, nurses, teachers, managers in enterprises, pupils, or students, is another promising option (Herpertz et al., Reference Herpertz, Herzog and Taubner2016). Other settings that have been proposed as targets for interventions include the early years of life, for example, supporting parents, parenting, and the parent–infant relationship to enhance infant and maternal mental health (Barlow et al., Reference Barlow, Schrader McMillan, Kirkpatrick, Ghate, Barnes and Smith2010) and families with parents suffering from a mental disorder (Taubner et al., Reference Taubner, Wolter and Rabung2015). Mothers with a borderline personality disorder, for example, may be supported by enhancing their capacities for mentalization and empathy. This applies to foster families as well (Midgley et al., Reference Midgley, Besser, Fearon, Wyatt, Byford and Wellsted2019).

Focusing on healthy aging, at the workplace and in general, is proposed by several stakeholders (McDaid and Park, Reference McDaid and Park2011). In the UK, for example, a Ministry for Loneliness has been established (https://www.nytimes.com/2018/01/17/world/europe/uk-britain-loneliness.html). Depending on its outcome, this could be a model for other countries as well. Finally, early identification and treatment or referral in primary care may prevent chronic developments. For patients who do not have access to face-to-face psychotherapy, Internet-based interventions may be helpful (Andersson and Titov, Reference Andersson and Titov2014; Andrews et al., Reference Andrews, Basu, Cuijpers, Craske, McEvoy, English and Newby2018).

Internet-based therapy achieved similar results as face-to-face therapy with comparable effect sizes (0.38) in relation to TAU (Andrews et al., Reference Andrews, Basu, Cuijpers, Craske, McEvoy, English and Newby2018).

All of these possibilities need to be evaluated rigorously as to their effectiveness v. potential harms, for example, over-diagnosis and over-treatment. To-date some prevention programs have yielded only small-to-medium effect sizes (Taubner et al., Reference Taubner, Wolter and Rabung2015).

Discovering new treatments

For discovering new treatments, research should allow more exploration of high-risk, out-of-the box ideas and accidental discoveries, for example, by not only reporting adverse events but also large unanticipated beneficial effects, by using online patient forums or by studying the effects of non-prescription recreational drugs (Nutt, Reference Nutt2014).

A paradigm shift in funding: not more and more of the same

There is no industry funding research in psychotherapy and the industry has largely shifted away from funding pharmacotherapy trials for mental disorders given the limited success to-date (Smith, Reference Smith2011). Studies addressing the renewed research agenda and the issues listed above need to be properly supported by funding organizations. Decisions on funding from existing public agencies and other funders are often biased toward specific types of inbred research with limited returns, providing just more of the same, for example, funding primarily one form of treatment (Nicholson and Ioannidis, Reference Nicholson and Ioannidis2012; Lorsch, Reference Lorsch2015; MQ, 2015). As advances often spring from unexpected sources, supporting a variety of different (treatment) approaches increases the chance for important discoveries. Initiatives to promote funding of unbiased studies are needed. Payers, insurance companies, and public funders should consider supporting the proposed agenda, given the large burden of disease, accompanying costs and unanswered questions.

Acknowledgements

We thank Juergen Matzat for reviewing and approving this article. Juergen Matzat is an official patient representative of the The Federal Joint Committee (http://www.english.g-ba.de), the highest decision-making body of the joint self-government of physicians, dentists, hospitals, and health insurance funds in Germany.

Author contributions

All authors have contributed to this article and concur on its content.

Conflict of interest

None.

References

Albani, C, Blaser, G, Geyer, M, Schmutzer, G and Brähler, E (2010) Outpatient psychotherapy in Germany from the patient perspective. Part 1: health care situation. Psychotherapeut 55, 503–514.Google Scholar
Altmann, U, Thielemann, D, Zimmermann, A, Steffanowski, A, Bruckmeier, E, Pfaffinger, I, Fembacher, A and Strauss, B (2018) Outpatient psychotherapy improves symptoms and reduces health care costs in regularly and prematurely terminated therapies. Frontiers in Psychology 9, 748.Google Scholar
Andersson, G and Titov, N (2014) Advantages and limitations of internet-based interventions for common mental disorders. World Psychiatry 13, 4–11.Google Scholar
Andrews, G, Basu, A, Cuijpers, P, Craske, MG, McEvoy, P, English, CL and Newby, JM (2018) Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: an updated meta-analysis. Journal of Anxiety Disorders 55, 70–78.Google Scholar
Barlow, J, Schrader McMillan, A, Kirkpatrick, S, Ghate, D, Barnes, J and Smith, M (2010) Health-led interventions in the early years to enhance infant and maternal mental health: a review of reviews. Child and Adolescent Mental Health 15, 178–185.Google Scholar
Barth, M, Kriston, L, Klostermann, S, Barbui, C, Cipriani, A and Linde, K (2016) Efficacy of selective serotonin reuptake inhibitors and adverse events: meta-regression and mediation analysis of placebo-controlled trials. British Journal of Psychiatry 208, 114–119.Google Scholar
Baskin, TW, Tierney, SC, Minami, T and Wampold, BE (2003) Establishing specificity in psychotherapy: a meta-analysis of structural equivalence of placebo controls. Journal of Consulting and Clinical Psychology 71, 973–979.Google Scholar
Boutron, I, Moher, D, Altman, DG, Schulz, KF and Ravaud, P (2008) Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Annals of Internal Medicine 148, 295–309.Google Scholar
Carpenter, JK, Andrews, LA, Witcraft, SM, Powers, MB, Smits, JAJ and Hofmann, SG (2018) Cognitive behavioral therapy for anxiety and related disorders: a meta-analysis of randomized placebo-controlled trials. Depression and Anxiety 35, 502–514.Google Scholar
Chalmers, I, Bracken, MB, Djulbegovic, B, Garattini, S, Grant, J, Gulmezoglu, AM, Howells, DW, Ioannidis, JPA and Oliver, S (2014) How to increase value and reduce waste when research priorities are set. Lancet 383, 156–165.Google Scholar
Cipriani, A, Reid, K, Young, AH, Macritchie, K and Geddes, J (2013) Valproic acid, valproate and divalproex in the maintenance treatment of bipolar disorder. Cochrane Database of Systematic Reviews 10, CD003196.Google Scholar
Cipriani, A, Furukawa, TA, Salanti, G, Chaimani, A, Atkinson, LZ, Ogawa, Y, Leucht, S, Ruhe, HG, Turner, EH, Higgins, JPT, Egger, M, Takeshima, N, Hayasaka, Y, Imai, H, Shinohara, K, Tajika, A, Ioannidis, JPA and Geddes, JR (2018) Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis. Lancet 391, 1357–1366.Google Scholar
Clark, DM (2018) Realizing the mass public benefit of evidence-based psychological therapies: the IAPT Program. Annual Review of Clinical Psychology 14, 159–183.Google Scholar
Clark, DM, Canvin, L, Green, J, Layard, R, Pilling, S and Janecka, M (2018) Transparency about the outcomes of mental health services (IAPT approach): an analysis of public data. Lancet 391, 679–686.Google Scholar
Cloitre, M (2015) The ‘one size fits all’ approach to trauma treatment: should we be satisfied? European Journal of Psychotraumatology 6, 27344.Google Scholar
Cox, J, Martinez, R and Southam-Gerow, M (2019) Treatment integrity in psychotherapy research and implications for the delivery of quality mental health services. Journal of Consulting and Clinical Psychology 87, 221–233.Google Scholar
Cristea, IA, Gentili, C, Cotet, CD, Palomba, D, Barbui, C and Cuijpers, P (2017a) Efficacy of psychotherapies for borderline personality disorder: a systematic review and meta-analysis. JAMA Psychiatry 74, 319–328.Google Scholar
Cristea, IA, Stefan, S, Karyotaki, E, David, D, Hollon, SD and Cuijpers, P (2017b) The effects of cognitive behavioral therapy are not systematically falling: a revision of Johnsen and Friborg (2015). Psychological Bulletin 143, 326–340.Google Scholar
Cuijpers, P, Karyotaki, E, Weitz, E, Andersson, G, Hollon, SD and van Straten, A (2014) The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis. Journal of Affective Disorders 159, 118–126.Google Scholar
Cuijpers, P, Cristea, IA, Karyotaki, E, Reijnders, M and Huibers, MHJ (2016) How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence. World Psychiatry 15, 245–258.Google Scholar
Cuijpers, P, Karyotaki, E, Reijnders, M and Ebert, DD (2019) Was Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression. Epidemiology and Psychiatric Sciences 28, 21–30.Google Scholar
Curtiss, J, Andrews, L, Davis, M, Smits, J and Hofmann, SG (2017) A meta-analysis of pharmacotherapy for social anxiety disorder: an examination of efficacy, moderators, and mediators. Expert Opinion on Pharmacotherapy 18, 243–251.Google Scholar
Dakin, E and Arean, P (2013) Patient perspectives on the benefits of psychotherapy for late-life depression. American Journal of Geriatric Psychiatry 21, 155–163.Google Scholar
Driessen, E, Hollon, SD, Bockting, CL, Cuijpers, P and Turner, EH (2015) Does publication bias inflate the apparent efficacy of psychological treatment for major depressive disorder? A systematic review and meta-analysis of US National Institutes of Health-funded trials. PLoS ONE 10, e0137864.Google Scholar
Eckshtain, D, Kuppens, S, Ugueto, A, Ng, MY, Vaughn-Coaxum, R, Corteselli, K and Weisz, JR (2019) Meta-analysis: 13-year follow-up of psychotherapy effects on youth depression. Journal of the American Academy of Child and Adolescent Psychiatry Epub ahead of print. doi: 10.1016/j.jaac.2019.04.002Google Scholar
Fava, G, Tomba, E and Sonino, N (2012) Clinimetrics: the science of clinical measurements. International Journal of Clinical Practice 66, 11–15.Google Scholar
Friborg, O and Johnsen, TJ (2017) The effect of cognitive-behavioral therapy as an antidepressive treatment is falling: reply to Ljotsson et al. (2017) and Cristea et al. (2017). Psychological Bulletin 143, 341–345.Google Scholar
Furukawa, TA, Cipriani, A, Atkinson, LZ, Leucht, S, Ogawa, Y, Takeshima, N, Hayasaka, Y, Chaimani, A and Salanti, G (2016) Placebo response rates in antidepressant trials: a systematic review of published and unpublished double-blind randomised controlled studies. The Lancet. Psychiatry 3, 1059–1066.Google Scholar
Glasziou, P, Meats, E, Heneghan, C and Shepperd, S (2008) What is missing from descriptions of treatment in trials and reviews? BMJ (Clinical Research Ed.) 336, 1472–1474.Google Scholar
Glasziou, P, Macleod, M, Chalmers, I, Ioannidis, JPA, Al-Shahi Salman, R and Chan, AW (2014) Research: increasing value, reducing waste – authors’ reply. Lancet 383, 1126–1127.Google Scholar
Gomez, AF, Barthel, AL and Hofmann, SG (2018) Comparing the efficacy of benzodiazepines and serotonergic anti-depressants for adults with generalized anxiety disorder: a meta-analytic review. Expert Opinion on Pharmacotherapy 19, 883–894.Google Scholar
Guidi, J, Brakemeier, E, Bockting, C, Cosci, F, Cuijpers, P, Jarrett, R, Linden, M, Marks, I, Peretti, C, Rafanelli, C, Rief, W, Schneider, S, Schnyder, U, Sensky, T, Tomba, E, Vazquez, C, Vieta, E, Zipfel, S, Wright, J and Fava, G (2017) Methodological recommendations for trials of psychological interventions. Psychotherapy and Psychosomatics 87, 276–284.Google Scholar
Gustavsson, A, Svensson, M, Jacobi, F, Allgulander, C, Alonso, J, Beghi, E, Dodel, R, Ekman, M, Faravelli, C, Fratiglioni, L, Gannon, B, Hilton Jones, D, Jennum, P, Jordanova, A, Jönsson, L, Karampampa, K, Knapp, M, Kobelt, G, Kurth, T, Lieb, R, Linde, M, Ljungcrantz, C, Maercker, A, Melin, B, Moscarelli, M, Musayev, A, Norwood, F, Preisig, M, Pugliatti, M, Rehm, J, Salvador-Carulla, L, Schlehofer, B, Simon, R, Steinhausen, H-C, Stovner, LJ, Vallat, J-M, Van den Bergh, P, van Os, J, Vos, P, Xu, W, Wittchen, H-U, Jönsson, B and Olesen, J (2011) Costs of disorders of the brain in Europe 2010. European Neuropsychopharmacology 21, 718–779.Google Scholar
Heeren, A, Mogoase, C, Philippot, P and McNally, RJ (2015) Attention bias modification for social anxiety: a systematic review and meta-analysis. Clinical Psychology Review 40, 76–90.Google Scholar
Herpertz, S, Herzog, W and Taubner, S (2016) Prevention of mental and psychosomatic disorders in occupational and training settings. Mental Health & Prevention 4, 1–2.Google Scholar
Hoffmann, TC, Erueti, C and Glasziou, P (2013) Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials. BMJ (Clinical Research Ed.) 347, f3755.Google Scholar
Hoffmann, TC, Glasziou, P, Boutron, I, Milne, R, Perera, R, Moher, D, Altman, DG, Barbour, V, Macdonald, H, Johnston, M, Lamb, SE, Dixon-Woods, M, McCulloch, P, Wyatt, JC, Chan, AW and Michie, S (2014) Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ (Clinical Research Ed.) 348, g1687.Google Scholar
Hoffmann, TC, Walker, MF, Langhorne, P, Eames, S, Thomas, E and Glasziou, P (2015) What's in a name? The challenge of describing interventions in systematic reviews: analysis of a random sample of reviews of non-pharmacological stroke interventions. BMJ Open 5, e009051.Google Scholar
Hoffmann, T, Oxman, A, Ioannidis, JPA, Moher, D, Lasserson, T, Tovey, D, Stein, K, Sutcliffe, K, Ravaud, P, Altman, D, Perera, R and Glasziou, P (2017) Enhancing the usability of systematic reviews by improving the consideration and description of intervention. BMJ (Clinical Research Ed.) 358, j2998.Google Scholar
Huhn, M, Tardy, M, Spineli, LM, Kissling, W, Forstl, H, Pitschel-Walz, G, Leucht, C, Samara, M, Dold, M, Davis, JM and Leucht, S (2014) Efficacy of pharmacotherapy and psychotherapy for adult psychiatric disorders: a systematic overview of meta-analyses. JAMA Psychiatry 71, 706–715.Google Scholar
Imai, H, Tajika, A, Chen, P, Pompoli, A and Furukawa, TA (2016) Psychological therapies versus pharmacological interventions for panic disorder with or without agoraphobia in adults. Cochrane Database of Systematic Reviews 10, CD011170.Google Scholar
Insel, T (2017) https://www.wired.com/2017/05/star-neuroscientist-tom-insel-leaves-google-spawned-verily-startup/?mbid=social_twitter_onsiteshare.Google Scholar
Ioannidis, JPA (2005) Why most published research findings are false. PLoS Medicine 2, e124.Google Scholar
Ioannidis, JPA (2008) Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials? Philosophy, Ethics, and Humanities in Medicine 3, 14.Google Scholar
Jauhar, S, McKenna, PJ, Radua, J, Fung, E, Salvador, R and Laws, KR (2014) Cognitive-behavioural therapy for the symptoms of schizophrenia: systematic review and meta-analysis with examination of potential bias. British Journal of Psychiatry 204, 20–29.Google Scholar
Johnsen, TJ and Friborg, O (2015) The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: a meta-analysis. Psychological Bulletin 141, 747–768.Google Scholar
Kopta, S, Howard, K, Lowry, J and Beutler, L (1994) Patterns of symptomatic recovery in psychotherapy. Journal of Consulting and Clinical Psychology 62, 1009–1016.Google Scholar
Kraemer, HC and Kupfer, DJ (2006) Size of treatment effects and their importance to clinical research and practice. Biological Psychiatry 59, 990–996.Google Scholar
Lambert, M (2013) The efficacy and effectiveness of psychotherapy. In Lambert, M. (ed.), Bergin and Garfield's Handbook of Psychotherapy and Behavior Change. New York: Wiley, pp. 169–218.Google Scholar
Leichsenring, F and Leweke, F (2017) Social anxiety disorder. New England Journal of Medicine 376, 2255–2264.Google Scholar
Leichsenring, F, Abbass, AA, Hilsenroth, MJ, Leweke, F, Luyten, P, Keefe, JR, Midgley, N, Rabung, S, Salzer, S and Steinert, C (2017) Biases in research: risk factors for non-replicability in psychotherapy and pharmacotherapy research. Psychological Medicine 47, 1000–1011.Google Scholar
Leichsenring, F, Sarrar, L and Steinert, C (2019) Drop-outs in psychotherapy: a change of perspective. World Psychiatry 18, 32–33.Google Scholar
Leucht, S (2014) Measurements of response, remission, and recovery in schizophrenia and examples for their clinical application. Journal of Clinical Psychiatry 75(Suppl. 1), 8–14.Google Scholar
Leucht, S, Leucht, C, Huhn, M, Chaimani, A, Mavridis, D, Helfer, B, Samara, M, Rabaioli, M, Bacher, S, Cipriani, A, Geddes, JR, Salanti, G and Davis, JM (2017) Sixty years of placebo-controlled antipsychotic drug trials in acute schizophrenia: systematic review, Bayesian meta-analysis, and meta-regression of efficacy predictors. American Journal of Psychiatry 174, 927–942.Google Scholar
Li, X, Zhu, L, Su, Y and Fang, S (2017) Short-term efficacy and tolerability of venlafaxine extended release in adults with generalized anxiety disorder without depression: a meta-analysis. PLoS ONE 12, e0185865.Google Scholar
Linden, M and Schermuly-Haupt, ML (2014) Definition, assessment and rate of psychotherapy side effects. World Psychiatry 13, 306–309.Google Scholar
Liu, H, Li, X, Han, B and Liu, X (2017) Effects of cognitive bias modification on social anxiety: a meta-analysis. PLoS ONE 12, e0175107.Google Scholar
Loerinc, AG, Meuret, AE, Twohig, MP, Rosenfield, D, Bluett, EJ and Craske, MG (2015) Response rates for CBT for anxiety disorders: need for standardized criteria. Clinical Psychology Review 42, 72–82.Google Scholar
Lorsch, JR (2015) Maximizing the return on taxpayers’ investments in fundamental biomedical research. Molecular Biology of the Cell 26, 1578–1582.Google Scholar
Markowitz, JC and Milrod, BL (2015) What to do when a psychotherapy fails. The Lancet. Psychiatry 2, 186–190.Google Scholar
McDaid, D and Park, A (2011) Investing in mental health and well-being: findings from the DataPrev project. Health Promotion International 26 (Suppl 1), i108–i139.Google Scholar
McKay, KM, Imel, ZE and Wampold, BE (2006) Psychiatrist effects in the psychopharmacological treatment of depression. Journal of Affective Disorders 92, 287–290.Google Scholar
Midgley, N, Besser, SJ, Fearon, P, Wyatt, S, Byford, S and Wellsted, D (2019) The Herts and Minds study: feasibility of a randomised controlled trial of mentalization-based treatment versus usual care to support the wellbeing of children in foster care. BMC Psychiatry 19, 215.Google Scholar
MQ (2015) MQ Landscape Analysis. UK Mental Health Research Funding.Google Scholar
Munafò, MR, Nosek, BA, Bishop, DV, Button, KS, Chambers, CD, Percie du Sert, N, Simonsohn, U, Wagenmakers, EJ, Ware, J and Ioannidis, JPA (2017) A manifesto for reproducible science. Nature Human Behavior 1, 0021.Google Scholar
Nahum-Shani, I, Qian, M, Almirall, D, Pelham, WE, Gnagy, B, Fabiano, GA, Waxmonsky, JG, Yu, J and Murphy, SA (2012) Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods 17, 457–477.Google Scholar
Nicholson, JM and Ioannidis, JPA (2012) Research grants: conform and be funded. Nature 492, 34–36.Google Scholar
Nutt, D (2014) Help luck along to find psychiatric medicines. Nature 515, 165.Google Scholar
Open Science Collaboration (2015) Estimating the reproducibility of psychological science. Science 349, aac4716.Google Scholar
Öst, L-G (2008) Cognitive behavior therapy for anxiety disorders: 40 years of progress. Nordic Journal of Psychiatry 47, 5–10.Google Scholar
Oxford Centre www.cebm.net/2009/06/oxford-centre-evidence-based-medicine-levels-evidence-march-2009.Google Scholar
Patel, V, Saxena, S, Lund, C, Thornicroft, G, Baingana, F, Bolton, P, Chisholm, D, Collins, PY, Cooper, JL, Eaton, J, Herrman, H, Herzallah, MM, Huang, Y, Jordans, MJD, Kleinman, A, Medina-Mora, ME, Morgan, E, Niaz, U, Omigbodun, O, Prince, M, Rahman, A, Saraceno, B, Sarkar, BK, De Silva, M, Singh, I, Stein, DJ, Sunkel, C and UnÜtzer, J (2018) The Lancet commission on global mental health and sustainable development. Lancet 392, 1553–1598.Google Scholar
Rousmaniere, T, Abbass, AA and Frederickson, J (2014) New developments in technology-assisted supervision and training: a practical overview. Journal of Clinical Psychology 70, 1082–1093.Google Scholar
Rush, AJ, Trivedi, MH, Wisniewski, SR, Nierenberg, AA, Stewart, JW, Warden, D, Niederehe, G, Thase, ME, Lavori, PW, Lebowitz, BD, McGrath, PJ, Rosenbaum, JF, Sackeim, HA, Kupfer, DJ, Luther, J and Fava, M (2006) Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. American Journal of Psychiatry 163, 1905–1917.Google Scholar
Sakaluk, JK, Kilshaw, RE, Williams, AJ and Rhyner, KT (2019) Evaluating the evidential value of empirically supported psychological treatments (ESTs): a meta-scientific review. Journal of Abnormal Psychology 128, 500–509.Google Scholar
Schalkwijk, S, Undurraga, J, Tondo, L and Baldessarini, RJ (2014) Declining efficacy in controlled trials of antidepressants: effects of placebo dropout. International Journal of Neuropsychopharmacology 17, 1343–1352.Google Scholar
Schroter, S, Glasziou, P and Heneghan, C (2012) Quality of descriptions of treatments: a review of published randomised controlled trials. BMJ Open 2, e001978.Google Scholar
Sherman, RE, Anderson, SA, Dal Pan, GJ, Gray, GW, Gross, T, Hunter, NL, LaVange, L, Marinac-Dabic, D, Marks, PW, Robb, MA, Shuren, J, Temple, R, Woodcock, J, Yue, LQ and Califf, RM (2016) Real-world evidence – what is it and what can it tell us? New England Journal of Medicine 375, 2293–2297.Google Scholar
Shimokawa, K, Lambert, MJ and Smart, DW (2010) Enhancing treatment outcome of patients at risk of treatment failure: meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology 78, 298–311.Google Scholar
Smith, K (2011) Trillion-dollar brain drain-enormous costs of mental health problems in Europe not matched by research investment. Nature 478, 15.Google Scholar
Springer, KS, Levy, HC and Tolin, DF (2018) Remission in CBT for adult anxiety disorders: a meta-analysis. Clinical Psychology Review 61, 1–8.Google Scholar
Steinert, C, Kruse, J and Leichsenring, F (2016) Long-term outcome and non-response in psychotherapy: are we short-sighted. Psychotherapy and Psychosomatics 85, 235–237.Google Scholar
Swift, J, Greenberg, R, Tompkins, K and Parkin, S (2017) Treatment refusal and premature termination in psychotherapy, pharmacotherapy, and their combination: a meta-analysis of head-to-head comparisons. Psychotherapy (Chic) 54, 47–57.Google Scholar
Tajika, A, Ogawa, Y, Takeshima, N, Hayasaka, Y and Furukawa, TA (2015) Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. British Journal of Psychiatry 207, 357–362.Google Scholar
Taubner, S, Wolter, S and Rabung, S (2015) Effectiveness of early-intervention programs in German-speaking countries – a meta-analysis. Mental Health & Prevention 3, 69–78.Google Scholar
Thase, ME (2014) Large-scale study suggests specific indicators for combined cognitive therapy and pharmacotherapy in major depressive disorder. JAMA Psychiatry 71, 1101–1102.Google Scholar
Tolin, D, McKay, D, Forman, EM, Klonsky, ED and Thombs, BD (2015) Empirically supported treatments: recommendations for a new model. Clinical Psychology: Science and Practice 22, 317–338.Google Scholar
van Dessel, N, den Boeft, M, van der Wouden, JC, Kleinstauber, M, Leone, SS, Terluin, B, Numans, ME, van der Horst, HE and van Marwijk, H (2014) Non-pharmacological interventions for somatoform disorders and medically unexplained physical symptoms (MUPS) in adults. Cochrane Database of Systematic Reviews 11, CD011142.Google Scholar
van Os, J, Guloksuz, S, Vijn, TW, Hafkenscheid, A and Delespaul, P (2019) The evidence-based group-level symptom-reduction model as the organizing principle for mental health care: time for change? World Psychiatry 18, 88–96.Google Scholar
Vigo, D, Thornicroft, G and Atun, R (2016) Estimating the true global burden of mental illness. The Lancet. Psychiatry 3, 171–178.Google Scholar
Wampold, BE and Imel, ZE (2015) The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work. New York: Routledge.Google Scholar
Watts, SE, Turnell, A, Kladnitski, N, Newby, JM and Andrews, G (2015) Treatment-as-usual (TAU) is anything but usual: a meta-analysis of CBT versus TAU for anxiety and depression. Journal of Affective Disorders 175, 152–167.Google Scholar
Webb, CA, deRubeis, RJ and Barber, J (2010) Therapist adherence/competence and treatment outcome: a meta-analytic review. Journal of Consulting and Clinical Psychology 78, 200–211.Google Scholar
Weisz, JR, Jensen-Doss, A and Hawley, KM (2006) Evidence-based youth psychotherapies versus usual clinical care: a meta-analysis of direct comparisons. American Psychologist 61, 671–689.Google Scholar
Weisz, JR, Kuppens, S, Eckshtain, D, Ugueto, AM, Hawley, KM and Jensen-Doss, A (2013) Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. JAMA Psychiatry 70, 750–761.Google Scholar
Weisz, JR, Kuppens, S, Ng, MY, Eckshtain, D, Ugueto, AM, Vaughn-Coaxum, R, Jensen-Doss, A, Hawley, KM, Krumholz Marchette, LS, Chu, BC, Weersing, VR and Fordwood, SR (2017) What five decades of research tells us about the effects of youth psychological therapy: a multilevel meta-analysis and implications for science and practice. American Psychologist 72, 79–117.Google Scholar
Weisz, JR, Kuppens, S, Ng, MY, Vaughn-Coaxum, RA, Ugueto, AM, Eckshtain, D and Corteselli, KA (2019) Are psychotherapies for young people growing stronger? Tracking trends over time for youth anxiety, depression, attention-deficit/hyperactivity disorder, and conduct problems. Perspectives on Psychological Science 14, 216–237.Google Scholar
Williams, T, Hattingh, CJ, Kariuki, CM, Tromp, SA, van Balkom, AJ, Ipser, JC and Stein, DJ (2017) Pharmacotherapy for social anxiety disorder (SAnD). Cochrane Database of Systematic Reviews 10, CD001206.Google Scholar