Equivalence and non-inferiority trials typically compare the effects of a treatment of interest with the current gold-standard treatment as the comparator. For this approach, the definition of equivalence or non-inferiority margins (NIM) is crucial, as we had noted earlier (Rief and Hofmann, Reference Rief and Hofmann2018). Leichsenring et al. (Reference Leichsenring, Abbass, Driessen, Hilsenroth, Luyten, Rabung and Steinert2018) opposed our arguments, based on other examples and their own trials advocating for psychodynamic treatments.

First, we want to thank Leichsenring and colleagues for their thorough report and interest on our paper. The authors present an impressive variety of NIMs that have been used in prior publications, confirming our argument that there is no clear consensus for defining NIMs. We also accept their critique that we did not differentiate between non-inferiority and equivalence trials; our critique applies to both types and does not require this differentiation.

The reasons for erroneous non-inferiority results of two treatments can be manifold. Poor study quality, insensitive ascertainment or statistical analyses procedures, poor implementation of (comparator) treatments are just a few examples, as outlined in our original commentary. Even if Leichsenring et al. do not appreciate our examples, they do not provide a convincing argument against these conclusions. We continue to encourage the scientific community to consider the pivotal question with regard to NIMs: What is the risk of erroneous equivalence conclusions that we want to accept as responsible clinicians and scientists in the era of the replicability crisis?

We explicitly reject Leichsenring's notion that we misrepresented the results of the Steinert et al. (Reference Steinert, Munder, Rabung, Hoyer and Leichsenring2017) study. Therefore, we present the original results in Fig. 1 here again. While the authors concluded that results indicate equality between two treatments, we noted that for some variables, significant differences are reported. The overall effect as depicted by the diamond of Fig. 1 clearly indicates a significant disadvantage at post-treatment of psychodynamic treatments compared with other psychological interventions [g = −0.153; 90% confidence intervals (CI) −0.23 to −0.08], or compared with cognitive behavioral therapy (CBT) (g = −0.16; 90% CI −0.24 to −0.08; please note: CI are similar if tested for equivalence compared with superiority). These results are of special interest because the financial sponsor was a German psychodynamic society, and the principal investigators are well-known for their advocacy in favor of psychodynamic treatments (despite one CBT-coauthor). This example illustrates one major point we wanted to make: the very same dataset can confirm the superiority of one treatment, and the equivalence of both treatments, depending on the chosen methodological approach. Again, Leichsenring et al. do not reject this conclusion, which is mainly built on methodological reasoning independent of this specific example, and which we consider as relevant to be discussed in the scientific community. For more details and critical aspects of non-inferiority/equivalence trials, we refer the reader to our original commentary.

Fig. 1. Superior, inferior, or equivalent? The example of the Steinert et al. study.

The scientific community has to define better and clearer criteria for NIMs. We do not believe that it is ethically justifiable to recommend a treatment that is 10 or 20% less effective than the current gold standard interventions. For some of the examples of NIMs summarized by Leichsenring et al. the treatments may be even 30 or 50% less effective than comparator treatments, even though authors reported them as ‘equally effective treatment alternatives’. This is not acceptable, neither ethically nor scientifically.