Schizophrenia-spectrum disorders (SSDs) are characterized by a variety of psychotic symptoms including delusions, hallucinations, and disorganized speech which frequently interfere with multiple aspects of everyday life (5th ed.; DSM–5; American Psychiatric Association, 2013). Individuals with SSDs often experience aberrant thought patterns including belief inflexibility, a tendency to jump-to-conclusions, and an over-confidence in judgments that are hypothesized to undergird the formation of delusions and other positive symptoms (Moritz et al., Reference Moritz, Andreou, Schneider, Wittekind, Menon, Balzan and Woodward2014a, Reference Moritz, Veckenstedt, Andreou, Bohn, Hottenrott, Leighton and Roesch-Ely2014b). Prior research shows that as many as 80% of people with SSDs suffer from persistent psychotic symptoms even after optimal pharmaceutical interventions (Chien, Leung, Yeung, & Wong, Reference Chien, Leung, Yeung and Wong2013). These observations emphasize the need for novel therapies to target thought processes that may spawn these treatment-resistant, often life-limiting positive symptoms.

Metacognition, which refers to one's ‘knowledge and cognition about cognitive phenomena’ (Flavell, Reference Flavell1979) has been a key target of novel therapies for improving aberrant thought processes, resultant symptoms and function in SSDs. Several different metacognitive interventions have been developed, most prominently Metacognitive training (MCT) (Moritz & Woodward, Reference Moritz and Woodward2007), and Metacognitive and Insight Therapy (MERIT; Vohs et al., Reference Vohs, Leonhardt, James, Francis, Breier, Mehdiyoun and Lysaker2018) for people with SSDs.

The first approach, MCT consists of psychoeducation, cognitive bias training, and strategy training and focuses on raising awareness of cognitive biases (Moritz et al., Reference Moritz, Andreou, Schneider, Wittekind, Menon, Balzan and Woodward2014a, Reference Moritz, Veckenstedt, Andreou, Bohn, Hottenrott, Leighton and Roesch-Ely2014b). In calling attention to individuals’ cognitive biases, this training attempts to ‘sow seeds of doubt’ by enhancing metacognitive knowledge and creating corrective ‘ah-ha’ experiences, encouraging clients to question their process of arriving at beliefs, and to gather more information before important beliefs are formulated (Moritz et al., Reference Moritz, Andreou, Schneider, Wittekind, Menon, Balzan and Woodward2014a, Reference Moritz, Veckenstedt, Andreou, Bohn, Hottenrott, Leighton and Roesch-Ely2014b). This treatment hypothesizes that positive symptoms commonly associated with schizophrenia, particularly delusional thinking, will diminish as aberrant thought processes are remediated.

In MERIT, treatment is formulated in response to the common first-person experience of schizophrenia as consisting of a loss of a previously cohesive sense of self (Lysaker & Lysaker, Reference Lysaker and Lysaker2008). Consequently, MERIT is targeted at helping the client bind small, disconnected experiences of thoughts and feelings as they relate to themselves and others, to help the client re-integrate these experiences into a more complex and nuanced sense of who they and others are as unique individuals, and to use that information to negotiate novel psychological and social challenges (Moritz and Lysaker, Reference Moritz and Lysaker2018). Given each person's unique history and expectations, treatment is tailored to the specific individual, and metacognitive skill is assessed on an ongoing basis and therapy tailored in an ongoing manner to the changing levels of metacognitive capacity.

Nine previous meta-analyses have assessed the efficacy of metacognitive interventions in SSDs. Of these nine studies, six assessed solely MCT (Eichner & Berna, Reference Eichner and Berna2016; Jiang, Zhang, Zhu, Li, & Li, Reference Jiang, Zhang, Zhu, Li and Li2015; Liu, Tang, Hung, Tsai, & Lin, Reference Liu, Tang, Hung, Tsai and Lin2018; Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022; Philipp et al., Reference Philipp, Kriston, Lanio, Kuhne, Harter, Moritz and Meister2019; van Oosterhout et al., Reference van Oosterhout, Smit, Krabbendam, Castelein, Staring and van der Gaag2016) while two assessed MCT along with other metacognitive interventions (Burlingame, Svien, Hoppe, Hunt, & Rosendahl, Reference Burlingame, Svien, Hoppe, Hunt and Rosendahl2020; Sauve, Lavigne, Pochiet, Brodeur, & Lepage, Reference Sauve, Lavigne, Pochiet, Brodeur and Lepage2020). In addition, one study assessed two forms of metacognitive therapy, MCT and MERIT, but analyzed each therapy separately (Lopez-Morinigo et al., Reference Lopez-Morinigo, Ajnakina, Martinez, Escobedo-Aedo, Ruiz-Ruano, Sanchez-Alonso and David2020). Four found improvement on cognitive biases and positive symptoms (Eichner & Berna, Reference Eichner and Berna2016; Liu et al., Reference Liu, Tang, Hung, Tsai and Lin2018; Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022; Sauve et al., Reference Sauve, Lavigne, Pochiet, Brodeur and Lepage2020), three found effects of metacognitive therapy on some targeted outcomes or compared to some control groups but not others (Jiang et al., Reference Jiang, Zhang, Zhu, Li and Li2015; Lopez-Morinigo et al., Reference Lopez-Morinigo, Ajnakina, Martinez, Escobedo-Aedo, Ruiz-Ruano, Sanchez-Alonso and David2020; Philipp et al., Reference Philipp, Kriston, Lanio, Kuhne, Harter, Moritz and Meister2019), and two found nonsignificant results (Burlingame et al., Reference Burlingame, Svien, Hoppe, Hunt and Rosendahl2020; van Oosterhout et al., Reference van Oosterhout, Smit, Krabbendam, Castelein, Staring and van der Gaag2016).

The four most recent meta-analyses provide the most relevant insight into the effects of metacognitive interventions on different classes of outcomes in SSD (Burlingame et al., Reference Burlingame, Svien, Hoppe, Hunt and Rosendahl2020; Lopez-Morinigo et al., Reference Lopez-Morinigo, Ajnakina, Martinez, Escobedo-Aedo, Ruiz-Ruano, Sanchez-Alonso and David2020; Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022; Sauve et al., Reference Sauve, Lavigne, Pochiet, Brodeur and Lepage2020). The Sauve et al., Reference Sauve, Lavigne, Pochiet, Brodeur and Lepage2020 meta-analysis included 29 studies incorporating 15 variations of meta-cognitive therapies, some single-session interventions. Outcomes specifically targeted by the treatment, cognitive biases and positive symptoms, showed a small and moderate effect of treatment, respectively. The authors also reported a moderate effect of treatment on insight. Another meta-analysis of 12 RCTs published in the same year (Lopez-Morinigo et al., Reference Lopez-Morinigo, Ajnakina, Martinez, Escobedo-Aedo, Ruiz-Ruano, Sanchez-Alonso and David2020) focused on clinical and cognitive insight as primary outcomes only. The authors found significant effects of metacognitive therapies on cognitive insight in a six-study analysis. The very small number of studies using inconsistent outcome measures made conclusions regarding the effect of metacognitive therapies on clinical insight difficult.

In the most recent meta-analysis in this area, Penney et al. (Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022) included 43 studies and focused exclusively on a single form of metacognitive therapy: MCT. This analysis found overall small to moderate positive effect sizes for proximal outcomes targeted by the treatment, including positive symptoms, hallucinations, and cognitive biases and a large treatment effect for delusions. Notably, Penney's analysis was the first to find positive effects on some distal outcomes: small effect size improvements for self-esteem and negative symptoms, and small to moderate effect sizes for functioning. Effects of MCT on outcomes not directly targeted by the intervention were diminished in studies with greater study bias, suggesting value in revisiting these outcomes including research designs of only the highest quality.

The current meta-analysis was designed to update and expand knowledge from previous meta-analyses by: (1) including multiple forms of metacognitive therapy for SSD to assess efficacy of this treatment more broadly than the most recent analysis in this area (e.g. Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022; 10 studies of meta-cognitive therapies were excluded in this most recent meta-analysis as they did not study MCT therapy specifically); (2) including a broad range of outcomes (e.g. social cognitive skills, functioning) to help assess treatment generalization; (3) excluding brief, 1-session metacognitive bias-training interventions that may have obscured potential generalization effects of more sustained metacognitive treatments in meta-analyses without this exclusion criteria (e.g. Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022; Sauve et al., Reference Sauve, Lavigne, Pochiet, Brodeur and Lepage2020).

Based on its presumed mechanism of action, as well as previous empirical results, we hypothesized that metacognitive therapies would produce effects on summary measures of positive symptoms, delusions, hallucinations and clinical and cognitive insight, that would generalize to functioning in SSD.

Material and methods

The study was pre-registered through PROSPERO (CRD42022318713).

Search strategy

The meta-analysis was conducted according to the Preferred Reporting for Systematic Reviews and Meta-analysis (PRISMA) guidelines (Moher et al., Reference Moher, Liberati, TeMoher, Liberati, Tetzlaff and Altman2009). Searches were finalized in February of 2022 in PubMed/MEDLINE, PsycINFO, CINAHL, and ProQuest: Dissertation and Thesis Global. Covidence systematic review software (Veritas Health Innovation Ltd, 2022) was used to aid in screening and data extraction. See online Supplementary materials for search strings and Fig. 1 for a PRISMA flow diagram of search results. The reference lists located from all searches, previous meta-analyses, and unpublished articles, were screened to identify potentially eligible articles. Eligibility was assessed by three independent raters in three rounds: titles, abstracts, and full texts. In case of disagreement, publications were reexamined to reach consensus under the supervision of the senior author (MK).

Figure 1. PRISMA.

Inclusion/exclusion criteria

Articles were included if they met the following criteria: (1) were published between 1980 (year of publication of DSM-III) and March, 2022, (2) Sample included at least 70% people with schizophrenia/schizoaffective, delusional disorder, psychotic disorder not otherwise specified, or schizophreniform disorder.; (3) Intervention was sustained (more than 1 session), and focused on training on knowledge and feelings around cognitions; more specifically the intervention had to aim to enhance the ability to synthesize thoughts and feelings about the self and others to better negotiate the day-to-day challenges of life, and/or must address cognitive biases by raising these biases into awareness to reduce distortions in perception of the world and self.; (4) Included a control groups consisting of people with schizophrenia-spectrum illness; (5) if metacognitive therapy was part of a hybrid intervention, metacognitive therapy must have been trained in more than 50% of the training sessions; (6) written in English. Articles that did not report quantitative data that were analyzable by meta-analytic techniques were excluded unless usable data was obtained from correspondence between the research team and study authors. Articles that used outcome measures as a method of intervention were excluded.

Data extraction

Data from each included study was independently extracted by pairs amongst the four authors (GM, MH, AP, MK) using a standardized data extraction spreadsheet. Data extracted included: location/year of the study; premorbid, clinical, and demographic characteristics of the patient and control samples and outcome measures. Interrater reliability, measured from a random sample of data extracted across four coders was 98.2%. Discrepancies between coders were resolved by consensus.

Study outcome measures

Outcome measures consisted of proximal (positive symptoms, delusions, hallucinations, cognitive bias) or distal (cognitive insight, self-esteem, function, depression, anxiety, neurocognitive flexibility, social cognition, metacognition, and clinical insight) outcomes. Tests studied in previous meta-analyses of metacognitive therapies for psychosis-spectrum illness were grouped into these general domains according to previous practice (Penney et al., Reference Penney, Sauve, Mendelson, Thibaudeau, Moritz and Lepage2022). Scales not studied in previous meta-analyses were grouped according to the domain they were presumed to measure. A complete list of included measures and their placement in domains is presented in the online Supplementary.

Study quality

The quality of each study was assessed using the Clinical Trial Assessment Measure (CTAM; Tarrier and Wykes, Reference Tarrier and Wykes2004), a 15-item measure designed to evaluate the quality of methodology for psychological treatment studies. CTAM assesses sample characteristics, allocation to treatment, assessments, control groups, and analysis and active treatment descriptions. The maximum score is 100. CTAMs were all scored and then checked by the senior author (MK). All ratings were sent to the study authors for approval and 40% of the scores reflected author feedback as part of the final CTAM score. Interrater reliability on the CTAM among authors was 89%.

Statistical analysis

Effect-size analyses were conducted according to procedures suggested by Rosenthal and Hedges and Olkin, using Comprehensive Meta-Analysis v. 3 software (Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2014). For purposes of the present study, Hedge's g score was defined as the difference between the metacognitive therapy group v. the non-metacognitive treated schizophrenia-spectrum illness group, expressed in standard deviation units (M_MCT-M_control/SD_pooled across groups) at the termination of treatment. This approach assured consistency with some previous meta-analyses in this research area (e.g. Lopez-Morinigo et al., Reference Lopez-Morinigo, Ajnakina, Martinez, Escobedo-Aedo, Ruiz-Ruano, Sanchez-Alonso and David2020), and has been shown to be a more conservative approach to understanding potential effects in meta-analyses of treatment studies as compared to the use of change scores (Fu & Holmer, Reference Fu and Holmer2016). Statistics were converted to g using Hedges and Olkin formulas (Hedges & Olkin, Reference Hedges and Olkin1985). Pooled standard deviation was calculated using the Rosenthal formula (Rosenthal, Reference Rosenthal1991). Effect sizes were characterized as small (0.2), medium (0.5), or large (0.8; Cohen, Reference Cohen1988). For studies with multiple measures assessing the same domain (e.g. delusions), we selected the measure within that domain that was most frequently used across studies to decrease measure heterogeneity. If a specific measure was used in 10 studies or more within a specific outcome domain we computed the effect-size for that measure to assess whether overall domain effects were similar to those generated from a single measure to provide an index of test heterogeneity on summary effect measures. By expressing effect size in standard deviation units, we were able to make a direct comparison of outcomes across studies. Positive effect size values indicated better scores in metacognitive therapy-treated samples relative to non-metacognitive-treated controls. When negative effect size values were considered improvement in metacognitive therapy-treated schizophrenia samples (e.g. HDRS ratings), we multiplied these values by −1 for ease of communication. When multiple control groups were reported in a specific study, the most active control group was selected as a comparator with metacognitive therapy. Usable data was missing from 8 studies that met our inclusion/exclusion criteria. The authors of these papers were contacted and 75% returned data suitable for analysis.

Effect size synthesis

Individual values of g were thereafter combined across studies and weighted according to their precision. In this approach, larger sample-size, more precise (less variable estimates) are accorded a greater weight in the creation of the summary effect-size estimate using a random-effects model. Potential differences in effect sizes between studies were analyzed using the method of Hedges and Olkin (Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2011; Hedges & Olkin, Reference Hedges and Olkin1985). This procedure computes mean weighted effect sizes and 95% confidence intervals (CI) for each variable subset and allows for the testing of the influence of each individual factor on the overall results using the Q statistic. The two-tailed statistical test was conducted with the criterion for statistical significance set at a level of p = 0.05.

Heterogeneity

We used two measures of heterogeneity: (1) the I² which provides an estimate of the proportion of variability in a meta-analytic dataset attributable to different studies. The I² statistic was interpreted as follows: might not be important (0% – 40%); may represent moderate heterogeneity (30% – 60%); may represent substantial heterogeneity (50% – 90%); considerable heterogeneity (75% – 100%) (Higgins, Thompson, Deeks, & Altman, Reference Higgins, Thompson, Deeks and Altman2003). (2) To assess stability of underlying effects we also used a test for heterogeneity Q_T which is based on the sum of squares of the individual effect sizes around the mean when each square is weighted by the inverse of the estimated variance of the effect size. Q has an asymptotic chi-square distribution and is analogous to the analysis of variance.

Moderator analyses

For outcome measures with significant heterogeneity with I²exceeding 50% and > ten studies, random effects meta-regression analyses were conducted. We evaluated a number of demographic, illness, assessment, and treatment moderator variables. Continuous variables evaluated included: sample mean participant age, gender distribution (% male), mean positive symptoms, racial/ethnic distribution, estimated IQ, duration of treatment, and intensity of treatment, and measures of study quality. Categorical variables were: in v. outpatient sample, active control (yes/no). Continuous data were analyzed with a continuous meta-regression model with a z-test for significance of model fit. Group comparisons were made with ANOVA-type summary values and were estimated for the group effect.

Publication bias

To partially address the ‘file-drawer’ problem, we calculated a fail-safe N using the Orwin method, which provides an estimate of the number of studies with null results that would be needed to render the obtained effect size not clinically meaningful (Orwin, Reference Orwin1983). In the absence of a universally accepted clinical significance level for effect sizes, we assumed a Hedges’ g of 0.1 would cease to reflect a meaningful degree of difference between treatment and control groups, as scores from 96% of participants from the two groups would overlap at this effect-size.

Results

Study characteristics

Sample characteristics

Forty-four studies satisfied our inclusion and exclusion criteria, yielding a participant size of 2423 with an average age of 37.93 (s.d. = 7.80) years. Within the studies included in the analysis, an average of 96.22% (s.d. = 9.75) of the sample had a DSM diagnosis of SCZ and the mean duration of illness was 14.76 years (s.d. = 6.30). At baseline, the sample had a mean PANSS positive score of 16.86 (s.d. = 5.42) and a mean total score of 70.64 (s.d. = 16.97). Twenty-three studies used outpatient participants, 8 had an inpatient sample, and 12 were mixed. Six studies included early stage/FEP participants. Additional information on sample characteristics across all studies is provided in Table 1.

Table 1. Demographic and clinical characteristics of samples included in the meta-analysis

Note: CPZ, chlorpromazine equivalent; IQ, Intelligence Quotient.

Treatment characteristics

A total of 34 studies included MCT and/or MCT + while the remaining 10 studies included other forms of metacognitive therapies. Twenty studies included an active control condition; 24 had an inactive control. For details on individual study control conditions, see online Supplementary 4. Only eight studies reported that participants had received compensation for some component of the study.

Forty studies had group structured metacognitive interventions, while the remaining four studies were individual interventions. The average number of sessions was 12.28 (s.d. = 7.79) over the course of an average treatment length of 8.81 weeks (s.d. = 5.23), with an average of 1.5 sessions/week (s.d. = 0.64).

Study quality

The average score on the Clinical Trials Assessment Measure (CTAM) across all published studies was 59.02 (s.d. = 16.9). Twenty-one (48.0%) of studies were classified as methodologically strong when a CTAM total score of 65 or greater was used to classify a study as of adequate methodological quality.

Meta-analysis results

Table 2 includes effect sizes and heterogeneity calculations for each domain. Additionally, Table 2 includes individual effect sizes for specific measures in each domain that were included in more than 10 studies. Forest plots for all significant outcomes can be found in the online Supplementary materials.

Table 2. Effect size calculations for cognitive, symptomatic, and functional outcomes

Note: DoF, degrees of freedom; CI, confidence interval; Nfs, fail-safe n; PANSS, Positive and Negative Syndrome Scale; PSYRATS, Psychotic Symptom Rating Scale; NA, not applicable; *p < 0.05.

Effects of treatment on proximal outcomes

Metacognitive therapies produced moderate improvement of delusions (g = 0.32; CI: 0.093/0.546) and summary measures of positive symptoms (g = 0.30; CI: 0.169/0.433) relative to a control condition. These effect sizes were consistent for the PSYRATS D (g = 0.39) and PANSS Positive subscale (g = 0.27). There were significant, small effects of metacognitive interventions on cognitive bias (g = 0.25; CI: 0.063/0.426). Effects on hallucinations were non-significant.

Effects of treatment on distal outcomes

Metacognitive interventions had a moderate effect on functioning (g = 0.31; CI: 0.093/0.522). Negative symptoms (g = 0.24; CI: 0.063/0.426), clinical insight (g = 0.29; CI: 0.016/0.569), and social cognition (g = 0.27; 0.015/0.521) showed small but significant effects following metacognitive interventions. The specific PANSS Negative subscale (g = 0.24) is also consistent with the mean effect size of the negative symptom domain. Effects on cognitive insight, depression, and neurocognitive flexibility were non-significant.

Moderator analysis

All sample, study, and treatment characteristics included in the moderator analysis are provided in online Supplementary material. All significant outcomes with significant heterogeneity and a minimum of 10 studies were analyzed. Studies with longer treatment duration were associated with a diminished effect of metacognitive therapies on delusions (slope: −0.1019, s.e.: 0.0507, z = −2.01, p = 0.0444, k = 19). Studies with a greater quality design (higher CTAM) were associated with enhanced effects of therapy on delusions (slope: 0.0143, s.e.: 0.0059, z = 5.82, p = 0.0444, k = 19) but were associated with more modest effects of metacognitive therapies on social cognition (slope: −0.0110, s.e.: 0.0057, z = −1.92, p = 0.0544, k = 12). There was no effect of metacognitive therapy on social cognition (Q_b = 20.762, df = 1, p = 0.000; g = 0.676 v. −0.096) and negative symptoms (Q_b = 4.518, df = 1, p = 0.034; g = 0.405 v. 0.125) in studies with an active control group. The effect of metacognitive therapies on negative symptoms was amplified in studies where participants received compensation for their participation (Q_b = 8.345, df = 2, p = 0.015; g = 0.155 v. 0.577) and in samples with a longer duration of illness (slope: 0.0263, s.e.: 0.0122, z = 2.16, p = 0.0308, k = 14). MCT produced larger effects on positive symptoms than other approaches to metacognitive therapy (Q_b = 7.287, df = 1, p = 0.007; g = 0.337 v. −0.011). All other moderator analyses were non-significant (all p values ⩾0.073).