The well-written paper by Álvarez-Jiménez et al ^{Reference Álvarez-Jiménez, Hetrick, González-Blanch, Gleeson and McGorry1} attempts to address a major concern in the management of psychosis, namely weight gain with antipsychotic medication, which has an overarching impact on the management of psychosis. The question it purports to answer is clearly focused and the search strategy thorough and systematic.

However, from the description of the conduct of included trials and assessment of the risk of bias presented in the online Table DS2, it becomes clear that several poor-quality trials were included, with only 2/10 having used an intention-to-treat analysis and 1/10 disclosing allocation concealment.

Proper randomisation is particularly important in small trials as it is relied upon to produce groups with similar baseline characteristics. Poor-quality randomisation would instead produce unequal groups with questionable validity of the results. ^{Reference Hollis and Campbell2}

The attrition rate is particularly high (up to 50%) for the control group in this case. Empirical evidence suggests that participants who adhere to medication tend to do better than those who do not, even after adjustment for all known prognostic factors and irrespective of assignment to active treatment or placebo.

In the absence of an intention-to-treat analysis, the results are biased in an unpredictable manner, compounded by the small size of the trials. Similar problems extend to the subgroup analysis. The authors confirm that the effect size is reduced in the better-quality studies. Three out of four trials in the nutritional therapy subgroup analysis were of poor quality; similarly, four out of five studies in the comparison of individual v. group therapy. Hence, by including poor-quality trials with larger treatment effects in the analysis, the beneficial effect of the intervention has been overestimated.

The choice of mean weight change as an outcome measure is an interesting one as it actually masks the heterogeneity between individuals in small trials. In simple terms, if one person in the intervention arm of the trial, loses 20 kg it skews the results in favour of the intervention even if the other five individuals gained 2 kg each, giving a group mean weight loss of 10 kg. It would perhaps have been more appropriate to have chosen a dichotomous definition of significant weight change (say 5%), so that it would be clear how many individuals actually benefited from the intervention.

The reviewers could have chosen to request the raw data from individual trials, to allow them the opportunity to account for those who withdrew, redo the intention-to-treat analysis and calculate dichotomous weight change outcomes. This, however, would still not resolve the basic problem with regard to quality in individual studies. Well-designed, large-scale pragmatic trials with longer periods of follow-up are needed before undertaking further review in this area, an implication which has been acknowledged by the authors.