Over the recent years, meta-analysis has become a very influential tool to synthesize information from multiple primary studies of similar design. Widespread use of increasingly complex statistical methods makes it ever more challenging to adequately assess the results reported and conclusions drawn in meta-analyses of psychopharmacological studies.

This study aimed to identify potential fallacies of meta-analytic reporting and interpretation by in-depth examination of recent publications on anti-depressant medication.

Published meta-analytic datasets were re-analysed and the results and possible interpretations obtained in this way were compared with the published results and interpretations.

Several widespread methodological problems were identified in the example studies. As most important among these appear the choice of effect size measures and modeling approaches, as well as the related risk of data dredging. Concerning the level of granularity, two pitfalls encountered were inappropriate aggregation of original data and lack of adequate subgroup analyses. Finally, a low level of transparency regarding data and methodology often hampers re-analysis and cross-checking of reported findings by peers.

The difficulty of replicating meta-analytic results on independent data leads to the often conclusive nature of meta-analytic findings, and therefore a realistic assessment of the limitations of the respective analysis is pertinent. To this end, practically relevant quality criteria for readers to bear in mind when dealing with meta-analytic publications are summarized in a ten point checklist for a rough assessment of the quality of meta-analyses by the reader.