Hostname: page-component-cd9895bd7-mkpzs Total loading time: 0 Render date: 2024-12-22T16:10:24.926Z Has data issue: false hasContentIssue false

Replication and contradiction of highly cited research papers: a lesson for the Secretary of State for Health?

Published online by Cambridge University Press:  02 January 2018

Roxanne Keynejad
Affiliation:
South London and Maudsley NHS Foundation Trust, London, UK, email: [email protected]
Tony Rao
Affiliation:
South London and Maudsley NHS Foundation Trust, London, UK
Rights & Permissions [Opens in a new window]

Abstract

Type
Columns
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an open-access article published by the Royal College of Psychiatrists and distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
Copyright © Royal College of Psychiatrists, 2015

Trainees seeking membership of the Royal College of Psychiatrists must, through Paper B examinations, demonstrate understanding of critical review, statistical methods and evidence-based practice. This ensures that no psychiatrist progresses through training without an appreciation of how the findings of isolated studies, even those which are highly cited, should be interpreted and applied in clinical practice. The continuing requirement for critical appraisal skills was supported by a recent study of highly cited psychiatric research reporting effective treatments. Reference Tajika, Ogawa, Takeshima, Hayasaka and Furukawa1 Tajika et al's 10-year follow-up of 83 papers found that only 16 were replicated, and standardised mean differences had been overestimated by 132%. This is consistent with previous studies reporting a devaluation trend, whereby highly cited clinical research findings are contradicted or weakened over time, Reference Ioannidis2 and relates to the ongoing problem that positive results are more likely to be published than negative ones. Reference Schmucker, Schell, Portalupi, Oeller, Cabrera and Bassler3

Junior doctors in England currently find themselves in dispute with the government, which has threatened to impose a new contract. Reference Keynejad4 The issue is distinct, but not entirely divorceable from a wider debate about 7-day National Health services. The government's argument originates from a single study of mortality rates across the week, Reference Freemantle, Ray, McNulty, Rosser, Bennett and Keogh5 although politicians quoted the findings a month before the paper's acceptance by the journal. Reference Hall6 Commenting on the methodological decision to reclassify Monday and Friday as part of the weekend, in contrast to previous analyses, Reference Freemantle, Richardson, Wood, Ray, Khosla and Shahian7 Professor Alistair Hall queried ‘whether such a decision might have been data-driven rather than based on a prior hypothesis – thereby introducing significant bias.’ Many are concerned that government rhetoric neglects the authors' own comment that ‘it is not possible to ascertain the extent to which these deaths may be preventable; to assume they are avoidable would be rash and misleading.’ This prompted the editor of the BMJ to take the unusual step of writing to the Secretary of State for Health, ‘to register my concern about the way in which you have publicly misrepresented an academic article published in the BMJ’. Reference Rimmer and Kmietowicz8

The ‘weekend effect’ has been noted in reverse in psychiatric studies of violent incidents in intensive care Reference Walker and Seifert9 and special hospital Reference Larkin, Murtagh and Jones10 settings. What interpretation should psychiatrists anticipate that politicians could make of this phenomenon, should it cross their radar? For all of Paper B's laudable focus on evidence-based practice, it sets up today's junior doctor for a disappointing confrontation with political reality.

Psychiatrists know better than practitioners of less stigmatised specialties the importance of engaging with the media's narrative about health and disease. A dose-response relationship has been observed for awareness of the anti-stigma Time to Change campaign and improved knowledge and attitudes towards people with mental illness, Reference Evans-Lacko, Corker, Williams, Henderson and Thornicroft11 but these were more likely to be eroded during periods of economic decline.

Our patients have never needed a properly staffed health service more than they do in today's climate of austerity and uncertainty. It is for them that trainees join with multi-disciplinary colleagues to oppose changes lacking an evidence base, which eliminate safeguards, threaten the well-being and diversity of the workforce and compromise its ability to deliver the high-quality service our patients deserve. We take solace from unprecedented steps taken by seniors to advocate for our struggle. 12 In the midst of so much bias, we have found at least one truth.

References

1 Tajika, A, Ogawa, Y, Takeshima, N, Hayasaka, Y, Furukawa, TA. Replication and contradiction of highly cited research papers in psychiatry: 10-year follow-up. Br J Psychiatry 2015; 207: 357–62.CrossRefGoogle Scholar
2 Ioannidis, JPA. Contradicted and initially stronger effects in highly cited clinical research. JAMA 2005; 294: 218–28.CrossRefGoogle ScholarPubMed
3 Schmucker, C, Schell, LK, Portalupi, S, Oeller, P, Cabrera, L, Bassler, D, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS One 2014; 9: e114023.CrossRefGoogle ScholarPubMed
4 Keynejad, RC. Medicine's social movement: #JuniorContract? Lancet Psychiatry 2015; 16 Oct. Available at: http://www.thelancet.com/pb/assets/raw/Lancet/pdfs/S2215036615004745.pdf.Google Scholar
5 Freemantle, N, Ray, D, McNulty, D, Rosser, D, Bennett, S, Keogh, BE, et al. Increased mortality associated with weekend hospital admission: a case for expanded seven day services? BMJ 2015; 351: h4596.CrossRefGoogle ScholarPubMed
6 Hall, AS. Increased mortality associated with weekend hospital admission: a case for expanded seven day services? BMJ 2015; 351: h4596.Google Scholar
7 Freemantle, N, Richardson, M, Wood, J, Ray, D, Khosla, S, Shahian, D, et al. Weekend hospitalization and additional risk of death: an analysis of inpatient data. J R Soc Med 2012; 105: 7484.CrossRefGoogle ScholarPubMed
8 Rimmer, A, Kmietowicz, Z. BMJ editor writes to Hunt over misuse of weekend mortality data. BMJ 2015; 351: h5624.CrossRefGoogle ScholarPubMed
9 Walker, Z, Seifert, R. Violent incidents in a psychiatric intensive care unit. Br J Psychiatry 1994; 164: 826–8.Google Scholar
10 Larkin, E, Murtagh, S, Jones, S. A preliminary study of violent incidents in a special hospital (Rampton). Br J Psychiatry 1988; 153: 226–31.CrossRefGoogle Scholar
11 Evans-Lacko, S, Corker, E, Williams, P, Henderson, C, Thornicroft, G. Effect of the Time to Change anti-stigma campaign on trends in mental-illness-related public stigma among the English population in 2003–13: an analysis of survey data. Lancet Psychiatry 2014; 1: 121–8.CrossRefGoogle ScholarPubMed
12 Royal College of Physicians. Royal College leaders express concern on junior doctor contract negotiations. Royal College of Physicians, 2015; 10 Oct. Available at: https://www.rcplondon.ac.uk/press-releases/royal-college-leaders-express-concern-junior-doctor-contract-negotiations (accessed 16 October 2015).Google Scholar
Submit a response

eLetters

No eLetters have been published for this article.