Post-traumatic stress disorder is a disabling condition resulting from a range of traumas and affecting many people worldwide. This month's Cochrane Corner review systematically searched and reported findings from 66 randomised controlled trials of pharmacotherapy for PTSD, 54 of which were included in a meta-analysis. Evidence was shown for the benefit of selective serotonin reuptake inhibitors, mirtazapine and amitriptyline in treatment response. This Round the Corner commentary critically appraises the review's findings, concluding that the summative evidence was of poor quality owing to the low number of studies, the high risk of bias and significant heterogeneity.

This study reports the first assessment of published comments in the family medicine literature using structured codes, which produced commentary annotations that will be the foundation of a knowledge base of appraisals of family medicine trials. Evidence appraisal occurs in a variety of formats and serves to shed light on the quality of research. However, scientific discourse generally and evidence appraisal in particular has not itself been analyzed for insights. A search strategy was devised to identify all journal comments indexed in PubMed linked to controlled intervention studies published in a recent 15-year period in major family medicine journals. A previously developed structured representation in the form of a list of appraisal concepts was used to formally annotate and categorize the journal comments through an iterative process. Trends in family medicine evidence appraisal were then analyzed. A total of 93 comments on studies from five journals over 15 years were included in the analysis. Two thirds of extracted appraisals were negative criticisms. All appraisals of measurement instruments were negative (100%). The participants baseline characteristics, the author discussions, and the design of the interventions were also criticized (respectively 91.7%, 84.6% and 83.3% negative). In contrast, appraisals of the scientific basis of the studies were positive (81.8%). The categories with the most appraisals were, most generally, those focused on the study design, and most specifically, those focused on the scientific basis. This study provides a new data-driven approach to review scientific discourse regarding the strengths and limitations of research within academic family medicine. This methodology can potentially generalize to other medical domains. Structured appraisal data generated here will enable future clinical, scientific, and policy decision-making and broader meta-research in family medicine.

Baillon et al’s Cochrane review included 430 participants with agitation in dementia from five randomised controlled trials. Overall, the reviewers found that valproate was no better than placebo for the treatment of agitation in people with dementia; however, the quality of the studies included was very variable. Adverse effects and events were higher in the treatment group compared with the controls, although these finding were largely based on low-quality data with incomplete reporting; thus, valproate's safety profile is of concern. This review demonstrates that there is insufficient evidence to change current treatment guidelines.

Introduction: The use of free open access medicine, particularly open educational resources (OERs), by medical educators and learners continues to increase. As OERs, especially blogs and podcasts, rise in popularity, their ease of dissemination raises concerns about their quality. While critical appraisal of primary research and journal articles is formally taught, no training exists for the assessment of OERs. Thus, the ability of educators and learners to effectively assess the quality of OERs using gestalt alone has been questioned. Our goal is to determine whether gestalt is sufficient for emergency medicine learners (EM) and physicians to consistently rate and reliably recommend OERs to their colleagues. We hypothesized that EM physicians and learners would differ substantively in their assessment of the same resources. Methods: Participants included 31 EM learners and 23 EM attending physicians from Canada and the U.S. A modified Dillman technique was used to administer 4 survey blocks of 10 blog posts per subject between April and August, 2015. Participants were asked whether they would recommend each OER to 1) a learner or 2) an attending physician. The ratings reliability was assessed using single measures intraclass correlations and their correlations amongst the groups were assessed using Spearman’s rho. Family-wise adjustments were made for multiple comparisons using the Bonferroni technique. Results: Learners demonstrated poor reliability when recommending resources for other learners (ICC= 0.21, 95% CI 0.13-0.39) and attending physicians (ICC = 0.16, 95% CI=0.09-0.30). Similarly, attendings had poor reliability when recommending resources for learners (ICC= 0.27, 95% CI 0.18-0.41) and other attendings (ICC=0.22, 95% CI 0.14-0.35). Learners and attendings demonstrated moderate consistency between them when recommending resources for learners (rs=0.494, p<.01) and attendings (rs=0.491, p<.01). Conclusion: Using a gestalt-based rating system is neither reliable nor consistent when recommending OERs to learners and attending physicians. Learners’ gestalt ratings for recommending resources for other learners and attendings were especially unreliable. Our findings suggests the need for structured rating systems to rate OERs.

Objectives: Case series constitute a weak form of evidence for effectiveness of health technologies. However, for a variety of reasons, such studies may be included in health technology assessments. There are no clear criteria for assessing the quality of case series. We carried out an empirical investigation of the association between outcome frequency and methodological characteristics in a sample of health technology assessments.

Methods: Systematic reviews of functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, and percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic stable angina were identified as containing more than forty case series. Data were extracted by one reviewer and checked by a second on population characteristics, outcomes, and the following methodological features: sample size, prospective/retrospective approach, consecutive recruitment, multi- or single-center organization, length of follow-up, independence of outcome measurement, and date of publication. Association between methodological features and outcome were explored in univariate and multivariate analyses using parametric and nonparametric tests and robust regression or analysis of variance/analysis of covariance, as appropriate.

Results: Included reviews contained between forty-two and seventy-six case series studies, involving 5 to 172,283 participants. Reporting of methodological features was poor and limited the analyses. In general, we found little evidence of any association between methodological characteristics and outcome. Sample size is used as an inclusion criterion in many reviews of case series but was consistently shown to have no relationship to outcome in all analyses. A prospective approach was not associated with outcome. Insufficient data were available to explore consecutive recruitment. Mixed results were shown for length of follow-up, independence of outcome measurement, and publication date.

Conclusion: We found little evidence to support the use of many of the factors included in tools used for quality assessment of case series. Importantly, we found no relationship between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, for example, blinding of outcome measurement, but these examples were not shown consistently across the small number of examples studied. Further research into the determinants of quality in case series studies is required to support health technology assessment.