Objectives: Evidence-based guideline committees are multidisciplinary and explicitly consider the existing evidence. They are thus in an ideal position to identify research gaps. However, gaps have not been systematically identified through guidelines. We pilot tested a method to systematically identify and classify gaps from evidence-based guidelines.

Methods: We reviewed all evidence-based guidelines published by the Cystic Fibrosis Foundation. We identified research gaps as topics for which there was insufficient evidence (recommendations were not made or consensus recommendations were made) and topics specified as needing further research. We characterized gaps using a standard framework and classified them by type of management issue, specificity of target population, and age of target population.

Results: We identified sixty-two research gaps in five guidelines (mean = 12.4/guidelines document). While thirteen gaps were topics specified as needing further research, most (n = 49) were topics with insufficient evidence. Of these forty-nine, recommendations were not made for twenty-two topics while consensus recommendations were made for twenty-seven topics. Most gaps were issues of comparative effectiveness (44/62), addressed the general cystic fibrosis population (40/62), and were specific to infants (33/62). Relevant comparisons and outcomes were explicitly stated for only 7 percent and 16 percent of gaps respectively.

Conclusions: Almost 80 percent of the gaps were not topics identified as future research needs in the guidelines documents but rather were topics with insufficient evidence for making recommendations. Although we used cystic fibrosis in the United States as an example, the method we developed could be applied in other settings, including other countries and for different diseases.

Objective: We have used multidisciplinary expert panels to assess the health benefits from two different emergency medical service programs in Norway. This gave the opportunity to study the reliability of the expert panel method.

Methods: Two panels assessed case reports for 18 children, and two other panels assessed case reports for 64 adult patients. The assessments of each case report were compared. These assessments were also compared with assessments of the same case reports, done by the same panels 1 and 9 years earlier.

Results: Two different panels agreed on the benefit/no benefit conclusion in at least 75% of the patients, both for children and adult patients (kappa 0.88–0.50). For groups of patients assessed to have some health benefit, the magnitude of the benefit estimates differed by 25% between the panels. When the same panels assessed the same patient groups twice, 1 and 9 years apart, their estimates of total benefit differed up to 30%. However, estimates for single patients, as well as estimates from single panel members, varied considerably more.

Conclusions: Use of multidisciplinary expert panels is a useful method for estimating health benefits on program level or for groups of patients. But assessments from single panelists, and for single patients may be seriously biased.

Objectives: To assess the extent to which abstracts of methodological research, initially presented at meetings on systematic reviews, have gone on to be published as full articles.

Methods: Full publication was assessed in three ways: a search was carried out of The Cochrane Library; a search was conducted using MEDLINE; and a questionnaire was sent to the contact author of each abstract.

Results: Approximately half of the abstracts had not been, or were unlikely ever to be, published in full.

Conclusions: The rate of full publication of abstracts related to the methodology of systematic reviews seems similar to that for randomized trials.