Systematic literature reviews are essential submissions to Prehospital and Disaster Medicine (PDM). They are an important element in the scientific process that allows for compilation of known evidence regarding an investigative topic or question. The systematic review is a means to establish what knowledge and ideas have been established on a topic and to determine the strengths and weaknesses of the evidence that is the basis of the knowledge.
Both disaster and prehospital health and medicine are newly developing fields of science. During the past three to four decades, an established scientific literature base has formed for the disciplines. This literature base has become appropriate for synthesizing to understand established research findings by means of structured literature reviews. The Editorial Board of PDM encourages the publication of high-quality, structured literature reviews. Unfortunately, many reviews are conducted without formal assessment of available literature or findings analysis. The objective of this discussion is to summarize types of literature review and the usual applications of each method.
One issue with literature review nomenclature is variations in the terminology used to describe the different formats used. For this discussion, the current common nomenclature is used. But, within the literature, one will find variation in terminology used to describe literature review processes.
Systematic Review
Systematic reviews provide a structured analysis of known evidence that can reliably determine health and medical practice standards and public policy. Systematic reviews define a topic and identify, summarize, and evaluate the findings of all well-designed research for that topic that is reported in the literature. This review method uses strict criteria designed to limit bias and emphasize scientific validity with the aim to produce an impartial analysis. Systematic reviews are the preferred method for rigorous literature review. A systematic review requires a focused objective, defined literature eligibility criteria (and exclusion criteria), a reproducible research protocol, a structured literature search to locate all eligible studies and reports, a scientific assessment of the included literature, and a systematic synthesis of the validity, flaws, and findings of the literature reviewed. This type of review requires attention to non-biased analysis of the literature by use of a defined scientific method, such as described within the PRISMA Statement.Reference Moher, Liberati, Tetzlaff and Altman1 Systematic reviews require minimization of error both in conduct of the review and within the literature included within the review. A systematic review should originate with a declared research protocol that is included in the publication of the review manuscript. Before initiating such a review, it is important that a researcher be familiar with the underlying literature available for the review topic and assure there is an adequate quantity of literature with scope and quality for review. As with any scientific study, a systematic review should be designed and described such that future investigators can reproduce the review. Reputable scientific journals require that for systematic reviews, the 27 item PRISMA Checklist be followed and submitted along with a manuscript.2 Important in the checklist is detailed descriptions of review methods and results.
Meta-Analysis (Quantitative and Qualitative)
Meta-analyses are designed to critically and systematically evaluate the evidence present for a disease, treatment, or health issue. Meta-analysis is a subset of systemic review and the same PRISMA Checklist used for systematic review should be followed in conducting a meta-analysis. 2 The same attention invested in systematic review is required for meta-analysis, with an additional requirement that data accumulated during a meta-analysis be homogeneous such that standard statistical methods or qualitative analysis can be applied to the pooled data. It has been shown that combining randomized and non-randomized studies into a single meta-analysis tends to show stronger treatment effects for non-randomized research.Reference Ioannidis, Haidich and Pappa 3 Therefore, meta-analyses of randomized studies is considered most appropriate. An advantage of meta-analyses over other forms of research is the ability to examine heterogeneity among key studies that are used to develop health and medical standards.Reference Greenland 4
Both quantitative and qualitative meta-analyses are thorough and rigidly structured in review of appropriate literature. Qualitative meta-analysis reviews qualitative literature to evaluate and interpret themes and concepts, often establishing primary and secondary population characteristics and ideas. Qualitative meta-analyses are also used to study cultural and environmental variations among populations. Quantitative meta-analyses review quantitative studies to improve objective analysis of pooled data, narrow or show flaws in reported data through pooled analysis, and assess validity of quantitative literature sources. When substantial, high-quality literature is available to appropriately pool data or information from that literature base, meta-analysis is preferred as a review method.
Cochrane Review
Cochrane Reviews are another subset of systematic reviews designed to investigate the effects of interventions for prevention, treatment, and rehabilitation in health care. Important for a Cochrane Review is that the review objective is designed to investigate a clearly formulated research question. Cochrane reviews follow rigid guidelines and defined research methods 5 that make the reviews highly respected. Cochrane reviews are ultimately designed to determine if there is conclusive evidence to support a specific treatment or concept. The Cochrane Community is an open group that supports cataloging Cochrane Reviews in an extensive library available to the public. The Cochrane Group provides online training for design and conduct of systematic reviews and provides guides and handbooks to assist researchers. 5
Scoping Review
Scoping reviews are designed to determine the volume of literature on a specific topic as well as an overview of the detail of the information regarding the topic.Reference Munn, Peters, Stern, Tufanaru, McArthur and Aromataris 6 While not designed to provide structured review and analysis of defined topics, the scoping method is used to map the available literature in a field of interest, to clarify concepts or definitions in the literature, to report on how research is conducted for a topic or in a field, to identify characteristics or factors related to a concept, as a preliminary assessment before a systematic review, and to identify and analyze knowledge gaps for a topic or field.Reference Munn, Peters, Stern, Tufanaru, McArthur and Aromataris 6 Scoping reviews differ from systematic reviews in purpose (as described above). Systematic reviews consolidate and narrow concepts or findings within the literature, while scoping reviews are focused toward clarification of a concept or theory. Until recently, there was lack of consensus for best methods in conducting scoping reviews. While general agreement is that scoping reviews require an a-priori review protocol, a search strategy, and structured data extraction technique, there has been variation in published literature labeled as scoping review. To address this issue, the PRISMA Group has developed a checklist similar to that used for systematic reviews.Reference Tricco, Lillie and Zarin 7
Narrative Review
Often called a “traditional literature review,” narrative literature reviews are unstructured reviews of literature used to establish a theory and the context for or justify a research focus. Narrative reviews are commonly found in the introduction or background section of a formal manuscript and used to introduce and justify research described in that manuscript. Narrative reviews may also be published as stand-alone papers focused on theory or frames of thought regarding the literature on a particular topic (Theoretical Literature Review). Narrative reviews may also outline methods or research design that is used for a particular study topic (Methodological Literature Review), or provide a historical evaluation of the development of theory or concepts for a topic (Historical Literature Review). Narrative reviews may be conducted using search words within standard literature databases, but do not describe the specific methods for selecting and reviewing the literature retrieved. Narrative reviews have many of the characteristics of scoping reviews, but lack a-priori review protocols or a pre-defined analysis approach. Narrative reviews often include an author’s assumptions and biases and generally cannot be replicated (as with systematic reviews).
Sub-forms of the narrative review method are prevalent in the health and medicine literature.Reference Grant and Booth 8 Listed below are three of the more common of these types of reviews.
Critical Review
Critical reviews don’t include the rigorous study design of a systematic review and are a form of narrative review. The objective of a critical review is to develop perspectives on a research topic using available literature. Often, this form of review is limited to recent literature, but may also be used to show changes in perspective over time periods.
Conceptual Review
The conceptual review is another subset of narrative review methodology. Conceptual reviews do not include a literature search protocol and systematic data extraction techniques. This type of review is common within editorials and concept papers. Conceptual reviews evaluate the general consensus of the literature on a given research topic and explore how this understanding was reached.
They are designed to show the current understanding of a topic and suggest if a better understanding or consensus is needed. As with scoping reviews, conceptual reviews have potential to show gaps in the knowledge base for a specified research area.
State-of-the Art Review
This sub-form of narrative review is used to focus on recent research and describe what is currently known and agreed upon for a review topic. Common to this form of review is a discussion of areas of agreement and disagreement for the review topic.
Narrative reviews which are also identified as traditional reviews may be given names other than those identified in this discussion. “Standard review,” “comprehensive review,” “snow-ball review,” “bibliographic review,” and other similar terms are occasionally used to identify what is actually a basic narrative review. As a point, “comprehensive review” was an older term for systematic review, but the term systematic review has replaced this older terminology due to the misuse of “comprehensive” by authors in identifying what is an actual narrative review.
Summary
In summary, as with original research, there is a hierarchy for strength of results from different review methods (Table 1). At the time of writing this Editorial, a search of the PubMed 9 database showed 96,807 published papers under the search term “disaster” and 11,991 under the search term “prehospital.” With a robust and broad literature in prehospital and disaster medicine, high-quality literature reviews designed to condense the knowledge base are a logical step to advance the science in these two general fields.
Table 1. Forms of Literature Review and Meta-Analyses
