Background
Systematic reviews (SRs) of primary treatment and diagnostic studies form the basis of health technology assessments (HTAs) and evidence-based clinical practice guidelines. They thus inform health policy and clinical decision making, ensuring reliable, unbiased and reproducible health care decisions both on the individual and population-based level (Reference O’Rourke, Oortwijn and Schuller1).
SRs are usually conducted in a professional environment in specialized research teams working at HTA agencies, universities, and international organizations, also in collaboration with each other. Examples of HTA agencies are the UK National Institute for Health and Care Excellence (NICE), the French Haute Autorité de Santé (HAS), the German Institute for Quality and Efficiency in Health Care (IQWiG), and the Canadian Drug and Health Technology Agency (CADTH). Examples of universities are the Specialist Unit for Review Evidence at Cardiff University (UK), the Department of Health Research Methods, Evidence & Impact at McMaster University (Canada), and the Institute for Evidence-Based Healthcare at Bond University (Australia). Examples of international organizations are the Cochrane Collaboration and JBI.
The above organizations have in common that SRs are conducted by a team of researchers with specific areas of expertise, as no single researcher would be able to cover all the areas of expertise required. An SR team should routinely include methodological experts such as statisticians and information specialists to ensure the production of high-quality, that is, accurate and reproducible, SRs (Reference Metzendorf and Featherstone2;Reference Uttley and Montgomery3). Depending on the topic of the SR, other experts such as ethicists, epidemiologists, and health economists may also be involved.
However, although the call for routine involvement of these experts in SRs is supported by research findings, inadequate resources often hamper the implementation in small research groups (Reference Waffenschmidt, van Amsterdam-Lunze and Gomez4). This especially applies to information specialists – even though it is a “core methodological recommendation” to involve them, they “tend to fly somewhat under the radar” (Reference Metzendorf and Featherstone2).
In the guideline for preparing the implementation of the HTA Regulation of the European Parliament (5), the European HTA network EUnetHTA requires that one HTA team member has expertise in information retrieval and one has expertise in statistical analysis (6). These requirements for European HTA were the rationale for our commentary on the role of information specialists and statisticians in HTA to provide background information on these professions and to highlight the importance of their involvement.
The commentary describes the qualifications required for information specialists and statisticians involved in SRs, as well as their tasks, the methodological challenges they face, and potential future areas of involvement.
Tasks and qualifications
Information Specialists
Information specialists often work alone, as professional teams usually exist only in larger organizations. The importance of collaborating with peers has resulted in the establishment of information specialist networks such as the Cochrane Information Specialist Group (7) and the HTAi Interest Group Information Retrieval (8).
The main responsibility of information specialists is to provide the evidence base for SRs; in detail, the tasks include (Reference Waffenschmidt, van Amsterdam-Lunze and Gomez4;Reference Littlewood, Fenton and Bridges9–Reference Spencer and Eldredge12):
-
• being involved in the formulation of the research question,
-
• selecting the type of information retrieval (e.g., comprehensive or focused), information sources (e.g., bibliographic databases, study registries), search techniques (e.g., Boolean search, citation searching), and study selection tools (e.g., EndNote, EPPI Reviewer) to be used,
-
• developing and peer-reviewing search strategies,
-
• conducting searches and update searches,
-
• reporting information retrieval methods and results,
-
• performing librarian tasks (e.g., full-text ordering, subscription management, reference list checks, reference and repository management, provision of advice on copyright and other questions relating to scholarly communications, dissemination of completed reports), and
-
• further developing information retrieval methods (e.g., validation of different search techniques, including automation and digitalization; development of search filters).
Qualifications
Information specialists come from a variety of backgrounds with different types of training and levels of experience (Reference McGowan and Sampson11;Reference Booth and Beecroft13;Reference Tran14). Only a few countries, such as the United Kingdom, the United States, and Canada, offer formal university degrees (e.g., Master of Library and Information Science (15)). Information specialists from other countries such as Germany sometimes also hold a degree related to health care or other sciences, but have no formal university degree in library and information science (Reference Knüttel, Krause, Semmler-Schmetz, Reimann and Metzendorf16). However, a variety of training options exist to acquire the skills needed. In our opinion, the following minimum requirements should be met to qualify as an information specialist in an SR team:
-
• A master’s degree in library and information science or another health-related or scientific qualification.
-
• Several years of experience in information retrieval/evidence-based medicine/health science.
-
• An understanding of health care.
-
• Evidence of continued education and training in information retrieval methods.
Statisticians
Statisticians work in similar settings as information specialists (organizational group or alone). Various networking options are given, for example, by working groups within national and international societies such as the International Biometric Society (IBS), which contains a large number of countries, the International Society for Clinical Biostatistics (ISCB), and other statistical networks such as the Statistical Methods Group of the Cochrane Collaboration.
During the twentieth century, there was an explosive growth in the development of statistical methodology, leading to a strong increase in the involvement of statisticians in biomedical and epidemiological research (Reference Ellenberg17). Statisticians are nowadays involved from the planning phase of a research project to the data analysis and finally to the interpretation and publication of the results. Statisticians are coresponsible for an appropriate and efficient study design, an adequate data analysis, and a correct interpretation of the findings (Reference Zapf, Huebner, Rauch and Kieser18;Reference Zapf, Rauch and Kieser19).
Consequently, international regulatory guidelines for conducting clinical trials for drug approval require that statistical expertise should be utilized throughout all stages of a trial and that a “qualified statistician” takes responsibility for all statistical aspects (Reference Zapf, Huebner, Rauch and Kieser18;Reference Gerlinger, Edler and Friede20). As in clinical trials, a statistician should also be involved in all stages of an SR and is responsible for the following tasks (Reference Zapf, Rauch and Kieser19;Reference Higgins, Li, Deeks, JPT, Thomas, Chandler, Cumpston, Li, Page and Welch21–Reference Schünemann, Higgins, Vist, Higgins, Thomas, Chandler, Cumpston, Li, Page and Welch25):
-
• Assessing the risk of bias of the results of the included studies.
-
• Choosing appropriate effect measures and summarizing the study results by means of appropriate methods for evidence synthesis (e.g., meta-analysis, network meta-analysis).
-
• Exploring heterogeneity.
-
• Assessing the assumptions of the chosen method for evidence synthesis.
-
• Performing required subgroup and sensitivity analyses.
-
• Presenting the SR results by means of appropriate graphs (e.g., forest, funnel, and network plots) and tables (e.g., summary of findings tables).
-
• Interpreting the SR results.
Qualifications
Although graduate programmes are far more common for statisticians than for information specialists, there is no unique way of becoming a statistician and the educational systems for applied statistics vary widely across countries (Reference Zapf, Huebner, Rauch and Kieser18). The ISCB drafted a reflection paper to discuss what constitutes a “qualified statistician” within the framework of drug approval (Reference Gerlinger, Edler and Friede20). According to this paper, the following requirements should be fulfilled:
-
• Typical career paths are given by, but are not limited to, special university degree programmes such as Master of Science in Medical Biometry and certificates such as the certificate “Biometry in Medicine” of the German Region of IBS (Reference Feldmann26).
-
• The academic qualification should be at least a master’s degree in statistics or an equivalent degree.
-
• Several years of practical experience in the planning, analysis, and reporting of clinical trials, including methods of computational statistics and software, are required.
-
• Continued education and training, for example, via attending relevant conferences and workshops.
Equivalent requirements should be fulfilled for statisticians involved in SRs to ensure their high quality. In our opinion, the following minimum requirements should be met to qualify as a statistician in an SR team:
-
• A master’s degree in statistics or an equivalent qualification (e.g., epidemiology with further education in clinical biostatistics).
-
• Several years of practical experience in the planning, analysis, and reporting of clinical or epidemiological trials.
-
• Several years of practical experience in computational statistics and statistical software.
-
• Practical experience in methods of evidence-based medicine, meta-analysis, and other forms of evidence synthesis.
-
• Continued education and training in new methods for evidence synthesis.
-
• An understanding of health care.
Key arguments
Methodological Challenges in SRs
Besides a massive increase in the amount of scientific literature that needs to identified and screened, there has been a strong expansion of statistical and information retrieval methods, which are also becoming more and more complex, thus making it difficult to keep up to date and impossible for a single researcher to cover (Reference Lefebvre, Glanville, Wieland, Coles and Weightman27). The first challenge in an SR emerges early in the planning phase, as it is difficult to judge how complex the research question could become and what hurdles could arise during the course of the project; methodological expertise is thus already essential at this early stage. The next challenge is the choice of the appropriate method for each case of use, including the consideration of resources. At this stage too, the involvement of information specialists and statisticians is essential and helps to minimize bias and produce valid robust results as well as to reduce research waste (Reference Waffenschmidt, van Amsterdam-Lunze and Gomez4).
Further Involvement of Information Specialists and Statisticians in SRs
After the establishment of the Cochrane Collaboration in the 1990s, the routine involvement of information specialists and statisticians was soon specified on the organizational level (e.g., introduction of the first “Trials Search Coordinators” in Cochrane). Similar requirements are currently being implemented within the planned harmonization of European HTA from 2025 onwards: As noted above, the European HTA network EUnetHTA requires both expertise in information retrieval and in statistical analysis (6). In our opinion, these requirements should also apply to other areas such as scientific journals (peer reviews of SRs), as well as assessment and reporting guidelines for SRs (inclusion of items in AMSTAR (Reference Shea, Reeves and Wells28) and PRISMA checklists (Reference Page, McKenzie and Bossuyt29;Reference Rethlefsen, Kirtley and Waffenschmidt30) specifying the methodological expertise required).
Coping with Limited Resources
In resource-limited settings, it will not always be possible to include methodological experts as an integral part of an SR team. However, other solutions have been developed to make use of their expertise. For example, a network of information specialists has been established within EUnetHTA (Joint Action 3) to provide input to individual assessments conducted by smaller HTA agencies (Reference Waffenschmidt, van Amsterdam-Lunze and Gomez4). A similar network of statisticians has also been established. In addition, many HTA agencies that do not directly employ methodological experts seek their advice on a freelance basis or use support services such as libraries with information management services. Online consultations can be provided to facilitate access to external expertise.
PhD students or other early stage researchers at universities usually have little experience in conducting SRs. Information specialists and statisticians should therefore be directly involved in researcher development, for example through workshops. In addition, advice on SRs should be available on an ad hoc basis. The implementation of these measures would avoid bias and the waste of resources, thus enabling the production of efficiently produced SRs of high quality.
Conclusion
SRs are becoming more and more complex to conduct and methodological experts such as information specialists and statisticians should routinely be involved right from the start of a project. They ensure the high methodological quality of an SR by applying up-to-date and appropriate methods, resulting in unbiased and thus reliable results. This increases the trustworthiness of SRs as the basis for informed health policy and clinical decision making.
Acknowledgments
The authors thank Natalie McGauran for editorial support.
Funding statement
This research received no specific grant from any funding agency, commercial, or not-for-profit sectors.
Competing interest
The authors declare none.
