In the theme section on information retrieval for health technology assessment (HTA), we are pleased to present a series of high-quality articles. The articles address various approaches to HTA searching and information management. Information retrieval and management is an integral part of the HTA review process. Those conducting searches must, therefore, understand the nature of the different domains of HTA, know what kind of information is needed for each domain, and know how to find it. These articles describe developments in HTA searching and other information-related activities.

Objectives: Bowel cancer is the second most common cancer in England and Wales, accounting for approximately 13,000 deaths per year. Economic evaluations and national guidance have been produced for individual treatments for bowel cancer. However, it has been suggested that Primary Care Trusts develop program budgeting or equivalent methodology demonstrating a whole system approach to investment and disinvestment. The objective of this study was to provide a baseline framework for considering a whole system approach to estimate the direct costs of bowel cancer services provided by the National Health Service (NHS) in England.

Methods: A treatment pathway, developed in 2005, was used to construct a service pathway model to estimate the direct cost of bowel cancer services in England.

Results: The service pathway model estimated the direct cost of bowel cancer services to the NHS to be in excess of £1 billion in 2005. Thirty-five percent of the cost is attributable to the screening and testing of patients with suspected bowel cancer, subsequently diagnosed as cancer-free.

Conclusions: This study is believed to be the most comprehensive attempt to identify the direct cost of managing bowel cancer services in England. The approach adopted could be useful to assist local decision makers in identifying those aspects of the pathway that are most uncertain in terms of their cost-effectiveness and as a basis to explore the implications of re-allocated resources. Research recommendations include the need for detailed costs on surgical procedures, high-risk patients and the utilization of the methods used in this study across other cancers.

Objectives: The aim of this study was to assess empirically the social and ethical risks associated with implantable radio-frequency identification (RFID) devices.

Methods: Qualitative research included observational studies in twenty-three U.S. hospitals that have implemented new patient identification systems and eighty semi-structured interviews about the social and ethical implications of new patient identification systems, including RFID implants.

Results: The study identified three primary social and ethical risks associated with RFID implants: (i) unfair prioritization of patients based on their participation in the system, (ii) diminished trust of patients by care providers, and (iii) endangerment of patients who misunderstand the capabilities of the systems.

Conclusions: RFID implants may aggravate inequalities in access to care without any clear health benefits. This research underscores the importance of critically evaluating new healthcare technologies from the perspective of both normative ethics and empirical ethics.

Objectives: Our objective was to document the pattern of mobile phone usage by medical staff in a hospital setting, and to explore any perceived benefits (such as improved communications) associated with mobile phones.

Methods: This cross-sectional survey was conducted in Waterford Regional Hospital, Ireland, where bleep is the official system of communication. All non-consultant hospital doctors, of medical disciplines only, were asked to participate. The questionnaire was designed to explore the pattern and different aspects of mobile phone usage.

Results: At the time of study, there were sixty medical junior doctors, and the response rate was 100 percent. All participants used mobile phones while at work, and also for hospital-related work. For 98.3 percent the mobile phone was their main mode of communication while in the hospital. Sixty-two percent (n = 37) made 6–10 calls daily purely for work-related business, and this comprised of ≥80 percent of their daily usage of mobile phones. For 98 percent of participants, most phone calls were work-related. Regarding reasons for using mobile phones, all reported that using mobile phone is quicker for communication.

Conclusions: Mobile phone usage is very common among the medical personnel, and this is regarded as a more efficient means of communication for mobile staff than the hospital paging system.

Objectives: The aim of this study was to examine open-source electronic health record (EHR) software to determine their level of functionalities according to the International Organization for Standardization (ISO) standards.

Methods: ISO standards were used as a guideline to determine and describe the reference architecture and functionalities of a standard electronic health record system as well the environmental context for which the software has been built. Twelve open-source EHR systems were selected and evaluated according to two-dimensional criteria based on ISO/TS 18308:2004 functional requirements and ISO/TR 20514:2005 context of the EHR system.

Results: Open EHR software programs mostly fulfill structural, procedural, evolutional, and medicolegal requirements at the minimal and full functionality levels. Communication, privacy, and security requirements are accomplished in less than 23 percent of the cases, mainly at minimal functional level. Ethical, cultural, and consumer requirements still need to be fulfilled by free and open-source EHR applications.

Conclusions: Most analyzed systems had several functional limitations. Nevertheless, especially for clinicians and decision makers in developing countries, open-source EHR systems are an option. The limited functionalities are likely to become requirements for further releases of open-source EHR systems.

Objectives: The aim of this study was to compare the clinical effectiveness and cost of percutaneous radiofrequency ablation (PRFA) and surgical resection (SRS) for the management of early stage Hepatocellular Carcinoma.

Methods: A systematic literature search of articles in English, French, and Chinese was performed using online databases. Only articles with patients classified as Child-Pugh Class A or B, with tumor size <5 cm were included. A meta-analysis was carried out to estimate the survival rate and disease-free survival rate following PRFA or SRS treatments. The cost of each treatment was estimated from the third party perspective. Univariate sensitivity analyses were used to study the relative importance of each component cost.

Results: We identified six studies (one randomized controlled trial (RCT) and five comparative cohort studies) meeting our inclusion criteria. There is good evidence that among Child-Pugh A patients for whom both SRS and PRFA are available options, survival rates following either procedure are comparable, while complications are more frequent and hospitalization longer following SRS. The evidence concerning recurrence rates and disease-free survival is less clear. Whereas the RCT indicates comparable outcomes with either procedure up to 3 years, the results of five cohort studies (with possible selection bias), particularly those with a mix of Child-Pugh A and B patients, favor the surgical option. SRS, costs approximately Canadian $8,275 more per case than PRFA.

Conclusions: Continuing doubts on this issue can only be resolved by a substantial RCT. Meanwhile, for early stage HCC patients classified as Child-Pugh A, who despite a possibly higher recurrence rate, prefer the less invasive PRFA to open surgery with its attendant risks, there is sufficient evidence to justify such a choice. For those classified as Child-Pugh (B) it is possible that overall survival is equally good with PRFA, but the evidence is less certain.

Objectives: We describe, in general, the principles used in priority setting and, in particular, policy processes and decision making in Norway.

Methods: A newly established council for setting priorities in health care is presented to illustrate how health technology assessments (HTAs) can support national advisors in complex priority-setting processes.

Results and Conclusions: Setting priorities in health care is a complex task. Careful thinking is, therefore, required in determining the components of a system for priority-setting. Based on recent Norwegian experiences, we believe that the following generic parts may provide some of the solution: a common set of values; an organizational structure made up of key stakeholders; supporting mechanisms in the form of HTA organizations and documented best evidence; and loyalty to decisions by stakeholders responsible for implementing national policies.

Objectives: The objective of this study is to profile the health technology assessments (HTAs) produced in Canada and other selected countries and assess their potential to inform policy making about health systems in jurisdictions other than the ones for which they were produced, and to develop and pilot test prototypes for packaging and assessing the relevance of HTAs for health system managers and policy makers.

Methods: We compiled an inventory of all HTAs that were produced by nine HTA agencies between September 2003 and August 2006; coded the title and abstract of each HTA according to the technologies assessed, methods used, and whether or not context-specific actionable messages were provided; developed a prototype for a structured, decision-relevant HTA summary and for a relevance-assessment form; and pilot-tested the prototypes using semistructured telephone interviews with a purposive sample of Canadian healthcare managers and policy makers.

Results: Our review of the 223 HTAs identified that: (i) 44 HTAs addressed health system arrangements (20 percent); (ii) 205 incorporated a systematic review (92 percent), whereas only 12 incorporated a sociopolitical assessment using explicit methods (5 percent); and (iii) 50 contained context-specific actionable messages (22 percent). Our interviews identified significant support for both the general idea of an HTA summary and the prototype's specific elements, but mixed views about using peer assessments of relevance.

Conclusions: Those involved in supporting the use of HTAs in policy making about health systems may wish to produce structured decision-relevant summaries for their systematic review-containing HTAs to increase the prospects for their HTAs being used outside the jurisdiction for which they were produced.

Objectives: The aim of this study was to present and illustrate an instrument to measure the level of innovation at country level.

Methods: The data used are the Organisation for Economic Co-operation and Development (OECD) health data 2009, in particular the information on use of medical technology. Two composite scales expressing a relative level of adoption of innovations in health care are regressed, using multilevel regression analysis, on country characteristics. The country characteristics are selected as proxies on availability or scarcity of resources in a country. We expect that scarcity will promote adoption of innovations that enhance efficiency, and that availability of resources will promote advanced, expensive innovations.

Results: Two scales were constructed. One scale indicates the use of efficiency-enhancing innovations (day case treatment), and the other scale indicates availability of advanced technical innovations. The application of day case treatment is significantly associated with education level (+), the ratio of people aged 15–64 versus younger and older people (+) and the number of hospital beds (−). Availability of advanced medical devices are associated with the expenditure on health (+), demographic dependency (−), number of hospital beds (+), and the annual reduction of hospital beds (−).

Conclusions: Diffusion of innovations is influenced by characteristics of the country and of the healthcare system; fewer resources encourage diffusion of innovations that enhance efficiency and more resources encourage diffusion of complex, expensive devices. This indicates that decisions by healthcare professionals on which innovation to adopt is embedded in a context that is influenced and shaped by the availability of resources on macro level.

Objectives: A set of fifteen key principles (KP) has been recently proposed to guide decisions on the structure of HTA programs, the methods of HTA, the processes for conducting HTA and the use of HTA findings in decision-making. The objective of this research is to explore whether these KPs are relevant and useful in Latin America (LA), and to what extent they are being applied.

Methods: A Web-based survey was sent to 11,792 HTA researchers and users in LA to explore the perceived relevance of each KP, its current level of application and the gap between these two.

Results: We received 1,142 responses from nineteen LA countries (9.7 percent response rate). The subgroup of KP related to Methods and to the Use of HTA received the higher mean scores in the relevance scale (9.00 and 8.94). Level of current application scored low in all KP (3.2 to 4.9). Higher gaps were observed in principles related to the use of HTA in decision making and to the processes for conducting HTA. Countries with more developed HTA showed higher scores in the degree of current application (5.3 versus 3.4, p < .01) and lower gaps (3.84 versus 5.21, p < .01). Researchers, compared with research users, scored the relevance of the KPs higher.

Conclusions: KPs seem to be very relevant to most HTA researchers and users in LA. However, the current level of application was considered uniformly poor. Higher gaps were observed in KPs related to the link between HTA and decision making, highlighting one of the major challenges for the countries in the region.

In the article “Are key principles for improved health technology assessment supported and used by health technology assessment organizations?” by Neumann et al. (3), the authors set out to evaluate the quality of processes used by health technology assessment (HTA) programs based on a set of principles they published in this journal previously (2). We believe that portions of the current article, as they relate to our own organization, are misleading or inaccurate, and welcome this opportunity to correct the inaccuracies.

Thank you for the opportunity to respond to the letter by Gibson and Little. The authors raise several points that require a response. First, they claim that we inaccurately characterized the DERP/Washington State Medicaid agency, and did not appreciate that the DERP and the Washington State Medicaid agency have different missions. On the contrary, however, we fully recognized that DERP's mission is to conduct systematic reviews and not to make recommendations, whereas Washington State uses the reviews in making recommendations for their Medicaid enrollees. We specifically noted in the study that “Washington Medicaid is one of fourteen participants in the DERP. DERP researchers conduct health technology assessments for drug classes. Participants in the DERP, such as the Washington Medicaid program, retain local authority for interpreting DERP reports and for decision making regarding which drugs to pay for.” We chose to analyze DERP/Washington State as a single entity for our exercise because we were interested in analyzing the link between the HTA conducted and the decisions that follow them.

Objectives: The aim of this study is to review briefly different methods for determining the optimal retrieval of studies for inclusion in a health technology assessment (HTA) report.

Methods: This study reviews the methodology literature related to specific methods for evaluating yield from literature searching strategies and for deciding whether to continue or desist in the searching process.

Results: Eight different methods were identified. These include using the Capture–recapture technique; obtaining Feedback from the commissioner of the HTA report; seeking the Disconfirming case; undertaking comparison against a known Gold standard; evaluating retrieval of Known items; recognizing the Law of diminishing returns, specifying a priori Stopping rules, and identifying a point of Theoretical saturation.

Conclusions: While this study identified a variety of possible methods, there has been very little formal evaluation of the specific strengths and weaknesses of the different techniques. The author proposes an evaluation agenda drawing on an examination of existing data together with exploration of the specific impact of missing relevant studies.

Objectives and Methods: Health technology assessment (HTA) often requires the identification and review of economic evaluations and models. This study surveys the available specific and general resources to search to identify economic evaluations. It also provides information on efficient searching of those resources and comments on the current evidence-base.

Results: Published checklists recommend searching for economic evaluations in specific information resources which collect economic evaluations such as NHS EED and HEED, followed by top-up searches of large biomedical bibliographic databases (such as MEDLINE and EMBASE). Other resources such as the HTA and DARE databases can yield reports of economic evaluations. Searches within NHS EED and HEED can be made more efficient by using database-specific search options. Searches within large biomedical databases such as MEDLINE and EMBASE require the use of economic search terms called search filters. Search filters are highly sensitive, retrieving most economic evaluations, but suffer from low precision returning many irrelevant records which need to be assessed.

Conclusions: It is relatively easy to identify rapidly a high proportion of economic evaluations but more research is required to improve the efficiency of this process. There are key high yield resources to search but more evidence is required on their overlap and unique contribution to searches. The value of other resources, particularly those providing access to gray literature, should be explored. Research into efficient retrieval requires clear definitions of economic evaluations to allow comparison across studies.

Background: Comprehensive health technology assessments (HTAs) include thorough reflections on ethical issues associated with health technologies, their use, and value-based decisions in the assessment process. As methods of information retrieval for effectiveness assessments are not applicable to information retrieval on ethical issues, a specific methodological approach is necessary.

Objectives: In the absence of existing adapted methods, our objective was to develop a methodological approach for the systematic retrieval of information on ethical issues related to health technologies.

Method and Results: A literature search was conducted to verify the non-existence of published comprehensive methodological approaches for the information retrieval on ethical issues for HTAs, and resulted in no hits. We, therefore, developed a step-by-step workflow following the workflow of information retrieval for effectiveness assessments: Step 1: Translation of the search question using the PICO scheme and additional components. Step 2: Concept building by modeling and linking search components. Step 3: Identification of synonyms in all relevant languages. Step 4: Selection of relevant information sources. Step 5: Design of search strategies for bibliographic databases. Step 6: Execution of search strategies and information seeking, including hand-searching. Step 7: Saving of retrieval results and standardized reporting of the process and results. Step 8: Final quality check and calculation of precision and recall.

Conclusions: Systematic searching for information on ethical issues related to health technologies can be performed following the common retrieval workflow for effectiveness assessments, but should be performed separately applying adapted procedures and search terms on ethical issues relevant to the research question.

Background: Information retrieval (IR) in health technology assessment (HTA) calls for transparency and reproducibility, but common practice in the documentation and presentation of this process is inadequate in fulfilling this demand.

Objectives: Our objective is to promote good IR practice by presenting the conceptualization of retrieval and transcription readable to non-information specialists, and reporting of effectively processed search strategies.

Methods: We performed a comprehensive database search (04/2010) to synthesize the current state-of-the-art. We then developed graphical and tabular presentation methods and tested their feasibility on existing research questions and defined recommendations.

Results: No generally accepted standard of reporting of IR in HTA exists. We, therefore, developed templates for presenting the retrieval conceptualization, database selection, and additional hand-searching as well as for presenting search histories of complex and lengthy search strategies. No single template fits all conceptualizations, but some can be applied to most processes. Database interface providers report queries as entered, not as they are actually processed. In PubMed®, the huge difference between entered and processed query is shown in “Details.” Quality control and evaluation of search strategies using a validated tool such as the PRESS checklist is suboptimal when only entry-query based search histories are applied.

Conclusions: Moving toward an internationally accepted IR reporting standard calls for advances in common reporting practices. Comprehensive, process-based reporting and presentation would make IR more understandable to others than information specialists and facilitate quality control.

Objectives: The aim of this study was to assess systematically the scope of evidence and purposes for which evidence is used in decision-analytic models of cost-effectiveness and to assess the implications for search methods.

Methods: A content analysis of published reports of models was undertaken. Details of cited sources were extracted and categorized according to three dimensions; type of information provided by the evidence, type of source from which the evidence was drawn and type of modeling activity supported by the evidence. The analysis was used to generate a classification of evidence. Relationships within and between the categories within the classification were sought and the implications for searching considered.

Results: The classification generated fourteen types of information, seven types of sources of evidence and five modeling activities supported by evidence. A broad range of evidence was identified drawn from a diverse range of sources including both research-based and non–research-based sources. The use of evidence was not restricted to the population of model parameters but was used to inform the development of the modeling framework and to justify the analytical and methodological approach.

Conclusions: Decision-analytic models use evidence to support all aspects of model development. The classification of evidence defines in depth the role of evidence in modeling. It can be used to inform the systematic identification of evidence.

Objectives: The aim of this study was to analyze the information needs of Spanish health technology assessment (HTA) agencies and units to facilitate access to the resources they require to substantiate their reports.

Methods: A questionnaire was designed and distributed among HTA bodies to ascertain the actual situation of subscriptions to information resources and what information specialists from these bodies considered would be the ideal subscription situation. Their information needs were then studied, and the resources that best met these needs were put forward. Following this definition, a subscriptions policy was adopted with suppliers and publishers.

Results: The survey showed that HTA bodies share a minimum of core subscriptions that includes open sources (MEDLINE, DARE) and sources that the government subscribes to for the health community (ISI Web of Science, Cochrane Library Plus). There was no common approach to determining which databases to subscribe to (UpToDate, EMBASE, Ovid EBMR, CINAHL, or ECRI). After identifying the information needs, a list of resources was proposed that would best cover these needs and, of these, subscription to the following was proposed: Scopus, Ovid EBMR, Clinical Evidence, DynaMed, ECRI, and Hayes.

Conclusions: There are differences in the way that HTA agencies and units access the different resources of biomedical information. Combined subscription to several resources for documentation services was suggested as a way of resolving these differences.

Background: The Centre for Reviews and Dissemination (CRD) funded through the United Kingdom National Institute for Health Research undertakes systematic reviews evaluating the research evidence on health and public health questions of national and international importance. CRD is involved in methods research and produces internationally accepted guidelines for undertaking systematic reviews. A core product of the Centre are the CRD databases; a key resource for health professionals, policy makers, and researchers around the world. Research evidence evaluating the effects and cost-effectiveness of healthcare interventions is growing year on year, and can be difficult and time consuming to identify and appraise. The databases assist decision makers by systematically identifying and critically describing systematic reviews and economic evaluations, appraising their quality and highlighting their relative strengths and weaknesses; and providing a comprehensive listing of ongoing and published health technology assessments.

Methods and Results: This article briefly outlines the processes behind each of the three databases, gives details of the records, the information they contain and the value added by CRD. An overview of the new public interface for the CRD databases to be launched later this year is given. Improvements and innovations to supplement existing functions are listed, including enhanced and new search options, the addition of quick links to relevant material from within individual records, an integrated PubMed search and the ability to select personal preferences.

Conclusions: CRD's databases provide free access to an important knowledge base and are used widely to underpin evidence informed healthcare decisions in the United Kingdom and internationally.