Objectives: Many of the currently used health technologies have never been systematically assessed or are misused, overused or superseded. Therefore, they may be ineffective. Active identification of ineffectiveness in health care is gaining importance to facilitate best care for patients and optimal use of limited resources. The present research analyzed processes and experiences of programs for identifying ineffective health technologies. The goal of this study was to elucidate factors that facilitate implementation.

Methods: Based on an overview article, a systematic literature search and unsystematic hand-search were conducted. Further information was gained from international experts.

Results: Seven programs were identified that include identification, prioritization and assessment of ineffective health technologies and dissemination of recommendations. The programs are quite similar regarding their goals, target groups and criteria for identification and prioritization. Outputs, mainly HTA reports or lists, are mostly disseminated by means of the internet. Top–down and bottom–up programs both have benefits in terms of implementation of recommendations, either as binding guidelines and decisions or as nonbinding information for physicians and other stakeholders. Crucial facilitators of implementation are political will, transparent processes and broad stakeholder involvement focusing on physicians.

Conclusions: All programs can improve the quality of health care and enable cost reduction in supportive surrounding conditions. Physicians and patients must be continuously involved in the process of evaluating health technologies. Additionally, decision makers must support programs and translate recommendations into concrete actions.

Objectives: The aim of this study was to identify and rank the sources for the detection of potentially obsolete technologies (POTs).

Methods: A specific questionnaire related to the search strategies and sources used for the identification of POTs and also for ineffective, inefficient or harmful health technologies was sent to the Health Technology Assessment International's Information Resources Group (HTAi-IRG) group. With the obtained information and taking into account the sources used for the identification of new and emerging technologies, a second questionnaire was elaborated and sent to EuroScan and International Network of Agencies for Health Technology Assessment (INAHTA) members, who had to select and score them. For the final ranking, the number of votes and the median score were taken into account.

Results: Seven HTAi-IRG members answered to the first questionnaire. Seventeen agencies answered to the second one (thirteen EuroScan members and four more members from INAHTA), but only seven had worked in the identification of POTs and one of them using only experts for it. The remaining six agencies answered the part related to devices, diagnostics, and procedures; five of them did it for settings and programmes and only three for drugs. The Canadian Agency for Drugs and Technologies in Health (5 votes; median = 2), Cochrane Collaboration (5 votes; median = 3), NICE (4 votes; median = 1), Food and Drug Administration (4 votes; median = 1.5), and EuroScan (4 votes, median = 2) were the most relevant sources for devices and diagnostics.

Conclusions: There is little experience on POTs identification. The identified sources provide mostly indirect information and further research should take place to determine the best use of them.

Objectives: Assessments of health technologies increasingly include economic evaluations conducted alongside clinical trials. One particular concern with economic evaluations conducted alongside clinical trials is the generalizability of results from one setting to another. Much of the focus relating to this topic has been on the generalizability of results between countries. However, the characteristics of clinical trial design require further consideration of the generalizability of cost data between centers within a single country, which could be important in decisions about adoption of the new technology.

Methods: We used data from a multicenter clinical trial conducted in the United Kingdom to assess the degree of variation in costs between patients and between treatment centers and the determinants of the degree of such variation.

Results: The variation between patients was statistically significant for both the experimental and conventional treatments. However, the degree of variation between centers was only statistically significant for the experimental treatment. Such variation appeared to be a result of hospital practice, such as payment mechanisms for staff and provision of hostel accommodation, rather than variations in physical resource use or substantive differences in cost structure.

Conclusions: Multicenter economic evaluations are necessary for determining the variations in hospital practice and characteristics that can in turn determine the generalizability of study results to other settings. Such analyses can identify issues that may be important in adopting a new health technology. Analysis is required of similar large multicenter trials to confirm these conclusions.

Perspectives in an economic analysis of medical technology reflect who makes decisions about the use of or payment for medical resources. Commonly used perspectives include those of providers, insurers, the individual, and society. Perspective is a critical determinant of study design, affecting the time horizon, types of resources considered, and economic cost measures assigned to those resources. Individuals involved in technology assessment for either research or policy-making purposes should be aware of the complexities of defining costs from different perspectives.

Objectives: To elucidate important differences between a health technology assessment (HTA) and a systematic review, using an HTA of positron emission tomography (PET) as an example.

Methods: Interviews with seventeen individuals who were authors or users of the PET HTA.

Results: Those interviewed identified seven areas in which HTAs often differ from traditional systematic reviews: (i) methodological standards (HTAs may include literature of relatively poor methodological quality if a topic is of importance to decision-makers), (ii) replication of previous studies (relatively common for HTAs but not systematic reviews), (iii) choice of topics (more policy oriented for HTAs, while systematic reviews tend to be driven by researcher interest), (iv) inclusion of content experts and policy-makers as authors (policy-makers more likely to be included in HTAs, although there are potential conflicts of interest), (v) inclusion of economic evaluations (more often with HTAs, although economic evaluations based upon poor clinical data may not be useful), (vi) making policy recommendations (more likely with HTAs, although this must be done with caution), and (vii) dissemination of the report (more often actively done for HTAs).

Conclusions: This case study of an HTA of PET scanning confirms that HTAs are a bridge between science and policy and require a balance between the ideals of scientific rigor and the realities of policy making.

Objectives: The aim of this study was to evaluate the costs and effectiveness associated with no screening, Helicobacter pylori serology screening, and the 13C-urea breath test (UBT) for gastric cancer in the Chinese population.

Methods: A Markov model simulation was carried out in Singaporean Chinese at 40 years of age (n = 478,500) from the perspective of public healthcare providers. The main outcome measures were costs, number of gastric cancer cases prevented, life-years saved, quality-adjusted life-years (QALYs) gained from the screening age to death, and incremental cost-effectiveness ratios (ICERs), which were compared among the three strategies. The uncertainty surrounding ICERs was addressed by scenario analyses and probabilistic sensitivity analysis using Monte Carlo simulation.

Results: The ICER of serology screening versus no screening was $25,881 per QALY gained (95 percent confidence interval (95 percent CI), $5,700 to $120,000). The ICER of UBT versus no screening was $53,602 per QALY gained (95 percent CI, $16,000 to $230,000). ICER of UBT versus serology screening was $470,000 per QALY gained, for which almost all random samples of the ICERs distributed above $50,000 per QALY.

Conclusions: It cannot be confidently concluded that either H pylori screening was a cost-effective strategy compared with no screening in all Chinese at the age of 40 years. Nevertheless, serology screening has demonstrated much more potential to be a cost-effective strategy, especially in the population with higher gastric cancer prevalence.

Background: In recent years, there has been growing interest in evaluating the health and economic impact of medical devices. Payers increasingly rely on cost-effectiveness analyses in making their coverage decisions, and are adopting value-based purchasing initiatives. These analytic approaches, however, have been shaped heavily by their use in the pharmaceutical realm, and are ill-adapted to the medical device context.

Methods: This study focuses on the development and evaluation of left ventricular assist devices (LVADs) to highlight the unique challenges involved in the design and conduct of device trials compared with pharmaceuticals.

Results: Devices are moving targets characterized by a much higher degree of post-introduction innovation and “learning by using” than pharmaceuticals. The cost effectiveness ratio of left ventricular assist devices for destination therapy, for example, decreased from around $600,000 per life year saved based on results from the pivotal trial to around $100,000 within a relatively short time period.

Conclusions: These dynamics pose fundamental challenges to the evaluation enterprise as well as the policy-making world, which this paper addresses.

Purpose: To estimate the benefit and cost of using radiotherapy (RT) in the initial management of lung cancer in the general population.

Methods: We identified indications for RT in the initial management of small cell and non–small cell lung cancer through a review of the literature. The proportion of patients with each specific indication for treatment was determined using epidemiological observations from cancer registry data and from the literature. We estimated the benefit gained from RT use for each indication in the model using values published in the literature. We estimated the cost of RT for each indication using published Canadian data. The total benefit and cost was calculated for all indications combined. Results are reported in 2001 Canadian dollars.

Results: The mean benefit was 7 months of survival for each lung cancer patient treated with curative intent and 3 months of symptom control for each patient treated with palliative intent. The average cost was $9,881 per life year gained and $13,938 per year of symptom control gained. Sensitivity analysis revealed values between $7,905 and $19,762 per year of survival gain and between $10,368 and $27,875 per year of symptom control gained.

Conclusions: Using RT in the initial management of lung cancer can provide considerable gains in survival and symptom control. The cost of RT for the initial management of lung cancer is inexpensive compared with a common cut off of $50,000 per life year gained.

Objectives: The cost-effectiveness of alternating pressure-relieving devices, mattress replacements, and mattress overlays compared with a standard hospital (high-specification foam mattress) for the prevention and treatment of pressure ulcers in hospital patients in the United Kingdom was investigated.

Methods: A decision-analytic model was constructed to evaluate different strategies to prevent or treat pressure ulcers. Three scenarios were evaluated: the prevention of pressure ulcers, the treatment of superficial ulcers, and the treatment of severe ulcers. Epidemiological and effectiveness data were obtained from the clinical literature. Expert opinion using a rating scale technique was used to obtain quality of life data. Costs of the devices were obtained from manufacturers, whereas costs of treatment were obtained from the literature. Uncertainty was explored through probabilistic sensitivity analysis.

Results: Using £30,000/QALY (quality-adjusted life year) as the decision-maker's cut off point (the current UK standard), in scenario 1 (prevention), the cost-effective strategy was the mattress overlay at 1, 4, and 12 weeks. In scenarios 2 and 3, the cost-effective strategy was the mattress replacement at 1, 4, and 12 weeks. Standard care was a dominated intervention in all scenarios for values of the decision-maker's ceiling ratio ranging from £5,000 to £100,000/QALY. However, the probabilistic sensitivity analysis results reflected the high uncertainty surrounding the choice of devices.

Conclusions: Current information suggests that alternating pressure mattress overlays may be cost-effective for the prevention of pressure ulcers, whereas alternating pressure mattress replacements appears to be cost-effective for the treatment of superficial and severe pressure ulcers.

Background: Endoscopic retrograde cholangiopancreatography (ERCP) is considered the gold standard for imaging of the biliary tract but is associated with complications. Less invasive imaging techniques, such as magnetic resonance cholangiopancreatography (MRCP), have a much lower complication rate. The accuracy of MRCP is comparable to that of ERCP, and MRCP may be more effective and cost-effective, particularly in cases for which the suspected prevalence of disease is low and further intervention can be avoided. A model was constructed to compare the effectiveness and cost-effectiveness of MRCP and ERCP in patients with a previous history of cholecystectomy, presenting with abdominal pain and/or abnormal liver function tests.

Methods: Diagnostic accuracy estimates came from a systematic review of MRCP. A decision analytic model was constructed to represent the diagnostic and treatment pathway of this patient group. The model compared the following two diagnostic strategies: (i) MRCP followed with ERCP if positive, and then management based on ERCP; and (ii) ERCP only. Deterministic and probabilistic analyses were used to assess the likelihood of MRCP being cost-effective. Sensitivity analyses examined the impact of prior probabilities of common bile duct stones (CBDS) and test performance characteristics. The outcomes considered were costs, quality-adjusted life years (QALYs), and cost per additional QALY.

Results: The deterministic analysis indicated that MRCP was dominant over ERCP. At prior probabilities of CBDS, less than 60 percent MRCP was the less costly initial diagnostic test; above this threshold, ERCP was less costly. Similarly, at probabilities of CBDS less than 68 percent, MRCP was also the more effective strategy (generated more QALYs). Above this threshold, ERCP became the more effective strategy. Probabilistic sensitivity analyses indicated that, in this patient group for which there is a low to moderate probability of CBDS, there was a 59 percent likelihood that MRCP was cost-saving, an 83 percent chance that MRCP was more effective with a higher quality adjusted survival, and an 83 percent chance that MRCP had a cost-effectiveness ratio more favorable than $50,000 per QALY gained.

Conclusions: Costs and cost-effectiveness are dependent upon the prior probability of CBDS. However, probabilistic analysis indicated that, with a high degree of certainty, MRCP was the more effective and cost-effective initial test in postcholecystectomy patients with a low to moderate probability of CBDS.

Objectives: In Austria, research in health technology assessment (HTA) has been conducted since the 1990s. The aim of this study is to analyze whether the HTA research program of the Institute of Technology Assessment (ITA) and the Ludwig Boltzmann Institute for HTA (LBI-HTA) have had an impact on the Austrian healthcare system.

Methods: We applied qualitative and quantitative empirical research methods, such as interviewing, download analysis, questionnaire, retrospective routine data analysis, and media analysis. Data were analyzed according to a conceptual framework, considering seven impact categories (awareness, acceptance, process, decision, practice, final outcomes, enlightenment) and different target groups.

Results: A rising number of downloads and single HTA reports with high media interest were identified. Interviews showed that HTA reports have increasingly been used for investment and reimbursement decisions, as well as for the preparation of negotiations. Economic impact was indicated by decreased expenditures due to HTA recommendations. Overall, knowledge about evidence-based medicine increased and, in places, an “HTA culture” can be recognized. Yet, several decision-making processes occur at all levels without the use of HTA.

Conclusions: The analysis demonstrated an impact within all predefined categories; however, it depends on the system level and its target groups. HTA reports are primarily used by hospital management, (social) insurances, and the Austrian Ministry of Health. Nevertheless, there is still potential to increase the impact of HTA. Therefore, the inclusion of HTA in decision-making processes in Austria needs to move from a voluntary basis to a mandatory one.

Objectives: The INTEGRATE-HTA project provided methodology to evaluate complex technologies. This study provides guidance on how to retrieve and critically appraise available evidence on moderators and predictors of treatment effects and on patient preferences for treatment outcomes as a source of complexity.

Methods: Search filters for PubMed were developed by hand-searching a large volume of articles reporting on relevant aspects. Search terms were retrieved from selected papers and algorithmically combined to find the optimal combination of search terms. For the development of the appraisal checklists literature was searched in PubMed and Google Scholar together with citation chasing. For the CHecklist for the Appraisal of Moderators and Predictors (CHAMP) a Delphi procedure was used to value a set of eligible appraisal criteria retrieved from the literature.

Results: Search filters were developed optimized for different accuracy measures. The final version of CHAMP consists of a seventeen questions covering the design, analysis, results and transferability of results of moderator and predictor analysis. The final checklist for appraisal of literature on patient preferences for treatment outcomes consist of six questions meant to help the user to identify relevant quality issues together with a guidance toward existing tools concerning the appraisal of specific preference elicitation methods.

Conclusions: Incorporating knowledge on subgroups for whom a specific treatment will produce more benefit holds the promise of better targeting and, ultimately, enhancing overall effectiveness and efficiency of healthcare technology. Finally, incorporating information on preferences for treatment outcomes will foster health technology assessment that addresses outcomes that are important to patients.

Objectives: Model for ASsessment of Telemedicine Applications (MAST) is a health technology assessment (HTA) inspired framework for assessing the effectiveness and contribution to quality of telemedicine applications based on rigorous, scientific data. This study reports from a study of how it was used and perceived in twenty-one pilots of the European project RENEWING HEALTH (RH). The objectives of RH were to implement large-scale, real-life test beds for the validation and subsequent evaluation of innovative patient-centered telemedicine services. The study is a contribution to the appraisal of HTA methods.

Methods: A questionnaire was administered for project leaders of the pilots. It included questions about use and usefulness of MAST for (i) preceding considerations, (ii) evaluation of outcomes within seven domains, and (iii) considerations of transferability. Free text spaces allowed for proposals of improvement. The responses covered all pilots. A quantitative summary of use and a qualitative analysis of usefulness were performed.

Results: MAST was used and considered useful for pilot evaluations. Challenges included problems to scientifically determine alternative service options and outcome within the seven domains. Proposals for improvement included process studies and adding domains of technological usability, responsible innovation, health literacy, behavior change, caregiver perspectives and motivational issues of professionals.

Conclusions: MAST was used according to its structure. Its usefulness in patient centered pilots can be improved by adding new stakeholder groups. Interdependencies between scientific rigor, resources and timeliness should be addressed. Operational options for improvements include process studies, literature reviews and sequential mini-HTAs for identification of areas for more elaborate investigations.

Drugs have a central place among medical technologies, and medical technology assessment can learn from the established regulation of drug technology. This article outlines how users' experiences are not part of the basis on which decisions are made today, although this knowledge is imperative for identifying the problems that are not uncovered or foreseen by today's drug assessments. Further, users' interests might not be part of assessments that are based on the controlled clinical trial. A framework for drug technology assessments based on a user perspective is suggested.

To share the existing, albeit limited, experience among nations and to explore the feasibility of international collaboration on the identification and early evaluation of health care technologies, an workshop was arranged in September 1997. Twenty-seven policy makers and researchers from twelve countries attended the meeting and concluded that: the policy environment in most European countries is characterized by insufficient data on safety, effectiveness, and cost-effectiveness of health care innovations; that an early warning system is perceived as an essential mechanism for facilitating communication among policy makers, technology experts, and health professionals; and that collaboration on early warning activities might be even more useful than traditional collaboration in health technology assessment.

Objectives: Our objective was to document the pattern of mobile phone usage by medical staff in a hospital setting, and to explore any perceived benefits (such as improved communications) associated with mobile phones.

Methods: This cross-sectional survey was conducted in Waterford Regional Hospital, Ireland, where bleep is the official system of communication. All non-consultant hospital doctors, of medical disciplines only, were asked to participate. The questionnaire was designed to explore the pattern and different aspects of mobile phone usage.

Results: At the time of study, there were sixty medical junior doctors, and the response rate was 100 percent. All participants used mobile phones while at work, and also for hospital-related work. For 98.3 percent the mobile phone was their main mode of communication while in the hospital. Sixty-two percent (n = 37) made 6–10 calls daily purely for work-related business, and this comprised of ≥80 percent of their daily usage of mobile phones. For 98 percent of participants, most phone calls were work-related. Regarding reasons for using mobile phones, all reported that using mobile phone is quicker for communication.

Conclusions: Mobile phone usage is very common among the medical personnel, and this is regarded as a more efficient means of communication for mobile staff than the hospital paging system.

To reduce harm to the environment resulting from the production, use, and disposal of health technologies, there are different options for how health technology assessment (HTA) agencies can consider environmental information. We identified four approaches that HTA agencies can use to take environmental information into account in healthcare decision making and the challenges associated with each approach. Republishing data that is in the public domain or has been submitted to an HTA agency we term the “information conduit” approach. Analyzing and presenting environmental data separately from established health economic analyses is described as “parallel evaluation.” Integrating environmental impact into HTAs by identifying or creating new methods that allow clinical, financial, and environmental information to be combined in a single quantitative analysis is “integrated evaluation.” Finally, evidence synthesis and analysis of health technologies that are not expected to improve health-related outcomes but claim to have relative environmental benefits are termed “environment-focused evaluation.”

Objectives: Despite controversies, endoscopic thoracic sympathectomy (ETS) has been used as a treatment for excessive sweating of hands and face and for facial blushing. This study aims to evaluate the effectiveness of ETS for the current indications in a systematic review.

Methods: Controlled clinical trials and cohort studies with more than 100 patients were included. Abstracts were searched from MEDLINE and CCTR from 1966 to June 2004. Two reviewers extracted the data and assessed study quality. Data on effectiveness and safety were synthesized qualitatively.

Results: We did not find any controlled clinical trials. Fifteen prospective studies were included. The internal and external quality of these studies were poor overall. Follow-up was commonly less than 2 years, during which time excessive sweating and facial blushing seemed to decrease among most patients. Immediate complications related to thoracoscopy occurred in up to 10 percent of patients. Compensatory sweating below breast level was reported in up to 90 percent of the patients. Other common side effects included dryness of face and hands, gustatory sweating, and neuralgic pain. Several other less common side effects were reported.

Conclusions: The evidence of the effectiveness of ETS is weak due to a lack of randomized trials. The intervention leads to severe immediate complications in some of the patients, and to persistent side-effects for many of the patients.

Objectives: The aim of this study was to compare the cost and the effectiveness of two biennial fecal occult blood screening tests for colorectal cancer: a guaiac nonrehydrated test (G-FOBT) and an immunochemical test (I-FOBT) with the absence of screening.

Methods: A Markov model was developed to compare these strategies in a general population of subjects aged 50 to 74 over a 20-year period.

Results: Compared with the absence of screening, G-FOBT and I-FOBT were associated with a decrease in colorectal cancer mortality of 17.4 percent and 25.2 percent, respectively. With regard to cost-effectiveness, expressed as cost per life-year gained, I-FOBT was the most effective and most costly alternative. Compared with no screening, G-FOBT and I-FOBT presented similar discounted incremental cost-effectiveness ratios: €2,739 and €2,819 respectively per life-year gained. When compared with G-FOBT, I-FOBT presented an incremental cost-effectiveness ratio of €2,988 per life-year gained. Sensitivity analyses showed the strong influence of the I-FOBT lead time, of the participation rate to screening for I-FOBT, and of the purchase price of the I-FOBT on the discounted incremental cost-effectiveness ratios.

Conclusions: Compared with the absence of screening and with G-FOBT, the biennial two-stool immunochemical test can be considered a promising method for mass screening for colorectal cancer.