Background: The Alberta Hip and Knee Replacement Project developed a new evidence-based clinical pathway (NCP) for total hip (THR) and knee (TKR) replacement. The aim was to facilitate the delivery of services in a timely and cost-effective manner while achieving the highest quality of care for the patient across the full continuum of care from patient referral to an orthopedic surgeon through surgery, recovery, and rehabilitation. The purpose of this article is to provide an overview of the study design, rationale, and execution of this project as a model for health technology assessment based on comparative effectiveness of alternative clinical pathways.

Methods: A pragmatic randomized controlled trial study design was used to evaluate the NCP compared with the standard of care (SOC) for these procedures. The pragmatic study design was selected as a rigorous approach to produce high quality evidence suitable for informing decisions between relevant interventions in real clinical practice. The NCP was evaluated in three of the nine regional health authorities (RHAs) in Alberta with dedicated central intake clinics offering multidisciplinary care teams, constituting 80 percent of THR and TKR surgeries performed annually in Alberta. Patients were identified in the offices of twenty orthopedic surgeons who routinely performed THR or TKR surgeries. Evaluation outcome measures were based on the six dimensions of the Alberta Quality Matrix for Health (AQMH): acceptability, accessibility, appropriateness, effectiveness, efficiency and safety. Data were collected prospectively through patient self-completed questionnaires at baseline and 3 and 12 months after surgery, ambulatory and inpatient chart reviews, and electronic administrative data.

Results: The trial design was successful in establishing similar groups for rigorous evaluation. Of the 4,985 patients invited to participate, 69 percent of patients consented. A total of 3,434 patients were randomized: 1,712 to SOC and 1,722 to the NCP. The baseline characteristics of patients in the two study arms, including demographics, comorbidity as measured by CDS and exposure to pain medications, and health-related quality of life, as measured by Western Ontario and McMaster Universities Osteoarthritis Index and Short Form-36, were similar.

Conclusions: The Alberta Hip and Knee Replacement Project demonstrates the feasibility and advantages of applying a pragmatic randomized controlled trial to ascertain comparative effectiveness. This is a model for health technology assessment that incorporates how clinical pathways can be effectively evaluated.

Background: In the debate on improving the quality and efficiency of the United States healthcare system, comparative effectiveness research is increasingly seen as a tool for reducing costs without compromising outcomes. Furthermore, the recent American Recovery and Reinvestment Act explicitly describes a prioritization function for establishing a comparative effectiveness research agenda. However, how such a function, in terms of methods and process, would go about identifying the most important priorities warranting further research has received little attention.

Objectives: This study describes an Agency for Healthcare Research and Quality-funded pilot project to translate one current comparative effectiveness review into a prioritized list of evidence gaps and research questions reflecting the views of the healthcare decision makers involved in the pilot.

Methods: To create a prioritized research agenda, we developed an interactive nominal group process that relied on a multistakeholder workgroup scoring a list of research questions on the management of coronary artery disease.

Results: According to the group, the areas of greatest uncertainty regarding the management of coronary artery disease are the comparative effectiveness of medical therapy versus percutaneous coronary interventions versus coronary artery bypass grafting for different patient subgroups; the impact of diagnostic testing; and the most effective method of developing performance measures for providers.

Conclusions: By applying our nominal group process, we were able to create a list of research priorities for healthcare decision makers. Future research should focus on refining this process because determining research priorities is essential to the success of developing an infrastructure for comparative effectiveness research.

Objectives: The aim of this study was to review studies reporting cost-effectiveness of exercise-based interventions in treatment of various diseases.

Methods: Systematic literature search using several databases. Abstracts initially screened independently by two authors, full-text articles again evaluated by two authors, who decided whether an article should be included. Included were scientifically valid articles describing controlled studies that included an exercise-based intervention in the treatment of an established medical condition, and also reported on the cost-effectiveness of the intervention, or its effect on the utilization of health services. Quality was assessed with an established approach.

Results: A total of 914 articles were identified, of them 151 were obtained for closer review. Sixty-five articles describing sixty-one studies were included. Most (82 percent) were randomized trials. Twenty-eight studies dealt with musculoskeletal disorders, fifteen with cardiology, four with rheumatic diseases, four with pulmonary diseases, three with urinary incontinence, and two with vascular disorders. There was one study each in the fields of oncology, chronic fatigue, endocrinology, psychiatry, and neurology. Exercise interventions in musculoskeletal disorders were deemed to be cost-effective in 54 percent, in cardiology in 60 percent, and in rheumatic diseases in 75 percent of the cases. There was some evidence that exercise might be cost-effective in intermittent claudication, breast cancer patients, diabetes, and schizophrenia.

Conclusions: The number of studies assessing cost-effectiveness of exercise interventions in various diseases is still limited. The results show large variation but suggest that some exercise interventions can be cost-effective. Most convincing evidence was found for rehabilitation of cardiac and back pain patients; however, even in these cases, the evidence was partly contradictory.

For the past several years, we have discussed the idea of producing a publication on the history of health technology assessment (HTA). It seemed important to us to develop this history now, while those who lived it can give their own accounts. An exception is Seymour Perry, the first president of ISTAHC and the Director of the first national public program in HTA, the U.S. National Center for Health Care Technology (NCHCT). See the In Memoriam in this issue.

Objectives: This study assesses the impact of the English National Health Service (NHS) Health Technology Assessment (HTA) program using the “payback” framework.

Methods: A survey of lead investigators of all research projects funded by the HTA program 1993–2003 supplemented by more detailed case studies of sixteen projects.

Results: Of 204 eligible projects, replies were received from 133 or 65 percent. The mean number of peer-reviewed publications per project was 2.9. Seventy-three percent of projects claimed to have had had an impact on policy and 42 percent on behavior. Technology Assessment Reports for the National Institute for Health and Clinical Excellence (NICE) had fewer than average publications but greater impact on policy. Half of all projects went on to secure further funding. The case studies confirmed the survey findings and indicated factors associated with impact.

Conclusions: The HTA program performed relatively well in terms of “payback.” Facilitating factors included the program's emphasis on topics that matter to the NHS, rigorous methods and the existence of “policy customers” such as NICE.

Objectives: This article presents an overview of the practical methods and tools to support transnational Health Technology Assessment (HTA) that were developed and pilot tested by the European network for HTA (EUnetHTA), which involved a total of sixty-four Partner organizations.

Methods: The methods differ according to scope and purpose of each of the tools developed. They included, for example, literature reviews, surveys, Delphi and consensus methods, workshops, pilot tests, and internal/public consultation.

Results: Practical results include an HTA Core Model and a Handbook on the use of the model, two pilot examples of HTA core information, an HTA Adaptation Toolkit for taking existing reports into new settings, a book about HTA and health policy making in Europe, a newsletter providing structured information about emerging/new technologies, an interactive Web-based tool to share information about monitoring activities for emerging/new technologies, and a Handbook on HTA capacity building for Member States with limited institutionalization of HTA.

Conclusions: The tools provide high-quality information and methodological frameworks for HTA that facilitate preparation of HTA documentation, and sharing of information in and across national or regional systems. The tools will be used and further tested by partners in the EUnetHTA Collaboration aiming to (i) help reduce unnecessary duplication of HTA activities, (ii) develop and promote good practice in HTA methods and processes, (iii) share what can be shared, (iv) facilitate local adaptation of HTA information, (v) improve the links between health policy and HTA.

Objectives: There is considerable interest in the potential for harmonizing health technology assessments (HTA) across jurisdictions. This study aims to consider four HTAs of drug eluting stents to determine the degree to which the methods adopted, evidence considered, and resulting recommendations diverge.

Methods: Four HTAs of drug eluting stents were selected for inclusion and evaluated using a framework developed to systematically capture information on the process adopted, the evidence considered and the recommendations of each HTA.

Results and Conclusions: The findings suggest that, although there is a common core data set considered by most of the agencies, differences in the approach to HTA, heterogeneity of studies, and the limited relevance of research findings to local practice meant that the core data set had only limited influence on the resulting recommendations. Of the HTA agencies considered in the analysis, many sought to generate additional primary research from local settings to help inform the development of recommendations that were relevant to local practice. This raises questions about the extent to which HTA methods can be harmonized across jurisdictions.

Technology is generally defined as “science or knowledge applied to a definite purpose.” Technology assessment has been defined as a form of policy research that examines short- and long-term consequences (for example, societal, economic, ethical, legal) of the application of technology. The goal of technology assessment was said to be to provide policy makers with information on policy alternatives. Health technology assessment (HTA) grew out of this field and was developed in the U.S. Office of Technology Assessment (OTA). However, the OTA was closed in 1995. The links between technology assessment and health technology assessment were more-or-less lost after the dissolution of OTA, and few workers in the field of HTA seem familiar with the roots of the field in the more general and social-oriented technology assessment.

Objectives: The aim of this study was to explore the use of surrogate outcomes—a substitute outcome that predicts final patient-related outcomes—in cost-effectiveness models (CEM) within health technology assessment (HTA) reports and provide guidance for their future use.

Methods: Our sampling frame was all UK HTA Program monograph series reports published in 2005 and 2006. Reports were included if they addressed a treatment effectiveness/efficacy question and included a CEM based on a surrogate outcome. The two authors independently applied inclusion and exclusion criteria, and the following data was extracted from included reports: source of surrogate outcome, level of evidence for validation of the surrogate outcomes, methods used in report to quantify link between surrogate outcome and final outcome, and consideration of the uncertainty associated with using surrogate outcomes in the results or conclusions of report.

Results: Of 100 HTA reports, 35 complied with the inclusion criteria. Of these, four (11 percent) reports included a CEM based on a surrogate outcome. All four reports sourced treatment-related changes in surrogate outcome through a systematic review of the literature. One provided Level 1 surrogate evidence (randomized controlled trial data showing a strong association between the change in surrogate outcome and change final outcome); two reported Level 2 evidence (observational study data); and one provided Level 3 evidence (disease natural/ history data). The transparency of quantification and exploration of uncertainty of the surrogate and final outcome relationship varied considerably across all four reports.

Conclusions: Recommendations are made for the use of surrogate outcomes in future HTA reports.

Objectives: The aim of this study was to identify and rank the sources for the detection of potentially obsolete technologies (POTs).

Methods: A specific questionnaire related to the search strategies and sources used for the identification of POTs and also for ineffective, inefficient or harmful health technologies was sent to the Health Technology Assessment International's Information Resources Group (HTAi-IRG) group. With the obtained information and taking into account the sources used for the identification of new and emerging technologies, a second questionnaire was elaborated and sent to EuroScan and International Network of Agencies for Health Technology Assessment (INAHTA) members, who had to select and score them. For the final ranking, the number of votes and the median score were taken into account.

Results: Seven HTAi-IRG members answered to the first questionnaire. Seventeen agencies answered to the second one (thirteen EuroScan members and four more members from INAHTA), but only seven had worked in the identification of POTs and one of them using only experts for it. The remaining six agencies answered the part related to devices, diagnostics, and procedures; five of them did it for settings and programmes and only three for drugs. The Canadian Agency for Drugs and Technologies in Health (5 votes; median = 2), Cochrane Collaboration (5 votes; median = 3), NICE (4 votes; median = 1), Food and Drug Administration (4 votes; median = 1.5), and EuroScan (4 votes, median = 2) were the most relevant sources for devices and diagnostics.

Conclusions: There is little experience on POTs identification. The identified sources provide mostly indirect information and further research should take place to determine the best use of them.

Objectives: The aim of this study was to develop and test a generic framework to enable international collaboration for producing and sharing results of health technology assessments (HTAs).

Methods: Ten international teams constructed the HTA Core Model, dividing information contained in a comprehensive HTA into standardized pieces, the assessment elements. Each element contains a generic issue that is translated into practical research questions while performing an assessment. Elements were described in detail in element cards. Two pilot assessments, designated as Core HTAs were also produced. The Model and Core HTAs were both validated. Guidance on the use of the HTA Core Model was compiled into a Handbook.

Results: The HTA Core Model considers health technologies through nine domains. Two applications of the Model were developed, one for medical and surgical interventions and another for diagnostic technologies. Two Core HTAs were produced in parallel with developing the model, providing the first real-life testing of the Model and input for further development. The results of formal validation and public feedback were primarily positive. Development needs were also identified and considered. An online Handbook is available.

Conclusions: The HTA Core Model is a novel approach to HTA. It enables effective international production and sharing of HTA results in a structured format. The face validity of the Model was confirmed during the project, but further testing and refining are needed to ensure optimal usefulness and user-friendliness. Core HTAs are intended to serve as a basis for local HTA reports. Core HTAs do not contain recommendations on technology use.

Objectives: Technology assessment systems for interventional procedures (including surgical operations, minimally invasive procedures, and others) have lagged behind those for pharmaceutical treatments. Such systems have been introduced in some countries during the past decade amid debate about how they should be organized, but there is no collated information about where they exist or how they work. This study was designed to provide hitherto unavailable information about the existence, organization, methods, and outputs of systems aimed at influencing the use of interventional procedures in different countries.

Methods: Data were gathered from a questionnaire survey of key informers associated with healthcare technology assessment (HTA) organizations in different countries.

Results: Responses were received from key informers working for twenty-eight HTA organizations in twenty-five countries (response rate 83 percent). Information about a national system for assessing interventional procedures was obtained for fifteen countries. There was substantial variability in the type and funding of these organizations, the systems used for the selection of procedures, the types and sources of evidence used, the personnel involved in the appraisal of the evidence, the arrangements for consultation on the draft assessment, the format of assessment recommendations, the status of the guidance, and the use of guidance from other countries.

Conclusion: Guidance on interventional procedures is produced variably in different countries—and not at all in some. Greater international collaboration in the assessment of new interventional procedures could help to optimize the efficiency of existing systems as well as the quality of the assessments, by capitalizing on the outputs from scarce (international) resources and expertise.

Objectives: Several studies have shown that a key determinant of successful health technology assessment (HTA) uptake is a clear, fair, and consistent decision-making process for the approval and introduction of health technologies. The aim of this study was to gauge healthcare providers' and managers' perceptions of local level decision making and determine whether these processes offer a conducive environment for HTA. An Area Health Service (AHS) aimed to use the results of this study to help design a new process of technology assessment and decision making.

Methods: An online survey was sent to all health service managers and healthcare providers working in one AHS in Sydney, Australia. Questions related to perceptions of current health technology decisions in participants' own institution/facility and opinions on key criteria for successful decision-making processes.

Results: Less than a third of participants agreed with the statements that local decision-making processes were appropriate, easy to understand, evidence-based, fair, or consistently applied. Decisions were reportedly largely influenced by total cost considerations as well as by the central state health departments and the Area executive.

Conclusions: Although there are renewed initiatives in HTA in Australia, there is a risk that such investments will not be productive unless policy makers also examine the decision-making contexts within which HTA can successfully be implemented. The results of this survey show that this is especially true at the local level and that any HTA initiative should be accompanied by efforts to improve decision-making processes.

Objectives: The aim of this study is to evaluate the cost-effectiveness of Magnetic Resonance Guided Focused Ultrasound (MRgFUS) compared with alternative treatments for uterine fibroids in the United States.

Methods: We used techniques of decision analysis and data from secondary sources to develop and estimate an economic model of the management of uterine fibroids among premenopausal women. Patients in the model receive treatment with MRgFUS, uterine artery embolization (UAE), abdominal myomectomy, hysterectomy, or pharmacotherapy. The model predicts total costs (including subsequent procedures) and quality-adjusted life-years (QALYs) for each treatment strategy over a lifetime horizon, discounted at 3 percent, from a societal perspective. Data on treatment efficacy and safety were obtained from published and unpublished studies. Costs (2005 US$) were obtained from an analysis of a large administrative database and other secondary sources. Lost productivity costs were included in the base-case analysis, but excluded in a sensitivity analysis.

Results: UAE was associated with the most QALYs (17.39), followed by MRgFUS (17.36), myomectomy (17.31), hysterectomy (17.18), and pharmacotherapy (16.70). Pharmacotherapy was the least costly strategy ($9,200 per patient), followed by hysterectomy ($19,800), MRgFUS ($27,300), UAE ($28,900), and myomectomy ($35,100). Incremental cost-effectiveness ratios (cost per QALY gained) were $21,800 for hysterectomy, $41,400 for MRgFUS, and $54,200 for UAE; myomectomy was more costly and less effective than both MRgFUS and UAE. Results were sensitive to MRgFUS recurrence rates, MRgFUS procedure costs, and assumptions about quality of life following hysterectomy.

Conclusions: Our findings suggest that MRgFUS is in the range of currently accepted criteria for cost-effectiveness, along with hysterectomy and UAE.

Objective: The Israeli National Health Insurance Law stipulates a National List of Health Services (NLHS) to which all residents are entitled from their HMOs. This list has been updated annually for almost a decade using a structured review and decision-making process. Although this process has been described in detail in previous papers, none of these have fully addressed legitimacy and fairness. We examine the legitimacy and fairness of the process of updating the NLHS in Israel.

Methods: We assessed the priority-setting process for compliance with the four conditions of accountability for reasonableness outlined by Daniels and Sabin (relevance, publicity, appeals, and enforcement). These conditions emphasize transparency and stakeholder engagement in democratic deliberation.

Results: Our analysis suggests that the Israeli process for updating the NLHS does not fulfill the appeals and enforcement conditions, and only partially follows the publicity and relevance conditions, outlined in the accountability for reasonableness framework. The main obstacles for achieving these goals may relate to the large number of technologies assessed each year within a short time frame, the lack of personnel engaged in health technology assessment, and the desire for early adoption of new technologies.

Conclusions: The process of updating the NLHS in Israel is unique and not without merit. Changes in the priority-setting process should be made to increase its acceptability among the different stakeholders.

Health technology assessment (HTA) (1), also known as healthcare technology assessment or medical technology assessment, is a form of policy research that systematically examines short- and long-term consequences of the application of a health technology, a set of related technologies, or an issue related to technology. The goal of HTA is to provide input to decision making in policy and practice. The essential properties of HTA are this orientation to decision making and its multidisciplinary and comprehensive nature.

Background: Despite the mandate to examine the medical, ethical, and economic implications of the development and use of health technology, health technology assessment (HTA) reports often emphasize the epidemiologic and economic aspects, and omit ethical considerations. This study examines both whether and how ethical issues are incorporated into HTA.

Objectives: We aim to (i) review a set of Canadian HTA reports for ethics content, (ii) describe the strategies used to incorporate ethically relevant information into HTA, and (iii) determine the presence of implicit ethical issues in a sample of HTA reports.

Methods: Descriptive and qualitative content analysis of 608 HTA reports produced by six Canadian HTA agencies from January 1997 to December 2006.

Results: We found that (i) a minority (17 percent) of Canadian HTA reports addressed ethical issues, (ii) secondary research predominates while primary analysis is rare, (iii) implicit ethical issues are present in HTA reports that do not purport to address ethics.

Conclusions: Canadian HTA reports rarely explicitly, and then only superficially, address ethics, though implicit ethical issues abound.

Objectives: The aim of this study was to analyze and describe process and outcomes of two pilot assessments based on the HTA Core Model, discuss the applicability of the model, and explore areas of development.

Methods: Data were gathered from HTA Core Model and pilot Core HTA documents, their validation feedback, questionnaires to investigators, meeting minutes, emails, and discussions in the coordinating team meetings in the Finnish Office for Health Technology Assessment (FINOHTA).

Results: The elementary structure of the HTA Core Model proved useful in preparing HTAs. Clear scoping and good coordination in timing and distribution of work would probably help improve applicability and avoid duplication of work.

Conclusions: The HTA Core Model can be developed into a platform that enables and encourages true HTA collaboration in terms of distribution of work and maximum utilization of a common pool of structured HTA information for national HTA reports.

Objectives: Europe has many health technology assessment (HTA) agencies, each producing their own HTA reports. Adapting HTA reports for different contexts could reduce the need for multiple reports on the same health technology with resultant saving of time and resources. This study aims to examine and understand the process of adaptation, and to develop a toolkit that would help the adaptation of reports produced by other countries.

Methods: The methods used were a review of the literature; a survey of twenty-nine European HTA organizations, two rounds of a Delphi survey, a face-to-face meeting of twenty-one European network for Health Technology Assessment (EUnetHTA) representatives, iterative rounds of review, and two rounds of quality assurance testing (termed applicability testing).

Results: Descriptions of previous examples of adaptation in the literature are sparse. Most respondents had previous experience in adapting reports, and all believed that adaptation was useful, and there was the ability to benefit from the use of a toolkit to aid in the process. EUnetHTA Partners developed and tested an adaptation toolkit. The toolkit is composed of a series of checklists and resources that identify or clarify the relevance, reliability, and transferability of data and information from existing reports.

Conclusions: Consensus of opinion from twenty-nine European organizations/networks has indicated that the adaptation of HTA reports would be desirable and beneficial. A toolkit was developed to help with the adaptation of HTA reports produced in other settings. This collection of resources is available for use by all HTA agencies and can be accessed at: http://www.eunethta.net/upload/WP5/EUnetHTA_HTA_Adaptation_Toolkit_October08.pdf.

Objectives: This study describes the development of a framework for health technology decisions, for Ontario Health Technology Advisory Committee (OHTAC) in Ontario, Canada.

Methods: OHTAC convened a “Decision Determinants Sub-Committee” in January 2007, which undertook a systematic literature review and conducted key informant interviews to develop an explicit decision-making framework.

Results: The “Decision Determinants Sub-Committee” offered recommendations about decision criteria, and the process by which decisions are made. Decision criteria include (i) overall clinical benefit, (ii) consistency with societal and ethical values, (iii) value for money, and (iv) feasibility of adoption into the health system. The decision process should be transparent and fair and should use a deliberative process in delivering recommendations.

Conclusions: This methodology is currently being pilot tested in a live environment: OHTAC. It will be evaluated and revised according to its feasibility, acceptability, and perceived usefulness.