Objectives: This study assesses the impact of the English National Health Service (NHS) Health Technology Assessment (HTA) program using the “payback” framework.

Methods: A survey of lead investigators of all research projects funded by the HTA program 1993–2003 supplemented by more detailed case studies of sixteen projects.

Results: Of 204 eligible projects, replies were received from 133 or 65 percent. The mean number of peer-reviewed publications per project was 2.9. Seventy-three percent of projects claimed to have had had an impact on policy and 42 percent on behavior. Technology Assessment Reports for the National Institute for Health and Clinical Excellence (NICE) had fewer than average publications but greater impact on policy. Half of all projects went on to secure further funding. The case studies confirmed the survey findings and indicated factors associated with impact.

Conclusions: The HTA program performed relatively well in terms of “payback.” Facilitating factors included the program's emphasis on topics that matter to the NHS, rigorous methods and the existence of “policy customers” such as NICE.

Objectives: The aim of this study was to explore the use of surrogate outcomes—a substitute outcome that predicts final patient-related outcomes—in cost-effectiveness models (CEM) within health technology assessment (HTA) reports and provide guidance for their future use.

Methods: Our sampling frame was all UK HTA Program monograph series reports published in 2005 and 2006. Reports were included if they addressed a treatment effectiveness/efficacy question and included a CEM based on a surrogate outcome. The two authors independently applied inclusion and exclusion criteria, and the following data was extracted from included reports: source of surrogate outcome, level of evidence for validation of the surrogate outcomes, methods used in report to quantify link between surrogate outcome and final outcome, and consideration of the uncertainty associated with using surrogate outcomes in the results or conclusions of report.

Results: Of 100 HTA reports, 35 complied with the inclusion criteria. Of these, four (11 percent) reports included a CEM based on a surrogate outcome. All four reports sourced treatment-related changes in surrogate outcome through a systematic review of the literature. One provided Level 1 surrogate evidence (randomized controlled trial data showing a strong association between the change in surrogate outcome and change final outcome); two reported Level 2 evidence (observational study data); and one provided Level 3 evidence (disease natural/ history data). The transparency of quantification and exploration of uncertainty of the surrogate and final outcome relationship varied considerably across all four reports.

Conclusions: Recommendations are made for the use of surrogate outcomes in future HTA reports.

Objectives: The aim of this study is to evaluate the cost-effectiveness of Magnetic Resonance Guided Focused Ultrasound (MRgFUS) compared with alternative treatments for uterine fibroids in the United States.

Methods: We used techniques of decision analysis and data from secondary sources to develop and estimate an economic model of the management of uterine fibroids among premenopausal women. Patients in the model receive treatment with MRgFUS, uterine artery embolization (UAE), abdominal myomectomy, hysterectomy, or pharmacotherapy. The model predicts total costs (including subsequent procedures) and quality-adjusted life-years (QALYs) for each treatment strategy over a lifetime horizon, discounted at 3 percent, from a societal perspective. Data on treatment efficacy and safety were obtained from published and unpublished studies. Costs (2005 US$) were obtained from an analysis of a large administrative database and other secondary sources. Lost productivity costs were included in the base-case analysis, but excluded in a sensitivity analysis.

Results: UAE was associated with the most QALYs (17.39), followed by MRgFUS (17.36), myomectomy (17.31), hysterectomy (17.18), and pharmacotherapy (16.70). Pharmacotherapy was the least costly strategy ($9,200 per patient), followed by hysterectomy ($19,800), MRgFUS ($27,300), UAE ($28,900), and myomectomy ($35,100). Incremental cost-effectiveness ratios (cost per QALY gained) were $21,800 for hysterectomy, $41,400 for MRgFUS, and $54,200 for UAE; myomectomy was more costly and less effective than both MRgFUS and UAE. Results were sensitive to MRgFUS recurrence rates, MRgFUS procedure costs, and assumptions about quality of life following hysterectomy.

Conclusions: Our findings suggest that MRgFUS is in the range of currently accepted criteria for cost-effectiveness, along with hysterectomy and UAE.

Objectives: An important option for the medical treatment of obstructive sleep apnea-hypopnea syndrome (OSAHS) is continuous positive airway pressure (CPAP) during sleep. This study reports on the cost-effectiveness of CPAP compared with dental devices and lifestyle advice. The work was commissioned by the NHS HTA Programme to inform the National Institute of Health and Clinical Excellence's (NICE) appraisal of CPAP.

Methods: A Markov model compared the interventions over the expected patient lifetime. The primary measure of cost-effectiveness was the incremental cost per quality-adjusted life-year (QALY) gained. The QALY incorporated the impact of treatments on daytime sleepiness, blood pressure and health-related quality of life (HRQoL).

Results: On average, CPAP was associated with higher costs and QALYs compared with dental devices or lifestyle advice. In the base-case analysis, the incremental cost-effectiveness ratio (ICER) for CPAP compared with dental devices was around £4,000 per QALY (2005–06 prices). The probability that CPAP is more cost-effective than dental devices or lifestyle advice at a threshold value of £20,000 per QALY was 0.78 for men and 0.80 for women. Several sensitivity analyses were undertaken and it was found that the ICER for CPAP consistently fell below £20,000 per QALY gained, apart from in a subgroup with mild disease.

Conclusions: The model suggests that CPAP is cost-effective compared with dental devices and lifestyle advice for adults with moderate or severe symptomatic OSAHS at the cost-effectiveness thresholds used by NICE. This finding is reflected in the NICE guidance.

Objectives: Schizophrenia imposes a great burden on society, and while evaluation should play an important role in informing society's efforts to alleviate these burdens, it is unclear what “endpoints” should be chosen as the objective of such analyses. The objectives of the study were to elicit endpoints directly from patients with schizophrenia, to ascertain whether patients are sufficiently cognoscente to express what endpoints are and are not important to them and to rank the relevant endpoints.

Methods: We applied principles of patient-centered health technology assessment to identify and value endpoints from the patient's perspective. Focus groups were conducted to elicit endpoints, using interpretive phenomalogical analysis (IPA) to guide the collection, analysis and interpretation of data. Patient interviews were subsequently used to elicit patient preference over endpoints. Respondents were presented with cards outlining the endpoints and asked to remove irrelevant cards. They where then asked to identify and rank their five most relevant endpoints in order of importance. Interviews were recorded for the purposed of triangulation, and data was analyzed using descriptive statistics. Patients were recruited from five geographically diverse cities in Germany. Eligibility required a diagnosis of schizophrenia by a physician and treatment with an antipsychotic medication for at least one year. Respondents were excluded if they were experiencing an acute episode.

Results: Thirteen endpoints emerged as important from the focus groups spanning side-effects, functional status, processes of care and clinical outcomes. Respondents could clearly identify relevant and irrelevant endpoints, and rank which factors were important to them. Triangulation between field notes of the ranking exercise and recordings confirmed that rankings were not arbitrary, but justified from the respondents' point of view.

Conclusions: Patients with schizophrenia can express preferences over endpoints. Our results show that qualitative methods such as IPA can be used to identify factors, but ranking exercises provide a more robust method for ranking the importance of endpoints. Future research involving patients with schizophrenia ranking outcomes is needed to identify variations across patients and methods such as conjoint analysis could prove beneficial in identifying acceptable tradeoffs across endpoints.

Objectives: Mini-HTA (health technology assessment) is increasingly being applied in Denmark as an input for decisions on the use of health technologies. Mini-HTA is a form or check list with questions concerning the prerequisites for and consequences of health technologies. At the national level, the National Board of Health uses mini-HTA when hospitals apply for permission to introduce new treatments. Mini-HTA is also compulsory in Danish Regions' annual collection of early warnings. At the local level some hospitals have made mini-HTA compulsory when clinical departments apply for funding for new technologies. The objective of this study is to assess the quality of the information included in mini-HTA used at Danish hospitals and to discuss the consequences of this to decision making.

Methods: The quality of mini-HTA is assessed by use of an INATHA checklist for HTA reports. Data consists of reviews of the quality in fifty-two mini-HTAs produced by Danish hospitals in 2008.

Results: The mini-HTAs generally include descriptions of the assessed technology and the comparator, but information about the selection and interpretation of the clinical literature and other data is often missing. The level of evidence for the clinical effects and the main references are generally included. Only 25 percent of the mini-HTAs include a quantitative estimate of the size of the clinical effects. Organizational consequences inside the clinical department is described in 81percent of the cases and 92 percent includes a cost estimate.

Conclusions: The results show that the quality of the information in many cases is insufficient. There is a strong need for quality assurance of mini-HTAs to improve the accuracy of the information, however, without harming the timeliness and the limited use of resources in producing the reports.

Objectives: Research has shown that effectiveness, cost-effectiveness, and severity of illness each play a role in drug reimbursement decisions. However, the role of budget impact in such decisions is less obvious. Policy makers almost always demand a budget impact estimate yet seem reluctant to formally include budget impact as a rationing criterion. Health economists even reject budget impact as a legitimate criterion. For these reasons, it is important to examine its use in rationing decisions, and rationales underlying its use.

Methods: We trace several rationales supporting the use of budget impact through a literature review, supplemented by semistructured interviews with eleven key stakeholders involved in drug reimbursement decisions in the Netherlands.

Results: Budget impact arguments are used in certain instances, although policy makers appear uncomfortable with its use because well described rationales still are lacking. In addition, we identify the following rationales to support budget impact as a rationing criterion: opportunity costs, loss aversion, uncertainty and equal opportunity.

Conclusions: Budget impact plays a role in drug reimbursement decisions and has rationales to support its use. However, policy makers do not easily admit that they consider budget impact and are even reluctant to explicitly use budget impact as a formal criterion. A debate would strengthen the theoretical foundation of budget impact as a legitimate criterion in the context of drug reimbursement decisions. Such discussion of budget impact's role will also enhance policy-makers' accountability.

Objectives: Traditionally, patients with chronic hepatitis C virus (HCV) infection have not received treatment until their infection reaches the moderate to severe stage. The aim of this systematic review was to assess the clinical effectiveness of pegylated (PEG) and non-pegylated interferon (IFN) alfa and ribavirin (RBV) for the treatment of adults with histologically mild HCV.

Methods: We performed a sensitive search of fourteen electronic bibliographic databases for literature that met criteria defined in a research protocol. Two reviewers independently selected studies, extracted data and assessed methodological quality.

Results: Ten randomized, controlled trials (RCTs) were included. Treatment with PEG + RBV combination therapy resulted in significantly higher sustained virological response (SVR) rates than treatment with IFN + RBV combination therapy. Treatment for 48 weeks with PEG + RBV was significantly more effective than the same treatment for 24 weeks. Significantly higher SVR rates were seen with IFN + RBV compared with either IFN monotherapy or no treatment. In the meta-analysis (four IFN trials), the relative risk of not experiencing an SVR was 0.59 (95 percent CI, 0.51 – 0.69) and was statistically significant (p < .00001). SVRs were higher for patients with genotype non-1 compared with genotype 1 for both PEG + RBV and IFN + RBV treatments.

Conclusions: Patients with histologically mild HCV can be successfully treated with both PEG and IFN combination therapy, and response rates are broadly comparable with those achieved in patients with advanced disease. Treating patients in the early milder stages of HCV is, therefore, a clinically effective option.

Objectives: Registers have proven to be a valuable instrument in the evaluation of arthroplasty procedures and the performance of implants. The aim of this study was to describe the structure, functioning, and content of arthroplasty registers in Europe and other parts of the world.

Methods: A search of technical reports was carried out through the Internet and in Medline/PubMed. The exhaustiveness of the information was confirmed using the links to Web pages of other registers and contacts with key people. Aims, methods in data collection and evaluation, internal structure and organization, participants, validity of the data, and other variables were assessed for each arthroplasty register using a qualitative content analysis of the texts.

Results: Fifteen arthroplasty registers were identified which published sufficient information to conduct a comparative analysis. Eight additional registers were identified but no information was available on the Internet or in English. Most registers were initiatives of an orthopaedic society receiving governmental funding. Data were collected using standardized clinical forms and additional information from clinical-administrative datasets or other registers (mortality, implant costs, hip fractures). The main outcome measure of these registers is survival of the prostheses. Registers use the Internet and their annual reports as the main strategy for the dissemination and feed-back of their results.

Conclusions: Scientific or professional societies and the public health administration should collaborate in the development of arthroplasty registers. To adequately assess the results of observational data information on the structure, the process of arthroplasty interventions and patients characteristics should be collected.

Objectives: Constructive Technology Assessment (CTA) is a means to guide early implementation of new developments in society, and can be used as an evaluation tool for Coverage with Evidence Development (CED). We used CTA for the introduction of a new diagnostic test in the Netherlands, the 70-gene prognosis signature (MammaPrint®) for node-negative breast cancer patients.

Methods: Studied aspects were (organizational) efficiency, patient-centeredness and diffusion scenarios. Pre-post structured surveys were conducted in fifteen community hospitals concerning changes in logistics and teamwork as a consequence of the introduction of the 70-gene signature. Patient-centeredness was measured by questionnaires and interviews regarding knowledge and psychological impact of the test. Diffusion scenarios, which are commonly applied in industry to anticipate on future development and diffusion of their products, have been applied in this study.

Results: Median implementation-time of the 70-gene signature was 1.2 months. Most changes were seen in pathology processes and adjuvant treatment decisions. Physicians valued the addition of the 70-gene signature information as beneficial for patient management. Patient-centeredness (n = 77, response 78 percent): patients receiving a concordant high-risk and discordant clinical low/high risk-signature showed significantly more negative emotions with respect to receiving both test-results compared with concordant low-risk and discordant clinical high/low risk-signature patients. The first scenario was written in 2004 before the introduction of the 70-gene signature and identified hypothetical developments that could influence diffusion; especially the “what-if” deviation describing a discussion on validity among physicians proved to be realistic.

Conclusions: Differences in speed of implementation and influenced treatment decisions were seen. Impact on patients seems especially related to discordance and its successive communication. In the future, scenario drafting will lead to input for model-based cost-effectiveness analysis. Finally, CTA can be useful as a tool to guide CED by adding monitoring and anticipation on possible developments during early implementation, to the assessment of promising new technologies.

Objectives: The aim of this study was to assess the status of translating research findings into practice at a major academic healthcare system in Central Texas.

Methods: We conducted a cross-sectional survey addressing knowledge of and participation in translational research of physicians, residents, nurses and third- and fourth-year medical students in a major academic healthcare system in Central Texas.

Results: Out of 508 respondents, 428 (84.3 percent) completed all questions. A total of 68.9 percent of faculty reported having sufficient education and training to conduct research versus 44.4 percent of residents and 35.6 percent of nurses. Fifty-eight percent of faculty, 53 percent of residents and 9 percent of nurses reported current involvement in research activity. A total of 55.6 percent of residents reported that their departments provide them with protected time for research versus 18.4 percent of faculty and 10.3 percent of nurses. In addition, 33.9 percent of nurses reported interest in participating in research but do not know how to start. There were 86.4 percent of faculty, 77.8 percent of residents, and 58 percent of nurses who indicated they were familiar with translational research. However, only 42.7 percent of faculty, 46.7 percent of residents and 35.6 percent of nurses indicated they were aware of any changes in the delivery of care that resulted from research projects.

Conclusions: The study results suggested failure to leverage members of the healthcare team in a systematic process to ensure translation of research findings into practice. Results highlighted the need to merge culture of safety and quality improvement with research while dealing with the daily pressures of patient care.

Objectives: Fatigue is the most common symptom reported by cancer patients. The inclusion of health-related quality of life (HRQL) measures in routine clinical care of cancer patients may improve the management of fatigue. The primary objective of this study is to provide evidence on the magnitude of change in fatigue subscale scores using the Functional Assessment of Cancer Therapy-Fatigue (FACT-F) that is clinically important.

Methods: Consecutive patients with advanced primary lung cancer attending a Canadian tertiary care cancer and, prior to undergoing palliative chemotherapy, were enrolled in the study. Patients completed a battery of questionnaires [FACT-F, Qualitative Patients Self-report of Fatigue Level (QPSRF)] at baseline, follow-up and 2 weeks after their final cycle of chemotherapy. Clinicians assessed the patients using the Eastern Cooperative Oncology Group (ECOG) Performance Status Scale at baseline and each follow-up visit. FACT-F change scores were computed as the mean change in score (end of study score minus baseline score).

Results: A total of 43 patients with mean age of 59 years were enrolled in the study. Results revealed a mean change in FACT-F subscale score of 5.0 (SE 1.06) for those who rated themselves as more tired, 1.28 (SE 1.00) for those who rated themselves as the same (no change), and −1.52 (SE 0.84) for those patients who rated themselves as less tired.

Conclusions: We provide evidence on the magnitude of change in FACT-F score that is associated with the perception by patients of improvement in fatigue and magnitude of change in score that is associated with worsening in fatigue.

Objectives: This research integrates existing literature on information technology (IT) in hospitals, and proposes and validates a comprehensive IT capacities assessment tool in these settings.

Methods: A comprehensive literature review was conducted on Medline until September 2006 to identify studies that used specific IT measures in hospitals. The results were mapped and used as a basis for the development of the proposed instrument, which was tested through a survey of Canadian healthcare organizations (N = 221).

Results: A total of seventeen studies provided indicators of clinical and administrative IT capacities in hospitals. Based on the mapping of these indicators, a comprehensive IT capacities assessment instrument was developed including thirty-four items exploring computerized processes, thirteen items assessing contemporary technologies, and eleven items investigating internal and external information sharing. A time frame was inserted in the tool to reflect “plans for” versus “current” implementation of IT; in the latter, the extent of current use of computerized processes and technologies was measured on a (1–7) scale. Overall, the survey yielded a total of 106 responses (52.2 percent response rate), and the results demonstrated a good level of reliability and validity of the instrument.

Conclusions: This study unifies existing work in this area, and presents the psychometric properties of an IT capacities assessment tool in hospitals. By developing scores for capturing IT capacities in hospitals, it is possible to further address important research questions related to the determinants and impacts of IT sophistication in these settings.

Verhagen et al. (5) suggest, in a study done in 2002, that methodologic quality of individual trials do not influence the conclusions of a landmark meta-analysis on thrombolytic therapy. This meta-analysis was studied because it was believed that it represents the true effect because its conclusions remain valid 15 years after publication. They incorporated the results of quality assessment in five different ways in the calculation of the pooled odd ratios (ORs): (i) component analysis, (ii) visual plot, (iii) quality score as a threshold score, (iv) quality score as a weighting factor, and (v) cumulative pooling. They did not find much discrepancy using either of these methods of quality assessment.

We read with great interest the excellent paper of Gutiérrez-Ibarluzea et al. on the review of current policies of screening for colorectal cancer in European countries (12).Colorectal cancer screening has been a hot topic in health technology assessment and medical decision making (13;15;18). The study by Gutiérrez-Ibarluzea and colleagues focused mainly on the “old” fifteen member states of the European Union; however, colorectal cancer represents a large epidemiological (3;11) and economic (4) burden for the society and the healthcare financing agency in Eastern European countries. We would like to highlight some important aspect of colorectal cancer screening in Hungary.

We agreed with Gulácsi et al. on their perception about our article that did not include those eastern European countries that have recently joined the European Union. It does not mean that we tried to provide partial information or that we wanted to diminish the information with other purposes. In fact, the title of our article when it was sent to be published was: “Screening for colon-rectal cancer in the Europe of fifteen, Norway and Switzerland. So equal, so different. Is it time for a common approach?” and so was the title of the oral communication presented in the Annual Meeting of HTAi held in Montreal last summer (7). Even when we believed that this title was more accurate to the data provided in the article, we finally followed the suggestions of the reviewers to make the title short and comprehensive.