Objectives: Mini-HTA (health technology assessment) is increasingly being applied in Denmark as an input for decisions on the use of health technologies. Mini-HTA is a form or check list with questions concerning the prerequisites for and consequences of health technologies. At the national level, the National Board of Health uses mini-HTA when hospitals apply for permission to introduce new treatments. Mini-HTA is also compulsory in Danish Regions' annual collection of early warnings. At the local level some hospitals have made mini-HTA compulsory when clinical departments apply for funding for new technologies. The objective of this study is to assess the quality of the information included in mini-HTA used at Danish hospitals and to discuss the consequences of this to decision making.

Methods: The quality of mini-HTA is assessed by use of an INATHA checklist for HTA reports. Data consists of reviews of the quality in fifty-two mini-HTAs produced by Danish hospitals in 2008.

Results: The mini-HTAs generally include descriptions of the assessed technology and the comparator, but information about the selection and interpretation of the clinical literature and other data is often missing. The level of evidence for the clinical effects and the main references are generally included. Only 25 percent of the mini-HTAs include a quantitative estimate of the size of the clinical effects. Organizational consequences inside the clinical department is described in 81percent of the cases and 92 percent includes a cost estimate.

Conclusions: The results show that the quality of the information in many cases is insufficient. There is a strong need for quality assurance of mini-HTAs to improve the accuracy of the information, however, without harming the timeliness and the limited use of resources in producing the reports.

Objective: To determine the incremental cost-effectiveness of magnetic resonance imaging (MRI) and computed tomography (CT) in young adults presenting with equivocal neurological signs and symptoms. Designs and methods: A decision analysis of long-term survival using accuracy data from a diagnostic technology assessment of MRI and CT in patients with suspected multiple sclerosis, information from the medical literature, and clinical assumptions. Main results: In the baseline analysis, at 30% likelihood of an underlying neurologic disease, MRI use has an incremental cost of $101,670 for each additional quality-adjusted life-year saved compared with $20,290 for CT use. As the probability of disease increases, further MRI use becomes a cost-effective alternative costing $30,000 for each quality-adjusted life-year saved. If a negative MRI result provides reassurance, the incremental costs of immediate MRI use decreases and falls below $25,000 for each quality-adjusted life-year saved no matter the likelihood of disease. Conclusions: For most individuals with neurological symptoms or signs, CT imaging is cost-effective while MR imaging is not. The cost-effectiveness of MRI use, however, improves as the likelihood of an underlying neurological disease increases. For selected patients who highly value diagnostic information, MRI is a reasonable and cost-effective use of medical resources when even the likelihood of disease is quite low (5%).

The International Network of Agencies for Health Technology Assessment (INAHTA) spans the globe as a network of 50 publicly-funded health technology assessment (HTA) agencies supporting health system decision making for 1.4 billion people in thirty countries. Agency members are non-profit HTA organizations that are part of, or directly support, regional or national governments. Recently, INAHTA surveyed its members to gather perspectives from agency leadership on the most important issues in HTA today. This paper describes the top 10 challenges identified by INAHTA members. Addressing these challenges requires a call for action from INAHTA member agencies and the many other actors involved in the HTA ecosystem. In opening this call for action, INAHTA will lead the way; however, a comprehensive undertaking from all players is needed to effectively address these challenges and to continue to evolve HTA in its role as a strong and effective contributor to health systems.

This article considers the distinction between “explanatory” and “pragmatic” aims in clinical trials—the distinction between testing a biological hypothesis and providing evidence to permit a choice between alternative treatment policies. The choice of treatments to compare, the selection of patients for the trial, the study size, and how the treatment comparison should be made are among the matters discussed. In general, where explanatory and pragmatic aims conflict, the pragmatic aim will often take priority.

There is growing interest globally in using real-world data (RWD) and real-world evidence (RWE) for health technology assessment (HTA). Optimal collection, analysis, and use of RWD/RWE to inform HTA requires a conceptual framework to standardize processes and ensure consistency. However, such framework is currently lacking in Asia, a region that is likely to benefit from RWD/RWE for at least two reasons. First, there is often limited Asian representation in clinical trials unless specifically conducted in Asian populations, and RWD may help to fill the evidence gap. Second, in a few Asian health systems, reimbursement decisions are not made at market entry; thus, allowing RWD/RWE to be collected to give more certainty about the effectiveness of technologies in the local setting and inform their appropriate use. Furthermore, an alignment of RWD/RWE policies across Asia would equip decision makers with context-relevant evidence, and improve timely patient access to new technologies. Using data collected from eleven health systems in Asia, this paper provides a review of the current landscape of RWD/RWE in Asia to inform HTA and explores a way forward to align policies within the region. This paper concludes with a proposal to establish an international collaboration among academics and HTA agencies in the region: the REAL World Data In ASia for HEalth Technology Assessment in Reimbursement (REALISE) working group, which seeks to develop a non-binding guidance document on the use of RWD/RWE to inform HTA for decision making in Asia.

Objectives: To obtain further information from members of the International Network of Agencies for Health Technology Assessment (INAHTA) on the involvement of consumers in their programs.

Methods: A questionnaire for a survey was developed and sent to member agencies in November 2010. Survey responses were compared with those from an earlier survey conducted in 2005.

Results: Of the thirty-three agencies that provided responses, 67 percent involve consumers in some aspects of their health technology assessment (HTA) programs, compared with 57 percent in 2005. As in the earlier survey, most agencies reporting involvement have contact with consumer or patient organizations and a large minority also involve individual consumers. Summaries of HTA reports that are intended to be easily understood by consumers are prepared by 84 percent of the agencies, and 42 percent involve consumers in dissemination of HTA material. In both areas, there was some increase from the levels previously reported.

Conclusions: The survey results suggest that there is a trend to increased involvement of consumers by the INAHTA agencies in their programs but that the level of involvement remains relatively limited. The manner of consumer participation varies between agencies.

Objectives: Health technology assessment (HTA) carried out for policy decision making has well-established principles unlike hospital-based HTA (HB-HTA), which differs from the former in the context characteristics and ways of operation. This study proposes principles for good practices in HB-HTA units.

Methods: A framework for good practice criteria was built inspired by the EFQM excellence business model and information from six literature reviews, 107 face-to-face interviews, forty case studies, large-scale survey, focus group, Delphi survey, as well as local and international validation. In total, 385 people from twenty countries have participated in defining the principles for good practices in HB-HTA units.

Results: Fifteen guiding principles for good practices in HB-HTA units are grouped in four dimensions. Dimension 1 deals with principles of the assessment process aimed at providing contextualized information for hospital decision makers. Dimension 2 describes leadership, strategy and partnerships of HB-HTA units which govern and facilitate the assessment process. Dimension 3 focuses on adequate resources that ensure the operation of HB-HTA units. Dimension 4 deals with measuring the short- and long-term impact of the overall performance of HB-HTA units. Finally, nine core guiding principles were selected as essential requirements for HB-HTA units based on the expertise of the HB-HTA units participating in the project.

Conclusions: Guiding principles for good practices set up a benchmark for HB-HTA because they represent the ideal performance of HB-HTA units; nevertheless, when performing HTA at hospital level, context also matters; therefore, they should be adapted to ensure their applicability in the local context.

Objectives: Rapid reviews are characterized as an accelerated evidence synthesis approach with no universally accepted methodology or definition. This modified Delphi consensus study aimed to develop a comprehensive set of defining characteristics for rapid reviews that may be used as a functional definition.

Methods: Expert panelists with knowledge in rapid reviews and evidence synthesis were identified. In the first round, panelists were asked to answer a seventeen-item survey addressing a variety of rapid review topics. Results led to the development of statements describing the characteristics of rapid reviews that were circulated to experts for agreement in a second survey round and further revised in a third round. Consensus was reached if ≥70 percent of experts agreed and there was stability in free-text comments.

Results: A panel of sixty-six experts participated. Consensus was reached on ten of eleven statements describing the characteristics of rapid reviews. According to the panel, rapid reviews aim to meet the requirements and timelines of a decision maker and should be conducted in less time than a systematic review. They use a variety of approaches to accelerate the evidence synthesis process, tailor the methods conventionally used to carry out systematic reviews, and use the most rigorous methods that the delivery time frame will allow.

Conclusions: This study achieved consensus on ten statements describing the defining characteristics of rapid reviews based on the opinion of a panel of knowledgeable experts. Areas of disagreement were also highlighted. Findings emphasize the role of the decision maker and stress the importance of transparent reporting.

Background: Ethics has been part of health technology assessment (HTA) from its beginning in the 1970s, and is currently part of HTA definitions. Several methods in ethics have been used in HTA. Some approaches have been developed especially for HTA, such as the Socratic approach, which has been used for a wide range of health technologies. The Socratic approach is used in several ways, and there is a need for harmonization to promote its usability and the transferability of its results. Accordingly, the objective of this study was to stimulate experts in ethics and HTA to revise the Socratic approach.

Methods: Based on the current literature and experiences in applying methods in ethics, a panel of ethics experts involved in HTA critically analyzed the limitations of the Socratic approach during a face-to-face workshop. On the basis of this analysis a revision of the Socratic approach was agreed on after deliberation in several rounds through e-mail correspondence.

Results: Several limitations with the Socratic approach are identified and addressed in the revised version which consists of a procedure of six steps, 7 main questions and thirty-three explanatory and guiding questions. The revised approach has a broader scope and provides more guidance than its predecessor. Methods for information retrieval have been elaborated.

Conclusion: The presented revision of the Socratic approach is the result of a joint effort of experts in the field of ethics and HTA. Consensus is reached in the expert panel on an approach that is considered to be more clear, comprehensive, and applicable for addressing ethical issues in HTA.

This study summarizes the experience with health technology assessment (HTA) at the European level. Geographically, Europe includes approximately fifty countries with a total of approximately 730 million people. Politically, twenty-seven of these countries (500 million people) have come together in the European Union. The executive branch of the European Union is named the European Commission, which supports several activities, including research, all over Europe and in many other parts of the world. The European Commission has promoted HTA by several policy positions and has funded a series of projects aimed at strengthening HTA in Europe. Around fifteen of the European countries now have formal national programs on HTA and some also have regional public programs. All countries that are members of the European Union and do not have a national approach to HTA have an interest in becoming more involved. The HTA projects sponsored by the European Commission have focused on networking and collaboration among established agencies and institutions for HTA, however, also on capacity building, support, and facilitation in creating mechanisms for HTA in European countries that still do not have any program in the field.

This paper provides an overview of the effects of in vitro fertilization (IVF) on the children born from it. One of the main problems with IVF to date remains the high incidence of multiple pregnancies, which carry an inherent higher risk of preterm delivery and, therefore, of increased morbidity and mortality in newborns. Further, singleton pregnancies and twin pregnancies from IVF compared to control singleton or twin pregnancies appear to be at higher risk of preterm birth and low birth weight. Whether this is an effect of the procedure per se or is related to maternal factors, or a combination of both, remains to be studied. The risk of congenital malformations does not, with the available data, seem to be elevated. As of now, it remains unclear whether embryo freezing is a safe procedure. Psychomotor development of children born through IVF does not seem to be disturbed. Until further and more extensive studies are conducted, it remains unclear whether IVF poses long-term risks for the children.

Objectives: Use of complementary and alternative medicine (CAM) is growing in all Western countries. The goal of this study was to evaluate quality of randomized controlled trials (RCTs) of CAM interventions for specific diagnoses to inform clinical decision making.

Methods: MEDLINE and related databases were searched for CAM RCTs. Visual review was done of bibliographies, meta-analyses, and CAM journals. Inclusion criteria for review and scoring were blinded RCT, specified diagnosis and intervention, complete study published between January 1, 1966 and July 31, 1998 in an English-language, peer-reviewed journal. Two reviewers independently scored each study.

Results: More than 5,000 trials were found, but only 258 met all study inclusion criteria. The main cause for rejection (> 90%) was that the study was not an RCT or had no blinding. Mean score across 95 diagnosis/intervention categories was 44.7 (S.D. ± 14.3) on a 100-point scale. Ordinary least-squares regression found date of publication, biostatistician as author or consultant, published in one of five widely read English-language medical journals and diagnosis/intervention category of hypertension/relaxation as significant predictors of higher scores.

Conclusions: The overall quality of evidence for CAM RCTs is poor but improving slowly over time, about the same as that of biomedicine. Thus, most services are provided without good evidence of benefit.

Objectives: The aim of this study was to estimate the relationship between the financial impact of a new drug and the recommendation for reimbursement by the Australian Pharmaceutical Benefits Advisory Committee (PBAC).

Methods: Data in the PBAC summary database were abstracted for decisions made between July 2005 and November 2009. Financial impact—the upper bound of the values presented in the PBAC summary database—was categorized as ≤A$0, >A$0 up to A$10 million, A$10 million up to A$30 million, and >A$30 million per year. Descriptive, logistic, survival, and recursive partitioning decision analyses were used to estimate the relationship between the financial impact of a new drug indication and the recommendation for reimbursement. Multivariable analyses controlled for other clinical and economic variables, including cost per quality-adjusted life-year gained.

Results: Financial impact was a significant predictor of the recommendation for reimbursement. In the logistic analysis, the odds ratios of reimbursement for drug submissions with financial impacts ≥A$10 million to ≥A$30 million or >A$0 to <A$10 million compared with ≤A$0 were 0.12 (95 percent confidence interval [CI]: 0.03–0.51) and 0.16 (95 percent CI: 0.04–0.60), respectively. In the recursive partition decision analysis, the first split of the data was for submissions with a positive financial impact compared with those with a zero or negative financial impact.

Conclusions: In Australia, financial impact on the drug budget is an important determinant of whether a new drug is recommended for reimbursement when cost-effectiveness estimates and other clinical and economic variables are controlled.

Objectives: To compare four contingent valuation elicitation methods as a means to estimate the value of a pneumococcal vaccine in Bangladesh and to test if the elicitation methods are subject to bias and if they produce valid responses.

Methods: Three hundred sixty-one households with at least one child under 5 years of age were recruited in Dhaka, Bangladesh. Subjects were cluster-randomized to various elicitation methods: open-ended, dichotomous choice (at one of two asking prices), payment card (one of two cards with differing ranges). The dichotomous choice method was then followed up with a bidding game methodology, with the dichotomous choice price acting as the starting price for the bidding game. Analysis focused on summary statistics, demand curve estimation and multivariate regression analysis to test for validity and bias.

Results: Thirty-one households refused to participate, leaving a total of 330 participating households (a 91.4 percent response rate). Willingness to pay estimates varied significantly across the methods (p < .001), with average estimates varying between $2.34 and $18 (US). The open-ended elicitation method was found to produce highly inflated values that were insensitive to construct validity tests. The dichotomous choice method produced quantity (demand) estimates rather than value estimates, and there was some evidence of yea saying. The payment card elicitation method was found to be affected by range bias. The bidding game elicitation method was found to be less sensitive to starting point bias and yea-saying.

Conclusions: Different elicitation format do give rise to different demand curves; however, this may be partially due to the fact that they do not measure the same outcome. For example, the dichotomous choice format produces a demand curve, while the payment card, open-ended and bidding game produce inverse demand curves. All formats are prone to multiple biases. When choosing an elicitation format, it is important to first consider the purpose and use of the data. Each elicitation method has strengths and weaknesses and can be used for different purposes in technology assessment.

Objectives: We assessed the impacts of a Citizens’ Reference Panel on the deliberations of a provincial health technology advisory committee and its secretariat, which produce, recommendations for the use of health technologies in Ontario, Canada.

Methods: A fourteen-member citizens’ reference panel was convened five times between February 2009 and May 2010 to participate in informed, facilitated discussions to inform the assessment of individual technologies and provincial health technology assessment processes more generally. Qualitative data collection methods were used to document observed and perceived impacts of the citizens’ panel on the health technology assessment (HTA) process.

Results: Panel impacts were observed for all technologies reviewed, at two different stages in the HTA process, and represented macro- (raising awareness) and micro-level (informing recommendations) impacts. These impacts were shaped by periodic opportunities for direct and brokered exchange between the Panel and the expert advisory committee to clarify roles, foster accountability, and build trust. Our findings offer new insights about one of the main considerations in the design of deliberative participatory structures—how to maintain the independence of a citizens’ panel while ensuring that their input is considered at key junctures in the HTA process.

Conclusions: Citizens’ panels can exert various impacts on the HTA process. Ensuring these types of structures include opportunities for direct exchange between citizens and experts, to clarify roles, promote accountability, and build trust will facilitate their impacts in a variety of settings.

Objectives: Investments in health research should lead to improvements in health and health care. This is also the remit of the main HTA program in the Netherlands. The aims of this study were to assess whether the results of this program have led to such improvements and to analyze how best to assess the impact from health research.

Methods: We assessed the impact of individual HTA projects by adapting the “payback framework” developed in the United Kingdom. We conducted dossier reviews and sent a survey to principal investigators of forty-three projects awarded between 2000 and 2003. We then provided an overview of documented output and outcome that was assessed by ten HTA experts using a scoring method. Finally, we conducted five case studies using information from additional dossier review and semistructured key informant interviews.

Results: The findings confirm that the payback framework is a useful approach to assess the impact of HTA projects. We identified over 101 peer reviewed papers, more than twenty-five PhDs, citations of research in guidelines (six projects), and implementation of new treatment strategies (eleven projects). The case studies provided greater depth and understanding about the levels of impact that arise and why and how they have been achieved.

Conclusions: It is generally too early to determine whether the HTA program led to actual changes in healthcare policy and practice. However, the results can be used as a baseline measurement for future evaluation and can help funding organizations or HTA agencies consider how to assess impact, possibly routinely. This, in turn, could help inform research strategies and justify expenditure for health research.

This reflection on the history of the International Society of Technology Assessment in Health Care is an effort to describe the creation of the Society and its first 10 years of activity. Without analyzing the forces that spurred the growth of technology assessment internationally or linking events, policies, and changes in the various countries, this essay focuses on the persons and events that surrounded the birth and growth of the Society in the past decade.

Objectives: To determine whether unpublished dissertations are an important source of trials for systematic reviews.

Methods: In a review of infant massage, we identified 17 dissertations. We recorded whether each dissertation was included in the review and, if so, whether it contributed data to any analyses.

Results: Thirteen dissertations were excluded, 11 because of poor reporting of methodology. Three dissertations were also published as journal papers. The data from the remaining dissertation were not included in any analysis. Reviewing data on the Cochrane Library, only one of 878 reviews included data from a dissertation that might have changed a review's conclusions.

Conclusions: Searching for and retrieving unpublished dissertations involves considerable time and effort and appears to influence the conclusions of a review only rarely.

This paper reports the results of a study of the association between volume of activity and patient outcome in 26 intensive care units in the United Kingdom. Hospital fatality rates showed a statistically significant (p =.016) negative association with volume. However, mean APACHE II scores, as a measure of the severity of case mix, were also negatively associated with volume (p =.021). Thus, one explanation of the lower death rates at higher volumes is that larger units admit less severely ill patients. For patients admitted immediately after surgery, the correlation between severity standardized mortality ratios and volume (while not significant, p <.1) suggests there may be a volume-output effect unexplained by severity.

Objectives: To introduce a health-related quality of life measure for home care and institutional long-term care settings based on the Minimum Data Set (MDS) and the Health Utilities Index Mark 2 (HUI2).

Methods: Health attributes of Health Related Quality of Life (HRQOL) were identified, and suitable constructs were determined. Items from the MDS were mapped to the HUI2. Scores for the Minimum Data Set Health Status Index (MDS-HSI) were calculated using the HUI2 scoring function. Measurement properties are examined and reported. HRQOL scores were compared across study populations and to an external reference population. Random samples were drawn from long-term care clients in private households (n=377), supportive housing apartments (n=80), two residential care facilities (n=166), and a chronic care hospital (n=274) in Ontario, Canada. All sampled residents were assessed for health-related items using the MDS.

Results: The MDS-HSI results provide preliminary evidence of good validity. Institutional populations had lower overall HRQOL scores than community populations. Comparisons to existing Canadian national data support construct validity.

Conclusions: The MDS-HSI provides a summary outcome measure and an indicator of health status in the six supporting attributes. Longitudinal research is required to assess the sensitivity of the measure to changes over time. Further research is also required to establish the consistency between the preference weights used in this application of the HUI2 and those that would be derived from a frail elderly population.This research was supported by the Seniors Independence Program (grant 4687-06-93-458) and the Providence Centre Foundation. Thanks to Copernicus Lodge, Providence Centre, and the Red Cross Homemaker Program for participating in this study. Also thanks to Roger Roberge and Dan Dekoker from Statistics Canada for their help in obtaining and analyzing the NPHS data. International copyright for the MDS-NH and MDS-HC is held by interRAI. International copyright for HUI2 and HUI3 materials are held by Health Utilities, Inc.