Cross-design synthesis is a new (and still evolving) strategy for providing quantitative results that capture the strengths and minimize the weaknesses of different kinds of research. The strategy, which is being developed to answer questions about the effects of treatment in medical practice, includes (a) identifying complementary research designs and studies conducted according to those designs; (b) completing an in-depth assessment of each study with respect to the chief potential bias(es) that are associated with its design; (c) making “secondary adjustments” of study results to correct known biases; and (d) developing synthesis frameworks and models that will minimize the impact of hidden biases.

In this paper we develop a framework to categorize the concepts used to estimate indirect costs in economic evaluation. We apply this framework in a literature review of economic evaluation studies. We searched all English language literature from 1994–1996. Following the application of a search algorithm, which yielded 25 articles, we abstracted information from these articles to determine the methods used to identify the relative contribution of indirect costs to the cost-effectiveness ratio, the time horizon selected by the authors, the identification of those activities that were foregone, the time given up by these activities, and the valuation placed on this time. These methods were then assessed. Indirect costs, as they have been measured, significantly influence efficiency ratios. A wide variation exists among studies in how they incorporate each of the components of indirect costs. All of the four components that were identified will affect the measurement of indirect cost. Future guidelines in this area should address the entire measurement process.

There is a growing interest in the potential use of conjoint analysis (CA) as a tool for estimating patient preferences within the economic evaluation of healthcare technologies. It has been suggested that where cost is included as one of the attributes within the exercise, CA can be used to estimate willingness to pay (WTP) for healthcare interventions indirectly within the framework of cost-benefit analysis. While recognizing the potential advantages of CA as a technique for the assessment of benefits in health care, this paper recommends several avenues of investigation that need to be carefully explored before an informed judgment can be made as to the validity of eliciting WTP values using the CA methodology. It is argued that much can be learned from the use of qualitative research techniques in examining individuals' understanding and interpretation of CA questions where cost is included as an attribute in the exercise. Additionally, further research is required in defining techniques for establishing the most appropriate levels for the cost attribute in a CA exercise and in determining the sensitivity of WTP estimates to the levels that are chosen for the cost attribute.

Background: Economic evaluation as an integral part of health technology assessment is today mostly applied to established technologies. Evaluating healthcare innovations in their early states of development has recently attracted attention. Although it offers several benefits, it also holds methodological challenges.

Objectives: The aim of our study was to investigate the possible contributions of economic evaluation to industry's decision making early in product development and to confront the results with the actual use of early data in economic assessments.

Methods: We conducted a literature research to detect methodological contributions as well as economic evaluations that used data from early phases of product development.

Results: Economic analysis can be beneficially used in early phases of product development for various purposes including early market assessment, R&D portfolio management, and first estimations of pricing and reimbursement scenarios. Analytical tools available for these purposes have been identified. Numerous empirical works were detected, but most do not disclose any concrete decision context and could not be directly matched with the suggested applications.

Conclusions: Industry can benefit from starting economic evaluation early in product development in several ways. Empirical evidence suggests that there is still potential left unused.

From the heydays of HTA in the 1970s, it has been argued that ethics should be a part of HTA. Despite more than 30 years with repeated intentions, only few HTA reports include ethical analysis, and there is little agreement on methods for integrating ethics. This poses the question of why it is so important to integrate ethics in HTA? The article analyzes ten arguments for making ethics part of HTA. The validity of the arguments depend on what we mean by “integrating,” “ethics,” and “HTA.” Some of the counterarguments explain why it has taken so long to integrate ethics in HTA and why there are so many ethical approaches. Nevertheless, some of the arguments for making ethics part of HTA appear to be compelling. Health care is a moral endeavor, and the vast potential of technology poses complex moral challenges. A thorough assessment of technology would include reflection on these moral aspects. Ethics provides such a moral reflection. Health technology is a way to improve the life of human individuals. This involves questions of what “the good life” is, and hence ethical issues. Trying to ignore such questions may inflict with the moral foundation of health care: to help people. Additionally, HTA is an evaluation, and as such also a reflection on values. Hence, there is a profound affinity between HTA and ethics. Accordingly, ethics cannot be “integrated” in HTA as ethics is already a constitutive part of HTA. However, ethics can be acknowledged and emphasized.

Objectives: To evaluate the implementation of clinical guidelines for diabetes mellitus in general practice with a specific computer-based clinical decision support system (CDSS) as part of the intervention.

Methods: Randomized study with health center as unit. General practice in Sør- and Nord-Trøndelag counties in Norway, 380,000 inhabitants. Seventeen health centers with 24 doctors and 499 patients with diabetes mellitus were in the intervention group and 12 health centers with 29 doctors and 535 patients were in the control group. Main outcome measures were group differences in fractions of patients without registrations (process evaluation) and mean group differences for the same variables (patient outcome evaluation).

Results: Statistically significant group differences were experienced for fractions of patients without registration of cigarette smoking (intervention group, 82.6%; control group 94.5%), body mass index (78.2% vs. 93.0%), and sufficient registrations for calculation of risk score for myocardial infarction (91.1% vs. 98.3%); all during 18 months. Large center variations were shown for all variables. The only statistically significant group difference was −2.3 mm Hg (95% CI, −3.8, −0.8) in diastolic blood pressure in favor of the intervention group. Statistically insignificant differences in favor of the intervention group were HbA1c, −0.1% (95% CI, −0.4, 0.1), systolic blood pressure, −1.2 mm Hg (95% CI, −4.4, 2.0). Statistically insignificant differences in favor of the control group were fractions of smokers, +3.0% (95% CI, −4.0, 10.0), body mass index, +0.3 kg/m2 (95% CI, −0.8, 1.4), risk score in female +0.1 (95% CI, −5.1, 5.2), and risk score in male +2.6 (95% CI, −14.2, 19.5).

Conclusions: Implementation of clinical guidelines for diabetes mellitus in general practice, by means of a CDSS and several procedures for implementation, did not result in a clinically significant change in doctors' behavior or in patient outcome.

Surrogate outcome measures may speed up clinical research if they can be measured earlier in a study than the primary outcome of interest. We review the justification for their use and conclude that reliance on them may be harmful. Results obtained with surrogates should therefore be regarded as preliminary. Large, definitive trials with clinically relevant outcomes should always be performed before new interventions are accepted.

Objective: To identify and compare clinical practice guideline appraisal instruments.

Methods: Appraisal instruments, defined as instruments intended to be used for guideline evaluation, were identified by searching MEDLINE (1966–99) using the Medical Subject Heading (MeSH) practice guidelines, reviewing bibliographies of the retrieved articles, and contacting authors of guideline appraisal instruments. Two reviewers independently examined the questions/statements from all the instruments and thematically grouped them. The 44 groupings were collapsed into 10 guideline attributes. Using the items, two reviewers independently undertook a content analysis of the instruments.

Results: Fifteen instruments were identified, and two were excluded because they were not focused on evaluation. All instruments were developed after 1992 and contained 8 to 142 questions/statements. Of the 44 items used for the content analysis, the number of items covered by each instrument ranged from 6 to 34. Only the instrument by Cluzeau and colleagues included at least one item for each of the 10 attributes, and it addressed 28 of the 44 items. This instrument and that of Shaneyfelt et al. are the only instruments that have so far been validated.

Conclusions: A comprehensive, concise, and valid instrument could help users systematically judge the quality and utility of clinical practice guidelines. The current instruments vary widely in length and comprehensiveness. There is insufficient evidence to support the exclusive use of any one instrument, although the Cluzeau instrument has received the greatest evaluation. More research is required on the reliability and validity of existing guideline appraisal instruments before any one instrument can become widely adopted.

Forces from communities and from health care are pushing toward more consumer involvement in health care internationally. This article addresses the philosophies and rationales behind this trend and traces the history and development of consumer advocacy in health care.

Evaluating the impact of computer-based medical information systems requires not only an understanding of computer technology but also an understanding of complex social and behavioral processes. This essay discusses the need for evaluation of health care information systems, a set of evaluation questions based on assumptions about the impact of technology on organizations, and recommendations for reducing barriers to the implementation of health care information systems.

Objectives: Systematic review of the clinical and cost-effectiveness of donepezil, rivastigmine, and galantamine for people suffering from Alzheimer's disease.

Methods: Sixteen electronic databases (including MEDLINE, the Cochrane Library, and Embase) and bibliographies of related papers were searched for published/unpublished English language studies, and experts and pharmaceutical companies were consulted for additional information. Randomized controlled trials (RCTs) and economic studies were selected. Clinical effectiveness was assessed on measurement scales assessing progression of Alzheimer's disease on the person's global health, cognition, functional ability, behavior and mood, and quality of life. Cost-effectiveness was presented as incremental cost per year spent in a nonsevere state (by Mini Mental Health State Examination) or quality-adjusted life-year.

Results: Twelve of 15 RCTs included were judged to be of good quality. Although donepezil had beneficial effects in Alzheimer's patients on global health and cognition, rivastigmine on global health, and galantamine on global health, cognition, and functional scales, these improvements were small and may not be clinically significant. Measures of quality of life and behavior and mood were rarely assessed. Adverse effects were usually mild and transient. Cost-effectiveness base case estimates ranged from £2,415 savings to £49,476 additional cost (1997 prices) per unit of effect for donepezil and a small savings for rivastigmine. Estimates were not considered robust or generalizable.

Conclusions: Donepezil, rivastigmine, and galantamine appear to have some clinical effect for people with Alzheimer's disease, although the extent to which these translate into real differences in everyday life remains unclear. Due to the nature of current economic studies, cost-effectiveness remains uncertain and the impact on different care sectors has been inadequately investigated. Further research is needed to establish the actual benefits of acetylcholinesterase inhibitors (AChEls) for people with Alzheimer's disease and their caregivers, the relationship of these changes to clinical management, and careful prospective evaluation of resource and budgetary consequences.

Objectives: The aim of this study was to describe the current issues surrounding Coverage with Evidence Development (CED). CED is characterized by restricted coverage for a new technology in parallel with targeted research when the stated goal of the research or data collection is to provide definitive evidence for the clinical or cost-effectiveness impact of the new technology.

Methods: Presented here is information summarized and interpreted from presentations and discussions at the 2008 Health Technology Assessment International (HTAi) meeting and additional information from the medical literature. This study describes the differences between CED and other conditional coverage agreements, provides a brief history of CED, describes real-world examples of CED, describes the areas of consensus between the stakeholders, discusses the areas for future negotiation between stakeholders, and proposes criteria to assist stakeholders in determining when CED could be appropriate.

Results: Payers could interpret the evidence obtained from a CED program either positively or negatively, and a range of possible changes to the reimbursement status of the new technology may result. Striking an appropriate balance between the demands for prompt access to new technology and acknowledging that some degree of uncertainty will always exist is a critical challenge to the uptake of this innovative form of conditional coverage.

Conclusions: When used selectively for innovative procedures, pharmaceuticals, or devices in the appropriate disease areas, CED may provide patients access to promising medicines or technologies while data to minimize uncertainty are collected.

Objectives: This article presents an overview of the practical methods and tools to support transnational Health Technology Assessment (HTA) that were developed and pilot tested by the European network for HTA (EUnetHTA), which involved a total of sixty-four Partner organizations.

Methods: The methods differ according to scope and purpose of each of the tools developed. They included, for example, literature reviews, surveys, Delphi and consensus methods, workshops, pilot tests, and internal/public consultation.

Results: Practical results include an HTA Core Model and a Handbook on the use of the model, two pilot examples of HTA core information, an HTA Adaptation Toolkit for taking existing reports into new settings, a book about HTA and health policy making in Europe, a newsletter providing structured information about emerging/new technologies, an interactive Web-based tool to share information about monitoring activities for emerging/new technologies, and a Handbook on HTA capacity building for Member States with limited institutionalization of HTA.

Conclusions: The tools provide high-quality information and methodological frameworks for HTA that facilitate preparation of HTA documentation, and sharing of information in and across national or regional systems. The tools will be used and further tested by partners in the EUnetHTA Collaboration aiming to (i) help reduce unnecessary duplication of HTA activities, (ii) develop and promote good practice in HTA methods and processes, (iii) share what can be shared, (iv) facilitate local adaptation of HTA information, (v) improve the links between health policy and HTA.

Background: Health systems face rising patient expectations and economic pressures; decision makers seek to enhance efficiency to improve access to appropriate care. There is international interest in the role of HTA to support decisions to optimize use of established technologies, particularly in “disinvesting” from low-benefit uses.

Methods: This study summarizes main points from an HTAi Policy Forum meeting on this topic, drawing on presentations, discussions among attendees, and an advance background paper.

Results and Conclusions: Optimization involves assessment or re-assessment of a technology, a decision on optimal use, and decision implementation. This may occur within a routine process to improve safety and quality and create “headroom” for new technologies, or ad hoc in response to financial constraints. The term “disinvestment” is not always helpful in describing these processes. HTA contributes to optimization, but there is scope to increase its role in many systems. Stakeholders may have strong views on access to technology, and stakeholder involvement is essential. Optimization faces challenges including loss aversion and entitlement, stakeholder inertia and entrenchment, heterogeneity in patient outcomes, and the need to demonstrate convincingly absence of benefit. While basic HTA principles remain applicable, methodological developments are needed better to support optimization. These include mechanisms for candidate technology identification and prioritization, enhanced collection and analysis of routine data, and clinician engagement. To maximize value to decision makers, HTA should consider implementation strategies and barriers. Improving optimization processes calls for a coordinated approach, and actions are identified for system leaders, HTA and other health organizations, and industry.

Promulgation of practice guidelines in medicine has increased interest in the structure of clinical policy making. It is argued that with a generic efinition of policy as “the rules governing the behavior of individuals or institutions,” clinical policy making is analogous to legislative policy making. Practice guidelines emphasize the advantages of making clinical policy more explicit. The structure of legislative policy making as evolved over many years to meet the challenge of making both the policies and the process of policy making explicit. Processes to promulgate clinical policies may be able to exploit this experience to improve clinical policy making and thereby retain control of the process within medicine. Generic steps are outlined for making decisions with incomplete information; synthesis of facts, vested interests, and values; involvement of stakeholders; and implementation of policy. An illustration of the use of the generic steps to make and implement a clinical policy for cesarean birth follows, with evaluations of its impact on the behavior and satisfaction of clinical stakeholders.

Objectives: The aim of our study is to compare five European drug reimbursement systems, describe similarities and differences, and obtain insight into their strengths and weaknesses and formulate policy recommendations.

Methods: We used the analytical Hutton Framework to assess in detail drug reimbursement systems in Austria, Belgium, France, the Netherlands, and Sweden. We investigated policy documents, explored literature, and conducted fifty-seven interviews with relevant stakeholders.

Results: All systems aim to balance three main objectives: system sustainability, equity and quality of care. System impact, however, is mainly assessed by drug expenditure. A national reimbursement agency evaluates reimbursement requests on a case-by-case basis. The minister has discretionary power to alter the reimbursement advice in Belgium, France, and the Netherlands. All systems make efforts to increase transparency in the decision-making process but none uses formal hierarchical reimbursement criteria nor applies a cost-effectiveness threshold value. Policies to deal with uncertainty vary: financial risk-sharing by price/volume contracts (France, Belgium) versus coverage with evidence development (Sweden, the Netherlands). Although case-by-case revisions are embedded in some systems for specific groups of drugs, systematic (group) revisions are limited.

Conclusions: As shared strengths, all systems have clear objectives reflected in reimbursement criteria and all are prepared to pay for drugs with sufficient added value. However, all systems could improve the transparency of the decision-making process; especially appraisal lacks transparency. Systems could increase the use of (systematic) revisions and could make better use of HTA (among others cost-effectiveness) to obtain value for money and ensure system sustainability.

Objectives: A systematic review of the effectiveness and costs of different guideline development, dissemination, and implementation strategies wasundertaken. The resource implications of these strategies was estimated, and a framework for deciding when it is efficient to develop and introduce clinical guidelines was developed.

Objectives: Emotional disclosure has been widely publicized as having beneficial effects on physical and psychological health. A full systematic review was undertaken, with standard health technology appraisal methods, with the aim to assess the effects of emotional disclosure on healthy participants and those with pre-existing morbidity, particularly on longer-term physical health, performance, and psychological outcomes.

Methods: Randomized controlled trials of emotional disclosure were obtained from database searches (Medline (1966–2003), Embase (1980–2003), Cochrane Library (2002, issue 4), Web of Science (1981–2003), Cinahl (1982–2003), and Theses (March 2003), Internet sites (including Professor J.W. Pennebaker's home pages), and personal contacts. Quality was assessed qualitatively and by Jadad score. Meta-analysis was conducted, using Revman 4.1 software, where more than two trials reported the same outcome.

Results: Sixty-one trials were found meeting the inclusion criteria. Most had less than 100 participants and the median Jadad score was 0. A wide variety of physical, physiological, immunological, performance, and psychological outcomes were measured, but fewer were reported. There was no clear improvement for emotional disclosure compared with controls in objectively measured physical health and most other outcomes assessed.

Conclusions: The opinion that this intervention is beneficial needs to be reassessed in light of the totality of evidence available.

Health technology assessment (HTA) currently focuses on efficiency, rather than equity, on the basis that its primary objective is to maximize population health. Yet a strict cost-effectiveness approach sometimes conflicts with important equity concerns, such as the reduction of socioeconomic health inequalities. Managing such equity–efficiency trade-offs on the basis of intuition is unsatisfactory in a democracy, as it arouses suspicions of special pleading and favoritism toward vested interests. Over the next few decades, therefore, decision making may progress through up to three further stages of development observed historically in other areas of resource allocation. Stage two involves case law, limited to principles distilled from precedent. Stage three involves codification, seeking to generalize these principles without specifying their relative weights. Finally, at stage four, quantitative trade-offs are incorporated into a formula. At stage four, deliberation centers on adjustments to the formula, which would then be applied impartially, transparently, and fair-mindedly to all future decisions. Methods already exist for valuing equity–efficiency trade-offs, based on established methodological principles for valuing trade-offs between different dimensions of health. Early findings indicate that the general public thinks that social class inequalities are more inequitable than those by smoking status, with inequalities between the sexes somewhere in between. Relative weights can be calculated from these data, although the data are not yet comprehensive enough to do this credibly for current policy purposes. In the mean time, the equity–efficiency trade-offs suggested by current decisions can be estimated using standard cost-effectiveness analysis. This is because every departure from a strict cost-effectiveness approach has an opportunity cost. The size of that opportunity cost is a test of how much weight a particular equity concern is deemed to merit.

Medical technology offers enormous potential for scalable medicine—to improve the quality and access in health care while simultaneously reducing cost. However, current medical device innovation within companies often only offers incremental advances on existing products, or originates from engineers with limited knowledge of the clinical complexities. We describe how the Hacking Medicine Initiative, based at Massachusetts Institute of Technology has developed an innovative “healthcare hackathon” approach, bringing diverse teams together to rapidly validate clinical needs and develop solutions. Hackathons are based on three core principles; emphasis on a problem-based approach, cross-pollination of disciplines, and “pivoting” on or rapidly iterating on ideas. Hackathons also offer enormous potential for innovation in global health by focusing on local needs and resources as well as addressing feasibility and cultural contextualization. Although relatively new, the success of this approach is clear, as evidenced by the development of successful startup companies, pioneering product design, and the incorporation of creative people from outside traditional life science backgrounds who are working with clinicians and other scientists to create transformative innovation in health care.