Background: Increasingly, healthcare decision makers demand quality evidence in a short timeframe to support urgent and emergent decisions related to procurement, clinical practice, and policy. Health technology assessment (HTA) producers are responding by developing innovative approaches to evidence synthesis that can be executed more quickly than traditional systematic review. These approaches, and the broader implications they bring to bear on health decision making and policy development, however, are generally neither well-understood nor well-described. This study intends to contribute to an emerging literature around methodological approaches to rapid review in HTA by outlining those developed and implemented by the Canadian Agency for Drugs and Technologies in Health (CADTH).

Methods: Since 2005, CADTH has developed and implemented a rapid review approach that synthesizes evidence to support informed healthcare decisions and policy. Rapid Response reports are tailored to the identified needs of Canadian health decision makers, representing a range of options with regard to depth, breadth, and time-to-delivery.

Results: Preliminary observations indicate that CADTH's approach to rapid evidence review is generally well-received by Canadian health decision makers; real-world case studies provide pragmatic examples of how health decision makers have used Rapid Response reports to support evidence-informed health decisions across Canada.

Conclusions: Rapid review is becoming an increasingly important approach to evidence synthesis, both within and external to the field of HTA. Transparent reporting of the methods used to develop rapid review products will be critical to the assessment of their relevance, utility and effects in a range of contexts.

Objectives: The aim of this study was to evaluate the effectiveness of bilateral cochlear implantation (CI) compared with unilateral CI for deaf children in the context of the Republic of Kazakhstan health system.

Methods. A literature search was conducted, using the PubMed, Cochrane, and Embase data bases for studies that compared the effectiveness of bilateral and unilateral CI in children. The search included English language, publications from 2002–2012. Two reviewers independently evaluated all relevant studies. Administrative data relevant to CI in Kazakhstan were obtained from the Ministry of Health.

Results: Three relevant systematic reviews and an health technology assessment report were found. There was evidence of incremental benefits from bilateral CI but the quality of the available studies was poor and there was little information on longer term outcomes. No conclusions could be drawn regarding later incremental improvements to speech perception, learning, and quality of life. To date, in the Republic of Kazakhstan there is not full coverage of audiological screening due to the lack of medical equipment. This leads to late detection of hearing-impaired children and a long rehabilitation period, requiring more resources. Age of implantation in children is late and only a small minority attend general schools.

Conclusions: The clinical effectiveness of bilateral CI, an expensive health technology, requires further study. Given the current situation in Kazakhstan with audiological screening and access to unilateral CI, there appeared to be other priorities for improving services for children with profound hearing impairment.

Objectives: The aim of this study was to describe 8-year results from post-implementation surveillance of neuroreflexotherapy (NRT), a health technology proven effective for treating neck and back pain.

Methods: Post-implementation surveillance included all patients undergoing NRT across five regions within the Spanish National Health Service (SNHS). Validated methods were used to assess pain, disability, adverse events, use of health resources, and patient satisfaction. Logistic regression models were developed to identify the variables associated with the risk of a pain episode requiring more than one NRT intervention. The number of relapses among discharged patients during the 8-year period was calculated.

Results: Between January 1, 2004, and June 30, 2012, 9,023 patients (median age: 53 years), presenting 11,384 subacute (25.2 percent) and chronic (74.8 percent), neck or back pain episodes, were discharged after receiving NRT. Spinal pain improved in 89 percent of cases, 83 percent abandoned drugs, and 0.02 percent required spine surgery. The only adverse event was skin discomfort (8.0 percent of patients). Number of patient complaints was 0, and answers to a standardized questionnaire reflected a high degree of satisfaction (response rate: 76.7 percent). Of the pain episodes, 18.9 percent required more than one NRT intervention; logistic regression models identified the variables associated with this. Over the 8-year period, the proportion of discharged patients referred for treatment due to relapse at the same level for neck, thoracic, and low back pain, was 16.4 percent, 6.5 percent, and 14.5 percent respectively.

Conclusions: Post-marketing surveillance for a non-pharmacological technology is feasible within the SNHS. These results support generalizing NRT across the entire SNHS under the current validated application conditions.

The most recent discussions of the HTAi Policy Forum are being showcased in this issue of our Journal. The topic of adaptive approaches to technology management has been a policy area of continuing importance and relevance to health technology assessment agencies and policy makers as well as industry. The latest Policy Forum publication (Husereau et al., in this issue) demonstrates that the discussions at the meeting about this complex topic were very wide indeed.

Objectives: The aim of this study was to assess incidence, time frame, and outcome of “Coverage with Evidence Development” (CED) decisions in the Swiss Basic Health Insurance scheme.

Methods: Analysis of all controversial medical technologies submitted to review by the Swiss Federal Office of Public Health (FOPH) from 1996 to 2012 with focus on decisions with constraints. Description of types of technology, type of initial decision, duration of evaluation period, final decision, and search for potential factors associated with changes over time.

Results: Forty-five (37.5 percent) of 120 controversial health technologies were classified as “yes, in evaluation, reimbursed” for a certain period of time and thirty-five (29.2 percent) as “no, in evaluation, not reimbursed” by the Federal Department of Home Affairs from 1996 to 2012. The rate of CED decisions ranged between zero and nine per year and was influenced by type of technology and calendar year. Forty-four of forty-five decisions were subject to further restrictions, to a “center or a specialist” (76 percent), “indications” (49 percent), “registry” (31 percent), or “other” (49 percent). The time to a final decision ranged from 1.5 to 11 years (median, 6 years). No factors associated with initial decision and final outcome could be identified.

Conclusions: CED as a reality in Switzerland might have enabled patients to obtain access to promising technologies early in their life cycle. CED might have acted as a trigger to a successful implementation of a comprehensive national registry. The lack of qualitative data stresses the urgent need for evaluation of the HTA decisions and their impact on patient outcome and costs.

Objectives: Procedures and new medical devices are typically introduced into healthcare systems with limited evidence, when they might be ineffective or unsafe. Systematic data collection (“registers”) can provide valuable “real world” evidence, but difficulties in funding registers are a major obstacle. A good economic case for the value of registers would therefore be useful.

Methods: (i) Literature search on specific purposes of registers. (ii) Surveys (a) of senior clinicians involved with registers, seeking examples of beneficial outcomes, and (b) of administrators, regarding costs of running registers. (iii) A scoping exercise for possible methods to value (financially) the outputs of registers.

Results: Four main categories of beneficial outcomes from registers were identified. These were—safety and quality assurance; training and quality improvement; complementing trial evidence and reducing uncertainty; and supporting trial research. Explicit examples of all these are presented, together with information about the costs of registers. Combining these with the scoping exercise we present suggestions for a methodology of assessing the value of registers across each of the categories.

Conclusions: This study is unique in addressing methods for determining the financial value of registers, based on the amount they cost versus the financial benefits which may result from the evidence generated. Developing the suggested methods could support the case for funding new registers, by showing that their use can benefit healthcare systems through more efficient use of resources, so justifying their costs.

Objectives: The European Network for Health Technology Assessment (EUnetHTA) Joint Action Project Framework, 2010–12, Work Package 7B (WP 7B), was aimed at facilitating collaboration between HTA agencies to avoid duplication of assessment efforts. A major task of WP 7B was to collect information on planned and ongoing (POP) projects by EUnetHTA agencies and to incorporate this information in a POP Database. We analyzed whether the Database served its intended purpose.

Methods: A survey was sent to all fifty-seven EUnetHTA partners, complemented by telephone interviews with the Ludwig Boltzmann Institute of HTA in Austria (lead institution). Furthermore, detailed documentation on the activities of the POP Database was provided to the research team at CAST (University of Southern Denmark) by the lead institution.

Results: Forty-two of fifty-seven agencies (74 percent) responded to the survey. Eleven collaborations initiated by agencies themselves were reported. The scope of these collaborative activities was usually limited to information exchange on for example literature search protocols. A slight reduction of duplication of effort was documented. In addition, twelve collaborations at the full report level were initiated by the lead institution.

Conclusion: While the POP Database has the potential to reduce duplication of effort, this has not been realized during the 3-year period of the EUnetHTA Joint Action Project Framework, 2010–12. Further evidence needs to be gathered to determine whether the POP Database is effective and whether the benefits outweigh the resources required to maintain it.

Objectives: The aim of this study was to assess if the use of Markov modeling (MM) or discrete event simulation (DES) for cost-effectiveness analysis (CEA) may alter healthcare resource allocation decisions.

Methods: A systematic literature search and review of empirical and non-empirical studies comparing MM and DES techniques used in the CEA of healthcare technologies was conducted.

Results: Twenty-two pertinent publications were identified. Two publications compared MM and DES models empirically, one presented a conceptual DES and MM, two described a DES consensus guideline, and seventeen drew comparisons between MM and DES through the authors’ experience. The primary advantages described for DES over MM were the ability to model queuing for limited resources, capture individual patient histories, accommodate complexity and uncertainty, represent time flexibly, model competing risks, and accommodate multiple events simultaneously. The disadvantages of DES over MM were the potential for model overspecification, increased data requirements, specialized expensive software, and increased model development, validation, and computational time.

Conclusions: Where individual patient history is an important driver of future events an individual patient simulation technique like DES may be preferred over MM. Where supply shortages, subsequent queuing, and diversion of patients through other pathways in the healthcare system are likely to be drivers of cost-effectiveness, DES modeling methods may provide decision makers with more accurate information on which to base resource allocation decisions. Where these are not major features of the cost-effectiveness question, MM remains an efficient, easily validated, parsimonious, and accurate method of determining the cost-effectiveness of new healthcare interventions.

Objectives: The aim of this study was to evaluate cost effectiveness of coronary artery bypass graft (CABG) versus percutaneous coronary intervention (PCI) with stenting from Iran society perspective.

Methods: A retrospective study was carried out to estimate the annual cost and health related quality of life (HRQoL) of 109 patients who underwent coronary revascularization (PCI [n = 75] and CABG [n = 34]). A Markov model has been developed to determine the cost effectiveness of CABG compared with PCI. We used the model to calculate lifetime costs, life-years (LYs), and quality-adjusted life-years (QALYs) of each strategy. We also used probabilistic sensitivity analysis to test model robustness.

Results: We found that discounted QALY lived per person in CABG versus PCI group in 5 years, 10 years, and lifetime time horizon were (3.8 ± 0.13 versus 3.88 ± 0.14), (6.4 ± 0.23 versus 6.33 ± 0.22), and (8.74 ± 0.29 versus 8.33 ± 0.27), respectively. The estimated medical cost of CABG and PCI per patient in 5 years, 10 years, and lifetime time horizon were (USD 6,819 ± 765 versus 9,011 ± 1,816), (USD 8,852 ± 1,348 versus 12,034 ± 2,375), and (USD 14,037± 4,201 versus 18,798 ± 5,821), respectively. The incremental cost-effectiveness ratio results showed CABG is a dominate alternative in 10 years and lifetime time horizon.

Conclusions: This study demonstrated that despite higher initial cost and lower HRQoL, CABG is a cost-effective revascularization strategy compared with PCI for patients with multivessel coronary artery disease in long-term.

Background: Evidence-based policy making is increasingly used for better resource allocation. Queensland Department of Health has developed a new model to introduce innovative health technologies through a health technology assessment (HTA) program.

Structure: A state-wide committee and several sub-committees at health service district level were established to oversee the HTA program and to monitor the uptake of technologies. The committees are supported by a multidisciplinary secretariat comprising staff with key HTA skills.

Process: The process starts with HTA applications, which are then shortlisted according to prespecified criteria. A due diligence process adopting a rapid evidence assessment approach is used to evaluate the applications. Based on the assessment, recommendations are made using a deliberative decision-making process guided by well-recognized tools. With positive recommendation, a technology is piloted in constrained local setting before its system-wide diffusion.

Outcome: The HTA program has assisted health administrators in prioritizing their health technology agendas. It has gained trust and wide support from policy makers and is increasingly used to support funding allocations, indicating the increasing awareness of and confidence in the program.

Conclusions: The HTA program is a valuable process to assist evidence-based policy development and to guide better resource allocation.

Objectives: The E Sibling Project aims to address the needs of siblings of individuals affected by psychosis through provision of a comprehensive online intervention. The online intervention comprises four core elements, including: information on psychosis; various coping and promoting well-being strategies; siblings' blogs and discussion forum with peers; and an “Ask the Experts” function. After the intervention-prototype was developed, we tested its feasibility, usability and acceptability by siblings.

Methods: We evaluated the usability of the intervention-prototype using a non-randomised usability study with siblings of individuals diagnosed with psychosis. The usability study adapted Poulson et al's framework to collect subjective feedback from participants on ease of use, perceived usefulness and acceptability, together with objective usage data on the intervention.

Results: Twenty siblings were recruited to the usability test; 19 tried out the resource-prototype over a 4-week period and 17 completed the online evaluation after using the intervention. In total, 906 page-views were made by the participants and each spent about two hours using the resource. Participants evaluated the intervention as helpful, relevant and useful in terms of content, design and usability. Developments are needed to improve the navigation and intuitiveness of the resource.

Conclusions: Using an internet-based information-giving and peer support intervention to promote wellbeing and coping is feasible and acceptable to siblings of people with psychosis.

Objectives: Reporting bias potentially threatens the validity of results in health technology assessment (HTA) reports. Our study aimed to explore policies and practices of HTA agencies regarding strategies to include previously unpublished data in their assessments, focusing on requests to industry for unpublished data.

Methods: We included international HTA agencies with publicly available methods papers as well as HTA reports. From the methods papers and recent reports we extracted information on requests to industry and on searches in trial registries, regulatory authority Web sites and for conference abstracts.

Results: Eighteen HTA agencies and seventy-three reports were included. Agencies’ methods papers showed variability regarding requests to industry (requests are routinely carried out in seven cases, not mentioned in six, at the discretion of HTA authors in three, and based on manufacturer applications in two), which were reflected in the reports investigated. As reporting of requests was limited, it often remained unclear whether unpublished data had been received. Searches in trial registries, at regulatory authorities or for conference abstracts are described as a routine or optional part of the search strategy in the methods papers of 9, 11, and 8 included agencies, respectively. A total of 52 percent, 39 percent, and 16 percent of reports described searches in trial registries, at regulatory agencies, and hand searching of conference proceedings.

Conclusion: International HTA agencies currently differ considerably in their efforts to address the issue of unpublished data. Requests to industry may constitute one strategy to access and include unpublished data, while agencies can learn from each other concerning successful practice.

Objectives: The aim of this study was to present the updated methodological guidelines for economic evaluations of healthcare interventions (drugs, medical devices, and other interventions) in Belgium.

Methods: The update of the guidelines was performed by three Belgian health economists following feedback from users of the former guidelines and personal experience. The updated guidelines were discussed with a multidisciplinary team consisting of other health economists, assessors of reimbursement request files, representatives of Belgian databases and representatives of the drugs and medical devices industry. The final document was validated by three external validators that were not involved in the previous discussions.

Results: The guidelines give methodological guidance for the following components of an economic evaluation: literature review, perspective of the evaluation, definition of the target population, choice of the comparator, analytic technique and study design, calculation of costs, valuation of outcomes, definition of the time horizon, modeling, handling uncertainty and discounting. We present a reference case that can be considered as the minimal requirement for Belgian economic evaluations of health interventions.

Conclusions: These guidelines will improve the methodological quality, transparency and uniformity of the economic evaluations performed in Belgium. The guidelines will also provide support to the researchers and assessors performing or evaluating economic evaluations.

Objectives: Clinical endpoints are defined as valid measures of clinical benefit or harm due to treatment, that describe the impact of treatment on how a patient feels, functions, and survives. The choice of endpoints and the manner in which they are reported have a major impact on the relative effectiveness assessment (REA) of pharmaceuticals. The aim of this article is to describe the guideline development process and the key findings that set a framework for appropriate use of endpoints in REAs in Europe.

Methods: A multi-health technology assessment (HTA)-agency collaborative process in EUnetHTA JA1 was used to scope, draft, and finalize methodological guidelines for REA in Europe.

Results: Patient-relevant clinical endpoints can be broadly categorized into: mortality, morbidity and health-related quality of life. A clinical endpoint is a main symptom or sign of a disease that is clinically relevant, valid, reproducible and responsive to change. Preference is for long-term or final endpoints whenever possible. Surrogate endpoints may be used when there is compelling evidence of a clear and consistent correlation of treatment effects on the surrogate and final outcome of interest.

Conclusions: The relevance and hierarchy of the different types of clinical endpoints depend on the research question, disease, and the treatment investigated. Not only the primary endpoint, but also other relevant endpoints are assessed in comparison to adequate comparator(s). This simultaneous assessment of all relevant endpoints is a hallmark of REA.

Medical technology offers enormous potential for scalable medicine—to improve the quality and access in health care while simultaneously reducing cost. However, current medical device innovation within companies often only offers incremental advances on existing products, or originates from engineers with limited knowledge of the clinical complexities. We describe how the Hacking Medicine Initiative, based at Massachusetts Institute of Technology has developed an innovative “healthcare hackathon” approach, bringing diverse teams together to rapidly validate clinical needs and develop solutions. Hackathons are based on three core principles; emphasis on a problem-based approach, cross-pollination of disciplines, and “pivoting” on or rapidly iterating on ideas. Hackathons also offer enormous potential for innovation in global health by focusing on local needs and resources as well as addressing feasibility and cultural contextualization. Although relatively new, the success of this approach is clear, as evidenced by the development of successful startup companies, pioneering product design, and the incorporation of creative people from outside traditional life science backgrounds who are working with clinicians and other scientists to create transformative innovation in health care.

Objectives: The objective of this study was to explore the degree to which databases other than MEDLINE contribute studies relevant for inclusion in rapid health technology assessments (HTA).

Methods: We determined the extent to which the clinical, economic, and social studies included in twenty-one full and four rapid HTAs published by three Canadian HTA agencies from 2007 to 2012 were indexed in MEDLINE. Other electronic databases, including EMBASE, were then searched, in sequence, to assess whether or not they indexed studies not found in MEDLINE. Assessment topics ranged from purely clinical (e.g., drug-eluting stents) to those with broader social implications (e.g., spousal violence).

Results: MEDLINE contributed the majority of studies in all but two HTA reports, indexing a mean of 89.6 percent of clinical studies across all HTAs, and 88.3 percent of all clinical, economic, and social studies in twenty-four of twenty-five HTAs. While EMBASE contributed unique studies to twenty-two of twenty-five HTAs, three rapid HTAs did not include any EMBASE studies. In some instances, PsycINFO and CINAHL contributed as many, if not more, non-MEDLINE studies than EMBASE.

Conclusions: Our findings highlight the importance of assessing the topic-specific relative value of including EMBASE, or more specialized databases, in HTA search protocols. Although MEDLINE continues to be a key resource for HTAs, the time and resource limitations inherent in the production of rapid HTAs require that researchers carefully consider the value and limitations of other information sources to identify relevant studies.

Objectives: The aim of this study was to describe the techniques used in achieving consensus on common standards to be implemented in the EUnetHTA Information Management System (IMS); and to describe how interoperability between tools was explored.

Methods: Three face to face meetings were organized to identify and agree on common standards to the development of online tools. Two tools were created to demonstrate the added value of implementing interoperability standards at local levels. Developers of tools outside EUnetHTA were identified and contacted.

Results: Four common standards have been agreed on by consensus; and consequently all EUnetHTA tools have been modified or designed accordingly. RDF Site Summary (RSS) has demonstrated a good potential to support rapid dissemination of HTA information. Contacts outside EUnetHTA resulted in direct collaboration (HTA glossary, HTAi Vortal), evaluation of options for interoperability between tools (CRD HTA database) or a formal framework to prepare cooperation on concrete projects (INAHTA projects database).

Conclusions: While being entitled a project on IT infrastructure, the work program was also about people. When having to agree on complex topics, fostering a cohesive group dynamic and hosting face to face meetings brings added value and enhances understanding between partners. The adoption of widespread standards enhanced the homogeneity of the EUnetHTA tools and should thus contribute to their wider use, therefore, to the general objective of EUnetHTA. The initiatives on interoperability of systems need to be developed further to support a general interoperable information system that could benefit the whole HTA community.

Objectives: The aim of this study was to examine the empirical and methodological cost-effectiveness evidence of surgical interventions for breast, colorectal, or prostate cancer.

Methods: A systematic search of seven databases including MEDLINE, EMBASE, and NHSEED, research registers, the NICE Web site and conference proceedings was conducted in April 2012. Study quality was assessed in terms of meeting essential, preferred and UK NICE specific requirements for economic evaluations.

Results: The seventeen (breast = 3, colorectal = 7, prostate = 7) included studies covered a broad range of settings (nine European; eight non-European) and six were published over 10 years ago. The populations, interventions and comparators were generally well defined. Very few studies were informed by literature reviews and few used synthesized clinical evidence. Although the interventions had potential differential effects on recurrence and mortality rates, some studies used relatively short time horizons. Univariate sensitivity analyses were reported in all studies but less than a third characterized all uncertainty with a probabilistic sensitivity analysis. Although a third of studies incorporated patients’ health-related quality of life data, only four studies used social tariff values.

Conclusions: There is a dearth of recent robust evidence describing the cost-effectiveness of surgical interventions in the management of breast, colorectal and prostate cancers. Many of the recent publications did not satisfy essential methodological requirements such as using clinical evidence informed by a systematic review and synthesis. Given the ratio of potential benefit and harms associated with cancer surgery and the volume of resources consumed by these, there is an urgent need to increase economic evaluations of these technologies.

Background: Many jurisdictions delivering health care, including Canada, have developed guidance for conducting economic evaluation, often in the service of larger health technology assessment (HTA) and reimbursement processes. Like any health intervention, personalized medical (PM) interventions have costs and consequences that must be considered by reimbursement authorities with limited resources. However, current approaches to economic evaluation to support decision making have been largely developed from population-based approaches to therapy—that is, evaluating the costs and consequences of single interventions across single populations. This raises the issue as to whether these methods, as they are or more refined, are adequate to address more targeted approaches to therapy, or whether a new paradigm for assessing value in PM is required.

Objectives: We describe specific issues relevant to the economic evaluation of diagnostics-based PM and assess whether current guidance for economic evaluation is sufficient to support decision making for PM interventions.

Methods: Issues were identified through literature review and informal interviews with national and international experts (n = 10) in these analyses. This article elaborates on findings and discussion at a workshop held in Ottawa, Canada, in January 2012.

Results: Specific issues related to better guiding economic evaluation of personalized medicine interventions include: how study questions are developed, populations are characterized, comparators are defined, effectiveness is evaluated, outcomes are valued and how resources are measured. Diagnostics-based PM also highlights the need for analyses outside of economic evaluation to support decision making.

Conclusions: The consensus of this group of experts is that the economic evaluation of diagnostics-based PM may not require a new paradigm. However, greater complexity means that existing approaches and tools may require improvement to undertake these more analyses.

Objectives: The aim of this study was to analyze whether health technology assessment (HTA) reports published by SBU have influenced decisions, guidelines, clinical practice, or research priorities in Sweden.

Methods: All twenty-six SBU reports between 2006 and 2010 were analyzed. For each project, we searched publications and documentation that reflected impact on decisions, guidelines, research or clinical practice. Written documentation, before–after surveys or register-based time series data were used when available. Based on a conceptual model and on the available evidence, we determined whether HTA reports had a high, moderate, or low impact.

Results: HTA reports influenced comprehensive decisions to a high or moderate degree. In the case of fortifying flour with folic acid to a high degree. In ten cases, HTA reports were the primary source of clinical guidelines developed by the National Board of Health and Welfare (NBHW) or professional associations. In the cases of dyspepsia and gastro-esophageal reflux, as well as mild head injury, the HTA reports had a high impact on clinical practice. It was also obvious from this review that research had been initiated as a result of the knowledge gaps identified by HTA reports. In three cases, we had no adequate documentation, suggesting that the impact of the HTA report had been low.

Conclusions: Many interrelated forces change practice, but the cases presented here indicate that HTA reports have had a high impact on clinical guidelines, as well as a moderate or high impact on comprehensive decisions, the initiation of research and changes in clinical practice.