Mental health problems can lead to costs and benefits in other sectors (e.g. in the education sector) in addition to the healthcare sector. These related costs and benefits are known as intersectoral costs and benefits (ICBs). Although some ICBs within the education sector have been identified previously, little is known about their extensiveness and transferability, which is crucial for their inclusion in health economics research.

The aim of this study was to identify ICBs in the education sector, to validate the list of ICBs in a broader European context, and to categorize the ICBs using mental health as a case study.

Previously identified ICBs in the education sector were used as a basis for this study. Additional ICBs were extracted from peer-reviewed literature in PubMed and grey literature from six European countries. A comprehensive list of unique items was developed based on the identified ICBs. The list was validated by surveying an international group of educational experts. The survey results were used to finalize the list, which was categorized according to the care atom.

Additional ICBs in the education sector were retrieved from ninety-six sources. Fourteen experts from six European countries assessed the list for completeness, clarity, and relevance. The final list contained twenty-four ICBs categorized into input, throughput, and output.

By providing a comprehensive list of ICBs in the education sector, this study laid further foundations for the inclusion of important societal costs in health economics research in the broader European context.

While involving patients in health technology assessment (HTA) has become increasingly common and important around the world, little is known about the optimal methods of evaluating patients’ involvement (PI) in HTA. This scoping review was undertaken to provide an overview of currently available methods for the evaluation of PI, specifically the impact of PI on HTA recommendations.

A literature search was conducted using nine databases as well as a grey literature search of the websites of 26 organizations related to the conduct, practice or research of HTA to identify articles, reports and abstracts related to the evaluation of PI impact in HTA.

We identified 1,248 unique citations, six of which met our eligibility criteria. These six records (five articles, and one report) were all published after 2012. Four assessed the impact of patient experience submissions on final HTA recommendations; one evaluated the impact of direct involvement on HTA committees, and one assessed impact of multiple forms of involvement. Methods of evaluation included quantitative analyses of reimbursement decisions, qualitative interviews with those directly involved in an assessment, surveys of patient groups and committee members, and the review of HTA reports.

Quantitative evaluation of PI based on associations with funding decisions may not be feasible or fully capture the relevant impact of PI in the assessment of health technologies. Rather, a combination of both qualitative and quantitative strategies may allow for the most comprehensive assessment of the impact of PI on HTA recommendations when possible.

Quality-adjusted life-years (QALYs) and disability-adjusted life-years (DALYs) are commonly used in cost-effectiveness analysis (CEA) to measure health benefits. We sought to quantify and explain differences between QALY- and DALY-based cost-effectiveness ratios, and explore whether using one versus the other would materially affect conclusions about an intervention's cost-effectiveness.

We identified CEAs using both QALYs and DALYs from the Tufts Medical Center CEA Registry and Global Health CEA Registry, with a supplemental search to ensure comprehensive literature coverage. We calculated absolute and relative differences between the QALY- and DALY-based ratios, and compared ratios to common benchmarks (e.g., 1× gross domestic product per capita). We converted reported costs into US dollars.

Among eleven published CEAs reporting both QALYs and DALYs, seven focused on pharmaceuticals and infectious disease, and five were conducted in high-income countries. Four studies concluded that the intervention was “dominant” (cost-saving). Among the QALY- and DALY-based ratios reported from the remaining seven studies, absolute differences ranged from approximately $2 to $15,000 per unit of benefit, and relative differences from 6–120 percent, but most differences were modest in comparison with the ratio value itself. The values assigned to utility and disability weights explained most observed differences. In comparison with cost-effectiveness thresholds, conclusions were consistent regardless of the ratio type in ten of eleven cases.

Our results suggest that although QALY- and DALY-based ratios for the same intervention can differ, differences tend to be modest and do not materially affect comparisons to common cost-effectiveness thresholds.

Traditionally, health technology assessment (HTA) focuses on assessing the impact of pharmaceutical technologies on health and care. Resources are scarce and policy makers aim to achieve effective, accessible health care. eHealth innovations are increasingly more integrated in all healthcare domains. However, how eHealth is assessed prior to its implementation in care practices is unclear. To support evidence-informed policy making, this study aimed to identify frameworks and methods for assessing eHealth's impact on health care.

The scientific literature in five bibliographical databases was systematically reviewed. Articles were included if the study was conducted in a clinical setting, used an HTA framework and assessed an eHealth service. A systematic qualitative narrative approach was applied for analysis and reporting.

Twenty-one HTA frameworks were identified in twenty-three articles. All frameworks addressed outcomes related to the technical performance and functionalities of eHealth service under assessment. The majority also addressed costs (n = 19), clinical outcomes (n = 14), organizational (n = 15) and system level aspects (n = 13). Most frameworks can be classified as dimensional (n = 13), followed by staged (n = 3), hybrid (n = 3), and business modeling frameworks (n = 2). Six frameworks specified assessment outcomes and methods.

HTA frameworks are available for a-priori impact assessment of eHealth services. The frameworks vary in assessment outcomes, methods, and specificity. Demonstrated applicability in practice is limited. Recommendations include standardization of: (i) reporting characteristics of eHealth services, and (ii) specifying assessment outcomes and methods following a stepped-approach tailored to the functional characteristics of eHealth services. Standardization might improve the quality and comparability of eHTA assessments.

We designed, developed, and implemented a new hospital-based health technology assessment (HB-HTA) program called Smart Innovation. Smart Innovation is a decision framework that reviews and makes technology adoption decisions. Smart Innovation was meant to replace the fragmented and complex process of procurement and adoption decisions at our institution. Because use of new medical technologies accounts for approximately 50 percent of the growth in healthcare spending, hospitals and integrated delivery systems are working to develop better processes and methods to sharpen their approach to adoption and management of high cost medical innovations.

The program has streamlined the decision-making process and added a robust evidence review for new medical technologies, aiming to balance efficiency with rigorous evidence standards. To promote system-wide adoption, the program engaged a broad representation of leaders, physicians, and administrators to gain support.

To date, Smart Innovation has conducted eleven HB-HTAs and made clinician-led adoption decisions that have resulted in over $5 million dollars in cost avoidance. These are comprised of five laboratory tests, three software-assisted systems, two surgical devices, and one capital purchase.

Smart Innovation has achieved cost savings, avoided uncertain or low-value technologies, and assisted in the implementation of new technologies that have strong evidence. The keys to its success have been the program's collaborative and efficient decision-making systems, partnerships with clinicians, executive support, and proactive role with vendors.

The terminology used to describe community participation in Health Technology Assessment (HTA) is contested and frequently confusing. The terms patients, consumers, public, lay members, customers, users, citizens, and others have been variously used, sometimes interchangeably. Clarity in the use of terms and goals for including the different groups is needed to mitigate existing inconsistencies in the application of patient and public involvement (PPI) across HTA processes around the world.

We drew from a range of literature sources in order to conceptualize (i) an operational definition for the “public” and other stakeholders in the context of HTA and (ii) possible goals for their involvement. Draft definitions were tested and refined in an iterative consensus-building process with stakeholders from around the world.

The goals, terminology, interests, and roles for PPI in HTA processes were clarified. The research provides rationales for why the role of the public should be distinguished from that of patients, their families, and caregivers. A definition for the public in the context of HTA was developed: A community member who holds the public interest and has no commercial, personal, or professional interest in the HTA process

There are two distinct aspects to the interests held by the public which should be explicitly included in the HTA process: the first lies in ensuring democratic accountability and the second in recognising the importance of including public values in decision making.

To synthetize the state of the art of methods for identifying candidate technologies for disinvestment and propose an evidence-based framework for executing this task.

An interpretative review was conducted. A systematic literature search was performed to identify secondary or tertiary research related to disinvestment initiatives and/or any type of research that specifically described one or more methods for identifying potential candidates technologies, services, or practices for disinvestment. An iterative and critical analysis of the methods described alongside the disinvestment initiatives was performed.

Seventeen systematic reviews on disinvestment or related terms (health technology reassessment or medical reversal) were retrieved and methods of 45 disinvestment initiatives were compared. On the basis of this evidence, we proposed a new framework for identifying these technologies based on the wide definition of evidence provided by Lomas et al. The framework comprises seven basic approaches, eleven triggers and thirteen methods for applying these triggers, which were grouped in embedded and ad hoc methods.

Although identification methods have been described in the literature and tested in different contexts, the proliferation of terms and concepts used to describe this process creates considerable confusion. The proposed framework is a rigorous and flexible tool that could guide the implementation of strategies for identifying potential candidates for disinvestment.

This study sought to explore main barriers and facilitators to implementing health technology assessment (HTA) in Kuwait from the perspective of key stakeholders.

Semi-structured qualitative interviews were conducted with ten key stakeholders: seven healthcare providers working at various departments of the Kuwaiti Ministry of Health (MOH), and three academics with substantial experience in teaching HTA or related fields. Interviews were conducted face-to-face, audio-recorded, and transcribed verbatim. Data were analyzed using an inductive thematic approach.

Participating stakeholders reported several factors that might act as a barrier to building HTA in Kuwait: minimal awareness of HTA, lack of institutional and human capacity, a fragmented healthcare system, poor communication between researchers and policy makers, the country's wealth, politics, as well as data quality, availability, and sharing. Institutionalizing HTA as a politically empowered body, enforcing its recommendation by law, and benefiting from neighboring countries' experiences were suggested as possible ways to move forward.

Studies exploring the unique challenges that high-income developing countries may face in implementing HTA are still scarce. The results of this study are consistent with evidence coming from other developing countries, while also suggesting that the abundance of financial resources in the country is a double-edged sword; it has the potential to facilitate the development of HTA capacity, but also hinders recognizing the need for it.

Consideration of ethical, legal, and social issues plus patient values (ELSI+) in health technology assessment (HTA) is challenging because of a lack of conceptual clarity and the multi-disciplinary nature of ELSI+. We used concept mapping to identify key concepts and inter-relationships in the ELSI+ domain and provide a conceptual framework for consideration of ELSI+ in HTA.

We conducted a scoping review (Medline and EMBASE, 2000–2016) to identify ELSI+ issues in the HTA literature. Items from the scoping review and an expert brainstorming session were consolidated into eighty ELSI+-related statements, which were entered into Concept Systems® Global MAX™ software. Participants (N = 38; 36 percent worked as researchers, 21 percent as academics; 42 percent self-identified as HTA experts) sorted the statements into thematic groups, and rated them on importance in making decisions about adopting technologies in Canada, from 1 (not at all important) to 5 (extremely important). We used Concept Systems® Global MAX™ software to create and analyze concept maps with four to sixteen clusters.

Our final ELSI+ map consisted of five clusters, with each cluster representing a different concept and the statements within each cluster representing the same concept. Based on the concepts, we named these clusters: patient preferences/experiences, patient quality of life/function, patient burden/harm, fairness, and organizational. The highest mean importance ratings were for the statements in the patient burden/harm (3.82) and organizational (3.92) clusters.

This study suggests an alternative approach to ELSI+, based on conceptual coherence rather than academic disciplines. This will provide a foundation for incorporating ELSI+ into HTA.

The aim of this study was to provide an overview of the methodological characteristics and compare the assessment methods applied in health technology assessments (HTAs) of public health interventions (PHIs).

We defined a PHI as a population-based intervention on health promotion or for primary prevention of chronic or nonchronic diseases. HTAs on PHIs were identified by systematically searching the Web pages of members of international HTA networks. We included only full HTA reports published between 2012 and 2016. Two reviewers extracted data on the methods used to assess effectiveness/safety, as well as on economic, social, cultural, ethical, and legal aspects using a-priori standardized tables.

We included ten HTAs provided by four different organizations. Of these, all reports assessed the effectiveness of the interventions and conducted economic evaluations, seven investigated social/cultural aspects, and four each considered legal and ethical aspects, respectively. Some reports addressed applicability, context/setting, and intervention fidelity issues in different ways. We found that most HTAs adapted their methods to some extent, for example, by including nonrandomized studies, expanding the search strategy, involving stakeholders, or applying a framework to guide the HTA process.

Our analysis provides a comprehensive overview of methods applied in HTAs on public health interventions. We found that a heterogeneous set of approaches is used to deal with the challenges of evaluating complex public health interventions.

Traditional decision rules have limitations when a new technology is less effective and less costly than a comparator. We propose a new probabilistic decision framework to examine non-inferiority in effectiveness and net monetary benefit (NMB) simultaneously. We illustrate this framework using the example of repetitive transcranial magnetic stimulation (rTMS) and electroconvulsive therapy (ECT) for treatment-resistant depression.

We modeled the quality-adjusted life-years (QALYs) associated with the new intervention (rTMS), an active control (ECT), and a placebo control, and we estimated the fraction of effectiveness preserved by the new intervention through probabilistic sensitivity analysis (PSA). We then assessed the probability of cost-effectiveness using a traditional cost-effectiveness acceptability curve (CEAC) and our new decision-making framework. In our new framework, we considered the new intervention cost-effective in each simulation of the PSA if it preserved at least 75 percent of the effectiveness of the active control (thus demonstrating non-inferiority) and had a positive NMB at a given willingness-to-pay threshold (WTP).

rTMS was less effective (i.e., associated with fewer QALYs) and less costly than ECT. The traditional CEAC approach showed that the probabilities of rTMS being cost-effective were 100 percent, 39 percent, and 14 percent at WTPs of $0, $50,000, and $100,000 per QALY gained, respectively. In the new decision framework, the probabilities of rTMS being cost-effective were reduced to 23 percent, 21 percent, and 13 percent at WTPs of $0, $50,000, and $100,000 per QALY, respectively.

This new framework provides a different perspective for decision making with considerations of both non-inferiority and WTP thresholds.

Ethics has been considered among the core domains of health technology assessment (HTA), but there are still disputes regarding ethical analysis. This study aimed to examine full final reports of the European Network for Health Technology Assessment (EUnetHTA) in terms of their compliance with the ethical methodology and ethical perspective of the HTA Core Model®.

The study examines seven full final HTA reports of EUnetHTA written based on the methodology proposed in the HTA Core Model®. The reports were analyzed using the following parameters: competency of the person/group who conducted ethical analysis, assessment elements, and the methodology of ethical analysis.

The results show that, although the HTA Core Model® helped to standardize the final reports of the assessment, there are still concerns regarding the competency of the ethical analysis team, the perspectives on the purpose of ethical analysis, data sources and viewpoints of various stakeholders, use of ethical analysis methodology, and the evaluation of the ethical appropriateness of the entire HTA process.

The HTA Core Model® helped to standardize the final reports on the HTA; however, not all issues with the content and outcomes were solved. The lack of expertise in ethics and insufficiency of the teams regarding ethical analysis are other existing problems. This study also demonstrated that stakeholder viewpoints in general and patient perspectives, in particular, have been overlooked in the HTA process.

Timely access to innovative medical technologies driven by accelerated patient access pathways can substantially improve the health outcomes of patients who often have few therapeutic alternatives. We analyzed lead-times for the medical procedure reimbursement coverage process undertaken in South Korea from 2014 to 2017, which is considered one of the most important factors contributing to delays in patient access to new medical technologies.

This analysis was performed using the open datasets source of “Medical Procedure Expert Evaluation Committee (MPEEC)” meeting results and medical procedure coverage application information published on the Health Insurance Review and Assessment Service Web site.

From 2014 to 2017, 90 percent of all new coverage determinations took on average >250 days with almost 20 percent taking more than 2 years (>750 days), The average lead-time from the medical procedure coverage application to MPEEC meeting in 2015 was 435.0 ± 214.7 days (n = 26), which was significantly shorter than the average lead-time in 2014 (624.9 ± 290.3 days, n = 16) (p < .05). The average lead-time from application to official enforcement in 2015 was significantly shorter than that of 2014 (540.8 ± 217.4; n = 16 versus 734.1 ± 299.7 days; n = 26, respectively) (p < .05).

While this analysis showed a general trend of a reduction in the time taken to receive a positive coverage determination for a new medical technology, the average lead-time remains well over the government mandated 100 days. To continue this trend and further enhance the patient access pathway for medical procedure coverage determinations, some measures can be applied. In particular, the extended “One-Stop Service” program encompassing coverage determinations is one such recommendation that could be considered.

Indirect comparisons via a common comparator (anchored comparisons) are commonly used in health technology assessment. However, common comparators may not be available, or the comparison may be biased due to differences in effect modifiers between the included studies. Recently proposed population adjustment methods aim to adjust for differences between study populations in the situation where individual patient data are available from at least one study, but not all studies. They can also be used when there is no common comparator or for single-arm studies (unanchored comparisons). We aim to characterise the use of population adjustment methods in technology appraisals (TAs) submitted to the United Kingdom National Institute for Health and Care Excellence (NICE).

We reviewed NICE TAs published between 01/01/2010 and 20/04/2018.

Population adjustment methods were used in 7 percent (18/268) of TAs. Most applications used unanchored comparisons (89 percent, 16/18), and were in oncology (83 percent, 15/18). Methods used included matching-adjusted indirect comparisons (89 percent, 16/18) and simulated treatment comparisons (17 percent, 3/18). Covariates were included based on: availability, expert opinion, effective sample size, statistical significance, or cross-validation. Larger treatment networks were commonplace (56 percent, 10/18), but current methods cannot account for this. Appraisal committees received results of population-adjusted analyses with caution and typically looked for greater cost effectiveness to minimise decision risk.

Population adjustment methods are becoming increasingly common in NICE TAs, although their impact on decisions has been limited to date. Further research is needed to improve upon current methods, and to investigate their properties in simulation studies.

It is important to capture all health effects of interventions in cost-utility analyses conducted under a societal or healthcare perspective. However, this is rarely done. Household spillovers (health effects on patients’ other household members) may be particularly likely in the context of technologies and interventions to change behaviors that are interdependent in the household. Our objective was to prospectively collect outcome data from household members and illustrate how these can be included in a cost-utility analysis of a behavior change intervention in chronic obstructive pulmonary disease (COPD).

Data were collected from patients’ household members (n = 153) alongside a randomized controlled trial of a COPD self-management intervention. The impact of the intervention on household members’ EQ-5D-5L scores (primary outcome), was evaluated. Analyses were then carried out to estimate household members’ quality-adjusted life-years (QALYs) and assess the impact of including these QALYs on cost-effectiveness.

The intervention had a negligible spillover on household members’ EQ-5D-5L scores (−0.007; p = .75). There were also no statistically significant spillovers at the 5 percent level in household member secondary outcomes. In the base-case model, the inclusion of household member QALYs in the incremental cost-effectiveness ratio (ICER) denominator marginally increased the ICER from GBP 10,271 (EUR 13,146) to GBP 10,991 (EUR 14,068) per QALY gained.

This study demonstrates it is feasible to prospectively collect and include household members’ outcome data in cost utility analysis, although the study highlighted several methodological issues. In this case, the intervention did not impact significantly on household members’ health or health behaviors, but inclusion of household spillovers may make a difference in other contexts.

Healthcare organizations have invested efforts on hospital-based health technology assessment (HB-HTA) and enterprise risk management (ERM) processes for novel systems to obtain more accurate data on which to base strategic decisions. This study proposes to analyze how HB-HTA and ERM processes can share personal resources and skills to achieve principles with value-oriented results.

Literature on ERM and HB-HTA and data from interviews with healthcare managers compose the research data sources, which were submitted to a qualitative data analysis. It was oriented to identify the association between ERM and HB-HTA application in hospitals and the common principles between both processes, in addition to proposing the capability to share personal resources between both teams in a matrix.

The common principles and personal background suggested for HB-HTA and ERM teams allowed the build of a matrix identifying how both teams can work in an integrated manner being more effective and value-oriented. The shared resource matrix reports how each professional (with a specific background) may interact with each activity associated to HB-HTA or ERM implementation guidelines.

The identification of common principles and capabilities between ERM and HB-HTA suggested advances with the literature from both research areas. The opportunity to share personal resources also contributes to the implementation of those processes in hospitals with less financial resources, approaching its own management to be more efficient with the care chain.

Very few practical frameworks exist to guide the formulation of recommendations at hospital-based health technology assessment (HTA) units. The objectives of our study were: (i) to identify decision criteria specific to the context of hospital-based health technologies and interventions, (ii) to estimate the extent to which the expert community agrees on the importance of the identified criteria, (iii) to incorporate the identified criteria into a decision-aid tool, and (iv) to illustrate the application of a prototype decision-aid tool.

Relevant decision criteria were identified using existing frameworks for HTA recommendations, our past experience, a literature search, and feedback from a survey of diverse stakeholders.

Based on the survey results, twenty-three decision criteria were incorporated into the final framework. We defined an approach that eschewed a scoring system, but instead relied on a visual means for arriving at a final recommendation, by juxtaposing the importance rating for each criterion against the results of the health technology assessment. For a technology to be approved, a majority of criteria considered important should also have received favorable findings.

We created a simple and practical decision-aid tool that incorporates all decision criteria relevant to a hospital-based HTA unit. With its ease of use and accessibility, our tool renders the subjective decision-making process more structured and transparent.

As healthcare decision makers continue to face challenges in health services delivery to their patients, disinvestment programs are being established for a sustainable healthcare system. This study aimed to collect data and information by means of a survey of disinvestment candidates and ongoing disinvestment projects in the health technology assessment (HTA) community.

An online survey was conducted to collect information on disinvestment candidates and activities from members of the Health Technology Assessment International Disinvestment & Early Awareness Interest Group, the EuroScan International Network and International Network of Agencies for Health Technology Assessment.

Among the 362 invitees, twenty-four unique responses were received, and almost 70 percent were involved in disinvestment initiatives. The disinvestment candidates identified represented a range of health technologies. Evidence or signaling of clinical ineffectiveness or inappropriate use typically led to the nomination of disinvestment candidates. Health technology assessments and reassessments were usually conducted to evaluate the technology in question, and decisions usually led to the limited use of the technology. Barriers to disinvestment decisions included the strength of interest and advocacy groups, insufficient data for assessments, a systematic decision process and political challenges, while obstacles to their implementation were clinicians’ reluctance and insufficient funding and incentives.

The survey results suggested that disinvestment activities are occurring in the HTA community, especially in the public sector. Future research can further investigate the processes and methods used to reach and implement disinvestment decisions from our survey respondents and explore to form closer ties between the HTA and clinical communities.

There is no established methodology to assess the feasibility of medicine price data sources. Against this backdrop, a framework to guide the selection of most appropriate price data sources for pharmacoeconomic research has been developed.

A targeted literature review was carried out. Dimensions discussed in literature as relevant for medicine price comparisons and practical experience of the authors in medicine price studies informed the conceptional work of the framework development. A draft version of the framework was reviewed by peer pricing experts. The feasibility of the framework was tested in case studies.

According to the developed framework (called Re-ADAPT), a medicine price data source should meet the following criteria: reliability and sustainability; accessibility at a cost that users can afford; provision of medicine price information at the date(s) required; information for the defined geographic area, or at least in a representative way; coverage of the pharmaceuticals and at the price type(s) required. Easy handling and provision of additional information were defined as supportive assets of candidate data sources (secondary criteria). The case studies confirmed the feasibility of the Re-ADAPT framework. In some cases, however, it can be difficult to disentangle assessment criteria (particularly geographic area, scope of pharmaceuticals and price types) for separate consideration, given their interlinkage.

While selection of the most appropriate data sources will remain a challenge, the Re-ADAPT framework aims to provide practical guidance and thus contribute to a more careful, balanced, and evidence-based selection of data sources for medicine price studies.

Social and cultural aspects are rarely assessed in health technology assessments (HTA), despite being part of most HTA definitions. One hypothesis for the reason why they are hardly considered in HTA is that we lack relevant assessment methods. Accordingly, this review aims at providing an overview of methodological approaches to address social and cultural aspects related to health technologies in HTA.

We conducted a comprehensive literature search by searching fourteen databases and a hand-search of two pertinent journals. Additionally, we sent a query to all member agencies of the International Network of Agencies for Health Technology Assessment (INAHTA) asking them for methods they use to assess social and cultural aspects.

A total of 125 publications met our inclusion criteria. We grouped the methodological approaches into checklists for experts, literature reviews, stakeholder participatory approaches, primary data collection methods, and combinations of methodological approaches.

There is a wide variety of methods available for assessing social and cultural aspects of health technologies, some of which have been applied in HTA. The presented overview of the different approaches and their merits can facilitate the assessment of these aspects, and improve the knowledge regarding (potential) success and failure of the implementation of a health technology.