The PriTec Tool is an automatically executable multicriteria web application developed in 2009 by the Galician Health Knowledge Agency (avalia-t; ACIS) for the prioritization of technologies susceptible of post-introduction observation or obsolescence. Currently, the tool has been updated and improved to support the selection process of technologies to be assessed for inclusion into the National Public Health Care Portfolio. The aim of this work is to present the updated version of the tool (PriTec Tool 2) with the new functionalities.

The development of the tool was based on a mixed-method approach, comprising a systematic review, followed by a five-step process: (i) development of the preliminary proposal of prioritization criteria and domains; (ii) contextualization and validation of the criteria and domains by a multidisciplinary group of key stakeholders; (iii) assessment of validity, reliability and suitability of criteria; (iv) weighting of domains; and (v) evaluation of applicability, reliability and reproducibility of the tool.

The tool consists of 15 criteria categorized in 5 domains. The web application ranks the technologies through automatic computation of the weighted average of the different criteria and generates a comparative analysis of the individual or working group results. The application allows access to different options: working groups, case studies or technology comparison. It allows for individual prioritizations or managing working groups. When applied to prioritize the Spanish Network of Health Technology Assessment (HTA) Agencies yearly workplan it achieved an intraclass correlation coefficient of 0.71 (95% confidence interval 0.62, 0.88).

The updated PriTec Tool-2 can be very useful to guide decision-making regarding the assessments that would be mostly needed to ensure health, equity and sustainability. The tool stands out for its simplicity and ease of application. It is acknowledged that the tool could be of great interest to policy makers, HTA bodies and other health decision-makers worldwide.

Systematic reviews (SRs) are central to evaluating therapies but have high costs in time and money. Many software tools exist to assist with SRs, but most tools do not support the full process, and transparency and replicability of SR depends on performing and presenting evidence according to established best practices. In order to provide a basis for comparing between software tools that support SR, we performed a feature-by-feature comparison of SR tools.

We searched for SR tools by reviewing any such tool listed the Systematic Review Toolbox, previous reviews of SR tools, and qualitative Google searching. We included all SR tools that were currently functional, and required no coding and excluded reference managers, desktop applications, and statistical software. The list of features to assess was populated by combining all features assessed in four previous reviews of SR tools; we also added five features (manual addition, screening automation, dual extraction, living review, and public outputs) that were independently noted as best practices or enhancements of transparency/replicability. Then, two reviewers assigned binary ‘present/absent’ assessments to all SR tools with respect to all features, and a third reviewer adjudicated all disagreements.

Of 53 SR tools found, 29 were excluded, leaving 24 for assessment. Thirty features were assessed across six classes, and the inter-observer agreement was 86 percent. DistillerSR (Evidence Partners; n = 26/30, 87%), Nested Knowledge (Nested Knowledge; n = 25/30, 83%), and EPPI-Reviewer Web (EPPI-Centre; n = 24/30, 80%) support the most features followed by Giotto Compliance (Giotto Compliance; n = 23/30, 77%), LitStream (ICF; n = 22/30, 73%), and SRDB.PRO (VTS Software; n = 21/30, 70%). Seven tools support fewer than half of all features assessed: RobotAnalyst, SyRF, Data Abstraction Assistant, SWIFT-Review, SR-Accelerator, RobotReviewer, and COVID-NMA. Notably, only 10 tools (42%) support direct search, 7 (29%) offer dual extraction, and 13 (54%) offer living/updatable reviews.

DistillerSR, EPPI-Reviewer Web, and Nested Knowledge each offer a high density of SR-focused web-based tools. By transparent comparison and discussion regarding SR tool functionality, the medical community can choose among existing software offerings and note the areas of growth needed, most notably in the support of living reviews.

Producing new health technology assessments (HTA) can be a time-consuming process. With finite resources in HTA agencies, limited capacities in countries without formalized HTA processes, and growing interest for lifecycles approaches valuing health technologies; innovative and efficient HTA processes are needed. “Adaptive HTA”, referring to the pragmatic use of HTA methods and existing (HTA) evidence, might offer solutions. We will present the results from a scoping review that mapped existing tools, methods, practices to transfer existing HTAs; and reflect on these findings given our own experiences of adaptation processes in LMICs.

We undertook a scoping review and systematically searched five electronic databases. Inclusion of articles followed strict in- and exclusion criteria. Data extraction focused on information regarding tools, methods, and practices that could aid the transferability of HTA analysis. Here, HTAs referred to full-HTAs and other HTA products, as partial HTAs, economic evaluations, or systematic reviews. Lastly, we mapped the possible overarching factors that can affect transferability.

The search (November 2020) identified 2030 hits, of which 19 were included. Most HTA transfers followed five steps that closely resemble a de novo HTA process. The identified transferability tools, often checklists, were merely aids or a “catalyst” for the transfer and provided limited guidance for the whole transfer process. Contrastingly, we identified three frameworks that can support the whole process: European Network for HTA (EUnetHTA) Adaptation Toolkit, TRANSFER framework for systematic reviews, and paper series on systematic reviews for economic evaluations. Lastly, our findings pointed to various challenges and knowledge gaps; especially for transfers in low and middle income countries evidence is limited.

The re-use of existing evidence in HTA reports is not new; and readily part of de novo and adaptive processes. The innovative nature of adaptive HTA comes from its ability to unpack the process of adaptation and transferability. Simultaneously, this scoping review highlighted gaps in existing adaptive methods, and could aid future adaptive HTA process for experienced and new HTA-doers.

In areas where public confidence is low and there is a lack of understanding around behaviors, such as COVID-19 vaccine hesitancy, there is a need to explore novel sources of evidence. When leveraged using artificial intelligence (AI) techniques, social media data may offer rich insights into public concerns around vaccination. Currently, sources of ‘soft-intelligence’ are underutilized by policy makers, health technology assessment (HTA) and other public health research agencies. In this work, we used an AI platform to rapidly detect and analyze key barriers to vaccine uptake from a sample of geo-located tweets.

An AI-based tool was deployed using a robust search strategy to capture tweets associated with COVID-19 vaccination, posted from users in London, United Kingdom. The tool’s algorithm automatically clustered tweets based on key topics of discussion and sentiment. Tweets contained within the 12 most populated topics with negative sentiment were extracted. The extracted tweets were mapped to one of six pre-determined themes (safety, mistrust, under-representation, complacency, ineffectiveness, and access) informed using the World Health Organization’s 3Cs vaccine hesitancy model. All collated tweets were anonymized.

We identified 91,473 tweets posted between 30 November 2020 and 15 August 2021. A sample of 913 tweets were extracted from the twelve negative topic clusters. Of these, 302 tweets were coded to a vaccine hesitancy theme. ‘Safety’ (29%) and ‘mistrust’ (23%) were the most commonly coded themes; the least commonly coded was ‘under-representation’ (3%). Within the main themes, adverse reactions, inadequate assessment, and rushed development of the vaccines as key findings. Our analysis also revealed widespread sharing of misinformation.

Using an AI-based text analytics tool, we were able to rapidly assess public confidence in COVID-19 vaccination and identify key barriers to uptake from a corpus of geo-located tweets. Our findings support a growing body of evidence and confidence surrounding the use of AI tools to efficiently analyze early sources of soft-intelligence evidence in public health research.

The COVID-19 pandemic has highlighted the need for rapid assessment of potential health technologies that can improve health outcomes in COVID-19 patients, as well as helping pressurized health service provision. Medical technologies play a key role in the COVID-19 pandemic, especially diagnostic tests and respiratory technologies. This study evaluates the rapid response work that the medical technology evaluation programme (MTEP) at the National Institute for Health and Care Excellence (NICE) has done in response to the COVID-19 pandemic.

Companies routinely submit medical technologies for evaluation by NICE through HealthTech Connect, which is an online portal for devices, diagnostics and digital technologies intended for use in the NHS or wider United Kingdom health and care system. During the COVID-19 pandemic, companies were able to use a designated email address if they perceived their technology may benefit the healthcare system regarding the COVID-19 pandemic. This new system bypassed the usual full registration and data submission. All technologies were reviewed that were submitted via HealthTech connect and email between March 2020 and June 2021.

During this period, 20 technologies were submitted to MTEP. Most of these technologies were submitted via email. These technologies consisted of a mix of digital, diagnostic, and respiratory technologies. Seven technologies were selected for a rapid COVID-19 MedTech innovation briefing (MIB), with one specifically addressing issues around waiting lists because of knock-on effects of COVID-19 restricting normal clinical work. A further six technologies were not selected because of limited evidence, while one was not selected because it was not perceived as innovative. The other five technologies were progressed as normal MIBs as there was not enough evidence of potential benefits related to COVID-19 to expedite to a rapid COVID-19 MIB. In total, two technologies were selected for medical technology guidance (myCOPD and Anaconda) and are currently in development.

MTEP has responded to the COVID-19 pandemic by prioritising and producing rapid COVID-19 MIBs on technologies to improve health and social care.

A source of debate among the health technology assessment (HTA) community is what perspective should be taken in health economic evaluations. Many stakeholders advocate that a societal perspective is taken in order to include a comprehensive range of costs and outcomes and (in theory) make societally optimal decisions. The Second Panel on Cost-Effectiveness in Health and Medicine recommended that a societal perspective be presented alongside a health sector one. The Second Panel included environment as one item on its impact inventory—alongside productivity, education, and others—intended to support the use of a societal perspective. However, many HTA agencies, including the National Institute for Health and Care Excellence (NICE), have continued to use health sector-specific evaluations to inform decision-making. The presentation seeks to examine whether consideration of the environmental impact of healthcare requires/implies the formal adoption of a societal perspective in health economic analyses.

The presentation will provide an overview of the societal perspective, explaining how it differs from a health sector perspective and describing its main strengths and weaknesses. We then present policy analysis undertaken by NICE’s Science Policy and Research team to identify reasons for measuring environmental impact in HTAs and examine whether these align with the broader arguments for or against adopting a societal perspective in economic analyses.

Three reasons for considering environmental impact are identified: (i) to support parallel policies which demand healthcare system transformation against emissions targets; (ii) to ensure planetary and human health, in the future as well as the present; and (iii) to offset future healthcare resource use. We show that only the third reason aligns with arguments related to the choice of perspective for economic analyses. Moreover, this reason is arguably better aligned to maintaining a (potentially modified) health sector perspective. The implications of the results will be discussed with reference to updating reimbursement decision-making frameworks, such as those used by NICE, to account for the environmental consequences of healthcare.

In light of government and healthcare system commitments to reducing the carbon footprint of healthcare, health technology assessment (HTA) agencies are increasingly motivated to investigate how to consider environmental sustainability in their assessments and guidance. This constitutes a major departure from the existing remits and objectives of most agencies, which typically focus on improving population health outcomes. This presentation seeks to identify options for incorporating environmental impact data into HTA and to examine the main challenges, focusing on the National Institute for Health and Care Excellence (NICE) as a case study.

We present four broad approaches that could be pursued, informed by policy analysis undertaken by NICE. The strengths, weaknesses and implications of each approach are assessed.

The first option is to act as an ‘information conduit’, aggregating and distributing in a standardized format environmental impact information that is provided voluntarily by health technology manufacturers. The second is to present complementary analyses of environmental impact data, separately but alongside results from established health economic analyses (‘parallel evaluation’ model). The third is to incorporate environmental impact data into health economic analyses, for example by monetizing environmental outcomes, so that quantitative estimates of treatment value are directly affected by environmental benefits and costs (‘integrated evaluation’ model). The fourth is to create new decision-making frameworks for evaluating healthcare interventions that are not expected to improve health-related outcomes, but claim to have relative environmental benefits.

We conclude that these approaches are not mutually exclusive, and all involve some degree of benefit and risk. We explain why the parallel evaluation model may be the most appropriate approach for NICE as a first response to the increased demand for guidance on the environmental impact of health technologies. We also outline activities being undertaken by NICE and other agencies such as the Canadian Agency for Drugs and Technologies in Health to develop new methodologies for incorporating environmental impact data into their HTAs.

Climate change as the result of human action and the risks it poses to human health are well established. In healthcare there is increasing attention to climate and environmental impacts of the use of medical and health technologies. As part of a lifecycle approach, health technology assessment (HTA) needs to take climate and environmental impacts into account. In 2020, the new definition of HTA added the dimension ‘environmental aspects’, with which the value of health technologies can be determined and assessed in terms of their impact on the environment. This led several HTA organizations to explore opportunities for including environmental impacts in HTA procedures. It is, however, yet unclear how many researchers and HTA organizations are already working on this sustainability dimension, in what way, with which (international) partners, and what they have achieved as of now. Furthermore, the complex relations between the climate crisis, environmental pollution, health and care are difficult to trace, and methods are scarce. In HTA, there is an increasing need for outcome measures that, in addition to clinical utility, effectiveness, efficiency or satisfaction, also quantify the environmental impact of medical interventions (i.e., green metrics).

We report on (i) a scoping of international (research) groups and (HTA) organizations that are working on green metrics; (ii) a literature review into the state of affairs with regard to metrics and methods; and (iii) an impact analysis of possible future inclusion of green metrics in HTA procedures. We supplemented a review of (grey) literature with interviews with HTA organizations pioneering with green metrics, and we have conducted a review of available scientific literature, yielding examples of incorporation of environmental aspects into HTA and reports on practical implications.

Carbon dioxide emissions and pollution by the health sector are currently being explored as green metrics. Differences between direct and indirect environmental impacts complicate the evaluation.

Green metrics should eventually make it possible to assess sustainability in healthcare as part of a lifecycle approach.

At the United Nations Climate Change Conference (COP26) in 2021, over 40 countries made commitments to low carbon, sustainable health care. Respiratory care provides a case study to explore how existing evidenced-based guidance can inform progress towards more sustainable care pathways and technologies. Our aim is to identify whether environmental aspects of health technology assessment (HTA) are referenced in guidance and the extent to which the four principles of sustainable health care (prevention, self-care, streamlining, and low carbon technology) are promoted in guidance.

Internet searches enabled identification of current national guidance on management of respiratory diseases in English, French or Polish. Guidances were reviewed to identify references to environmental aspects of HTA and recommendations that align with each of the four sustainable healthcare principles.

Guidance on respiratory care is produced by varied stakeholders globally. Some principles of sustainable health care are frequently reflected in guidance to improve quality of care, but others are missed where environment sustainability is not considered. Reference to HTAs incorporating environmental impacts is lacking. There is limited engagement with the environmental impacts of inhalers in guidance. Guidance created by clinician groups (e.g., Greener Practice) and research networks (e.g., Centre for Sustainable Health Systems) has responded more quickly to the need to address sustainability concerns compared to guidance produced by national public bodies.

HTA organizations may need to take a broader perspective, incorporating environmental impacts in assessments. This could have an influential role in enabling evidence-informed guidance and development of sustainable care pathways and technologies. Limitations of our study were lack of evaluation of local guidance due to limited capacity, language restrictions, and subjectivity in assessing whether each sustainable healthcare principle was addressed in guidance. There may be limited transferability of our results to other specialties or settings. Further research on the sustainability impacts and relative merits of different health technologies and care pathways is required to inform HTA and guidance.

Patients are increasingly involved in the decision-making process for health technology assessment (HTA), but the question of at what stage they can be involved is still controversial. In Kazakhstan, the HTA process began in 2010. Over the past 2 years, implementation of a project to develop a priority-setting tool based on evidence-informed deliberative processes has made it possible to discuss the participation of patients in HTA. We explored the possibilities of participation of patients or a patient-oriented group in the HTA process.

Structured interviews were held with eight people with interests in HTA. Two were representatives of universities, two from a patient-oriented group, two experts in HTA, and two physicians. Interviews were held online by mobile phone or Zoom for 25–30 minutes. Question structures were formed based on the report ‘Patient Involvement in Health Technology Assessment in Europe 2010’. Seven stages were considered.

All participants partially or completely agreed with the involvement of patients at the HTA stages of identification and prioritization. One or two did not agree with their involvement at the HTA assessment, information production, internal and external review, and diffusion and dissemination stages. Challenges for patient involvement in HTA can be related to other commitments for patients and their carers, lack of financial affordability, conflict of interest, and lack of capacity of the HTA agency to involve them. Five participants agreed on challenges for patients to being meaningfully involved in decision-making on health technologies. These included understanding which institution makes the decisions, finding an interlocutor within the decision-making body, and understanding the decision-making process. Other issues were technical and language difficulties, lack of commitment from decision-makers and the legal or policy framework for patient involvement in HTA decision-making.

Patients can participate in HTA, but the HTA agency must first prepare and agree on the level of patient participation, and develop measures to reduce barriers such as language difficulties, and patient obligations.

Patient involvement (PI) has become a key priority to the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS). As part of the national strategy to promote PI, an interest group was created in 2017 to share knowledge, develop methodologies and standardize PI processes. The aim of this work is to analyze the main activities of the Patient Interest Group 5 years after its launch and to reflect on possible needs and challenges.

Narrative description and an in-depth analysis of the main activities of the Interest Group from 2017 to the present.

The group is composed of HTA researchers from the 8 regional agencies in Spain and is supported by the Ministry of Health and the RedETS council. It currently has the participation of 26 researchers, organized into different working subgroups. The initial lines of work were the analysis of the situation, the development of procedures, and the initiation of training materials for patients on HTA. At present, the main projects are the development of metrics for evaluating the impact of patient participation, the development of procedural materials to promote methodological process standardizaton (e.g., a flowchart with the main process steps, checklists, templates), and the design and piloting of virtual training for patients in HTA. New lines include the analysis of the ethical challenges of PI and the feasibility of setting up an HTA patient registration system and a patient forum to facilitate participation. In addition, the interest group has promoted the exchange of relevant information for PI and the organization of capacity building activities.

The RedETS Patient Interest Group is encouraging the development of activities, reflection on collective experiences, and tools that facilitate PI in Spain. Among the main challenges are the need to ensure the quality and applicability of PI and to analyze the views of patients who have actively participated in HTAs.

Patient preferences (PPs) are an important source of evidence in health technology assessment (HTA). However, a methodological framework to achieve their integration in decision-making is lacking. We aim to investigate the potential role of evaluative frameworks to integrate PP evidence into HTA and decision-making.

We undertook a scoping review to identify potential methodological frameworks to consider PP evidence in HTA and evidence of the acceptability of these frameworks for decision-makers. We searched PubMed, Cochrane, and the grey literature to identify relevant studies, reports, or guidance documents. We restricted our search to the use of PP rather than patient experience data and excluded articles solely relating to deliberative approaches.

Frameworks identified as having the potential to integrate PP evidence included cost-utility analysis, cost-consequence analysis (CCA), the efficiency-frontier approach, and multi-criteria decision analysis. All have been used in various HTA contexts, but not necessarily for inclusion of PP evidence. Distinct benefits and challenges of integrating PP data were identified for each framework. These included the theoretical basis of the frameworks, their ability to consider non-health as well as health outcomes, and their ability to separate outcomes based on PPs from outcomes based on population preferences. There is limited evidence and no consensus on the application of these frameworks to consider PPs in HTA or on their acceptability for decision-makers. However, CCA has the advantage that it is both based on economic decision theory and it leaves patient preferences disaggregated from population preferences in an HTA.

The frameworks identified in this review offer potential approaches to systematically and transparently integrate PPs into HTA and decision-making. Based on the review findings, we propose a research agenda to explore the potential of CCA in particular. We anticipate that our findings will augment the recommendations of the Innovative Medicines Initiative PREFER project, which are expected to report in 2022.

Symptom-checkers are digital health applications (DHA) with diagnostic algorithms. These symptom-checkers claim to improve the diagnostic process and patient guidance. After asking the user to describe the symptoms using a chatbot interface, the symptom-checkers offer a list of potential diagnoses, and/or give recommendations for appropriate action (self-care, doctor’s visit, or emergency care). Because of the growing number and increasing use of these diagnostic DHA, there is a need to evaluate the evidence.

We updated a British evidence synthesis on symptom-checkers from the National Institute for Health Research (NIHR, 2019). For the systematic update search, we selected four databases. The following endpoints were selected: effectiveness, safety, diagnostic accuracy, triage accuracy, organizational and patient-relevant endpoints. For accuracy studies included from the update search, we assessed the risk of bias (RoB) using the quality assessment tool of diagnostic accuracy studies (QUADAS-2).

The NIHR-report included 27 studies. We added 14 additional studies via update search. One randomized-controlled-trial (RCT) reported a prolonged illness duration when using symptom-checkers (statistically non-significant). No harms when using symptom-checkers were identified (six observational studies). The diagnostic accuracy ranged from 14-84.3 percent (ten observational studies), the triage accuracy ranged from 33-100 percent (eleven observational studies). For organizational endpoints, the results were inconsistent (one RCT, six observational studies). The patient perspective indicates a high usability for symptom-checkers, but the limited description of symptoms and the missing verbal interaction with health personnel were mentioned as hindering factors (nine survey-studies). The QUADAS-2 assessment for RoB was low in one, and high in seven studies.

The studies were often conducted using fictitious case-vignettes, limiting the validity of the evidence. Therefore, the results for the diagnostic and triage accuracy are insufficient to demonstrate a benefit in real-world settings. Additionally, there is a concern for misdiagnosis and overdiagnosis. We recommend a continuous monitoring of these diagnostic DHA, using high-quality studies.

Innovative digital health technologies (DHTs) may present new aspects of value that are not appropriately accounted for in current health technology assessments. In discovering what value means in the context of DHTs, multi-stakeholder collaboration is essential.

A scoping literature review was conducted to identify current value assessment criteria and proposed methodologies across three health systems: United States of America, United Kingdom, and Germany. A Delphi exercise was conducted with stakeholders from the following groups: users, healthcare practitioners, decision-makers, supply-side actors, and influencers. Based on a review of assessment frameworks in the study countries and consultations with experts from each stakeholder group, researchers proposed value constructs in five domains: health inequalities, data rights and governance, technical and security, economic characteristics, clinical characteristics, and user preferences. In Delphi round one, participants commented on the proposed constructs and submitted their own. A thematic analysis identifying key concepts and themes of the participant proposed constructs and comments was used to incorporate this information for round two. Then, participants rated each value construct on an ‘importance’ Likert scale in two decision contexts: user-facing DHTs and system-facing DHTs. In round three, participants were presented with the consensus judgement for each construct, with the opportunity to change their answer. Value constructs with equal to or greater than 70 percent consensus were included in the final framework. Rounds four and five were, respectively, value judgements on a Likert scale and a presentation of consensus for a therapeutic area to test the final framework.

Initially 32 value constructs were proposed by researchers, 20 of which were changed or removed based on round one feedback. Additional constructs were added based on participant suggestions resulting in forty-five value constructs in round two. The final framework will be available after round three closes on 20 December 2022.

The multi-stakeholder Delphi approach ensures that all suggestions and value judgements are weighted equally across stakeholder groups. The resultant value framework can be used to inform policymaking around health technology assessment of DHTs.

We explore how the scope of analysis is defined in health technology assessment (HTA) and economic evaluation and consider the potential implications of these decisions.

The scope of analysis, including decisions about which methods and domains of HTA to include in the assessment, which costs, and health outcomes are most meaningful, and which comparators are the most relevant are typically informed by the needs of the decision-maker. We undertook two systematic scoping reviews to assess: (i) to what extent systems thinking is considered in literature-based technology assessments; and (ii) how the scope of the analysis is defined in economic evaluation using Clostridioides difficile infection as an exemplar. We synthesized the findings from these reviews and offer three key observations for future research and exploration in the field of HTA.

Our scoping reviews found that the scope of analysis in economic evaluations typically focus on single interventions, often ignoring upstream and downstream interventions. Similarly, published technology assessments have narrowly defined and inconsistent scopes of analysis, with limited consideration of indirect health and non-health impacts. Three key observations for the field of HTA include: (i) economic evaluations focus on the value of single heath interventions. A focus on a single health intervention may simplify the analysis; however, will this siloed decision-making lead to optimal health resource allocation? (ii) published assessments have inconsistently defined scopes of analysis. A decision problem that focuses on the needs of the decision-maker is practical; however, will inconsistencies in perspectives across assessments create unfair conceptualizations of value? (iii) HTA is technology-focused, not patient-focused. A technology-focused HTA system aligns with the technology diffusion process; however, does this move us away from the patient-centered mandate of HTA?

The dynamic nature of HTA leads to many conceptualizations of value. Considering the potential implications of narrowly defined, inconsistent, and technology-focused scopes of analyses may have consequences on achieving a patient-centered high-quality health system.

Conducting a cost-effectiveness analysis (CEA) is resource consuming, and therefore the Dutch National Health Care Institute (ZIN) only performs those for interventions with a high budget impact. Sometimes, cost-effectiveness (CE) estimates are clearly below or far above reference values, which makes full cost-effectiveness assessments less vital. The objective of this study was to develop an efficient and simplified method to identify interventions that are clearly cost-(in)effective.

The method makes use of headroom analysis. Several HTA experts and other relevant stakeholders have been asked to provide feedback on a preliminary version of the CE signal.

The method consists of five steps. In the first step (i) the relevant willingness-to-pay threshold is determined. Reference values are used by ZIN for the maximum willingness-to-pay per incremental quality-adjusted life year (QALY), depending on burden of disease.

In next step (ii) the health gain that can realistically be obtained with the new treatment is estimated. Hereby the effect of the intervention on the clinical outcomes, quality of life and gained life years is determined to estimate the number of QALYs gained, including uncertainty. Then (iii) the societal cost maximum (SCM) of the new treatment is calculated by multiplying step 2 with step 1. In step four (iv) the incremental treatment costs are estimated looking at both the costs and savings for both treatments options for the average patient. In the final step (v) the incremental treatment costs are compared to the SCM to determine if the intervention is probably cost effective, probably not cost-effective or if a conclusion cannot be drawn.

This method has proven to be feasible and could be a valuable addition to the current cost-effectiveness assessment toolbox. The CE-signal is being validated against performing a full cost-effectiveness analysis.

The very high costs of orphan drugs, together with the uncertainties regarding their (cost-)effectiveness raise questions regarding the efficiency and legitimacy of their health technology assessment (HTA) and appraisal process. The aim of the present, qualitative study was to investigate how experts on the reimbursement of these treatments perceive the HTA and appraisal process in their country. Moreover, it aimed to provide specific conditions and practical recommendations for their improvement.

Twenty-two European experts from 19 different countries were included in a qualitative survey and semi-structured interviews. Transcripts were analyzed using the qualitative data analysis software Nvivo. A grounded theory approach was adopted to develop a set of well-defined concepts from the cyclic analysis of the empirical data.

First, analysis of the expert interviews yielded five good practices for an efficient HTA and appraisal of orphan drugs: a high level of transparency, patient involvement, a clear decision-making structure with room for flexibility, mechanisms to minimize bias and an explicit consideration of the opportunity cost. Meanwhile, participants highlighted several barriers to the overall process, such as a lack of trust between the different stakeholders and imbalances in negotiation power. In addition, the results allowed to identify a number of ‘contextual’ determinants that may undermine the legitimacy of the final decision, such as bias and the perverse effects of the orphan drug legislation. Drawing from the experts’ experiences, a toolkit was developed that includes an extensive number of specific recommendations (and conditions) for decision-makers to improve the legitimacy and efficiency of their HTA and appraisal of orphan drugs.

Overall, the results showed that decision-makers should focus on limiting the impact of the contextual determinants rather than improving the methods included in the HTA. This will contribute to further legitimize reimbursement choices for orphan drugs towards the wider public.

On a regular basis, new and innovative medicines come to the market; with uncertainties surrounding their exact effect in patients. To address these uncertainties, disease-specific registries are commonly being used. The Dutch National Health Care Institute (Zorginstituut Netherland [ZIN]) started, in collaboration with stakeholders, a national program on the management and coordination of disease-specific registries. The main goal is to improve the quality and consistency of these registries and thereby increase the value of the data in monitoring innovative, expensive medicines. As a starting point for this program, we performed a study on the current state of these registries in the Netherlands.

Using an initial list of 114 registries, we sent out questionnaires to 58 disease-specific registries that collected information on medicine use. Thirty registries responded and their registries were used for the analysis.

Of 30 registries that responded, 15 registries collect information on orphan medicinal products. Most registries are in the field of internal medicine (n = 8), oncology (n = 6), and rheumatology (n = 5). The size of the registries ranged between 250 patients (orphan diseases) and more than 10,000 patients (oncology). Only 13 registries collect information on patient reported outcomes. Data collection is mostly performed manually and standard coding systems such as Systemized Nomenclature of Medicine (SNOMED), Logical Observation Identifiers, Names, and Codes (LOINC), and The International Classification of Diseases 10th Revision (ICD-10) are not routinely used. Finally, our results show that most registries are (partly) dependent on the funding of pharmaceutical companies.

Our assessment shows variation in the type, goal of, governance, and funding of the disease-specific registries investigated. We believe that these results show the importance of further national coordination of the disease-specific registries to increase the usability of their data to address the uncertainties surrounding these innovative, expensive medicines.

The National Institute for Health and Care Excellence (NICE) interventional procedures programme (IPP) issues guidance on the safety and efficacy of new interventional procedures (IPS). The IPP considers a range of evidence from randomized controlled trials (RCTs) to case series to make final recommendations. Real-world evidence (RWE) can provide additional information on long-term outcomes and patient population characteristics that are not easily captured by RCTs. This study explores the impact of RWE in complementing RCTs on long-term safety and efficacy for national guidance development.

We review the impact of RWE in IPS guidance (IPG) updates and change in guidance recommendations in the last 5 years. This is done by analyzing NICE guidance updates and supporting evidence. A range of RWE was considered in the supporting evidence, for example, registries and clinical audits.

The addition of RWE evidence to IPP guidance update has led to significant changes in the recommendations. For example, in one recent IPG, standard arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE showed an increase in long-term adverse effects. In another recent IPG update, special arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE reported long-term inferior efficacy and safety for the new intervention compared to current standard of care. A complete overview of results of the last 5 years will also be presented at the meeting.

These findings indicate that increased availability of RWE in HTA has the potential to impact national guidance recommendations. In addition, it shows how RWE can fill the evidence base gaps created by RCTs. Such data can confirm or contradict the findings of RCTs, or generate questions needing further research, or support disinvestment in non-effective technologies.

Several diagnostic tests are often adopted into diagnostic pathways for specific indications without strong evidence to support their use. In this context, real-world prospective cohort studies in combination with decision modelling can generate evidence to support decision-making. The Early Detection of neovascular Age-Related Macular Degeneration (EDNA) study was a prospective cohort designed to assess the diagnostic accuracy and cost-effectiveness of several diagnostic monitoring tests used in routine practice for the detection of neovascular age-related macular degeneration (nAMD) in the second eye of patients being treated for unilateral disease.

Five-hundred and fifty-two participants with newly diagnosed unilateral nAMD were monitored for up to 3 years in 24 UK eye clinics. The diagnostic monitoring performance of five index tests was compared: self-reported change in visual function, Amsler test, clinic measured change in visual acuity, fundus assessment by clinical examination or colour photography, and spectral-domain optical coherence tomography (SD-OCT). The reference standard was fundus fluorescein angiography (FFA). A patient-level state transition model was used to simulate the onset of nAMD in the second eye, and assess the impact of different tests on the timing of detection and treatment, and associated costs and quality adjusted life years (QALYs) over a 25-year time-horizon.

One hundred and forty-five (26.3%) patients developed active nAMD in the study eye, of whom 120 had an FFA at detection. SD-OCT had the highest sensitivity (91.7 percent (95% CI: 85.2-95.6) and provided high specificity (87.8% (95% CI: 83.8-90.9)). It generated more QALYs and lower health and personal social care costs compared to all other monitoring tests. The combination of SD-OCT with fundus-examination provided a marginal increase in sensitivity over OCT alone, but the associated incremental cost-effectiveness ratios was >GBP 100,000 per QALY.

The efficiency of diagnostic pathways for nAMD may be improved by using SD-OCT alone to monitor the second eye of people being treated for unilateral disease. Prospective cohort studies embedded into routine practice offer value for informing decisions surrounding the use of technologies already in routine use.