International development agendas increasingly push for access to healthcare for all through universal healthcare coverage. Health economic evaluations and health technology assessment (HTA) could provide evidence to support this but do not routinely incorporate consideration of equitable access.

We undertook an international scoping review of health economic evaluation and HTA guidelines to examine how well issues of healthcare access and equity are represented, evidence recommendations, and gaps in current guidance to support evidence generation in this area. Guidelines were sourced from guideline repositories and websites of international agencies and organizations providing best practice methods guidance. Articles providing methods guidance for the conduct of HTA, or health economic evaluation, were included, except where they were not available in English and a suitable translation could not be obtained.

The search yielded forty-seven national, four international, and nine independent guidelines, along with eighty-six articles providing specific methods guidance. The inclusion of equity and access considerations in current guidance is extremely limited. Where they do feature, detail on specific methods for providing evidence on these issues is sparse.

Economic evaluation could be a valuable tool to provide evidence for the best healthcare strategies that not only maximize health but also ensure equitable access to care for all. Such evidence would be invaluable in supporting progress towards universal healthcare coverage. Clear guidance is required to ensure evaluations provide evidence on the best strategies to support equitable access to healthcare, but such guidance rarely exists in current best practice and guidance documents.

The primary objectives of this umbrella review were to (a) quantify the relative importance, of “severity” and “rarity” criteria in health resource allocation; and (b) analyze the contextual factors influencing the relative importance. The secondary objective was to examine how “severity” and “rarity” criteria are defined.

Searches were carried out in PubMed and Embase to identify eligible systematic reviews. Quality appraisal of systematic reviews was undertaken. From identified systematic reviews, primary studies were extracted and further screened for eligibility. The inclusion of severity and rarity criteria and their respective weights in primary studies were examined. Descriptive and regression analyses were performed.

Twenty-nine systematic reviews were screened, of which nine met the inclusion criteria. Primary studies included in these systematic reviews were retrieved and screened, resulting in forty articles included in the final analysis. Disease severity was more frequently considered (n = 29/40) than disease rarity (n = 23/40) as an evaluation criterion. Out of all cases where both were included as evaluation criteria, disease severity was assigned higher weights 84 percent of the time (n = 21/25).

Our review found consistent evidence that disease severity is more relevant and preferred to rarity as a priority-setting criterion albeit constraints in statistical analysis imposed by limited sample size and data availability. Where funding for rare diseases is concerned, we advocate that decision-makers be explicit in clarifying the significance of disease severity and/or rarity as a value driver behind decisions. Our findings also reinforce the relevance of disease severity as a criterion in priority setting.

Health Technology Assessment (HTA) practitioners recognize the significance of qualitative methodologies that focus on how a technology is feasible, meaningfulness, acceptable, and equitable. This mapping aimed to delineate the frameworks employed to synthesize qualitative evidence and assess the quality of synthesis in HTA .

Mapping was conducted using Medline, LILACS, CINAHL, Embase, Web of Science, Scopus, PsycINFO, Cochrane Library, JBI, and ScienceDirect databases. Gray literature searches included PROQUEST, Open Grey, Canadian Agency for Drugs and Technologies in Health’s Grey Matters, Google Scholar, and HTA agency websites. The inclusion criteria were centered on global qualitative evidence synthesis frameworks. The data are presented in the tables.

Of the 2054 articles, 31 were included, mostly from Europe. Guide was the type of document more cited, and most authors are from HTA agencies and universities. Incorporating both patient and family perspectives is the most cited reason for include qualitative evidence. Regardless of the framework or tool, SPICE was the main acronym, and RETREAT was preferred for approach selection. Thematic synthesis dominated analytic methods, and CASP was the primary quality appraisal tool. GRADE-CERQual graded evidence synthesis, with ENTREQ as the top reporting guidance. The GRADE evidence-to-decision framework was mentioned for recommendations.

This mapping highlights the movement incorporate qualitative evidence in HTA employing specific frameworks. Despite the similarities among documents, most of them describe part of the process to synthesize qualitative evidence. Standardizing procedures to incorporate qualitative evidence into HTA can enhance decision-making. These findings offer essential considerations for HTA practice.

The costs of low back pain (LBP) are complex and difficult to estimate. This study aims to adapt the Cost for Patients Questionnaire (CoPaQ) for use in LBP populations.

In a cross-sectional qualitative study, we conducted cognitive interviews to assess the CoPaQ’s suitability for addressing costs related to LBP. Three groups of participants were included (n = 5 each): (i) persons with a history of LBP or primary caregiver, (ii) researchers with expertise in LBP, and (iii) primary care providers specialized in treating LBP. The interpretation, analysis, and summary of results used Knafl et al.’s qualitative content analysis method.

Persons with a history of LBP (n = 5), had a median age of 60 years (Interquartile Range (IQR): 26–71.5), and varying durations of LBP, the median duration of LBP 7 years (IQR: 4–32.5). Researchers (n = 5) had a median age of 33 years (IQR: 29–45). Primary care providers (n = 5) had a median age of 40 years (IQR: 37.5–65), and a background in chiropractic care (n = 3) and physiotherapy (n = 2). Content analysis of the interviews revealed sources of error with five pre-determined themes (clarity/comprehension, relevance, inadequate response definition, reference point, perspective modifiers) and one developed theme (organization). We modified the questionnaire for LBP populations based on the feedback.

Our study evaluated the content validity of a questionnaire that assesses the direct and indirect costs associated with LBP. Future studies should pilot this questionnaire with persons of varying LBP severity and compare it with cost diaries.

A conceptual framework, called Innovation of Health Technology Assessment Methods (IHTAM), has been developed to facilitate the understanding of how to innovate methods of health technology assessment (HTA). However, the framework applicability has not been evaluated in practice. Hence, we aimed to explore framework applicability in three cases of method innovation that are part of the HTx project and to develop a roadmap to improve framework applicability.

The IHTAM framework was applied to three cases of innovating HTA methods. We collected feedback from case study leaders and consortium members after a training session, an approximately 1-year follow-up of periodic case study meetings, and a general assembly meeting where innovation progresses of the three cases were reported through surveys and interviews. Feedback was then summarized using an open-coding technique.

According to feedback, the framework provided a structured way of deliberation and helped to improve collaboration among HTA stakeholders. However, framework applicability could be improved if it was complemented by a roadmap with a loop structure to provide tailored guidance for different cases, and with items to elaborate actions to be taken by stakeholders. Accordingly, a 48-item roadmap was developed.

The IHTAM framework was generally applicable to the three case studies. A roadmap, with loop structure and actionable items, could complement the framework, and may provide HTA stakeholders with tailored guidance on developing new methods. To further examine the framework applicability, we recommend stakeholders to apply the IHTAM framework and its roadmap in future practice.

The sandbox approach, developed in the financial technologies sector, creates an environment to collaboratively develop and test innovative new products, methods and regulatory approaches, separated from business as usual. It has been used in health care to encourage innovation in response to emerging challenges, but, until recently, has not been used in health technology assessment (HTA). This article summarizes our learnings from using the sandbox approach to address three challenges facing HTA organizations and to identify implications for the use of this approach in HTA.

We identified three challenging contemporary HTA-related topics to explore in a sandbox environment, away from the pressures and interests of “live” assessments. We convened a pool of 120 stakeholders and experts to participate in various sandbox activities and ultimately co-develop solutions to help HTA organizations respond to the identified challenges.

Important general learnings about the potential benefits and implementation of a sandbox approach in HTA were identified. Consequently, we developed recommendations to guide its use, including how to implement an HTA sandbox in an effective way and the types of challenges for which it may be best suited.

For many HTA organizations, it is difficult to carefully consider emerging challenges and innovate their processes due to risks associated with decision errors and resource limitations. The sandbox approach could reduce these barriers. The potential benefits of addressing HTA challenges in a collaborative “safe space” are considerable.

The management of rare diseases is rarely addressed among policymakers and public communities. It is hindered by the lack of information on its epidemiology and burden, especially from the perspective of patients and families with rare diseases. This study aims to understand the perceptions of rare disease patients and their families in the management of rare diseases in Malaysia.

A qualitative interview was used to explore the perceptions of patients and families regarding the management of rare diseases in Malaysia. In-depth interviews were conducted with the rare disease patients or their parents/guardians provided by three major rare disease advocacy groups, between 1 July and 15 September 2016. The interviews focused on two key areas: the challenges associated with rare disease and the issues related to accessing medication.

Out of the nineteen recruited participants, seventeen (89.5 percent) completed the interview sessions. The significance of awareness, knowledge, and support from others emerged as crucial for families and patients living with rare diseases. Despite facing delayed diagnosis and treatment, a majority of patients and parents expressed satisfaction with the advancements in rare disease management. Nevertheless, a prominent challenge revolves around access to enzyme replacement therapy for eligible patients.

This study emphasizes the importance of healthcare professionals understanding patient with rare diseases perceptions to tailor communication strategies, provide accurate information, and address concerns effectively. The message underscores the significance of collaboration between healthcare providers and patient support groups to deliver adequate health information, potentially enhancing patients’ understanding and their illness perceptions.

We draw from the Health Technology Assessment (HTA) literature to propose how hospitals and local health networks can prepare the key components of early economic evaluations to support the development and management of health service interventions.

Using the case example of a proposed intervention for older people in the Emergency Department (ED), a conceptual logic model of a new health service intervention is articulated to inform the structuring and population of a decision-analytic model using observed data on the existing care comparator and structured elicitation exercise of initial stakeholder expectations of intervention effects.

The elicited patient pathway probabilities and lengths of stay quantities profile which of the existing types of patients are expected to avoid the ED and how this impacts the lengths of stay across the system. The exercise also quantifies the stakeholders’ uncertainty and disagreement, with qualitative insights into why. The elicitation exercise participants draw upon the rationale for how the intervention is expected to affect a change within the local context, as captured within the logic model, together with the descriptive analyses of the characteristics and utilization of their target population. Feedback indicates the methods are acceptably robust yet pragmatic enough for healthcare delivery settings.

As proposed in this paper, HTA methods can be used to capture how key stakeholders initially expect a service intervention to affect a change within their local context. The example results can be used in a decision-analytic model to guide the development and management of an intervention.

Our objective was to explore procedures and methods used at health technology assessment (HTA) agencies for assessing medical devices and the underlying views of HTA practitioners about appropriate methodology to identify challenges in adopting new methodologies for assessing devices. We focused on the role of normative commitments of HTA practitioners in the adoption of new methods.

An online survey, including questions on procedures, scoping, and assessments of medical devices, was sent to members of the International Network of Agencies for Health Technology Assessment. Interviews were conducted with survey respondents and HTA practitioners involved in assessments of transcatheter aortic valve implantation to gain an in-depth understanding of choices made and views about assessing medical devices. Survey and interview questions were inspired by the “values in doing assessments of health technologies” approach towards HTA, which states that HTA addresses value-laden questions and information.

The current practice of assessing medical devices at HTA agencies is predominantly based on procedures, methods, and epistemological principles developed for assessments of drugs. Both practical factors (available time, demands of decision-makers, existing legal frameworks, and HTA guidelines), as well as commitments of HTA practitioners to principles of evidence-based medicine, make the adoption of a new methodology difficult.

There is a broad recognition that assessments of medical devices may need changes in HTA methodology. In order to realize this, the HTA community may require both a discussion on the role, responsibility, and goals of HTA, and resulting changes in institutional context to adopt new methodologies.

Computerized clinical decision support software (CDSS) are digital health technologies that have been traditionally categorized as medical devices. However, the evaluation frameworks for traditional medical devices are not well adapted to assess the value and safety of CDSS. In this study, we identified a range of challenges associated with CDSS evaluation as a medical device and investigated whether and how CDSS are evaluated in Australia.

Using a qualitative approach, we interviewed 11 professionals involved in the implementation and evaluation of digital health technologies at national and regional levels. Data were thematically analyzed using both data-driven (inductive) and theory-based (deductive) approaches.

Our results suggest that current CDSS evaluations have an overly narrow perspective on the risks and benefits of CDSS due to an inability to capture the impact of the technology on the sociotechnical environment. By adopting a static view of the CDSS, these evaluation frameworks are unable to discern how rapidly evolving technologies and a dynamic clinical environment can impact CDSS performance. After software upgrades, CDSS can transition from providing information to specifying diagnoses and treatments. Therefore, it is not clear how CDSS can be monitored continuously when changes in the software can directly affect patient safety.

Our findings emphasize the importance of taking a living health technology assessment approach to the evaluation of digital health technologies that evolve rapidly. There is a role for observational (real-world) evidence to understand the impact of changes to the technology and the sociotechnical environment on CDSS performance.

This study examined the application, feasibility, and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results.

We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between January 2011 and May 2023. We used Naïve Bayes, Lasso, and Support Vector Machine models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results, we tested hypotheses in relation to the appraisal guidance, advanced therapy medicinal product (ATMP) status, disease area, and age group.

The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in highly specialized technology (HST) appraisals compared to appraisals from the technology appraisal (TA) guidance (adjusted odds ratio = 1.44, 95 percent CI 1.09, 1.90, p = 0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model.

Using supervised learning models for text classification in NICE appraisals for RDTs is feasible, but the results of downstream analyses may be sensitive to the choice of classification model.

Published evidence on health service interventions should inform decision-making in local health services, but primary effectiveness studies and cost-effectiveness analyses are unlikely to reflect contexts other than those in which the evaluations were undertaken. A ten-step framework was developed and applied to use published evidence as the basis for local-level economic evaluations that estimate the expected costs and effects of new service intervention options in specific local contexts.

Working with a multidisciplinary group of local clinicians, the framework was applied to evaluate intervention options for preventing hospital-acquired hypoglycemia. The framework included: clinical audit and analyses of local health systems data to understand the local context and estimate baseline event rates; pragmatic literature review to identify evidence on relevant intervention options; expert elicitation to adjust published intervention effect estimates to reflect the local context; and modeling to synthesize and calibrate data derived from the disparate data sources.

From forty-seven studies identified in the literature review, the working group selected three interventions for evaluation. The local-level economic evaluation generated estimates of intervention costs and a range of cost, capacity and patient outcome-related consequences, which informed working group recommendations to implement two of the interventions.

The applied framework for modeled local-level economic evaluation was valued by local stakeholders, in particular the structured, formal approach to identifying and interpreting published evidence alongside local data. Key methodological issues included the handling of alternative reported outcomes and the elicitation of the expected intervention effects in the local context.

Telemedicine may improve healthcare access and efficiency if it demands less clinician time than usual care. We sought to describe the degree to which telemedicine trials assess the effect of telemedicine on clinicians’ time and to discuss how including the time needed to treat (TNT) in health technology assessment (HTA) could affect the design of telemedicine services and studies.

We conducted a scoping review by searching clinicaltrials.gov using the search term “telemedicine” and limiting results to randomized trials or observational studies registered between January 2012 and October 2023. We then reviewed trial registration data to determine if any of the outcomes assessed in the trials measured effect on clinicians’ time.

We found 113 studies and of these 78 studies of telemedicine met the inclusion criteria and were included. Nine (12 percent) of the 78 studies had some measure of clinician time as a primary outcome, and 11 (14 percent) as a secondary outcome. Four studies compared direct measures of TNT with telemedicine versus usual care, but no statistically significant difference was found. Of the sixteen studies including indirect measures of clinician time, thirteen found no significant effects, two found a statistically significant reduction, and one found a statistically significant increase.

This scoping review found that clinician time is not commonly measured in studies of telemedicine interventions. Attention to telemedicine’s TNT in clinical studies and HTAs of telemedicine in practice may bring attention to the organization of clinical workflows and increase the value of telemedicine.

The purpose of this study is to evaluate the validity of the standard approach in expert judgment for evaluating precision medicines, in which experts are required to estimate outcomes as if they did not have access to diagnostic information, whereas in fact, they do.

Fourteen clinicians participated in an expert judgment task to estimate the cost and medical outcomes of the use of exome sequencing in pediatric patients with intractable epilepsy in Thailand. Experts were randomly assigned to either an “unblind” or “blind” group; the former was provided with the exome sequencing results for each patient case prior to the judgment task, whereas the latter was not provided with the exome sequencing results. Both groups were asked to estimate the outcomes for the counterfactual scenario, in which patients had not been tested by exome sequencing.

Our study did not show significant results, possibly due to the small sample size of both participants and case studies.

A comparison of the unblind and blind approach did not show conclusive evidence that there is a difference in outcomes. However, until further evidence suggests otherwise, we recommend the blind approach as preferable when using expert judgment to evaluate precision medicines because this approach is more representative of the counterfactual scenario than the unblind approach.

The objective was to identify and describe the published guidance and current academic discourse of ethical issues and standards related to the use of Social Media Research for generating patient insights for the use by health technology assessment (HTA) or health policy decisions.

A scoping review of the literature was conducted in PubMed and Embase and identified 935 potential references published between January 2017 and June 2021. After title and abstract screening by three reviewers, 40 publications were included, the relevant information was extracted and data were collected in a mind map, which was then used to structure the output of the review.

Social Media Research may reveal new insights of relevance to HTA or health policies into patient needs, patient experiences, or patient behaviors. However, the research approaches, methods, data use, interpretation, and communication may expose those who post the data in social media channels to risks and potential harms relating to privacy, anonymity/confidentiality, authenticity, context, and rapidly changing technologies.

An actively engaged approach to ensuring ethical innocuousness is recommended that carefully follows best practices throughout planning, conduct, and communication of the research. Throughout the process and as a follow-up, there should be a discourse with the ethical experts to maximally protect the current and future users of social media, to support their trust in the research, and to advance the knowledge in parallel to the advancement of the media themselves, the technologies, and the research tools.

The study, conducted by the French National Authority for Health (HAS), aimed to identify available online health technology assessment (HTA) training tools for patients, specifically those used by HTA bodies (HTAbs) and major selected European and international patient and consumer groups (PCGs), to inform an HTA training approach for HAS.

A literature search, a review of selected websites, semi-structured interviews with HTAbs and patient groups, and discussion within a dedicated working group to help target the needs and preferences of patients and consumers were conducted. Online HTA training tools relating to HTA and patient and public involvement (PPI), published or translated into English or French, were included in the study results.

Eighty-two online HTA training tools for patients and consumers were selected according to the specified inclusion criteria coming from sixteen international HTAbs, nine European and international PCGs, and thirteen other organizations. Two main categories emerged: the first relating to HTA and the second relating to PPI in HTA. The formats of these tools ranged from interactive and non-interactive formats with varying accessibility and assessment methods. No journal articles mentioned explicitly the content and format of PCG training tools.

This research served as a basis for HAS to develop their own HTA training tools and materials for patients and consumers. Two training tools were subsequently developed, guided by the needs and preferences of a patient and consumer working group, and were published on the HAS website in November 2022.

Horizon scanning for health technology appraisal (HTA) in England involves topic notification to the National Institute for Health and Care Excellence (NICE) via technology briefings. This activity is undertaken by the Innovation Observatory with submission timelines designed to ensure that HTA decisions align with regulatory approval time. In this paper, we aimed to track and assess the progression and current status of the topics notified for HTA and provide a descriptive analysis of these topics.

Technology briefings were mapped from submission to NICE technology appraisal/highly specialized technologies recommendations from April 2017 until October 2021. This was done using a combination of searches on Google and NICE website, searching a downloadable spreadsheet containing NICE topic selection decisions, and querying NICE Topic Selection team. Analysis was undertaken regarding type of indications and interventions of submitted topics and published guidance.

Six-hundred and ninety-three topics entered the NICE scoping process, of which 94 percent were prioritized. As of November 2021, approximately 39 percent of prioritized topics were in scoping/in progress, 31 percent were proposed/completed, 20 percent were suspended/terminated, and 4 percent were referred back to Innovation Observatory (IO) for further monitoring.

Our work demonstrates that horizon scanning for HTA is a complex and time-intensive process. Timelines and progress through HTA is challenging due to the growing number of innovative medicines, significant uncertainties, and limited transparency in clinical development and regulatory pathways. A better understanding of clinical trials and regulatory requirements may help eliminate some of this uncertainty and improve timely HTA.

Though there have been longstanding discussions on the value of ethics in health technology assessment (HTA), less awareness exists on ethics information retrieval methods. This study aimed to scope available evidence and determine current practices for ethics information retrieval in HTA.

Literature searches were conducted in Ovid MEDLINE, LISTA, Scopus, and Google Scholar. Once a list of relevant articles was determined, citation tracking was conducted via Scopus. HTA agency websites were searched for published guidance on ethics searching, and for reports which included ethical analyses. Methods sections of each report were analyzed to determine the databases, subject headings, and keywords used in search strategies. The team also reached out to information specialists for insight into current search practices.

Findings from this study indicate that there is still little published guidance from HTA agencies, few HTAs that contain substantial ethical analysis, and even less information on the methodology for ethics information retrieval. The researchers identified twenty-five relevant HTAs. Ten of these reports did not utilize subject-specific databases outside health sciences. Eight reports published ethics searches, with significant overlap in subject headings and text words.

This scoping study of current practice in HTA ethics information retrieval highlights findings of previous studies—while ethics analysis plays a crucial role in HTA, methods for literature searching remain relatively unclear. These findings provide insight into the current state of ethics searching, and will inform continued work on filter development, database selection, and grey literature searching.

This study aimed to assess whether there have been changes in the quality of clinical evidence submitted for government subsidy decisions on cancer medicines over the past 15 years.

We reviewed public summary documents (PSDs) reporting on subsidy decisions made by the Pharmaceutical Benefits Advisory Committee (PBAC) from July 2005 to July 2020. Information was extracted on the study design, directness of comparison, sample size, and risk of bias (RoB). Changes in the quality of evidence were assessed using regression analysis.

Overall, 214 PSDs were included in the analysis. Thirty-seven percent lacked direct comparative evidence. Thirteen percent presented observational or single-arm studies as the basis for decisions. Among PSDs presenting indirect comparisons, 78 percent reported transitivity issues. Nearly half (41 percent) of PSDs reporting on medicines supported by head-to-head studies noted there was a moderate/high/unclear RoB. PSDs reporting concerns with RoB increased by a third over the past 7 years, even after adjusting for disease rarity and trial data maturity (OR 1.30, 95% CI: 0.99, 1.70). No time trends were observed regarding the directness of clinical evidence, study design, transitivity issues, or sample size during any of the analyzed periods.

Our findings indicate that the clinical evidence supplied to inform funding decisions for cancer medicines is often of poor quality and has been deteriorating over time. This is concerning as it introduces greater uncertainty in decision making. This is particularly important as the evidence supplied to the PBAC is often the same as that supplied to other global decision-making bodies.