The Scottish Health Technologies Group (SHTG) set out to assess the impact of HTA products. Two questions were posed: Does advice from SHTG have influence? How is SHTG advice used?

SHTG adapted a tool developed by the International Network of Agencies for Health Technology Assessment (INAHTA). The INAHTA framework investigates indications of impact and categorizes outputs into levels of impact. Over three years, potential users of SHTG advice were contacted six to twelve months after advice was published and asked how the advice had been used. HTA outputs were categorized into the four levels of influence they achieved: ‘major influence’, ‘some influence’, ‘some consideration’ and ‘no known influence’.

HTA products were found to have been used in four main ways: ‘informed discussion’, ‘referenced’, ‘informed policy’ or ‘directly informed practice’. Levels of influence had steadily increased over the three years assessed. The findings were well received by internal audiences, with particular interest in the various ways HTA recommendations had been used. There was also feedback about ‘marking our own homework’. These results have informed a new SHTG strategy and supported clear messaging around the value of HTA.

SHTG has found a pragmatic, resource-light way to explore the impact of HTA outputs, which has proved valuable for driving strategy and messaging.

Managed Entry Agreements (MEAs) are increasingly used to address uncertainties arising in the Health Technology Assessment (HTA) process due to immature evidence of new, high-cost medicines on their real-world performance and cost-effectiveness. The literature remains inconclusive on the HTA decision-making factors that influence the utilization of MEAs. We aimed to assess if the uptake of MEAs differs between countries and if so, to understand which HTA decision-making criteria play a role in determining such differences.

All oncology medicines approved since 2009 in Australia, England, Scotland, and Sweden were studied. Four categories of variables were collected from publicly available HTA reports of the above drugs: (i) Social Value Judgments (SVJs), (ii) Clinical/Economic evidence submitted, (iii) Interpretation of this evidence, and (iv) Funding decision. Conditional/restricted decisions were coded as Listed With Conditions (LWC) other than an MEA or LWC including an MEA (LWCMEA). Cohen's κ-scores measured the inter-rater agreement of countries on their LWCMEA outcomes and Pearson's chi-squared tests explored the association between HTA variables and LWCMEA outcomes.

A total of 74 drug-indication pairs were found resulting in n = 296 observations; 8 percent (n = 23) were LWC and 55 percent (n = 163) were LWCMEA. A poor-to-moderate agreement existed between countries (−.29 < κ < .33) on LWCMEA decisions. Cross-country differences within the LWCMEA sample were partly driven by economic uncertainties and largely driven by SVJs considered across agencies.

A set of HTA-related variables driving the uptake of MEAs across countries was identified. These findings can be useful in future research aimed at informing country-specific, “best-practice” guidelines for successful MEA implementation.

The Department of Management and Incorporation of Technologies and Innovation in Health (DGITIS) acts as Conitec's Executive Secretariat. Among its attributions, it promotes the public/patient involvement in the health technology assessment (HTA) process. Recently, Conitec has been working on the inclusion of patient's testimonials about their illness experience in the plenary sessions, that is, the monthly meeting where technologies are assessed.

To support the action of including patient reporting in Conitec's HTA process, DGITIS developed research on HTA agencies websites worldwide. The main criteria was the inclusion of patients’ reports in their Committee meetings. DGITIS contacted some of these agencies and requested a listserv question to the International Network of Agencies for Health Technology Assessment (INAHTA) members. These findings supported the DGITIS for the inclusion of patient participation in Conitec's meetings, from the selection process to the actual participation.

For the Conitec's HTA process, the patients’ participation should occur in the prior session to the public consultation, guaranteeing the inclusion of their perspective since the recommendation process beginning. Hence, every demand for incorporation to be discussed at Conitec's meeting should be preceded by a public call for patients with the clinical condition. The DGITIS will also hold preparatory meetings, which will serve as moments for shared construction of knowledge and literacy.

The nomination process, so far, has been grounded as a consensus among the patients. Thus, Conitec acts as a mediator, connecting the involved stakeholders, in a way that they can autonomously organize themselves and indicate the main representative and an alternate one. With the inclusion of the patient's perspective in the Conitec's meeting, another form of patient participation was opened in the HTA process. Therefore, the consolidation of this participation space is feasible and contributes to enrich the Brazilian HTA process.

Medical devices play an essential role in health care, but they are also a leading causes of increasing healthcare expenditures. The purchase of technologies and the determination of how and when they should be used are among the most important decisions made by decision-makers, at the institutional level.

The present research focuses on the Portuguese health system and sheds light on the characterization of decision-making process by those involved in Magnetic Resonance Imaging (MRI) purchases.

To characterize the decision-making process, results from forty questionnaires and twenty-seven semi-structured interviews with key decision-makers were merged, using a mixed method approach. To assess competences for decision-making, a questionnaire was applied, and Exploratory and Confirmatory Factorial Analysis conducted.

Cost and suppliers’ characteristics are seen as the most important indicators to guide decisions. The decision is undertaken by a committee, in a bottom-up process, characterized by a bounded rationality, influenced by intuition and a consultant decision-maker. The reasoning and justification for selection of the committee members is unclear. The decision process is considered to be bureaucratic, time-consuming and long. Patients are negatively perceived as stakeholders in the process. Few studies were performed (mostly related to the workload of the Radiology Department) to support the decision and no national or international health technology assessment (HTA) study was used in the process, to guide decisions. Decision-makers have limited knowledge and training in areas of decision-making in the areas of health informatics, health economics and especially HTA. This may limit their ability to truly understand the future implications of their purchase decisions.

To foster HTA in decision-making processes, recommendations are made, in particular, to: (i) establish an HTA in-house unit, able to carry out studies considering the hospital context and aiming to inform managerial local decisions (ii) promote a team comprised of technology assessment multidisciplinary researchers but also professionals from the health institution able to carry out HTA studies (iii) foster common languages and values to increase uptake of HTA studies.

Accounts of patient experiences are increasingly used in health technology assessment (HTA) processes. However, we know little about their impact on the decision-making process. This study aims to assess the level and the type of impact of patient input to highly specialised technologies (HSTs) and interventional procedures (IPs) guidance at the National Institute for Health and Care Excellence (NICE).

A questionnaire was developed to capture quantitative and qualitative data on the amount and type of impact of patient input into NICE HTAs. It was completed by committee members of the guidance-producing programs after a discussion of the considered topics. The data were analyzed by topic and overall, for each program, and compared across programs.

Patient input was assessed on ten pieces of HST guidance published between January 2015 and November 2019, and on twenty-six pieces of IP guidance scoped between February 2016 and October 2018. A total of 96 responses were collected for HST and 440 for IP. The level of impact of patient input was higher for HST than for IP. For HST, no respondents stated that it had no impact, whereas in IP, 35 percent of respondents did. The most common types of impact found for HST and IP were that it helped interpret the other evidence and that it provided new evidence.

The impact of patient input is not necessarily explicit in changing recommendations, but it provides context, reassurance, and new information to the committee for the decision-making process in HTAs.

Prescribing of medicines in primary care in Wales has been exceptional in 2020 due to COVID-19 and the associated changes to the delivery of health services. The changes are likely to have harmful, albeit unintended, consequences, including disruption of pharmacy stock management; unpredictable changes in prescribing; and interruption to patients’ supply of medicines and reduced medication adherence. Changes in prescribing are unlikely to be distributed evenly across the country or population. Therefore, this study aimed to identify changes in GP prescribing compared with previous years, the variation of these changes, and factors related to the variation in changes, to identify patient subgroups for whom the impact is disproportionate.

We identified medicines of interest where concerns around prescribing have been raised and, for each of these medicines, retrieved monthly prescribing data for each GP practice in Wales (N = 492). We then linked these data with other publicly available data (for example, practice size, indices of multiple deprivation, disease prevalence).

We developed a novel approach to measure the impact of COVID-19 on GP prescribing. We compared observed with expected prescribing volume projected via time series modelling and differences were related to patient and practice characteristics using general estimating equations.

There was evidence of stockpiling of medicines during March 2020 (for example, oral-contraceptives and oral-anticoagulants with 11.6 and 18.5 percent increases from March 2019), followed by a short-term reduction in prescribing for oral-contraceptives (a reduction of 12.9 percent), but not oral-anticoagulants (an increase of 6.5 percent). However, GP level data show considerable deviation from the national trend for several GPs, which may be due to health and socio-demographic factors.

COVID-19 has had a major impact on primary care prescribing in Wales. The distribution of changes in prescribing will not be even across the country or the population. Identification of systematic variation in impacts on prescribing could identify geographical areas or patients in need of additional support to ensure uninterrupted and appropriate access to medicines.

Patient Support Programs (PSPs) have become a trend among pharmaceutical companies and a standard service offering to patients. The objective of the present study is to describe the status of PSPs in Lebanon and to assess the extent of knowledge and awareness among Lebanese patients about the PSPs.

A cross-sectional study was conducted between April and July 2017. A convenient sample of patients was randomly selected from outpatient clinics at four hospitals within the Greater Beirut Area. A questionnaire was used to address the study objective. Bivariate analysis was performed using the Chi-square test. Data were analyzed by using SPSS version 23.

Out of 385 patients who participated in the study, 45.45 percent were aged between 46 and 66 years. None of them indicated that they were enrolled in a PSP, and only 13 percent of the respondents were aware of the existence of such a program. In terms of adherence habits, 55.6 percent of the patients self-reported that they do not skip any dose of their medication and consume their medication as prescribed by their healthcare providers. The main reason for nonadherence reported by the majority of nonadherent participants 144 (84.2%) was simple forgetfulness.

There is a severe lack of awareness of PSPs in Lebanon. Given the important role that PSPs play in creating value for patients—in terms of healthcare follow-up practices, improved adherence habits, and cost savings—there should be a more substantial effort by pharmaceutical companies to expand and promote their PSPs in the Lebanese market.

Gaining the perspective of patients is invaluable in the design, management and reporting of research. As part of the process of facilitating clinical research into the effectiveness of a digital colposcope in a cervical cancer pathway, patients were involved from the outset.

Using funding made available by a Public Involvement Fund, a patient consultation group was established. The group's initial discussions informed the design of a feasibility study and funding application, which was submitted to the UK National Institute of Health Research (NIHR). A Patient and Public Involvement (PPI) representative was recruited and along with the consultation group, contributed to the ethical approvals for the study. The Patient Information Sheet and Consent Form were reviewed by the patients, to ensure readability, understandability and accessibility. The patient questionnaires and interview topics that are part of the feasibility study were also developed in conjunction with the PPI group, to make sure that women's concerns are being addressed in the research design and protocols.

The PPI consultation group's contributions helped strengthen the funding application and funding for a feasibility study was granted as part of the NIHR's Research for Patient Benefit funding scheme. Part of the grant will be used for training and reimbursement for time spent for the PPI representative. Data collection for the study is due to commence in the summer of 2021. The PPI group will be consulted at the beginning and end of the data collection period and will contribute to the data analysis and dissemination of the research output, including a Plain English Summary.

Involving patients greatly amplified the quality of the funding and ethical applications and will continue to benefit the ongoing research. Resources were widely available within the researcher's University and also through UK-wide schemes. Such resources are crucial and should be encouraged as part of all clinical research.

To summarize current evidence on patient and public involvement (PPI) in health technology assessment (HTA) in order to synthesize the barriers and facilitators, and to propose a framework to assess its impact.

We conducted an update of a systematic review published in 2011 considering the recent scientific literature (qualitative, quantitative, and mixed-methods studies). We searched papers published between March 2009 (end of the initial search) and December 2019 in five databases using specific search strategies. We identified other publications through citation tracking and contacting authors of previous related studies. Reviewers independently selected relevant studies based on prespecified inclusion and exclusion criteria. We extracted information using a pre-established grid.

We identified a total of 7872 publications from the main search strategy. Ultimately, thirty-one distinct new studies met the inclusion criteria, whereas seventeen studies were included in the previous systematic review. PPI is realized through two main strategies: (i) patients and public members participate directly in decision-making processes (participation) and (ii) patients or public perspectives are solicited to inform decisions (consultation or indirect participation). This review synthesizes the barriers and facilitators to PPI in HTA, and a framework to assess its impact is proposed.

The number of studies on patients or public involvement in HTA has dramatically increased in recent years. Findings from this updated systematic review show that PPI is done mostly through consultation and that direct involvement is less frequent. Several barriers to PPI in HTA exist, notably the lack of information to patients and public about HTA and the lack of guidance and policies to support PPI in HTA.

With unprecedented times, comes accelerated change. Hospitals in our region have begun to facilitate safe discharge for COVID-19 patients in the form of “The virtual COVID ward”. This has enabled patients to be monitored safely in the community using pulse oximetry, Florence (a telehealth mobile app) and remote consultations. Our objective is to expand upon this model by providing home oxygen therapy for these patients facilitated by telemedicine.

Patients were discharged with an oxygen concentrator if they had an oxygen requirement equal to or less than four litres/minute. Fraction of inspired oxygen needed to be stable and an early warning score of less than four was also required. Once admitted, the Florence app and daily remote consultations were crucial to closely monitor the patient's clinical status. The patient was instructed to enter oxygen saturations and heart rate into the app four times daily. The app would then alert our team if any patients observations deteriorate, triggering immediate assessment.

We have discharged ninety patients to the virtual ward, fifty-six of these with home oxygen. The average age was fifty-seven and the Clinical Frailty Score ranged between one and six. At present, ten patients have been re-admitted, four with increasing oxygen requirements, and six with unrelated symptoms. Two patients had oxygen concentrators installed at home after we were alerted to their desaturation by the Florence App. The re-admission rate is eleven percent, which mirrors that of other virtual wards (who do not provide home oxygen). In total, the ward has saved the trust 627 hospital inpatient ‘days’. Patients report increased satisfaction at playing a meaningful role in monitoring their own healthcare using the app.

Our novel model of supported discharge with oxygen therapy using telehealth demonstrates that it is possible to manage such patients, safely, in the community. Other trusts could utilise this model to reduce inpatient bed occupancy. Looking to the future, could telehealth be utilised further to facilitate other “Virtual wards” in the community?

The practice of public and patient engagement (PPE) in health technology assessment (HTA) has spread worldwide, yet gaps in knowledge remain. We carried out an institutional ethnography of the Canadian Agency for Drugs and Technologies in Health (CADTH) public and patient involvement in HTA.

The research took place over 15 months and included observational work in the institutional settings, text review, and interviews with individuals working for or involved with the agency.

We found that despite demonstrated commitment to PPE, organizational history, governance structure, and practices were impediments to a unified approach to PPE. Unclear role descriptions for committee members and differences in philosophy and priority given to PPE across the organization presented challenges to effective participation. The high degree of value given to evidence-based principles at times conflicted with meaningful integration of patient input. A lack of clear goals and processes, roles, and differential treatment of evidence in PPE served to minimize the importance of patient experiences and to displace their validity. An acknowledgment of conflicts between multiple epistemic traditions at work within HTA activities may strengthen organizational approaches to PPE.

HTA organizations can learn from this study by reflecting on the challenges described and the recommendations offered to address them. We suggest solidifying CADTH's commitment to PPE with clear agency-wide roles and direction, values, and outcomes, a comprehensive framework, and policy and procedures. An acknowledgment of diverse epistemic traditions, as well as leadership and expertise in PPE, will strengthen CADTH's PPE activities and sustain its leadership position in the HTA field.

Wearable Digital Health Technologies (WDHTs) can support and enhance self-management by giving individuals with chronic conditions more control over their health, safety and wellbeing. Involving patients early on in the design of these technologies facilitates the development of person-centered products. It may increase the potential uptake of (and adherence to) any intervention they are designed to deliver. This research aims to elicit chronic kidney disease (CKD) patients’ preferences for WDHTs that may help patients manage their conditions.

We used discrete choice experiments (DCE) to elicit preferences for WDHTs characterized by their generalizable characteristics. The study design was informed by a multi-stage mixed-method approach (MSMMA). This included a review of the published literature, focus group interviews and one-to-one interactions with CKD patients to identify relevant characteristics (that is, attributes and levels) associated with wearable DHTs. We collected the data from 113 patients (age ≥18 years) with stage 3 or above CKD. The analysis started with a conventional multinomial logit model and was extended by investigating heterogeneity in preferences via latent class models.

Our MSMMA yielded ten potential attributes for consideration in a choice task. The final list included five attributes, cross-checked and validated by the research team, and patient representatives. The most preferred attributes of WDHTs were device appearance, format and type of information provided, and mode of engagement with patients. Respondents preferred a discreet device, which offered options that individuals could choose from and provided medical information.

We show how to use MSMMA to elicit user preferences in (and to inform the) early stages of the development of WDHTs. Individuals with CKD preferred specific characteristics that would make them more likely to engage with the self-management support WDHT. Our results provide valuable insights that can be used to inform the development of different WDHTs for different segments of the CKD patients population, moving away from a one-size-fits-all provision and resulting in population health gains.

The American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) is a validated, risk-adjusted database for improving the quality and security of surgical care. ACS NSQIP can help participating hospitals target areas that need improvement. The aim of this study was to systematically review the literature analyzing the economic impact of using NSQIP. This paper also provides an estimation of annual cost savings following the implementation of NSQIP and quality improvement (QI) activities in two hospitals in Quebec.

In June 2018, we searched in seven databases, including PubMed, Embase, and NHSEED for economic evaluations based on NSQIP data. Contextual NSQIP databases from two hospitals were collected and analyzed. A cost analysis was conducted from the hospital care perspective, comparing complication costs before and after 1 year of the implementation of NSQIP and QI activities. The number and the cost of complications are measured. Costs are presented in 2018 Canadian dollars.

Out of 1,612 studies, 11 were selected. The level of overall evidence was judged to be of moderate to high quality. In general, data showed that, following the implementation of NSQIP and QI activities, a significant decrease in complications and associated costs was observed, which improved with time. In the cost analysis of contextual data, the reduction in complication costs outweighed the cost of implementing NSQIP. However, this cost analysis did not take into account the costs of QI activities.

NSQIP improves complication rates and associated costs when QI activities are implemented.

Zio XT Service was one of five Digital Health Technologies (DHTs) to be assessed by the National Institute for Health and Care Excellence (NICE) as part of their evaluation pilot. The King's Technology Evaluation Centre (KiTEC) act as an External Assessment Centre for NICE and worked on this pilot evaluation. The service comprises a I-Lead ECG patch, an inbuilt software that makes use of artificial intelligence (AI) algorithms to record, store and analyze ECG traces, and a team of cardiac physiologists.

Although the methods were based on NICE's existing Medical Technologies Guidance Process, they were modified to suit the assessment of DHTs. The process was split into two sections, with the option to discontinue the assessment if it was considered that insufficient evidence was available for the technology. Clinical experts and patients were consulted through the process and clinical, economic and technical evidence was considered. Costs for three care pathways were modelled.

A total of thirty relevant clinical studies were identified, with a further study being reviewed as part of a separate technical assessment, focusing on the AI component of the technology. Four of the studies were considered to be pivotal to the decision problem, one of which was a Randomized Controlled Trial. The technology was found to have a greater diagnostic yield than a standard ambulatory monitor, however diagnostic accuracy measures were absent in the literature. Three economic models were developed to represent three care pathways: patients with syncope, patients who have had a stroke or transient ischaemic attack and a third model assessing downstream costs associated with stroke treatment.

Digital Health Technologies and Artificial Intelligence Technologies pose novel and unique challenges to health technology assessment (HTA) bodies. Zio XT Service is a diagnostic tool, with both human and AI input, making it a particularly complex technology to assess. This work serves as a case-study in the evaluation of DHTs and AI and the lessons learned may contribute to the development of guidelines for such technologies.

Wearable digital health technologies (WDHTs) offer several solutions in terms of disease monitoring, management and delivery of specific interventions. In chronic conditions, WDHTs can be used to support individuals’ self-management efforts, potentially improving adherence to (and outcomes resulting from) interventions. Early health technology assessment (HTA) methods can inform considerations about the potential clinical and economic benefits of technology in the initial phases of the product's lifecycle, facilitating identification of those Research & Development (R&D) investments with the greatest potential stakeholders’ payoff. We report our experience of using early HTA methods to support R&D decisions relating to novel WDHT being designed to support self-management of chronic kidney disease (CKD).

We performed a literature review, focus-group interviews with patients, and qualitative interviews with the prototype development team to understand the relevant characteristics of WDHTs, quantify relevant clinical indications and existing technological constraints. An early economic evaluation was used to identify the key drivers of value for money, and a discrete choice experiment shed light onto patient preferences towards what key features the WDHT should have for the users to adopt it. Then a model-based cost-effectiveness analysis was undertaken incorporating headroom analysis, return on investment, one-way sensitivity analysis and scenario analyses using data from secondary sources.

The review of the literature, focus groups with CKD patients, and qualitative interviews with technology developers helped to understand relevant characteristics of WDHT and user preferences helped inform the next R&D iteration. Compared to the standard care, WDHT that support stage ≥3 CKD patients self-management at home by measuring blood pressure and monitor mobility has the potential to be cost-effective at conventional cost-effectiveness threshold levels. From the headroom analysis, novel WDHT can be priced up to GBP280 (EUR315, USD360) and still be cost-effective compared to standard home blood pressure monitoring.

Our study provides valuable information for the further development of the WDHT, such as defining a go/no-go decision, as well as providing a template for performing early HTA of Digital Health Interventions.

The MAVORIC trial compared mogamulizumab to vorinostat in patients with mycosis fungoides (MF) or Sézary syndrome (SS), subtypes of cutaneous T-cell lymphoma. However, the treatment comparison within MAVORIC may not represent an HTA relevant population from a UK perspective: (i) 72.6 percent of patients randomized to vorinostat switched to mogamulizumab and (ii) vorinostat is not used in current clinical practice in the UK. This study explores methods to adjust treatment effect estimates using different crossover adjustment methods and Real-World Evidence.

This medicine is subject to additional monitoring. This will allow quick identification of new safety information. See www.mhra.gov.uk/yellowcard for how to report side effects.

An advanced stage (stage ≥IIB MF and all SS) population was included. Three methods were considered for treatment crossover adjustment. A synthetic control arm was created using the Hospital Episodes Statistics (HES) dataset. Predicted survival for the MAVORIC control arm, post-crossover adjustment, was compared to the HES to inform the selection of the appropriate methods for adjustment. A direct comparison between mogamulizumab (reweighted to represent the distribution of MF/SS patients in the HES) and the synthetic control was also conducted.

Following crossover adjustment of the vorinostat arm, using the inverse probability of censoring weighting method, the overall survival (OS) hazard ratio (HR) estimate for mogamulizumab vs. vorinostat was 0.45 (95% confidence interval (CI): 0.19, 1.07). This adjustment method was considered the most appropriate based on an assessment of assumptions and a comparison of OS between the adjusted vorinostat data and the HES data. The OS HR estimate for reweighted mogamulizumab vs. synthetic control from HES was 0.33 (CI: 0.21, 0.50).

Real World Evidence from the HES database can be used to validate crossover adjustment methods and to better reflect current clinical practice in the UK. Results using both methods support each other.

This study aims to assess stakeholder perceptions on the challenges and value of real-world evidence (RWE) post approval, the differences in regulatory and health technology assessment (HTA) real-world data (RWD) collection requirements under the German regulation for more safety in drug supply (GSAV), and future alignment opportunities to create a complementary framework for postapproval RWE requirements.

Eleven semistructured interviews were conducted purposively with pharmaceutical industry experts, regulatory authorities, health technology assessment bodies (HTAbs), and academia. The interview questions focused on the role of RWE post approval, the added value and challenges of RWE, the most important requirements for RWD collection, experience with registries as a source of RWD, perceptions on the GSAV law, RWE requirements in other countries, and the differences between regulatory and HTA requirements and alignment opportunities. The interviews were recorded, transcribed, and translated for coding in Nvivo to summarize the findings.

All experts agree that RWE could close evidence gaps by showing the actual value of medicines in patients under real-world conditions. However, experts acknowledged certain challenges such as: (i) heterogeneous perspectives and differences in outcome measures for RWE generation and (ii) missing practical experience with RWD collected through mandatory registries within the German benefit assessment due to an unclear implementation of the GSAV.

This study revealed that all stakeholder groups recognize the added value of RWE but experience conflicting demands for RWD collection. Harmonizing requirements can be achieved through common postlicensing evidence generation (PLEG) plans and joint scientific advice to address uncertainties regarding evidence needs and to optimize drug development.