In England, the time gap between marketing authorization (MA) and guidance publication by National Institute for Health and Care Excellence (NICE) can limit patients’ access to new medicines. In this study, our aim was to identify medicine characteristics associated with the long time gap between MA and guidance publication and explore the influencing factors.

We identified 116 single technology appraisals from 2016 to 2020 using publicly available data, and extracted information on the year of appraisal completion, application type, experiences of similar appraisals, orphan medicinal products (OMPs), cancer medicines, and accelerated assessment. Multiple regression analyses were performed to analyze the associations between the medicine characteristics and key time periods related to health technology assessment and MA processes.

OMPs were associated with a long period between MA and guidance publication. Specifically, OMPs and cancer medicines were associated with slow guidance publication after the final scope (FS) development. However, there was no association between OMPs and the period between validation of MA application and FS development. Non-double-blinded randomized clinical trials and the use of comparators not specified in the FS were associated with slow guidance publication after the FS development.

Our results demonstrate that OMPs are associated with a longer period between MA and guidance publication by the NICE than non-OMPs; this may be attributed to the slow guidance publication after the FS development. These findings indicate the necessity to shorten the appraisal process for OMPs.

Conducting joint health technology assessments (HTA) is one of the main goals of the Beneluxa Initiative. To strengthen this collaboration, this study aimed to assess similarities and differences between past assessments of Beneluxa Initiative member countries (Austria, Belgium, Ireland and the Netherlands).

A retrospective comparative analysis was performed that investigated the similarities and differences in drug assessments in the period 2016 to 2020 in (i) the number and type of assessed indications; (ii) the conclusions within assessments performed by at least two member countries; and (iii) the main arguments leading to the conclusions through a qualitative analysis of selected cases, looking into the patient population, the intervention, comparator, outcome, timing, and included evidence.

The scope of HTA differs between the countries, with Belgium and Ireland assessing most, the Netherlands focusing on drugs above a budget impact threshold and Austria on outpatient drugs. Furthermore, indications might slightly differ between countries. Therefore, only 44 (10%) of the 444 included drug-indication combinations were assessed through a full HTA by all four countries. Between any pair of countries, the overlap was higher, from 63 (Austria-the Netherlands) to 188 (Belgium-Ireland). Added benefit conclusions matched exactly in 62 to 76 percent of the indications, depending on the compared countries. In the remaining cases, often a difference of one added benefit level was observed (e.g., higher versus equal relative effect). Contradictory outcomes were very rare. Differences were observed with regards to whether a cost-effectiveness analysis was performed. When assessing the underlying arguments within the reports for nine cases with different outcomes, it became clear that organizations agree on almost all aspects, and that differences are mostly attributable to slight differences in weighing of some aspects and uncertainties.

Overall, which indications are assessed differs, but for those indications that are assessed by multiple member countries, considerations and assessment outcomes are similar.

The need for timely access to innovative technologies has placed a special focus on the development of policies and practices that can guarantee the availability whilst ensuring the safety of these technologies after launch or licensure. The aim of this paper is to present and discuss Post-Launch Evidence Generation (PLEG) practices among health technology assessment (HTA) bodies at the European level to explore cross-border collaboration opportunities.

In December 2019, a survey composed of nine closed-ended questions with multiple choice answers about the PLEG practices in each country was sent to 25 partners of the European Network of Health Technology Assessment (EUnetHTA) Joint Action 3. In addition to the survey, the national practices were discussed during a face-to-face meeting with all partners of the dedicated work package. A quantitative analysis and a qualitative synthesis of the results was carried out.

Twelve HTA bodies completed the survey. Of these, 11 reported procedures in place for official PLEG requests. In nine of the agencies, the requests are made at the time of the assessment/appraisal. Data collection and analysis mainly lies with companies for pharmaceuticals (60%) while it is more the responsibility of the HTA body for medical devices (75%). Only one agency reported owning the data and being able to exchange the data without asking permission. During the face-to face discussions, it was acknowledged that PLEG practices differ between countries depending on the topic concerned, but most rely on the usage of registries (mainly disease registries) for data collection. Most agencies estimated that a European collaboration could take place.

PLEG practices are in the remit of many European HTA bodies. Data sharing should be anticipated as only some own the data and can exchange them without asking permission. European collaboration on PLEG could commence once the evidence gaps have been defined or during the production of the HTA reports in the case of joint assessments.

The objective of this study was to investigate whether glycated hemoglobin (HbA1c) is a valid surrogate for evaluating the effectiveness of antihyperglycemic drugs in diabetes mellitus (DM) trials.

We conducted a systematic review of placebo-controlled randomized clinical trials (RCTs) evaluating the effect of a treatment on HbA1c (mean difference between groups) and clinical outcomes (relative risk of mortality, myocardial infarction, stroke, heart failure, and/or kidney injury) in patients with DM. Then, we investigated the association between treatment effects on HbA1c and clinical outcomes using regression analysis at the trial level. Lastly, we interpreted the correlation coefficients (R) using the cut-off points suggested by the Institute for Quality and Efficiency in Healthcare (IQWiG). HbA1c was considered a valid surrogate if it demonstrated a strong association: lower limit of the 95 percent confidence interval (95 percent CI) of R greater than or equal to .85.

Nineteen RCTs were identified. All studies included adults with type 2 DM. None of the associations evaluated was strong enough to validate HbA1c as a surrogate for any clinical outcome: mortality (R = .34; 95 percent CI −.14 to .69), myocardial infarction (R = .20; −.30 to .61), heart failure (R = .08; −.40 to .53), kidney injury (R = −.04; −.52 to .47), and stroke (R = .81; .54 to .93).

The evidence from multiple placebo-controlled RCTs does not support the use of HbA1c as a surrogate to measure the effectiveness of antihyperglycemic drugs in DM studies.

The European Network for Health Technology Assessment (EUnetHTA) Joint Action 3 (JA3) aimed to develop a sustainable European model for scientific and technical collaboration on HTA. It succeeded EUnetHTA JA2, which focused on strengthening practical applications of approaches and tools in the European HTA collaboration. Compared to JA2, several changes in procedures and processes were undertaken throughout JA3 in order to improve the different steps in joint HTA production.

Findings and identified challenges regarding the assessment production processes from JA2 were considered as a basis. In JA3, vast majority of structured and informal feedback was gathered from the assessment teams and project managers via feedback surveys and meetings. Only limited informal feedback from stakeholders (such as patients, health care professionals, and health technology developers) that were involved in EUnetHTA assessments was collected. To this end, experiences were documented and recommendations for a future production process were developed.

During the course of JA3, the joint production resulted in 16 pharmaceutical assessments and 27 assessments of other technologies. The latter included medical devices, diagnostics, interventions, and screening. Due to the different context of pharmaceuticals and other technologies, some technology-specific changes needed to be made in their production process. However, the majority of implemented changes were made for both types of technologies to ensure maximum possible alignment in processes. The implemented changes affected several steps in the production process as well as the involvement of stakeholders in EUnetHTA assessments. The production and related project management of assessments was fine-tuned and resulted in clearer, standardized, and comprehensible processes that facilitated transparency and inclusiveness.

The procedural changes led to further standardization and elaboration of assessment production processes in preparation for a future European HTA system under the EU HTA Regulation. However, some methodological challenges remained to be tackled further in the currently ongoing EUnetHTA 21 service contract.

The European Network for Health Technology Assessment (EUnetHTA) was established in 2006. During its final project phase (joint action 3 [JA3]), it undertook an activity to define the scientific and technical principles of a model of health technology assessment (HTA) cooperation in Europe. This policy article presents the key learnings from JA3 partners about developing a model of HTA cooperation.

There were two phases to the activity: (i) A descriptive phase to describe the elements of HTA cooperation that were already in place in EUnetHTA JA3 and to identify which elements could be improved or were missing. (ii) An analytic phase synthesizing the data collected to identify learnings from the JA3 and to define the scientific and technical principles for a future model of HTA cooperation.

Learnings for developing HTA cooperation were identified in regard to the framework used to support the cooperation, the HTA activities undertaken, the involvement of internal and external actors, managing decision making and the required human resources and support services needed to undertake HTA activities and to coordinate collaboration.

These learnings coming from the experiences of the EUnetHTA JA3 are useful to inform discussions on a European Union regulation for HTA cooperation as well as subsequent work to set up the structures that will be defined in the regulation. The findings also have broader applicability and are relevant to individuals, groups, and organizations setting up HTA programs or establishing their own international collaborations.

Patient and public involvement (PPI) is an expectation when conducting a health technology assessment (HTA), but there is little guidance for those wishing to embed PPI when developing an HTA proposal. We wanted to ensure PPI was central in preparing a proposal for an HTA potentially of any intervention for pelvic organ prolapse (POP) in women.

We conducted an open process to recruit two PPI co-applicants who, after induction to the project, were jointly responsible for governance of PPI in partnership with the PPI Lead throughout project planning. We facilitated an online workshop with the PPI co-applicants and other women with a lived experience of POP to: develop our question and scope; decide interventions and outcomes for the evidence synthesis; discuss the care pathway for the economic evaluation component; and plan dissemination. The PPI co-applicants were encouraged to comment on the full proposal, while workshop attendees were invited to comment on the plain language summary. Our work adhered to United Kingdom (UK) Standards for Public Involvement. We obtained funding to facilitate PPI within the proposal and reimburse those with lived experience for their time.

Involving the co-applicants and workshop participants strengthened the HTA proposal by: solidifying the rationale based on lived experience; adding interventions to our evidence synthesis not previously considered; and highlighting dissemination outlets that appealed to the public. Comments on the full proposal and plain language summary ensured the proposal was accessible. However, we were unable to discuss everything we originally planned even though researcher time spent on embedding PPI into the proposal was substantial.

Including PPI can be valuable for developing HTA proposals. However, research is required to explore the appropriate level of involvement at the HTA proposal stage, particularly given the large amount of researcher time and additional resource needed to incorporate meaningful PPI.

The cyclic appropriate care program of the National Health Care Institute aims at enhancing quality of care in the Netherlands. A full cycle consists of four phases: (i) screening each ICD-10 chapter and selecting a care trajectory, (ii) in-depth analysis of the care trajectory and formulating actions for improvement, (iii) implementation, and (iv) evaluation. Collaboration with and between relevant parties from the field is key to its success. We describe the 2nd and 3th phase of this cycle, aimed at osteoporosis.

First, nine clinicians, one nurse practitioner, two physiotherapists and one pharmacist representing scientific organizations, two patient representatives and one health insurer, defined research questions. Second, by in-depth analysis on declaration data of health care activities, and comparing the results to the guidelines, research questions were addressed. Last, results were discussed among all parties and actions for making wiser choices and improving the appropriateness of care were formulated. For each action, the parties defined who would take the lead and who would be involved.

The analyses showed that only 26 percent of Dutch fracture patients 50 years of age and above underwent dexa-scanning, as opposed to more than 80 percent in the guideline. All parties underlined this percentage should be increased. Possible actions were defined: to describe the care trajectory in the update of the multidisciplinary guideline; to better inform fracture patients about the importance of dexa-scanning; and to make adjustments to the hospital electronic system in order to facilitate the ordering of dexa-scans. For these actions, medical specialists were in the lead. Other data-analyses showed that half of osteoporosis patients stopped using medication within two years. For the jointly defined actions aimed at raising medication adherence, such as better patient education, the general practitioner and pharmacist were in the lead. Currently, the National Health Care Institute facilitates implementation and monitors improvement.

Close collaboration with relevant parties led to well-considered actions on improvement of osteoporosis care.

Involving patients and community organizations in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE) and helps build public confidence in the healthcare decision-making. From 2019 to2021 NICE assessed a HIV-1 medicine for the first time. This presented new opportunities and challenges for engaging with Human Immunodeficiency Virus (HIV) community organizations who hadn’t participated in a NICE HTA before. To understand their experience, we collected feedback on the impact and experience of community involvement throughout the assessment.

We used a mixed-method approach using a survey and qualitative feedback from the committee lay member, community experts, and the community organizations. An impact survey was sent to key committee members, the NICE technical team and Associate Director. It included Likert scale questions and open text boxes to capture both quantitative and qualitative data. Additionally, qualitative feedback was gathered throughout the assessment’s lifecycle from the NICE team and the community stakeholders.

Results from the impact survey showed that the community input: had significant impact on the evaluation (100% (n=5)); helped interpret the other evidence and information (80% (n=4)); provided new evidence (60% (n=3)); and, was consistent with the other evidence (40% (n=2)). Examples of impact included highlighting the population heterogeneity, stigma, side effects and the effects of frequent clinic attendance. The key feedback from the qualitative data from the community organizations and experts in terms of their experience were: early support and support throughout from the NICE team; NICE’s flexibility in involving them; and, meaningful inclusion of their evidence in the committee slides.

Both the NICE committee and the community stakeholders recognized that the community input was valued and had an impact on the decision-making. To ensure meaningful community engagement, support and flexibility from NICE were required throughout the assessment. This level of engagement will be adopted in future for community organizations new to medicines HTAs at NICE.

Robot-assisted surgery (RAS) is being adopted rapidly internationally across a wide range of surgical procedures. Although a great deal of evidence of the clinical effectiveness of RAS has been generated, it is possible that the evidence base is not complete or persuasive in some areas where adoption is being considered. This review seeks to summarize systematic reviews (SRs) undertaken to date to illustrate the weight of evidence across specialties. We then take an in depth look at the quality of evidence across several indications where the adoption of RAS is currently underway.

A comprehensive literature search was conducted using Ovid Medline, Embase, and Cochrane Central Register of Systematic Reviews from January 2017 to April 2021 for SRs describing clinical effectiveness outcomes. The body of evidence was mapped across all specialties. For a selected number of indications currently under consideration in Scotland, results were comparatively summarized, and the quality of the reviews was evaluated with the AMSTAR-2 tool.

A total of 451 SRs were found. Most were in urology (n = 130) where RAS is well established, followed by colorectal (n = 63), hepatology (n = 58), and gynecology (n = 41). From within these latter three specialties, we selected six indications in which RAS is currently being considered for adoption in Scotland for in depth review (colorectal cancer surgery, hysterectomy, gastrointestinal oncological resection, hepatic, pancreatic and biliary surgery). Evidence for the clinical effectiveness of RAS versus conventional laparoscopic surgery is mixed across indications and outcomes. In colorectal cancer surgery, for example, evidence was positive for conversion rate and neutral for length of hospital stays, blood loss and postoperative complication and negative for operative time. For hysterectomy, evidence was positive for the length of hospital stays and neutral for operative time, blood loss, conversion rate and postoperative complication. The quality of the included reviews was judged to be critically low.

The currently available evidence of clinical effectiveness is mixed across indications and of low quality.

The purpose of this systematic review is to evaluate whether implantable versus injectable bulking agents (second-line therapies) are equal/superior in terms of effectiveness (severity, quality of life [QoL], sustainability) and safety (adverse events) for fecal incontinence (FI).

A systematic review was conducted and five databases were searched (Medline via Ovid, Embase, Cochrane Library, University of York Centre for Reviews and Dissemination, and International Network of Agencies for Health Technology Assessment database). In-/exclusion criteria were predefined according to the PICOS scheme. The Institute of Health Economics risk of bias (RoB) tool assessed studies’ internal validity. According to the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach, the strength of evidence for safety outcomes was rated. A qualitative synthesis of the evidence was used to analyse the data.

Six prospective uncontrolled trials (143 patients) were included. The evidence consists of six prospective single-arm, before-after studies fulfilling the inclusion criteria for assessing clinical effectiveness and safety for implantable bulking agents. FI severity (Cleveland Clinic FI Score) statistically significantly improved to three months (p<0.01) and six months (p<0.05) follow-up (five studies). Improvements in severity sustainability were reported after 12, 14 (p<0.01), and 36 (p<0.0001) months postoperatively. Improved disease-related QoL (FI QoL Score) was found (p<0.05) 12 months after surgery, and statistically significant improvements in QoL’s sustainability after 12 months (one study).

Procedure-related adverse events (n=3) occurred, where prostheses extruded during surgery, and anal discomfort/pain was felt (n=11). Device-related adverse events, i.e., prostheses’ dislodgement (n=31) and removed/extruded prostheses (n=3), occurred. Studies were judged with moderate/high RoB. The strength of evidence for safety was judged to be very low.

Implantable bulking agents might be an effective and safe minimally invasive option in FI treatment if conservative therapies fail. FI severity significantly improved, but not QoL, which needs to be explored in further studies. Due to the uncontrolled nature of the case series, comparative studies need to be awaited.

The Pharmaceutical Benefits Advisory Committee (PBAC) is an independent expert body that recommends new technologies for listing on the Pharmaceutical Benefits Scheme. Its decision-making process is evidence-based and considers a technology's clinical effectiveness, safety, and cost-effectiveness compared with other technologies. Since 2014, the PBAC has formally taken into account input from those impacted by the technology via an online consumer comments portal and has also reported on received comments in the Public Summary Documents (PSDs). Comments are welcomed from those whose health the technology is trying to improve, as well as carers, clinicians, and organizations. Our objective was to analyze and review consumer comments in the PBAC's decision-making process.

We extracted information about consumer comments from the PBAC PSDs from 2014–9. We conducted simple descriptive analyses.

Our findings reveal that two thirds of all submissions did not receive a single consumer comment. Of the remaining third, eight submissions (less than 1 percent) had a substantial number of consumer comments (>500). For these technologies, multiple submissions were required before a recommendation was issued. Submissions spanned multiple therapeutic areas, the therapeutic areas with the most consumer comments were genetic disease, pediatrics, and oncology.

In the light of our review, we have identified limitations to the current consumer comments process, and after an examination of the processes of other comparable health technology assessment agencies, we have identified a number of improvements that could be made to the PBAC's process to increase consumer engagement.

The purpose of this study was to systematically review the process for topic selection by health technology assessment (HTA) agencies around the world to provide the knowledge base for the improvement of topic selection frameworks in HTA agencies.

A systematic search was conducted in PubMed and EMBASE to identify papers up to February 2019. Gray literature was identified by screening the Web sites of HTA agencies on the nonprofit member list of the International Network of Agencies for Health Technology Assessment (INAHTA). Data were extracted for each HTA agency and synthesized, with issues including general contextual information about each agency and the process of topic selection.

Out of forty-nine nonprofit members of INAHTA, a total of seventeen HTA agencies with a framework for topic selection were identified from twenty-two included papers/documents. Multiple criteria were used for topic selection in all frameworks and agencies undertook multiple steps, which could include the specification of criteria for topic selection, identification of topics, short listing of potential topics, scoping of potential topics, scoring and ranking of potential topics, and deliberation and decision on final topics for HTA. Shortcomings were found in relation to methods of scoring and ranking as well as lack of monitoring and the evaluation of the process.

Our study provides insights into the current practice of topic selection in HTA agencies. Multiple criteria decision analysis methodology appears highly relevant to these processes. A consensus approach for the development of methods of topic selection would be valuable for the HTA community.

Treatment options for men with moderate-to-severe lower urinary tract symptoms due to benign prostatic hyperplasia (BPH) include medical therapy, minimally invasive surgical therapies (MISTs), and invasive surgical procedures. While these treatments are recommended by American Urological Association Guidelines, they have different clinical profiles impacting both efficacy and durability outcomes. Using an indirect comparison approach, this study assessed the clinical effects of combination therapy (CT) using alpha-blockers and 5-alpha reductase inhibitors, two emerging MISTs (prostatic urethral lift [PUL] and water vapor thermal therapy [WVTT]), and two invasive surgical procedures (photoselective vaporization of the prostate [PVP] and transurethral resection of the prostate [TURP]).

A systematic search of Medline, Embase, Cochrane Library, and relevant health technology assessment (HTA) databases was conducted to identify randomized and non-randomized clinical trials of the five treatments published prior to December 2020. Trials were included if they reported changes in International Prostate Symptom Score (IPSS) and retreatment rates, without any country or language restrictions. A random-effects network meta-analysis (NMA) with an aggregate regression model was performed to account for the baseline BPH severity and characteristic differences among men from the different trials.

A total of 237 of 3,104 retrieved abstracts were included for full-text review. Of these, 16 randomized and four non-randomized clinical trials were included in the NMA. The random-effects NMA showed among medical and minimally invasive therapies, WVTT had the greatest one-year IPSS improvement (-∆11.7), followed by PUL (-∆10.4) and CT (-∆10.3). The one-year IPSS improvement for TURP and PVP was comparable (-∆14.1 vs. -∆13.8, respectively; p-value=0.675). The one-year retreatment rates were lowest for WVTT (3.0%), followed by CT (3.6%), TURP (6.3%), PVP (7.8%), and PUL (8.0%).

WVTT provided greater clinical and durability benefits compared to other less invasive treatment options for men with BPH. Given NMA is increasingly used in HTA processes, this study provided systematically synthesized evidence that could facilitate decision-makers in determining new technology coverage decisions globally.

Recently, there have been concerns regarding a trend toward poorer quality evidence being accepted by regulatory institutions and the consequent impact on health technology assessment (HTA) decision-making. This study aimed to determine whether there has been a change in the quality of evidence provided on cancer drugs proposed for listing on the Pharmaceutical Benefits Scheme, using data solely extracted from public summary documents (PSD) published by the Australian government.

PSDs published from July 2005–2020 were reviewed. Metrics associated with quality of evidence were extracted, including the directness of comparison, study design, sample size, and risk of bias (RoB). Additional data were extracted to provide greater context to any observed trends in quality of evidence. Analyses were performed across different time periods. Associations between the quality of evidence and time periods were explored using logistic regression analysis.

In total, 214 PSDs were included in the analysis. Only 13 percent of submissions provided a single arm study or observational study as the key evidence; however, 37 percent of submissions did not contain a direct (‘head-to-head’) comparison relevant to Pharmaceutical Benefits Advisory Committee (PBAC) decision-making. Among all submissions containing direct evidence, about half had findings of a moderate/high/unclear RoB. Among all submissions containing indirect comparisons, over half had transitivity issues. In submissions containing direct comparisons, there was an increase in the RoB over time even after adjusting for trial data maturity and the rareness of the drug indication (odds ratio [OR] 1.30; 95% confidence interval [CI] 0.99, 1.70). There were no clear time trends observed in sample size, directness, study design, or transitivity issues during any of the observed time periods.

In the last 7 years, a high proportion of cancer drug submissions presented findings with a high RoB and transitivity issues. As the evidence dossiers provided to the PBAC are often congruent with submissions made elsewhere, this poor evidence quality is of concern and can only lead to higher levels of decision-maker uncertainty.

Presenting an approach to synthesize quantitative and qualitative information from systematic reviews of multiple health interventions.

Within the context of an EUnetHTA multi-health technology assessment of twenty-three surgical techniques, we developed synthetic single tables, using color gradients and abbreviations, with information on which technologies had been compared, estimates of the size of differences for available comparisons, their clinical relevance, and certainty of the related evidence.

The proposed methodology provided, through a single depiction, information normally included in multiple figures/tables such as network plots, league tables, and summary of findings tables.

Transferring information on benefits, risks, and certainty of the available evidence on health interventions may be challenging, especially when assessing multiple treatments: more pieces of information need to be integrated in order to show an overall picture for each of the chosen outcomes, and usual reporting tools may be targeted to researchers more than to different kinds of decision makers. While more in-depth layers of information can always be added to satisfy needs of different audiences, the proposed tools could favor a quick interpretation of articulated scientific data by both decision makers and researchers.