Background
Medicinal products to treat rare diseases fulfill a high unmet need for small populations suffering from often severely debilitating or life-threatening illnesses (Reference Godman, Malmstrom, Diogene, Gray, Jayathissa and Timoney1). Although these rare disease treatments (RDTs) and ultra-RDTs (highly specialized treatments for very rare diseases) hold promise to improve health and quality of life (QoL), they also pose considerable challenges for national health systems to appraise and reimburse, primarily because they come with a substantial amount of evidential uncertainty and high costs (2). The uncertainties relating to determination of the value of RDTs stem from a number of challenges inherent to rare diseases: they affect a small number of people, natural history knowledge is often limited, and presentation is heterogeneous (Reference Nicod, Annemans, Bucsics, Lee, Upadhyaya and Facey3–Reference Pearson, Rothwell, Olaye and Knight6). Appraisal of RDTs often involves significant assumptions about disease progression that cannot be validated, and reimbursement is often only possible with a willingness to pay (WTP) that is above traditional thresholds. This raises the question as to whether appraisal processes for RDTs should differ from standard ones (2;Reference Nicod, Annemans, Bucsics, Lee, Upadhyaya and Facey3;Reference Henderson, Errea, Skedgel and Jofre-Bonet7–Reference Ollendorf, Chapman and Pearson9).
In related research, we found that countries differ in their approaches to appraising RDTs to determine clinical benefit and/or value for money. In some countries, standard appraisal processes are used. In others, standard appraisal processes are adapted in order to better deal with some of the common challenges encountered with RDTs or ultra-RDTs, and others have entirely separate appraisal processes for RDTs or ultra-RDTs (Reference Nicod and Kanavos10;Reference Nicod, Whittal, Drummond and Facey11). Our previous work characterized the features included in these separate or adapted appraisal processes for RDTs, referred to from now as “supplemental.” Supplemental processes vary across countries and include features such as different requirements for clinical and/or economic evidence, more lenience around evidence quality, greater disease-specific input from patient and clinical experts, additional considerations of value, different decision rules, different WTP thresholds, or conditional approval (Supplementary Figure 1) (Reference Nicod, Whittal, Drummond and Facey11).
This research was undertaken to enable a deeper understanding of how standard and supplemental appraisal processes are implemented in practice for two RDTs and whether the identified features of supplemental appraisal processes make a difference in the decision-making.
Methods
A case study design investigated the appraisal/reimbursement processes for two RDTs across various countries with and without supplemental processes for RDTs.
Country selection was based on availability of public health technology assessment (HTA) reports and representative of countries with and without supplemental processes. The sample consisted of three countries/country alliances using only standard processes (the cross-border collaboration BeNeLuXAI, France, and the Netherlands), five countries using only supplementary processes (Belgium, Germany, Norway, Scotland, and Sweden), and two countries having both standard and supplementary processes, which were applied to one treatment or the other (England, USA) (Table 1). All countries in Europe with supplemental processes were initially included, some of which were subsequently excluded as they either did not have any publicly available information or did not provide sufficient information in their public reports (Italy, Lithuania, Poland, Romania, and Slovakia). Because of the collaboration between countries that are part of the BeNeLuXAI country alliance, BeNeLuXAI and Belgium used the Netherland's assessment, the only difference being that for Belgium, an additional country-specific epidemiology and budget impact analysis was conducted.
Table 1. Overview of appraisal/reimbursement processes used for nusinersen and voretigene neparvovec
Note: QALY-based systems consider economic models that include QALYs as measures of clinical benefit. Non-QALY based systems do not use QALYs as the preferred measure for clinical benefit assessment and do not conduct any economic modeling.
OMP, orphan medicinal product; RDT designated by European Medicines Agency (EMA), ICER, Incremental Cost Effectiveness Ratio; WTP, Willingness to Pay.
a In England, Nusinersen went through the standard (STA) pathway and Voretigene went through the HST pathway.
b The (I.C.E.R.) recently implemented modifications to their Appraisal Framework that apply to ultra-RDTs. Nusinersen was assessed taking into account the ultra-RDT modifications. Voretigene was assessed prior to the modifications and so is considered standard, but the report claims to take into account the framework adaptations.
Two contrasting RDTs were sought: rare versus ultrarare treatment (ultra-RDTs have even higher levels of uncertainty than RDTs, and in some cases, differing country processes apply only to ultra-RDTs), affecting child versus adult (as this aspect often affects decisions), life-threatening versus disabling (acute, often short-term treatment vs. longer-term treatment). A list of potential RDT candidates was compiled based on desk research, recommendations from country HTA experts, availability of reports, and discussion among team members. This led to the following selection:
(1) nusinersen (Spinraza®): intrathecal injection every 4 months, indicated for 5q spinal muscular atrophy (SMA) (children and adults, life threatening in children, rare across all types).
(2) voretigene neparvovec (Luxturna®): one-off gene therapy, indicated for RPE65-mediated inherited retinal dystrophies (adults, long-term disabling, ultrarare).
HTA reports for nusinersen were retrieved for all 10 selected jurisdictions and for eight of the 10 selected jurisdictions for voretigene neparvovec (not available for Belgium or BeNeLuXAI).
The data from published HTA reports were extracted by country experts for Sweden and Norway and by AW for all other countries, using a template based on an existing methodological framework (Reference Nicod and Kanavos10). This framework provided a structure for data extraction and analysis by distinguishing what occurred in the following stages of the appraisal process: (i) evidence considered, (ii) interpretation of evidence, and (iii) pricing and reimbursement decision, with the main focus on interpretation of evidence. The framework enabled documentation of the appraisal/reimbursement recommendation/decision, the clinical trials considered, the main clinical and patient-reported endpoints, and a range of issues that appeared to have influenced the appraisal. This supported reflection on the impact of the different types of appraisal processes on decision making in each country. Results focus on those cases identified where features from the supplemental process (Supplementary Figure 1) were reported to impact the deliberation and/or where contrasts were seen across countries. To illustrate the impact of these features, summaries of the key points from the appraisal were extracted from the HTA reports and supported with direct quotes from the report text.
Results
Case Studies
Case Study #1: Nusinersen
SMA is a progressive condition caused by a missing gene that weakens muscles and causes problems with movement, bones, joints, and breathing. The most severe type affects babies (Type 1) and the least severe type affects adults (Type 4). The comparator at the time nusinersen was appraised was best supportive care (12). Physical debilitation reduces a person's ability to live independently. Babies with Type 1 rarely survive beyond the first few years of life (13). Nusinersen compensates for the missing gene that causes the condition. Treatment is continued as long as it benefits patients. It is a designated orphan medicinal product (OMP) in the EU and received marketing authorization under the European Medicines Agency accelerated assessment program in April 2017.
Case Study #2: Voretigene Neparvovec
Inherited retinal dystrophies (IRDs) are a group of rare genetic eye diseases caused by gene mutations leading to progressive vision loss. At the time of appraisal, the only comparator was supportive care. It is a severe and progressive condition that affects all aspects of patients’ lives in terms of independence, ability to work, social life, and ability to carry out day-to-day tasks (14). Voretigene neparvovec is a one-time gene therapy involving an injection in each eye under the retina (14). It is an OMP that received marketing authorization in the EU in November 2018 for a specific population (15), which was considered as ultrarare or highly specialized by HTA bodies that use that classification.
Appraisal Processes
The two selected RDTs underwent the standard appraisal process in four countries, the supplemental process for RDTs in five countries, and in two countries, one treatment underwent the standard process and the other the supplemental RDT process (England, US). Table 1 outlines the country processes and further delineates the countries that focus on added clinical benefit to determine reimbursement versus those that require cost utility analysis (QALY-based). Further details about each country's appraisal process can be found on the IMPACT HTA Web site (16).
The timing of the appraisal, reimbursement decision, and key elements of countries’ appraisals are presented in Table 2. We defined reimbursement decisions as “list” when it was granted for the full indication submitted, with inclusion of a financial managed entry agreement (MEA) in some cases; “conditional approval” if an outcomes-based MEA was in place; “restrict” if there were restrictions on use (e.g., restricted population or dispensing); and “reject” when the decision was negative.
Table 2. Details of country decisions for nusinersen and voretigene neparvovec
Notes:
- Poland, Romania, and Slovakia all approved Nusinersen for use, but due to a lack of sufficient information in publicly available reports, they are included only in the discussion.
- ICER is a base case without MEA.
- All appraised for the same indication listed in the disease/treatment description unless otherwise stated.
Grey, standard process; White, supplemental process; UO, ultra-orphan; HTA, health technology assessment; ICER, incremental cost effectiveness ratio; QALY, quality adjusted life year; Zin, Zorginstituut; MEA, managed entry agreement; NHS, National Health Service; SMA, spinal muscular atrophy; TC, transparency committee; NOK, Norwegian krone; SEK, Swedish kronor; OMP, orphan medicinal product.
a SMC initial restriction: type 1 was most significantly improved according to evidence, but as of July 2019, it was listed under the ultra-orphan pathway, and types 2 and 3 were also reimbursed.
Most countries with and without special processes for RDTs approved both treatments, some conditionally with an outcomes-based MEA. However, we have explored in more depth what evidence was considered, how it was interpreted by appraisal committees, and if any features of supplemental processes were reported as being important for decision-making.
Evidence Considered
In terms of the evidence considered by each country, only two countries, both with supplemental processes, had different evidence requirements for RDTs: Germany and Scotland. Scotland has a different submission form for ultra-RDTs, whereas Germany has a simplified process for RDTs and considers only the pivotal trials. For both nusinersen and voretigene neparvovec, it was not clear from the reports whether these differing evidence requirements had any impact on the decision. In countries with no differing evidence requirements, it was not clear from the reports if or to what extent the additional trials considered by some and not by others impacted the decision.
The clinical data considered by each country to inform their appraisal processes can be found in Supplementary Table 1.
Interpretation of the Evidence
Issues Raised
A number of issues were reported during the deliberative process for the two treatments that related to uncertainties in the clinical evidence and in the economic modeling. These issues were dealt with differently across countries. Table 3 presents a summary of the reported issues raised by each country for nusinersen and voretigene neparvovec (for the detailed list of issues, see Supplementary Table 2).
Table 3. Summary of issues raised across countries for nusinersen and voretigene neparvovec
Grey, standard process; White, supplemental process.
X, issue considered but not addressed; √, issue considered and addressed.
Additional information: Addressed = it was discussed but further explained/countered/considered acceptable anyway, Not addressed = it was just raised.
Although the types of issues raised were not distinctively different between or dependent on whether the RDTs were assessed through a standard or supplemental process, countries with supplemental processes more often referred to explicit criteria to handle such issues. In the HTA reports, each issue raised was not specifically elaborated upon as to whether or not it was deemed acceptable and why, although some selected issues were thoroughly discussed; these are exemplified in detail below. Notably, when summarizing the interpretation of the evidence, countries with supplemental processes often highlighted that there were many issues, such as low evidence quality, uncertainty around long-term effects, and higher ICERs than those normally accepted, but generally referred to criteria enabling them to manage these issues through greater leniency in terms of evidence quality or different WTP thresholds.
This relates to a supplemental process feature, more leniency when judging the quality of evidence, and is reflected in Table 3, in which more countries with supplemental processes addressed the issues raised (represented by √ instead of X) by further explanation, discussion, or considering it acceptable given the particular circumstance. In particular, the general willingness to accept greater uncertainty (last row of the table) was reported more in countries with supplemental processes (1/4 standard vs. 3/6 supplemental processes for nusinersen; 0/3 standard vs. 5/5 supplemental processes for voretigene neparvovec). The way in which these elements were reported is highlighted in the example report quotations below, first for countries with standard processes, followed by countries with supplemental processes, for nusinersen and voretigene neparvovec, respectively.
For nusinersen, England (STA) was the only country using a standard process that explicitly accepted greater uncertainty: “committee was prepared to take into account a wide range of factors … concluded that it was willing to be flexible in its considerations around uncertainty … ”. From the countries using supplemental processes, Scotland accepted greater clinical uncertainty in relation to its decision-making criteria: “ … a number of the criteria were satisfied: the absence of other treatments of proven benefit and a substantial improvement in life expectancy … ”. Scotland similarly accepted uncertainty in the economic evidence in relation to its decision-making criteria by stating, “As nusinersen is an ultra-orphan medicine, SMC can accept greater uncertainty in the economic case.” Norway's rationale for eventually accepting greater uncertainty was discussed in reference to its criteria to account for disease severity. “For very severe diseases with small patient populations, less extensive documentation requirements can be accepted.” Germany accepted greater uncertainty by granting automatic reimbursement for OMPs.
For voretigene neparvovec, only countries using supplemental processes explicitly accepted greater uncertainty. England (HST) accepted issues by referring to unmet need and rarity of the condition, stating: “The committee was aware that there is a high unmet need in this population because there are no current specific treatments available for this condition. It recognized that the results of clinical trials were uncertain because of the small sample size and limited follow up. However, it considered that the evidence showed that voretigene neparvovec improved visual performance and was likely to prevent disease progression …. ” Norway accepted greater uncertainty in the evidence by referring to its decision-making criteria: “ … when assessing drugs for particularly small groups of patients with very serious condition, a lower requirement for documentation and a higher use of resources compared to other measures can be accepted”. Sweden accepted uncertainty based on rarity: “The uncertainties in the documentation and the results presented by TLV are considered to be very high. However, these uncertainties should be understood based on the rarity of the disease.” Germany accepted greater uncertainty by automatic reimbursement for OMPs, and Scotland through its ultra-OMP process, permitting three years of reimbursement and further data collection.
Other Considerations
One of the supplemental RDT process features identified in our previous work is “broader consideration of value,” that is, considerations beyond clinical and cost-effectiveness. In addition to the issues raised, both supplemental and standard process countries reported other considerations during the deliberative process for both treatments, but differed somewhat in their approaches. Table 4 outlines the other considerations that countries took into account and highlights if these were raised by stakeholder input from patients and/or clinicians.
Table 4. Other considerations that contributed to the determination of value
√: other consideration mentioned in the HTA report.
P/C: other consideration mentioned in the HTA report raised by the patient (P) and/or clinical experts (C), or at a PACE meeting. Mod = Modifier: other consideration recognized as SMC modifier for which greater uncertainty and higher cost/QALY threshold may be accepted. Nature of population: children and young people; PACE, patient and clinician engagement; QoL, quality of life; substantially improved life expectancy relates to an “end of life” medicine, which is used to treat a condition that is at a stage that normally results in death in 3 years or less with currently available treatments.
Gray, standard process; White, supplemental process.
The most notable differences between the two treatments were that, for nusinersen, more countries with supplemental processes (6/6) took into account the impact on QoL of patients and carers than countries with standard processes (2/4) (based on expert input). For nusinersen and voretigene neparvovec, most countries documented that they took the standard elements relating to rare diseases into account, namely rarity, severity, and unmet need, regardless of whether they used standard or supplemental processes. For both treatments, innovative nature of treatment, nature of the population, financial burden of condition, substantially improved life expectancy, and impact on specialized services were documented only by a few countries, with the latter being more predominant in those with supplemental processes. The descriptions and quotations below highlight examples of how these considerations were raised across countries and what they were.
Impact on Quality of Life of Patients and Carers
Regardless of standard or supplemental process, the way this consideration was addressed was quite uniform for both RDTs among countries:
“Some patients will be able to live longer with a significantly better quality of life … ” (nusinersen Norway).
Nature of the Population (Children)
“Considering … its beginning being either early after birth (severe forms evolving rapidly), or later in childhood … ” (voretigene neparvovec, France).
Only France and England STA considered the nature of the population for nusinersen, but this is not explicitly outlined as a criterion to consider in the decision-making process. However, in NICE HST, the nature of the condition is one of the four parts of the decision-making quadrant.
Rarity, Severity, and Unmet Need
Rarity, severity, and unmet need were accounted for equally by countries using standard processes and supplemental processes for nusinersen and voretigene neparvovec:
“Committee acknowledged the difficulty of appraising drugs for very rare conditions” (rarity, nusinersen, England STA).
Rarity is an explicit part of the assessment criteria in Germany and the Netherlands, but not in the STA process in England, yet the challenges relating to rarity were still considered in England.
“SMA is a serious, progressive muscle disease” (severity, nusinersen, Netherlands). “Hereditary retinal dystrophy … is severe” (severity, voretigene neparvovec, Norway).
Severity is an explicit part of the criteria in Belgium, Norway, and Sweden for potentially accepting an ICER higher than traditional WTP levels. It was considered within England STA but is not part of their criteria.
“At present there is no alternative to symptomatic treatment … ” (unmet need, nusinersen, BeNeLuXAI).
Innovative Nature of the Product
Regardless of the type of process, some counties referred to the innovative nature of the product for both treatments, and did so by referring to aspects that could be considered to represent innovative nature:
“nusinersen has a novel mechanism of action … ” (nusinersen, US).
Financial Burden of Condition
Was raised only by one country with a standard process (England STA) and two countries with a supplemental process (Scotland). The financial burden of the condition is a criterion in Scotland's appraisal process, but not in England STA.
Impact on Specialized Services
Was raised only by four countries, three with supplemental processes (England HST, Scotland, and Sweden) and one with a standard process (France).
For a full list of quotations, see Supplementary Table 3.
Countries with supplemental processes more often referred to criteria that enabled them to consider factors beyond clinical and cost-effectiveness.
Stakeholder Input
For both nusinersen and voretigene neparvovec, stakeholder input (patients, clinicians, or Scotland's “patient and clinician engagement meeting” (PACE)) played a role in raising some of the other considerations beyond clinical and cost-effectiveness. The amount of stakeholder input was reported more in countries with supplemental processes than in countries with standard processes, which may be because countries with supplemental processes are providing greater detail in their reports.
For nusinersen, England's STA was the only standard process that reported stakeholder input, whereas from the supplemental process countries, Norway, Scotland, and the USA reported stakeholder engagement. The structured PACE meeting in Scotland captured a wide range of value elements (Table 4), and of all countries that accounted for stakeholder input, highlighted the most considerations to provide context regarding disease and treatment. In the supplemental processes of Norway and the USA, patients highlighted the impact on QoL, and in the USA, severity as well.
For voretigene neparvovec, stakeholder engagement highlighted more additional points of consideration in countries with supplemental processes. Of the standard process countries, only the USA reported stakeholder input in terms of influence on QoL. Of the supplemental process countries, England HST, Scotland, and Sweden reported stakeholder input.
Patient Reported Outcome Measures (PROMs)
No differences were seen in the use of PROMs between standard and supplemental process countries, for either treatment. The only differences were between countries with non-QALY based processes (France and Germany) and those with QALY based processes (all others). Given that they do not use cost-effectiveness analysis, France and Germany only focused on PROMs for a general, qualitative assessment of benefit for both treatments.
For nusinersen, PROMs were reported in the company submission in the form of the PedsQL questionnaire in the CHERISH study, and all countries who use economic models reported it as a factor in the deliberation (Belgium, BeNeLuXAI, Netherlands, Norway, Scotland, Sweden, and the USA). EQ-5D youth version was also considered in Sweden, which reported these PROMs as being important for the decision.
For voretigene neparvovec, a visual function questionnaire was submitted, along with vignettes. Regardless of having standard or supplemental processes, all countries reported numerous uncertainties with the PROM measures. For example, Germany noted that validity was too unclear to take the PROMs into account, and England noted that the lack of direct HRQoL measures was a key limitation. From the public HTA reports, it seems there were many limitations observed in the PROM data, such as a lack of validity, uncertain utility values, or low response rates, which might explain why QoL seemed to be more often considered during the deliberations based on input from patient and clinical experts than from PROMs.
Pricing and Reimbursement Decision
A number of differences in the interpretation of the evidence across countries were identified around the types of issues raised, whether they were deemed acceptable, the consideration of expert opinion, and other considerations. Despite these differences, most countries accepted both treatments for full, restricted, or conditional reimbursement. This can be further explained by the different process features applied when making the decision, allowing for different decision rules, conditional reimbursement to manage uncertainty, greater WTP, or a combination thereof.
The supplemental RDT process feature of “different appraisal rules for RDTs” was reported in two supplemental process countries, namely Germany and Scotland. For nusinersen and voretigene neparvovec, automatically proven benefit was granted in Germany due to its status as an OMP. In Scotland, voretigene neparvovec was designated as an ultra-OMP and within the new ultra-OMP pathway had an interim assessment that identified uncertainties. A data collection plan was agreed with the Scottish Government and a full appraisal will occur after three years.
Another feature of supplemental RDT processes is “differing WTP,” usually represented by acceptance of an ICER that is higher than traditional WTP thresholds. A higher WTP was seen in countries with both supplemental and standard processes.
For nusinersen, in terms of standard process countries, England STA accepted the higher ICER within the context of implementing MEA: “the cost-effectiveness estimates … above the range normally considered cost-effective …nusinersen should be recommended … for the duration of and within the conditions set out in the managed access arrangement.” From countries with supplemental processes, for example in Scotland, decision modifiers were applied and a higher WTP was accepted. Sweden stated that “a higher cost per QALY can usually be accepted when the severity is high or if there are few other treatments to choose from.”
For voretigene neparvovec, in using its standard process, I.C.E.R. in the USA stated that “for ultra-rare diseases, decision-makers in the US … often give special weighting to other benefits and to contextual considerations that lead to coverage and funding decisions at higher prices, and thus higher cost-effectiveness ratios … ”. This was seen in the considerations made in the I.C.E.R. report regarding QoL, innovation, rarity, severity, and unmet need. In using its supplemental process, England (HST) stated: “The committee was satisfied that voretigene neparvovec would offer significant QALY gains, and therefore applied this weighting in its consideration of its value for money.”
Most countries accepted a higher than usual ICER and approved both treatments, regardless of whether they used a standard or supplemental process. The rationale for this was often the additional criteria considered and the negotiation of an MEA.
MEAs were also identified as a supplemental RDT process feature but were used as a way to deal with uncertainty by countries with both supplemental and standard processes. For nusinersen, Scotland's ultra-RDT process required a three-year data generation period, followed by reassessment. England's standard STA process required a five-year MEA. Norway applied an MEA. For voretigene neparvovec, a finance-based agreement was provided in England (HST), and other countries made outcome-based agreements.
Discussion
This analysis sought to examine the impacts of standard and supplemental RDT appraisal processes in practice. Given two contrasting treatments, the differences in the decision-making process for nusinersen and voretigene neparvovec were not immediately obvious. For both treatments, standard and supplemental process countries often reported many issues, made additional considerations of value, and found ways to manage the high price. For countries with standard and supplemental processes, the following key points were observed:
(1) Only supplemental processes had different clinical or economic submission requirements for RDTS: Germany and Scotland. For Germany, this was the expedited process without the initial IQWiG assessment. In Scotland, this allowed presentation of evidence about wider aspects of value.
(2) Stakeholder input highlighted important factors in consideration of value beyond clinical and cost-effectiveness more often in countries with supplemental processes (England HST, Norway, Scotland, Sweden, and the USA) for both RDTs, particularly structured stakeholder input in the form of the PACE meeting in Scotland.
(3) Different thresholds/decision frameworks were seen in countries with supplemental processes; many countries with supplemental processes have formal criteria that allow acceptance of an ICER that is higher than the usual WTP threshold.
(4) Countries with standard and supplemental processes did not obviously differ in their reported assessments and considerations, but this lack of clear differences may be due to the fact that it was difficult to extract this information from the HTA reports. With the exception of the NICE reports, which clearly state what the “committee considered” for its decision, it was not possible to judge from the other HTA reports which considerations impacted the decision most. This points to a need for better reporting of all considerations relevant for the decision, as these are the only public facing documents that can allow an understanding of how decisions are made.
(5) Stakeholder input can come from patient and clinical group submissions, structured peer review comments on the evidence, meetings with experts, and also from comments during the appraisal; it was not always possible to differentiate between these sources from the reported information, which similarly points to a need for clearer reporting.
(6) For both treatments, only a few countries extended their considerations beyond more conventional elements (i.e., QoL, rarity, severity, and unmet need) to aspects such as the innovative nature of the treatment, the nature of the population, the financial burden of the condition, substantial improvement in life expectancy, and impact on specialized services. These are equally important and could be more frequently taken into account alongside the more common considerations.
(7) Although clear distinctions between countries were not obvious, countries with supplemental processes were more likely to have formal criteria in place for addressing the specificities of RDTs, and more often referred to this in their reports. There were instances when countries with standard processes also “applied” the features of supplemental processes. This more informal consideration of RDTs may lead to less consistent decision making in countries without supplemental processes.
(8) It is relevant to note that financial agreements between manufacturers and payers are confidential and we have no insight into this. Although recommendations for reimbursement were mostly positive for both treatments regardless of whether a standard or supplemental process was used, we cannot fully conclude the impact of the type of process on the outcome. Supplemental processes may, for instance, have had a different impact on the financial agreement than standard processes, but this remains unknown.
This analysis highlights the complexity of the appraisal processes for RDTs. It found that the previously identified features of supplemental RDT processes do seem to be applied more in countries with supplemental processes, but are also applied less formally in countries with standard processes, which may lead to more inconsistency. These features may have driven the deliberative process of appraisal differently for those countries with supplemental processes, but this is unclear from reports. As a result, another workstream in this research is undertaking ethnographic observation of appraisal committees to further explore what impacts decisions about RDTs.
Although we cannot determine from the HTA reports to what degree these considerations factored into the decision itself, in general, these elements of supplemental processes seem to enable structured and consistent consideration of the specificities of each rare disease and treatment during appraisal.
Strengths and Limitations
The strength of this research lies in enabling a deeper understanding of how standard and supplemental RDT appraisal processes play out in practice. To our knowledge, this is the first publication to adopt the approach of comparing standard with supplemental appraisal processes for RDTs. The investigation of a wide variety of country HTA reports is unique, as many reports are in national languages and not easily accessible without time and resources investment to translate and understand the report contents.
As a result, these case studies provide invaluable insights into the details of the HTA decision-making process for RDTs and a reference point for understanding how various countries report handling the appraisal of these treatments.
It is also important to note the limitation that the analysis was based on publicly information provided in HTA reports, which cannot capture the full picture of the appraisal process, but is simultaneously a strength in that it highlights the degree of transparency that HTA bodies demonstrate in reporting their decision-making processes.
Additionally, a limited number of RDTs and countries were examined based on feasibility, and both RDTs were accepted in almost all included countries. A wider picture could be gained by looking at more RDTs and more countries. With these case studies, we explore possible explanations, but we cannot make definitive statements about the reasons for country differences as the public reports do not provide enough information to clearly identify the key points that ultimately impacted decisions.
Conclusion
This analysis of public HTA reports illustrates how standard and supplemental appraisal processes for RDTs are implemented. Although in these case studies both RDTs were generally accepted for full or conditional reimbursement, making it difficult to conclude how much of a difference supplemental processes make, they do reflect the finding by Nicod et al. (Reference Nicod, Whittal, Drummond and Facey11) that supplemental RDT processes have a number of different mechanisms/more formal criteria than standard processes, which can enable RDT specificities to be accounted for in the decision-making process. This includes, for instance, differing evidence requirements for RDTs, being more lenient around the quality of evidence and accepting higher uncertainty, broader considerations, and a higher WTP threshold. Such formal criteria may help more accurately appraise the value and challenges of RDTs in a more structured way and aid in consequent decisions being more transparent and consistent.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000337.
Acknowledgments
We would like to thank Sheela Upadhyaya for her significant input throughout the development of this manuscript, as well as Douglas Lundin, Niklas Hedberg, and Leung-Ming Yu for their invaluable contributions to extracting information from their respective country reports.
Funding
This work was supported by the European Commission's Horizon 2020 research and innovation program and was undertaken under the auspices of the IMPACT-HTA project—Work Package 10 on appraisal of orphan medicinal products (Grant number 779312). The results presented here reflect the authors’ views and not the views of the European Commission. The European Commission is not liable for any use of the information communicated.
Conflicts of Interest
Dr. Whittal reports grants from EC H2020, during the conduct of the study; personal fees from Dolon Ltd., with no conflicts, outside the submitted work. Dr. Nicod reports grants from the European Commission's Horizon 2020 research and innovation program, during the conduct of the study, and part-time employment with Dolon Ltd., but no conflicts. Dr. Drummond reports grants from the European Commission, during the conduct of the study; grants and personal fees from several pharmaceutical companies, outside the submitted work. Dr. Facey reports grants from EC H2020, during the conduct of the study; personal fees from Novartis, personal fees from Sanofi, personal fees from Dolon, personal fees from FIPRA, and personal fees from INAMI, outside the submitted work.
Open access
