Background
As part of the Sustainable Development Goals, all United Nations member states have committed to achieving universal health coverage (UHC) (1;2). To achieve UHC, many countries have developed health benefits packages, which are a list of interventions that are available for individuals free of charge or at affordable rates subsidized by the government. The design of health benefits packages depends on the context in each country which is often based on a prioritization process set up to incorporate technical and/or social values. Managing the transition to UHC has meant that many countries are attempting to widen access to health care, and expanding the treatments available in their benefits packages, without healthcare costs ballooning to unmanageable levels.
Health technology assessment (HTA) is a multidisciplinary process that uses explicit methods to determine the value of health technologies at different points throughout the product life cycle (Reference O’Rourke, Oortwijn and Schuller3). The purpose of HTA is to inform decision making and thus to promote an equitable, efficient, and high-quality health system (4). Globally, the growing commitment to UHC is underpinned by an increasing role for HTA to support evidence-informed policy (1;5;6). Every country or jurisdiction has different processes and the decision about if and how to implement HTA in their decision-making process depends on several factors, including national policy goals and criteria for defining their essential health services (Reference Teerawattananon, Rattanavipapong and Lin7). Health economic evaluations are one of the HTA tools often used by policy makers to help decide whether new health technologies offer better value-for-money compared to current standards of care (Reference Drummond, Sculpher and Claxton8).
Impact assessments of HTA are often incorporated into a country’s monitoring and evaluation of HTA and appraisal processes. Monitoring of the impact of HTA reports should be a quality measurement element of any HTA agency (Reference Velasco Garrido, Kristensen and Nielsen9). Only one-third of the HTA agencies from the International Network of Agencies for Health Technology Assessment (INAHTA) reported to have a formal impact assessment, which included the utilization of HTA reports, public awareness and media coverage, satisfaction, changes in the policy process, changes in policy decisions, and changes in practice (Reference Berndt, Schuller and Hailey10). Previous studies highlighted the benefit of HTA studies and outcomes in terms of health benefits, economic benefits, and other outcomes such as time-to-access of medicines (Reference Chinnery, Bashevoy and Blatch-Jones11–Reference Grieve and Briggs13).
In Thailand, HTA plays an important part in healthcare coverage decisions at the national level, including the development of: both the National List of Essential Medicines (NLEM) (the pharmaceutical reimbursement list) since 2008, and the Universal Coverage Benefits Package (UCBP) (the nonpharmaceutical benefits list) since 2009 (Reference Mohara, Youngkong and Velasco14;Reference Teerawattananon, Tritasavit, Suchonwanich and Kingkaew15). In the past decade, numerous health technologies have been evaluated using HTA, and as a result these technologies have been funded by public organizations. Most appraisals for new and high-cost health technologies considered cost-effectiveness and budget impact evidence for the decision-making process in public health insurance programs (Reference Hampson, Henshall and Towse16). The economic aspect of HTA is particularly important due to its quantifiable nature, which facilitates comparisons between health interventions. An additional benefit of using economic evaluations is support for price negotiation (Reference Teerawattananon, Tritasavit, Suchonwanich and Kingkaew15). However, economic evidence is not the only information used in the appraisal and decision-making process for priority setting new health technologies, but also ethics, equity, acceptability, and feasibility are taken into consideration as well (Reference Teerawattananon, Tritasavit, Suchonwanich and Kingkaew15).
The Thai experience shows how establishing HTA institutions and incorporating HTA processes into national level decision-making processes can strengthen the reliability and validity of decisions. HTA processes can serve as a priority-setting tool that aligns with the country’s overall policy and direction. While contextual factors must be considered to establish suitable HTA processes, considering cost-effectiveness information enhances the “good value-for-money” approach which is especially crucial in low- and middle-income country (LMIC) settings where resources are limited. Despite misconceptions that persist about HTA and feasibility issues, and hindrances to HTA processes created by values, attitudes and politics (Reference Teerawattananon, Painter and Dabak17), establishing HTA is still a worthwhile ambition.
Institutionalizing HTA requires resources and strong policy commitments. Kim et al. underline the importance of illustrating the value of HTA and HTA agencies by generating evidence on the return on investment as well as the economic and health impact of HTA (Reference Kim, Sharma and Teerawattananon18). Such efforts can highlight the fact that while HTA agencies require financial support, they are an efficient value-for-money policy tool. As such, the objective of this study is to simulate the expected net monetary value of different allocated funding criteria focusing on the use of cost-effectiveness information using data from Thailand. In addition, it can help to demonstrate the value of continued and strengthened investment in HTA in LMICs. Hereafter, mentions of “HTA research” specifically refer to cost-utility and budget impact analysis evidence.
Methodology
A previously developed simulation model that used real-world data from the United Kingdom and Malawi, the Evaluating the Value of a Real-world HTA agency (EVORA) model, was adapted to the Thai context. More details about the original model can be found in the associated publication (Reference Morton and Barlow19;Reference Barlow, Morton and Dabak20). The simulation model, developed in Microsoft Excel, allows users to simulate the impact of using HTA-based decision criteria in a public healthcare system, where it then adopts or rejects hypothetical new health technologies using a threshold-based HTA decision rule. The simulation model then provides a quantitative comparison of alternative approaches to making decisions on which considered health technologies should be allocated funding, between an HTA-based funding rule and a counterfactual first-come, first-served (FCFS) funding rule. The time horizon of the model was 5 years.
Comparisons are given in terms of the overall financial impact (in monetary values) and anticipated health benefits in quality-adjusted life-years (QALYs), which could be expected from pursuing different funding approaches. These comparisons therefore provide a measurable estimate of the potential benefit of using HTA-based decision rules (in terms of costs and benefits). All costs are reported in Thai Baht (THB) and US Dollars (USD), a conversion rate of USD1 = THB33.4231 from 31 Jan 2022 was used. The primary output of the model was the net monetary benefit (NMB) of using HTA-based decision criteria compared to a counterfactual decision rule (Reference Messori and Trippoli21). The NMB was calculated using the following equation, where CET refers to the cost-effectiveness threshold (which is THB160,000 [USD4,787]) and HTA and FCFS represent the HTA-based and FCFS decision rule scenarios, respectively.
$$ \mathrm{NMB}\hskip0.35em =\hskip0.35em \left({\mathrm{QALYs}}_{\mathrm{HTA}}-{\mathrm{QALYs}}_{\mathrm{FCFS}}\right)\times \mathrm{CET}-\left({\mathrm{Costs}}_{\mathrm{HTA}}-{\mathrm{Costs}}_{\mathrm{FCFS}}\right). $$The threshold-based decision rules for the HTA scenario used in the model are:
1. Incremental cost-effectiveness ratio (ICER) versus CET decision rule only, with no budget limit.
2. ICER versus CET decision rule up to a budget limit.
The impact of HTA is measured in terms of spend and health gain and then compared with a healthcare system which implements decisions based on two possible criteria:
1. FCFS (random) health technology selection up to a budget limit.
2. FCFS (random) health technology selection up to a limit of technologies.
A stakeholder meeting was held to discuss and refine the methods of this study and included a range of public health professionals, representatives from the NLEM and UCBP committees, and researchers in Thailand. Based on the discussions that took place in the stakeholder meeting, the most appropriate HTA-based and counterfactual funding rules for Thailand were identified. The discussion led to a consensus agreement to conduct two sets of analyses:
Analysis 1: Budget limit. A comparison between the ICER versus CET decision rule with a budget limit HTA scenario, compared to a FCFS health technology selection up to a budget limit scenario;
Analysis 2: Technology limit. A comparison between the ICER versus CET decision rule with no budget limit scenario, compared to a FCFS health technology selection up to a maximum number of technologies scenario.
When the maximum number of projects that could be approved was set for the counterfactual scenario, this value was assumed to be ten projects per year (i.e., fifty projects over the model time horizon). The total number of simulated projects for consideration in each year was thirty for both analyses. These parameters were assumed based on historical data for the average number of topics considered and nominated for HTA analyses in Thailand.
Three key adaptations were made to the original model to make it more appropriate for the Thai context: (i) real-world cost-effectiveness data from Thai HTA studies were used; (ii) the Thai CET was used; and (iii) a cost of conducting HTA research was added to the model. All HTA reports developed as part of the coverage decision process for the NLEM and UCBP committees from 2008 to 2020 were collected and reviewed. These reports were selected against the inclusion criteria that the analyses must have been assessed by either the committees for the NLEM or UCBP, the analyses must be cost-utility analyses (and reported incremental costs and incremental QALYs). The exclusion criteria were HTA reports that reported other study designs, such as feasibility studies or budget impact analyses. Duplicated HTA reports were also excluded. In total, thirty-seven HTA reports from the NLEM process, and twenty-two reports from the UCBP process were identified, with a total of 185 individual technology appraisals (see Figure 1). The final number of reports included for the analyses were thirty-four HTA reports from NLEM and twenty-one HTA reports from UCBP.
Figure 1. Flow diagram showing screening, identification, eligibility, and inclusion of health technology assessment reports for data extraction. HTA, health technology assessment; NLEM, National list of essential medicines; UCBP, universal coverage benefit package.
From the fifty-five included HTA reports, the intervention, comparator, year of the study, size of population, budget of project, incremental cost (government perspective), incremental cost (societal perspective), incremental QALYs were extracted from each technology appraisal. The characteristics of the included technology appraisals are described in Table 1. Further details are available in Supplementary File 1. These data were used as input parameters in the model. Approximately one-third of the included technology appraisals were for the treatment and management of cancers. The CET of THB160,000 (USD4,787) per QALY matched the threshold recommended by the Subcommittee for Development of Benefits Package and Service Delivery, the National Health Security Office (NHSO) in Thailand, which was adopted in 2013 (Reference Chaikledkaew and Kittrongsiri22). The budget limit was estimated based on the mean year-on-year increase in the NHSO budget between 2012 and 2021. It was set at THB5,836 million (USD175 million) per year. The average cost of performing HTA research (THB213,244 [USD6,380]) was added to the model, estimated from the budgets allocated for the technology appraisal projects for the NLEM process.
Table 1. Characteristics of Included HTA Reports
Abbreviation: HTA, health technology assessment.
Results
Low correlation (0.297) was found between the incremental costs and incremental QALYs of the included technology appraisals (Figure 2), which suggested that most of the health interventions that were being assessed in Thailand are high cost with a small increment in health benefits.
Figure 2. Scatter plot of incremental costs and incremental quality-adjusted life-years for included Thai health technology assessment appraisals. USD1 = THB33.4231.
The results of the analyses are listed in Table 2 below. In both analyses, the HTA-based decision rule scenario was associated with a greater NMB compared to the counterfactual FCFS scenarios. The NMB ranged from THB24,238 million (USD725 million) to THB759,328 million (USD22,719 million). HTA-based decisions led to fewer costs and superior health outcomes (more QALYs). Three fewer projects were funded under the HTA-based decision rule in the budget limited analysis, as only cost-effective technologies up to the total budget limit were approved. Whereas three more projects were funded in the HTA-based decision rule in the technology limit analysis.
Table 2. Model Results
Note: Budget limit: ICER versus CET up to a budget limit compared to FCFS up to a budget limit; Technology limit, CET only compared to FCFS up to a number of technologies limit. USD1 = THB33.4231.
Abbreviations: CET, cost-effectiveness threshold; FCFS, first-come, first-served; ICER, incremental cost-effectiveness ratio; QALYs, quality-adjusted life-years; THB, Thai Baht.
Box plots of the incremental costs and incremental QALYs accrued from HTA and the counterfactual in both analyses are displayed in Figure 3. The average cost per additional QALYs was THB140,000 (USD4,189) for the FCFS scenario in the budget limit analysis, compared to THB70,000 (USD2,094) in the HTA scenario. In the corresponding technology limit analysis, the average cost per QALY was THB551,000 (USD16,486) in the FCFS scenario, and THB269,000 (USD8,048) in the HTA scenario. These results show that when comparing a variety of plausible HTA funding scenarios to plausible counterfactual non-HTA based decision criteria, HTA-based decision making was shown to yield superior health outcomes and lower costs.
Figure 3. Box plots of incremental QALY gains and incremental costs (Thai Baht) for explored HTA and FCFS scenarios. (A,B) Budget limit analysis. (C,D) Technology limit analysis. FCFS, first-come, first-served; HTA, health technology assessment; QALY, quality-adjusted life-year. USD1 = THB33.4231.
Discussion
The results demonstrate that an HTA-led framework for evaluating decisions of adoption or rejection of new health technologies yields superior outcomes to a FCFS based decision-making process. Furthermore, this work shows that the original model developed by the EVORA team can be adapted using local data to provide contextualized results, to better understand the potential impact of HTA-based decision rules compared to other decision-making processes for the adoption or rejection of health technologies (Reference Morton and Barlow19;Reference Barlow, Morton and Dabak20). To the authors’ knowledge, this research is the first to estimate the potential impact of HTA-based decision rules in a middle-income country context. One key, and perhaps unexpected, difference between the Thai data and UK data from the original model was that the correlation between incremental costs and incremental QALYs from the Thai HTA data was much lower than for the UK data (0.297 compared to 0.998), meaning that HTA studies in Thailand tended to have less correlation between the incremental costs of interventions and the incremental QALYs gained, compared to the UK data. This may have been due to the selection process for the data informing this correlation measure. The 185 technology appraisals that comprised the Thai data set were largely pharmaceutical interventions. Whereas the UK data set included ten technology appraisals, these were varied and included interventions such as acupuncture, telemedicine programs, screening programs and ventilators. However, none of the considered interventions in the UK data set were pharmaceutical interventions. Greater NMBs of HTA-based decision rules can be demonstrated when decisions are made in this case. Additionally, this analysis illustrates the importance of using local data to produce context-specific estimates.
The two examined decision-making strategies in this analysis may be understood as two extremes, with decision making for the adoption or rejection of new health technologies either based strictly on the cost-effectiveness of a new technology, or alternatively, a random choice process. Allocation criteria using the combinations of economic value and affordability are increasingly important for reimbursement decision in LMICs (Reference Hampson, Henshall and Towse16). It is reasonable to assume that, in reality, policy makers who are making decisions without an HTA-based framework and cost-effectiveness analysis may outperform a random decision-making process; however, as the decision-making criteria in these circumstances are not explicit it could not be simulated in the model. Therefore, a key limitation of the model is that it is a simplification of reality and may underestimate the economic performance of a real-world counterfactual scenario to HTA-based decision-making. Furthermore, no explicit uncertainty analysis was conducted for this study, though the average cost and outcome estimates for each included HTA report were used. Hence, this study assumes that all HTA studies were conducted and presented appropriately, which underlines the importance of HTA studies following national or international guidelines to ensure quality and consistency. In Thailand, HTA methods and process guidelines have been applied for all HTA studies to be used to inform coverage decisions in Thailand since 2007.
This analysis considered an HTA-based process only as a methodology to understand the cost-effectiveness of health technologies; however, HTA is a more multifaceted approach that considers social and ethical issues, amongst others, as is well-documented elsewhere (Reference O’Rourke, Oortwijn and Schuller3;Reference Teerawattananon, Rattanavipapong and Lin7), and provides other unquantifiable benefits, such as increased transparency, reduced corruption and increased empowerment for participating stakeholders (decision makers, physicians and healthcare providers, academics, patients and the public). Another benefit of HTA which was not considered in this analysis is the impact that HTA has on price negotiations with manufacturers of health technologies, that is, the model assumes that the incremental cost of a new health technology remains the same regardless of the decision-making criteria in use. We strongly suspect that the use of CETs for reimbursement decisions have a downward pressure on prices charged by manufacturers (Reference Teerawattananon, Tritasavit, Suchonwanich and Kingkaew15).
We also speculate that the presence of an HTA-based decision-making process would impact the mix of interventions under consideration, particularly in systems where manufacturers are able to submit to HTA agencies for public reimbursement (which is not the case in Thailand), by discouraging submissions for interventions that are prohibitively expensive and/or ineffective. Furthermore, the data used in the model were obtained from the Thai system, where an HTA-based decision rule is in place and topics are selected systematically as part of the HTA process. As a result, the mix of interventions has likely been biased toward a more cost-effective mix than would be the case without the topic selection process, which in turn would increase the estimated benefit from using an HTA-based decision rule. All of these unconsidered benefits of HTA would further increase the NMB of an HTA-based decision-making process. The estimates produced in this analysis only illustrate the potential impact of using HTA-based decision rules, and do not consider the issue of implementation as other factors may contribute to the access of health interventions. Nonetheless, it was assumed that implementation issues would be unaffected between the HTA-based decision rule scenario and the counterfactual scenario.
Conclusions and recommendations
This research adds to a growing body of literature that aims to quantitatively understand the potential benefits of HTA using an empirical approach in collaboration with relevant stakeholders (Reference Grieve and Briggs13;Reference Allen, Liberti, Walker and Salek23;Reference Salas-Vega, Bertling and Mossialos24). This research can provide useful lessons for other LMICs. Firstly, the study illustrates the importance of monitoring and evaluation of HTA processes and data collection, which made the adaptation of the simulation model to the Thai setting possible. HTA agencies are urged to monitor their HTA impact to improve and optimize evidence-based decision-making process (Reference Barlow, Morton and Dabak20;Reference Frønsdal, Facey and Klemp25). This research can be used by researchers in other LMICs to perform contextualized research, similar to this analysis, in their own settings. The findings of this research are instructive regarding potential benefits that HTA can provide by improving resource allocation decision making and the consequential health and economic benefits that can be captured by doing so. Countries not yet using HTA methods could consider this evidence as support for their HTA processes which ultimately can be used to inform resource allocation decisions for their health budgets.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Authors Contributions
All authors contributed to the design and implementation of the research and review of the manuscript.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0266462322000277.
Open access
