1. Background
Health technology assessment (HTA) is the systematic evaluation of the properties, effects, or impacts of health technology (Goodman, Reference Goodman1998; WHO, 2021).Footnote 1 It is used by both public and private organisations to inform decisions about the availability of interventions within the health system, with profound implications for the health and wellbeing of members of society.
HTA makes significant use of empirical evidence, but its evaluative aim is inherently normative – that is, value-based (Hofmann et al., Reference Hofmann, Cleemput, Bond, Krones, Droste, Sacchini and Oortwijn2014, Reference Hofmann, Bond and Sandman2018; Legault et al., Reference Legault, Gagnon, Parent, Bellemarre, Beland, Kocsis-Bedard, Bernier, Dagenais, Charles-Etienne and Patenaude2021). The normative grounds for healthcare priority-setting are highly contested: there is little societal agreement about what constitutes a fair distribution of scarce healthcare resources. Healthcare priority-setters and those undertaking HTA for policy purposes thus face a significant challenge in seeking to make well-justified decisions that will be accepted by all as fair and legitimate in the face of disagreement about what is the right decision.Footnote 2 This challenge may be even greater when priority-setting decisions are made by private organisations that lack the political authority of public institutions.
To date, problems of legitimacy and fairness have been addressed largely through procedural means. Norman Daniels and James Sabin's well-known accountability for reasonableness framework, for example, is a widely adopted approach that rests heavily on the requirement that those employing HTA to inform healthcare priority-setting are transparent about their decisions and the reasoning behind them (Daniels and Sabin, Reference Daniels and Sabin1997). However, defensible HTA decision-making also requires that the substantive rationales underpinning such decisions stand up to moral scrutiny and are clearly enough articulated to allow for reasoned debate between stakeholders who may disagree about the course of action (Culyer and Lomas, Reference Culyer and Lomas2006; Daniels and Van der Wilt, Reference Daniels and Van der Wilt2016). An approach that incorporates practical public reasoning – in which decisions are made on behalf of the public, in public view, through a chain of reasoning that brings normative commitments together with empirical evidence in reaching a conclusion that can be morally justified on both procedural and substantive grounds – has the potential to not only improve decision-making, but also to strengthen fairness and the perceived legitimacy of the HTA body and its decisions (Weale, Reference Weale2010; Rumbold et al., Reference Rumbold, Weale, Rid, Wilson and Littlejohns2017; Charlton and Weale, Reference Charlton and Weale2021).Footnote 3
To date, the practice of practical public reasoning in healthcare priority-setting has been hindered by limitations in the language used to articulate normative aspects of HTA (Bellemare et al., Reference Bellemare, Dagenais, K-Bedard, Beland, Bernier, Charles-Etienne, Gagnon, Legault, Parent and Patenaude2018; Charlton and Weale, Reference Charlton and Weale2021). This paper attempts to address this challenge.
2. Normativity in HTA: an enigmatic sphere, ambiguously described
There have been many calls in the literature for increased attention to be given to the ethical dimensions of HTA (Hofmann, Reference Hofmann2008; Saarni et al., Reference Saarni, Hofmann, Lampe, Lühmann, Mäkelä, Velasco-Garridod and Autti-Rämö2008; Schokkaert, Reference Schokkaert2015; Bellemare et al, Reference Bellemare, Dagenais, K-Bedard, Beland, Bernier, Charles-Etienne, Gagnon, Legault, Parent and Patenaude2018; Legault et al., Reference Legault, Gagnon, Parent, Bellemarre, Beland, Kocsis-Bedard, Bernier, Dagenais, Charles-Etienne and Patenaude2021; Oortwijn and Sampietro-Colom, Reference Oortwijn and Sampietro-Colom2022). However, across academic literature and in the description and practice of applied HTA, normative aspects of decision-making are described using a variety of imprecise terms, which are employed inconsistently and lack the sensitivity to distinguish between different types of normative commitment. This gives rise to ambiguity about the rationale for individual priority-setting decisions. While such ambiguity may serve a useful function in helping decision-makers to respond pragmatically to ethically and politically challenging circumstances – that is, to ‘muddle through elegantly’ (Hunter, Reference Hunter1995; Mechanic, Reference Mechanic1997; Calnan et al., Reference Calnan, Hashem and Brown2017) – it also potentially facilitates dubious practices such as the deliberate obfuscation of reasons that cannot be morally justified or the abandonment of principles in favour of political, professional, or personal interests. Intentional or not, ambiguity in the language used to describe normative aspects of HTA is therefore problematic.
One term frequently employed in this context is ‘social value judgement’, whose use was popularised through a 2005 document produced by the UK's National Institute for Health and Care Excellence (NICE) – an acknowledged HTA innovator. ‘Social Value Judgements: Principles for the development of NICE guidance’ used the term to describe what NICE understood to be its normative commitments, defining it in explicitly moral terms as ‘an ethical opinion […] that a particular course of action, institutional arrangement or method of analysis ought to be implemented, or is itself good’ (NICE, 2005). NICE's definition of ‘social value judgement’ has since undergone several iterations.Footnote 4 At present, it grounds the term firmly on social rather than moral norms, describing a social value judgement as a judgement that ‘take[s] account of society's expectations, preferences, culture and ethical principles’ (NICE, 2021). It is unclear what this definition implies about the relationship between social and moral values, however, or how this notion should be applied to ethical questions on which societal consensus is lacking.Footnote 5 It is also unclear how NICE distinguishes between a social value judgement and a principle – an alternative term that has recently become more prominent in NICE documentation (NICE, 2020).
A lack of consistency and clarity in adopted language is also evident within and across other HTA bodies. Australia's Pharmaceutical Benefits Advisory Committee (PBAC) distinguishes between ‘quantifiable’ and ‘less-readily quantifiable’ considerations in HTA, including in the latter category specific normative issues such as ‘implicit equity and ethical assumptions such as age, or socioeconomic and geographical status’ (PBAC, 2016). France's Haute Autorité de Santé (HAS) refers to such issues as part of its ‘assessment of ethical aspects’, which it defines as those matters that ‘involve the values that concern the conditions of living together’ – that is, ‘social values’ (HAS, 2014). The Canadian Agency for Drugs and Technologies in Health (CADTH) also conducts an ‘ethics review’ for certain complex technologies, intended to describe the ‘ethical issues relevant to the drug's target population(s), evidentiary basis, use, implementation and outcomes’ (CADTH, 2023).
The Swedish Agency for Health Technology Assessment and Assessment of Social Services (SBU) groups together ‘ethical and social aspects’ related to a technology's application (SBU, 2018), while the US Institute for Clinical and Economic Review (ICER) additionally includes legal issues within this category of ‘contextual considerations’ (ICER, 2020). In contrast, the HTA Core model – a methodological framework developed by a network of European HTA agencies – treats ‘ethical analysis’, ‘patient and social’, and ‘legal’ as separate domains, while acknowledging the relevance of ‘value judgements’ to all domains of HTA (EUnetHTA, 2016). Given the variation in language across just these few examples, common understandings are difficult to identify, and it is perhaps unsurprising that HTAGlossary.net (2021) includes none of these terms within its 300+ entries.
Similar issues are also present in the academic literature. In a 2012 journal issue focused on ‘social values’ in health priority-setting, one paper describes these as ‘the values of the public or society, including their moral values’, suggesting that the latter is a subset of the former (Biron et al., Reference Biron, Rumbold and Faden2012), while another describes them solely as ‘the moral or ethical values of a particular society’ (Clark and Weale, Reference Clark and Weale2012). Stafinski et al. (Reference Stafinski, Menon, Marshall and Caulfield2011) typify the health economic literature in defining social values in non-moral terms as the ‘distributive preferences of the public for healthcare among populations’. Unusually, Orr et al. (Reference Orr, Wolff and Morris2011) distinguish between the preferences of individuals and the ‘citizen perspective’, defining social values as ‘values that the public generally feel are important for society at large, often regardless of one's own preferences’. More unusually still, Nicod and Kanavos (Reference Nicod and Kanavos2016) define them as ‘non-elicited preferences’ that, in the context of HTA, ‘originate from the individual appraisal committee member's value judgment based on their experience or on what they believe society would prefer’. According to a recent systematic review, many authors avoid definitional problems by simply not specifying the nature of the normative concepts that they describe, in particular ‘whether they consist of social values, moral norms, or value judgments’ (Bellemare et al., Reference Bellemare, Dagenais, K-Bedard, Beland, Bernier, Charles-Etienne, Gagnon, Legault, Parent and Patenaude2018).
The aim of this paper is to address such difficulties directly by defining key terms and presenting a tool that allows reasoning in HTA to be more clearly and precisely articulated. Our hope is that this will be of practical use to HTA practitioners and policymakers, while also supporting the work of those who seek to hold healthcare priority-setters to account. In addition, we hope that the paper will stimulate more constructive discussion in the academic literature by reducing confusion across disciplinary boundaries.
3. Methods
This paper represents the output of a collaboration involving 24 co-authors, all of whom have significant knowledge and expertise in the policy, ethics, and/or economics of healthcare priority-setting. The frameworkFootnote 6 that it describes was developed through a group method in which the problem identified above was characterised through several rounds of discussion, before an approach to addressing it emerged through further cycles of idea generation, discussion, and refinement (Steyaert and Bouwen, Reference Steyaert, Bouwen, Cassell and Symon2004). The preparation of the current paper was led by a small writing groupFootnote 7 with all other co-authors contributing to the critical revision of an initial draft. The proposed framework is offered as a starting point for further debate and development and it is hoped that engagement with others working in this field will contribute to its legitimacy as a practical tool to support priority-setting.
We proceed by providing a brief overview of the proposed framework, before introducing each of its key concepts and illustrating its utility through application to a simplified hypothetical case. We then discuss the framework's advantages in terms of its ability to facilitate transparency across the chain of reasoning and increase moral scrutiny of decision-making. The paper concludes by considering how the framework might be used to improve the legitimacy and fairness of HTA as a tool for public policy.
4. Results
4.1 Overview: a tool for articulating normative reasoning in HTA
The framework rests on the view that HTA is an activity which is shaped by a range of evaluative considerations, claims, and beliefs. It terms these normative commitments. Normative commitments differ from one another both in their content (i.e. in their conception of how the world ought to be) and in their degree of specification (i.e. the extent to which they guide specific action). We have argued in the previous section that current limitations of language regularly obscure the chain of reasoning that links such commitments in justifying decisions informed by HTA.
The framework addresses this limitation by distinguishing between values, principles, standards, and case-based judgements: four relatively distinct types of normative commitment that range from the highly abstract (values) to the context specific (case-based judgements) (Figure 1; Table 1). The lines between these categories are not absolute. But, in general, values will be set at the organisational level and may also reflect beliefs held across the health system (or even society) as a whole. Principles will usually be organisational, standards will typically relate specifically to a programme of technology assessment, and case-based judgements will be made in response to the consideration of an individual technology and/or indication. Use of this framework therefore facilitates a structured articulation of the chain of reasoning that underlies decision-making – from more abstract and indeterminate values down to highly specified case-based judgements – and clarifies the role that empirical evidence has played in this chain.
Figure 1. Overview of the framework.
Table 1. Definitions
The practical utility of this approach will be illustrated through the framework's application to a hypothetical case, introduced below. It is important to note that this case is, by design, simplified and is not intended to reflect the approach of any particular HTA body. As will become evident, it is also not constructed to illustrate a morally justified decision. Rather, it aims to demonstrate how the framework can make normative aspects of HTA more transparent, such that the rationale for decisions is made more open to reasoned debate and scrutiny.
4.2 The hypothetical case: GehrigoleFootnote 8 for the treatment of amyotrophic lateral sclerosis
Gehrigole is a new drug treatment for amyotrophic lateral sclerosis (ALS), one of several disorders in which degeneration of motor neuron cells in the brain and nervous system leads to muscle weakness. ALS generally affects people aged 50–65 and typically leads to substantially reduced quality of life and death within 3–5 years of diagnosis, often due to respiratory failure. Gehrigole's manufacturer claims that its new drug counteracts ALS by acting on a molecular pathway that has not previously been the target of pharmacological treatment. As a result, the national healthcare regulator has formally designated Gehrigole an ‘innovative medicine’.
In a large head-to-head randomised controlled trial (RCT) conducted in South-East Asia over 3 years, Gehrigole appeared to show some benefits compared to standard treatment. When initiated immediately following diagnosis, patients receiving Gehrigole reported a clinically meaningful improvement in symptoms and their requirement for assisted ventilation was delayed by an average of 53 days compared with the control group. Average length of post-diagnosis survival was also 20 days longer than in the control group, although this relationship did not reach statistical significance. Based on application of a standardised quality of life questionnaire, these benefits were associated with a utility gain of approximately 0.15 quality-adjusted life-years (QALYs) compared with existing treatment.
In assessing this evidence, the national regulator expressed some concern about approving a novel medicine based on a single, relatively short trial and noted that there was uncertainty about how generalisable the findings would be to its own population. However, as a result of Gehrigole's ‘innovative medicine’ designation, the regulator judged that it was appropriate to tolerate a relatively high level of uncertainty and deemed this sufficient evidence to grant marketing authorisation.
Priority-setter A has authority to make recommendations on behalf of the national health system. It delegates its decision-making about individual cases to an independent committee made up of clinicians, health economists, and patient representatives. This committee is using HTA to inform its view on whether Gehrigole should be adopted.
4.3 Values
Values are the most abstract of the framework's four types of normative commitment.
Value is generally understood to be a claim about something's worth. However, philosophers and economists tend to differ in their application of the concept to HTA. For economists, value is usually an empirical claim about the extent to which certain states or things are observed (or believed) to be preferred over others. This notion of value plays an important role in HTA, primarily as a source of empirical evidence about a technology's anticipated effects, in terms of either comparative clinical effectiveness or cost-effectiveness.Footnote 9 This economic conception of value is, however, distinct from the philosophical understanding of a value as an abstract end that is worth pursuing because it is ‘good’ or ‘right’: ends such as justice, liberty, dignity, and happiness. It is this latter conception of value that the framework describes as a form of normative commitment.
Within this philosophical conception, a distinction can be drawn between socially and morally based values. In a society in which there is a strong sense of ‘common morality’, these may be closely aligned. However, social and moral values can also diverge. Society may hold dubious values which, though popular, cannot be morally justified. Similarly, individuals may hold self-interested preferences that differ from what they believe is ‘right’.Footnote 10 Preference-derived social values may also not reflect the views of ‘society’ in any simple sense; rather, they may represent the views of the majority, or of special interests, or of a group charged with expressing such views on behalf of society, rather than reflecting the plurality of views present across society as a whole (Baker et al., Reference Baker, Mason, McHugh and Donaldson2021; Charlton and Weale, Reference Charlton and Weale2021).
In its formal charter, priority-setter A promises that it will act in accordance with three values: justice, liberty, and integrity. It does not commit to any single moral theory but considers these values to be morally based because they reflect its desire to act for the good of society. It also considers them to be socially based in so far as they are thought to reflect society's views about how public bodies ought to act. The influence of these values is also evident in the legislation through which priority-setter A was established and in which its statutory role is defined: to ‘provide unbiased recommendations on the fair distribution of healthcare resources across the population’. Priority-setter A pledges to be guided by these values in its consideration of Gehrigole, as in all its actions.
4.4 Principles
All attempts to set healthcare priorities rest on values, whether or not they are explicitly acknowledged. However, values are, by their nature, insufficiently specified to determine the choice of one course of action over another. One way of specifying how particular values are to be understood and interpreted is through the stipulation of principles. Principles broadly capture the conditions under which an action might be regarded as right or wrong and therefore serve as a pledge to act in a certain way, as a guide for behaviour, or as the basis for part of a chain of reasoning.
Principles can be categorised as either substantive or procedural. Substantive principles relate to what decisions are made and why, specifying features that ought to be manifested in the decision outcome. Procedural principles relate to how decisions are made, specifying features that ought to be present in the decision-making process. The claim that health systems should generate as much population health as possible – a principle of health maximisation – is a substantive principle that represents one possible specification of the value of justice. However, the value of justice might also be specified through a procedural principle of consistency, according to which priority-setting decisions should be reached through the application of a consistent set of procedures. In some cases, different specifications of a given value may give rise to principles that are in tension with one another, as may principles derived from different values. Such tensions can be resolved either through further specification of general rules of action (i.e. through standards) or on a case-by-case basis (i.e. through case-based judgements).
Priority-setter A specifies its commitment to the value of justice through three substantive principles. First, it states that cases that are alike in morally relevant respects should be treated similarly. It refers to this as its principle of equality. Second, it proposes that everyone should have the ability to live life in a reasonable state of health, and that priority should therefore be given to addressing the clinical needs of those in poor health. It refers to this as its principle of clinical need. Third, it considers that the health service (which operates under a fixed budget) exists to serve everyone and that, all else being equal, adoption of a technology that displaces more health than it generates is unjust. It refers to this as its principle of efficiency. Priority-setter A acknowledges that these principles may sometimes be in tension with one another and that there is no moral or societal consensus about how they should be balanced. It also acknowledges that uncertainty in the available evidence may sometimes make it difficult to reach reliable conclusions about how the use of any given technology might impact on the realisation of these principles.
In seeking to act justly and with integrity, priority-setter A has also specified several procedural principles. These include principles of consistency, transparency, inclusiveness, independence, and consensus-based decision-making. It has also pledged to act in accordance with the value of liberty, which it believes is directly served by respecting the rule of law. This last principle takes precedence over all other substantive and procedural principles.Footnote 11
4.5 Standards
At a further level of specification lie standards: ways of doing things that are accepted by authority, precedent, custom, or general consent. Like principles, standards can be either substantive or procedural.
In their operationalisation of values and principles, standards act as a bridge between relatively abstract normative commitments and the specific judgements that must be made in response to individual cases. Formal standards are standards that have been codified in policy. For example, the so-called ‘reference case’ in HTA tends to comprise multiple formal substantive standards that govern how a technology's potential impacts should be measured and evaluated,Footnote 12 while other policies may stipulate when exceptions to the reference case might be considered acceptable. Formal procedural standards may describe different features of the overall decision-making process, such as how participants in appraisal processes will be selected, how the process will be facilitated, and whether meetings will be open to the public (Oortwijn et al., Reference Oortwijn, Husereau, Abelson, Barasa, Bayani, Canuto Santos, Culyer, Facey, Grainger, Kieslich, Ollendorf, Pichon-Riviere, Sandman, Strammiello and Teerawattananon2022). Unlike formal standards, informal standards are established through precedent and are not codified in policy. While formal standards are likely to be applied consistently – because departure from them will be visible and can be easily challenged – informal standards are less obvious to external stakeholders and may be applied inconsistently. Over time, informal standards may become entrenched and eventually formalised.Footnote 13
Standards offer a way of specifying normative commitments, but they may also be heavily informed by empirical evidence, including economic conceptions of value. For example, an insurer might use evidence on the public's willingness to pay for a particular drug to inform standards concerning who is given access to that drug and under what terms. Indeed, the technical nature of some standards may conceal their normativity from all but those with relevant expertise. Standards concerning discount rates, for instance – that is, the rate at which future health gains should be discounted compared with health gains experienced immediately – are highly technical in their use of empirical evidence (Attema et al., Reference Attema, Brouwer and Claxton2018), but also rest on normative choices (O'Mahony and Paulden, Reference O'Mahony and Paulden2014). The normative nature of such technically complex standards often goes unacknowledged and is rarely discussed during the ‘ethical analysis’ phase of an HTA process or as part of HTA bodies' reflection on their ‘social value judgements’.
One type of formal standard frequently adopted by HTA bodies is a cost-effectiveness threshold. Different conceptions of this standard are discussed in Box 1. HTA bodies that do not rely on cost-effectiveness analysis may instead base their decisions on formal standards used to determine the added clinical benefit of particular technologies, which then informs pricing decisions.
Box 1. Conceptions of cost-effectiveness
1. NICE (2021).
2. Bullement et al. (Reference Bullement, Taylor, McMordie, Waters and Hatswell2019).
Priority-setter A has specified its substantive principle of efficiency by embedding several formal standards within its reference case. These include use of cost-utility analysis, measurement of health gains in QALYs, and the exclusion of non-health and indirect effects. The use of a reference case also partially operationalises its procedural principle of consistency. In addition, priority-setter A has established a formal standard which requires it generally to reject technologies whose cost exceeds $50,000/QALY. Although direct evidence for the opportunity cost associated with the adoption of new technologies is not available for this health system, this ‘cost-effectiveness threshold’ has been established through precedent and is considered by priority-setter A to be the point at which a new technology's adoption is likely to displace more health than it generates. It is therefore an attempt to operationalise its principle of efficiency, as well as its principle of equality and its conception of the value of justice, by ensuring that known and unknown beneficiaries of the health system are treated equally, absent a specific reason to do otherwise.
One such reason is disease severity, arising from its principle of clinical need. The relative weight of this principle compared with other substantive principles is not specified by any formal standard. However, priority-setter A has shown itself willing in several previous cases to recommend technologies whose costs exceed $50,000/QALY if clinical need is judged to be substantial and the estimated resource impact is small. It has also shown itself willing to tolerate greater uncertainty if the potential health benefits are large, although this is also not formally acknowledged or justified. These constitute informal standards that are relatively unspecified and may be interpreted differently in different cases, potentially undermining priority-setter A's substantive principle of equality and its procedural principle of consistency.
In addition to these substantive standards, several procedural standards are in place to detail how such decisions should be reached in a way that accords with its principles of inclusiveness, independence, and consensus-based decision-making.
In consideration of these standards, the manufacturer of Gehrigole has presented its evidence as a cost-utility analysis. This analysis includes evidence relating to patients' functional abilities, survival, and quality of life but excludes evidence about indirect and non-health effects such as the impact on family members and on patient employment and earnings. The manufacturer is aware of priority-setter A's informal standard relating to clinical need and has drawn attention to the severity of ALS, the relatively large health benefits potentially offered by Gehrigole and its relatively small resource impact. The manufacturer has highlighted in its communications with priority-setter A that it considers this evidence to be robust and its view that Gehrigole is highly likely to deliver the clinical benefits presented in its analysis.
4.6 Case-based judgements
The most specified type of normative commitment is case-based judgements: considered conclusions reached through the evaluation of information relevant to a specific context or case. The framework classifies case-based judgements into two forms: evidential and ethical.
Evidential judgements are case-based judgements about what one ought to believe about the world, given the information that is available. In the context of HTA, these relate primarily to the anticipated outcomes of a technology's adoption and the credence given to different sources of evidence on these outcomes.
Evidential judgements draw substantially on empirical data, which may require a high level of technical competence for their proper interpretation and application to decision-making. However, evidential judgements also incorporate normative commitments which can be taken to ground knowledge claims; these include widely accepted epistemic standards, such as the definition of statistical significance, and norms for the conduct and reporting of clinical trials. Other evidential judgements relate to how individuals and priority-setting committees interpret and attribute weight to the data that is presented to them. These might include judgements about how much credibility should be ascribed to evidence produced by different parties and about how uncertainty should affect the conclusions reached. As in the case of standards, the highly technical nature of such judgements may conceal the extent to which they are based on normative commitments as well as empirical data.
Some evidential judgements can be codified as standards. However, it will always be necessary to consider the features of the individual case in deciding how to apply these standards. For instance, a priority-setter might specify as a standard that evidence generated by RCTs should be considered more reliable than ‘real-world’ observational data, but the weight given to evidence produced by a particular RCT will depend on judgements made about the strengths and limitations of that particular trial. Individual cases will also sometimes generate novel case-based judgements. For instance, decision-makers might occasionally be faced with new types of technology or new ways of collecting evidence, to which existing standards are not applicable, or, in the absence of clearly defined standards, may find themselves making case-based judgements about how uncertainty should affect the conclusions reached. Over time, formalisation of such judgements as standards is likely to improve the consistency and transparency of decision-making.
The manufacturer's submission indicates that Gehrigole comes at an incremental cost of $45,000/QALY compared to standard of care. Priority-setter A has not specified in its formal standards its attitude to risk or how it responds to uncertainty in the available evidence. In evaluating this estimate of cost-effectiveness, it therefore relies on informal standards and several case-based evidential judgements. First, it notes that its evidence for decision-making is derived from a single trial conducted in South-East Asia. It judges that the results of this trial may not be generalisable to its own health system due to differences in standard of care and patient demographics. Second, it notes that though the trial was randomised and controlled – and that RCTs are usually considered to be a reliable source of evidence – it was of relatively short duration and the manufacturer's approach to extrapolating survival data is the most generous of several possible scenarios. It judges that, given the failure of the observed effect to reach statistical significance even using this approach, any apparent survival benefit should be considered highly uncertain. Third, it observes that the manufacturer has a significant commercial interest in securing adoption of its drug and is therefore incentivised to take a favourable view of the likely magnitude of clinical benefits and the robustness of the available evidence. It therefore judges – as it has in previous cases – that the manufacturer's model is highly likely to overestimate the drug's clinical effectiveness and that Gehrigole is likely to have been priced based on the maximum cost thought likely by the manufacturer to be deemed acceptable by the committee.
Taking these evidential judgements together, priority-setter A concludes that Gehrigole is unlikely to offer any significant survival benefit and that the drug's cost probably exceeds $50,000/QALY. However, it accepts that Gehrigole does likely offer some benefits in terms of quality of life – albeit less than the 0.15 QALYs suggested by the available evidence – and that its novel mechanism of action represents a significant scientific advance. It agrees with the manufacturer's assessment that the financial impact of Gehrigole's adoption on the health system would amount to around $15 million per year.
Operating alongside evidential judgements are case-based ethical judgements about how one ought to act. In the context of HTA these relate primarily to whether a particular technology should be adopted for use and on what grounds.
The ethical judgements made during HTA are informed by empirical information and the evidential judgements that arise from it. However, while empirical evidence is required to establish the likely effects of a technology's adoption and may additionally indicate how some people would trade these off against each other, such data do not in themselves determine what should be done. Rather, in deciding whether to adopt a technology, a decision-maker will be required to make a series of ethical judgements. As demonstrated below, in attempting to balance such judgements, priority-setters may occasionally find themselves reaching conclusions that are not morally coherent.
As in the case of evidential judgements, the consistency and transparency of decision-making is likely to be improved over time by codifying certain case-based ethical judgements as standards. However, attention will always need to be given to the specific features of the individual case in deciding how these standards should be applied and the need to make case-based ethical judgements will likely continue to arise, even in a highly formalised process.
After weighing the evidence concerning Gehrigole's likely effects, priority-setter A makes several case-based ethical judgements. Underpinning each of these is the evidential judgement that Gehrigole's cost exceeds $50,000/QALY and that its adoption would therefore likely lead to a net loss of QALYs from the population.
First, priority-setter A considers how to weigh its concern for efficiency against its principle of clinical need. It notes the severity of ALS – specifically, the short-life expectancy and poor quality of life experienced by patients at an advanced stage of the disease – and acknowledges that members of this group are in extremely poor health and therefore warrant prioritisation. Historically, it has traded efficiency off against clinical need in such cases by showing a willingness to exceed the usual cost-effectiveness threshold (an informal standard). It is aware that this approach is potentially incoherent in that it fails to give equivalent weight to the needs of those within the general population who are in a comparable state of poor health. Nevertheless, it makes an ethical judgement that to deviate from an established (if informal) standard would be to contravene the principle of formal equality, which requires it to treat identifiable cases that are alike in morally relevant respects similarly. It therefore decides to accept an incremental cost-effectiveness ratio somewhat greater than $50,000/QALY for Gehrigole.
Second, it considers patient population size. Following extensive discussion and reflection on its normative commitments, priority-setter A can find no reason why ALS's relative rarity compared with other conditions should justify deviation from its adopted principles. However, it notes that in previous cases it has been willing to accept a somewhat higher cost per QALY for rare conditions because of their relatively small impact on resources. Therefore, as in relation to severity, it makes an ethical judgement that its principle of equality requires it to maintain this established informal standard, despite its potential lack of moral justification. It therefore further uplifts its view of what constitutes an acceptable cost per QALY for Gehrigole.
Third, priority-setter A gives attention to Gehrigole's pharmacological innovativeness, recalling its previous evidential judgement that the drug represents a significant scientific advance. It situates the normative value of this innovativeness in Gehrigole's potential to contribute to an eventual cure for ALS, though it acknowledges that such an outcome is highly uncertain and that no further data can be gathered concerning its likelihood. It also notes its formal standard of excluding such indirect effects from consideration. However, given the significant unmet clinical need associated with ALS, it makes the ethical judgement that an exception should be made in this case and that these highly uncertain indirect benefits should be included in its assessment of Gehrigole. It therefore makes a further evidential judgement that whatever cost per QALY is derived from the available evidence, this is likely to undervalue Gehrigole's worth to society.
Finally, priority-setter A considers the age profile of the ALS patient population, noting that individuals in late middle-age often have children and older family members for whom they care (and who care for them). It is satisfied that this fact does not give rise to any direct health benefits that have not already been accounted for, but it considers whether there is a moral case for exceptionally including other indirect benefits of Gehrigole given the substantial impact of ALS on dependents' non-health-related quality of life. It makes the ethical judgement that to do so would be to prioritise treatment of these patients based on their age and social role and that, while it could be argued that these are morally relevant considerations, this would constitute illegal discrimination and would contravene the rule of law. It therefore does not take this factor into account in its decision-making.
4.7 Recommendation
A single case will give rise to multiple judgements that might be used to justify a range of actions. The role of the decision-maker is to draw on these in reaching a resolution or proposal either to adopt or to reject the technology in question: a recommendation.
Some case-based judgements are likely to be repeated across cases, reflecting quite straightforwardly the values, principles, and standards of the priority-setter, and giving rise to a relatively uniform set of recommendations. But the idiosyncrasies of individual cases, the interpersonal dynamics of particular groups, and the discretion granted to decision-makers in deciding how to apply and balance different normative commitments mean that recommendations can rarely be predetermined: two committees may reasonably use the same information to reach different conclusions on different days. It is therefore only by articulating the full chain of reasoning that complete transparency about the rationale for a recommendation can be achieved.
Priority-setter A concludes that though Gehrigole likely does not meet the usual criteria for adoption, in this case its recommendation is justified by the severity and rarity of ALS and the drug's innovative nature. Priority-setter A therefore recommends that the health system adopt Gehrigole for the treatment of ALS.
5. Discussion
The preceding paragraph, in which priority-setter A gives the key reasons for its recommendation, is reflective of the type of abridged summary typically provided in HTA reports.Footnote 14 However, this offers a far from complete account of the normative conclusions reached in relation to Gehrigole or the role played by evidence in its evaluation. It is also the case that a full rationale for the approach taken to HTA is often absent at the level of policy; while some HTA bodies offer an account of the ‘values’, ‘principles’, ‘morals’, or ‘ethics’ that guide their approach to decision-making, it is often unclear how these have been derived, how they relate to one another, or how they are reflected in the more specified standards and judgements adopted in individual cases.Footnote 15 In contrast, the more detailed narrative facilitated by the framework illustrates the full chain of reasoning that underlies priority-setter A's recommendation of Gehrigole, opening this decision up to scrutiny and debate.
In this section, we highlight four specific ways in which use of the framework can enhance the legitimacy and fairness of decisions guided by HTA, by increasing transparency across the chain of reasoning and by ensuring that decisions can be defended as morally justifiable.
5.1 By making explicit the normative considerations that influence decision-making
In the case of Gehrigole, the concluding paragraph explicitly links priority-setter A's recommendation to three normative considerations: disease severity, rarity, and innovation. However, the full narrative facilitated through use of the framework makes clear that several other factors are implicated in the chain of reasoning. The most obvious is cost-effectiveness, which though alluded to is not explicitly acknowledged in the recommendation, potentially obscuring (at least to the non-expert) its central role in decision-making and the fact that the committee believes that the drug will displace more QALYs from the health system than it will deliver.Footnote 16 Also unacknowledged is the consideration given to patient age and social role, which contribute to the reasoning underlying priority-setter A's response to Gehrigole, despite ultimately being excluded. This significantly abridged account of priority-setter A's reasons also prevents external stakeholders from identifying any factors that may have been unintentionally overlooked or excluded on unclear grounds. Normative considerations underpinning priority-setter A's evidential judgements are similarly absent from this brief summary, such as its view that the manufacturer's commercial interests are likely to have influenced its estimate of Gehrigole's cost-effectiveness.
While reference to some of these factors may be included within the more detailed report that tends to accompany priority-setting decisions, their dispersal across lengthy and often highly technical documents significantly curtails their accessibility (Charlton, Reference Charlton2021). Moreover, limitations in the language currently adopted to describe such considerations hinder the articulation of a chain of reasoning that can be comprehended by those both involved in, and external to, the decision-making process. The framework offers a means of much more clearly and explicitly setting out such reasoning.
5.2 By specifying how normative considerations are understood and applied
Divorced from the chain of reasoning from which they are derived, reference to the normative importance attributed to factors such as disease severity, rarity, and innovation can give rise to ambiguity about the precise role that such factors play in decision-making and whether their treatment can be morally justified.
Taking disease severity as an example, HTA bodies frequently cite this factor in justifying a technology's adoption (Golan et al., Reference Golan, Hansen, Kaplan and Tal2011; Angelis et al., Reference Angelis, Lange and Kanavos2018; Kaur et al., Reference Kaur, Prinia, Lakshmi, Downey, Sharma and Teerawattananon2019). But they typically do not stipulate how severity is understood and do not acknowledge the values and principles that motivate its consideration. In contrast, the structured narrative facilitated by the framework specifies priority-setter A's understanding of severity and explicitly justifies the ethical judgement that stems from it: the view that the clinical needs of ALS patients should be prioritised due to their ‘short life expectancy and poor quality of life’.
This type of articulation can also reveal the widely differing normative commitments on which identical recommendations can be based.
Let us consider another organisation, priority-setter B. While priority-setter A's approach centres on the value of justice, understood primarily in terms of health outcomes, priority-setter B's approach rests on the values of human dignity and social solidarity. Drawing on these values, priority-setter B considers that the most severe diseases are those that most seriously diminish human dignity and that, in such cases, solidarity requires that society does what it can to reverse this harm. On this understanding, it judges ALS to be a very severe disease not because of its direct impact on QALYs, but because of its symptomology and the extent to which this undermines dignity; for example, by limiting patients' ability to walk, talk, eat, and breathe without assistance. Priority-setter B is therefore willing to suffer very significant opportunity cost to gain access to the relatively limited clinical benefits that Gehrigole seems to offer.
While both priority-setters A and B choose to recommend Gehrigole's adoption on the grounds of severity, the framework reveals that these decisions are based on very different normative commitments. It thereby opens such commitments up to reasoned debate amongst stakeholders, ensuring that justifications are offered for decisions grounded in normative commitments, and providing an opportunity for judgements that cannot be morally justified to be scrutinised and corrected.
5.3 By clarifying the relationship between empirical data and normative judgements
HTA draws on a variety of normative commitments which both shape and are shaped by empirical evidence and the application of technical expertise to its interpretation. However, when the evaluative conclusions of HTA become divorced from the reasoning that supports them, the distinction between the empirical and the normative – and the contribution of each to decision-making – often becomes blurred. Use of the framework is intended to clarify how evidence has been interpreted and evaluated during HTA and the role that it has played in decision-making. It also provides a structure for more clearly communicating uncertainty in HTA outputs (the importance of which has been highlighted elsewhere (Trowman et al., Reference Trowman, Powers and Ollendorf2021)) and through which inconsistencies in a priority-setter's attitude to risk and uncertainty can be identified and rectified.
The framework defines an evidential judgement as a case-based judgement about what one ought to believe about the world, given the available information. It therefore makes explicit the normative content inherent to judgements that may seem, on the face of it, to be predominantly technical. For example, priority-setter A acknowledges that the extrapolation of survival data can reasonably be modelled in different ways and that its evidential judgement about Gehrigole's cost per QALY is based in part on its view that the manufacturer's choice of model may have been influenced by its commercial interests. While HTA reports commonly include detailed (and often highly technical) information about such matters as model choice and estimated cost per QALY, acknowledgement of the normative considerations that underpin these types of judgement is typically lacking.
Case-based judgements that appear predominantly normative can also contain important empirical content. Priority-setter A states in its conclusion that Gehrigole's recommendation is justified in part by its ‘innovative nature’, implying that it considers innovative pharmacology to be inherently valuable.Footnote 17 However, the more detailed narrative situates the value of Gehrigole's innovativeness in its potential contribution to an eventual cure. The question of how large the future health benefits of such a cure might be, and how likely they are to be realised, are extremely difficult to resolve, but are nevertheless empirical rather than normative in nature. Still, priority-setter A's decision to contravene its own formal standard by taking account of these indirect benefits is shown through this narrative to be a normative one, based not on a notion of the inherent value of innovation but on the ‘significant unmet clinical need associated with ALS’.Footnote 18
Perhaps most importantly, the framework makes explicit that evidential judgements cannot, by themselves, determine policy. That is, case-based judgements about what should be believed based on the available evidence (evidential judgements) cannot dictate how one ought to act as a result (ethical judgements). The truth of this statement can be easily overlooked where priority-setting is based on a single distributive principle, operationalised via a formal standard: for example, through strict application of a cost-effectiveness threshold. However, even under more nuanced approaches, the framework fulfils an important role in calling attention to normative judgements that might otherwise be misread (even by those making them) as technical.
5.4 By helping to identify and resolve moral incoherence
In ethics, coherence is understood to be the alignment of considered moral judgements about particular cases with the principles, rules, or theoretical considerations that are believed to ground them (Daniels, Reference Daniels2016). One way of achieving such alignment is via reflective equilibrium, a dynamic process in which abstract and particular normative commitments that do not align with one another are re-examined, deliberated, and mutually adjusted until any inconsistencies or contradictions are resolved (Rawls, Reference Rawls1999; Arras, Reference Arras and Steinbock2007). Under current norms, the conceptual and terminological issues already discussed can make it difficult for HTA bodies (and their stakeholders) to identify the commitments that have contributed both to a general approach to decision-making and to the judgements reached in specific cases. The framework can be used to make such commitments explicit, allowing the values, principles, standards, and case-based judgements adopted during assessment to be examined for alignment and facilitating reflective equilibrium.
In some cases, sources of incoherence may be easily identified, even without the additional clarity provided by the framework. For example, an HTA body that claims that its sole normative concern is efficiency, but which repeatedly recommends technologies that offer very poor value for money, is obviously in a state of incoherence. However, the framework improves accountability by making sources of incoherence explicit, requiring them to be either acknowledged or resolved. Priority-setter A echoes the behaviour of several real-world HTA bodies by assigning additional weight to the health needs of patients suffering from severe diseases. However, it fails to assign equivalent weight to other severely ill patients who suffer a proportion of the opportunity cost; a logical flaw that typically goes unacknowledged by HTA bodies but which priority-setter A is required to acknowledge through application of the framework.Footnote 19 The framework can also be used to highlight less obvious instances of incoherence. Take, for example, priority-setter A's treatment of rarity. Priority-setter A claims that its substantive approach is underpinned by its principles of efficiency and equality. These principles are reflected as standards in priority-setter A's use of a formal cost-effectiveness threshold, which ensures both that technologies are not recommended if they are likely to be inefficient and that all technologies are held to the same standard of efficiency. However, an informal standard has emerged from previous cases which allows this threshold to be exceeded if the estimated impact on resources is small. Priority-setter A acknowledges that patient population size (and, by extension, resource impact) does not appear to be a morally relevant consideration.Footnote 20 This informal standard therefore appears to contravene its principle of equality because it requires that cases that are alike in morally relevant respects (i.e. rare and common diseases) be treated differently. However, given that rare diseases have previously been prioritised under application of this standard, failure to apply it in the case of Gehrigole would also require similar cases to be treated differently. These principles and standards are incoherent.
Having identified this problem, priority-setter A has several options. It can reject the informal standard that requires it to prioritise technologies for rare diseases and maintain that rarity is not a morally relevant consideration. Alternatively, it can accept the standard of prioritising rare diseases, working to understand the values and principles that have led to its establishment. In pursuing this option, priority-setter A might find it necessary to revise its values and principles or to reconsider how these are weighed against one another. If, for example, it finds that it has previously prioritised drugs indicated for rare diseases on the grounds that manufacturers deserve to be fairly rewarded for their investment in research, then this might lead to the acknowledgement of a further principle: a principle of reasonable commercial returns. Or if it discovers that its prioritisation of rare diseases in the past has derived from a belief that severely ill people have a right to potentially curative treatment, even when the costs are extremely high, then consideration should be given to how this might be formally incorporated into priority-setter A's approach and what changes may need to be made to accommodate it coherently. A third option, of course, would be for priority-setter A simply to accept that its decisions and approach are incoherent and thus cannot be morally justified. However, to do so would substantially weaken both its normative and perceived legitimacy and would make it vulnerable to entitled challenge from those who suffer disadvantage as a result of its recommendations. Such a response would also, by its nature, be unethical.
6. Conclusion
Decisions about which health care interventions to provide, and to whom, have substantial implications for the health and wellbeing of society and are fundamentally grounded in normative considerations about which reasonable people will disagree. These decisions must, therefore, be transparently reported and morally justified if they are to be seen as legitimate. At present, limitations in the language used to articulate normative reasoning are likely to undermine both perceived legitimacy and fairness.
The proposed framework provides a tool for more clearly articulating the rationale on which priority-setting decisions are based and allows decision-makers to be more explicit about how the available evidence has been evaluated, and the role that it has played in guiding them towards their conclusions. As such, it constitutes an attempt to strengthen the legitimacy of HTA as a tool for healthcare priority-setting and is offered for further debate and development by those in the academic and policy communities.
While we are mindful of the fact that language is socially and culturally situated and that the complete standardisation of long-established terms is neither feasible nor necessarily desirable, we believe that the concepts and definitions contained here can act as a useful reference point for those wishing to anchor their own terminology, contributing to greater clarity across disciplinary boundaries. The framework is also intended to assist those who seek to evaluate decision-making in order to hold healthcare priority-setters to account.Footnote 21 Most importantly, we hope that the tool set out here will facilitate practical public reasoning by providing HTA practitioners and policymakers with the means to conceptualise, articulate, and evaluate the normative basis of their decision-making better and more easily, to the benefit of all those whose health these decisions impact.
Financial support
VC is funded by a Wellcome Trust Society and Ethics Doctoral Studentship (grant number 203351/Z/16/Z). AR's work is supported in part by the Clinical Center Department of Bioethics, which is in the Intramural Program of the US National Institutes of Health. LR's work is funded by Arnold Ventures. The views expressed here are those of the authors and do not necessarily reflect the views or policies of the organisations by which they are employed and/or funded.
Competing interests
AR is an employee of the Clinical Center Department of Bioethics, which is in the Intramural Program of the US National Institutes of Health. LR's research is supported by Arnold Ventures, a philanthropic organisation which invests in evidence-based solutions that maximise opportunity and minimise injustice. ZP-W is an employee of Leukaemia Care, a charitable body which receives funding from a wide range of sources, including the pharmaceutical industry. Details are available via the organisation's website and annual report. MS is an employee of the University of York, which receives funding to undertake technology assessment for the UK National Institute for Health and Care Excellence (NICE), which undertakes technology appraisal on behalf of the National Health Service in England and whose decision-making processes might be impacted by the issues raised in this manuscript. KS is an employee of NICE. All other authors have no competing interests.
Ethical standard
As this research was theoretical in nature, institutional ethics approval was not required.
Open access
