Introduction
Brain tumors have among the worst prognosis of all cancer types. In England, the 1-year survival rate is 40 percent, whereas the 5-year survival rate is as low as 15 percent (1). This, at least in part, may relate to late presentation and diagnosis. The symptoms experienced by patients with a brain tumor can be vague and nonspecific and, as such, have only a poor predictive value from a diagnostic perspective (Reference Hamilton and Kernick2;Reference Ozawa, Brennan, Zienius, Kurian, Hollingworth and Weller3). Headache, the most common symptom of brain tumors in adults, also occurs in 4.4 percent of all primary care consultations but has a positive predictive value of only 0.09 percent (Reference Hamilton and Kernick2). A study of symptom-based referral pathways for suspected brain tumor reported a positive predictive value (PPV) of 2.8 percent for severe red flag symptoms in terms of detecting a brain tumor on subsequent brain imaging (Reference Zienius, Grant and Brennan4). There is clinical need for new tests to support brain tumor diagnosis that reduce both diagnostic delay and unnecessary imaging.
The liquid biopsy proposed here is based on Fourier-transform infrared (FTIR) spectroscopy applied to serum from a standard blood sample. The spectral data are collected and analyzed using pattern recognition and machine learning algorithms to detect disease-specific signatures (Reference Butler, Brennan, Cameron, Finlayson, Hegarty and Jenkinson5). Liquid biopsy results predict the presence or absence of cancer and may also suggest cancer type.
The test was developed using data from 433 patient blood samples, including those with and without brain tumors. Using a fivefold cross-validation strategy to assess accuracy in the development data, a sensitivity of 92.8 percent and a specificity of 91.5 percent were reported (Reference Hands, Clemens, Stables, Ashton, Brodbelt and Davis6). This result was further validated in a larger retrospective cohort of 724 patients with a reported sensitivity of 93.2 percent and a specificity of 92.8 percent (Reference Butler, Brennan, Cameron, Finlayson, Hegarty and Jenkinson5).
Translation of entirely new diagnostic technologies to the clinic is highly challenging with complex and internationally varying regulatory and reimbursement decision making (Reference Rogowski, Hartz and John7). Early economic evaluation has been proposed as a method to guide the development of medical tests from the lab to the clinic (Reference Buisman, Rutten-Van Mölken, Postmus, Luime, Uyl-De Groot and Redekop8). By obtaining estimates of cost-effectiveness before the final-stage clinical research or commercialization, efficiency in translation can be improved. Clinical applications that are unlikely to be able to demonstrate cost-effectiveness can be halted, whereas more promising applications can be processed (Reference Hartz and John9).
In an early economic evaluation (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10), we mapped a clinical pathway integrating spectroscopic liquid biopsy as a triage test in both primary and secondary care. Patients would be tested with a liquid biopsy prior to referral for brain imaging studies; those testing positive would be prioritized to urgent imaging (within 1 week), whereas those testing negative would have standard referral (around 4 weeks) or follow-up without further investigation. It is expected that all patients seen in secondary care would ultimately receive imaging tests, whereas many in primary care could forgo imaging for the time being if they received a negative liquid biopsy result. Under current standard of care, all patients would be referred for brain imaging via either specialist or open-access services.
Cost-effectiveness analysis results suggested the potential for the technology to be cost-effective subject to confirmation of diagnostic accuracy and test cost. The next stage in translation was identified to be a prospective validation study of test accuracy with patient blood samples gathered prior to diagnosis and treatment. This study required symptomatic patients only, without “healthy” controls, as would be the case in the proposed clinical setting. A study of this type has now reached a preplanned interim analysis [Brennan et al., under review], necessitating an update of the economic evaluation to inform the next stage of development. A cohort study design was used, recruiting from high-risk settings in secondary care. At the preplanned interim analysis when recruitment had reached 400 patients, the prospective validation study reported a test sensitivity of 81 percent and a specificity of 80 percent. This iteration of the early economic evaluation updates the cost-effectiveness analysis with these estimates to help inform the next stage of translation of the technology.
We also examine a new extension of the clinical pathway. Recent research has established that there is scope to differentiate between brain tumor types as part of the same liquid biopsy based on FTIR serum spectroscopy (Reference Cameron, Butler, Smith, Hegarty, Jenkinson and Syed11). The ability to provide a prediction of the tumor type may have additional diagnostic value. This could be particularly beneficial in cases when brain imaging is inconclusive for the tumor type. For example, the aggressive grade IV glioma, glioblastoma (GBM), may have similar radiological appearance with primary central nervous system lymphoma (PCNSL) and brain metastases, but all have very different treatment pathways (Reference Campos, Davey, Hird, Pressnail, Bilbao and Aviv12;Reference Xiao, Yan, Wang, Osman and Zhao13). This issue was highlighted as an important problem by clinical experts in focus groups (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). Therefore, in this study, we sought to explore the impact of providing tumor-type classification on healthcare costs and patient management pathways through an extension of the cost-effectiveness model.
This study aims to inform the development of the test in two ways:
(1) Estimate the cost-effectiveness of the spectroscopic liquid biopsy in both primary and secondary care in the UK NHS based on prospective diagnostic performance.
(2) Estimate the costs–consequences of providing additional tumor-type predictions to clinicians at the point of confirmation of brain tumor diagnosis by imaging.
Methods
A cost-effectiveness analysis was conducted from a UK NHS perspective to evaluate the effect of the addition of a triage blood test for brain cancer to standard diagnostic practice. Life-years (LY), quality-adjusted life-years (QALY), and healthcare costs were estimated for both current standard practice and practice with the addition of the test. The time horizon of the analysis was 2 years. Unit costs used a price year of 2017/2018. Costs in GBP were converted to EUR at a rate of 1.1405 (14).
A cost–consequence analysis was conducted to explore the consequences of providing additional information on the probability of tumor types. This analysis applies only to patients with a tumor confirmed by imaging. Healthcare costs, the probability of surgery, and the probability of biopsy were compared between standard practice and practice when the predicted probability of different tumor types is available. Surgery is both the major resource item and a significant source of possible harm for patients (Reference Jackson, Westphal and Quiñones-Hinojosa15). Therefore, reducing unnecessary surgery is the principal outcome of interest in addition to total healthcare costs.
A decision-analytic model was constructed to represent the alternative clinical pathways for suspected brain tumor patients with or without the availability of a triage blood test (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). A decision tree structure was selected because this can feasibly represent the alternative diagnostic pathways and outcomes over a short time horizon. A 2-year time horizon was selected because of the short duration of survival in this patient group, with less than 20 percent expected to survive beyond 2 years (Reference Teo, Martin, Owusu-Agyemang, Nowicki, Clark and Mackinnon16). The model structure is shown in Figure 1. The figure shows the decision tree for the diagnostic pathway using a spectroscopic liquid biopsy. Without the spectroscopic liquid biopsy, all patients in both primary and secondary care receive a standard referral to imaging. A positive liquid biopsy test provides the opportunity for a fast-track referral and consequently earlier diagnosis. The clinical pathways of patients presenting in primary and secondary care were evaluated separately. A key difference is that in primary care, a proportion of patients who have tested negative may forgo imaging referral, whereas in secondary care, all will receive at least a standard imaging referral. This feature also means that there may be harms from false negative results in primary care, leading to a longer time to diagnosis. The prevalence of brain tumors of patients presenting in secondary care (3 percent) is higher than in primary care (1 percent) (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). Prevalence in each setting was determined in the prior evaluation based on evidence from observational data and clinical expert opinion on the population that would be selected for testing in practice (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). LY and QALY gains are driven by a reduction in delays to diagnosis for those patients with a brain tumor, leading to increased expected survival time, which is explained in more detail in (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). The selected prevalence and the test accuracy in the base case imply a false negative rate per 10,000 patients tested of 57 in secondary care and 19 in primary care, with a corresponding false positive rate of 1,940 in secondary care and 1,980 in primary care.
Figure 1. Decision tree model and extension. Outcomes 1: LY 0.80, QALY 0.71. Outcomes 2: LY 2, QALY 2. Outcomes 3: LY 0.58, QALY 0.52. GBM, Glioblastoma; PCNSL, Primary Central Nervous System Lymphoma.
An extension of this decision tree was constructed for tumor-type information (Figure 1). Tumor-type classification probabilities allow varying diagnostic protocols for patients who test positive and have a tumor confirmed by imaging study (i.e., true positive for a brain tumor for the serum spectroscopy test). Three possible management routes were included in the model. Those who are predicted to have high grade glioma/GBM go directly to surgery. A prediction of primary central nervous system lymphoma (PCNSL) leads to a biopsy and medical therapy. Predicted metastatic cancer of unknown origin leads to further imaging investigation (with avoidance of brain surgery or biopsy in most cases). Under standard care, all these patients would be presumed to have a GBM and directed to surgical resection, with alternative diagnosis happening during or after the surgery.
Unit cost parameters are listed in Supplementary Table 1, and decision tree parameters are listed in Supplementary Table 2. Outcomes (healthcare costs and remaining LY and QALYs) for each leaf of the decision tree were specified as in (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). Survival, quality of life, and cost parameters for the cost–consequence analysis are displayed in Supplementary Table 3. Prices for imaging studies and surgery were sourced from NHS reference costs (2017–18).
Cost-effectiveness results for the base case analysis were summarized by the incremental cost-effectiveness ratio (ICER) at four selected spectroscopic liquid biopsy price points. Key parameters of prevalence and the effect of delay on survival were varied in one-way sensitivity analysis (OWSA), and these are presented on a cost-effectiveness plane. The OWSA of test sensitivity and specificity is available in (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). A probabilistic sensitivity analysis (PSA) was conducted to understand uncertainty across all key model parameters arising from sampling uncertainty. The PSA results are reported on a cost-effectiveness plane and as a cost-effectiveness acceptability curve (CEAC).
Results
In the base case of the cost-effectiveness analysis (Table 1), updated with prospective validation results, spectroscopic liquid biopsy testing would lead to a gain of 15.4 QALYs per 10,000 patients in primary care use and 65.4 QALYs per 10,000 patients in secondary care use. In primary care, at test costs below GBP 100 (EUR 114), testing was cost saving. At GBP 100 (EUR 114) per test, the ICER was GBP 13,279 (EUR 15,145), whereas at GBP 200 (EUR 228), the ICER was GBP 78,300 (EUR 89,301). In secondary care, the ICER ranged from GBP 11,360 (EUR 12,956) to GBP 43,870 (EUR 50,034) across the range of test costs. The results of the previously published, first iteration of the early evaluation using retrospective data are available in Supplementary Table 4 so that the effect of the attenuation of diagnostic performance on cost-effectiveness can be ascertained.
Table 1. Cost-effectiveness analysis base case results
a More effective and less expensive.
OWSA on prevalence (Supplementary Figure 1) and hazard ratio (HR) for delay to diagnosis (Supplementary Figure 2) show that these parameters are highly influential. Both are strong determinants of the QALY difference between diagnostic strategies and, therefore, influence cost-effectiveness. For example, at a test cost of GBP 100 (EUR 144), if prevalence were 0.5 percent instead of 1 percent in primary care, then the ICER would be >GBP 20,000 (EUR 22,810) and the technology may not be considered cost-effective.
The CEAC plot (Figure 2) demonstrates that at a test cost of GBP 100 (EUR 114), there is approximately a 75% probability of being cost-effective in primary care and a 50% probability of being cost-effective in secondary care at an ICER threshold of GBP 20,000 (EUR 22,810). At the lower end of the range of test costs (GBP 50, EUR 57), testing is highly likely to be cost-effective at any ICER threshold in primary care and at thresholds above GBP 15,000 (EUR 17,108) in secondary care. Supplementary Figure 3 shows the PSA results on the cost-effectiveness plane. This highlights the importance of uncertainty about effectiveness in both scenarios and uncertainty about costs for the primary care scenario only.
Figure 2. Cost-effectiveness acceptability curve, and primary care and secondary care scenarios at GBP 50 (EUR 57) and GBP 100 (EUR 114) test cost.
The cost–consequence analysis results of providing tumor-type information are reported in Table 2. Tumor-type predictions have the potential to reduce surgeries by approximately 8 per 100 brain tumor cases. The additional healthcare cost savings are approximately GBP 58,000 (EUR 66,149) per 100 cases. These estimates are sensitive to the relative frequency of GBM, PCNSL, and metastatic disease in the patient population (Supplementary Figure 4). Significantly larger cost savings are possible when other tumor types are relatively more prevalent among cases compared with GBM. The total budget impact for the NHS in England and Wales if this was rolled out nationally would be approximately GBP 1.3–2 million (EUR 1.5–2.3 million) assuming that 50–80% of the 4,500 malignant brain tumors diagnosed every year (17) had received testing.
Table 2. Costs–consequences per 100 cases with and without type information
Discussion
A prospective study of diagnostic accuracy has brought a spectroscopic liquid biopsy for brain cancer one step closer to being ready for clinical use. The updated cost-effectiveness results reported here demonstrate potential for the technology to be cost-effective in both primary and secondary care settings. Compared with the base case result in the previous iteration of the evaluation, effectiveness is reduced due to the attenuation of diagnostic accuracy in the prospective data. This iteration of the early economic evaluation also extended the clinical pathway to include a new feature of the technology—classification of tumor type. The cost–consequence analysis demonstrated some additional utility and cost savings from providing tumor-type classification probabilities.
New evidence in relation to diagnostic accuracy may emerge as more data are collected in ongoing studies. This can then be used to update the economic evaluation in a further iteration. Machine learning-based classification systems can require large numbers of observations to achieve optimal performance; therefore, it is possible that improvements in diagnostic accuracy may occur after more training data are collected. Prospective data from primary care would also be useful to determine if diagnostic accuracy is the same in this setting and alleviate concerns about possible spectrum bias.
This economic evaluation provides support for the continued development of a serum spectroscopy test for brain tumors. Remaining issues relate to the feasibility of integrating the test into routine practice. An important aspect of this is understanding how patient selection by clinicians would influence disease prevalence in the tested population. This was highlighted as an important parameter in the sensitivity analysis. Another issue, identified in the previous iteration of early evaluation (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10), is to what extent brain imaging studies would actually be reduced in primary care; that is, what proportion of test-negative patients would not be subsequently referred to imaging.
The major limitations of this analysis are related to the lack of direct evidence of how alternative diagnostic pathways will influence patient survival and quality of life. In the absence of this data, we have relied on extrapolation from observational data. Although these data are available from cohort studies (Reference Aggarwal, Herz, Campbell, Arkush, Short and Rees18;Reference Do, Gebski and Barton19), treating the association between time to treatment and survival as causal requires strong assumptions.
An open question is whether or not a randomized trial of alternative diagnostic strategies (test vs. no test) is required to demonstrate patient benefit for this technology. Whether a randomized trial(s) is desirable or not can be considered within existing frameworks (Reference Lord, Irwig and Bossuyt20). This requires the judgment of all relevant stakeholders regarding the feasibility of a trial as well as the validity of assuming survival and quality of life benefits of early diagnosis. Although a randomized trial offers the best possible evidence on which to base an adoption decision, the scale of the trial that would be required may prove prohibitively expensive to undertake, and it is well recognized that randomized trials for diagnostic tests are often unfeasible (Reference Schaafsma, Van Der Graaf, Rinkel and Buskens21). The vast majority of tests in clinical practice today do not have evidence of patient benefit from a randomized trial. A feasible solution in this case may be a postmarketing study, forgoing randomization in favor of a cohort study design. It has been suggested that an observational study assessing time to diagnosis and treatment may be optimal for a triage test when a randomized design is infeasible (Reference Lord, Irwig and Bossuyt20). A study of the hypothetical clinical utility of test results using a questionnaire survey among primary care clinicians may be a useful preceding step. These types of studies could address the two key remaining areas of uncertainty: prevalence in the referred population and clinical decision making in response to negative test results. Clear benchmarks for clinical utility could be determined and a risk-sharing model for reimbursement (Reference Carlson, Sullivan, Garrison, Neumann and Veenstra22) may be useful to explore. Determining the optimal design of future studies requires a full exposition and consultation with stakeholders, beyond the scope of this early economic evaluation.
Translation of the economic model to other countries or health systems will also require refining the above parameters and may need further development of the economic model. In the preceding evaluation, the USA healthcare system was considered using Medicare unit prices (Reference Gray, Butler, Board, Brennan, Chalmers and Dawson10). We now believe that differences between UK and USA health systems may necessitate a different model structure. Validation or adaption of the diagnostic and clinical pathway represented in the model to health systems in the United States is part of ongoing research.
Early economic evaluation has been useful in guiding the development of this liquid biopsy test for brain tumors. Making an early assessment of both costs and consequences of other novel tests in the domain of liquid biopsies for cancer may improve the focus of clinical research efforts in this area. Ultimately, comparative cost-effectiveness analysis with both existing diagnostic pathways and all potential new alternatives would be most useful for clinicians, patients, and health system decision makers.
A logical next stage of development for spectroscopic liquid biopsy for brain tumors is a large-scale, pragmatic program of test use in routine primary care practice. Additional studies in this setting are needed to resolve the remaining issues of uncertainty—prevalence in this patient population and referral behavior—and would demonstrate feasibility in real-world clinical practice.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000143.
Acknowledgments
The authors would like to thank clinical experts who took part in the previous iteration economic evaluation. Additional thanks to the patients who took part in the clinical validation study and without whom progressing to the present evaluation would not have been possible.
Funding
This research was supported by a grant from Innovate UK Precision Medicine Accelerator Grant (Project No. 23626).
Conflict of Interest
MJB (Chief Technical Officer), HJB (Head of Development), MGH (Chief Executive Officer), PMB (Clinical Advisory Board Lead), DSP (Chief Data Officer), and EG (Health Economics Consultant) are all employees/consultants of ClinSpec Diagnostics Ltd. JMC and MDJ declare no conflicts of interest. PMD reports grants from Innovate UK, grants from Cancer research UK, and other assistance from ClinSpec Dx during the conduct of the study. MJB, DSP, and HJB have patents issued for technology related to liquid biopsy for brain tumors. MJB reports personal fees from ClinSpec Diagnostics Ltd., both during the conduct of the study and outside the submitted work. In addition, MJB has a patent Discerning Brain Cancer Type 2019—PE959914GB pending to ClinSpec Diagnostics Ltd., a patent Infra-Red Spectroscopy System March 2017—GB2018/050821 pending to ClinSpec Diagnostics Ltd., a patent Method of Diagnosing Proliterative Disorders—WO 2014/076480 issued to ClinSpec Diagnostics Ltd., a patent Analysis of bodily fluids using infrared spectroscopy—GB2018/050821 pending to ClinSpec Diagnostics Ltd., and a patent Sample Container—PCT/GB2019/052898 pending to ClinSpec Diagnostics Ltd. HJB reports grants and other assistance from ClinSpec Diagnostics during the conduct of the study; grants and other assistance from ClinSpec Diagnostics, from null, outside the submitted work. In addition, HJB has a patent Methods of diagnosing proliferative disorders issued. MGH reports personal fees from ClinSpec Diagnostics Ltd., both during the conduct of the study and outside the submitted work. In addition, MGH has a patent Infra-Red Spectroscopy System March 2017—GB2018/050821 licensed to ClinSpec Diagnostics Ltd. and a patent Sample Container—PCT/GB2019/052898 licensed to ClinSpec Diagnostics Ltd. DSP reports grants and other assistance from ClinSpec Diagnostics Ltd., both during the conduct of the study and outside the submitted work. In addition, DSP Palmer has a patent issued relating to brain tumor diagnosis issued.
Open access
