Health technology assessments (HTA) provide a connection between the research and decision-making worlds and over the last 20 years have become central to informing healthcare policy decision making in many settings worldwide (Reference Sorenson, Drummond and Kanavos1). The HTA process involves a systematic evaluation of the direct (intended) and indirect (unintended) effects and impacts of healthcare technologies. Organizations such as the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, the Heath Care Insurance Board (CVZ) in the Netherlands, and the National Institute for Health and Care Excellence (NICE) in the United Kingdom now publish guidelines to ensure consistency and transparency in the HTA decision-making and reimbursement process (2–4).
Once the therapeutic value of novel interventions is established, evidence of value for money and efficiency is required before allocation of scarce healthcare resources in many settings. To ensure consistency and increase the quality of evidence submitted, NICE provides detail of a “reference case” for economic evidence used to support submissions (4). While some of NICE's requirements (such as discount rates and perspectives on costs) differ from those stipulated for non-UK settings, many are essential requirements for any economic evaluation in healthcare. These are clearly defined, relevant comparators; systematic literature reviews for health effects; synthesis of clinical evidence where applicable; suitable time horizons; health related quality of life (HRQoL) data reported by patients; and exploration of uncertainty (Reference Johannesson, Jonsson and Jonsson5–Reference Lopez-Bastida, Oliva and Antonanzas7). An often recommended criterion, which is not standard across the board, is that results are reported in terms of incremental cost per quality-adjusted life-year (QALY) whereby the HRQoL weights for the QALY are valued by the general population (4;Reference Gold, Siegel and Russell8).
The last decade has seen a rapid growth in the volume of literature describing economic evaluations of interventions in healthcare. In particular the number of articles describing cost-effectiveness evaluations in cancer has increased substantially as new pharmaceuticals are brought to market. Surgery, radiotherapy, and chemotherapy are the main treatments for cancer (9), and considerable proportions of healthcare budgets are consumed by these interventions. In 2010–2011, cancer was the third largest category of the overall UK healthcare budget, accounting for 5.4 percent of the overall cost at £5.8 bn (EUR 7.24 bn) (Reference Harker10), while total economic costs attributed to colorectal, breast, and prostate cancers were estimated to be £1.6 bn (EUR 2.0 bn), £1.5 bn (EUR 1.87 bn) and £0.6 bn (EUR 0.75 bn), respectively (Reference Leal11). Although the clinical and cost-effectiveness of novel pharmaceutical interventions for cancer is well documented due to the submission processes for reimbursement in many settings, it is unclear whether existing and novel surgical and radiotherapy interventions are treated with the same rigorous scrutiny.
The objective of the study was to examine the methodological and empirical cost-effectiveness evidence specifically in relation to surgical interventions in breast, colorectal and prostate cancer. Relevant literature was identified through systematic searches and methodological standards were assessed using quality criteria, both essential and preferred, and informed by the NICE reference case.
METHODS
Search Strategy
Systematic searches were undertaken (April 2012) in the following databases: Medline, EMBASE, CDSR, NHSEED, HTA, DARE, and EconLit. The CEA registry, the NICE website, recent conference proceedings, and reference lists of any included studies and existing reviews were also searched. No language, publication, or date restrictions were applied to the searches. The full search strategy is available online (Reference Ara, Barbieri and Weatherly12).
Study Selection Strategy
Studies were selected for inclusion in two stages. An initial sieve was conducted on title and abstract and full articles were retrieved for any potentially relevant studies. Studies were included in the reviews if they assessed the cost-effectiveness of a surgical technique in individuals with either breast, colorectal, or prostate cancer, and presented results in the form of an incremental cost per QALY or an incremental cost per life-year [LY]). Data were extracted using a customized template designed to capture relevant information (Table 1–2). Studies were quality appraised (Figure 2) in terms of meeting essential (items 1–7: patient group and indication clearly defined, comparators clearly defined, effectiveness evidence based on a systematic review, appropriate time horizon, HRQoL data reported directly by patients or carers, probabilistic sensitivity analysis used to quantify uncertainty), preferred (items 8–9: cost-effectiveness analysis using QALYs, HRQoL preference data valuation by representative sample of the public), and UK NICE specific (items 10–14: 3.5 percent discount rate, comparators used in NHS, UK setting, UK NHS and personal social services costs, cost per QALY below the NICE threshold value, assumed to be £30 k per QALY for evaluations in cancer (i.e., society is willing to pay £30k [EUR 37.4 k] for one additional QALY) requirements for economic evaluations in healthcare (4). Many of the criteria that were applied are relevant to economic models irrespective of setting and the preferred items overlap with those in published quality checklists (Reference Gold, Siegel and Russell8;Reference Drummond, Sculpher and Torrance13). The evidence was reviewed by a single researcher and any issues were discussed within the review team until a consensus was achieved in cases of uncertainty or lack of clarity.
Table 1. Characteristics of the Studies Included
BCS, breast conserving surgery; BSCRT, breast conserving surgery with radiation; CEA, cost-effectiveness analysis; LRARP, laparoscopic remotely assisted radical prostatectomy; LS, laparoscopic surgery; LY, life-year; MRM, modified radical mastectomy; MR, magnetic resonance; OS, open surgery; QALY, quality adjusted life year; RP, radical prostatectomy; RALP, Robot-assisted laparoscopic prostatectomy; RRP, retropubic radical prostatectomy; WW, watchful waiting.
Table 2. Methods and Study Conclusions
BCS, breast conserving surgery; BSCRT, breast conserving surgery with radiation; CEA, cost-effectiveness analysis; LRARP, laparoscopic remotely assisted radical prostatectomy; LS, laparoscopic surgery; LY, life year; MRM, modified radical mastectomy; MR, magnetic resonance; OS, open surgery; QALY, quality-adjusted life-year; RP, radical prostatectomy; RALP, Robot-assisted laparoscopic prostatectomy; RRP, retropubic radical prostatectomy; WW, watchful waiting.
RESULTS
The searches identified 1,133, 2,408, and 722 unique references for the breast, colorectal and prostate cancer reviews, respectively (Figure 1). Seventy-eight full papers were retrieved for detailed inspection. Of these, sixteen studies met the inclusion criteria (Table 1). One additional study published after completion of the searches was identified and has been included in this article.
Figure 1. Flow diagram for study inclusion.
Three studies were in breast cancer (publication dates 1997 to 2003) (Reference Malin, Keeler and Wang14–Reference Polsky, Mandelblatt and Weeks16), seven were in colorectal cancer (publication dates 2000 to 2011) (Reference Callejo, Guerra and Reza17–Reference Murray, Lourenco and de Verteuil23) and seven were in prostate cancer (publication dates 2000 to 2012) (Reference Calvert, Morgan and Catto24–Reference Ramsay, Pickard and Robertson30). Two of the three breast cancer studies were set in the United States (Reference Malin, Keeler and Wang14;Reference Polsky, Mandelblatt and Weeks16), and one was in Norway (Reference Norum, Olsen and Wist15). Two of the seven colorectal studies were set in the United Kingdom (Reference de Verteuil, Hernandez and Vale18;Reference Murray, Lourenco and de Verteuil23), two in the United States (Reference Jensen and Abcarian21;Reference Miller, Cantor and Peoples22), one in Canada (Reference Govindarajan, Naimark and Coburn19), one in New Zealand (Reference Hayes and Hansen20), and one in Spain (Reference Callejo, Guerra and Reza17). Three of the seven prostate studies were set in the United Kingdom (Reference Calvert, Morgan and Catto24;28;Reference Ramsay, Pickard and Robertson30), with the remaining four in Australia (Reference O’Malley, Jordan and O’Malley29), Denmark (Reference Hohwu, Borre and Ehlers25), Sweden (Reference Lyth, Andersson and Andren27), or the United States (Reference Jager, Severens and Thornbury26). The studies scan a broad publication range with six published before 2004 (older publications) and five published after 2010 (recent publications). Of particular interest, there were no publications for breast cancer after 2003.
Essential Basic Requirements
Defined Research Question
A basic requirement of any economic evaluation in health care is a clear and well defined decision problem which states both the exact clinical definition of the indication of interest, and detailed information on the interventions being compared (4). This enables reviewers and readers to assess if the analyses and results are relevant to their own settings and practice. The patient populations were reasonably well defined in the majority of studies (Figure 2), but there were exceptions. In the colorectal and prostate studies, the exact patient population or stage of cancer was either not clear or was not provided in some articles (Reference Callejo, Guerra and Reza17;Reference Hayes and Hansen20;Reference Jensen and Abcarian21;Reference Jager, Severens and Thornbury26).
Figure 2. Quality of included studies compared to the NICE reference case.
The interventions and comparators were generally clearly defined in all the studies. The three breast cancer studies compared breast conserving surgery with mastectomy with one also comparing reconstruction after surgery (Reference Malin, Keeler and Wang14). Six of the colorectal studies compared laparoscopic surgery with open resection (Reference Callejo, Guerra and Reza17;Reference de Verteuil, Hernandez and Vale18,Reference Hayes and Hansen20;Reference Jensen and Abcarian21;Reference Murray, Lourenco and de Verteuil23), one compared surgical resection with diagnostic or palliative surgery (Reference Miller, Cantor and Peoples22), and one compared emergency colonic stenting as a bridge to definitive surgery with emergency surgery (Reference Govindarajan, Naimark and Coburn19). Two of the prostate cancer studies compared radical prostatectomy with watchful waiting (Reference Lyth, Andersson and Andren27;28), one compared radical prostatectomy for those with non-diploid test with either radical prostatectomy for all or, watchful waiting (Reference Calvert, Morgan and Catto24), one compared radical prostatectomy based on magnetic resonance (MR) imaging with radical prostatectomy based on clinical staging (Reference Jager, Severens and Thornbury26), one compared laparoscopic remotely assisted radical prostatectomy with open radical prostatectomy (Reference O’Malley, Jordan and O’Malley29), one compared laparoscopic prostatectomy and robotic prostatectomy with open surgery (Reference Ramsay, Pickard and Robertson30), and the final study compared robot-assisted laparoscopic prostatectomy with retropubic radical prostatectomy (Reference Hohwu, Borre and Ehlers25).
Clinical Evidence
Another essential requirement relates to the clinical data used to quantify potential benefits of the interventions under evaluation. Evidence of a full literature search and, where appropriate, a synthesis of evidence on health effects informed by the systematic review, are mandatory to ensure the data used represent all the evidence available at the time of the study and the associated uncertainty surrounding the point estimates (2–4;6). The effectiveness rates used in the majority of the studies included in this study were not informed by systematic literature reviews or a synthesis of all available data. In breast cancer, none of the studies used effectiveness evidence informed by a systematic literature review or a synthesis of data. In colorectal cancer, five of the seven studies used the results of meta-analyses (Reference Callejo, Guerra and Reza17–Reference Hayes and Hansen20;Reference Murray, Lourenco and de Verteuil23). One study used event rates observed in a single hospital as per their research question (Reference Miller, Cantor and Peoples22), and one provided insufficient detail to determine the source of the effectiveness evidence used (Reference Jensen and Abcarian21). In prostate cancer, one study used the results of a meta-analysis (Reference Ramsay, Pickard and Robertson30), two used data from single randomized control trials (RCTs) with no evidence to suggest alternative evidence was used in sensitivity analyses (Reference Lyth, Andersson and Andren27;28), two used data from retrospective cohort studies (Reference Hohwu, Borre and Ehlers25;Reference O’Malley, Jordan and O’Malley29), one study was informed by a mixture of evidence from RCTs and observational data obtained from a systematic search supplemented by expert opinions (Reference Jager, Severens and Thornbury26), and one provided insufficient detail to determine the source (Reference Calvert, Morgan and Catto24).
Time Horizon
Many of the interventions will have a differential effect on recurrence rates and/or mortality risk hence the most appropriate time horizon for the costs and benefits would be a lifetime (4). However, for individuals with cancer, a lifetime horizon is potentially considerably shorter than that of the general population due to the reduced life expectancy associated with the condition. Over half (10/17) of the studies took account of longer-term costs and benefits associated with the interventions using a minimum of 10 year horizons (Reference Malin, Keeler and Wang14;Reference de Verteuil, Hernandez and Vale18–Reference Govindarajan, Naimark and Coburn19;Reference Murray, Lourenco and de Verteuil23;Reference Calvert, Morgan and Catto24;Reference Jager, Severens and Thornbury26–28). The remaining studies arguably underestimated both the costs and benefits of the interventions. One study used the average weighted survival duration (42 months for resected patients) (Reference Miller, Cantor and Peoples22). One restricted costs and QALYs to the first post-operative year (Reference Hohwu, Borre and Ehlers25), while one used 5-year retrospective data without extrapolating either costs or benefits (Reference Polsky, Mandelblatt and Weeks16). Two studies, which limited the perspectives to that of single hospitals, presented incremental benefits over short horizons without extrapolating to account for downstream benefits or costs. One restricted the time horizon to length of stay, estimating the incremental difference in recovery time (Reference Hayes and Hansen20), while the second study estimated the reduction in months (range, 3.79 to 8.67 months) of incontinence or erectile dysfunction (Reference O’Malley, Jordan and O’Malley29).
HRQoL
HRQoL data should be collected directly from patients with the conditions described in the economic model and preferably from patients in receipt of the interventions under evaluation (4;Reference Lopez-Bastida, Oliva and Antonanzas7). When it is not possible for patients with the conditions to provide these values, then proxy values may be obtained from their principle carer. However, values obtained from healthcare professionals are not appropriate and have been shown to differ substantially from those obtained from patients with the condition being valued (Reference Ashby, O’Hanlon and Buxton31).
The description of sources of HRQoL data was poor in many of the studies. While over a third of the studies (7/17) incorporated HRQoL data which were collected directly from individuals with the condition of interest, three studies used HRQoL data obtained from clinicians (Reference Malin, Keeler and Wang14;Reference Norum, Olsen and Wist15,Reference Miller, Cantor and Peoples22;Reference Murray, Lourenco and de Verteuil23), or nurses (Reference Norum, Olsen and Wist15;Reference Murray, Lourenco and de Verteuil23), and one study assumed HRQoL values based on the New Zealand social tariff (Reference Hayes and Hansen20). There was insufficient evidence to determine the source of the HRQoL data in six of the studies (Reference Norum, Olsen and Wist15;Reference Callejo, Guerra and Reza17;Reference Jensen and Abcarian21;Reference Calvert, Morgan and Catto24;Reference Jager, Severens and Thornbury26;Reference O’Malley, Jordan and O’Malley29).
Uncertainty
All economic evaluations should accurately characterize the decision uncertainty associated with the interventions under evaluation. At the very least, univariate sensitivity analyses should be conducted to illustrate both structural uncertainty (for example assumptions relating to the description of the clinical pathway, or the extrapolation of data) and uncertainty around key parameter values (for example if different sources for HRQoL or cost data are available) (4). Ideally, a full probabilistic sensitivity analysis using Monte Carlo simulations would be performed to demonstrate the uncertainty around the mean point estimates used (Reference Gold, Siegel and Russell8), and is encouraged by several decision-making bodies (3;4;6).
The majority of studies (13/17) performed univariate sensitivity analyses to illustrate the effect on results associated with variations in key parameter inputs. Many of the authors reported cost-effectiveness results were sensitive to changes in parameter values and particularly for recurrence rates, hernia rates, HRQoL values, and side-effects of treatments. The full range of uncertainty was poorly captured with less than one third (5/17) of the studies presenting the results of a full probabilistic sensitivity analysis (Reference Callejo, Guerra and Reza17;Reference Govindarajan, Naimark and Coburn19;Reference Murray, Lourenco and de Verteuil23;Reference Lyth, Andersson and Andren27;Reference Ramsay, Pickard and Robertson30). It is perhaps surprising that some of the newer publications did not report these results as this is now considered to be standard practice in health economics.
Preferred Requirements
Although the preferred outcome from economic evaluations varies depending on the reimbursement body, the cost per QALY, where the valuation of the HRQoL is based on public preferences, is now a pivotal requirement for submissions to NICE (4). The QALY describes both survival and HRQoL weights in a single metric and is the recommended gold standard to facilitate comparison across disparate conditions and equity in policy decisions (Reference Gold, Siegel and Russell8). Although all studies reported results in terms of a cost per QALY (or cost per quality adjusted month) (Reference Govindarajan, Naimark and Coburn19), few (4/17) used general population weights for all utilities (Reference de Verteuil, Hernandez and Vale18;Reference Hayes and Hansen20;Reference Calvert, Morgan and Catto24;Reference Hohwu, Borre and Ehlers25). One study mapped from HRQoL values onto a scale assuming 0 and 100 were equal to death and full health, respectively (Reference Norum, Olsen and Wist15), while two studies used patient values from the patients’ completed Euroqol VAS, again assuming 0 and 100 were equal to death and full health, respectively (Reference Polsky, Mandelblatt and Weeks16;Reference Lyth, Andersson and Andren27). Three studies elicited weights from clinicians using standard gamble (Reference Malin, Keeler and Wang14;Reference Miller, Cantor and Peoples22;Reference Murray, Lourenco and de Verteuil23), while a fourth used time-trade off to derive values from patients. Six studies provided insufficient detail to determine if general population weights were used (Reference Malin, Keeler and Wang14;Reference Norum, Olsen and Wist15;Reference Callejo, Guerra and Reza17;Reference Jensen and Abcarian21;Reference Jager, Severens and Thornbury26;Reference O’Malley, Jordan and O’Malley29).
UK Specific Requirements
Discount rates are both jurisdiction and time dependent and are used to ensure results reflect the present value of the cost and benefits accrued over the duration of the analyses (4). Not surprisingly, due to the differences in settings and the publication dates, only 3/17 of the studies used the current UK recommended discount rates (3.5 percent per annum) for costs and benefits (Reference Lyth, Andersson and Andren27;28;Reference Ramsay, Pickard and Robertson30). However, all studies that used horizons over one year did discount costs and effects.
While some reimbursement bodies restrict the perspective to the direct healthcare resource costs and savings directly attributed to a health service, others take a broader societal view and require indirect costs such as productivity losses or caregiver's time (Reference Johannesson, Jonsson and Jonsson5). Two-thirds (Reference Drummond, Sculpher and Torrance13/17) of the studies limited costs to direct healthcare costs (Reference Malin, Keeler and Wang14–Reference Polsky, Mandelblatt and Weeks16;Reference de Verteuil, Hernandez and Vale18–Reference Hayes and Hansen20;Reference Miller, Cantor and Peoples22–Reference Calvert, Morgan and Catto24;Reference Jager, Severens and Thornbury26–Reference Ramsay, Pickard and Robertson30), and two of these restricted costs to those directly incurred by hospitals (Reference Hayes and Hansen20–Reference Miller, Cantor and Peoples22). The remainder took a broader perspective and costs such as productivity losses (Reference Jensen and Abcarian21;Reference Hohwu, Borre and Ehlers25), and caregiver's time (Reference Jensen and Abcarian21), were included.
Applying the NICE threshold value of GBP 20–30 k (EUR 25–37.4 k) per QALY, irrespective of the time of publication, the setting or adherence to the NICE methods guide, breast conserving surgery plus axillary mode dissection compared with modified radical mastectomy would be considered cost-effective while breast conservation surgery with radiation therapy compared with mastectomy would not be considered cost-effective (Reference Malin, Keeler and Wang14). There are multiple reasons for this including cosmesis, patient preferences, and the added cost of reconstruction which is used after mastectomy in over 20 percent of all cases and recurrence rates requiring further interventions (32;Reference Hazebroek33). It should be noted that the latter comparator is no longer appropriate in the current clinical context as it is now current practice, with rare exceptions, for radiation to be administered after conservation therapy.
Applying the GBP 20–30 k (EUR 25–37.4 k) per QALY threshold to the studies on colorectal cancer, the probability that laparoscopic surgery would be considered cost-effective compared with open surgery ranged between 30 percent (Reference de Verteuil, Hernandez and Vale18) and 40 percent (Reference Murray, Lourenco and de Verteuil23) and univariate analyses produced results where laparoscopic surgery either dominated (higher costs and lower benefits) open surgery or was dominated (lower costs and larger benefits) by open surgery suggesting considerable uncertainty (Reference de Verteuil, Hernandez and Vale18;Reference Hayes and Hansen20;Reference Murray, Lourenco and de Verteuil23). The evaluation of emergency colonic stenting as a bridge to definitive surgery dominated emergency surgery in individuals with emergent, malignant left colonic obstruction (Reference Govindarajan, Naimark and Coburn19), and non-surgical treatment dominated both surgical resection and diagnostic/palliative surgery in individuals with recurrent colorectal cancer (Reference Miller, Cantor and Peoples22).
Applying the GBP 20–30 k (EUR 25–37.4 k) per QALY threshold to the studies for prostate cancer, radical prostatectomy treatment is unlikely to be considered cost-effective compared with watchful waiting due to considerable uncertainty around the incremental cost-effectiveness ratio (ICER). Again, the cost per QALY results ranged from below GBP 20 k (EUR 25 k) per QALY (except for men aged 75 and over) when the decision was based on patient survival only (Reference Lyth, Andersson and Andren27), to dominated when side effects of the surgical technique were taken into account (28). While a marker-based (DNA-ploidy) treatment selection policy would be considered cost-effective compared with watchful waiting using these thresholds, both policies dominated the “prostatectomy for all” option (Reference Calvert, Morgan and Catto24). However, the ICER was highly sensitive to factors such as sensitivity and specificity (Reference Calvert, Morgan and Catto24), and in one study the ICER ranged from approximately GBP 2 k (EUR 2.5 k) per QALY to GBP 84 k (EUR 105 k) per QALY (Reference Lyth, Andersson and Andren27). Robotic prostatectomy was likely to be cost-effective at GBP 30 k (EUR 37.4 k) per QALY in comparison to laparoscopic prostatectomy when the number of procedures performed per year with the robotic system was over 150 (Reference Ramsay, Pickard and Robertson30), whereas radical prostatectomy was cost-effective at GBP 20 k (EUR 25 k) per QALY threshold compared with open surgery (Reference O’Malley, Jordan and O’Malley29).
DISCUSSION
The limited numbers of studies included in the reviews highlight the scarcity of economic evidence for surgical procedures in these indications. The searches identified just three studies for breast cancer surgery, all were published before 2003 (Reference Malin, Keeler and Wang14–Reference Polsky, Mandelblatt and Weeks16). Of the seven colorectal cancer studies, only two were published after 2010;(Reference Callejo, Guerra and Reza17,Reference Jensen and Abcarian21) and of the seven prostate cancer studies, three were published after 2010 (Reference Hohwu, Borre and Ehlers25;Reference Lyth, Andersson and Andren27;Reference Ramsay, Pickard and Robertson30).
Although some UK specific requirements were included in the methodological checklist (Figure 1), the vast majority of criteria used to assess the scientific rigor of the studies are fundamental requirements for any economic evaluation in health care and are, therefore, generalizable to other settings and decision-making bodies. Scientific rigor is improving in economic evaluations generally, and older publications may not conform to current practice. The studies included in the reviews cover a broad publication range with six (6/17) published before 2004 (older publications) and five (5/17) published after 2010 (recent publications). Assessing the studies chronologically (Figure 2), there is a marked difference in the proportion that satisfies the essential criteria. For example, although only 5/17 of all the studies reported results of full probabilistic sensitivity analyses, none (0/6) were older publications and three (3/5) were recent publications. Similarly, although only 6/17 used a systematic review and meta-analysis to inform the clinical evidence used, none (0/6) were older publications and three (3/5) were recent publications. Similar rates were observed for the use of HRQoL data reported directly by the patients or carers with none (0/6) of the older publications using these data compared with two (2/5) of the recent publications. While the majority of the older evaluations were not undertaken using the current requirements for sources of evidence (clinical data informed by a systematic literature review/synthesis, HRQoL data from patients with the condition) and uncertainty (probabilistic sensitivity analysis) the recent publications would be expected to conform to these basic standards.
There were several essential requirements which were not satisfied in many of the publications. First, much of the clinical evidence was not supported by a systematic review of the literature or a synthesis of the relevant evidence, and very few authors presented the results of a full probabilistic sensitivity analysis. Relying on the effectiveness evidence from a single clinical study or hospital setting in isolation, with no reference to the relevant published evidence, can be misleading as the effectiveness evidence from a single study may differ substantially from other evidence in the area. This may hinder and even mislead policy decision makers in some cases. While it is not always possible to conduct a full systematic literature review and evidence synthesis within the resource allocations of a project, it is generally possible to compare the effectiveness evidence used with the literature. If the data used are comparable to the published literature, this will increase confidence in the results presented. If the data differ substantially from the published literature, then the results obtained using the different values should be compared (preferably using a probabilistic sensitivity analysis) to determine if the evaluation is sensitive to variations in these parameters. Exploring the uncertainty around the point estimates is particularly relevant if the evidence comes from a novel intervention where there is no existing evidence.
The second and closely related area where many of the studies failed to meet requirements was in the choice of time horizon used to sum the benefits and costs accrued from the alternative interventions. It is widely accepted that if an intervention has a differential effect on survival, the most appropriate horizon would be a lifetime. As the patients in the evaluations had cancer, then a shorter horizon (say 10 to 20 years) may be considered sufficient. However, many of the studies restricted the horizon to less than 5 years, or did not provide details of the length of horizon used. It is unlikely that a time horizon of 1 year will accurately reflect the full benefits of an intervention in patients with cancer. While some analysts may be reluctant to rely on assumptions to extrapolate beyond the duration of the clinical evidence, the effect of longer horizons should be explored in sensitivity analyses to determine what if any difference this makes to the results generated.
The final area where many of the studies failed to satisfy the essential requirements relates to the HRQoL data used within the analyses. Over half of the studies did not use data collected from patients (or their carers) with the particular conditions. Reporting standards for both the source of evidence and the methodology used to collect these data were particularly low. Many policy makers require that the preference-weights used to determine QALYs are obtained from the general population. There is an argument that in some health conditions, generic measures such as the EQ-5D may not be sensitive to small changes in dimensions of health or features of a particular condition. In these cases, a condition specific measure of health, such as the Functional Assessment of Cancer Therapy (FACT) for patients with cancer, may be more relevant (Reference Cella, Tulsky and Gray34). At the moment the FACT measure does not have a corresponding preference-weighted index which could be used to determine the QALYs in an economic evaluation. While several of the studies (4/17) conducted analyses to elicit preference weights for defined health states in the evaluations, in many of the studies it was impossible to determine what preference-weights, if any, had been used.
To our knowledge, little or no research has been conducted to date which has compared the equivalence of reporting standards for economic evaluation for surgical procedures and those for pharmaceutical products. Comparing methods and standards in economic evaluation of medical devices and drugs, authors highlighted key challenges for evaluating medical devices. These may generalize to surgical interventions including the practical difficulties in conducting randomized controlled trials (Reference Drummond, Griffin and Tarricone35–Reference Sorenson, Tarricone and Siebert36), the dependence of outcome on the surgeon's expertise (Reference Drummond, Griffin and Tarricone35). and the wider organizational impacts associated with introducing new devices which may differ by location (Reference Drummond, Griffin and Tarricone35–Reference Sorenson, Tarricone and Siebert36). In some countries (for example Australia and Canada), HTAs and cost-effectiveness analyses focus on pharmaceuticals rather than the full range of health-care technologies, which may go some way to explaining why there is a dearth of evidence on surgical interventions.
Given the scarcity of literature and relatively dated clinical evidence for some of the indications and procedures, it is reasonable to question what economic evidence is used to support clinical guidelines in these areas. In the United Kingdom, the number of HTA and NICE guidance issued can be used as an indicator of the priority given to cancer health care (37;38). Since its inception, NICE has issued guidance relating to twenty-eight (11;13) pharmaceutical interventions and seven (13) intervention procedures for breast (colorectal, prostate) cancer, and there have been forty-five (25;32) HTA submissions.
The most recent clinical guideline for early (and locally advanced) breast cancer has very limited de-novo evidence in terms of alternative surgical techniques (39;40), The cost-effectiveness literature was not reviewed under the following topics: “Surgery to the breast” (including Paget's disease), “evaluation and management of a positive lymph node” and “breast reconstruction” as “the Guideline Development Group (GDG) did not consider this topic as a health economic priority”. The only independent evaluation conducted to inform the GDG's recommendations examined alternative strategies used for staging of the cancer before surgery.
For colorectal cancer, the recommendations for laparoscopic surgery in the current UK clinical guidelines (41) were taken directly from NICE technology appraisal guidance on “Laparoscopic surgery for colorectal cancer TA105” (42), which used cost-effectiveness evidence reported in an article included in the current review (Reference Murray, Lourenco and de Verteuil23). Although this evidence suggested laparoscopic surgery was dominated by open surgery, the laparoscopic surgery was recommended over open surgery as the NICE committee informed by TA105 were “persuaded there were important differences” in length of stay and return to normal activities for laparoscopic surgery, and even though costs were greater and there was little direct evidence of quality of life benefits, “it was likely that such benefits exist . . . . . . . and these would be sufficient to make laparoscopic procedures cost-effective”. For prostate cancer, the existing cost-effectiveness evidence was considered to be limited as none included evidence from a recently published RCT, hence, a de novo model (included in this review) was constructed to incorporate these data (see 28).
Authors of a recent report expressed concern with regard to the future of academic surgery in the United Kingdom and highlighted the need for additional research activities in cancer surgery and in particular multidisciplinary research (40). Surgery is often associated with the greatest potential benefit (cure/not cure) and potential harm (risk of morbidity/mortality) when comparing modalities in the cancer treatment pathway (40). It is, therefore, likely that new surgical devices and changes in techniques and timing will be cost-effective and even potentially cost-saving compared with pharmaceutical interventions. However, such a proposition needs to be evidence-based and the scarcity of economic evidence on surgical techniques suggests that this is one area where the surgical community and policy-decision makers could benefit from a wider multi-disciplinary approach to research.
CONCLUSION
There is a dearth of recent robust evidence describing the cost-effectiveness of surgical interventions in these indications. Many of the recent publications did not satisfy essential requirements such as exploring the full uncertainty associated with the evidence or using clinical evidence informed by a systematic review and synthesis. Policy decisions for surgical techniques in these indications do not appear to be informed by robust economic evidence in all cases and due to the ratio of potential benefit and harm associated with cancer surgery there is an urgent need to increase economic activities in this area.
CONTACT INFORMATION
Roberta Ara, MSc(hons), ScHARR, Senior Research Fellow, School of Health and Related Research (ScHARR), Regent Court, 30 Regent Street, Sheffield, S1 4DA
Hasan Basarir, PhD, ScHARR, Research Fellow, The University of Sheffield, School of Health and Related Research (ScHARR), Regent Court, 30 Regent Street, Sheffield, S1 4DA
Anju D. Keetharuth, Phd, ScHARR, Research Associate, The University of Sheffield, School of Health and Related Research (ScHARR), Regent Court, 30 Regent Street, Sheffield, S1 4DA
Marco Barbieri, MSc(hons), Centre for Health Economics (CHE), University of York, Heslington, York YO10 5DD, UK
Helen L.A. Weatherly, MSc(hons), Senior Research Fellow, Centre for Health Economics (CHE), University of York, Heslington, York YO10 5DD, UK
Mark J.S. Sculpher, Phd, Professor of Health Economics, Deputy Director of the Policy Research Unit in Economic Evaluation of Health and Care Interventions (EEPRU), Centre for Health Economics (CHE), University of York, Heslington, York YO10 5DD, UK
Hashim Ahmed, FRCS(Urol), PhD, BM, BCh (Oxon), BA(Hons), MRC Clinician Scientist and Honorary Consultant Urological Surgeon, University College Hospital, 235 Euston Road, London, NW1 2BU
Steven Brown, MBChB FRCS MD, Consultant Colorectal Surgeon, Sheffield Teaching Hospitals, NHS Foundation Trust, Northern General Hospital, Herries Road, Sheffield, S5 7AU
CONFLICTS OF INTEREST
The authors have no conflicts of interest to declare.
Open access
