Objectives: Case series constitute a weak form of evidence for effectiveness of health technologies. However, for a variety of reasons, such studies may be included in health technology assessments. There are no clear criteria for assessing the quality of case series. We carried out an empirical investigation of the association between outcome frequency and methodological characteristics in a sample of health technology assessments.

Methods: Systematic reviews of functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, and percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic stable angina were identified as containing more than forty case series. Data were extracted by one reviewer and checked by a second on population characteristics, outcomes, and the following methodological features: sample size, prospective/retrospective approach, consecutive recruitment, multi- or single-center organization, length of follow-up, independence of outcome measurement, and date of publication. Association between methodological features and outcome were explored in univariate and multivariate analyses using parametric and nonparametric tests and robust regression or analysis of variance/analysis of covariance, as appropriate.

Results: Included reviews contained between forty-two and seventy-six case series studies, involving 5 to 172,283 participants. Reporting of methodological features was poor and limited the analyses. In general, we found little evidence of any association between methodological characteristics and outcome. Sample size is used as an inclusion criterion in many reviews of case series but was consistently shown to have no relationship to outcome in all analyses. A prospective approach was not associated with outcome. Insufficient data were available to explore consecutive recruitment. Mixed results were shown for length of follow-up, independence of outcome measurement, and publication date.

Conclusion: We found little evidence to support the use of many of the factors included in tools used for quality assessment of case series. Importantly, we found no relationship between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, for example, blinding of outcome measurement, but these examples were not shown consistently across the small number of examples studied. Further research into the determinants of quality in case series studies is required to support health technology assessment.

Objectives: Health needs has attracted the interest of policy-makers, health economists, and health professionals as modern health services try to satisfy individual and population health needs to optimize resource utilization. Health needs can be assessed by administering various types of survey or interview-based instruments. If health needs are to be satisfied in changing health agendas in developed and developing countries, it is essential to employ valid and reliable tools. Despite the importance of needs assessment, no comprehensive review of tools is currently available. We carried out a literature search to define and categorize existing health needs assessment tools.

Methods: We reviewed medical and social search engines for items containing specific health needs–related words to identify needs tools across a range of specialties. Papers were reviewed in terms of design, subject matter, psychometric features, and method of administration method.

Results: Thirty-one employed in 52 studies including cancer, mental health, palliative care, multiple sclerosis, and cardiovascular disease tools were identified.

Conclusions: This report summarizes available health needs instruments in a range of diseases to assist researchers in accessing health needs resources more easily and to encourage further research in this field.

Objectives: The feasibility and convergent validity of a cost diary and a cost questionnaire was investigated.

Methods: Data were obtained as part of a cost-utility analysis alongside a multicenter clinical trial in patients with resectable rectal cancer. A sample of 107 patients from 30 hospitals was asked to keep a weekly diary during the first 3 months after surgery, and a monthly diary from 3 to 12 months after surgery. A second sample of seventy-two patients from twenty-eight hospitals in the trial received a questionnaire at 3, 6, and 12 months after surgery, referring to the previous 3 or 6 months. Format and items of the questions were similar and included a wide range of medical and nonmedical items and costs after hospitalization for surgery.

Results: Small differences were found with respect to nonresponse (range, 79 to 86 percent) and missing questions (range, 1 to 6 percent between the diary and questionnaire). For most estimates of volumes of care and of costs, the diary and questionnaire did not differ significantly. Total 3-month nonhospital costs were €1,860, €1,280, and €1,050 in the diary sample and €1,860, €1,090, and €840 in the questionnaire sample at 3, 6, and 12 months after surgery, respectively (p =.50). However, with respect to open questions, the diary sample tended to report significantly more care.

Conclusions: For the assessment of health-care utilization in economic evaluations alongside clinical trials, a cost questionnaire with structured closed questions may replace a cost diary for recall periods up to 6 months.

Objectives: The aim of the study was to investigate the feasibility and effectiveness of searching selected databases to identify information required to populate a decision-analytic model.

Methods: Methods of searching for information to populate a decision-analytic model were piloted using a case study of prophylactic antibiotics to prevent recurrent urinary tract infections in children. This study explored how the information requirements for a decision-analytic model could be developed into searchable questions and how search strategies could be derived to answer these questions. The study also assessed the usefulness of three published search filters and explored which resources might produce relevant information for the various model parameters.

Results: Based on the data requirements for this case study, 42 questions were developed for searching. These questions related to baseline event rates, health-related quality of life and outcomes, relative treatment effects, resource use and unit costs, and antibiotic resistance. A total of 1,237 records were assessed by the modeler, and of these, 48 were found to be relevant to the model. Search precision ranged from 0 percent to 38 percent, and no single database proved the most useful for all the questions.

Conclusions: The process of conducting specific searches to address each of the model questions provided information that was useful in populating the case study model. The most appropriate resources to search were dependent on the question, and multiple database searching using focused search strategies may prove more effective in finding relevant data than thorough searches of a single database.

Objectives: Although ethics has been on the agenda in health technology assessment (HTA) since its inception, the integration of moral issues is still not standard and is performed in a vast variety of ways. Therefore, there is a need for a procedure for integrating moral issues in HTA.

Methods: Literature review of existing approaches together with application of various theories in moral philosophy and axiology.

Results: The article develops a set of questions that addresses a wide range of moral issues related to the assessment and implementation of health technology. The issues include general moral issues and moral issues related to stakeholders, methodology, characteristics of technology, and to the HTA process itself. The questions form a kind of checklist for use in HTAs.

Conclusions: The presented approach for integrating moral issues in HTA has a broad theoretical foundation and has shown to be useful in practice. Integrating ethical issues in HTAs can be of great importance with respect to the dissemination of HTA results and in efficient health policy making.

Objectives: An example of technology assessment in dental care by evaluating the (cost-)effectiveness of types of three-surface inlays (gold, laboratory-fabricated ceramic, and chairside CAD/CAM ceramic) is provided.

Methods: MEDLINE, EMBASE, and the Cochrane Library were searched for studies published between 1966 and June 2003 that reported annual survival probabilities and annual observations. The longevity of different types of inlays was measured by the number of failure-free years. Annual survival rates from different studies were pooled by weighing the rates of each study by the inverse of the variance of the effect estimate. A cost-effectiveness analysis from the perspective of German private health insurers was performed using billing charges.

Results: Three, five, and two case series on laboratory-fabricated ceramic, chairside CAD/CAM ceramic, and gold inlays, respectively, were included. Over a 9-year observation period, the number of undiscounted failure-free years was 8.62 (95 percent confidence interval, 8.40–8.85), 8.65 (8.58–8.73), and 8.76 (8.72–8.80) for laboratory-fabricated ceramic, chairside CAD/CAM ceramic, and gold inlays, respectively. Laboratory-fabricated ceramic inlays were the most expensive.

Conclusions: While laboratory-fabricated ceramic, chairside CAD/CAM ceramic, and gold inlays had a strikingly similar failure-free survival rate, laboratory-fabricated ceramic inlays had the highest costs and, thus, were less cost-effective than chairside CAD/CAM ceramic and gold inlays.

Objectives: The aim of this investigation was to assess the incremental cost-effectiveness of replacing bare metal coronary stents (BMS) with drug-eluting stents (DES) in the Province of Quebec, Canada.

Methods: The strategy used was a cost-effectiveness analysis from the perspective of the health-care provider, in the province of Quebec, Canada (population 7.5 million). The main outcome measure was the cost per avoided revascularization intervention.

Results: Based on the annual Quebec rate of 14,000 angioplasties with an average of 1.7 stents per procedure and a purchase cost of $2,600 Canadian dollar (CDN) for DES, 100 percent substitution of BMS with DES would require an additional $45.1 million CDN of funding. After the benefits of reduced repeat revascularization interventions are included, the incremental cost would be $35.2 million CDN. The cost per avoided revascularization intervention (18 percent coronary artery bypass graft, 82 percent percutaneous coronary intervention [PCI]) would be $23,067 CDN. If DES were offered selectively to higher risk populations, for example, a 20 percent subgroup with a relative restenosis risk of 2.5 times the current bare metal rate, the incremental cost of the program would be $4.9 million CDN at a cost of $7,800 per avoided revascularization procedure. Break-even costs for the program would occur at DES purchase cost of $1,161 for 100 percent DES use and $1,627 for selective 20 percent DES use for high-risk patients for restenosis (RR = 2.5). Univariate and Monte Carlo sensitivity analyses indicate that the parameters most affecting the analysis are the capacity to select patients at high risk of restenosis, the average number of stents used per PCI, baseline restenosis rates for BMS, the effectiveness ratio of restenosis prevention for DES versus BMS, the cost of DES, and the revascularization rate after initial PCI. Sensitivity analyses suggest little additional health benefits but escalating cost-effectiveness ratios once a DES penetration of 40 percent has been attained.

Conclusions: Under current conditions in Quebec, Canada, selective use of DES in high-risk patients is the most acceptable strategy in terms of cost-effectiveness. Results of such an analysis would be expected to be similar in other countries with key model parameters similar to those used in this model. This model provides an example of how to evaluate the cost-effectiveness of selective use of a new technology in high-risk patients.

Objectives: The cost-effectiveness of alternating pressure-relieving devices, mattress replacements, and mattress overlays compared with a standard hospital (high-specification foam mattress) for the prevention and treatment of pressure ulcers in hospital patients in the United Kingdom was investigated.

Methods: A decision-analytic model was constructed to evaluate different strategies to prevent or treat pressure ulcers. Three scenarios were evaluated: the prevention of pressure ulcers, the treatment of superficial ulcers, and the treatment of severe ulcers. Epidemiological and effectiveness data were obtained from the clinical literature. Expert opinion using a rating scale technique was used to obtain quality of life data. Costs of the devices were obtained from manufacturers, whereas costs of treatment were obtained from the literature. Uncertainty was explored through probabilistic sensitivity analysis.

Results: Using £30,000/QALY (quality-adjusted life year) as the decision-maker's cut off point (the current UK standard), in scenario 1 (prevention), the cost-effective strategy was the mattress overlay at 1, 4, and 12 weeks. In scenarios 2 and 3, the cost-effective strategy was the mattress replacement at 1, 4, and 12 weeks. Standard care was a dominated intervention in all scenarios for values of the decision-maker's ceiling ratio ranging from £5,000 to £100,000/QALY. However, the probabilistic sensitivity analysis results reflected the high uncertainty surrounding the choice of devices.

Conclusions: Current information suggests that alternating pressure mattress overlays may be cost-effective for the prevention of pressure ulcers, whereas alternating pressure mattress replacements appears to be cost-effective for the treatment of superficial and severe pressure ulcers.

Objectives: The object of this study was to determine, taking into account uncertainty on cost and outcome parameters, the cost-effectiveness of high-dose chemotherapy (HDC) compared with conventional chemotherapy for advanced breast cancer patients.

Methods: An analysis was conducted for 300 patients included in a randomized clinical trial designed to evaluate the benefits, in terms of disease-free survival and overall survival, of adding a single course of HDC to a four-cycle conventional-dose chemotherapy for breast cancer patients with axillary lymph node invasion. Costs were estimated from a detailed observation of physical quantities consumed, and the Kaplan–Meier method was used to evaluate mean survival times. Incremental cost-effectiveness ratios were evaluated successively considering disease-free survival and overall survival outcomes. Handling of uncertainty consisted in construction of confidence intervals for these ratios, using the truncated Fieller method.

Results: The cost per disease-free life year gained was evaluated at 13,074€, a value that seems to be acceptable to society. However, handling uncertainty shows that the upper bound of the confidence interval is around 38,000€, which is nearly three times higher. Moreover, as no difference was demonstrated in overall survival between treatments, cost-effectiveness analysis, that is a cost minimization, indicated that the intensive treatment is a dominated strategy involving an extra cost of 7,400€, for no added benefit.

Conclusions: Adding a single course of HDC led to a clinical benefit in terms of disease-free survival for an additional cost that seems to be acceptable, considering the point estimate of the ratio. However, handling uncertainty indicates a maximum ratio for which conclusions have to be discussed.

Objectives: The aim of this study was to develop a decision-analytic model to assess the cost-effectiveness of spinal cord stimulation (SCS), relative to nonsurgical conventional medical management (CMM), for patients with failed back surgery syndrome (FBSS).

Methods: A decision tree and Markov model were developed to synthesize evidence on both health-care costs and outcomes for patients with FBSS. Outcome data of SCS and CMM were sourced from 2-year follow-up data of two randomized controlled trials (RCTs). Treatment effects were measured as levels of pain relief. Short- and long-term health-care costs were obtained from a detailed Canadian costing study in FBSS patients. Results are presented as incremental cost per quality adjusted life year (QALY) and expressed in 2003 Euros. Costs were discounted at 6 percent and outcomes at 1.5 percent.

Results: Over the lifetime of the patient, SCS was dominant (i.e., SCS is cost-saving and gives more health gain relative to CMM); a finding that was robust across sensitivity analyses. At a 2-year time horizon, SCS gave more health gain but at an increased cost relative to CMM. Given the uncertainty in effectiveness and cost parameters, the 2-year cost-effectiveness of SCS ranged from €30,370 in the base case to €63,511 in the worst-case scenario.

Conclusions: SCS was found to be both more effective and less costly than CMM, over the lifetime of a patient. In the short-term, although SCS is potentially cost-effective, the model results are highly sensitive to the choice of input parameters. Further empirical data are required to improve the precision in the estimation of short-term cost-effectiveness.

Objectives: Chondral defects of the knee cartilage are prevalent. Autologous chondrocyte implantation (ACI) and mosaicplasty are increasingly used to treat symptomatic knee defects. This study assessed the costs and health status outcomes after ACI and mosaicplasty.

Methods: Patients were eligible to participate in this cross-sectional study if they received ACI or mosaicplasty at the Royal National Orthopaedic Hospital between 1997 and 2001 or were on a waiting list for ACI. Secondary-care resource use was collected to 2 years postoperatively using a resource collection proforma. Participants responded to postal questions about sociodemographic characteristics and knee-related (Modified Cincinnati Knee Rating System) and general health status (EQ-5D).

Results: Fifty-three ACI, twenty mosaicplasty, and twenty-two patients waiting for ACI participated. The average cost per patient was higher for ACI (£10,600: 95 percent confidence interval [CI], £10,036–£11,214) than mosaicplasty (£7,948: 95 percent CI, £6,957–£9,243). Postoperatively, ACI and mosaicplasty patients (combined) experienced better health status than those waiting for ACI. ACI patients tended to have better health status outcomes than mosaicplasty patients (not statistically significant). Estimated average EQ-5D social tariff improvements for quality-adjusted life year (QALY) calculations were 0.23 (ACI) and 0.06 (mosaicplasty). Average costs per QALY were £23,043 (ACI) and £66,233 (mosaicplasty). The incremental cost effectiveness ratio (ICER) for providing ACI over mosaicplasty was £16,349.

Conclusions: Average costs were higher for ACI than mosaicplasty. However, both the estimated cost per QALY and ICER for providing ACI over mosaicplasty fell beneath an implicit English funding threshold of £30,000 per QALY. Prospective studies should include measures of utility to confirm the estimated cost utility ratios of ACI and mosaicplasty.

Objectives: The use of ultrasonography and computed tomography (CT) in the diagnosis of appendicitis in adult patients was compared.

Methods: Systematic review and meta-analysis of current evidence in two clinical situations: unselected nonpregnant, adult patients with symptoms of appendicitis, and more selective use in only those patients who still have an equivocal diagnosis subsequent to routine clinical investigations.

Results: Meta-analysis of eligible studies shows CT to have better sensitivity and specificity than ultrasound in both clinical situations.

Conclusions: Application of these findings in clinical practice and/or policy would need to evaluate the better diagnostic performance of CT against its cost and availability. In addition, it is imperative that future studies be conducted in patient populations that are well-defined with respect to prior investigations. Sequelae of false-negative and false-positive diagnoses should also be evaluated.

Objectives: The objective of physician continuing medical education (CME) is to help them keep abreast of advances in patient care, to accept new more-beneficial care, and discontinue use of existing lower-benefit diagnostic and therapeutic interventions. The goal of this review was to examine effectiveness of current CME tools and techniques in changing physician clinical practices and improving patient health outcomes.

Methods: Results of published systematic reviews were examined to determine the spectrum from most- to least-effective CME techniques. We searched multiple databases, from 1 January 1984 to 30 October 2004, for English-language, peer-reviewed meta-analyses and other systematic reviews of CME programs that alter physician behavior and/or patient outcomes.

Results: Twenty-six reviews met inclusion criteria, that is, were either formal meta-analyses or other systematic reviews. Interactive techniques (audit/feedback, academic detailing/outreach, and reminders) are the most effective at simultaneously changing physician care and patient outcomes. Clinical practice guidelines and opinion leaders are less effective. Didactic presentations and distributing printed information only have little or no beneficial effect in changing physician practice.

Conclusions: Even though the most-effective CME techniques have been proven, use of least-effective ones predominates. Such use of ineffective CME likely reduces patient care quality and raises costs for all, the worst of both worlds.

Objectives: The aim of this investigation was to study the effect of general practitioners' (GP's) characteristics on two decisions: whether to have the Helicobacter pylori rapid test (HPRT) in the office laboratory and whether to use this test or a similar hospital-based serological test in a typical clinical situation described in a vignette.

Methods: Discrete choice analysis with binary logit models were used to predict the probability that a general practice has the HPRT, as well as the GP's probability of using the HPRT or a similar test in this clinical situation.

Results: We found that the number of consultations per week has a positive effect on the probability of having the HPRT, indicating that the size of the practice affects the decision to have such a test in the repertoire. Furthermore, four variables significantly increased the probability of using one of the lab tests: more if located in urban practices, more by solo practitioners, more when the GP stated a high probability for H. pylori associated disease, and more when the GP had the rapid test available in the practice. In our analysis, the remuneration system is endogenous and does not have a significant effect on the two decisions.

Conclusions: Our study demonstrates that characteristics of the GP affect the availability and use of a specific laboratory analysis.

Objectives: There is relatively little evidence on the test–retest reliability of utility scores derived from multiattribute measures. The objective was to estimate test–retest reliability for Health Utilities Index Mark 2 (HUI2) and Mark 3 (HUI3) utility scores in patients recovering from hip fracture.

Methods: We enrolled an inception cohort of hip fracture patients within 3 to 5 days of surgery. Baseline assessments included the Functional Independence Measure (FIM™), Folstein Mini-Mental State Examinations, and the HUI2 and HUI3 questionnaire. Follow-up assessments at 1, 3, and 6 months also included a global change question. Test–retest reliability was assessed as agreement between 3- and 6-month scores using the intraclass correlation coefficient (ICC). Two approaches were used to classify patients as stable; a third approach based on the generalizability theory was also used. Patients were classified as stable if their FIM™ overall scores changed by 10 points or fewer and if they classified themselves as having experienced no or only a little change according to their global change question.

Results: Complete data at both the 3- and 6-month assessments based on self-report were available for 196 patients; 141 patients with complete data were classified as stable. The ICCs for HUI2 and HUI3 for stable patients were 0.71 and 0.72; the ICCs derived from the generalizability theory were 0.76 and 0.77.

Conclusions: Test–retest reliability for HUI in this cohort was similar to reliability estimates for other preference-based multiattribute and generic health-profile measures—in the acceptable range for making valid group-level comparisons.

Objectives: The diffusion of cost-utility analyses (CUAs) through the medical literature was examined, documenting visible patterns and determining how they correspond with expectations about the diffusion of process innovations.

Methods: This study used 539 CUAs from a registry. It includes data elements comprising year of publication, the research center in which the study was performed, the clinical area covered by the CUA, and the specific journal. Finally, each paper was assigned to a journal type that could be one of the three categories: health services research, general medicine, or clinical specialty.

Results: When the average number of publications is plotted against time, the plot reveals an S-shaped curve. It appears that, whereas CUAs initially were published more frequently in general medical or health services research journals, there was a clear increase in the diffusion of CUA into subspecialty journals over time. The concentration ratio for research centers as measured by the Herfindhal–Hirschman Index decreased over time.

Conclusions: The spread of CUA through the medical literature follows patterns identified for the diffusion of other new technologies and processes. Future research should focus on what impact this spread has had on the practice of medicine and formulation of health policy.

Objectives: This work has investigated under what conditions cost-effectiveness data from a cluster randomized trial (CRT) are suitable for analysis using a cluster-adjusted nonparametric bootstrap. The bootstrap's main advantages are in dealing with skewed data and its ability to take correlations between costs and effects into account. However, there are known theoretical problems with a commonly used cluster bootstrap procedure, and the practical implications of these require investigation.

Methods: Simulations were used to estimate the coverage of confidence intervals around incremental cost-effectiveness ratios from CRTs using two bootstrap methods.

Results: The bootstrap gave excessively narrow confidence intervals, but there was evidence to suggest that, when the number of clusters per treatment arm exceeded 24, it might give acceptable results. The method that resampled individuals as well as clusters did not perform well when cost and effectiveness data were correlated.

Conclusions: If economic data from such trials are to be analyzed adequately, then there is a need for further investigations of more complex bootstrap procedures. Similarly, further research is required on methods such as the net benefit approach.

Objectives: As health technology assessments (HTA) may have considerable impact on health-care decisions, it is essential to guarantee the expected quality of these assessments. Variations in the methods used or lack of transparency can affect the important role of HTA reports.

Methods: In our study, we analyzed the methods used in two corresponding HTA reports to assess the validity of two key papers, which were included in both reports. Also the discussions and the final conclusions of both reports were compared.

Results: The authors of the two HTA reports used different instruments to assess the validity of the original studies. A minor problem is differences that were found in the assessments of identical validity aspects of the included studies and in the information provided in the HTA reports. A more serious problem was found in both HTA reports which identified the weakness of the key papers and expressed these limitations in the discussions (read mainly by scientists) but not in the conclusions (read mainly by policy-makers).

Conclusions: The results of this study may be important for any institution that prepares recommendations for policy-makers. In the case of HTA reports, no new checklists are necessary as the checklist offered by the INAHTA contains the needed information. It may be necessary, however, to consider an “intramural quality board,” which helps the member organizations guarantee both the transparency of the original studies included in HTA reports and the transparency of the HTA report itself.

In a recent publication in the “International Journal of Technology Assessment in Health Care” (7), Kildemoes and Kristiansen claim to address “Cost-effectiveness of interventions to reduce the thrombolytic delay for acute myocardial infarction.” Their study is based on a “Master of Public Health Assessment” thesis published by Kildemoes in the year 2001 (6). Three years ago, the author was informed that several of her assumptions were incorrect. In this letter, we will address six of the erroneous assumptions made by Kildemoes and Kristiansen.

We appreciate the interests in our work and observe that we agree that earlier thrombolysis for AMI can reduce the AMI case fatality. The question is how much and at what price. There are no data available to directly address this issue, and we developed a simulation model to quantify costs and health consequences of less thrombolytic delay by using public awareness campaigns, telemedicine, or a combination of the two. Inevitably, such a model needs to be based on several uncertain parameter values. We performed a range of sensitivity analyses so readers of the analysis could see the effect of replacing our base case values with others that the reader might consider more appropriate. Due to space restrictions, we had to omit a table with sensitivity analyses that would have addressed several of the concerns Terkelsen and coworkers have.