Objectives: Since 1988, four government-funded health technology assessment (HTA) agencies have been established in Canada. This paper is a descriptive review of reports issued by these organizations during the period from 1988 to 1998.

Methods: Publications from the national and three provincial HTA agencies in Canada were obtained and reviewed. Only the 117 assessment reports that were reported to have undergone external review were included in this analysis. Each report was classified on a standard abstraction form according to criteria such as technology type(s), assessment focus, whether a specific policy question was clearly stated and relevant decision maker(s) identified, description of search strategy and selection criteria, sources of data and assessment methods used, whether recommendations or conclusions were made, and duplication or overlap of reports. The trends in these qualities over the 10-year period were also examined.

Results: Therapeutic technologies have received the most attention from all four agencies, although the focus on devices, drugs, and procedures varied between agencies. The policy question under investigation was clearly identified in 82% of reports, and 71% clearly identified the decision maker toward whom the assessment was targeted. Efficacy or effectiveness was examined in 81% of reports, and costs were assessed in 65% of studies. These were the two most frequently examined aspects. Almost all assessments were descriptive literature reviews; 9% included meta-analyses and 32% had cost analyses or economic evaluations. Forty-four percent of reports had a clear description of the literature search strategy, and selection criteria were clearly specified in 38% of studies, but there was considerable variation among agencies in the level of description of these methods. Conclusions were clearly stated in 83% of the assessments' conclusions, and 13% had recommendations. When analyzed longitudinally, it is apparent that the quality of reports has improved markedly during the past decade. This was determined by examining the clarity of specifying the policy question(s) under investigation, the identification of the target audience of decision makers for the information, and by evaluating the thoroughness of the description of the methods used in the assessment.

Conclusions: Canadian government agencies have contributed a considerable quantity of health technology assessments. There has been very little duplication of technologies evaluated, and the quality of the assessment reports has markedly improved during the past decade.

Switzerland has a mixed public and private healthcare system. All citizens are enrolled in compulsory basic health insurance. A 1996 law allows people to choose among different sickness funds and managed care plans. The federal government is empowered to act on important health issues, but the 26 cantons have prime responsibility in health care and social welfare. They have their own laws on health care, hygiene, hospitals, and social welfare. These laws are not harmonized. The system is complex, with a mix of public (mainly hospitals) and private (mainly doctors' offices) providers. The health services are decentralized. Ambulatory care was traditionally provided in doctors' offices, but the last decade has seen the development of centers for day surgery, group practices, and managed care plans. Decisions on placement, location, and extension of services are decentralized. The payment system is very complex. Current trends include global budgets, cost analyses, and prices related to patient categories. However, coverage policy is developed centrally and includes both traditionally established services and new technologies. New technologies are added to the list only after evaluation by the Federal Coverage Committee. The coverage process integrates health technology assessment (HTA). Coverage can be granted in stages, including limited coverage and temporary coverage. Technologies and coverage can be reevaluated on the basis of registries or assessment information. The structure of the Swiss healthcare system does not lend itself to the establishment of a national HTA program. However, recent moves include the development of a coordinating mechanism for HTA in Switzerland.

Objective: To assess the influence of physician specialty and the way in which patient data are presented in the treatment recommended for patients with coronary artery disease.

Methods: In a prospective study, 3,628 patients with significant coronary artery disease who had been referred to 1 of 10 heart centers in the Netherlands as possible candidates for either percutaneous transluminal coronary angioplasty (PTCA) or coronary artery bypass graft surgery (CABG) were recruited. Within each center, the recommended treatment is determined by a team consisting of cardiologists only, cardiovascular surgeons only, or cardiologists and cardiovascular surgeons (i.e., composite teams). The main outcome measures are the proportions of patients for whom PTCA, CABG, or noninvasive (medical) therapy was recommended.

Results: Composite teams made 71% of recommendations, surgeon-only teams, 12%, and cardiologist-only teams, 17%. Cardiologist-only teams primarily recommended patients to PTCA, surgeon-only teams to CABG, while combined teams made more evenly distributed recommendations (p < .001). Although the patients discussed by the three types of teams were clinically different, the recommendation patterns remained significant after adjusting for these differences (p < .001). For patients with recent myocardial infarction, direct presentation of the case to the team by the referring cardiologist reduced the likelihood that CABG would be recommended.

Conclusions: The treatment recommended to patients with coronary artery disease is affected by the composition of the team providing the recommendation. These findings have important implications for clinical decision making for patients with cardiovascular disease.

Objective: Severe to profound hearing impairment affects one-half to three-quarters of a million Americans. To function in a hearing society, hearing-impaired persons require specialized educational, social services, and other resources. The primary purpose of this study is to provide a comprehensive, national, and recent estimate of the economic burden of hearing impairment.

Methods: We constructed a cohort-survival model to estimate the lifetime costs of hearing impairment. Data for the model were derived principally from the analyses of secondary data sources, including the National Health Interview Survey Hearing Loss and Disability Supplements (1990–91 and 1994–95), the Department of Education's National Longitudinal Transition Study (1987), and Gallaudet University's Annual Survey of Deaf and Hard of Hearing Youth (1997–98). These analyses were supplemented by a review of the literature and consultation with a four-member expert panel. Monte Carlo analysis was used for sensitivity testing.

Results: Severe to profound hearing loss is expected to cost society $297,000 over the lifetime of an individual. Most of these losses (67%) are due to reduced work productivity, although the use of special education resources among children contributes an additional 21%. Life time costs for those with prelingual onset exceed $1 million.

Conclusions: Results indicate that an additional $4.6 billion will be spent over the lifetime of persons who acquired their impairment in 1998. The particularly high costs associated with prelingual onset of severe to profound hearing impairment suggest interventions aimed at children, such as early identification and/or aggressive medical intervention, may have a substantial payback.

Objective: Extracorporeal shock wave therapy (ESWT) as an emerging technology in orthopedics has been assessed in Austria with the objective to establish a scientific basis for pending and pressing health policy decisions. Despite encouraging results within some indications and the promising prospect of a noninvasive treatment for some orthopedic diseases, it seemed crucial to assess this new field of application in the light of evidence-based standards, without forgetting the reality of healthcare decisions. This article presents the results of the abovementioned assessment in the context of the policy-making process in Germany, Austria, and Switzerland.

Methods: The study is based on an overview of the already existing reviews on the effectiveness of ESWT as carried out by different actors (health insurances and orthopedists), and on an overview of additional literature covering all relevant aspects (mode of action, recommendations, economic estimates). Orthopedic societies, several institutions involved in health technology assessment, and health insurance agencies were contacted in search of further relevant literature, ongoing studies, and assessments.

Results: Until now, clear evidence has not been available on the effectiveness of ESWT in four currently recommended indications (tendinosis calcarea of the shoulder, epicondylopathia humeri radialis, calcaneal spur, and pseudarthrosis), although a substantial body of literature has been produced. Despite encouraging results in some indications, the lack of coherent therapy schemes and the poor quality in the design of clinical studies has been serious enough to hinder the drawing of reasonable conclusions.

Conclusions: Due to the rapid diffusion of ESWT and the pressure on reimbursement agencies to prove the ineffectiveness of ESWT, the policy option of a pragmatic approach is recommended to controll the diffusion of an uncertain health technology, alongside active research that would allow a more comprehensive appraisal of this potentially interesting treatment strategy in orthopedics.

Though the desire to live is fundamental and it is never easy to accept death, a full acceptance is based on the understanding that death is an inherent part of all life and is necessary for the continuation and vitality of species. (1)

Objective: This study investigates the influence of different aspects of methodologic quality on the conclusions of a systematic review concerning treatments of acute lateral ankle sprain.

Method: A data set of a systematic review of 44 trials was used, of which 22 trials could be included in this study. Quality assessment of the individual studies was performed using the Delphi list. We calculated effect sizes of the main outcome measure in each study in order to evaluate the relationship between overall quality scores and outcome. Next, we investigated the impact of design attributes on pooled effect sizes by subgroup analysis.

Results: The quality of most studies (82%) was low; only 4 of 22 trials were of high quality. Studies with proper randomization and blinding procedure produce a slightly higher (not statistically significant) effect estimate compared to the other studies.

Conclusion: Previous research has suggested that methodologically poorly designed studies tend to over-estimate the effect estimate. Our study does not confirm these conclusions.

The National Health Service (NHS) provides universal health coverage for all British citizens. Most services are free of charge, although modest copayments are sometimes applied. About 11% of the population also has private insurance. General practitioners, generally the first point of contact for accessing the system, are independent contractors who serve as gatekeepers for specialist and hospital services and enjoy substantial clinical autonomy. Hospitals are public and are regionalized, but the 1990 reforms made them self-governing trusts that contract with local purchasers (health authorities and general practitioner fundholders). Reforms beginning in 1990 moved the NHS away from a centralized administrative structure to more pluralistic arrangements in which competition, as well as management, influences how services develop. Health technology and health technology assessment (HTA) have gained increasing attention in the NHS during this period, as part of a wider NHS Research and Development (R&D) Strategy. The strategy promotes a knowledge-based health service with a strong research infrastructure and the capacity to critically review its own needs. HTA is the largest and most developed of the programs within the strategy. It has a formal system for setting assessment priorities involving widespread consultation within the NHS, and a National Co-ordinating Centre for Health Technology Assessment. The stategy supports related centers such as the U.K. Cochrane Centre and the NHS Centre for Reviews and Dissemination. A hallmark of the HTA program is strong public participation. The United Kingdom has made a major commitment to HTA and to seeking effective means of reviewing and disseminating evidence.

Objectives: Inappropriate hospital admissions are commonly believed to represent a potential for significant cost reductions. However, this presumes that these patients can be identified before the hospital stay. The present study aimed to investigate to what extent this is possible.

Methods: Consecutive admissions to a department of internal medicine were assessed by two expert panels. One panel predicted the appropriateness of the stays from the information available at admission, while final judgments of appropriateness were made after discharge by the other.

Results: The panels correctly classified 88% of the appropriate and 27% of the inappropriate admissions. If the elective admissions predicted to be inappropriate had been excluded, 9% of the costs would have been saved, and 5% of the gain in quality-adjusted life-years lost. The corresponding results for emergency admissions were 14% and 18%.

Conclusions: The savings obtained by excluding admissions predicted to be inappropriate were small relative to the health losses. Programs for reducing inappropriate health care should not be implemented without investigating their effects on both health outcomes and costs.

Objectives: To evaluate the implementation of clinical guidelines for diabetes mellitus in general practice with a specific computer-based clinical decision support system (CDSS) as part of the intervention.

Methods: Randomized study with health center as unit. General practice in Sør- and Nord-Trøndelag counties in Norway, 380,000 inhabitants. Seventeen health centers with 24 doctors and 499 patients with diabetes mellitus were in the intervention group and 12 health centers with 29 doctors and 535 patients were in the control group. Main outcome measures were group differences in fractions of patients without registrations (process evaluation) and mean group differences for the same variables (patient outcome evaluation).

Results: Statistically significant group differences were experienced for fractions of patients without registration of cigarette smoking (intervention group, 82.6%; control group 94.5%), body mass index (78.2% vs. 93.0%), and sufficient registrations for calculation of risk score for myocardial infarction (91.1% vs. 98.3%); all during 18 months. Large center variations were shown for all variables. The only statistically significant group difference was −2.3 mm Hg (95% CI, −3.8, −0.8) in diastolic blood pressure in favor of the intervention group. Statistically insignificant differences in favor of the intervention group were HbA1c, −0.1% (95% CI, −0.4, 0.1), systolic blood pressure, −1.2 mm Hg (95% CI, −4.4, 2.0). Statistically insignificant differences in favor of the control group were fractions of smokers, +3.0% (95% CI, −4.0, 10.0), body mass index, +0.3 kg/m2 (95% CI, −0.8, 1.4), risk score in female +0.1 (95% CI, −5.1, 5.2), and risk score in male +2.6 (95% CI, −14.2, 19.5).

Conclusions: Implementation of clinical guidelines for diabetes mellitus in general practice, by means of a CDSS and several procedures for implementation, did not result in a clinically significant change in doctors' behavior or in patient outcome.

Objectives: To describe the types of report that are cited by protocols and reviews included in the Cochrane Database of Systematic Reviews.

Methods: The citation for each reference included in the Cochrane protocols and reviews published in the Cochrane Database of Systematic Reviews in issue 1, 1999 of The Cochrane Library were categorized by reference type (e.g., journal article, conference proceeding, book chapter, personal communication, Cochrane review).

Results: Of a total of 24,913 citations, 21,694 (87.1%) were references to journal articles. There was a significant difference between the proportion of references to studies that were journal articles (12,348 of 13,472; 91.7%) and the proportion of other references in this category (9,346 of 11,441; 81.7%).

Conclusion: The great majority of studies included in Cochrane reviews at the beginning of 1999 had been published as journal articles.

The papers on the 16 European countries presented in this issue show their great diversity of health systems. Health technology assessment (HTA) has been institutionalized in a number of members of the European Union and has a growing impact on health policy. A remaining challenge is to see HTA visible and useful at the European level, which requires more active coordination of national and regional activities. A network of HTA programs and researchers has been established. The HTA-Europe Steering Committee has suggested a number of actions by the European Commission to promote further coordination. The most important conclusion of the HTA-Europe report is that a permanent coordinating structure is needed at the European level.

Objective: The rise in the number of implantations of cardiac pacemakers is of some concern to decision makers in the health sector. We assessed the intrinsic and relative clinical efficacy of cardiac pacemakers in current clinical indications to find out whether scientific or clinical arguments might justify differences in market prices.

Methods: We retrieved papers on cardiac pacing (January 1993–April 1998) from five databases (MEDLINE, HealthSTAR, EMBASE, Cochrane Library, and PASCAL). The citations in these papers were used to seek further articles. We selected the articles that met the criteria of evidence-based medicine (EBM) (randomized and nonrandomized controlled trials) and classified them according to clinical indication and type of evaluation (either of the intrinsic efficacy of a pacemaker versus a control or of the relative efficacy of different pacing modes).

Results: A total of 542 references were retrieved, but under 10% met our EBM criteria. Very few were comparative studies versus controls; most were recent and tended to use endpoints other than survival. Clinical efficacy was not proven on the basis of EBM criteria, even in common indications (e.g., sick sinus syndrome). Studies comparing different pacing modes were rarely randomized and did not provide consistent evidence for the superiority of any pacing mode in a given indication.

Conclusions: Knowledge of the natural history of the diseases for which cardiac pacing is indicated is scarce. There is an approximately 20-year gap between technological progress and clinical evaluation that cannot be easily bridged because of methodologic difficulties and ethical issues. Current guidelines on pacemaker use either rely on expert opinion or highlight present inadequacies and make recommendations for future work. Available clinical efficacy data do not justify the wide differences in the price of cardiac pacemakers.

Objectives: Cost-effectiveness analysts often present cost-effectiveness results by age to help inform decisions about the use of an intervention. Yet it is not known how well studies model the risks and costs associated with age. We reviewed published studies to examine their modeling of age differences.

Methods: MEDLINE searches identified all cost-effectiveness analyses published between 1985 and 1997 that included adults 50 years of age and older, were based on data for developed countries, and compared cost-effectiveness ratios for adults of different ages or for initiation of an intervention at different ages; 36 articles met these criteria. They were reviewed to determine the extent to which they incorporated age-specific data. Studies that justified using the same data for all ages were counted as having varied the data element by age.

Results: All studies varied life expectancy by age. Most also varied the incidence/prevalence of the target condition and the case fatality rate. Only 36% varied the effectiveness rate of the intervention by age. Costs were usually assumed constant: 42% of studies varied the cost of treating adverse effects and 17% varied the cost of treating the target condition. Whether a data element was varied did not appear to be related to the pattern of cost-effectiveness ratios by age.

Conclusions: Many studies have not modeled age differences in sufficient detail to ensure that differences in cost-effectiveness ratios by age are accurate and a sound basis for decisions. As cost-effectiveness analysis becomes more widespread, analysts should strive to incorporate more complete age-specific data.

This paper discusses the challenges facing health technology assessment (HTA) in Europe, based on an explicit analysis of the characteristics of an “optimal” HTA system. It has three objectives: a) to elaborate an explicit system of policy goals and the characteristics of an optimal HTA system that facilitates the achievement of these goals; b) to identify the general institutional incentive barriers (government and market failures) that prevent the attainment of an optimal HTA system in Europe; and c) to argue that evaluation of the implications of health technologies for equity and inequality in health is an essential part of this optimal system and a considerable challenge for HTA decision makers, especially as national governments realign policy toward equity goals.

Background: Despite the growing medical and public interest in reducing exposure to allogeneic blood, little is known about the use of alternatives to allogeneic transfusion. This study was conducted to determine the availability of these technologies in Canadian hospitals and was undertaken under the auspices of the International Study of Peri-Operative Transfusion (ISPOT), a 10-country study of the effectiveness of, attitudes toward, and practices related to the use of alternatives to allogeneic transfusion.

Methods: A cross-sectional national mail survey of Canadian hospitals with greater than 50 medical/surgical beds. Chiefs of anesthesia, surgery, and the divisions of cardiac, orthopedic, vascular, and urology were initially mailed a brief postcard asking which of seven technologies were used in their center. This was then followed up with a one-page questionnaire asking how frequently the technologies were used, their thoughts on the appropriateness of the use of the technologies, barriers to their greater use, and reasons for nonuse of the technologies.

Results: Response rates to the postcard survey ranged from 70%–98%, depending on the technology and type of surgery, and ranged from 27%–53% for the follow-up questionnaire. All technologies were used most frequently in cardiac surgery. Aprotinin, tranexamic acid, aminocaproic acid, desmopressin, and cell salvage were reported used in over 70% of cardiac surgery centers. Of these, tranexamic acid and cell salvage were the only ones used routinely in some centers. Acute normovolemic hemodilution and erythropoietin were used in 45% and 20% of cardiac centers, respectively. The drugs were used in less than 15% of orthopedic, vascular, and urologic divisions, with the exception of desmopressin in urologic and vascular surgery and aminocaproic acid in urologic surgery. The techniques of cell salvage and acute normovolemic hemodilution were used in 30%–45% of these divisions, with the exception of cell salvage, which was used in less than 15% of urology units. In more than 60% of cases, the technologies were considered to be used “about right,” although an important minority felt that they were underused.

Conclusions: In general, alternatives to perioperative allogeneic transfusion were rarely used except in cardiac surgery.

Objectives: To estimate the potential for cost reduction in the acute care setting and the required investment in the home care setting of implementing an outpatient/early discharge strategy for operable (stages I and II) breast cancer in Canada.

Methods: Data from a community hospital were augmented by expert knowledge and incorporated into the breast cancer submodel of Statistics Canada's Population Health Model. For the estimated 90% of patients for whom this approach was assumed to be appropriate, the resource utilization for outpatient breast-conserving surgery and 2 days of hospitalization for those women undergoing mastectomy was quantified and costed, as were the appropriate home care services. A 5% readmission rate for complications was assumed. Cost per case, total cost burden, investment in home care, savings in acute care, and net savings were calculated. Sensitivity analyses were performed around readmission rates and home care/surgical follow-up costs. All costs were determined in 1995 Canadian dollars.

Results: The cost of initial treatment for the 15,399 women diagnosed with stages I and II breast cancer in 1995 in Canada was estimated to be $127.6 million. Hospitalization made up 53% of these costs. Under the outpatient/early discharge strategy, the acute care cost of initial breast cancer management could be reduced by $47.2 million, with an investment in home care of $14.5 million ($453 per patient), resulting in an overall net saving of $33 million. Under this strategy, hospitalization would contribute only 21% to the total care cost.

Conclusions: If Canadian surgeons and healthcare administrators were to work together to put in place processes to support ambulatory breast cancer surgery and if resources were redirected to the provision of home-based post-operative care, there would be potential for a large net healthcare saving and preservation of high-quality patient care.

[bull ] Many interventions in health and health care are implemented at the level of geographic area or health service organizational unit. Such interventions will be implemented for entire clusters of healthy subjects or patients.

[bull ] Evaluation of cluster level interventions may be difficult because only a small number of clusters may be available for study and the responses of individuals are correlated within clusters.

[bull ] This review synthesized current methodologic recommendations into a checklist for investigators:

1. Recognize the cluster as the unit of intervention or allocation;

2. Justify the use of the cluster as the unit of intervention or allocation;

3. Include a sufficient number of clusters;

4. Randomize clusters wherever possible;

5. In nonrandomized studies include a control group;

6. In single group studies include repeated measurements over time;

7. Allow for clustering when estimating the required sample size;

8. Consider matching or stratifying clusters where appropriate;

9. Choose between repeated cross-sectional and cohort designs in studies with follow-up;

10. Allow for clustering at the time of analysis;

11. Allow for confounding at both individual and cluster level; and

12. Include estimates of intraclass correlation and components of variance in published reports.

[bull ] The present level of adherence to these recommendations is low.

Objectives: As health care budgets continue to face close scrutiny, any new acquisition must be evaluated for both costs and outcomes. This study was undertaken to demonstrate the application of an economic framework for the evaluation of a multileaf collimator as an example of a new technology that can have both quantifiable and nonquantifiable benefits for patients, staff, and cancer care institutions.

Methods: Using financial data from the Northeastern Ontario Regional Cancer Centre (NEORCC) and a recognized staffing model, a commercial spreadsheet, developed to economically characterize the principal radiotherapy processes has been used to determine the net incremental annual cost of a multileaf collimator (MLC).

Results: The incremental annual cost of purchasing an MLC is estimated at approximately $85,000 (1997 CDN $). Without increasing patient throughput, this increases the average cost of a course of radiotherapy by approximately CDN $200. Savings can be accrued by decreasing mold room activity, increasing the hourly patient capacity on each treatment machine, and decreasing sick time due to strain injuries.

Conclusions: Although the clinical outcome of techniques facilitated by MLCs, such as intensity-modulated radiation therapy, are unknown at this time, an economic context within which to objectively evaluate this technology is presented. The framework presented suggests a method of quantifying outcome-justified expenditures, such as improved patient outcome and greater treatment flexibility, which, when offset against the incremental annual equipment cost, may be used to help justify the acquisition of multileaf technology.

Objectives: To consider the impact of rapid health technology assessments undertaken as part of a program in a provincial healthcare system in response to urgent requests for advice.

Methods: Review of the development and preparation of 20 rapid assessment reports, communication with decision makers within the healthcare system, and appraisal of data subsequent to preparation of the reports.

Results: Fourteen of the assessments were judged to have had an influence on policy and other decisions, as judged by responses from those who had requested advice. Another four were considered to have provided guidance, while having less immediate influence on decisions, and two others had no apparent impact. Quality of the assessments was considered acceptable, on the basis of literature that subsequently became available and from comments received.

Conclusions: These brief reports are considered to be a useful component of a health technology assessment program. However, they should be regarded as provisional appraisals and followed up with more detailed evaluation where possible.