Objectives: The objectives of this paper were: a) to determine what can be learned from conclusions of systematic reviews about the evidence base of medicine; and b) to determine whether two readers draw similar conclusions from the same review, and whether these match the authors' conclusions.

Methods: Three methodologists (two per review) rated 160 Cochrane systematic reviews (issue 1, 1998) using pre-established conclusion categories. Disagreements were resolved by discussion to arrive at a consensual score for each review. Reviews' authors were asked to use the same categories to designate the intended conclusion. Interrater agreements were calculated.

Results: Interrater agreement between two readers was 0.68 and 0.72, and between readers and authors, 0.32. The largest categories assigned by methodologists were “positive effect” (22.5%), “insufficient evidence” (21.3%), and “evidence of no effect” (20.0%). The largest categories assigned by authors were “insufficient evidence” (32.4%), “possibly positive” (28.6%), and “positive effect” (26.7%).

Conclusions: The number of reviews indicating that the modern biomedical interventions show either no effect or insufficient evidence is surprisingly high. Intterrater disagreements suggest a surprising degree of subjective interpretation involved in systematic reviews. Where patterns of disagreement emerged between authors and readers, authors tended to be more optimistic in their conclusions than the readers. Policy implications are discussed.

Objective: To determine the extent to which systematic reviews of public health interventions influenced public health decisions and which factors were associated with influencing these decisions.

Methods: This cross-sectional follow-up survey evaluated the use of five systematic reviews in public health decision making. Independent variables included characteristics of the innovation, organization, environment, and individual. Primary data were collected using a telephone survey and a self-administered organizational demographics questionnaire. Public health decision makers in all 41 public health units in Ontario were invited to participate in the study. Multiple linear regression analyses on the five program decisions were conducted.

Results: The systematic reviews were perceived as having the greatest amount of influence on decisions related to program justification and program planning, and the least influence on program evaluation decisions. The greater the perception that one's organization valued the use of research evidence for decision making and that ongoing training in the critical appraisal of research literature was provided, the greater the perception of the influence the systematic review had on public health decisions.

Conclusions: Organizational characteristics are important predictors of the use of systematic reviews in public health decision making. Future dissemination strategies need to promote the value of using systematic reviews for program decision making as well as promote ongoing training in critical appraisal among intended users in Ontario.

Objective: This article outlines basic methods for quantifying reproducibility and responsiveness of evaluative outcome measures.

Methods: The background noise in stable patients provides the desired information to quantify the reproducibility. From this, the smallest real difference (SRD) for longitudinal differences can be derived. We propose to use the SRD to define responsiveness: from all patients who change according to an external criterion, we take the percentage that changes at least SRD on the outcome measure. A more complete picture of the responsiveness of the outcome measure arises when the receiver operating characteristic (ROC) is constructed, corresponding to all possible cut-off change scores. The proposed methods are illustrated with an empirical example.

Results: In the illustration the methods appeared to be very useful and complemented each other. We could evaluate whether the observed change score was larger than that expected due to chance. With the methods it was possible to evaluate both the ability of an instrument to detect change if there is a real change in health status (sensitivity to change) and the ability to detect absence of change if there is no real change (specificity to change).

Conclusion: We presented the use of SRDs and ROC curves for quantifying reproducibility and responsiveness. We started with the basic notions and arrived at methods that are both understandable and useful.

Objectives: In quality-adjusted life-years (QALY) models, it is customary to weigh life-years with quality of life via multiplication. As a consequence, for positive health states a longer duration has more QALYs than a shorter duration (i.e., longer is better). However, we have found that for poor health states, many prefer to live only a limited amount of time (i.e., longer is worse). Such preferences are said to be maximum endurable time (MET). In the present contribution, the following questions are asked: a) How low does the utility have to be in order for a MET to arise? and b) Do MET preferences occur when patients judge hypothetical health states?

Methods and Results: We reanalyzed data from 176 students for the hypothetical health states of “living with migraines” and “living with metastasized cancer.” For utilities smaller than 0.7 (ranging from 0 to 1), the MET preference rate was larger than 50%. High MET preference rates were also found in two new studies on migraine and esophageal cancer patients, who evaluated hypothetical health states related to their disease.

Conclusions: We discuss the interpretation of the MET preferences and the preference reversal phenomenon. Standard QALY models imply that longer is better. However, we find that more often, longer is worse for poorly evaluated health states. Consider the following question: are 3 years with a weight of 0.3 equally as valuable as 1 year with a weight of 0.9? Our results suggest that the 3-year period may be less valuable because for poor health, many will prefer a 1-year over a 3-year period.

Objective: To evaluate the total cost of minilaparotomy cholecystectomy (MC) and laparoscopic cholecystectomy (LC) and perform a cost-effectiveness (CE) analysis.

Methods: One hundred adult subjects with painful gallstone disease were randomized: 59 cases for LC and 41 cases for MC. Patients with gallstones shown on ultrasound with normal common bile duct and no history of icterus were included after an informed consent. Cases with acute cholecystitis and raised alkaline phosphatase were excluded. LC was performed using CO2 insufflation and a Storz 2D video camera. MC was done by transverse rectus cutting incision. Outcome was coded as success or failure. Success was defined as operation without injury to bile duct, viscera or vessels, minimal pain and discomfort at 4 weeks, no wound infection up to 4 weeks, and resumption of work within 2 weeks of operation. The total cost of each case included cost of investigations, cost of disposable articles for operation, cost of drugs, cost of hospital stay, and cost of operation including anesthesia. LC and MC were done with reusable instruments. A “societal viewpoint” has been taken in the cost calculations.

Results: There were 50/59 successful outcomes in LC and 15/40 outcomes in MC group. Total cost for LC was 386,769 rupees (Rs) and for MC was Rs 205,041. CE in LC was Rs 7,735 and in MC was Rs 13,669. Incremental CE ratio comparing LC with MC was 3,028.33.

Conclusion: LC is a more cost-effective method for treatment of gallstone disease.

Assessing the costs and benefits of developing a clinical practice guideline is important because investments in guidelines compete with investments in other clinical programs. Despite the considerable number of guidelines in many industrialized countries, little is known about their costs and cost-effectiveness. The authors have developed specific measures to determine the cost-effectiveness of guidelines, using a German evidence-based guideline on obesity for the diagnosis and treatment of obese patients as a model. The measures are: the number of people needed to cure, the number of people needed to prevent from developing the disease in question, and the number of people to treat in order to break even.

Objectives: The objective of this study was to assess the quality of economic analysis studies published in the medical and economical literature assessing the clinical management of functional dyspepsia.

Methods: Bibliographic search in the main biomedical databases, in articles from bibliographic references, health technology assessment reports, and in gray literature. A specific protocol with economic and clinical items was designed for the evaluation.

Results: Overall, 18 of 162 studies met the inclusion criteria for the assessment. The compared treatment options were very diverse. The main methodologic deficiencies were in perspective of analysis, inclusion of indirect costs, and sources of clinical information.

Conclusions: Specific checklists with clinical and economical items may help to better assess the quality of economic analysis in the field of functional dyspepsia. The methodologic rigor in the application of economic analysis techniques, as well as the use of appropriate clinical outcome measures, is essential to guarantee the reproducibility of the studies.

Objectives: To document the mammography data that are gathered by the organized screening programs participating in the International Breast Cancer Screening Network (IBSN), the nature of their procedures for data quality assurance, and the measures used to assess program performance and impact.

Methods: A detailed questionnaire covering multiple aspects of quality assurance in screening mammography was mailed to IBSN representatives in 23 countries.

Results: Countries collect a wealth of screening mammography data, much of it computerized. Most countries have designated staff for data quality assurance. All provide staff training, and most have documentation requirements for data collection. Nearly all have one or more procedures to maintain data confidentiality. Countries are heterogeneous in collecting and assessing data to monitor screening program performance and impact.

Conclusions: Demonstrating that population-based screening mammography reduces breast cancer mortality requires collection of high-quality data on key aspects of the multi-step screening process. Assuring the quality of data collection systems for screening mammography programs is an important and evolving area for IBSN countries.

Objectives: To analyze the response patterns and trends of 68 surveys of successive NIH consensus panels' views on the NIH consensus process.

Methods: Each panel's responses were compared to an “average” panel's responses calculated by determining the mean response for each survey question across panels.

Results: The results show a stable pattern of panelists' generally positive views. However, several conferences were judged very positively and some very negatively compared to the norm. Most negatively viewed conferences occurred early in the consensus program's history.

Conclusions: The disparate perceptions are discussed and interpreted as reflecting favorable panels' views of recent changes in the NIH Consensus Development Program.

Objectives: To study considerations used by professional and academic leaders to assess the position of new cardiovascular drugs in the therapeutic regimen in relationship to professional characteristics and the level of prescribing.

Methods: Interviews with 39 internists, cardiologists, general practitioners, and hospital pharmacists about considerations regarding the therapeutic position and prescribing of a new cardiovascular drug (losartan or atorvastatin) and professional characteristics. Considerations were classified according to Rogers' characteristics of an innovation, i.e., referring to the drug's relative advantage, compatibility, or complexity. Proportions of respondents mentioning advantageous, comparable, and/or disadvantageous characteristics were used to construct patterns to analyze an overall evaluation of the drugs in relation to professional characteristics and level of prescribing.

Results: The majority of considerations referred to the degree of relative advantage, but different subjects were emphasized for both drugs. Overall patterns of evaluation were generally intermediate and negative. The respondents' profession, mentioning commercial sources of information and self-qualification as a (moderately) early adopter of new drugs differentiated the overall evaluation of the drugs, in contrast to expertness and academic affiliation. The level of prescribing differentiated the overall evaluation only in the case of losartan.

Conclusions: These professional and academic leaders critically evaluated the claims when assessing the position of the drugs in the therapeutic regimen but did not show consensus in their considerations. Accepted principles for prescribing were considered when assessing the therapeutic position of the drugs but resulted in varied tendencies for prescribing.

Background: Although pharmacoeconomic studies constitute a valuable tool for better managing drug consumption, the conditions under which such studies would be performed in Latin American countries have not been explored.

Objectives: The aim of this paper is to evaluate the potential advantages of and pitfalls in doing pharmacoeconomic research in Latin America and to propose avenues to facilitate the development of this field in the region.

Methods: The Canadian guidelines for the economic evaluation of pharmaceuticals served as a structured framework to assess, both prospectively and retrospectively, the conditions under which the pharmacoeconomic component of a clinical trial held in Mexico and Brazil would be and actually was conducted.

Results: The conditions under which pharmacoeconomic evaluations are conducted must be improved if studies are to contribute to the better management of scarce resources across the entire health care system.

Conclusions: The creation of a public funding agency, the reappraisal of administrative data as a management tool in both the public and the private sectors, and the establishment of national guidelines should be considered within the framework of reforms aimed at allowing healthcare systems to meet their objectives of efficiency and equity.

Objectives: The objective of this study was to assess the effectiveness of government regulation of private pharmacy practice in a low-income country.

Methods: The intervention comprised inspections of the pharmacies, information, and distribution of documents to drug sellers and sanctions. It was implemented at two different intensity levels, active and regular intervention. The methods used to assess the effect of the interventions were interviews with the district drug inspectors, drug sellers and customers, inspection of drug purchases, and indicator surveys of pharmacies. Indicators for pharmacy-specific quality as well as for dispensing quality were developed.

Results: The main finding was one of strong overall improvements from initially low levels. The improvements were particularly marked by increases in the availability of essential materials for dispensing by 34% and in order in the pharmacy by 19%. Information given to customers increased from 35% to 51% and the mixing of different drugs in the same package went down from 17% to 9%. The pharmacies in the active intervention districts showed greater improvements for four of the six indicators, although statistically significant compared with the regular intervention districts only for the essential materials indicator.

Conclusions: It was concluded that the regulatory activities have probably been an important factor behind the service quality improvements. It appeared feasible as well as effective to regulate private pharmacy practice in this particular low-income setting.

Objectives: While evaluating the effect of a community-wide informational intervention, this study explored access, health, and demographic factors related to the use of medical reference books, telephone advice nurses, and computers for health information.

Methods: A random sample of households in the intervention city (Boise, Idaho) and two control cities were surveyed about their use of health information in 1996. Shortly thereafter, the Healthwise Communities Project (HCP) distributed health information to all Boise residents. A follow-up survey was conducted in 1998. Overall, 5,909 surveys were completed for a 54% response rate.

Results: The HCP intervention was associated with statistically significant increases in the use of medical reference books and telephone advice nurses. The increased use of computers for health information was marginally significant. Few access, health, or demographic factors were consistently associated with using the different resources, except that people with depression used more of all three information resources, and income was not a significant predictor.

Conclusion: Providing free health information led to an increase in use, but access, health, and demographic factors were also important determinants. In particular, poor health status and presence of a chronic illness were associated with health information use. These results suggest that healthy consumers are less interested in health information, and it may take other incentives to motivate them to learn about prevention and healthy behaviors.

Objectives: To assess the extent to which abstracts of methodological research, initially presented at meetings on systematic reviews, have gone on to be published as full articles.

Methods: Full publication was assessed in three ways: a search was carried out of The Cochrane Library; a search was conducted using MEDLINE; and a questionnaire was sent to the contact author of each abstract.

Results: Approximately half of the abstracts had not been, or were unlikely ever to be, published in full.

Conclusions: The rate of full publication of abstracts related to the methodology of systematic reviews seems similar to that for randomized trials.

Objectives: In a context where sleep laboratories are overwhelmed by a growing demand to diagnose obstructive sleep apnea syndrome (OSAS), efficient substitutive solutions to in-laboratory polysomnography should be found. To compare the effectiveness and costs of home unattended polysomnography (Hpsg) and telemonitored polysomnography (TMpsg), a cost minimization study was performed.

Methods: In a crossover trial, 99 patients underwent on two consecutive nights TMpsg and Hpsg according to a randomized order. A legibility recording criterion was retained to measure effectiveness. A microcosting study of TMpsg and Hpsg was performed. The risks to adopt home strategy or telemonitored strategy, according to different scenario chosen to reach the diagnosis in case of failure of Hpsg or TMpsg, were analyzed.

Results: The recording was considered to be ineffective in 11.2% of TMpsg (95% CI, 4.9–17.4) and in 23.4% (95% CI, 19.12–27.68) of Hpsg. The effectiveness differential was 12.2% (95% CI, 1.8–22.6) (p = .02). Assuming that in case of failure PSGs would be re-realized in the same condition to reach the diagnosis, then TMpsg could be selected if Hc/TMc (cost of Hpsg/cost of TMpsg) > 0.97; Hpsg could be selected if Hc/TMc < 0.76. If 0.76 ≤ Hc/TMc ≤ 0.97, the choice of TMpsg would be ambiguous. TMc was estimated to be $244, while Hc was $153 (Hc/TMc = 0.63).

Conclusion: Unless some specific geographical situations generate significant transport costs, the implementation of a strategy based on unattended polysomnography at home is cost-saving compared to a telemonitoring strategy.

Objective: To determine the actual costs of extracorporeal shock wave therapy (ESWT) in patients with tendinitis of the supraspinatus muscle.

Methods: A comparison of the costs of surgical treatment versus the costs for ESWT was made. The total accrued costs were determined 12 weeks after intervention, using a sample group of 60 patients with calcifying or noncalcifying tendinitis of the supraspinatus muscle.

Results: The costs per case ranged from EUR 2,700 to EUR 4,300 per patient for ESWT and from EUR 13,400 to EUR 23,450 for surgical treatment, dependent on the method of calculation. Approximately 65% of the per-patient cost is attributable to productivity losses in the workplace.

Conclusions: In comparable short-term results, costs for operative treatment are 5–7 times higher than for ESWT. The greater trauma caused by an operative procedure leads to patients being off work for a longer period and thus a correspondingly higher social economic burden.

Objectives: The main objective of this article is to demonstrate the usefulness of dynamic modeling for an economic assessment of technology in health care. Specifically, this approach is applied to assess the impact of the use of hearing aids in Dutch health care.

Methods: The population is divided into different health classes between which, over time, transitions occur. Transition probabilities are derived from exogenous data. The transitions are associated with economic and societal costs and benefits. People who are satisfied with their hearing aids experience benefits. These benefits are expressed by quality-adjusted life-years (QALYs). Costs are made during transitions (mainly the fitting of hearing aids). A cohort analysis is carried out, starting with people in a particular age group. The starting point is a fixed number of people within this age group, who are followed during their whole lifetime.

Results: Costs per QALY ratios are calculated for two health programs. The Fitting Hearing Aid Program describes the present situation in the Netherlands; the Post-purchase Counseling Hearing Aid Program is a hypothetical addition to the first program, where an intervention based on a Dutch study is undertaken to improve satisfaction with hearing aids. Future benefits and costs are discounted at a rate of 5%.

Conclusions: The dynamic modeling approach provides a more realistic picture than a static approach. Particularly, the cost-effectiveness of the Fitting Hearing Aid Program is compared with the Post-purchase Counseling Hearing Aid Program.

Objectives: To assess whether sentinel lymph node biopsy (SLNB), an alternative to axillary lymph node dissection in treating female breast cancer, affords any cost savings.

Methods: We profile cumulative treatment costs of 811 breast cancer patients, 555 of whom received SLNB. Univariate and multivariate statistical tests are used to appraise whether these cost profiles differ between SLNB and other patients.

Results: The statistical results are mixed. However, none supports the conjecture that SLNB necessarily lowers the cost of treating the average breast patient.

Conclusions: SLNB may be cost-effective, but longer term costs and outcomes must be estimated before firm conclusions can be reached.

Cancer is a general term used to describe many different types of malignant tumors. Cancers have the capacity for uncontrolled growth and can spread to different parts of the body. In Sweden, over 40,000 individuals per year (approximately as many men as women) are diagnosed with some form of cancer.