This discussion of the Office of Technology Assessment (OTA) health program will emphasize the early work of OTA to define the new field of health technology assessment (HTA, then called “medical technology assessment”). During its 21 years of life, the OTA Health Program produced approximately eighty full assessments, plus a large number of shorter technical documents, briefing papers, and so on. Most of these had little to do with the shaping of HTA itself, although they may have been important in certain other areas concerned with health.
Health was not initially thought of as a high priority topic for technology assessment in the Congress. However, in 1974–75, some important members of Congress, notably Senator Edward Kennedy, who was chairman of the OTA Board several times, believed that OTA should enter the health field. A person was hired to start up this activity. Part of the initial work of the Health Program was based on a 1975 letter from Senators Kennedy and Javits which asked OTA to “examine current Federal policies and existing medical practices to determine whether a reasonable amount of justification should be provided before costly new medical technologies and procedures were put into general use.” This work became the basis of much of the work in the Health Program for the next several years. (To undertake full-scale studies at OTA required a letter or letters from at least one chairman or ranking minority member of a Congressional Committee.)
Banta was hired to begin to work on topics suggested by this letter. He had been interested in such issues, especially since publication of Cochrane's book, Effectiveness and Efficiency (Reference Cochrane4). (Cochrane had addressed the annual meeting of the Institute of Medicine in Washington, DC in 1971, where Banta heard his remarks. Others who also became influential on this issue were also present. Notably, the source of the Kennedy/Javits letter was probably stimulated by Cochrane's speech.) Clyde Behney joined OTA to work on this issue as well. (Behney became Health Program Manager when Banta was promoted to Assistant Director of OTA, and later succeeded Banta as Assistant Director of OTA, a position which he held until OTA was abolished in 1995.)
From 1975 to 1983, OTA was generally little known to the majority of members of the Congress, and the Health Program was able to carry out studies without much Congressional interest or “interference.” Several reports, discussed below, were based on the Kennedy/Javits letter. Eventually, these reports became known in much of the world and influenced further developments in health and medical technology assessment.
In addition to the staff members of OTA already named, key people in these reports included Jane Sisk, PhD, and Bryan Luce, DrPH, both economists. The Health Program had an advisory committee that gave much useful input. The Committee was chaired by Frederick Robbins, Nobel Laureate, and during part of his chairmanship, the President of the Institute of Medicine (IOM) of the National Academy of Sciences. Each assessment report also had its advisory panel with a carefully chosen chairman and members representing important “stakeholders,” but also generally people who had the reputation of being knowledgeable and experienced, and also willing to listen and respond to other points of view. These advisory committees were very important, even essential, to the reports and their eventual impact. The members of the committees also commented many times what a good experience being on the committee had been, and they undoubtedly promoted the reports to the outside world, which helped establish OTA as source of health information in the outside world and in the Congress.
One confusion we often ran into concerning the Health Program was making people understand that OTA was not a national agency working for the public. It was part of the U.S. Congress, and formally worked only for the Congress. Few health technologies are of direct concern to the Congress, so carrying out assessment of specific technologies was never a central task for the Health Program. Instead, the challenge was to identify policy issues toward health technology that were or might be the concerns of the Congress. Some assessments of specific technologies were done, but these were usually done as part of methodological explorations or to illustrate concerns with health technology. The United States did develop a national HTA program, the National Center for Health Care Technology (NCHCT), but the NCHCT only existed for 2 years, and the United States has not, since then, had an HTA-dedicated national agency and approach to health technology.
OTA had a great advantage from the standpoint of the staff, compared with other policy analysis organizations, such as the Institute of Medicine (IOM). In IOM studies, the committee controls the study and the staff is answerable to that committee, although of course IOM has overall quality control. However, at OTA the staff controlled the studies, and although each study had an advisory committee, these were truly advisory in nature. Of course, most of the time this was a distinction without a difference, but on occasion it resulted in a stronger report. In addition, for staff morale, it was important that they ultimately had the say as to what went into a report. In our opinion, the reports were better because the people in control knew the Congressional environment and also had a general expertise.
THE REPORT ON DEVELOPMENT OF MEDICAL TECHNOLOGY: OPPORTUNITIES FOR ASSESSMENT
This first report in the series had the key aim of relating assessment of medical or health technology to the broader field of technology assessment (12). In addition, although Banta was eager to tackle the problem of efficacy, he realized that he did not know enough about the general field, the research and development (R&D) process for medical technology, or how to work with an advisory panel to produce an effective report. In effect, this first report was a kind of “shake-down” cruise.
The report was conceived as examining the assessment of social impacts of medical technology, with the later planned report on efficacy and safety to be a paired report. The first report did not have much visible impact, which may have been a reflection of lack of interest in such social issues, a lack of interest that largely continues today. Or, indeed, it may have reflected the diffuseness of the report, trying to deal with too many complicated issues.
Banta and his main colleague and staff member, Dr. Joshua Sanes, now Professor of Molecular and Cellular Biology at Harvard University, were quite proud of the chapter they wrote on the process of R&D for medical technology. Ultimately, though, the Health Program Manager at that time did not believe that this chapter was well-integrated into the report, so he had it moved to an appendix. Otherwise, the report reviewed how technology assessment had been developed, including experiences in the Executive Branch of the U.S. government, and speculated as to how medical technology assessment might be developed and used in policy making.
Actually, the chapter that was moved to the Appendix has probably had the greatest impact. It presented the idea of the diffusion of technology and the diffusion curve. The stages in diffusion became the underlying model for much of the Health Program's later work, especially in that formal governmental policies could be related to different phases in the diffusion of health technology.
Another interesting chapter of the report presented a list of questions that might be used in carrying out a social assessment of a medical technology. The questions were drawn mainly from 11 key articles concerning the social implications of medical technology, referenced in the report. One of those key sources was the report from the National Institutes of Health assessment of the totally implantable artificial heart (5). The assessment was a “comprehensive assessment,” so it was used to illustrate the answers to the questions concerning, for example, the implications for the patient, the implications for the family, the implications for society, and so forth.
THE REPORT ON ASSESSING THE EFFICACY AND SAFETY OF MEDICAL TECHNOLOGIES
This might be the most important report done by OTA in terms contributions to the development of HTA (6). The issue of efficacy and safety, as indicated by the Kennedy/Javits letter and the Cochrane book, was increasingly visible. However, as yet, relatively little attention had been paid to the issue in methodological terms, and there had been almost no consideration about how such information might be used in policy and practice.
The report was developed and written by Banta and Behney, with enormous help from an excellent advisory panel chaired by Dr. Lester Breslow. Later, Banta and Behney especially commended one panel member, Professor Kenneth Warner, for his contributions. The chapters of the report included definitions and methods, several case studies illustrating problems of efficacy and safety in technology of that time, assessment activities in the United States, focusing on the federal government, and status and availability of information on efficacy and safety. Indeed, its definitions of medical technology, of efficacy and safety, and of medical technology assessment were widely used as a standard for many years and are still accepted today, in the cases of efficacy, effectiveness, and safety.
In many ways, this report was ground-breaking. It pointed out the pervasive lack of accessible information on efficacy and safety, despite more-than-adequate methods of assessment. It also pointed to many problems that resulted from this lack, and the limited use of such information in clinical practice and policy making. It promoted the use of synthesis of information to produce statements of efficacy and safety (presaging the Cochrane Collaboration and other efforts to make such information widely available). It also gave a wealth of information on efforts to assess efficacy, focusing especially on the program of the National Institutes of Health to fund important clinical trials of (especially) new technologies.
The core of the report was a simple model (a more complicated version of the model was also presented) of the assessment process: (i) identification of the technology to be assessment, including setting priorities between candidates for assessment; (ii) testing or carrying out studies, especially concerned with efficacy and safety; (iii) synthesis of available information on efficacy and safety to reach conclusions in the case of a specific technology; and (iv) dissemination of the conclusion to those who needed the information and could act on it.
This model was followed in many HTA programs around the world, and was the organizing framework in the EUR-ASSESS report, in which HTA in Europe was examined and stimulated (Reference Banta1).
One thing that the report did not do was to develop a scheme for the uses of such information in policy making. That is peculiar, considering that this had been an important goal for the study. The truth is that the researchers and the advisory panel were excited by the topics and information that were being covered, and the subject of use of the information was almost forgotten. In addition, policy analysis was not common at that time, at least not in the health field, and no one involved in the study was experienced in such activities. The omission was realized in the last meeting of the advisory panel. “What do we do,” we said to each other. It was agreed that we would publish the report as it was, but add a short discussion to “open the door” to a further discussion on uses of the information.
This report was truly ground-breaking. It was used, to our knowledge, in many parts of the world and influenced perspectives and policies in many countries. One example was that Dr. Jose Laguna, Dean of the School of Public Health of Mexico at that time began to require (at least) his best students to read the report as a fundamental contribution to health. Dr. Laguna knew the report thoroughly and discussed it in detail with his students. (Dr. Luis Duran, a present-day leader of the health sector in Mexico reported this to Banta during Banta's visits to Mexico in 1999.)
The report was the most thorough discussion of efficacy and safety available at that time. The subject was not covered, or not covered in depth, in standard discussions of evaluation, including clinical trials. The report's distinctions between efficacy and effectiveness were also a key element of the study.
In addition, the cases covered important subjects in medical technology, and almost all of the seventeen cases subsequently underwent some re-evaluation and reconsideration. Several clinical trials followed. Some visible cases, for example, were electronic fetal monitoring, pap smear for cervical cancer, mammography, prophylactic antibiotics in surgery, skull X-ray examinations, surgery for coronary artery disease, and hysterectomy.
THE REPORT ON THE IMPLICATIONS OF COST-EFFECTIVENESS ANALYSIS OF MEDICAL TECHNOLOGY
This study was begun in 1978 as a result of rising healthcare costs and resulting proposals to try to rationalize expenditures on health care through use of cost-effectiveness analysis (13). Behney directed the study, and the advisory panel was chaired by the late Dr. John Hogness. The study had a generous budget, and included seven other professional staff, several contractors, and consideration of several related issues. As in the efficacy and safety report, the discussion of methodological issues was thorough and sound (14). An exhaustive literature review was carried out by Dr. Kenneth Warner of the University of Michigan, with the help of Dr. Bryan Luce. At approximately that time, there were increasing calls to cover psychotherapy in the Medicare program, and a Congressional fellow with OTA, Dr. Leonard Saxe, examined the evidence on the efficacy of psychotherapy and policy issues surrounding its possible coverage, considering cost-effectiveness (Reference Banta and Saxe3). Banta, who was the Program Manager, requested and received from Behney a small amount of money to examine the use of cost-effectiveness analysis and other evaluative tools in making policy in several other countries) (Reference Banta and Kemp2). As far as we know, the information concerning HTA in other countries was the first such information available to the general public.
A particular contribution of the report on cost-effectiveness was to examine different U.S. Federal government programs to see whether analysis of cost-effectiveness might be of assistance in making decisions in those programs. In particular, the issue of basing insurance coverage on such information was discussed, probably the first time this issue, which later became a key issue in the United States as well as other countries, was systemically raised and analyzed.
Several policy options were presented, suggesting how cost-effectiveness analysis could be used in formal decision making. Again, this was the first time these issues had been publicly considered, as far as we are aware.
Another aspect of the report is that analyses were commissioned of several technologies or groups of technologies to examine issues related to their cost-effectiveness. Each was published as a separate monograph in the Cost-Effectiveness series. In several cases, these cases were truly forward-looking. For example, the report on psychotherapy, referred to above, made it clear that psychotherapy could be examined for cost-effectiveness, in a similar way to other technologies. A report on a common dental problem, surgery for gum disease, questioned whether surgery was a correct or cost-effective approach, again anticipating events that came to more attention some years later.
The range of authors commissioned to prepare these cases illustrates another way that OTA affected the history of HTA: involving a large number of people who had played or who would subsequently play a major role in health policy in a common effort to understand these new issues surrounding cost-effectiveness assessment (e.g., John Bunker, Milton Weinstein, Stuart Schweitzer, Duncan Neuhauser, William Stason. Harvey Fineberg, David Eddy, Judith Wagner, Richard Rettig, Jonathan Showstack, and Steven Schroeder,
STRATEGIES FOR MEDICAL TECHNOLOGY ASSESSMENT
The report on strategies did not develop or present any truly new material, but it drew together the material from these early three reports into a comprehensive synthesis of issues related to HTA, especially methods of HTA and policies toward health technology (18). The report was able, though, to examine U.S. Federal government policies in a depth that had not been achieved in the earlier reports, so it was more critical and constructive than the earlier efforts. In effect, the report proposed a method for managing technological change in health care. Little of this was applied, unfortunately, because the United States was beginning a shift to the political right at that time, and there was little interest in expanding and strengthening the government role in health care. Much of the report is still relevant, and could have lessons for the situation today, if a truly national health policy were developed in the United States.
OTA AND COVERAGE OF PREVENTIVE SERVICES
Until the late 1970s, the U.S. Medicare program, the Federal insurance program covering health services for elderly people (and certain others) did not cover preventive services at all. Apparently, this was because the Congress was skeptical of the value of prevention (as was the U.S. medical profession at that time) and was concerned that the expenditures on prevention would be difficult to control.
During 1979 OTA carried out a detailed analysis of U.S. government policies toward vaccines (16). The most interesting part of the report from the standpoint of history of HTA is that the report included a cost-effectiveness analysis of pneumococcal vaccine. The pneumococcal vaccine had been licensed by the Food and Drug Administration in 1977, based on extensive studies of efficacy and safety. One of the main goals of the vaccine was to prevent pneumococcal disease in elderly people, but sales were slow. One reason for this situation was the fact that preventive procedures were not covered by the Medicare program. The OTA analysis showed that the vaccine was quite cost-effective. After publication of the report, Merck Sharp and Dohme, one of the vaccine manufacturers, put the OTA report in a special cover and its representatives took the report to every Congressional office, apparently saying something like, “You see, our vaccine is quite cost-effective, according to your own OTA.” The Congress passed an amendment to cover the pneumococcal vaccine in the Medicare program.
At least one observer protested to OTA that the outcome was perverse: to cover the pneumococcal vaccine, when the more effective and important vaccine against influenza was not covered by the Medicare program. OTA staff agreed. OTA then carried out a cost-effectiveness analysis of influence vaccine, published in 1981, which demonstrated a high degree of cost-effectiveness for that vaccine (11). After a period of time, Congress also passed an amendment to require the Medicare program to cover the influenza vaccine.
After these events, the House Ways and Means Committee requested that OTA analyze the cost-effectiveness of selected preventive health services to the elderly under the Medicare program, and the Senate Labor and Human Resources Committee requested that OTA provide information on the value of preventive services for the American people. A series of analyses on preventive services followed (7–10;17). Those that OTA found to be cost-effective also became part of the Medicare program. In the case of those technologies not found to be cost-effective, such as cholesterol screening, the technology was not covered.
This prevention program for the elderly undoubtedly had large health benefits for enrollees in the Medicare program, and set important precedents concerning coverage of health services based on cost-effectiveness.
However, this mechanism was unwieldly and time-consuming. Although doubtless of value, it did not give a sound approach to covering preventive services for the Medicare population. Therefore, in 1990, OTA carried out a special analysis concerning options for Medicare coverage, such as amending the Medicare law to cover cost-effective services linked to a program to be developed in the Executive Branch to analyze preventive services for cost-effectiveness (15). Unfortunately, this report seemed to have little impact.
CONCLUSIONS
During its life, the OTA health program carried out many worthwhile policy-oriented assessments. However, from the standpoint of HTA, the early OTA reports are recognized around the world for defining the field of HTA. We are not aware of any dissent to this conclusion. Moreover, almost no significant amendments to these early results have occurred. One significant exception occurred during the EUR-ASSESS project, funded by the European Commission during the period 1994–97 (Reference Banta1). It should also be mentioned that several countries, notably the United Kingdom and the Netherlands, have had programs on HTA methodology. These programs have certainly made advances in such areas as biostatistics, but in terms of the policy-relevance of HTA, we are not aware of any significant changes in the conclusions of the OTA reports.
A pioneering activity in HTA was examining preventive activities for cost-effectiveness to determine whether they should be included in Medicare, the main public health insurance program in the United States. This series of reports anticipated what has become perhaps the largest area of use of HTA in policy making today, the use of HTA in insurance coverage decisions.
How was OTA so successful? Basically, we believe that the circumstances were right. There was widespread concern about the implications of technology in general and health technology in particular. The background of technology assessment and the visibility of OTA beginning in the late 1960s and early 1970s played an important role. It could be that the beginning work in Harvard University, certain U.S. Federal programs, and Sweden would have led to a similar outcome. However, certainly, the fact that OTA was able to spend some years thinking about and analyzing ways to approach health technology also was a critical event in the field's development.
CONTACT INFORMATION
David Banta, MD, MPH ([email protected]), 9 route de Bragelogne, 10210 Villiers-le-Bois, France
Clyde J. Behney, MBA ([email protected]), Deputy Executive Director, Institute of Medicine, National Academy of Sciences, 500 Fifth Street, NW, Washington, DC 20001
