1 Introduction
Most, if not all patients, consult with their physician regarding medical treatment, and the physician plays a major role in the treatment decisions of patients (Reference AuerbachAuerbach, 2001; Reference Degner and SloanDegner & Sloan, 1992). An important finding of recent years is that patients and the general public poorly interpret risk reductions associated with treatment options (see Reference Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz and WoloshinGigerenzer, Gaissmaier, Kurz-Milcke, Schwartz & Woloshin, 2007, for a review). With patients relying on the recommendations of their physicians, we must ask whether physicians understand the reports of risk reduction behind the treatment options they recommend.
Risk reductions associated with medical treatment are typically reported either in relative or absolute terms. Relative risk reduction presents risks associated with a treatment relative to some baseline (e.g., a control group). A risk reduction from 10% to 5% for individuals administered a treatment represents a relative risk reduction of 50%, compared to an overall reduction of 5% in absolute terms. Perhaps not surprisingly, patients and people in general with no formal training in statistics anticipate treatments to be more effective when risk reductions are presented to them in relative than in absolute terms (Reference Akl, Oxman, Herrin, Vist, Terrenato, Sperati, Costiniuk, Blank and SchünemannAkl et al., 2011; Reference Chao, Studts, Abell, Hadley, Roetzer and DineenChao et al., 2003; Reference CoveyCovey, 2007; Reference FeinsteinFeinstein, 1992; Reference Rolison, Hanoch and Miron-ShatzRolison, Hanoch & Miron-Shatz, 2012; Reference Wegwarth, Gaissmaier and GigerenzerWegwarth, Gaissmaier & Gigerenzer, 2010). Of perhaps greater concern, physicians—who typically receive instruction in statistics as part of their basic medical training Footnote 1—also succumb to the lure of relative risk reports, and are found to recommend treatment options more on basis of relative than of absolute risk reduction (Reference Akl, Oxman, Herrin, Vist, Terrenato, Sperati, Costiniuk, Blank and SchünemannAkl et al., 2011; Reference Bobbio, Demichelis and GiustettoBobbio, Demichelis & Giustetto, 1994; Reference Forrow, Taylor and ArnoldForrow, Taylor & Arnold, 1992; Reference Moxey, O’Connell, McGettigan and HenryMoxey, O’Connell, McGettigan & Henry, 2003; Reference Naylor, Chen and StraussNaylor, Chen & Strauss, 1992; Reference Nexøe, Gyrd-Hansen, Kragstrup, Kristiansen and NielsenNexøe, Gyrd-Hansen, Kragstrup, Kristiansen & Nielsen, 2002).
Since physicians, patients, and the general public alike are unduly influenced by relative risk information, it is surprising that relative risk reduction is the most commonly used method of delivering statements about the efficacy of clinical trials in media reports (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000), and in the technical reports on which physicians base their recommendations (Reference Bucher, Weinbacher and GyrBucher, Weinbacher & Gyr, 1994; Reference Fahey, Griffiths and PetersFahey, Griffiths & Peters, 1995). In a survey of media reports, Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al. (2000) found that 83% of reports described risk reductions only in relative terms, compared to just 17% that provided the absolute risks.
Some authors suggest either discouraging the use of relative risk information, or combining it with absolute risk information (Reference Laupacis, Sackett and RobertsLaupacis, Sackett & Roberts, 1988; Reference Rolison, Hanoch and Miron-ShatzRolison et al., 2012; see also Reference Schwartz and MeslinSchwartz & Meslin, 2008), while others advocate policies for improving health literacy in the general public (Reference Schwartz, Woloshin, Black and WelchSchwartz, Woloshin, Black & Welch, 1997). Despite the merits of these approaches, there still remains the possibility that physicians’ misunderstanding of relative risk reduction has been overestimated due to the default assumption, common to the previous studies, that physicians are aware of the baseline risks. Without the baseline risk, the efficacy of a treatment cannot be assessed, and any recommendations based on it may be misled. Depending on whether a baseline risk is 10%, 5%, or 1%, a relative risk reduction of 50% could mean that 50, 25, or 5 lives in every 1,000 will be saved by a treatment. Can we assume that physicians always know the baseline risks, especially when the baseline risks refer to domains outside the physician’s everyday practice? And when physicians know the baseline risk, is their understanding of the relative risk reduction accurate? Literature has showed that patients and the general public overestimate the relative risk reduction even when the baseline risk is explicitly provided (Reference Chao, Studts, Abell, Hadley, Roetzer and DineenChao et al., 2003; Reference Wolf and SchorlingWolf & Schorling, 2000). To date, however, no previous study has examined the influence of baseline risk information on physicians’ interpretations of relative risk information.
In the current article, we present physicians with reports of a fictitious clinical trial of a new treatment for hypertension—a chronic but treatable cardiovascular disease (Reference Ong, Cheung, Man, Lau and LamOng, Cheung, Man, Lau & Man, 2007). The report provides the risk reduction of the treatment either in terms of its absolute risk reduction, or in terms of its relative risk reduction with or without including the baseline risk information. Our aims were (a) to examine whether physicians’ willingness to prescribe a treatment and their ratings of its effectiveness would be influenced by the method in which risk reduction is provided (relative or absolute risk reduction), and (b) whether explicitly reporting the baseline risk in combination with information about relative risk reduction would influence the physicians’ willingness to prescribe and ratings of effectiveness.
2 Method
2.1 Participants
A total of 95 physicians (55 men, 40 women; mean age=37.27, SD=10.64), of which 41 were interns and 54 resident physicians at a public hospital in Northeast Italy were contacted at their workplace. All physicians agreed to participate in the study on a voluntary basis.
2.2 Materials and Procedure
Physicians were presented with one of three vignettes, adapted from a previous study by Forrow et al. (1992). The vignette reported the results of a clinical trial of a new drug treatment for hypertension. Physicians were randomly assigned to receive the vignette reporting the clinical trial outcome either as an absolute risk reduction, a relative risk reduction, or a relative risk reduction that included the baseline risk. The vignette that referred to an absolute risk reduction followed (translated from Italian; N=32):
A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the experimental drug treatment reduced the overall mortality rate over the 5 year period from 7.8% in the “routine treatment” control group to 6.3%. The 1.5% reduction in total mortality observed over the 5 year period was statistically significant.
The vignette that referred to a relative risk reduction followed (N=30):
A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the group treated with the experimental drug registered a reduction in the overall mortality rate over the 5 year period by 20.3% as compared with rate in the “routine treatment” control group. This reduction was statistically significant.
The vignette that referred to a relative risk reduction including the baseline risk followed (N=33):
A randomized controlled study of over 6,000 men with a moderate form of hypertension (DBP 90-104) conducted to evaluate the effectiveness of an experimental drug treatment, showed that the group treated with the experimental drug registered a reduction in the overall mortality rate over the 5 year period by 20.3% as compared with rate in the “routine treatment” control group (the “routine treatment” control group’s overall mortality rate over the 5 year period was equal to 7.8%). This reduction was statistically significant.
Regarding the vignette, physicians were asked to rate the effectiveness of the drug treatment on a 7 point scale ranging from −3 (“Not effective at all”) to 3 (“Very effective”), and their willingness to prescribe the drug treatment on the same 7 point scale from −3 (“Certainly not inclined to prescribe”) to 3 (“Certainly inclined to prescribe”).
3 Results
Physicians’ mean group ratings of treatment effectiveness and their willingness to prescribe are provided in Figure 1. A one way independent analysis of variance (ANOVA), including presentation method (absolute risk reduction, relative risk reduction, relative risk reduction including baseline risk) as an independent factor, and including physicians’ experience (residency vs. internship) and gender as covariates, was conducted separately on physicians’ ratings of effectiveness and their willingness to prescribe. The analysis revealed significant effects of presentation method both on physicians’ ratings of effectiveness (without covariates: F (2, 92)=13.53, MS e=0.71, p<.001, η2=.23; with covariates: F (2, 90)=13.02, MS e=0.71, p<.001, η2=.22) and their willingness to prescribe (without covariates: F (2, 90)=9.56, MS e=0.66, p<.001, η2=.17; with covariates: F (2, 90)=9.17, MS e=0.65, p<.001, η2=.16). There were no significant effects of gender (ratings of effectiveness, F (1, 90)=0.08, MS e=0.71, p=.773; willingness to prescribe, F (1, 90)=0.53, MS e=0.65, p=.470) or experience (ratings of effectiveness, F (1, 90)=1.78, MS e=0.71, p=.185; willingness to prescribe, F (1, 90)=2.86, MS e=0.71, p=.094).
Figure 1: Physicians’ ratings of the effectiveness of a new treatment (Panel A) and their willingness to prescribe it (Panel B) by method of risk presentation. Bars represent 1 standard error below and above mean ratings. ARR = Absolute Risk Reduction, RRR = Relative Risk Reduction.
Follow-up t-test comparisons (with Bonferroni correction) revealed significant mean group differences on ratings of effectiveness between the absolute risk reduction method and the relative risk reduction method, regardless of whether (p<.001) or not (p<.001) the baseline risk was included, such that physicians rated the treatment as more effective when information was provided as a relative (including baseline risk, M=2.24, SD=0.75; not including baseline risk, M=2.07, SD=0.64) rather than as an absolute risk reduction (M=1.22; SD=1.10). Regarding physicians’ willingness to prescribe, compared to the absolute risk reduction method (M=1.19, SD=0.90), physicians were significantly more willing to prescribe the treatment when presented as a relative risk reduction (including baseline risk, M=2.06, SD=0.70, p<.001; not including baseline risk, M=1.73, SD=0.83, p=.029). Including the baseline risk in combination with the relative risk reduction did not affect physicians’ ratings of effectiveness (p=.213) nor their willingness to prescribe (p=.205) compared with when the baseline risk was not included with the relative risk reduction. Footnote 2
4 Discussion
The physician plays an important role in the treatment decisions of patients (Reference AuerbachAuerbach, 2001; Reference Degner and SloanDegner & Sloan, 1992). It is thus crucial that we examine how physicians evaluate reports of new clinical trial outcomes, and how their evaluations influence their willingness to prescribe new treatments to patients and their expectations about the effectiveness of new treatments.
In this study, we provided physicians the risk reduction associated with a new (but fictitious) treatment. Physicians were presented the treatment’s benefits as an absolute risk reduction, as a relative risk reduction, or as a relative risk reduction including the baseline risk. In this respect our study is unique. In fact previous studies (e.g., Forrow et al., 1992), as well as media reports of clinical outcomes (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000) typically do not report the baseline risk, a crucial aspect of relative risk information. If the baseline risk is unknown, the relative risk reduction cannot be interpreted. We reasoned that participants of previous studies could have overestimated the effectiveness of treatments because they did not know the baseline risk information.
As expected, physicians claimed that the new treatment would be more effective and that they would be more willing to prescribe it when its risk reduction was presented to them in relative rather than in absolute terms. To our surprise, however, physicians appeared entirely unaffected by the addition of the baseline risk in combination with the relative risk reduction.
Why do physicians’ not take into account baseline risk information when evaluating clinical outcomes? We do not know whether physicians are not aware that the baseline risk information is relevant, or whether they do not have the statistical or numerical skills to integrate baseline risk information with relative risk reduction (see, for example, Reference Barbey and SlomanBarbey & Sloman, 2007; Reference Kahneman and TverskyKahneman & Tversky, 1973). It is clear, however, that physicians, like others, are unduly influenced by the “big” numbers that relative risk reductions present. Indeed, this may also explain the popular use of relative risk information in media reports of clinical outcomes (Reference Moynihan, Bero, Ross-Degnan, Henry, Lee and WatkinsMoynihan et al., 2000).
It would seem that, even when all relevant statistical information is made available, the format in which risks are reported to physicians influences strongly their expectations of a treatment’s effectiveness and their willingness to prescribe a treatment to patients. Our current results echo the strong recommendations made by others that information about absolute risk reduction should be made available to physicians (as well as to the lay public) when information about relative risk reduction is used (Reference Laupacis, Sackett and RobertsLaupacis et al., 1988; Reference Rolison, Hanoch and Miron-ShatzRolison et al., 2012). The lure of relative risk information may be especially strong among the general public who typically do not have direct training in interpreting health risk statistics (Reference Galesic and Garcia-RetameroGalesic & Garcia-Retamero, 2010; Reference Schwartz, Woloshin, Black and WelchSchwartz et al., 1997). Although better than the general public, physicians also show difficulties with numerical skills necessary for interpreting health risk statistics (Reference Estrada, Barnes, Collins and ByrdEstrada, Barnes, Collins & Byrd, 1999). Thus, we suggest also that more extensive statistical and numerical training be made available to physicians.
In conclusion, based on our current results we strongly recommend that health professionals be cognizant of the potential biasing effects of risk information presented in relative risk terms. We urge that absolute risk information also be made available when clinical outcomes are reported to physicians.
Open access
