R2. Why do deep societal problems persist? Two competing perspectives

In our paper, we talk about individual- and system-oriented policies, drawing upon the widely used metaphor according to which social, economic, and political life is viewed as a game (or rather a complex system of interlocking games). The players of each game – citizens, politicians, corporations, think tanks, university researchers, governments, corporations, and many others – interact with one another, seeking to further their objectives, which might take any form, from complete selfishness, to the promotion of the interests of the group, to universal altruism (see Bowles, for an insightful discussion of why a single concept of utility may be insufficient to capture the diversity of such objectives). The outcome of any game depends both on the rules of the game itself (the focus of the s-frame), and how – and how well – the players can play according to those rules (the focus of the i-frame).Footnote ³

Persistent societal problems arise when particularly crucial games “go wrong,” leading to outcomes such as climate change, inadequate financial provision for retirement, the obesity “epidemic,” plastic pollution, and the spiraling costs of health care. Substantial societal problems typically develop over decades and continue to grow despite widespread alarm, active programs of research, and highly motivated and informed groups campaigning for change.

Why do such problems persist? One possibility is that, although the players all genuinely want to fix the problem, they can't figure out, or agree on, what to do. More research, perhaps, is required. On this reading, the various players, despite their diverse concerns, would be willing to help to reduce carbon emissions, plastic pollution, inequality, or spiraling healthcare costs, if only they knew how. We believe that this “good faith” perspective is often implicit in policy thinking, including behavioral public policy.

But there is a second, very different explanation, which assumes that societal problems persist because of conflicts of interests between the players. What may seem to be deeply pathological outcomes of social and economic games may in fact be highly beneficial to some participants in the game. If solving a societal problem damages powerful and concentrated interests, these interests will work to block reform.

According to this conflictual perspective, the key obstacle to solving many of society's problems lies not with helping well-intentioned actors find better policies. Indeed, we suggest that the well-intentioned often know very well what to do, at least in general terms. What is lacking is the political influence and popular support to drive through reform, in the teeth of powerful and well-funded opposition. Smoking provides a relatively uncontroversial case. After early and compelling evidence about the harms of tobacco, it was clear that smoking needed to be reduced; and also relatively uncontroversial what measures (s-frame taxes and bans, combined with large and consistent public information campaigns) would be effective. But these measures (as well as the underlying science) were relentlessly contested by “big tobacco,” with their considerable financial and lobbying power. Indeed, tobacco companies continue to aggressively promote smoking in many countries around the world, even as profitable Western markets decline. Thus, more than 20% of the global population now smokes, and, according to the World Health Organization (WHO),Footnote ⁴ half of these people will die of smoking-related diseases. As the case of smoking illustrates, where conflicts of interest are creating or maintaining major societal problems, the solution will typically require campaigning and building political coalitions to change the “rules of the game” so that the wishes of concentrated special interests do not prevail.

Johnson & Dana strongly endorse such a conflictual perspective, and review a substantial body of research corroborating our point that existing, concentrated, interests are often behind legislation and regulations that support their interests. They provide, however, a useful caution to our call for s-frame reform (see also Heath): “C&L are surely right that traditional regulations, whether through bans or incentives, will change behavior more than nudging. Yet a public choice analysis suggests that this is a reason for more, not less, caution in proposing regulation: Poor nudging can waste resources; poor regulation can lay waste to us all.” We agree, but stress that the poorest regulations of all are those that, as is so often the case, have been crafted by powerful interests opposed to change – the phenomenon of “regulatory capture” (e.g., Laffont & Tirole, Reference Laffont and Tirole1991). It is therefore especially important to apply behavioral science to design, and build support for, better s-frame policies.

Some commentators take the conflictual perspective for granted, and stress the need for reforms in different areas, and at different scales, to address them (e.g., Bright, Parry, & Thoma [Bright et al.]; Edelman; Strohminger & Táíwò). Other commentators, rather to our surprise, downplay the conflictual origins of social problems. Indeed Sunstein, fresh from an earlier commentary accusing us of being “reactionaries” (Sunstein, Reference Sunstein2022; for our response, Chater & Loewenstein, Reference Chater and Loewenstein2022) now makes the rather mystifying claim that our conflictual analysis can be dismissed as a “conspiracy theory.” Sunstein's charge would, if valid, apply to almost all academic studies of the political and policy-making process, where the conflictual analysis is taken for granted across the ideological spectrum, from Karl Marx (Reference Marx2004/1867) to Chicago School Economics (e.g., Becker, Reference Becker1983; Stigler, Reference Stigler1971). Indeed, the conflictual perspective is entirely standard in fields such as political science, political economy, public health, climate policy, the sociology of science, and many more (Bartels, Reference Bartels2016; Brownell & Warner, Reference Brownell and Warner2009; Mann, Reference Mann2021; Oreskes & Conway, Reference Oreskes and Conway2011).Footnote ⁵

The difference between the good faith versus conflictual perspective is crucial in understanding political debate concerning persistent social challenges – and the role of i- and s-frame interventions. The good faith perspective takes the superficial content of this debate at face value, accepting as genuine tobacco companies' expressed doubts that smoking kills, fossil-fuel companies' dismissal of the idea that greenhouse gases cause rising global temperatures, the gun lobby's questioning that the availability of assault weapons impacts the scale of mass shootings, and the insurance industry's view that the US model of private health insurance provides good value health care for the average citizen. The conflictual perspective suggests, instead, that for these and many other apparent debates about “the facts” are phony. The facts are widely and long agreed upon by any serious individual who doesn't have an economic stake in disbelieving them. Uncertainty and confusion arises less from genuine uncertainty than from deliberate obfuscation and disinformation from powerful interests that would be disadvantaged by s-frame reform.

It doesn't require a conspiratorial mindset to appreciate that many current policy debates regarding persistent social problems are similarly phony. Powerful interests benefitting from the status quo will, of course, continually attempt to raise doubts about the “quality of the evidence,” or will agitate endlessly for “more research” before action is taken (while often blocking the ability to carry out that research). But, according to the conflictual perspective, these are moves in an economic and political game – aimed at delaying s-frame reform.Footnote ⁶ If persistent social problems typically arise from political log-jams caused by competing interests, better insights into how to change individuals' behaviors are likely to have only marginal impacts. From this perspective, behavioral science will contribute to better policy primarily by helping to overcome the special interests that block s-frame reform.

R3. The definitions of i-frame and s-frame

Some commentators (e.g., Hallsworth; Madva, Brownstein, & Kelly [Madva et al.]) question the clarity of our i-frame/s-frame distinction. For example, Hallsworth worries that “the distinction does not offer much clarity and holds up poorly under scrutiny.” By contrast, we suggest that the core distinction is simple, clear, and is indeed already deeply woven into the social sciences. S-frame interventions involve changing the rules of the game; i-frame interventions attempt to modify the actions of the players, within the existing rules. Distinguishing rules and actions is basic to seeing the social and economic world in terms of games at all: A game is, after all, simply an interaction between agents (players) governed by some set of rules.

Interestingly, libertarian paternalism (Sunstein & Thaler, Reference Sunstein and Thaler2003; see also Camerer, Issacharoff, Loewenstein, O'Donoghue, & Rabin, Reference Camerer, Issacharoff, Loewenstein, O’Donoghue and Rabin2003, and Heath for insightful discussion) relies on the same distinction, but with a different slant. Here, changing the rules of the game (s-frame change) is viewed as infringing individual autonomy; “nudging,” or providing information, advice, or education (i-frame change), is viewed as preserving individual autonomy. Where we differ from libertarian paternalism is not on the i-frame/s-frame distinction, but on which approach should be prioritized when dealing with major social challenges.

The natural viewpoint of policy makers (and we suspect the general public) is that when games go wrong, we need to change the rules of the game or “system” (e.g., introducing regulations, subsidies, and taxes to decarbonize the economy). The libertarian paternalist perspective offers a superficially attractive “light-touch” alternative (e.g., for climate change, defaulting people into green electricity tariffs; or helping people compare their energy consumption with that of their neighbors). Our contention is that light-touch liberty-preserving alternatives are rarely a meaningful alternative to, and can distract from, much-needed rule changes. But more to the point here: The very concept of libertarian paternalism requires distinguishing between interventions that change the rules of the game (thus impacting individual autonomy); and “nudges” that don't change the rules but encourage players to respond differently.

Now any such discussion (whether from a libertarian perspective, or from our own) is relative to which game, and which players, we are focusing on. Banning displays of cigarettes in shops is an i-frame nudge if we focus on consumer choice (consumers can still buy cigarettes, but it is now harder); it is an s-frame nudge if we focus on the interaction between retailers and the legal system (the retailers aren't merely discouraged from displaying cigarettes – they are legally required not to do so). But whichever game is the focus, changing the rules of the game, rather than nudging the players, is likely to be more effective. Focusing on consumers, bans and taxes will be more impactful than nudges that make cigarettes less salient in shops. Focusing on retailers, banning cigarette displays outright will be more impactful than nudging them to make displays less salient. If any game is going persistently wrong, the policy maker's, and the public's, first thought should be how to change the rules. This typically means facing up to a trade-off between welfare and liberty, to be resolved by the normal political processes. For most persistent social problems, to follow the libertarian paternalist in hoping this trade-off can be dodged, by helping players play better within the existing rules, is to pursue a mirage.

Some commentators propose introducing a third, intermediate level of analysis (a group-frame, Bingley et al. or community-frame, Caggiano et al.). We are very sympathetic to this emphasis on groups of many kinds, including popular movements (Mermelstein & Preston; see also Cole, Reference Cole2016). Regarding theory and terminology, though, we believe it is simpler to stick to the basic distinction (changing the rules of the game vs. the actions of the players), while allowing that the players in the game (i.e., system) under study can be groups (or other aggregate entities) rather than individuals (just as we might model a wage negotiation game between labor unions and businesses, rather than individual workers and managers). Similarly, there may be interesting i-frame/s-frame issues within groups or organizations (e.g., concerning struggles for control within a social movement).

R4. Crowd-out, crowd-in, and the impetus for s-frame reform

Several commentators doubt our assertion in the target article and in prior work (Hagmann, Ho, & Loewenstein, Reference Hagmann, Ho and Loewenstein2019) that nudges may “crowd out” support for more substantive policies. We will turn later to an obvious, material, crowd-out effect: That armies of researchers testing nudge interventions are not engaged in other policy-related research. Here, we focus on the narrower question of whether exposure to i-frame interventions, whether in the news or in practice, diminishes support for structural changes.

Sunstein, who has extensive experience in public policy, claims to have never seen such crowd-out. This conflicts with other top-level policy makers we have interacted with who, reading our paper or hearing us present it, have reported on situations in which the (false) promise of nudges diminished support for more substantive reforms. Sunstein dismisses the systematic pattern we identify of corporate support for the i-frame as no more than “arresting stories,” but seems to have complete confidence in his own personal observations from his time in the White House.

Sunstein seems to misunderstand our worry that behavioral scientists' focus on i-frame interventions can weaken support for systemic reform. He claims “If we were making a list of 100 reasons why system reform has not happened in some important area (such as climate change), the fact that some behavioral scientists have been enthusiastic about i-frame interventions could not possibly make the list.” We are certainly not claiming that crowd-out is one of the most important causes of lack of progress on climate change (or any other issue). This would be to radically overestimate the power of behavioral insights, or indeed any other source of policy recommendations, when compared with the vast political and commercial forces battling for control of the climate agenda. We are not expecting behavioral public policy to single-handedly change the world; we are hoping that more reflection on our focus as a discipline may increase the degree to which we can collectively contribute in a positive direction as far as possible.

We are surprised that Sunstein expresses such confidence that i-frame research findings will not reduce the perceived need for s-frame change. To requote a passage we highlighted, he recently wrote (Sunstein, Reference Sunstein2021, p. 548):

This very description pitches i-frame interventions as an alternative to what was “long thought” to be the best approach: A carbon tax. Sunstein seems to imply that the best approach may not be a carbon tax, or similar s-frame reform, but that i-frame nudges may provide an alternative. Behavioral scientists need to be very cautious about conveying such an impression, whether intentionally or not.

Sunstein also dismisses our research as “unreliable non-evidence, including surveys finding that if you tell people about an i-frame intervention, you can reduce support for an s-frame intervention.” We assume Sunstein cannot be suggesting that survey evidence is always unreliable – indeed, he has a long track-record of using, and drawing strong conclusions from, survey methods. But if that is not the implication of his statement, then why are the diverse empirical studies we review – all showing large and highly statistically significant crowd-out effects – viewed as “unreliable non-evidence?”Footnote ⁷

Zhao & Chen address the empirical question more substantively, and point to a recent meta-analysis concluding that there is no systematic negative spillover between different environmental behaviors (although also showing weak or non-existent positive spillovers). They cite Thomas, Sautkina, Poortinga, Wolstenholme, and Whitmarsh's (Reference Thomas, Sautkina, Poortinga, Wolstenholme and Whitmarsh2019) study, which found that the English plastic-bag charge changed behavior and increased support for other charges to reduce plastic waste. Of course, as a plastic-bag charge is a paradigm s-frame policy, this demonstrates a positive spillover from s-frame reform to i-frame reform, rather than the opposite.Footnote ⁸ Zhao & Chen note that positive spillovers might be more likely when i-frame interventions target intrinsic motivation. Koppel, Robertson, Doell, Javeed, Rasmussen, Rathje, Vlasceanu, & Van Bavel (Koppel et al.) acknowledge the lack of empirical support for positive spillover, but claim that positive, crowd-in, effects could be obtained if nudges played on social identity, paralleling Walton & Yeager's claim that nudges could have large effects if more accurately targeted.Footnote ⁹

We acknowledge that there are situations in which positive spillovers could, and very likely do, occur – in which being nudged to engage in a certain behavior increases public support for more substantive measures. Newell, Vigouroux, & Greenwell (Newell et al.) provide a persuasive example: Although there is little if any evidence that carbon-footprint calculators reduce personal emissions, they suggest that “knowledge about how our personal actions can collectively make a difference in tackling environmental problems can be a powerful motivator for supporting proenvironmental action.”

In closing this subsection, we note that Hagmann et al. (Reference Hagmann, Ho and Loewenstein2019) did not intend the conclusion of their paper, “Nudging out support for a carbon tax,” to be that nudges always crowd out more substantive policies, but as a response to claims that such effects do not occur (e.g., Sunstein; Thaler). Indeed, as we note in our target article, Hagmann et al.'s final study showed that these effects disappear when participants are informed of the likely small impact of the nudge (green-energy defaults).Footnote ¹⁰ We continue to believe, along with many commentators (e.g., Bright et al.; Hertwig; He & Kang) and, apparently, many companies and their public relations agencies, that the i-frame can provide a potent distraction from s-frame change, unless we are very clear about the modest impacts that i-frame interventions are likely to have.Footnote ¹¹

R5. i-Frame and s-frame interventions: Complements or substitutes?

A number of commentators (e.g., Cherry & Kallbekken; Collier, Harris, Jecks, & Bendtsen; Hagger & Hamilton; Koppel et al.; Madva et al.; Newell et al.; Ruggeri et al.; Sunstein; Zhao & Chen) note that i-frame and s-frame policies should be viewed as complements, not substitutes: That is, that surely both are required. We agree, and regret if we failed to convey this clearly. We did go some distance in that direction in the target article, stating for example, that:

Among the illustrations we provide is the case of a carbon tax (or, nearly equivalently, a cap-and-trade scheme), widely recognized as an essential part of any successful response to climate change. Designing and implementing a carbon tax that is both effective and acceptable to the public will, as we discuss in our paper, require key decisions (some involving i-level issues) that can usefully be informed by behavioral research.

Although i-level and s-level interventions can and should be synergistic, a very real danger arises when i-frame interventions (typically with modest scope and effect sizes) are framed as alternatives to s-frame change. The problem is not primarily that nudges, information provision (for example, calorie labels, labels for recycling, kite-marks for sustainability), boosts (Hertwig), financial education, computational tools for assisting decision making (Johnson & Mrkva) and the like are being oversold by their inventors. The danger is rather that they are being overbought by policy makers hoping that difficult s-frame policy challenges can be avoided by supposed i-frame alternatives.

Indeed, our central theme is that powerful interests opposed to s-frame reform exploit the prospect of i-frame change as a substitute for s-frame change. Thus, as we document, fossil-fuel companies have promoted individual carbon footprints; pension companies frame long-term retirement provision as a matter of personal prudence; food companies focus on individual choice, and especially levels of exercise, as the root cause of the obesity epidemic; companies generating vast quantities of plastic waste sponsor advertising campaigns focusing on individual responsibility for littering; the gun lobby promotes the slogan that “guns don't kill people, people kill people,” and so on. Viewed through the lens of the conflictual analysis of persistent social problems, this is to be expected, as a tactic to reduce pressure for s-frame solutions and to focus instead on i-frame interventions, which are likely to have only marginal (although often worthwhile and positive) impacts.

In the debate over the substitutability versus complementarity of i-frame and s-frame policies, an obvious point gets lost: Researchers have limited time and resources. As Roberto writes, “Resources spent developing, pursuing, and touting relatively ineffective i-frame interventions draw resources away from the development and implementation of more effective s-frame solutions.” “Attentional and physical resources are limited. A researcher spending time investigating or promoting an i-frame solution is not spending that time investigating or promoting an s-frame solution. Funding dollars spent on i-frame research is not spent on s-frame work.” Roberto concludes that “Behavioral scientists who want to develop solutions to the world's biggest problems should focus their efforts on s-frame solutions.” We agree. Similarly, Newell et al. also point to the “scarcity of academic or bureaucratic resources” as reasons for why i-frame and s-frame interventions can act as substitutes rather than complements.

R6. Are i-frame interventions really so ineffective?

Surprisingly few commentators (e.g., Johnson & Mrkva) challenge our conclusions about the (in)effectiveness of nudges.Footnote ¹² We stress that in some policy contexts, i-frame ergonomics, and in particular choice architecture, can be important (e.g., Johnson, Reference Johnson2022). But often better s-frame design is more crucial. Thus, no amount of nudging to help people to save can compensate for a fundamentally flawed pension system (see below). We completely agree with Johnson & Mrkva that policy makers need to be alert to, and regulate against, profitable but welfare-destroying “dark nudges.”

Beyond the studies we cite (e.g., by DellaVigna & Linos), it is useful to consider a more recent analysis (Saccardo et al., Reference Saccardo, Dai, Han, Raja, Vangala and Croymans2022) of two randomized controlled trials (RCTs) of nudges to increase coronavirus disease-2019 (COVID-19) vaccinations (N = 187,134) and 111 nudge RCTs ranging across policy areas involving 22 million people. The paper concludes that “nudges’ estimated efficacy is higher when outcomes are more narrowly (vs. broadly) defined and measured over a shorter (vs. longer) horizon, which can partially explain why nudges evaluated by academics show substantially larger effect sizes than nudges evaluated at scale by a government agency.”Footnote ¹³

Relatedly, and consistent with this theme, Thaler notes that: “the range of interventions studied by behavioral scientists is truncated by what I call permission bias: you can only test what you can get the approval to try. It is wrong to infer from this fact of life that behavioral scientists are using the wrong ‘frame.’ Rather, they face constraints! It also makes it problematic to judge the potential impact of possible behavioral policy interventions based on the set of randomized controlled experiments behavioral scientists have been allowed to run.” It is indeed possible that nudges might be more effective if unconstrained by policy-making realities. But these same constraints inevitably bind not only on what RCTs are possible, but also what policies can be implemented. Moreover, if Saccardo et al.'s (Reference Saccardo, Dai, Han, Raja, Vangala and Croymans2022) findings hold more broadly, we might anticipate that large-scale and longer-term trials, if and when they could be run, would produce disappointingly modest effects. More generally, our argument, echoing others (e.g., Deaton, Reference Deaton2020; Deaton & Cartwright, Reference Deaton and Cartwright2018a, Reference Deaton and Cartwright2018b), is that the focus on RCTs itself is extremely restrictive, and largely excludes s-frame reform. Systemic changes typically affect everyone, and usually cannot be randomly allocated to some people and not others. We discuss this point further in section R8, dealing with research methods.

Walton & Yeager suggest that nudges might be more effective if more accurately targeted. They cite two meta-analyses finding that a specific type of nudge, mindset interventions (which they acknowledge that we don't discuss but, somewhat mysteriously, claim that we nevertheless mischaracterize) has modest overall effects. But they argue that: “One should not expect strong main effects but variable effects in different contexts (for i-frame interventions) … C&L ignore this, claiming that small average effects for nudges indict the field. Yet the question should be: how and under what circumstances can effects be optimized?” Perhaps, but the proof will be in the pudding.

Newell et al. propose that discouragement about small nudge effect sizes arises from excessive expectations. We agree that “behavioural scientists should not overhype the potential impact of i-frame interventions beyond what is justified by their typically modest results.” Indeed, as shown in Hagmann et al. (Reference Hagmann, Ho and Loewenstein2019), this is crucial for encouraging complementarity rather than rivalry between s- and i-frame approaches – because it is critical for policy makers, opinion formers, and the general public to appreciate that, in most policy domains, i-frame interventions are too weak to substitute for s-frame change.

R7. Have behavioral scientists been engaging with the s-frame all along?

Some commentators (Hallsworth; Sunstein; Thaler) stress that behavioral insights have already often been applied to inform systemic change.Footnote ¹⁴ According to this critique, where behavioral insights researchers are in, or are close to, government, they have frequently focused on systemic policy (e.g., laws, taxes, mechanism design). Where this occurs, it is to be applauded and extended.

Perhaps differences in perspective partly reflect a disagreement on what counts as substantive s-frame reform. Focusing on the crucial case of what he terms “saving for retirement” Thaler writes “In four decades of behavioral economic research on this topic, the focus has always been on making the system work better for humans. Isn't that the s-frame?” Not necessarily. Indeed, the very framing of retirement provision as a problem of “saving” reflects an individualist perspective. Retirement is, in most countries with successful pension systems, not a matter of individuals saving (and making investment decisions) for their own retirement; it is often a process that is primarily organized by the state.Footnote ¹⁵

Substantial s-frame reform for pension provision in the United States needs to be far more radical than the innovations presented by auto-enrolment and auto-escalation. The entire defined-contribution retirement system, with its unrealistic assumptions about individual saving, highly regressive tax breaks, requirement that individuals make their own (uninformed) investment choices, ease in permitting withdrawals and borrowing against retirement savings, and ignorance of the realities of life at low or even median income-levels in the United States, is fundamentally flawed.

The benefits of defined-benefit schemes (and other collective schemes) are clear: They pool risks across many workers. By contrast, defined-contribution schemes place the risk regarding life-expectancy and investment performance firmly with the individual. Most people attempting to provide for their retirement under such schemes have little idea what level of savings will be sufficient, or how to draw down the money once they are retired. As it happens, across the population, defined-contribution pensions are also seriously underfunded: Many people with such schemes face severe financial hardship in retirement.Footnote ¹⁶

Thaler claims: “Private sector defined-benefit plans, like typewriters and dial telephones, are obsolete technologies few people pine for.” But the transition has not occurred because of public enthusiasm for defined-contribution schemes; but because they are cheaper for employers and more profitable for pension providers (Hassel, Naczyk, & Wiß, Reference Hassel, Naczyk and Wiß2019). The outcome has been disastrous. The median 401(k) balance in the United States in 2019 for people who have a 401(k) in the age range 55–64 was an utterly inadequate $144,000.Footnote ¹⁷ Worse, 48% of that age group have no 401(k) at all. If defined-benefit schemes were the typewriters of their day, it seems we must now make do with a broken pencil.Footnote ^18, Footnote ¹⁹

The challenge of s-frame pension reform is substantial, but is dwarfed by the technological and social transformations required to tackle problems such as climate change or healthcare reform. This “inconvenient truth” is rarely confronted in behavioral public policy (Roberto, see also Lamberton). For example, Jackson notes how little social science policy research (including behavioral science) has focused on addressing the growing problem of inequality in the United States, United Kingdom, and many other countries. Of course, where there are small-scale “quick wins,” we should embrace them.Footnote ²⁰ But we must not lose sight of the scale of reforms required to bring about really substantive change; nor of the inevitability that such reforms will be vigorously opposed by those benefitting from the status quo.

There is doubtless room for further debate on how usefully and distinctively behavioral science can shape government policy. But this issue is largely independent of our argument that: (1) that a large fraction of the behavioral science community has focused on i-frame interventions to the detriment of exploring routes to sorely needed systemic reform; and (ii) the biggest stumbling block to reform is not lack of insight (behavioral or otherwise) about what to do, but achieving the political momentum to overcome the powerful interests opposing system change. A major role of behavioral science should, therefore, be to help identify how to mobilize popular support to overcome powerful vested interests.

R8. Why is there so little s-frame behavioral public policy research?

A theme that we touch upon, but do not discuss in detail, in our target article is the tendency for the most widely used social science methods to bias policy toward the i-frame. Jackson concurs, arguing that “the social sciences are increasingly ill-equipped to design system-level reforms. Long-standing trends in social science – in particular, increasing specialization, emphasis on causal inference, and the replication crisis – are barriers to system-level policy development.” Strohminger & Táíwò likewise argue that “structural factors bias and perpetuate behavioral science toward the i-frame”; we agree, though we are skeptical of some of their specific proposals (e.g., choosing which research projects to fund by lottery). Gal & Rucker have elsewhere noted that “applied behavioural science tends to overvalue interventions that can be readily tested using experiments,” and argued that such constraints on research “drives the popularity of light interventions and nudges and unnecessarily limits the scope and ambition of the field” (Gal & Rucker, Reference Gal and Rucker2022). Thaler also points out the biasing effect of constraints on research: “An important point to stress is that behavioral scientists, whether they are in academia or nudge units, do not have the authority to experiment with most of the rules and regulations in a given domain. No nudge unit has the ability to say, hey, let's try a carbon tax in half the country and strict emission rules in the other and see how it goes. In practice they are often limited to messaging campaigns, which are less impactful.”

Note, though, that the limits on s-frame research go beyond what Thaler refers to as “permission bias” – that researchers need political permission to test their ideas. Commercial funding for science is very substantial, and, given severe limits on government funding of research, inevitably influences the types of research that gets done. Worse, commercial interests can suppress science. Koerth (Reference Koerth2023) notes that there is little evidence on effective gun controls in the United States because laws banning the use of gun-tracing information for research purposes was enacted by gun-lobby-influenced legislators.Footnote ²¹ In consequence, Koerth notes, a recent major Rand report (Smart et al., Reference Smart, Morral, Ramchand, Charbonneau, Williams, Smucker and Xenakis2023) that adopts strict criteria to evaluate studies on gun control finds just three studies between 1995 and 2020 meet these criteria. But these are, in any case, the wrong standards for s-frame policies, where experimental studies are generally impractical (and well-controlled “natural” experiments are rare); instead, historical and cross-national (and sub-national) comparisons, while imperfect, are hugely informative.

More broadly, s-frame policy innovation is inevitably, as Cherry & Kallbekken observe in their insightful commentary, a process involving intuition and exploration, as is true for systemic changes ranging from restructuring a business, changing a law, to widening the franchise, expanding civil rights, or setting up entirely new institutions. Any experimentation in system-level reform typically occurs in a fairly ad hoc manner, as the new systemic change is trialed and continually adapted to deal with the inevitable stream of hitches and unintended consequences (Hausmann & Rodrik, Reference Hausmann and Rodrik2003; Mulgan, Reference Mulgan2021; Sanger & Levin, Reference Sanger and Levin1992).

A cautious approach to policy innovation relying on experimental, or similar, evidence as a precondition would have ruled out most of the major transformational developments in human history. Consider the transitions toward democracy, revolutions in agriculture, manufacturing, and the invention of information technology (IT) (and their economic and organizational consequences), the modern financial system, the creation of the welfare state, or the international institutions of the United Nations (UN), and many more. All these huge s-frame innovations would have been hopelessly hamstrung if each move forward had to be grounded in a solid basis in RCTs.

Research and its influence on policy can be skewed in other ways, too. Scientists, ourselves included, are naturally driven by curiosity. But what is most interesting is not always what is most important. Roberto writes:

Without abandoning curiosity as an important and legitimate criterion for the selection of research projects, a “nudge” for researchers in the direction of policy impact might be beneficial.

R9. Final thoughts: Behavioral science and the struggle against special interests

We agree that behavioral insights are key to enacting and implementing successful policy reform (e.g., Hallsworth; Sunstein; Thaler). A key, but under-appreciated role that the behavioral sciences can play is in winning the “political battle” against special interests (which are not seriously attempting to engage with the truth or find policy solutions to maximize human welfare). Radical systemic change often comes from the bottom-up, as well as from top-down. Understanding which policies gather popular support (Fitzgerald, Lamberton, & Walsh, Reference Fitzgerald, Lamberton and Walsh2016) and how to design policies (e.g., carbon taxes, health care, or pension reform) to maximize that support are key challenges.

Major societal problems require, we believe, major systemic change. Early social and economic theorists as varied as Thomas Hobbes, Jean-Jacques Rousseau, Adam Smith, David Ricardo, Auguste Compte, and Karl Marx, focused at least as much on changing the social world as on understanding it (Mulgan, Reference Mulgan2021). But major s-level change, even where it is widely agreed to promote human welfare overall, will typically be bitterly opposed by those benefitting from the status quo. The challenge in formulating, and building support for, reforms that address the fundamental challenges that face our societies is formidable, but more than worth confronting. All available tools must be deployed, and where i-frame solutions can contribute, behavioral scientists should pursue these enthusiastically. But to really make a difference, behavioral public policy needs to refocus its insights and energy on s-frame reforms: Almost always, deep policy problems require us not just to nudge the players, but to change the game.