We are grateful to the many authors for providing such thoughtful and stimulating responses to our article. We only have space to respond to a few points here, but we look forward to continuing these discussions in print and in person.

First, we would like to acknowledge the many instances where respondents raise valuable points that we failed to make (or did not treat in enough depth). We agree with the need for better cost–benefit analysis and more consideration of well-being (O'Donnell; Delaney). We agree that there is much that academic, public and private sectors can learn from each other – if findings are shared (Caldwell). We agree that we should refer more to collective notions of identity and agency – for example, “solidarity, community, compassion,” as Lepenies et al. put it. We see this as similar to van der Linden's point about making nudges more “socially situated.” Part of the neglect stems from the emphasis that economics puts on individual actors and nudging's focus on achieving marginal shifts in large populations rather than looking in more depth at the plurality of group values and preferences.

We welcome the call from Delaney to reflect more on “the longer intellectual foundations of behavioural science.” It is worth saying that we are aware of the range of thinkers in this field, including the many contributions of Herbert Simon. We hope to demonstrate this variety in our forthcoming work on behavioural government, given Simon's seminal contributions to the study of administrative behaviour. We acknowledge the complexities underlying the apparent simplicity of ‘nudge for good’, as noted by Spencer and Lepenies et al. We are aware that government action to achieve ‘the good’ is the subject of a long and illustrious history of debate, which should feed into contemporary discussions about paternalism more generally (not just in terms of behavioural policy). As we have written previously (Hallsworth & Sanders, Reference Hallsworth, Sanders and Spotswood2016), we also reject any crude attempt by policy-makers to categorise behaviours as ‘undesirable, irrational or unwise’ simply because they do not align with their personal preferences. The work on ‘ecological rationality’ is particularly valuable here (Todd & Gigerenzer, Reference Todd and Gigerenzer2012).

We value Gauri's perspective on applying behavioural science in developing countries. We agree that there is a need to help smaller organisations to innovate and evaluate in contexts where randomised controlled trials (RCTs) are not possible, for reasons of logistics or small sample sizes. We also realise that our discussion of cultural variation seems to implicitly position developing countries as venues for replicating existing findings. As Gauri suggests, the process of developing appropriate interventions is more dynamic than that and may involve adapting the intervention on an ongoing basis in response to specific challenges. However, we also want to clarify one point Gauri makes. The fact that tax letters with a ‘hard’ tone outperformed those with social norms in Poland is actually in line with the original studies he mentions. These did not claim that social norms would necessarily outperform enforcement action, but rather that social norms may produce comparable results with much lower costs and less harm to the taxpayer–government relationship.

We agree that more work is needed on the practical ethics of behavioural policy. Delaney's call for the development of applied ethical frameworks and professional standards is persuasive. We are aware that the Behavioural Insights Team (BIT) has responsibilities as a first mover in this field and we will explore ways of putting the thinking we have done on ethics in a useful format. We interpret Bovens’ question about charitable donations as: is it ethically problematic to ask a chief executive officer (CEO) to ask his/her employees to contribute to charity, since this may be exploiting existing power structures in a problematic way? We may conclude that the question rests on points of detail: how visible is evidence of having contributed or not (which would seem to neutralise Bovens’ concerns about ‘invasion of privacy’)? Does the CEO have a history of coercive and threatening behaviour? Many possible perspectives are available for interpreting these details, which means that practitioners would benefit from a useable guide (or checklist) that helps them navigate these options.

We agree with Hansen that there is a need to be careful that ‘quick wins’ do not prevent behavioural insights becoming more deeply integrated into public policy. As we highlight in our paper, we believe ‘thorny’ problems are a big opportunity for behavioural insights. However, Hansen raises two points that we want to reflect on in more depth. The first is that our guiding principles have led us to make only technocratic changes to policy implementation. The reality is that we have been fortunate enough to influence the development of many different policies, thanks to our origin within the UK government. Examples include mental health, unemployment, obesity and domestic violence. The complex and indirect nature of policy-making means that this work is not as easy to document as the technocratic tweaks and therefore is often not as prominent.

Second, we agree that an in-depth understanding of context and behaviours is crucial to developing successful solutions and interventions, but we disagree that practitioners are not doing so already (see Holden et al., Reference Holden, Egan, Snijders and Service2017). Our own methodology, TEST (Target, Explore, Solution, Trial), emphasises the need to conduct wide-ranging exploratory fieldwork. Although TEST has not been formally published, we will soon publish an extended guide on the qualitative dimension of behavioural insights projects. We welcome Hansen's efforts to develop better diagnostic tools and look forward to seeing BASIC and others in action. It is worth noting, though, that a lot of his recommendations rest on a rather technocratic and rationalistic concept of policy-making (proceeding in stages, matching solutions to problems). There is much evidence that policy-making in the real world does not work this way (Lindblom, Reference Lindblom1959; Hallsworth et al., Reference Hallsworth, Rutter and Parker2011), and therefore we need frameworks that are capable of being adapted and used opportunistically.

Finally, we welcome the discussion of reactance and public consultation from de Jonge et al. We agree that this is an important discussion, and BIT has previously pushed for greater use of deliberative forums in the policy-making process (Halpern, Reference Halpern2015). Indeed, we helped to run one for the Victorian government in Australia recently, which helped to change the direction of the ensuing policy work.Footnote ¹ We have seen occasional evidence of reactance from our own interventions – mostly notably when a stock image seemed to create a strong backfire effect on organ donation rates. But it is also worth noting that policies may cause initial reactance before becoming accepted and effective in the long run – for example, hoarding of plastic bags before a 5p charge was introduced, which later proved to be a popular and successful policy.

We are in favour of creating more accurate beliefs about persuasion knowledge, as de Jonge et al. suggest. But we also want to note an important fact that they neglect. In most political systems, we explicitly assign the role of anticipating reactance to politicians (and, to a certain extent, communications professionals), who form a crucial part of the policy-making process. These are the people who often have the skill to anticipate what features of a policy may trigger reactance – what will ‘play badly’ with the public. The fact that policy-making is not purely technocratic may reassure Sutherland as well, since ministers and communications professionals often bring the marketing perspective that he laments economists lack. One question for de Jonge et al. is: what does the persuasion knowledge model add to the current practices of these actors?

There are some points we find more problematic or less persuasive. First, we think our view on the strengths and limitations of RCT is broadly fair (for a new discussion, see Hallsworth, Reference Hallsworth and Lewis2018). We think that there is a danger of a ‘focusing illusion’, whereby commentators interested in RCTs end up thinking that they are more common than they actually are. They are still not common in the UK government, at least. While RCTs may not be suitable in all instances, we hardly think they are being overused. Part of the reason for this is Caldwell's point that some government actors may want to find evidence to support their existing views – we argue that this is more difficult to do with results from a RCT. We also do not agree with Gauri that the focus on RCTs has been at the expense of moving earlier in the cycle into policy design. At least in our experience, RCTs have helped make the case for behavioural science to be ‘at the policy table’ and have provided data to feed into the policy design. We have found RCTs to be a path to policy impact, rather than a technocratic dead end.

We really value the fact that Maynard and Munafo provide a practical proposal to increase peer review and transparency – we appreciate that they have made the effort to make their recommendations concrete and we will be looking at the scheme they suggest. We wish to note that we are not set up as an academic institution and, as noted in our comments on ‘reverse impact’, we are rarely funded to go through the peer review process. Therefore, alternative options are very welcome.

However, Munafo and Maynard's argument that we could recruit PhD students to write up papers for us assumes that the studies are in fact publishable in journals that meet the career goals of most academics. Some BIT projects test important theoretical questions, but many will be light on theory – which makes them more difficult to publish in prestigious journals. This fact also means that there is less danger that we are contributing to “the draining of valuable resources into research questions that have already been answered.”

In addition, it is worth considering who the peers are who should peer review our work. Our analysis is subject both to our standard quality assurance procedures and to scepticism and review by the government organisations with which we partner. In terms of publication bias, it should be noted, as Maynard and Munafo do in passing, that BIT is often dependent on public sector clients to grant permission for publication. We always push for publication and (not mentioned by Maynard and Munafo) we do feature many null results in the array of trials presented in our annual updates.

We were not convinced by some of the re-categorisations of our article attempted by Spencer. In our view, categorising the challenges as being about ‘impact evaluation’ unduly skews things towards questions of measurement. We are more concerned with whether, for example, effects deteriorate with repeated exposure, rather than how these effects are measured per se. We also do not see the opportunities we highlight as being about ‘scope and scale’ – behavioural government, for example, is about taking on a qualitatively different kind of challenge. We also do not think that it makes much sense to describe all the opportunities as thorny problems. For example, sometimes the barrier to scaling is not a deficiency of knowledge or planning, but of will and resourcing. We also do not think that it makes sense to upgrade the description of them to ‘wicked’ problems, for the reasons just stated – in fact, we consciously stepped back from employing this often overused term. We also do not say anywhere that standard economic tools should be discarded, nor do we introduce a distinction between ‘behavioural’ and ‘traditional’ tools.

This distinction between ‘behavioural’ and ‘traditional’ brings us on to our final comments, which concern some of the curious statements in Lepenies et al.’s paper. First, we want to deal with the statement that, “In essence, there is a tendency to claim an intervention to be a ‘behavioural’ policy tool in virtue of the fact that it has been tested via RCT.” We are not aware of anyone who is making this claim, perhaps because it would be a very strange thing to say. Of course, we are happy to revise this view if any evidence is presented to the contrary.

What we found particularly interesting was the underlying, unacknowledged binary opposition driving the first half of the article. The authors seem very troubled by a concern that we are blurring the lines between ‘behavioural’ and ‘traditional’ policy tools. We are “overly permissive” in what counts as a behavioural intervention – instead, it is “good and proper” to create “clear demarcations,” a limit beyond which there is a protected space where behavioural science “has nothing to say about how to form good policy.” The language of transgression and contamination is quite striking, and brings to mind Douglas's (Reference Douglas1966) observations on ‘boundary maintenance’. We note the strong injunctive and normative tone of some passages:

Let us engage on the substance here. The last statement refers to the sugared drink tax introduced in the UK. We think there is a strong case that this is not “standard” or “old-fashioned” regulation, since details of the tax's design – namely, the tiered structure – have been constructed to ensure that the vast majority of producers have a salient and proximal incentive to reformulate. This is not the case with most “standard” soft drinks taxes, which are often at a flat rate and are applied on the basis of volume.

The first statement refers to our suggestion that a behavioural approach to policy may indicate that the best policy option is not to change behaviour, but to mitigate its consequences. We pointed towards the case where a policy-maker tries to reformulate food, rather than persuade people to make healthier choices. First, it is worth noting how strange it is to say that command-and-control regulation is not a “behavioural stimulus,” since it clearly attempts to stimulate certain behaviours by food producers. But it is not a stimulus to change consumer behaviour – and that is our point. This is why it is so puzzling that Lepenies et al. say that “[this regulation] ought not to be construed or presented as a behavioural intervention. It is not seeking to change the behaviour in question, but rather its corollaries.” In our view, it is clear that the policy-maker decided that the most relevant “behaviour in question” – the one that was most effective or cost-effective to change – was that of food producers, not consumers. Regulation was the result. We are not sure why behavioural science must be erased from the picture once “traditional” regulation enters.

The authors dismiss a central role for behavioural science in policy as follows: “While this may be true in a trivial sense – for instance, we might say all regulation is about behaviour in the sense that it aims to alter the behaviour of some actor(s) or other – the breadth of such a framing is difficult to legitimise.”

Why is this the case? Assertions about triviality and legitimacy are not enough. There are two dangers that we can identify from the article. One is “[t]he danger is that behavioural policy is mistakenly identified with ‘whatever behavioural policy units do’.” From our perspective, we consider that a non-trivial amount of what we do is not directly inspired by the behavioural sciences and strongly resist the idea that behavioural policy units have a monopoly on behavioural public policy. We are happy to make this clearer.

However, there appears to be a worse problem: namely, that giving power to such units is dangerous for evidence-based policy because they are “also political actors.” Again, we can see an implicit opposition between the ‘behavioural’ and the ‘traditional’, with the former to be treated as an object of suspicion. We are not sure who are the wholly disinterested actors that should be trusted with the task of evidence-based policy instead, given that politics is an essential part of the policy-making process in a democracy.

Finally, if we say that a clear demarcation between the ‘behavioural’ and the ‘traditional’ is desirable, how do we do it? Should the demarcation be on the basis of policy instrument (i.e. “regulation is not behavioural”)? Or should it be done on the basis of policy topic, which is what is implied by the criticism that “substantial swaths of policy” are inappropriately categorised? Or both? Without knowing this, it is hard to respond to the authors’ call to limit our claims on behalf of the behavioural sciences.

Once again, we would like to thank the authors for the considerable effort they have taken to respond here. There are many points that we will take on board and that we hope will be useful for the field in general.