Identifying relevant research

In order to identify relevant research that helps answer my research question, I follow a set of pre-defined inclusion criteria. This is summarised in Table 1.

Table 1. Inclusion criteria

The literature search took place in December 2021 and covered websites of financial regulators, competition authorities and nudge units, websites of international organisations, a number of databases and search engines. In addition, I reviewed all editions of five relevant journals between 2015 and 2020. Finally, I reviewed the bibliographies of all the selected papers. This search yielded 35 papers that met the inclusion criteria. Details of the literature search and the list of studies included in the review are shown in the Supplementary Material Appendix.

Data extraction

I record data about the relevant studies at two levels. First, I select the characteristics that do not change within a paper and record these for all the 35 studies. These include study design, geographic area, population, involvement of an authority and if there was one, the policy stage at which they carried out the study.

Second, I record as a separate observation each estimated impact from the quantitative studies for all interventions that met the definition of nudge. Most papers report their results as a percentage point change, and therefore, I focus on these measures. The final dataset contains 797 observations on 102 nudges from the 26 quantitative papers. However, only 476 of these estimates are comparable percentage point estimates. These belong to 89 different nudges from 19 papers. Note that over 40% of these observations come from three papers (Adams et al., Reference Adams, Baker, Hunt, Kelly and Nava2015; Oxera-CESS, 2016; Charles et al., Reference Charles, Gaglani, Gilchrist, Goodier, Harrington-Clark, Sansoye, Scott and Vass2019). Further details of the data extraction process and the number of estimates per paper are available in the Supplementary Material Appendix.

For each estimate, I record 54 variables. These include details of the design (e.g. type of study, intervention and product), the outcome measures used (e.g. search or switching, self-reported or not) and the estimation (e.g. specification, sample size, standard error).

Analysis methods

I treat the qualitative and the quantitative studies separately throughout the analysis. I first review the qualitative papers and draw out the most important/common themes. For the quantitative papers, I perform two types of analysis on the extracted dataset. First, I calculate the average impact of interventions overall, by study design, by product, by type of nudge and outcome measure, and the corresponding pooled standard errors and confidence intervals. Second, as a cross-check on the previous analysis, I run univariate and multivariate OLS regressions with dummies included for study design, product, type of nudge and outcome measure, using robust standard errors clustered by the paper. Note, however, that variation across the different dimensions is often limited and as such, does not allow us to fully isolate the impact of these features.

Study characteristics

Table 2 shows a morphological box of the 35 studies included.

Table 2. Morphological box of the studies included

Abbreviations: FCA - Financial Conduct Authority; EC - European Commission; CMA - Competition & Markets Authority.

¹ One study runs the same experiment using a population of product users and a population of students.

In terms of study design, the 35 papers contain qualitative analyses, laboratory and field experiments and ex post data analyses. All but two laboratory experiments are online; respondents did not have to show up in person in a laboratory. The field experiments are all randomised controlled trials (RCTs) but, in some cases, participation was voluntary and/or the outcomes were measured through a survey. Finally, the ex post analyses include two evaluations and two studies that took existing datasets and used them to analyse the impact of some change that happened (without specifically designing the intervention or the data generation for the purposes of the analysis).

The majority of the studies originate from the UK but there are a few from the US, the EU (studies that cover several countries with coordination at the EU level), Ireland, Germany and Mexico. The reason why the UK has the most studies is because its regulators and governing bodies have been at the forefront of applying behavioural research in competition analyses that often assess search and switching. The Financial Conduct Authority (FCA) issued 15 publications that passed all the inclusion criteria, but there are also studies from the Competition & Markets Authority (CMA), its predecessor, the Office of Fair Trading (OFT), from Pension Wise and The Money & Pensions Service. Other authorities that carried out relevant research are the European Commission (EC), the Bureau of Consumer Financial Protection (BCFP) in the US, the Central Bank of Ireland and the Competition and Consumer Protection Commission (CCPC) in Ireland and the Mexican Banking Commission (CNBV). The papers with no involvement of any authority are mostly academic papers from the US.

More than half of the papers where an authority was involved carried out the research with the purposes of testing possible remedies for already identified problems. All but one of these studies belong to the FCA or the CMA. Others used research to explore issues and solutions but without having done a full analysis of market failures. As already mentioned, only two papers evaluated the impact of an intervention that had been put in place.

Over two-thirds of the studies drew samples from users of the product in question (in one case, potential users), with some restricting their sample to certain groups of consumers (e.g. those nearing retirement for pensions or those close to automatic reenrolment for insurance). One paper run experiments with both users and students, and the rest relied on a nationally representative sample or grown-up population.

Nudge characteristics

Table 3 shows a morphological box of the characteristics of the 102 nudges covered in the 26 quantitative papers included in the review. These nudges were implemented in a number of different retail financial markets. Insurance includes add-on, car rental, home, contents, health, prescription drug, motor and pet insurance, and also extended warranties.

Table 3. Morphological box of nudges in the quantitative papers

The impact of more than half of the nudges was measured on switching metrics, about a quarter on search metrics and less than quarter on both.

Table 3 also splits the nudges into two categories: informational and structural. Informational nudges are those that provide, simplify or highlight information, including reminders and disclosures. In contrast, structural nudges change the choice architecture more profoundly. Examples include (i) sending a letter with a tear-off return switching form pre-filled for a switch to the best internal rate and a prepaid, addressed envelope (field experiment, Adams et al., Reference Adams, Hunt, Palmer and Zaliauskas2021); (ii) offering to book an appointment with an independent government-operated advisor (Pension Wise) when customers call their pension provider (field experiment, Farghly et al., Reference Farghly, Milward, Holt, Solomon and Sandbrook2020); or (iii) introducing add-on insurance upfront vs only at the point-of-sale (online laboratory experiment, Duke et al., Reference Duke, Huck, Lewis, Sinn, Wallace and Ellison2014). These nudges do not only reduce the cognitive load of searching and switching, or simply draw the customer's attention to certain relevant information but also change the task the customer faces. The Supplementary Material Appendix contains further details and examples of each category.

Note that the majority of nudges are classified as informational. Disclosures, reminders and simplifications can be treated separately from the informational category. However, these only account for a small number of informational nudges and using them as separate categories does not significantly change the overall results. Thus, in this analysis, I treat all informational nudges as a single category.

Qualitative studies

All the nine qualitative studies are from the UK or Ireland and all of them are commissioned by regulators. They test interventions in cash savings, current accounts, mortgages and payday loans through interviews, focus groups and surveys. They mostly cover informational nudges. One exception is a survey which also explores consumers’ views on a default intervention: being automatically booked into an appointment about switching before the initial fixed rate expires on a mortgage (Savanta ComRes, 2020). Note, however, that according to a narrow definition (DellaVigna & Linos, Reference DellaVigna and Linos2022), even this intervention would not qualify as default as it does not change the outcome automatically if an individual remains passive. A broader definition encompasses all interventions that consist of pre-selecting one choice option to increase the likelihood of its uptake (Jachimowicz et al., Reference Jachimowicz, Duncan, Weber and Johnson2019). No other research (including the quantitative studies) tested any form of default intervention, which is somewhat surprising, given the popularity of defaults in other policy areas. This lack of default interventions could be due to the fact that the outcome measure is a deliberate act of the consumer (shopping around or switching), which she may or may not decide to do. This contrasts with a choice between different options (e.g. retirement savings plans) in which case defaults can be more easily applied while the consumer remains passive.

The primary purpose of these qualitative studies is to explore consumers’ reactions to a nudge and to identify features that are more likely to make them work. Overall, they suggest that communications need to be clear and standardised, include a graphical representation and personalised information on the (financial) benefits of search and switching, as well as information about the process itself. Consumers are in general of the view that there is little to gain by shopping around and switching for financial products and they consider the process to be cumbersome. As a result, nudges that highlight potential savings for that particular consumer (rather than in general) and help with the process receive the most favourable feedback in these studies. The review of these papers reveals a number of lessons for the practical implementation of nudges.

First, it is difficult to find a channel that can grab consumers’ attention. Consumers view pop-ups as spam (Archer et al., Reference Archer, Fu, Ohta and Thomas2014), question the authenticity of text messages (Collaborate Research, 2017), miss prompts that are embedded into annual statements (Optimisa Research, 2016) and rarely read standalone letters (Collaborate Research, 2017). Online and mobile app notifications were suggested in a couple of interviews (Optimisa Research, 2016; Collaborate Research, 2017) but there is less past experience with these and it needs to be explored whether they would indeed work in practice.

Second, while most studies find that new communications work best when they arrive from the consumers’ own provider, they also find that providers telling their customers to switch away causes confusion (Worton & Reynolds, Reference Worton and Reynolds2015; Worton et al., Reference Worton, Jones and Light2016; Collaborate Research, 2017). This suggests that nudging consumers to switch products within provider is more likely to work than nudging them to switch away to another provider. More internal switching could help the problem of price discrimination between engaged and disengaged consumers, but it is less effective in increasing the competitive pressure on firms. In fact, theoretical modelling suggests that the imposition of measures encouraging internal switching could even be detrimental to consumers (Beckert & Siciliani, Reference Beckert and Siciliani2021).

Finally, while in almost all studies a large proportion (20–60%) of respondents indicate that a nudge would encourage them to search or switch, it appears that any prompt is more likely to work for those who are already considering switching (CBI, 2017; Savanta ComRes, 2020) and will not change the behaviour of those who are otherwise reticent to switch (Collaborate Research, 2017). This is of concern as in the presence of price discrimination, increasing the engagement among already engaged consumers may not affect or even increase prices the less engaged consumers face (Fletcher, Reference Fletcher2021).

Quantitative review

The findings below are based on the 19 papers with comparable quantitative estimates using the meta-analysis and regression analysis. Detailed results are available upon request.

The average impact of nudge interventions that aim to increase consumer search and switching is between 4 and 6 percentage points. This varies slightly depending on whether the estimates are weighed using the inverse of the number of estimates by paper or by nudge, and also whether less reliable estimates (such as those that come from non-causal analyses or use self-reported outcome measures) are included or not.

However, the overall average is likely to overestimate the real impact of nudges on search and switching. When looking at the results by study design, I find that laboratory experiments show a four times higher impact than field experiments, which are in turn higher than the results of ex post analyses once less reliable estimates are excluded. In particular, the estimated average increase in search and switching is between 7 and 12 percentage points in the laboratory, 2–3 percentage points in the field and basically zero in ex post analyses. This is shown in Figure 1.

Figure 1. Average impact of nudge intervention by study design.

The regression analysis also shows that the coefficient of the laboratory experiment dummy is 9 percentage points with and without controlling for the type of nudge, the outcome measure and product (i.e. laboratory experiments estimate a 9 percentage point higher impact than field experiments). These coefficients are statistically significant and robust to excluding less reliable estimates.

The difference in the estimated impact in laboratory and field experiments is likely to reflect the difficulties of replicating a real-world situation in a laboratory environment. There is a criticism in the literature that results of laboratory experiments cannot easily be extrapolated to the real world, including for the following reasons:

• • Selection effect: those who volunteer to take part in an experiment are likely to be different from those who do not and non-random selection of participants can bias results (Levitt & List, Reference Levitt and List2007b);

• • Information provision: in the laboratory participants make decisions based on a complete description of rules but in reality social interactions and past experience can lead to very different patterns of behaviour (Erev & Greiner, Reference Erev, Greiner, Fréchette and Schotter2015);

• • Level of abstraction: high level of abstraction in the laboratory allows for tight control, which helps establish causal effects (Charness, Reference Charness, Fréchette and Schotter2015), but an abstraction may interact with the treatment and as such could alter the findings (e.g. including vs excluding brand names when testing an intervention; Nieboer, Reference Nieboer2020);

• • Time commitment: many economic decisions take more time to reach than the typical time limits in a laboratory session (Reiley, Reference Reiley, Fréchette and Schotter2015);

• • Decision-making process: certain elements are difficult to replicate in the laboratory, such as inertia (participants of a laboratory experiment are there to make a decision so this important barrier to acting on information is less pronounced than in the real world; Iscenko et al., Reference Iscenko, Duke, Huck and Wallace2014), or heuristics, which evolved in familiar field settings and may not manifest themselves in the laboratory (Harrison & List, Reference Harrison and List2004);

• • Incentives: the stakes of the decisions and the costs of effort in the laboratory are different from real-world situations and will influence how players behave in the laboratory (Levitt & List, Reference Levitt and List2007a).

These concerns are relevant for the experiments at hand. The sample is usually drawn from large panels of market research companies and those who subscribe to these may have more time than those who do not, the latter group being less likely to have time to search for the best deals in reality. Some of the elements of consumer decision making that are difficult to replicate in the laboratory (such as inertia or brand loyalty) are present in retail financial markets. In addition, any metric of shopping around or switching inevitably requires less time and effort in a laboratory environment than what it takes to actually shop around for financial products or to switch between providers. It is thus likely that the difference in average impact that we observe is simply a demonstration of the above-mentioned criticisms.

Note, however, that the criticism only concerns the quantitative results – the external validity of qualitative findings of laboratory experiments is generally not in doubt (Levitt & List, Reference Levitt and List2007a; Iscenko et al., Reference Iscenko, Duke, Huck and Wallace2014; Charness, Reference Charness, Fréchette and Schotter2015; Gneezy & Rey-Biel, Reference Gneezy, Rey-Biel, Fréchette and Schotter2015). This is in line with how some of the laboratory experiments included in this review position their results: their main findings relate to which intervention had the largest impact (e.g. Burke et al., Reference Burke, Hayes, Ng and Smart2020) and some specifically argue that the key outcome is the ranking of the different treatments, not necessarily the magnitudes of differences between them (Oxera-CESS, 2016). Unfortunately, these interventions have not been tested in the field and as a result, it is not possible to say whether the ranking obtained from laboratory experiments is confirmed in the field.

Regarding the difference between field experiments and ex post analyses, a review of the studies in the latter group reveals that even the few that are available suffer from methodological issues, such as not being able to establish causality between the intervention and the observed changes. Given this, the results that ex post analyses show a smaller impact than field experiments using only reliable (causal) estimates is not necessarily robust. However, it is plausible that ex post analyses show smaller effects than field experiments because the former incorporates second-round effects that the latter is unable to take into account. For instance, field experiments are usually implemented in close cooperation with some selected firms, whereas a market-wide requirement does not tend to be so closely supervised. If not all firms comply with the requirements fully, estimated effects will be smaller (see, e.g., Charles et al., Reference Charles, Gaglani, Gilchrist, Goodier, Harrington-Clark, Sansoye, Scott and Vass2019, who show that the estimated impact is higher assuming full rather than actual compliance level). Similarly, ex post analyses may find a different effect because they incorporate the impact of the suppliers’ response on the search and switching metrics. For example, the FCA required insurers to show last year's premium in the annual renewal letter, and as a result, insurers did not increase their premium as much as they would have without the intervention (Charles et al., Reference Charles, Gaglani, Gilchrist, Goodier, Harrington-Clark, Sansoye, Scott and Vass2019). This may have contributed to the limited increase in switching rates (albeit still achieved the desired outcome by putting pressure on firms’ pricing).

Thus, while ex post analyses could in theory provide the most comprehensive assessment of the impact of nudge interventions, given the lack of these studies and their methodological issues, I consider that the average impact obtained in field experiments (2–3 percentage points) is likely to be the currently most reliable estimate of the impact of nudge interventions on search and switching.

When looking at different types of nudges, I find that the difference between the average impact of the two categories is 13–15 percentage points with informational nudges averaging around 2–4 percentage points and structural nudges having an average impact of 17 percentage points. This is confirmed in the regression analysis that controls for study design, product and outcome measure (search vs switching). Note, however, that several nudges in the structural category were tested in a laboratory environment so we need to treat the estimated difference with caution. Looking at field experiments only, regression analysis suggests that structural nudges still have a 6–8 percentage point higher impact than informational nudges. The results are highly significant, although there are many fewer data points for structural nudges. In addition, out of the four studies that tested both types of nudges, three (Suter et al., Reference Suter, Duke, Harms, Joshi, Rzepecka, Lechardoy, Hausemer, Wilhelm, Dekeulenaer and Lucica2017; Burke et al., Reference Burke, Hayes, Ng and Smart2020; Adams et al., Reference Adams, Hunt, Palmer and Zaliauskas2021) clearly found larger effects of the structural intervention. The fourth study (Hunt et al., Reference Hunt, Kelly and Garavito2015) does not provide causal analysis for the structural nudge; it is thus unclear how the estimated result compares with the impact of the tested informational nudges. Overall, we can still conclude that nudges that change the choice architecture more profoundly have a significantly higher impact on search and switching than nudges that only provide, simplify or highlight information.

There is no clear evidence that nudge interventions aiming to increase consumer search or switching would work significantly better for certain products than for others. Only interventions in cash savings appear to have a robust impact of 3–6 percentage points based on the regression analysis, but it must be noted that a large part of this is internal switching, that is, when the consumer moves to a different product with the same provider. Internal switching does not bring the same benefits for competition as when consumers switch between different providers, and could even be detrimental (Beckert & Siciliani, Reference Beckert and Siciliani2021).

There is only weak evidence that the impact of interventions varies by consumer groups. Nine studies investigated heterogeneity in the results, including splitting consumers by age, gender, education level, income or by how much they could gain by switching. One clear finding is that including last year's premium next to the new premium offered in insurance renewal letters is more effective when consumers face a larger price increase relative to a previous price they paid (Adams et al., Reference Adams, Baker, Hunt, Kelly and Nava2015). The rest of the significant results do not appear to be robust or consistent across studies, and may indeed just be random findings.

In terms of outcome measures, there is an indication that it is easier to nudge people to shop around than to switch. Simple means of estimated impacts are 4–7 percentage points higher for outcome measures of search than for outcome measures of switching. However, field experiments measure the impact of nudges more often on switching, whereas laboratory experiments tend to use outcome measures of search. Indeed, the difference between the average impact on search and switching measures becomes insignificant if I control for products and study design in the regression analysis. When looking at field experiments only, though, I estimate that there is a significant 3–4 percentage point higher impact on search than on switching.

Out of the 26 papers with quantitative analysis, 7 did not include a comparable percentage point estimate and therefore were not part of the calculation of average impacts above. Furthermore, in addition to their main analysis, one study (Seira et al., Reference Seira, Elizondo and Laguna-Müggenburg2017) included a description of an additional ex post analysis in the Supplementary Material Appendix, which again did not contain a percentage point estimate. Four of these papers found no real impact of the tested interventions. Another three used an absolute number as their outcome measure, such as the number of mortgage lenders contacted (BCFP, 2018) and the number of quotes looked at in currency transfer services (BIT, 2018; Burke et al., Reference Burke, Hayes, Ng and Smart2020). The estimated change in relative terms varies between 5% and 28%, but the absolute changes are small in all three cases: an increase from 1.6 to 2.0, from 1.8 to 2.1 and from 2.8 to 2.9. The remaining one paper (Timmons et al., Reference Timmons, Barjaková, Mcelvaney and Lunn2019) estimated the impact of a detailed guidance on consumers’ willingness to switch. They concluded that after reading the guidance respondents self-assessed confidence increased and participants who felt more competent were more willing to switch. However, this is a subjective outcome measure that describes future intentions, rather than past actions and as such, cannot be expected to reliably estimate the quantitative impact of an intervention. Overall, I conclude that the results in these papers do not change the picture drawn from the quantitative analysis.

A final issue to consider is publication bias, which may lead to the results of a meta-analysis attempting to estimate the average impact being biased upwards. As documented in a previous meta-study, the difference between the average impact of nudge interventions in academic publications and in a comprehensive set of studies by nudge units suggests the presence of publication bias in academia (DellaVigna & Linos, Reference DellaVigna and Linos2022). While this may be an issue in general, I believe that publication bias for this review is less of a concern, for the following reasons.

First, there are five papers in the dataset that are ‘purely’ academic, that is, were only published in scientific journals without any involvement of authorities. Three of these (Bhattacharya et al., Reference Bhattacharya, Hackethal, Kaesler, Loos and Meyer2012; Keys et al., Reference Keys, Pope and Pope2016; Johnson et al., Reference Johnson, Meier and Toubia2019) find no impact of the interventions tested and one of them (Marzilli Ericson et al., Reference Marzilli Ericson, Kingsdale, Layton and Sacarny2017) finds a 6 percentage point increase in shopping around but no impact on switching. Only the remaining one purely academic study finds a significant, almost 10 percentage point impact on switching (Kling et al., Reference Kling, Mullainathan, Shafir, Vermeulen and Wrobel2012). It is, therefore, unlikely that the results of academic publications are heavily biased upwards.

Second, as far as policy research is concerned, about half of the quantitative studies with involvement of an authority are prepared for or by the FCA and the FCA claims to publish the results of all experimental trials it carries out (Smart, Reference Smart2016). Again, while it is possible that some relevant studies could not be included in the review as they were not published, the indication is that the impact of that is limited.

Finally, even if there is undetected publication bias, it would only strengthen the conclusion that nudge interventions have a limited impact on the proportion of consumers who shop around or switch between products.

Thus my overall conclusion from the quantitative review remains that nudge interventions on average increase consumer search and switching by 2–3 percentage points in retail financial markets. Structural nudges appear to be more effective than informational nudges (Q1), but there is no clear evidence that nudges would work better for some products (Q2) or for any consumer groups (Q3). The review also reveals that different study designs lead to significantly different estimates and that laboratory experiments are likely to overestimate the real impact of interventions. Ex post evaluations and specifically designing interventions so that their causal impact can be measured could help further evidence accumulation.