Introduction
Encouraging pervasive sustainable behaviour change, beyond mere intentions, remains one of the most pressing challenges for public policy. Previous approaches have heavily relied on incentives and appeals targeting people's extrinsic motivation including economic incentives directly rewarding sustainable behaviour or more abstract rewards such as social recognition. However, extrinsically motivated interventions have often failed to achieve long-lasting behaviour change (Frey and Rogers, Reference Frey and Rogers2014; Kaiser et al., Reference Kaiser, Henn and Marschke2020; Gravert and Olsson, Reference Gravert and Olsson2021). It has therefore been argued that for pro-environmental behaviour (PEB) to be sustained in the long run, it needs to be internalised and thus motivated by intrinsic factors (Steinhorst and Klöckner, Reference Steinhorst and Klöckner2017). Therefore, appealing to people's intrinsic motivation may be a promising strategy to promote long-term sustainable actions (Taufik et al., Reference Taufik, Bolderdijk and Steg2015; van der Linden, Reference van der Linden2015; Steg, Reference Steg2016; Van Der Linden, Reference Van Der Linden2018). This is in line with research showing that messaging appealing to the intrinsic motivational basis of PEB can be more effective than messages appealing to the monetary gains from PEB (Bolderdijk et al., Reference Bolderdijk, Steg, Geller, Lehman and Postmes2013; Asensio and Delmas, Reference Asensio and Delmas2015; Schwartz et al., Reference Schwartz, De Bruin, Fischhoff and Lave2015; Steinhorst and Klöckner, Reference Steinhorst and Klöckner2017). Additionally, it resonates with research acknowledging that crafting the right message or appeal is difficult and emphasising the need for more targeted messaging interventions (List et al., Reference List, Murphy, Price and James2021; Brody et al., Reference Brody, Dai, Saccardo, Milkman, Duckworth, Patel and Gromet2022).
Intrinsic motivation towards sustainable behaviour can come from a positive emotional reward or ‘warm glow’ from aligning one's actions with one's moral values (van der Linden, Reference van der Linden2015).Footnote 1 In the environmental context, warm glow has been shown to predict sustainable behaviour, but it remains unclear if it can be manipulated to encourage such behaviour (Taufik et al., Reference Taufik, Bolderdijk and Steg2015; Hartmann et al., Reference Hartmann, Eisend, Apaolaza and D'Souza2017; Kácha and Ruggeri, Reference Kácha and Ruggeri2018; Van Der Linden, Reference Van Der Linden2018). Moreover, it has often been argued that creating opportunities to experience warm glow from sustainable behaviour could initiate a positive feedback loop in which previously experienced warm glow gives rise to anticipated warm glow thus motivating future PEB (Hartmann et al., Reference Hartmann, Eisend, Apaolaza and D'Souza2017; Van Der Linden, Reference Van Der Linden2018; Brosch, Reference Brosch2021; Schneider et al., Reference Schneider, Zaval and Markowitz2021). However, empirical evidence for this positive reinforcement is lacking (Schneider and van der Linden, Reference Schneider and van der Linden2023).
In this study, we utilise a large-scale pre-registered online experiment to assess the impact of four different messaging interventions, targeting both intrinsic motivation (via emotional reward) and extrinsic motivation (via social reward), on people's willingness to act on climate change.Footnote 2 We randomly assign participants to one of four message interventions: (1) a warm glow appeal, which highlights the positive emotional reward from helping the environment, (2) a cold prickle appeal, which highlights the negative moral emotions of not helping the environment, (3) a social norm appeal which communicates a prescriptive (injunctive) norm and (4) a call-to-action condition which also includes basic information on climate change. It's important to highlight that the call-to-action and basic information about climate change were incorporated into all the previously mentioned conditions to ensure a clear within-treatment comparison. To increase (emotional) engagement, messages were administered in the form of short explainer-style animated videos. We quantify emotions related to PEB with the help of self-report measures, and willingness to act on climate change through a novel incentivised paradigm on pro-environmental effort similar to that of Lange and Dewitte (Reference Lange and Dewitte2022). The persistency of the messaging interventions was also evaluated by measuring pro-environmental effort 2 days after the main experimental survey.
Our study design introduces several key innovations: (1) a novel incentive-compatible measure of PEB through a real-effort task that is tedious and thus more accurately resembles pro-environmental effort, which is often perceived as entailing personal costs and/or extra physical effort. The more effort participants put into the task, the greater the amount of donations they can generate for Friends of the Earth, an environmentally charitable organisation. The measure is quantitative and allows us to observe time invested (‘quantity’) and actual performance on the task (‘quality’). Our innovative measure thus surpasses previous lab research which has primarily relied on self-reported intentions or windfall donations (Schneider et al., Reference Schneider, Zaval and Markowitz2021) and contributes to recent advances in the design of consequential measures of PEB for controlled experimental settings (Lange et al., Reference Lange, Steinke and Dewitte2018; Lange and Dewitte, Reference Lange and Dewitte2022). (2) A longitudinal design to explore if behavioural change persists, at least in the short term, and whether experienced warm glow mediates the relationship between past and future PEB. (3) The ability to measure self-reported emotions and explicitly test their role in the relationship between message interventions and pro-environmental action.
The present paper extends previous work along multiple dimensions. First, our study contributes to the emerging literature on warm glow as an important motivator of PEB (van der Linden, Reference van der Linden2015; Steg et al., Reference Steg, Lindenberg and Keizer2016; Chatelain et al., Reference Chatelain, Hille, Sander, Patel, Hahnel and Brosch2018; Kácha and Ruggeri, Reference Kácha and Ruggeri2018; Venhoeven et al., Reference Venhoeven, Bolderdijk and Steg2020; Gråd et al., Reference Gråd, Erlandsson and Tinghög2021) and to literature on (positive) emotions and climate change engagement (Lange and Dewitte, Reference Lange and Dewitte2020; Brosch, Reference Brosch2021; Schneider et al., Reference Schneider, Zaval and Markowitz2021; Shiota et al., Reference Shiota, Papies, Preston and Sauter2021). Schneider et al. (Reference Schneider, Zaval and Markowitz2021) and Schneider and van der Linden (Reference Schneider and van der Linden2023) review the recent literature and conclude that more research is required to explore actual behaviour (rather than intentions), using large-scale longitudinal studies (rather than cross-sectional designs looking at short-term individual pro-environmental actions). Our study addresses these major gaps. Second, research has stressed the potential for positive emotions (specifically warm glow) to form a positive feedback loop with climate change engagement (Van Der Linden, Reference Van Der Linden2018; Brosch and Steg, Reference Brosch and Steg2021; Schneider and van der Linden, Reference Schneider and van der Linden2023) and we are among the first studies to empirically explore whether appealing to intrinsic motives can kick-start such a self-reinforcing ‘virtuous cycle’. At the same time, our longitudinal design allows us to explore the persistence of treatment effects, at least in the short-term (Allcott and Rogers, Reference Allcott and Rogers2014; Bernedo et al., Reference Bernedo, Ferraro and Price2014; Brandon et al., Reference Brandon, Ferraro, List, Metcalfe, Price and Rundhammer2017; Hume et al., Reference Hume, John, Sanders and Stockdale2020; Gravert and Olsson, Reference Gravert and Olsson2021). In this respect, we also address the challenge of whether warm glow experiences can be exogenously manipulated in a controlled experimental setting (Hartmann et al., Reference Hartmann, Eisend, Apaolaza and D'Souza2017). Finally, we contribute to the literature on informational nudges and persuasive behavioural appeals and messages to promote desirable behaviours (DellaVigna and Gentzkow, Reference DellaVigna and Gentzkow2010; Goldberg et al., Reference Goldberg, Gustafson and van der Linden2020; Milkman et al., Reference Milkman, Gandhi, Patel, Graci, Gromet, Ho, Kay, Lee, Rothschild and Bogard2022). Message framing assumes that individuals are not only sensitive to the content of information but also to the way it is presented, as highlighted by Kahneman and Tversky (Reference Kahneman and Tversky1984), where emotional influences are integral to the dynamics of message framing especially with respect to climate change issues (Schneider et al., Reference Schneider, Zaval and Markowitz2021). Our paper particularly contributes to the ongoing debate within the climate change communication literature, regarding the relative efficacy of positively framed (e.g., emotions such as hope) versus negatively framed (e.g., emotions such as fear, guilt) communications (Rees et al., Reference Rees, Klug and Bamberg2015; Bissing-Olson et al., Reference Bissing-Olson, Fielding and Iyer2016; Charness and Dufwenberg, Reference Charness and Dufwenberg2016; Schneider et al., Reference Schneider, Zaval, Weber and Markowitz2017a; Adams et al., Reference Adams, Hurst and Sintov2020; Shipley and van Riper, Reference Shipley and van Riper2022). More generally, while there has been a notable enthusiasm among organisations and policymakers to employ persuasive appeals to align individuals’ behaviour with policy objectives (e.g., increasing citizens vaccination uptake, tax more or pro-environmetnal behaviour), the effectiveness of appeals has been inconsistent and subject to debate (DellaVigna and Linos, Reference DellaVigna and Linos2022; Bergquist et al., Reference Bergquist, Thiel, Goldberg and van der Linden2023). Some argue that to comprehensively enable behavioural change, it is important to account for heterogeneous responses to behavioural messages (Bryan et al., Reference Bryan, Tipton and Yeager2021; Mills, Reference Mills2022; Hallsworth, Reference Hallsworth2023) and underscore the value of large-scale empirical testing of messages to optimise their effectiveness (Duckworth and Milkman, Reference Duckworth and Milkman2022). We advance this literature with the help of a large-scale experiment with substantial sample size.
Our results indicate that directly appealing to warm glow motives was ineffective in boosting PEB, relative to a call-to-action group that received only basic information on climate change and a call to action. Both warm glow and cold prickle framing only partially succeeded in altering anticipated emotions, with cold prickle significantly reducing anticipated positive affect (PA), but warm glow failing to increase PA, when compared to the call-to-action condition. Social norm framing did not change emotions, as expected. Messages explicitly framed to strongly emphasise the personal emotional rewards of supporting the environment or the negative emotional consequences of neglecting it appear to be ineffective in mobilising pro-environmental effort. Analysis of sub-groups with high and low biospheric values suggests that cold prickle framing reduced pro-environmental effort in people with low biospheric values and warm glow messaging had a negative effect on individuals with high altruistic values. This suggests that climate change communications appealing to both negative and positive emotions may ‘back fire’ for certain people. Finally, it appears that the level of pro-environmental effort remains relatively stable in the short-term.
Study design
Logistics and randomisation
Data were collected via a pre-registered online experiment and recruitment of participants took place via the online crowdsourcing platform Prolific Academic (Palan and Schitter, Reference Palan and Schitter2018). The study was programmed with the survey software Qualtrics and hosted at the servers of the University of Cambridge. The study consisted of a three-wave design including a baseline survey (baseline wave) used for stratified randomisation and assignment to treatment conditions and two experimental surveys (main experiment and a follow-up). That said, the three waves were collected on 3 days during the week of 19 July 2021.Footnote 3 To incentivise participation in all parts, participants were informed that upon completion of all parts, they would be sent an additional bonus payment of £1.
Note that the baseline survey (N = 3,001, UK sample) which was conducted to implement a stratified randomisation procedure for the main experiment included socio-demographic questions, measures of subjective well-being and values orientation, and the real-effort task (without PEB framing) to measure baseline ability. After excluding 21 participants who failed an attention check, the remaining 2,980 were assigned to one of four treatment groups based on gender, baseline ability, self-reported past donation behaviour and life satisfaction, with balance checks performed using additional socio-demographic variables. A detailed overview of all variables used for balance checks can be found in Supplementary Appendix Table A1.
Experimental procedures
The main experimental survey design is shown in Figure 1. Participants were first shown some generic instructions for the pro-environmental effort task and subsequently completed six practice rounds to familiarise themselves with the task. The first three practice rounds had to be completed correctly, before the participant could proceed, while the second three were completed ‘at speed’ with the three second time limit for each trial. The practice round did not contribute to the generation of donations for the environmental charity and had the sole of objective to familiarising participants with the task. Participants were then informed about the objective of scoring as many correct completions as fast as possible during 30 s, and each correct completion generated a 2.5 p donation to the partnered environmental charity.
Figure 1. Experimental survey design.
After the instructions and prior to starting the mandatory part of the pro-environmental effort task participants were shown one of four treatment videos. They were then asked to rate the sentiment of the video (as a manipulation check) and report their anticipated emotions (‘how helping the environment would make them feel’), immediately followed by the thirty-second mandatory pro-environmental effort task. To conclude the mandatory part of the survey, participants reported on the perceived difficulty and enjoyability of the task. It is important to note that the financial reward (£0.40) for completing the survey was based entirely on the estimated time of 2–3 min required to complete the mandatory part only. Participants thus had no financial incentive (or perceived obligation) to complete the voluntary part of the survey.Footnote 4
At this point, participants were shown their Prolific completion code and required to verify their submission on Prolific.co. On the same page, they were notified about the possibility to complete the voluntary part of the questionnaire in which they could generate an additional donation for Friends of the Earth. Participants were clearly informed that this part of the survey was entirely voluntary, would not be financially compensated and that they could stop at any time. If they chose to continue, they were then shown the same real-effort task which they could continue for up to 10 min (200 trials) or exit at any time via an ‘exit button’. Upon completion or exit of the task, participants reported their experienced emotions (how helping the environment made them feel). Participants who did not participate in the voluntary part of the survey did not report their experienced emotions.
Finally, participants were asked to complete a follow-up survey 48 h after completion of the main experiment. The follow-up survey followed the same structure, containing the same mandatory and voluntary pro-environmental effort task and measures of anticipated and experienced emotions, but excluded the treatment videos.
Effort task and survey measures
Pro-environmental effort
A long-standing challenge for experimental research on PEB has been its measurement. Many studies have used self-report measures, but these have limitations (Lange and Dewitte, Reference Lange and Dewitte2019). Another common approach is to allow participants to donate part of their payoff to an environmental charity at the end of the survey (see, for e.g., Schneider et al., Reference Schneider, Zaval, Weber and Markowitz2017b). While this increases the degree of incentive-compatibility and reflects the trade-off between personal gain and pro-social gain, it is based on a single decision which may not accurately represent real-world PEB which is often effortful in addition to costly. More recently, novel approaches have been developed for the study of consequential PEB (Lange et al., Reference Lange, Steinke and Dewitte2018; Lange and Dewitte, Reference Lange and Dewitte2022). Lange et al. (Reference Lange, Steinke and Dewitte2018) present a lab-based ‘PEB-Task’ while Lange and Dewitte (Reference Lange and Dewitte2022) developed a web-based task in which participants can exert voluntary effort in exchange for donations to an actual environmental organisation.
In this study, we use an incentivised effort-donation paradigm. Participants completed a real-effort task (based on the Stroop Task (Stroop, Reference Stroop1935), adapted from McClanahan (Reference McClanahan2020)). Here, participants are shown one of four words (red, green, blue, yellow) randomly printed in one of the four colours and need to use their keyboard to enter the ink-colour of the words independently of the written word. They have a maximum of 3 s per word trial. The task is both cognitively demanding and relatively tedious, thus providing an ideal framework to measure pro-environmental effort. Each successfully completed trial generates a donation of 2.5 p for Friends of EarthFootnote 5. Participants had 10 min to complete up to 200 trials and earn a maximum of £5 for the charity. The payment was conditional on correct completions, making it an incentive-compatible measure of pro-environmental effort.Footnote 6 We obtained four outcome measures from the pro-environmental action task: (1) total donation generated in GBP, (2) share of participants who participated in the voluntary part of the survey and completed at least one trial (3) time spent (quantity dimension) and (4) share of correct trials (quality dimension). The total donation serves as the primary outcome variable as it combines time invested and performance.
Emotions
Our study measured emotions specifically tied to contributing to environmental protection, rather than general mood (Hartmann et al., Reference Hartmann, Eisend, Apaolaza and D'Souza2017). Moreover, we asked participants to reflect on both anticipated (‘how would helping the environment would make you feel) and experienced emotions (‘how did helping the environment make you feel’). Anticipated emotions were measured prior to completing the pro-environmental action task and just after the treatment videos had been shown, and experienced emotions were measured immediately after completing (or exiting) the pro-environmental effort task.Footnote 7
Five positively framed measurement items (Happy, Proud, Hopeful, Inspired, Warm) were used to construct a measure of PA (or ‘Warm Glow’), which incorporates different dimensions of emotional reward derived from the act of helping the environment. Additionally, five negatively framed items (Cold, Guilty, Anxious, Angry, Sad) were used to construct a measure of Negative Affect (or ‘Cold Prickle’), which captures potential negative moral emotions. Participants were asked to rate each of these items on a 10-point scale ranging from 0 (not at all) to 10 (very much). Positive and negative emotions items were averaged to construct scores for anticipated and experienced positive and negative affect. All four scores ranged from 0 to 10 and achieved an overall satisfactory scale validity as measured by the Cronbach's Alpha (0.92 for anticipated PA, 0.82 for experienced PA; 0.95 for anticipated negative affect and 0.82 for experienced negative affect. We acknowledge that the initial wording of the question may have limited the accurate assessment of negative affect, particularly in capturing the ‘cold prickle’ emotions associated with not engaging in environmentally friendly actions. In hindsight, the question should have been framed as ‘How would not helping the environment make you feel?’ to align better with the intended elicitation of negative emotions. Consequently, our main analysis focuses on PA and we report negative affect measures only in Supplementary Appendix.
Treatment messages
Participants were randomly assigned to view one of four treatment messages, presented in 2D animated explainer videos. The videos were 25–56 s long, featured animated characters, and were narrated by a professional voice-over artist with subtitles. All videos can be viewed on our designated YouTube channel (Link).
The standard video (video basic call-to-action information) was 25 s long and presented basic information on the issue of climate change. The script reads as follows:
[1] Call-to-Action Information [video link]Footnote 8
‘Emissions of carbon dioxide are a primary driver of climate change and present one of the world's most pressing challenges. Did you know? Cutting carbon emissions by half can limit global warming to 1.5°C and reduce the harmful impacts of climate change. Act now by contributing today.’
All other treatment videos were up to 56 s long and the introductory paragraph was identical to the above. In addition, the treatment messages included sentences highlighting positive emotions (warm glow video) of helping the environment, negative emotions (cold prickle video) of not helping the environment and the pro-environmental beliefs and behaviours of others (social norm video).
The scripts of the warm glow, cold prickle and social norm video included the following additional information:
[2] Warm Glow [video link]
‘Have you ever experienced that warm fuzzy feeling when helping others? You may get the same feeling when you make climate friendly choices. People who help the environment often feel uplifted, positive, and experience deep feelings of joy and happiness. When you help the environment, it creates a pleasant feeling known as ‘warm glow’, a rewarding emotion that makes you feel good about your contribution. Helping the environment reduces stress and will boost your well-being. Warm your heart and experience these positive emotions by contributing today.’
[3] Cold Prickle [video link]
‘Have you ever experienced that guilty feeling when you've let someone down? You may get the same feeling if you make climate damaging choices. People who do not act to help the environment often end up feeling guilty, shameful and regretful. When you fail to help the environment, it creates an unpleasant feeling known as ‘cold prickle’, a negative emotion that makes you feel bad about your inaction. You may end up feeling stressed and unhappy about your choices. You will feel bad about not contributing today.’
[4] Social Norm [video link]
‘Many people choose to contribute to the global effort to tackle climate change. 8 of 10 people in the UK believe we should do everything necessary, urgently in response to the climate crisis. Many people's decisions to help the environment have been informed by this social norm, which implies a shared expectation that the majority of people now engage in sustainable behaviour. Do your part by contributing today.’
We encourage the reader to view the videos on our YouTube channel for a firsthand understanding of the message framing used in all treatments. Our intention was to ensure a clear and discernible treatment effect with more explicit messages, as research in priming effects suggests that explicit situational cues can shape preferences and behaviour (Cohn and Maréchal, Reference Cohn and Maréchal2016). We also acknowledge the downsides of such an approach such as that explicit primes may cause demand effects and may trigger psychological reactance. We discuss our results in light of these limitations.
Summary statistics
Table 1 presents summary statistics for the socio-demographic characteristics of the sample that completed the first experimental survey (N = 2,689). Of these, 61% were female (information on the participant's gender was not available for two participants) and the average age was 37 years. The level of altruistic values (M = 5.4) and biospheric values (M = 5.1) was generally high in our sample. Over half of participants were educated to degree level (i.e., higher education qualification), while average household income was evenly distributed across the six income brackets.
Table 1. Summary statistics
Note: Table displays the summary statistics of socio-demographic variables for participants of the main experimental survey (N = 2698).
Charitable behaviour was relatively uncommon in our sample. About a quarter of participants indicated that they never donate or volunteer for charity while approximately half of participants contribute once a year. The final quarter of participants said they donate or volunteer at least once a month or more frequently. Similarly, 20% of respondents indicated that they do not donate any money to charity, whereas 45% of participants donate up to £50 per year. Only about 7% of the samples give more than £300 per year. We find that randomisation was successful in balancing all socio-demographic characteristics across the four experimental conditions, with the exception of biospheric values, which we subsequently control for in our main analysis.Footnote 9
Table 2 presents the mean values and standard deviations of our main dependent variables in both the experimental and follow-up surveys. The average donation generated for Friends of the Earth, our primary measure of pro-environmental effort, was £1.13 in the main experimental survey and £1.09 in the follow-up survey. In the experimental survey slightly less than half of all participants (47%) participated in the voluntary pro-environmental effort task, while this share decreased to 40% in the follow-up survey. Participants also spent slightly less time on the task in the follow-up survey (2.4 min) as opposed to the experimental survey (2.6 min), but marginally improved their ability which is reflected by a higher share of correct submissions (92 vs. 89%).
Table 2. Summary statistics: dependent variables
Notes: Summary statistics of the main outcome variables in both the experimental survey and follow-up survey. Share of correct submissions is only available for individuals who started/completed the voluntary pro-environmental effort task.
Results
Here, we present results from a series of linear OLS regressions to explore the effect of the treatment messages on donation behaviour. Details on the estimation strategy are provided in Supplementary Appendix Section 3.
We initially check that the videos were perceived as intended (for details, see Supplementary Appendix Section 4). First, we find that the treatment videos were perceived as intended (Supplementary Figure A1). We asked participants to rate the general sentiment of the video on a five-point Likert scale ranging from ‘extremely negative’ to ‘extremely positive’, immediately after viewing the video. Results indicate that nearly all (90%) found the warm glow message to be somewhat or extremely positive, while the cold prickle message was perceived as (somewhat or extremely) negative by most (60%). The fact that only 60% of participants saw the cold prickle message as negative should not be a major concern, as the distribution is clearly skewed towards negativity. This distribution also differs significantly from all other conditions where positive perceptions dominated. Regression analysis supports these findings (see Supplementary Table A2). Notably, the Warm Glow and Cold Prickle videos had the most contrasting perception distributions, with Cold Prickle peaking at somewhat negative and Warm Glow at extremely positive perceptions, underscoring the success of these treatments in conveying their intended messages on average. Second, we show that the treatment videos were only partially successful in manipulating anticipated PA (see Supplementary Appendix Figure A2 and Table A3 for affect scores, and Supplementary Figure A3 for individual emotion measures). We acknowledge that when comparing both Warm Glow and Cold Prickle conditions to the call-to-action condition, neither appeared to exhibit the anticipated significant shifts in affect. However, it is worth highlighting that when directly comparing the warm glow condition to the cold prickle condition, warm glow notably registers higher in PA than cold prickle, as expected. This comparative analysis between these two conditions suggests that our manipulation, while not entirely consistent across all comparisons, can be considered at least partially successful. We conclude that our manipulations were more effective at priming specific perceptions than directly inducing emotions. Nonetheless, these primed perceptions can potentially influence PEB.
Message effectiveness in main experiment
Figure 2 displays the share of participants that started the voluntary part of the survey (left panel) and the average amount of donations generated (in GBP), our primary measure of ‘pro-environmental effort’. Participants who did not participate in the voluntary part of the survey were coded as having generated a donation equal to zero.
Figure 2. Share of participants that started the voluntary part of the survey and mean donation generated across treatment conditions in main experimental wave.
Note: Donations of participants who did not participate in the voluntary part were coded as zero. Error bars indicate 95% confidence intervals. N = 2,698.
Between 45 and 49% of participants started the voluntary part of the survey, with no significant differences between conditions. Donations were highest in the call-to-action condition (£1.20), which provided only basic information on climate change and a call to action and were lowest amongst participants who viewed the warm glow treatment video.
Table 3 presents the OLS estimates for each treatment condition relative to the call-to-action condition for all four outcome variables. The results indicate that, after controlling for biospheric values (i.e., concern for environment), none of the treatment conditions had a statistically significant effect on donation behaviour, relative to the call-to-action condition (column 1). Moreover, the treatment messages had no effect on participation in the voluntary part of the survey (column 2), time spent on the PEB-task (column 3) or the share of correct completions in the PEB-task (columns 4). Consistent with previous research (de Groot and Steg, Reference de Groot and Steg2008), biospheric value orientation is found to be a significant predictor of all four measures of PEB. Supplementary Table A4 displays the coefficient estimates for the same analysis, incorporating supplementary control variables (age, gender, income, education). Our findings indicate that, in addition to biospheric values, both age and gender significantly predict donation behaviour. Women donate on average £0.29 more than men, while participants aged 50 or older donate £0.24 less. However, our treatment conditions remain statistically indistinguishable from zero. As an additional robustness check we estimate a linear hurdle model (Cragg, Reference Cragg1971), which combines a selection model (i.e., the decision to continue to the voluntary part of the survey) with an outcome model (i.e., the amount of donation generated). The marginal effect estimates presented in Supplementary Table A5 (column 2) suggest that none of the treatment conditions significantly affected the amount of donation generated.
Table 3. Direct effect of treatments on pro-environmental behaviour
Notes: OLS estimates of equation (1). In the first column, the dependent variable is the donation amount generated in GBP (£). In the second column the dependent variable is an indicator identifying subjects that participated in the voluntary part of the survey. In the third column, the dependent variable is a continuous measure of the time spent completing the real-effort task. In the fourth column, the dependent variable is a measure of effort given by the share of correct submissions in the real-effort task. Warm glow, cold prickle and social norm are treatment indicators identifying individuals randomly assigned to a respective condition. The omitted category is the Call-to-Action group. Robust standard errors are shown in brackets.
*p < 0.10, **p < 0.05, ***p < 0.01.
We also employed equivalence testing to ascertain statistical equivalence of the appeals on donation behaviour, our primary outcome of interest (List et al., Reference List, Sadoff and Wagner2011; Lakens et al., Reference Lakens, Scheel and Isager2018). Results from two one-sided tests (TOST) are presented in Supplementary Appendix Table A8 and suggest that, at a 5% significance level, the null hypothesis that effects are equal to or greater than the smallest effect size of interest (SESOI) is rejected, thereby providing evidence of equivalence for each of the treatment effect estimates.Footnote 10
Finally, we explore whether experienced emotions differ between treatment condition. Figure 3 displays average positive affect scores (controlling for differences in biospheric values) based on responses to 5 emotion items assessed at the end of the experimental survey, after completion of the PEB-task.
Figure 3. Experienced positive affect score by time invested in voluntary task.
Note: Error bars indicate 95% confidence intervals. Score range 0–10, N = 1,212 (full sample).
From regression analysis (see Figure 3 and Supplementary Appendix Table A6) we conclude that, after controlling for biospheric values, both the cold prickle and social norm treatment messages significantly decreased average PA (p = 0.035 and p = 0.091, respectively) in the full sample relative to the call-to-action condition. While these reductions are statistically significant at conventional levels, the observed effect sizes are relatively small (Cohen's d = 0.10–0.12) and thus unlikely to be considered ‘economically significant’. That said, it is unclear if such small changes in PA would produce detectable impacts on relevant economic behaviours or outcomes. Taking into consideration the time spent on the voluntary PEB-task, we find that these reductions are driven by the sub-group of participants who spent the most time on the voluntary PEB-task (i.e., over 8 min – Cohen's d = 0.22 for both conditions), while participants who spent less than 8 min on the task did not significantly differ in their experienced warm glow.Footnote 11 It is plausible that the negative emotional priming of the cold prickle message eroded experienced PA for those that nonetheless were motivated to help the environment and spent over 8 min on the task.
Heterogenous treatment effects in main experiment
Altruistic and biospheric values (i.e., concern for environment) have both been found to be important predictors of PEB. We thus hypothesised that our message frames may have heterogeneous effects for people who have higher levels and lower levels of baseline altruistic and biospheric values. We thus measured both types of values using a well-established 12-item values scale (de Groot and Steg, Reference de Groot and Steg2008). The scale is constructed based on responses to 12-items asking respondents to indicate to what extent each statement serves as a guiding principle in their lives. The corresponding items form reliable scales for biospheric values (Cronbach's alpha = 0.94) and altruistic values (Cronbach's alpha = 0.85), respectively. To categorise individuals into high and low value sub-groups we took the median split in our analysis sample. Above median individuals were considered as holding high levels of altruistic and biospheric values, whereas individuals below the median were labelled as holding low levels of values.Footnote 12 To explore differences between sub-groups we estimate equation (1b) which interacts the treatment indicators with the high altruistic or biospheric values indicator respectively (see Online Supplementary Appendix Section 3). Mean values and corresponding 95% confidence intervals are visualised in Figure 4 and the full regression output is presented in Supplementary Appendix Table A8.
Figure 4. Mean donation across treatment conditions in main experiment by biospheric and altruistic values. Notes: Bars with solid outlines display mean donations for the sub-sample of respondents with below median biospheric or altruistic values. Bars with dashed outlines display mean donations for the sub-sample of respondents with above median biospheric or altruistic values. Full regression output is presented in Supplementary Appendix Table A8. Error bars indicate 95% confidence intervals. N = 2,698.
Two interesting findings emerge from this analysis. For subjects with below median biospheric values, the cold prickle message significantly decreased donations compared to the Call-to-Action condition (p = 0.045). This finding suggests that highlighting the negative emotional consequences of failing to act pro-environmentally may be counterproductive to the objective of increasing PEB for people who are less inclined to hold biospheric values. Moreover, we observe that people with above median biospheric values donated less, on average, if they viewed the warm glow message, relative to the call-to-action condition. While this difference is not significant at meaningful levels (p = 0.11), a similar and more pronounced pattern emerges for people who hold high levels of altruistic values (p = 0.018). Both findings suggest that appealing to warm glow benefits or the cold prickle consequences may be counterproductive for certain individuals. Overall, these findings highlight the importance of considering individual differences in values and motivations when crafting messaging strategies to promote pro-environmental action.
Temporal patterns in behaviour: short-term effects
Here we present results from the longitudinal analysis utilising the full data collected in both experimental surveys (main experiment and follow-up). Figure 5 shows the mean donation generated across treatment conditions in both the main survey and the follow-up.
Figure 5. Mean donations in experimental survey (T1) and follow-up (T2) by treatment condition. Notes: donation of participants who did not participate in the voluntary part was coded as zero. Error bars indicate 95% confidence intervals. N = 5,295).
We find that donation behaviour is largely unchanged in the short-term and across treatment conditions. Donations slightly decreased in T2 for both the call-to-action condition and the social norm group, which performed best at T1. However, overall, none of the differences in mean values over time and across groups are statistically significant at the 10% level.
As in the first experimental survey (T1), participants were asked to rate their experienced emotions (how did helping the environment make you feel), after completing the voluntary PEB-task, in the follow-up survey (T2). Supplementary Figure A4 shows that experienced PA remained largely unchanged between T1 and T2 in the full sample. Focussing again on participants who spent the most time on the voluntary task at T1 (more than 8 min), we observe more substantial decreases in experienced PA in the warm glow (p = 0.045), cold prickle (p = 0.027) and social norm groups over time (p = 0.001). Only the Call-to-Action condition remained largely unchanged (p = 0.112).
Emotions as predictors of PEB
While our previous results suggest that emotions are difficult to manipulate, further exploratory analysis confirms and extends previous literature on the link between emotions, particularly warm glow and PEB. For instance, two recent studies found that anticipated warm glow predicts self-reported low-cost but not high-cost PEB (Van Der Linden, Reference Van Der Linden2018; Jia and van der Linden, Reference Jia and van der Linden2020). Similarly, Hartmann et al. (Reference Hartmann, Eisend, Apaolaza and D'Souza2017) explore to what extent the effect of altruistic value orientation on PEB is mediated by anticipated warm glow. To study this relationship, we conduct a simple correlational analysis (Panel A) and a mediation analysis (Panel B) in Table 4. We find that anticipated PA (‘warm glow’) is an important predictor of donations generated for an environmental charity (Column 1). A one-unit increase on the 10-point PA scale is associated with 9.6 p increase in donations, on average. Furthermore, we find that both biospheric and altruistic values (Column 2) are highly correlated with pro-environmental donations. A one-unit increase on the altruistic values and biospheric values scale is associated with a respective increase of 9.9 and 8.8 p in donations, on average. In columns (3) and (4) of Panel A, we also observe that all parameter estimates decrease when both values and anticipated PA are included in the model, suggesting that the effect of value orientation may be partially mediated by anticipated warm glow (PA).
Table 4. Mediation analysis: anticipated warm glow and value orientation
Notes: Panel A presents estimates from a correlational analysis in which the dependent variable is the donation amount generated (£) in columns (1) to (4). Robust standard errors are shown in brackets. Panel B presents estimates from a causal mediation analysis. ACME represents the Average Causal Mediation Effect. IV and MV refer to ‘independent variable’ and ‘mediating variable’, respectively. All models control for treatment assignment.
*p < 0.10, **p < 0.05, ***p < 0.01.
In Panel B, we provide outputs from a causal mediation analysis (methodological details provided in Supplementary Appendix Section 3). In both mediation models, the Average Causal Mediated Effect (ACME) is highly statistically significant, providing evidence of an indirect relationship between value orientation (the independent variable, IV), anticipated PA (the mediating variable, MV) and pro-environmental donations (the dependent variable, DV). The results suggest that approximately 24% of the effect of biospheric values on pro-environmental donations is mediated by anticipated warm glow from helping the environment (Column 3). Similarly, anticipated warm glow also mediates the effect of altruistic values on donations, however, to a slightly smaller extent: 14.14% of the total effect is mediated by anticipated PA.
Furthermore, our longitudinal design allows us to explore whether experienced warm glow in the main experiment reinforces PEB at T2. Figure 6 illustrates the indirect mediation model describing the relationship between donations in the main experiment and donations in the follow-up, mediated by experienced warm glow at T1.
Figure 6. Indirect effect of donations at T1 on donations at T2 mediated by experienced positive affect at T1.
We utilise our index of experienced PA (experienced warm glow) as our MV, which consists of five emotion items which may form warm glow experiences (happy, proud, hopeful, inspired, warm). It is important to note that the sample for this analysis is restricted to individuals who completed both surveys and also participated in the voluntary part of the survey at T1.Footnote 13 The regression outputs (Panel A) and formal mediation analysis outputs (Panel B) are presented in Table 5. Column (1) of Panel A shows that pro-environmental donations at T1 are a weak predictor of experienced PA at T1, which does not reach statistical significance at meaningful levels. The estimates in Column (2) of Panel A suggest that pro-environmental donations at T2 are largely driven by donations at T1. Moreover, experienced PA at T1 has only a small effect on donations at T2, which is weakly statistically significant (p = 0.051). Formal mediation analysis (Column 2, Panel B) confirms that there is no statistically significant indirect relationship between past donations, experienced warm glow and future donations. The ratio of the ACME to the total effect suggests that only 0.41% is mediated by warm glow experiences.
Table 5. Mediation analysis: experienced emotions and donations over time
Notes: Panel A presents estimates from a correlational analysis. The dependent variable in column (1) is experienced positive affect and experienced happiness at T1. The dependent variable in column (2) is the donation amount generated (£) at T2. Robust standard errors are shown in in brackets. Panel B presents estimates from a causal mediation analysis. ACME represents the Average Causal Mediation Effect.
*p < 0.10, **p < 0.05, ***p < 0.01.
The findings from this analysis suggest that experienced warm glow may not be as an important mediator as previously believed. Much rather, donation behaviour appears to be relatively constant over time and the mediating relationship with experienced warm glow is statistically insignificant. It is important to note that this analysis excludes individuals who made zero contribution to charity, thus limiting the sample to already highly motivated ‘green’ participants.
Discussion and conclusion
We find no evidence that appealing to intrinsic (‘warm glow’ and ‘cold prickle’) and extrinsic (‘social norm’) motives motivates people to act pro-environmentally any more than a simple call-to-action. Emphasising the positive ‘warm glow’ and negative ‘cold prickle’ emotions associated with acting (or failing to act) pro-environmentally can even backfire for those who are already more concerned about the environment by crowding-out donations. What can account for these findings?
Focusing first on the warm glow appeal, our analysis indicates that the appeal had no statistically significant effect on promoting PEB. This finding is at odds with recent field experiments, which found warm glow appeals to be significantly more effective than social norm or altruistic appeals in promoting generic altruistic behaviour (Bergquist et al., Reference Bergquist, Nyström and Nilsson2020; List et al., Reference List, Murphy, Price and James2021; Ferguson et al., Reference Ferguson, Lawrence, Bowen, Gemelli, Rozsa, Niekrasz, van Dongen, Williams, Thijsen, Guerin, Masser and Davison2023). It is important to note, however, that the studies by Bergquist et al. (Reference Bergquist, Nyström and Nilsson2020), List et al. (Reference List, Murphy, Price and James2021) and Ferguson et al. (Reference Ferguson, Lawrence, Bowen, Gemelli, Rozsa, Niekrasz, van Dongen, Williams, Thijsen, Guerin, Masser and Davison2023) focused on generic altruistic behaviour rather than PEB specifically and utilised implicit warm glow appeals.
Another explanation for our result is that the call-to-action used in all the treatment groups might have been enough to stimulate warm glow emotions, regardless of the specific treatment. Theories of pro-social preferences propose that pro-social acts can be driven by warm glow (for an overview see Meier (Reference Meier2007)), and a large literature in charitable giving has provided direct evidence for a warm glow effect on donations (e.g., Crumpler and Grossman, Reference Crumpler and Grossman2008; Konow, Reference Konow2010; Bischoff and Krauskopf, Reference Bischoff and Krauskopf2015). We observe high levels of self-reported anticipated warm glow (PA) in all four treatment groups, strongly predicting donation behaviour. The latter also indicates that participants may have already been familiar with warm glow experiences (i.e., they were already ‘glowing’ in the environmental domain) and thus may have been less susceptible to the warm glow treatment, making it difficult to observe a treatment effect in increasing warm glow.
An additional issue to consider is that explicit attempts by videos or other forms of communication to elicit warm glow emotions may lead individuals to perceive external influence on their emotional state rather than experiencing authentic emotions through personal agency. This in turn could lead to a sense of a sense of being coerced or influenced and might reduce the positive impact of the warm glow effect. Our findings suggest that making the private benefits of PEB overly salient (by explicitly emphasising the concept of warm glow) may crowd out motivation to act or even backfire for altruistic individuals. Directly and explicitly appealing to warm glow motives is thus unlikely to be an effective strategy to encourage PEB and warrants further investigation into whether implicitly appealing to warm glow emotions can be more effective than direct (explicit) appeals, for instance, by increasing the meaningfulness and satisfaction derived from the behaviour (Brosch and Steg, Reference Brosch and Steg2021). At the same time, implicit appeals through the refinement of tone and framing of the videos might also be able to minimise any negative effects.
Regarding the cold prickle messaging, which increased the salience of negative emotions such as guilt and shame, our study found that it significantly decreased both anticipated and experienced PA without inducing treatment effects on donations. This suggests that while the message was generally perceived as negative, it did not have the desired impact on PEB. Our heterogeneity analysis suggests that individuals with lower baseline biospheric values were significantly less likely to engage in PEB when exposed to cold prickle messaging. One plausible psychological mechanism for this finding is psychological reactance. Psychological reactance refers to the defensive response individuals may have when they perceive an attempt to restrict their freedom or manipulate their behaviour (Brehm, Reference Brehm1966; Rosenberg and Siegel, Reference Rosenberg and Siegel2018). It can manifest as a resistance to persuasion tactics and, in some cases, may lead to the reinforcement of existing attitudes and behaviours. In the case of individuals with low biospheric values, we hypothesise that they may interpret the guilt-inducing cold prickle messaging as an attempt to manipulate or coerce them into PEB, which could trigger a psychological reactance response, further contributing to their reluctance to engage in such behaviour. These findings echo previous research suggesting that guilt framing may not be the most effective strategy for promoting PEB among individuals with low environmental concern (Wonneberger, Reference Wonneberger2018), emphasising the need for caution when using guilt appeals in environmental campaigns for this target audience.
Regarding the injunctive social norm message, we found no effect on pro-environmental effort. We hypothesise that social norm messages might not work if participants already perceive the behaviour as socially normative, and therefore, the message may not provide new or additional information to motivate them to engage in the behaviour. Recent research shows that descriptive social norm messages are particularly effective if they correct people's misperceptions of the norm (Peter et al., Reference Peter, Boneva, Chopra and Falk2021). Future research should attempt to measure baseline beliefs to further explore this hypothesis. Additionally, it is worth noting that the lack of impact observed with the social norm message may also be attributed to the treatment's abstract nature. The content of the norm message focused on sustainable behaviour in a general context, without specifically addressing people's views on appropriate behaviour within the experimental task. This abstract framing may have failed to resonate with participants in a way that directly influences their actions in the specific experimental task.
Our study comes with its limitations as we found that all videos resulted in high levels of engagement, and also the call-to-action and social norm conditions were perceived as generally positive, leading to a potential ceiling effect to discern an additional treatment effect from warm glow messaging. That said, the call-to-action message on its own was highly successful, indicating that at least in our setting, simple messages were more effective than complex ones. Also, social desirability bias (i.e., respondents wanting to present themselves in a positive light) could have played a role for the high engagement and the high number of respondents opting for the voluntary pro-environmental action in our survey. If so, our results should be regarded as representing an upper bound of engagement, as respondents may have been more inclined to act pro-environmentally due to social desirability considerations, although recent empirical evidence suggests that this is unlikely (Mummolo and Peterson, Reference Mummolo and Peterson2019). Our study also raises concerns about experimenter demand effects and the potential that participants felt they were being manipulated by the explicit wording of the video messages. While we cannot empirically test this claim, some indication is provided by Gråd et al. (Reference Gråd, Erlandsson and Tinghög2021). They find that donations to a charity were unaffected by different types of nudges (including social norm and moral norm) for those subjects who felt that the nudges were an attempt to manipulate their donation behaviour. They conclude that, ‘if someone feels pressured or tricked into an action, the pro-social act might be less rewarding in terms of experienced warm glow’ (Gråd et al., Reference Gråd, Erlandsson and Tinghög2021, p.3). In line with this, we find that experienced warm glow, measured after the real-effort task, was lower in all three treatment conditions compared to the call-to-action condition.
In summary, our findings highlight the difficulty of crafting clear and concise messages that resonate with individuals’ existing motivations and beliefs. In this experiment, we found that providing basic information about climate change combined with a call to action was more effective in inducing PEB than explicitly framed messages attempting to elicit ‘warm glow’ and ‘cold prickle’ emotions. Further research is thus needed to understand how to best harness the potential of emotional incentives to mobilise pro-environmental action. Future research may focus on understanding how to best design intervention strategies, which take into account the reciprocal relationship between emotion and cognition (Brosch and Steg, Reference Brosch and Steg2021). Moreover, it will be crucial to understand how to tailor effective messaging strategies for different audiences and contexts.
Our findings thus imply that policy makers may exercise caution when making use of (explicit) emotion-based communication strategies (Chapman et al., Reference Chapman, Lickel and Markowitz2017; de Vries, Reference de Vries2020). Future policies incorporating emotional appeals should be grounded in affective science and demonstrate a thorough a-priori understanding of how the targeted emotions are integrated within a complex and integrated interpretive learning system (Chapman et al., Reference Chapman, Lickel and Markowitz2017), for instance their interaction with key judgments of risk and behavioural control (Brosch and Steg, Reference Brosch and Steg2021). Prior testing and validation of emotional appeals and an in-depth knowledge of the target audience will be crucial to avoid potential psychological reactance and ensure the effective use of scarce public resources. Moreover, emotions should be seen as a component of a comprehensive communication approach, rather than being viewed as a quick fix aimed at eliciting specific responses (Chapman et al., Reference Chapman, Lickel and Markowitz2017). Relatedly, future research could shift from a binary emotional framework (‘positive’ vs. ‘negative’) towards the development of hybrid communication strategies that consider appraisals and subsequent cognitive effects (Brosch and Steg, Reference Brosch and Steg2021). More research is needed to understand how emotions can be harnessed as a powerful force for good.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/bpp.2024.4.
Data availability
The experiment was pre-registered via the Open Science Framework (OSF) prior to data collection (OSF reference: https://osf.io/gbmv7). Data and code will be made available via the OSF repository: https://osf.io/e5kfs/.
Ethics statement
Ethical approval for the experiment was granted by an Institutional Review Board.
Open access
