Email mobilization
What is the effect of an unsolicited email encouragement on voter turnout? Campaigns currently employ a broad strategy of contact that includes both text messages (Roose Reference Roose2018) and email (Astor Reference Astor2019) to generate financial and electoral support. Voters opt-in to provide their email addresses to issue-advocacy groups, and national campaigns spend considerable effort to manage, merge, and then re-distribute these lists for use by political allies (Evers-Hillstrom and Erickson Reference Evers-Hillstrom and Erickson2019). The consequence of these strategies is that voters receive electronic communication from campaigns, though they may not have explicitly “opted in” to receive these messages. Experiments conducted between 2007 and 2012 are mixed in their findings about email’s ability to generate votes (see Table 1). In this study, we re-examine email mobilization because, despite little experimental evidence in support of its effectiveness, campaigns continue to spend considerable capital to send emails to voters for support.
Table 1 Mixed Evidence for Email Mobilization
When email has been effective the senders were a trusted, official source (Malhotra, Michelson, and Valenzuela Reference Malhotra, Michelson and Valenzuela2012). Outside of email mobilization, there is evidence that some efforts meant to increase turnout can inadvertently have the opposite effect (e.g., Cornwall and Kessler Reference Cornwall and Kessler2012; Grose and Russell Reference Grose and Russell2008; Kousser and Mullin Reference Kousser and Mullin2007; McCabe and Michelson Reference McCabe and Michelson2015).
Here, we test the impact of email messages that prime social norms about voting (Gerber and Rogers Reference Gerber and Rogers2009). The messages we send emanate from a publicly identifiable political science professor who is a member of our research team. We contact voters using a real identity to forestall concerns about deceptive messaging. While this experiment was designed to test the differential impact of injunctive and descriptive norms on turnout (Rivera, Hughes, and Gell-Redman Reference Rivera, Hughes and Gell-Redman2016), we find no clear evidence that voters react differently to different treatment messages. Instead, we were very surprised to observe a small, but persistent decrease in turnout among voters assigned to receive a treatment message.
Experimental setting, approach, and data
We target voters who provide a valid email in the publicly available October 10, 2016, Florida Division of Elections voter roll. Random assignment to receive an email message was blocked on congressional district and self-reported race (See Table 2 in the SI). Of the more than 12 million registered voters in Florida, 503,859 provide a valid email and are assigned to a condition (Rivera, Hughes and Gell-Redman Reference Rivera, Hughes and Gell-Redman2020).
Table 2 Comparison of Email Providers
Notes: SEs are omitted here but reported in the SI. Proportion Democrat is the proportion of registered Democrats among those registered as a Democrat or Republican. Column 2, Provide Email includes registered voters with email addresses that could not be reconciled to be valid, Assigned A Condition, includes only voters with a valid email. The Analytic Sample, used throughout the reported analysis, excludes voters who voted early in-person.
Of all votes cast during the 2016 general election in Florida, 39.7% were cast early in-person, 29.4% were by mail, and 30.5% were cast in-person on election day. Throughout the reported results, the analytic sample is the group of
$328,181$
Florida voters who provide a valid email address and did not vote early in-person. We include those who voted by mail, because some of these voters could have sent in their ballot after receiving a message. We exclude voters who cast a ballot early in-person, because these votes were cast before the distribution of our treatment messages. The analytic sample is notably younger than the whole set of registered voters, but otherwise broadly similar in terms of party affiliation, gender identification, and ethnic/racial identification (Table 2, columns 1 & 4).
We contact voters via a single email sent from an active email address with a.edu top-level domain. All emails contain the subject heading “Please Remember to Vote Tomorrow,” begin with a greeting, and close with a brief paragraph that directs questions about the voting process to the Florida Department of State. Footnote 1 In line with current campaign practices, we sent messages the morning of November 7, 2016, at 10am EST, one day before the election. Limitations on the sending infrastructure precluded the ability to measure whether a voter received or opened our emails (Hughes et al. Reference Hughes, Gell-Redman, Krishnaswami, Rodenberger, Monge and Crabtree2019). All results are intent-to-treat effects.
Results
Outcome data are drawn from the June 14, 2017, official voter roll. Voters assigned to receive no contact – that is, the control group – turned out to vote at a rate of 78.7 percent (see SI Section 4.1.1).
Footnote 2
Our main experimental finding, shown in Figure 1, is that those who were randomly assigned to receive any email message turned out at a 0.53 percentage point lower rate (
${\overline{\tau}} = - 0.53$
, robust
$SE = 0.17$
).
Footnote 3
Figure 1 Reduction in rate of turnout among registered voters assigned to receive an email, compared to voters of the same race/ethnicity in the control group. All points are estimated in separate models, on data partitions noted on the x-axis. Thick lines are 1.64 times the robust standard error; thin lines are 1.96 times the robust standard error. See Table 4 and Table 6 in the SI.
We further examine the effect of email messages within groups of self-identified black, Latino, and white voters.
Footnote 4
Among the 36,518 black voters in the analytic sample, 64.0 percent assigned to the control group voted, compared to 61.8 percent in the treatment group (
${{\overline{\tau}}_b} = - 2.168$
, robust
$SE = 0.585$
). Among the 63,404 Latino voters in the analytic sample, 73.9 percent assigned to the control group voted, compared to 73.0 percent in the treatment group (
${{\overline{\tau}}_l} = - 0.970$
, robust
$SE = 0.414$
). Finally, among the 204,054 white voters in the analytic sample, 84.0 percent assigned to the control group voted, compared to 83.7 percent in the treatment group (
${{\overline{\tau}}_w} = 0.309$
, robust
$SE = 0.193$
). Plainly stated, these messages may have kept as many as 1,389 voters from going to the polls (
$95\%\, robust \,CI = [ - 2249, - 531]$
).
Discussion
Email mobilization has attracted interest in part because of its potential to deliver results at very low cost. Our results show that this low-cost investment for campaigns may yield zero or even negative returns for some groups of voters. Like all field experiments, ours has idiosyncratic context and design features that require care in extrapolating to both existing findings and future contexts (Bates and Glennerster Reference Bates and Glennerster2017). Specifically, our treatment messages were sent in the competitive 2016 election by a university professor with no personal or institutional connection to voters in the study. The African American voters who received our messages live in the shadow of twin legacies of exclusion. Not only have black voters been shut out of the political process (Dawson Reference Dawson1995), but the black community has also been excluded from, and in the extreme, exploited by the research community (Brandt Reference Brandt1978). In fact, one explanation for the particularly sharp depressive effect of our message among these voters is a lingering mistrust of academic researchers, especially those who lack a personal connection to the community.
Ultimately, our design cannot support direct claims about why turnout was reduced. However, our preferred conjecture is that receiving a treatment message induced stress in some subjects. Approximately 700 voters chose to respond to our message; many of these people used worried or anxious language. Voters’ decisions to reply occur after exposure to treatment, and so we are circumspect about the causal inferences permitted by these responses (Coppock Reference Coppock2019). It is possible, however, that when voters perceive the political environment to be hostile or unfavorable the circumstances of voting cause stress, and unsolicited communication may reinforce those negative impressions and lower voters’ internal motivation to turnout (Hassell and Settle Reference Hassell and Settle2017). This perspective is consonant with careful observational work that demonstrates that black voters who perceive a lower chance of electing their preferred candidate are less responsive to mobilization efforts (McGowen Reference McGowen, White and Philpot2010).
There is no clear consensus within political science over the ethical norms that should guide experimentation (e.g., Desposato Reference Desposato2015). Our own view is that the guiding principle of experimentation should be to avoid doing harm wherever possible and to balance any potential harm against the knowledge to be gained. We expected that email messages that prime social norms might provide a low cost means to encourage turnout, and we reasoned that for at least some voters this would occur by increasing negative emotions (Gerber and Rogers Reference Gerber and Rogers2009). On the one hand, priming norms may lower voters’ internal motivation (Hassell and Settle Reference Hassell and Settle2017) by increasing shame, stress, or anxiety (Marcus, Neuman, and MacKuen Reference Marcus, Neuman and MacKuen2000; Panagopoulos Reference Panagopoulos2010); on the other hand, priming norms may increase internal motivation by increasing pride (Panagopoulos Reference Panagopoulos2010). The net effect of these and other forces cannot be known in advance. Because our messages decreased turnout, we must ask whether the study’s scientific benefits outweighed the harm done. Our answer is that the learning strongly reinforces findings in the literature that emphasize the importance of personal contact and connection to the community, particularly among minority voters.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2021.34
Data availability statement
The data, code, and compute environment required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/YIZEA7.
Acknowledgements
We are grateful for comments from attendees of the 2017 State Politics and Policy Conference.
Conflicts of interest
The authors have no conflicts of interest to declare.
Open access
