A common theme in campaign communications is that one must vote because who wins the upcoming election is a matter of enormous consequence. Campaign appeals invoking the critical importance of an election are frequently dramatic and often stretch credibility but are clearly viewed as an essential ingredient by nearly all successful campaigns. Creating a sense that the stakes of the contest are important (that is, that the electoral outcome matters) is perhaps the central feature of campaigns and mobilization efforts (see, for example, Sides and Vavreck Reference Sides and Vavreck2013; Vavreck Reference Vavreck2009).
By contrast, academic work has generally shied away from assessing the effectiveness of appeals that invoke factors related to the specific political context, such as the candidates running for office and the importance of the contested office. Instead, scholars have largely turned their attention toward mobilization messages grounded in how a citizen's general engagement with politics and desire to perform their civic and social duty may explain participation. Numerous field-experimental studies demonstrate that appeals to civic duty, social norms, and other factors unrelated to the stakes of the contest at hand have modest effects on turnout (see, for example, Gerber and Green Reference Gerber and Green2000; Gerber, Green, and Larimer Reference Gerber, Green and Larimer2008; Panagopolous Reference Panagopoulos2011).Footnote 1 However, that focus on the motivational power of appeals to democratic citizenship is in stark contrast to what actually unfolds in most campaigns, where turnout appeals are typically designed to create a sense of urgency related to why that race matters rather than emphasizing broad themes of civic engagement or democratic responsibility (see, for example, Fowler, Franz, and Ridout Reference Fowler, Franz and Ridout2016).Footnote 2
The possibility that appeals to the importance of an election might stimulate participation is consistent with resource-based, psychological, and rational-choice models of turnout, which all attribute a prominent role in the decision to vote to such perceived stakes (see, for example, Campbell et al. Reference Campbell1960; Riker and Ordeshook Reference Riker and Ordeshook1968; Verba, Scholzman, and Brady Reference Verba, Scholzman and Brady1995). Additionally, observational work shows that those who vote are more informed about and interested in the race, have more of a vested self-interest in its outcome, and see greater benefit to one candidate winning versus another (see, for example, Leighley and Nagler Reference Leighley and Nagler2014; Lewis-Beck et al. Reference Lewis-Beck2008). For this reason, scholars have long speculated that the failure to appreciate the consequences of election outcomes explains low turnout, with the demobilizing effect of that failure exacerbated in nonpresidential contests, where what is “at stake” may not be clear (see, for example, Hajnal Reference Hajnal2010; Leighley and Nagler Reference Leighley and Nagler2014). However, assessing the causal effect of perceptions of the stakes of an election outcome is difficult using observational data because unobserved factors may explain both information levels and voting, and voting may itself lead to knowledge of what is at stake (see, for example, Shineman Reference Shineman2018). Prior research also suggests reasons for skepticism: if those who are not engaged have low efficacy or trust and therefore believe the political system is beyond their control or comprehension (see, for example, Verba, Scholzman, and Brady Reference Verba, Scholzman and Brady1995), then merely knowing much is at stake would be unlikely to change patterns of participation.
In light of these conflicting findings, we seek to estimate the causal relationship between increasing the perceived importance of an election outcome—an election's stakes—and turnout. To do so, we present the first field test (to our knowledge) of the participatory effects of providing individuals with specific objective, nonpartisan information about why an election matters. We designed and implemented a large-scale voter-mobilization field experiment with approximately 950,000 subjects in which we mailed roughly 24,500 letters to registrants a few days before the November 2013 local elections in the state of Connecticut. The main aim of our two classes of experimental treatments, which provide various pieces of information about local spending or taxation, is to inform (or remind) citizens in concrete terms of the stakes of municipal elections. These treatments: (1) emphasize the general power of government in controlling resources; or (2) implicate poor government performance by framing taxes as being high. Additionally, we build into our design a standard nonpartisan get-out-the-vote (GOTV) civic duty appeal so that we estimate the incremental effectiveness of providing information about the material stakes of the election. We focus on local elections because they have low turnout dominated by unrepresentative subsets of eligible voters and because citizens are often uninformed about the powers of local governments despite their significant authority. These elections therefore provide an ideal environment to understand whether the dissemination of nonpartisan factual information that alters the perceived stakes of contests can increase participation.
We find that messages emphasizing the budgetary stakes of local elections and those emphasizing comparative tax burdens both appear effective in increasing participation in local contests. Compared to an untreated control group, these effects are substantial. A single letter emphasizing the budgetary resources that are controlled by local government increases turnout by between 2.7 and 2.9 percentage points, while a letter highlighting the comparatively high tax burden in a citizen's town is estimated to increase voting by 4.2 points. By contrast, a simple nonpartisan GOTV letter, which is the base to which these novel treatments are added, has a point estimate of 1.2 points, meaning that these effects are also larger than standard mobilization efforts in pairwise comparisons. Overall, these results support the argument that communication emphasizing the stakes of the election can have a substantial effect on turnout.
Our results provide causal evidence that inattention to the stakes of an election reduces turnout. Although prior work has documented this relationship using survey data about perceived differences between the candidates and reported reasons for abstention, our findings rule out standard concerns about omitted variable bias from observational analyses and survey response misreports (for example, post hoc rationalizations or social desirability bias). Some of the estimated treatment effects we present are of borderline statistical significance compared to no contact when we address potential concerns about multiple testing (in part, due to the modest sample sizes for each individual treatment). Multiple-testing corrections also lead our most successful treatments to not reach standard levels of statistical significance compared to the GOTV message, which means that these results warrant further replication. That said, the findings signal a potentially more successful approach to engage nonvoting registrants: convincing them about the policy consequences of an election rather than reminding them of an upcoming contest or telling them that it is their civic duty to vote. Assuming these results replicate, our findings suggest the use of these messages may be a powerful and underutilized tool for increasing engagement in local and other low-salience contests.
Can Invoking the Stakes of an Election Increase Turnout?
Scholars have long noted that low and unequal turnout plagues US elections, and that this pattern, consistently observed in presidential races, tends to be exacerbated in less salient elections, with lower and more unequal turnout in midterm, primary, and local contests (see, for example, Leighley and Nagler Reference Leighley and Nagler2014; Oliver Reference Oliver2012). Among the many proffered explanations for these low and distinct participation rates is variation across individuals in perceptions of how much an election's outcome matters. Many of the path-breaking theoretical explanations for voting maintain that such perceived stakes, whether derived from an electoral outcome in general or one candidate winning versus another, exert a critical influence on the decision to turn out (Campbell et al. Reference Campbell1960; Riker and Ordeshook Reference Riker and Ordeshook1968; Rosenstone and Hansen Reference Rosenstone and Hansen1993; Verba, Scholzman, and Brady Reference Verba, Scholzman and Brady1995). These theories anticipate that those who are more informed about and interested in politics, who perceive greater policy differences between the candidates and/or parties, or who care more about the outcome should be more likely to vote. In contrast, nonvoters should have greater difficulty seeing the importance of a particular electoral outcome. This should not be surprising: if the stakes are unclear, then an individual has much less of an incentive to vote.
A rich descriptive survey literature is consistent with this perspective. For example, higher levels of political information and interest are associated with a higher propensity to vote (Campbell et al. Reference Campbell1960; Leighley and Nagler Reference Leighley and Nagler2014; Lewis-Beck et al. Reference Lewis-Beck2008). In addition, wealthier individuals, who participate at higher rates than poorer Americans, perceive greater policy differences between presidential candidates (Leighley and Nagler Reference Leighley and Nagler2014). Reported indifference toward candidates reduces the inclination to vote in presidential and midterm elections (Adams, Dow, and Merrill Reference Adams, Dow and Merrill2006; Blais and Achen Reference Blais and Achen2019; Plane and Gershtenson Reference Plane and Gershtenson2004). Finally, survey respondents frequently justify decisions to abstain in terms of failing to see the stakes of the election (see, for example, Verba, Scholzman, and Brady Reference Verba, Scholzman and Brady1995). That work, however, leaves open several critical questions. Most centrally, it is unclear whether perceiving that more is at stake in a contest is a possible causal explanation for participation. As with any observational analysis, a crucial concern is that omitted variables could cause both participation and perceptions, a problem that is exacerbated in the survey context, where measurement error is ubiquitous. The clearest evidence that the stakes of an election matter in driving turnout is that those whose jobs, family well-being, or most important financial asset's value are directly tied to an election—for example, teachers, parents of school-aged children, and homeowners, respectively—are most likely to vote in municipal elections (Anzia Reference Anzia2011; Berry Reference Berry2009; Fischel Reference Fischel2005; Moe Reference Moe2006; Oliver Reference Oliver2012). Those studies, however, cannot distinguish among greater concern for the outcome, greater knowledge, or group mobilization as driving participation.
Additionally, even if abstainers stay home because they believe little is at stake, this does not mean that such beliefs are remediable—these individuals may be alienated, believe the political system is too complicated for them to understand or control (low efficacy), defer voting to others who they believe can act in their place, or have difficulty figuring out for whom they ought to vote. In these cases, merely knowing government is important may do little to bring people who know they ought to care to the polls. That possibility is reflected in the frequent failure of mobilization appeals that describe an election as important or that seek to heighten perceived differences between candidates (via efforts to persuade individuals to support a particular candidate) to mobilize registrants (see Green and Gerber Reference Green and Gerber2019).Footnote 3
Alternatively, for at least some individuals in some electoral contexts, their lack of motivation to vote may stem not from systematic ignorance about the importance of politics, but rather from an election-specific, solvable informational issue about the stakes of the contest. For those individuals, one seemingly fruitful way to increase their participation is to attempt to change their perceptions of why an outcome might matter to them—that is, to not simply tell them, absent any context, that the election is important (as other work that has failed to increase turnout has often done), but rather to provide information about the specific policy outcomes tied to that election. The content of such messages is likely particularly important in lower-salience elections where candidate- and campaign-level factors do less to engage potential voters. Simply put, given that Americans tend to be poorly informed about political matters in general and know even less about state and local affairs (see, for example, Delli Carpini and Keeter Reference Delli Carpini and Keeter1996; Hopkins Reference Hopkins2018), one might be able to increase participation just by making citizens aware of the concrete consequences of the contest's outcome.
Given this possibility, we identify two classes of messages that may potentially be useful in increasing participation and that have not been employed in prior field-experimental work. The first is a simple message emphasizing what is materially and specifically at stake in control of local government. This is designed to increase perceptions of the importance of elections for local offices in general. The second builds on the base of providing information about the specific importance of government and adds information about relative government performance. Both academic work on retrospective voting (see, for example, Arceneaux Reference Arceneaux2003; Fiorina Reference Fiorina1978; Niven Reference Niven2006) and candidate campaign communications emphasizing an incumbent's good or bad performance directly point to the prospects of increasing the motivation to vote by stressing how well or poorly current government performs. Failures, in particular, have the potential to motivate previous abstainers to understand the importance of elections so as to avoid undesirable policy outcomes.Footnote 4
Experimental Design
We designed and implemented a large-scale field experiment during the November 2013 local elections cycle in Connecticut to test the two classes of messages introduced earlier. The experiment targeted active, eligible registrants in towns that held municipal elections at that time. After excluding invalid and out-of-state mailing addresses, we randomly selected a single person from each (potentially multiperson) household for inclusion in the sampling frame.Footnote 5 We then constructed 16 assignment strata, which were composed of the cross between three age categories (less than or equal to 35, 36–54, and 55+) and five past participation patterns, based on meeting the first criteria as listed: New registrants (first registered after 1 November 2010), Ever local voters (had ever voted in a local election); Ever midterm voters (had ever voted in a midterm election); Ever presidential voters (any record of voting); and Never voters. For the past participation pattern categories, we distinguish New registrants from Never voters because the former did not have the opportunity to participate in all three types of elections prior to our field experiment. The other categories classify registrants based on the least salient race in which they had previously voted. Although this distinction (which proxies for vote propensity) is imperfect, it reduces to a manageable number the groups for which we conducted random assignment. (The final stratum was composed of 7,603 individuals, who we recruited to participate in a separate survey project.)
We then assigned treatments at different rates across strata so as to achieve adequate power to compare across these groups. Setting aside the “survey group” (from which we assigned 2,588 to treatments with the goal of being able to merge survey data to treatment effectiveness assessments in later work), this involved undersampling Ever local voters (because they had already displayed a relatively high propensity to vote in local elections) and oversampling Ever midterm voters and Ever presidential voters (with the goal of testing whether these groups could be induced to participate in municipal elections). Table 1 lists the counts of selected registrants in the initial sampling frame and overall treatment assignment rates (any treatment assigned) for each stratum.Footnote 6 Our sample is composed of 948,716 registrants, and we sent a total of 24,581 treatment mailings.
Table 1. Sampling frame and overall treatment assignment rates
Notes: Initial universe is all active Connecticut registrants listed in a September 2013 voter file residing in a town with a local election in November 2013 and who had a valid mailing address in the state, with a limit of one registrant selected at random from multiperson households. Additionally removed from the sampling frame are households with selected journalists or elected state and local officials.
In addition to varying the rate at which we treated the strata, the rate at which each treatment was assigned within strata differed across treatments (but was the same across strata). We assigned our “placebo” treatment, modeled after a standard GOTV treatment letter, at higher rates because we compare multiple treatments to this baseline. Treatment assignment rates (which are the same conditional on any treatment being applied across strata) are summarized in Table 2.Footnote 7 Our subsequent analysis accounts for the different rates of assignment across treatments and strata using inverse probability weights and indicators for stratum.
Table 2. Treatment assignment rates, by treatment
Treatment letters were mailed via US first-class mail from a Connecticut location under cover of “Connecticut Votes (CTV)” and timed to be delivered three to five days before the election. We measured turnout in the 2013 election as recorded in a Connecticut voter file dated February 11, 2014. Individuals were coded as having voted if they were listed in the file and had a turnout record for November 2013. Individuals who were listed as not voting or who were no longer present in the file were coded as not voting (we code the latter as not voting because a failure to vote may have been the reason they were purged).Footnote 8 A total of 31 per cent of registrants in the control group in our initial sampling frame voted in November 2013.
Treatment Descriptions
We designed treatments to mirror the two classes of messages described earlier. The basic template for all treatments is the text from our standard nonpartisan GOTV mobilization letter that both informs recipients about the upcoming election and invokes civic-duty norms. The treatment letters hold constant the GOTV message but provide additional content. This approach allows us to assess whether simple nonpartisan communication can increase participation in local elections and to benchmark our other treatments to this standard campaign mobilization tactic.Footnote 9 The text of the GOTV treatment read as follows:
Dear <FIRSTNAME> <MIDDLENAME> <LASTNAME>,
This letter is to remind you that a Municipal Election will be held on Tuesday, November 5th, 2013. Polls will be open from 6 AM to 8 PM on Election Day. Connecticut records show that as of September 15th, 2013 you were registered to vote.
Voting is a right and responsibility. Whichever candidate or party you prefer, we want to remind you to exercise your right to vote this November. The right to vote is an important American tradition. The whole point of democracy is that citizens are active participants in government and democracy functions best when everyone takes part in the voting process. This November, remember your rights and responsibilities as a citizen.
If you have any questions about the voting process, please visit the Secretary of State's website (http://www.sots.ct.gov) or call your local Registrar of Voters.
Sincerely,
The Connecticut Votes Team
Budget Treatments
Our first class of treatments sought to emphasize the material resources at stake in local government control by discussing local budgets. The first two versions of this treatment told recipients that local elections affected the allocation of local spending and informed them about spending levels in different ways (thereby informing them of the large amount of money controlled by local governments). For both of these treatments, our expectation is that, on average, citizens may not have a sense of the material stakes involved in local government because they do not understand how much money it controls (given, as noted earlier, that they are particularly uninformed about local government). As such, learning that amount may increase the importance of local elections that decide how those resources are spent. Notably, our treatment includes language making this control of resource allocation explicit: “One of the most important things (local officials) do is decide what to spend money on.” Thus, the treatment can operate through either mechanism by making citizens aware either that the bundle of local resources is large or that local elections decide the allocation of that bundle. As we do not have a measure of subjects' prior understanding of local government spending (and thus how the provided information comports with that understanding), we cannot disentangle through which of these two potential mechanisms perceptions of importance could be increased. Both of these possibilities, however, are compatible with our argument that these treatments raise the perceived stakes of local elections.
Across all Connecticut municipalities, average spending per capita in 2012 was approximately US$3,400, with a range from US$1,600 to US$7,100. The Budget own spending treatment text, which followed the GOTV language described earlier and emphasized what was at stake in the recipient's own locality, read as follows:
Your voice starts with your vote: In Connecticut, local elected officials make many important decisions that affect your life. One of the most important things they do is decide what to spend money on. In 2012, your city/town's budget was $XXXX per resident. By voting, you make sure local elected officials are thinking about you when they decide how to spend your money. Make sure your voice is heard!
The Budget average spending treatment is identical except that it replaces the town-level figure with the average across all towns (“In 2012, the average city/town budget in Connecticut was $3,354 per resident.”). Unlike the town-level treatment, therefore, this treatment is homogeneous across all towns (that is, it does not present different information depending on actual spending levels in a town/city), which means it is easier to interpret across contexts. We assume individuals use the statewide average to make inferences about spending levels in their own locality, which should increase the perceived importance of local election outcomes.
Finally, the third material resources treatment, Budget comparative spending, is conceptually different. It provides both pieces of information simultaneously along with a direct comparison of these amounts (“In 2012, your city/town's budget was $XXXX per resident, which is ZZ% (higher/lower) than the $3,354 average in all Connecticut cities/towns.”). Theoretically, how this treatment might work is ambiguous. First, it provides the same information about the average and local stakes of government control, but in comparing the two amounts to the state average benchmark, it may make the stakes in a particular city/town appear larger or smaller. If a locality's spending is above the state average, the stakes of controlling those resources are arguably higher because of the larger amount of money at stake, though places with higher spending may be those where citizens are already engaged and active in making demands. On the other hand, if a locality spends less than other towns, the stakes are lower, though citizens may also infer that their city should raise and spend more, which may increase the motivation to participate. Secondly, the treatment may make a locality appear to overspend or underspend compared to others in the state. This could lead to those in places that spend less being mobilized by underspending because they believe government should do more and/or those in places where spending appears too high being mobilized because it suggests government overreach. As such, we do not have clear ex ante predictions about the effectiveness of this comparative treatment relative to the other spending treatments because it mixes both making local government appear important and ambiguous performance judgments.
Tax Treatments
Our other set of treatments focused on tax rates rather than spending behavior. In Connecticut, as with many other states, local property taxes are an important source of revenue, and tax rates vary substantially due to differences in the relative wealth and homogeneity of localities. Our two primary tax treatments of interest present individuals with a comparison of their own tax rates to either a low or high state taxes baseline, which provides information about relative government performance. As with our budget treatments, the effectiveness of these tax treatments is consistent with two potential mechanisms that raise the stakes of participation. First, learning that one's taxes are higher rather than lower may make one see the stakes of government control as higher because it suggests government is doing more. Secondly, under the assumption that high taxes are unpopular because they suggest a greater burden than in other communities, when local taxes are perceived as being higher this may be especially likely to imply a current failure of government. Therefore, our primary expectation is that telling people their tax rates are comparatively higher than elsewhere in the state will increase turnout more.Footnote 10 We also include three additional tax treatments to rule out the possibility that the provision of the component pieces of information in our core tax treatments explains their observed effects (details provided below).
We note that it may also be the case that simply informing individuals about their relative tax burden, even if it is less onerous than elsewhere in the state, induces them to participate because it makes clear the stakes of local elections in setting taxes. That is, across the tax treatments, we hold constant that we provide information about the importance of local elections for setting local taxes and the town's mill rate, potential mechanisms to generically increase the stakes of these contests. In doing so, we can rule out those mechanisms, whereas we cannot do the same for our budget treatments.
To construct the Taxes comparatively higher and Taxes comparatively lower treatments without engaging in deception, we calculated two different measures of the average tax burden in Connecticut. The first, which yields a lower baseline level of taxation, is the average taxes calculated on a town-by-town basis across the state. In 2012, the average mill rate across Connecticut localities was 23.90. Our second measure is the average mill rate individuals experienced in Connecticut, which is higher at 28.18 because more populous towns tend to have higher tax rates. By constructing two measures of relative taxes, we can expose all recipients to messages that make their taxes appear higher or lower, whereas if we relied only on variation across localities, we would have to be concerned that places with higher (lower) taxes would respond to the comparative treatments differently because of some correlation between underlying citizen preferences and observed local tax rates. The complete treatments: (1) describe how taxes are calculated in Connecticut; (2) provide information about the resident's local tax (mill) rate; and (3) provide information about either the higher or lower baseline described earlier. The following is an example of the treatment text that, again, was added to the baseline GOTV content:
Your voice starts with your vote: In Connecticut, local elected officials make many important decisions that affect your life. One of the most important things they do is set the property tax rate, which affects the taxes that people who own real estate pay each year. This tax rate is called the mill rate. Each mill is the number of dollars that a person has to pay for each $1,000 in assessed value for their property (by state law, property is assessed at 70 per cent of its fair market value). Here is some information about local property taxes:
• Local taxes fund more than 72 per cent of local spending in Connecticut.
• [Taxes comparatively higher treatment] In 2012, the average city/town's mill rate in Connecticut was 23.90.
• [Taxes comparatively lower treatment] In 2012, the average person in Connecticut had a mill rate of 28.18.
• In 2012, your city/town's mill rate was XX.XX.
By voting, you make sure local elected officials are thinking about you when they are setting your tax rates. Make sure your voice is heard!
This treatment has many different components apart from the comparison of local tax rates to these different baselines, and so we also implemented three treatments that included each of those components in isolation to rule out the possibility that it was those components that generated any observed effects. These treatments are therefore not included out of theoretical interest, and, in fact, we do not expect these treatments to increase turnout (beyond the baseline GOTV treatment). Instead, they are included to test an assumption that these component parts are not responsible for any observed effects of the Taxes comparatively higher and Taxes comparatively lower treatments (that is, that these component parts only work in the theoretically motivated combination). The Taxes own mill rate is the previous treatment without any reporting of statewide taxes. The Taxes high state average and Taxes low state average are the previous treatment without the reference to the recipient's local taxes.
Data Analysis and Results
Our reporting proceeds in two steps. We first assess the average effect on turnout of receiving the baseline GOTV message so that we can benchmark the effects of the other treatments to this baseline. Next, we examine the absolute and comparative effectiveness of each of our two sets of treatments.
To estimate the effects of our different treatments,Footnote 11 we use ordinary least squares (OLS) regression with controls for the sixteen assignment strata discussed earlier (as indicators). As we varied the probability of being selected based on strata, treatment assignment rates are conditional on strata, and thus strata fixed effects are necessary when estimating average treatment effects across strata to avoid bias (see, for example, Duflo, Glennerster, and Kremer Reference Duflo, Glennerster and Kremer2007; Gerber and Green Reference Gerber and Green2012).Footnote 12 We also include town-level fixed effects (as indicators) and use weights to account for the different rates of assignment to treatment across treatments and strata.Footnote 13 Our most comprehensive specification is therefore:
$$Y = G \times {\rm Strata} + H \times {\rm Town} + K \times {\rm Controls} + B_0 + B_j \times {\rm \;Treatmen}{\rm t}_j + e$$which we estimate with robust standard errors. In the equation, the subscript j means that the coefficient Bj is the estimated effect of being sent treatment j relative to not being contacted. Additionally, we can compare treatment effects across treatments (relative to one another rather than to the uncontacted control group) by conducting linear combination-of-coefficients tests.
Results of our analysis appear in Table 3. Focusing just on the result for the baseline GOTV message, per the column 1 specification, we see that being sent this message is associated with a 1.4-point increase in turnout (p = .05, two-tailed). In substantive terms, baseline turnout (see third-from-bottom row of Table 3) is 31 per cent, which means the average effect of the GOTV message is to increase participation by about 4.3 per cent. The column 2 specification adds town-level fixed effects to account for average turnout across towns, while the column 3 specification also includes individual-level covariates as measured from the pre-treatment voter file (gender, partisanship, year of registration, and number in household).Footnote 14 These covariates slightly shrink the estimate to 1.3 points (p = 0.06, two-tailed) and 1.2 points (p = 0.07, two-tailed), respectively. We use the column 3 specification as our preferred specification because it accounts for the most potential sources of unobserved heterogeneity and is therefore a conservative benchmark. As is obvious from direct inspection of the different treatment coefficients across columns, the choice of specification has relatively minor effects on the substantive interpretation of our results, but several results have p-values close to the traditional frequentist benchmark of 0.05. We thus report all three specifications for the sake of transparency.
Table 3. Effect of mail treatment about election stakes on turnout
Notes: Cell entries are OLS coefficient estimates using inverse probability weighting (see text for details) with robust standard errors in parentheses. * p < 0.1; ** p < 0.05; *** p < 0.01, two-tailed tests. Columns 4–6 use Bonferroni-corrected alpha thresholds: * p < 0.02; ** p < 0.01; *** p < 0.002, two-tailed tests.
Next, we examine the effects of the novel treatments that sought to increase participation by raising the stakes of local elections. To preview our findings, we find that discussing local government spending (either one's own government or average government spending) and making one's tax burden seem comparatively higher increase turnout by between 2.7 and 4.2 points versus not being sent a letter. Compared to the baseline GOTV message (the effect of which we discuss in the previous paragraph), these treatment effects are larger (by between 1.5 and 2.9 points), but only the difference for the Taxes comparatively higher treatment reaches standard levels of statistical significance (in part, due to modest treatment letter sample sizes, especially compared to the extremely large no-contact control group).
Emphasizing the Budgetary Implications of Local Elections Increases Turnout
We first focus on the comparative effectiveness of the three treatments emphasizing the control of local resources at stake in local elections. As a reminder, the Budget average spending treatment is identical across localities, while the Budget own spending treatment reports a different figure depending on where people live. The Budget comparative spending treatment compares these two figures and so varies in content (and meaning) across localities.
Per column 3, both the Budget own spending and Budget average spending treatments are estimated to meaningfully increase participation in local elections by roughly similar magnitudes of 2.7 (p < 0.05, two-tailed) and 2.9 (p < 0.05, two-tailed) points compared to not being sent any mail. Notably, these coefficient estimates are at least twice as large as the estimated effect of the baseline GOTV treatment in this specification (1.2 points), the content of which is included in these treatments. At the same time, these differences compared to the GOTV message are not statistically significant at conventional levels: a linear combination-of-coefficients test of whether these differences are statistically distinguishable from the baseline GOTV effect in isolation yields one-sided p-values of 0.15 and 0.11, respectively. If we instead compare the average effect of the two budget treatments to the GOTV treatment, the estimated average effect is 2.8 percentage points (p < 0.01), and the 1.6-point difference with the GOTV message has a one-sided p-value of 0.07. While it would be ideal for ruling out sampling variability if the standard errors associated with these treatment effects were smaller, the substantive fact that the point estimates imply these treatments are twice as effective as the baseline GOTV message is encouraging, and, moreover, these messages are clearly effective compared to not being contacted at all.
In contrast to the clear effectiveness of emphasizing the resources at stake in local elections by noting one's own budget size or the state average, the Budget comparative spending treatment appears completely ineffective on average. This message has a point estimate near 0 (0.2 points), which is both substantively unimportant and worse than the baseline GOTV treatment delivered in isolation. Lacking clear theoretical expectations about the effects of this treatment across town spending levels, we investigated whether there was heterogeneity by town-level spending levels (an observational partitioning) and found no evidence that the treatment was more or less effective in places with lower or higher levels of spending (for details, see Table S3 in the Online Supporting Information). As we noted earlier, emphasizing a comparison may invoke different considerations for different people, and the ineffectiveness of this treatment is compatible with multiple effects, such that any mobilization effect is offset by a demobilization effect for other potential voters, or even simple sampling variability. Setting this aside, the key finding from this subsection is that the simple presentation of local or state average spending, which signals that local government controls a great deal of money, appears to increase participation in local elections.
Emphasizing High Local Tax Burdens Increases Turnout
If highlighting the spending side of the budgetary implications of local elections is a potentially effective way to increase turnout, are treatments that emphasize local taxes effective, particularly if taxes are higher than elsewhere? The results presented in column 3 provide initial evidence that the comparative tax treatments are effective in increasing participation, particularly when those comparisons make local taxes appear higher.
In particular, the Taxes comparatively higher treatment, which compares taxes to a lower state baseline and therefore makes local taxes appear higher, is associated with a 4.2-point increase in turnout (p < 0.01). This effect is 2.9 points larger (240 per cent) than the 1.2-point effect of the baseline GOTV treatment (p < 0.05, one-tailed test of difference of coefficients) and 2.3 points larger (125 per cent) than the 1.8-point effect of the Taxes comparatively lower treatment. The difference in the effects of the two tax treatments has a one-sided p-value of 0.14 (reflecting, in part, the smaller number of individuals assigned to each tax treatment than to the GOTV message). (The difference between the Taxes comparatively lower treatment and the GOTV message is small, at 0.6 points, and not significant.) Overall, a message emphasizing high local tax burdens appears promising for increasing turnout—it produces an estimated increase in turnout of approximately 13 per cent over baseline voting rates.Footnote 15
One concern with our interpretation of these results is that presenting either one's own mill rate or the different baseline statewide tax rates explains the combined effect of the mill rate and the comparison with a low state benchmark. However, the relative effectiveness of those treatment components in isolation shows these concerns are not warranted. Simply telling someone their mill rate, without any contextual information about what property is taxed and at what basis (in Connecticut, property is taxed at 70 per cent of fair market value) or whether that figure is high or low (Taxes own mill rate), depresses turnout by 1.4 points relative to no contact at all. Similarly, telling people that state taxes are low (Taxes low state average) has a very small estimated effect on turnout (0.04 points, not significant).Footnote 16 As these are the two key components that are combined to produce the Taxes comparatively higher treatment, these estimates help us rule out an argument that these components in isolation explain that effect.Footnote 17 Overall, this set of treatments provides promising results that messages that make local taxes appear higher are effective in increasing participation in local elections.
Adjusting Statistical Tests for Multiple Testing
One potential concern with the results discussed earlier is that we do not adjust the statistical tests to account for multiple testing. Specifically, because we have five theoretically motivated treatments (Budget own spending, Budget average spending, Budget comparative spending, Taxes comparatively higher, and Taxes comparatively lower), we essentially have five chances to identify a statistically significant relationship.Footnote 18 To address this concern, in columns 4–6 of Table 3, we present our statistical significance tests using the Bonferroni-corrected p-value threshold for tests using a 5 per cent alpha level (which is 0.05/5, or 0.01). Using this (perhaps overly) conservative threshold, both the Budget average spending and Taxes comparatively higher treatment effect estimates achieve statistical significance at the 0.05 level (one-sided p-value) compared to the control (see column 6). Using the same adjusted alpha for the comparison to the GOTV message, neither treatment achieves statistical significance (one-sided p-values of 0.04 and 0.11, respectively).
Additionally, we can also test whether these five treatments, when pooled, have effects that are statistically significantly larger than either the untreated control group or the baseline informational treatment. Using the column 3 specification from Table 3, we estimate the average effect of the five theoretically motivated treatments is to increase turnout by 2.4 points [standard error (SE) = 0.62 points, p < 0.01], an effect that is 1.1 points larger than the estimated 1.2-point effect of the informational baseline. The difference between the average effect of these treatments and the information baseline is not statistically significant (SE = 0.91 points, p = 0.10, one-tailed). As such, while the totality of the evidence presented earlier is consistent with our expectations, issues related to statistical imprecision, sampling variability, and concerns about false positives mean that uncertainty remains about the effectiveness of these sets of mobilization messages.
Conclusion
A central message of political campaigns is the substantial stakes of the contest in question. Campaigns, candidates, and parties all go to great lengths to emphasize the crucial significance of an election's outcome as a means to highlight the importance of showing up to the polls on Election Day. Yet, despite this strategy being a ubiquitous approach taken by political actors, scholars have generally ignored this approach when assessing the utility of various means to increase turnout. As such, there is little field-experimental work testing the effectiveness of these types of messages (outside of partisan/advocacy appeals) and none that attempt to benchmark the effects of these messages against a standard GOTV message.
We address this empirical oversight by conducting, to our knowledge, the first large-scale voter mobilization field experiment that tests the extent to which efforts to alter the perceived stakes of an election (beyond simply stating the election is important) can successfully activate registrants. Our findings suggest that such outreach can engage citizens. Treatments that emphasized the resources controlled by local governments (that is, their budgets) and the comparatively high tax burdens imposed upon residents were estimated to increase turnout compared to receiving no contact. Several of these messages exerted substantively important effects that were noticeably larger than a simple nonpartisan GOTV letter, though these differences were often borderline statistically significant, likely due to power considerations, leaving concerns about sampling variability.
Our results have a number of important implications. Perhaps most notably, they provide causal evidence that messages emphasizing the concrete stakes of an electoral contest may influence the decision to vote. Considerable prior work speculates that ignorance of an election's consequences raises the probability of abstention, but those analyses are limited in their ability to demonstrate a causal link in this relationship. Our approach, in contrast, shows that messages that should lead to perceiving greater electoral stakes increased participation. Furthermore, our experiment reveals that these perceptions are remediable. Simple outreach can clarify why a contest's outcome matters, and that clarification not only seemingly enhances one's understanding of why what government does is important, but also translates into a higher propensity to vote. That these messages exert a larger impact on turnout than a standard GOTV message signals that affecting these perceptions is a potentially powerful strategy to raise participation rates.
That said, the exact mechanism through which these messages increase turnout is still unclear. Our messages highlight the stakes of an election, but that can cause greater participation through a variety of causal pathways that implicate different theoretical mechanisms and inform different models of the decision to vote. Distinguishing among these competing accounts is beyond the scope of this article, but we note that doing so would require additional measurement of what sets of beliefs and motivations are implicated by our treatments. For the moment, the key point is that such material incentives appear to matter for voting in a way that prior field-experimental work has neglected.
While our results are derived from a large-scale field experiment, there are nonetheless reasons for caution in interpreting our results. Some relate to standard caveats about whether effects would arise in different electoral contexts, but we also note that despite the size of our experiment, the large number of treatments we tested means that we still have imprecision in our estimates. These apprehensions, which raise the possibility of false positives, are a greater concern when comparing the effects of our novel treatments to the baseline GOTV message because the differences in effects are smaller and standard errors are larger because fewer treatment letters were sent for the non-GOTV messages. For this reason, it would be desirable to replicate these treatments on an even larger scale and in different contexts, with sufficient power to accurately discern relative treatment differences of perhaps 1 point (a proportionally large effect that is at least 71 per cent of the effect of the GOTV treatment). As such, future studies should build upon our promising findings to speak definitively about the link between the stakes of elections and the decision to vote.
Supplementary Material
Online appendices are available at: https://doi.org/10.1017/S0007123421000600
Data Availability Statement
Replication data for this paper can be found at the journal's Harvard Dataverse page at: https://doi.org/10.7910/DVN/JNGCZT
Acknowledgments
We thank J. Andrew Harris, the anonymous reviewers, and the editor for their helpful comments and feedback. All errors are our own.
Financial Support
None.
Competing Interests
None.
Ethical Standards
The study was ruled exempt by the Yale University Human Subjects Committee (Protocol #:1310012890).
Open access
