Hostname: page-component-78c5997874-t5tsf Total loading time: 0 Render date: 2024-11-06T04:55:16.331Z Has data issue: false hasContentIssue false

Is it worth door-knocking? Evidence from a United Kingdom-based Get Out The Vote (GOTV) field experiment on the effect of party leaflets and canvass visits on voter turnout

Published online by Cambridge University Press:  03 October 2018

Joshua Townsley*
Affiliation:
School of Politics and International Relations, University of Kent, Canterbury, UK
*
*Corresponding author. Email: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

What impact do party leaflets and canvass visits have on voter turnout? Get Out The Vote (GOTV) experiments consistently find that campaigning needs to be personal in order to be effective. However, the imbalance between United States and European-based studies has led to recent calls for further European GOTV experiments. There are also comparatively few partisan experiments. I report the findings of a United Kingdom-based field experiment conducted with the Liberal Democrats in 2017. Results show that party leaflets boost turnout by 4.3 percentage points, while canvassing has a small additional effect (0.6 percentage points). The study also represents the first individual level experiment to compare GOTV effects between postal voters and in-person voters outside the United States.

Type
Original Articles
Copyright
© The European Political Science Association, 2018 

Introduction

There is an ongoing discussion over how levels of political participation are affected by various party campaign activities. The literature in the United States is particularly well developed. Get Out The Vote (GOTV) experiments show that campaigning has strong mobilizing effects when it involves face-to-face contact with voters (Gerber and Green Reference Gerber and Green2017). The precision-weighted effect on turnout of common campaign activities is summarized in Table 1 based on Green, McGrath and Aronow (Reference Green, McGrath and Aronow2013) meta-analysis review. The effect of face-to-face canvassing is around 2.5 percentage points, while other, less personal activities such as commercial phone calls and leaflets have a smaller effect. US experiments tend to produce mixed results on the effect of partisan treatment, however (e.g., McNulty Reference McNulty2005; Nickerson, Friedrichs and King Reference Nickerson, Friedrichs and King2006; Panagopoulos Reference Panagopoulos2009). Green, McGrath and Aronow (Reference Green, McGrath and Aronow2013) conclude from their review of leaflets and direct mail that partisan messages actually have a (statistically insignificant) negative effect of −0.213 percentage points on voter turnout.

Table 1 Effect of COMMON Get Out The Vote (GOTV) Tactics (Green, McGrath and Aronow Reference Green, McGrath and Aronow2013)

Note:

a The effect of leaflets and direct mail are the Intent to Treat effect, which represents the difference in turnout between those randomly assigned to treatment conditions compared to those assigned to the control conditions. The effects of face-to-face canvassing and phone calls are the Complier Average Causal Effects, which takes into account the one-sided non-compliance among subjects (Green, McGrath and Aronow Reference Green, McGrath and Aronow2013).

b “Conventional mailings” refer to leaflets that do not contain any social pressure messaging, which are shown to have stronger effects (see Green, McGrath and Aronow Reference Green, McGrath and Aronow2013 for a summary).

However, there is a sharp imbalance in the literature between United States and European-based studies (Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016; Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2017; Foos and de Rooij Reference Foos and de Rooij2017; John Reference John2017). Emerging evidence in Europe is seeking to rectify the imbalance (e.g., Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons and Liegey Reference Pons and Liegey2018; Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016; Cantoni and Pons Reference Cantoni and Pons2016; Foos and John Reference Foos and John2016; Pons Reference Pons2018; Nyman Reference Nyman2017)—and provides evidence that existing findings do not generalize well to non-US contexts. Bhatti et al.’s (2016) meta-study of nine canvassing experiments across six European countries finds that canvassing, for example, has a smaller effect—a precision-weighted effect of 0.78 percentage points, compared to 2.54 percentage points in the United States. There is good reason, therefore, to further investigate the effect of GOTV activities in Europe. Given the limited number of existing studies, scholars call for further campaign experiments to continue to redress the geographical imbalance (Nielsen Reference Nielsen2014; Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016).

While the European GOTV literature is growing, there are still comparatively few partisan experiments. In the United Kingdom, beyond a handful (Foos and John Reference Foos and John2016; Foos and de Rooij Reference Foos and de Rooij2017), existing GOTV experiments have focused on non-partisan interventions (e.g., John and Brannan Reference John and Brannan2008; Fieldhouse et al. Reference Fieldhouse, Cutts, Widdop and John2013). An experiment in Sweden found that party canvassing increases turnout by 3.6 percentage points (Nyman Reference Nyman2017), while Cantoni and Pons (Reference Cantoni and Pons2016) find an effect size of around 2 percentage points in France. Meanwhile, experiments that randomize party contact at the aggregate level often show null effects (Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons and Liegey Reference Pons and Liegey2018; Pons Reference Pons2018). In their UK experiment, Foos and John (Reference Foos and John2016) find that partisan leaflets and canvass visits mobilize supporters, but not overall turnout. Foos and de Rooij (Reference Foos and de Rooij2017) find that partisan phone calls boost turnout by 2–3 percentage points, but that calls with a stronger partisan tone had stronger effects on supporters. There is emerging evidence, therefore, of “differential mobilization”—whereby campaigns mobilize supporters but demobilize supporters of rival parties (Foos and John Reference Foos and John2016). But the comparatively small number of partisan experiments in Europe mean there is considerable uncertainty over these effects (Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016).

The first contribution of this study is to further redress the imbalance between United States-based and European-based GOTV experiments, and to present more evidence on the effect that party contact has on turnout. I report the results of a field experiment involving 6525 voters, conducted in co-operation with the Liberal Democrats during the English local elections in 2017. Unlike existing UK experiments, the study provides a test of whether party campaigning can affect turnout in a politically unfavorable area with relatively few existing supporters to mobilize. The design also deviates from most non-partisan GOTV studies by testing the effect of leaflets and the additional effect of a canvass visit, rather than directly comparing the two. In so doing, I aim to more closely replicate real party campaigns—that tend to use leaflets widely—and use canvassing more sparingly in addition to, rather than in instead of leaflets (Fisher et al. Reference Fisher, Cutts, Fieldhouse and Rottweiler2016). Results show that while the leaflet and canvass combination boosts turnout by 4.9 percentage points, leaflets alone also have a substantive impact on turnout of 4.3 percentage points. The additional effect of the canvass visit (0.6 percentage points) is statistically insignificant from the effect of the leaflet alone. While this experiment does not directly compare the two activities, it does find that party campaigning does not necessarily have to be personal in order to affect turnout.

In addition to examining overall effects, GOTV research has also looked at which types of voters are most affected by campaign contact. Arceneaux and Nickerson (Reference Arceneaux and Nickerson2009) found that across 11 GOTV campaigns, those closest to their “indifference threshold” (between voting and abstaining) were more likely to be mobilized than those who are either certain to vote or certain to abstain. An assessment of 24 experiments by Enos, Fowler and Vavreck (Reference Enos, Fowler and Vavreck2014) also finds that low propensity voters become increasingly easier to mobilize as the saliency of the election rises, but that at a typical election, high propensity voters are more likely to be mobilized. But few have tested whether campaign effects vary according to whether subjects are registered to vote by post, or in person. This represents an important gap in existing research. Due to their high turnout rate, postal voters are a major source of potential support for political parties. In US states such as Oregon and Washington, for instance, almost all votes are cast by mail (Alvarez, Beckett and Charles Stewart, Reference Alvarez, Beckett and Stewart2012). In the United Kingdom, one in five of the total votes cast at general elections are cast by mail (Rallings and Thrasher Reference Rallings and Thrasher2015), and around one in three of votes cast in local elections (Electoral Commission 2013). But we know little about whether the mode through which voters cast their ballots impacts on the effectiveness of parties’ campaigning.

The reasons for this are largely logistic. Postal voters and in-person voters tend to cast their ballots at different times, making it problematic to time experiment treatments. As a result, postal voters are usually excluded from experimental populations prior to randomization, or it is assumed that the way subjects cast their ballots is constant (Arceneaux, Kousser and Mullin Reference Arceneaux, Kousser and Mullin2012). The question of whether effects differ between the two has been subject to an aggregate-level natural experiment before (Arceneaux, Kousser and Mullin Reference Arceneaux, Kousser and Mullin2012), which found that mobilization efforts are less effective in areas with high levels of postal voting (or “vote-by-mail” in US terminology). The second contribution of this paper, therefore, is to present the first individual-level GOTV experiment that compares effects between postal voters and non-postal voters, to my knowledge. The experiment is designed in such a way as to replicate treatment within the same election campaign in the same area among (1) households containing postal voters and (2) non-postal voter households. I find that leaflets and canvass treatments boost turnout among in-person voters but has negligible effects on postal voter households.

Theory and expectations

Despite considerable GOTV research, the precise mechanism that drives the effect of powerful GOTV interventions is still unclear (Broockman and Kalla Reference Broockman and Kalla2016). Prominent scholars have argued that in order to establish a “social connection” with voters, campaign contact requires a “personal touch” that leaflets cannot offer. Mobilizing a voter, they argue, is akin to “inviting them to a social occasion” (Green and Gerber 2004, 92), and personal contact signals a campaign’s “willingness to devote time and energy” to voters (Green and Gerber 2015, 38). Alternative explanations suggest that it is the noticeability of a campaign intervention that drives its effect, as registered voters only require a reminder (Dale and Strauss Reference Dale and Strauss2009). These explanations are often associated with non-partisan contact, however, and existing research suggests that the mechanism connecting partisan contact to turnout is different.

First, we might expect party campaign appeals to reduce costs for voters. We know that, on the whole, voters’ capacity for processing political information, particularly in comparison to the other day-to-day information, is relatively limited (Mondak Reference Mondak1993). To process political information, voters make use of heuristics and cues as “shortcuts” to aid decision-making (Sniderman et al. Reference Sniderman, Richard, Philippe and Tetlock1991). While non-partisan campaigns might appeal to “civic duty” or exert social pressure (Gerber and Green Reference Gerber and Green2017), partisan campaigns present party cues to voters (Foos and de Rooij Reference Foos and de Rooij2017). Party labels, names, or messages, mean voters can more easily associate the candidate in question with their pre-formed opinions about that political party and their policies (Green, Palmquist and Schickler Reference Green, Palmquist and Schickler2002). This, in turn, helps to reduce the costs associated with making the decision over whether to support the candidate, support an alternative candidate, or not to vote at all.

As well as reducing costs, party contact can also affect the benefits associated with voting (Foos and de Rooij Reference Foos and de Rooij2017). Evidence shows that for individuals who are inclined to support the party, partisan contact can activate the expressive benefits that can be associated with turning out in support of that candidate (Green, Palmquist and Schickler Reference Green, Palmquist and Schickler2002). Partisan experiments in the United Kingdom also find evidence that the effect of party contact is related to the recipients’ pre-existing party identity (Foos and John Reference Foos and John2016; Foos and de Rooij Reference Foos and de Rooij2017). But these experiments have been conducted in areas in which the party is a dominant—or at least major—player with a significant base of underlying support to mobilize. In a less favorable local context, with fewer party supporters to mobilize, we might expect campaign contact to trigger pre-existing party identities to turnout against the party in question (Foos and de Rooij Reference Foos and de Rooij2017).

How would we expect the effects of party campaign activities to differ between postal voters and non-postal voters? Existing research on who is most influenced by GOTV mobilization paints a mixed picture. Earlier evidence suggests that those who are closer to their “threshold of indifference” between voting and abstaining are less responsive to campaign effects (Arceneaux and Nickerson Reference Arceneaux and Nickerson2009). Individuals close to the threshold are more likely to be mobilized to actually cast their ballots. Given that postal voters in the United Kingdom have a much higher rate of turnout than non-postal voters, especially at local elections (Rallings and Thrasher Reference Rallings and Thrasher2014), they are further away from their “indifference threshold” and thus less susceptible to additional mobilization efforts. More recent evidence shows that higher propensity voters are more likely to be influenced by GOTV campaigns (Enos, Fowler and Vavreck Reference Enos, Fowler and Vavreck2014). However, given the very high turnout rate of postal voters in the United Kingdom we would expect a “ceiling effect”, where the impact of campaign mobilization efforts is limited by the fact that most of the target group vote already. While this question has not been a topic of much experimental research, it echoes Arceneaux, Kousser and Mullin’s (Reference Arceneaux, Kousser and Mullin2012) aggregate level experiment in the United States that finds that voters in vote-by-mail areas are less affected by GOTV efforts than those in traditional voting areas. Based on these findings, we would anticipate that at the individual level, the effect of campaign contact on voter turnout will be lower among postal voters.

Research design

I conduct a randomized field experiment with the Liberal Democrats at the English local elections in 2017. The study population comprised registered postal voters and non-postal voters in the electoral division of Thedwastre North in Suffolk, England. At the 2013 local elections, turnout in Thedwastre North was 31.3 percent, compared to 31 percent nationally (Electoral Commission 2013). The area is a relatively pro-ConservativeFootnote 1 one, and the local authority supported leaving the European Union in the 2016 referendum, by 55–45 percent (BBC 2016). The setting therefore differs from those of similar partisan studies in the United Kingdom, which have tended to take place in areas that are relatively favorable politically (Foos and John Reference Foos and John2016; Foos and de Rooij Reference Foos and de Rooij2017). Taken from the local party’s voter database, the study population totaled 6525 registered voters in 3371 households.

Clustered random assignment

As the household was the unit of randomization and contact, voters were clustered into their households. In order to compare the effect of treatment between postal voters and non-postal voters, the former needed to be separated from the latter. This was not straight forward, however, as postal voters do not all live in the same household as other postal voters. Therefore, households were first separated based on whether or not they contained postal voters. Households containing at least one postal voter (i.e., households comprised entirely of postal voters, or a mix of postal voters and non-postal voters) were placed into the postal voter component of the experiment. Most subjects (76 percent) in this half of the experiment were postal voters. These “postal voter households” were then randomly assigned to either a control group, or one of two treatment groups. Meanwhile, the remaining households (all of which contained only non-postal voters, and no postal voters) were placed into the “non-postal voter household” component of the experiment, and randomly assigned to separate control or treatment groups.Footnote 2 The result was six randomly assigned groups: a control and two treatment groups in the postal voter household experiment (containing mostly postal voters—76 percent), and a control and two treatment groups for the “non-postal voter household” experiment. This process is summarized in Figure 1.Footnote 3

Fig. 1 Field experiment research design process Note: The relative sample sizes of the different groups were determined based on how many households we (the researcher and the local party), thought we could realistically contact. This was based on the timeframe and the personnel available at different times during the campaign. We wanted to ensure treatments were carried out two weeks prior to postal votes being sent out, and polling day, for postal voter households and non-postal voter households, respectively. We knew that there were fewer postal voter households than non-postal voter households, but we had approximately the same amount of time to contact each. Therefore, we decided to assign a larger proportion of postal voter households, which we were contacting during the first half of the campaign, to the treatment groups (534 out of 676 households, compared to 1179 out of 2695 for non-postal voter households).

The nature of the research design has empirical ramifications regarding what the experiment is testing. First, the cluster-randomized design cannot test the turnout effect of direct exposure to the treatments. Rather, any effects could also be the result of downstream effects within the household (see, for instance, Nickerson Reference Nickerson2008; Cutts and Fieldhouse Reference Cutts and Fieldhouse2009; Fieldhouse and Cutts Reference Fieldhouse and Cutts2016), caused by an individual being taken to the polls by their partner after they spoke to a canvasser. The design can estimate effects at the household level. Second, while all non-postal voter households contain entirely non-postal voters, some of the postal voter households contain non-postal voters (24 percent of subjects in the postal voter household experiment were not registered postal voters). Given this, the comparison between the two can only reveal how voters in households containing postal voters react to treatment, compared to non-postal voters.

Treatments

All treatments were partisan in nature and carried out by the researcher or a volunteer. Treatments were kept as “authentic” (i.e., as stylistically close to what the local campaign would do anyway), as possible. The first treatment was a Liberal Democrat party leaflet (Figure 2). A fortnight was spent touring the Thedwastre North area with the Liberal Democrat candidate to collate photographs and stories to populate the leaflet. The leaflet was printed in full color A3, and contained the legally required party imprint, as well as Liberal Democrat party branding. The leaflet contained numerous local policy issues and manifesto commitments covering issues that are under the remit of the council for which the election was being held, including libraries, transport and schools. This differed from the leaflets used in Foos and John’s (Reference Foos and John2016) similar UK party experiment, where the Conservative leaflet delivered was a “sorry we missed you” flyer (an image of this leaflet is in the Online Appendix for comparison; Figure A4). The aim was to test the impact of standalone leaflets that contain more information on the upcoming election. Party leaflets typically contain information about local issues and the candidate. Evidence shows that voters often respond to photographs and other personal candidate attributes (Mattes and Milazzo Reference Mattes and Milazzo2014), so the leaflet included photographs of and information about the candidate and colorful, eye-catching information about the issues at stake at the election.

Fig. 2 Lib Dem leaflet

While one treatment group was assigned to receive a Liberal Democrat leaflet, while the other group received the same leaflet at the same time, followed by a canvass visit from a volunteer. In all cases, the canvass visit proceeded the leaflet delivery by fewer by three days. The control group was not directly exposed to any Liberal Democrat campaign treatment. The canvass scripts (Boxes 1 and 2) were partisan in nature, again aiming to replicate personal visits carried out by campaigns at local elections. Canvassers were instructed to deliver the script, but not to stick rigidly to it, and to be as friendly as they liked. In sum, a random sample of households received either (a) a leaflet, or (b) a leaflet followed by a doorstep visit from a party activist. As this second treatment group received the same party leaflet, in essence, the study is designed to examine what, if any, additional effect a canvass visit has on turnout on top of a party leaflet.

Box 1 Canvass script for postal voter households

“Hello, my name is ______, I am a volunteer for the local Liberal Democrats. I’m just calling round to remind you that the Suffolk County Council elections are coming up, and that it’s important that you have your say in how our area is run.

The local Lib Dem candidate, Jon James, will be a strong voice for our area, and will fight to improve local transport and health services.

According to our records, you are registered for a Postal Vote. Your Postal Vote papers should be arriving over the next few days, so just keep an eye out for those, then you can vote, send it off, and you’ve had your say.

Thank you for your time, have a good day”

Box 2 Canvass script for non-postal voter households

“Hello, my name is ______, I am a volunteer for the local Liberal Democrats. I’m just calling round to remind you that the Suffolk County Council elections are coming up, and that it’s important that you have your say in how our area is run.

The local Lib Dem candidate, Jon James, will be a strong voice for our area, and will fight to improve local transport and health services.

Your local polling station is located at ____________.

Thank you for your time, have a good day”

In order to compare the effects among postal voter households, and non-postal voter households, they both received the same treatments. This presented difficulties in terms of timing, as postal voters are sent their ballots a few weeks prior to election day in order to allow enough time to vote. Meanwhile, non-postal voters vote on election day itself. To take this into account, the same treatments were applied to both, but at different times. Postal voter households were treated within the two-week period prior to their postal votes being sent out by the local (17 April). Leaflets were delivered between the 4 and 14 April, while those assigned to the canvass group were canvassed between 10 and 16 April. All contacts were therefore made just before postal voters received their ballots. Non-postal voter households began receiving their treatments a week later, all within the two-week period leading up to polling day. It is possible that the differential timing of the treatment could mean that turnout-inducing events might affect one type of household and not the other, thereby confounding the treatment method and time. However, given that both the postal voter and non-postal voter components of the experiment had their own control groups, the experiment is designed to measure the difference in turnout between the treatment groups and their respective control groups, to allow effect comparisons.

Contact rates and non-compliance

Table 2 shows the final sample sizes and contact rates. All leaflets in the postal voter household experiment, and the non-postal voter experiment were successfully delivered, meaning the leaflet contact rate (compliance being defined as a leaflet being successfully delivered to the household) was 100 percent. In total, 534 postal voter households, and 1179 non-postal voter households, received a leaflet. However, like most experiments involving canvass visits, one-sided non-compliance was an issue among those living in households assigned to receive a canvass visit in addition to the leaflet. Among this group, compliance is defined as the voter living in a household in which at least one person opened the door and interacted with the canvasser. In total, 262 postal voter households, and 595 non-postal voter households were assigned to receive a canvass visit in addition to a leaflet. The successful canvass contact rate was 28 percent. This is toward the lower end of the range found in other European canvassing experiments (e.g., Bhatti et al. (Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016) had a contact rate of 22 percent in their Labour Market Union, while Nyman (Reference Nyman2017) had a contact rate of 60 percent). The rate is also lower than the 40 percent found in the other UK party experiment involving door-to-door canvassing (Foos and John Reference Foos and John2016).Footnote 4 The low contact rate reduces the reliability of the canvass treatment effects, which is discussed further in the following section.

Table 2 Sample Size and Contact Rates by Treatment Group

Balance test and data assumptions

Randomization ensures that observable and unobservable characteristics of the experimental subjects should be balanced between treatment and control groups (Gerber and Green Reference Gerber and Green2012). Table 3 presents the balance of available individual-level characteristics across the treatment and control groups. This data were obtained through the local Liberal Democrats’ voter ID database. As Table 3 shows, the covariates do not vary appreciably between assignment groups, indicating that the randomization process was successful.Footnote 5

Table 3 Balance of Pre-Treatment Covariates Between Assignment Groups (% (n))

Note: Voted in 2009 most recent turnout data available in Thedwastre North. While more recent turnout data would be preferable, it is the balance between the groups that is crucial.

Analysis and results

Turnout data at the 2017 local election was obtained from the local council’s official records released after the election and matched to the experimental records. The results for the full sample, and for the postal voter and non-postal voter experiments separately, are presented in Table 4. The three models show the estimated Intent-To-Treat (ITT) effectsFootnote 6 of (1) Lib Dem campaigning, (2) a leaflet only, and (3) a canvass visit after a leaflet. The models are estimated using linear regression (standard errors are clustered at household level). The covariate-adjusted effects adjust for pre-treatment covariates in a multivariate linear regression framework. Adjusting for covariates has the advantages of correcting for any minor imbalances in the assignment process and increasing the precision of the estimates, while still producing unbiased effect estimates (Gerber and Green Reference Gerber and Green2012, 109). The covariate data were taken from the party’s voter database, and includes sex, party support, ward, postal voter registration, previous turnout, and age group. I focus on the covariate-adjusted estimates in the interpretation due to their greater level of precision, but both are presented. In order to illustrate the effects visually, I also present the predicted marginal (covariate-adjusted) effects of the treatments in Figure 3.

Fig. 3 Covariate-Adjusted Predicted Intent-To-Treat Effects of Treatments on Voter Turnout Note: the solid lines represent 95% confidence intervals.

Table 4 Experiment Results

Note: Robust standard errors (in parentheses) clustered on households.

All tests two-tailed.

***p<0.001, **p<0.01, *p<0.05, +p<0.1.

The study was pre-registered and included a pre-analysis plan.Footnote 7 The experiment was conducted in accordance with the pre-registration with no substantive deviations on the research design itself. There were two deviations concerning the theoretical discussion and analysis that were made due to data availability issues. These are outlined and explained in the Online Appendix (A5).

All effect sizes that are statistically significant at conventional levels of at least 95% confidence are highlighted in bold for ease of interpretation. Looking first at Model 1, subjects that were exposed to the Liberal Democrat campaign—in any form—were 3.6 percentage points more likely to vote, significant at the 95% level. This effect is stronger among the non-postal voter households, for whom the campaign increased turnout by 4.6 percentage points.

Models 2 and show present the effects of the two treatments. Model 2 shows the effect of the leaflet, which increased turnout by 2.8 percentage points across all voters, though this is statistically insignificant at conventional levels. Again, this effect is stronger among non-postal voting households at 4.3 percentage points. Model 3 shows the effect of the canvass visit and leaflet combination, which boosted turnout by 4.5 percentage points, significant at the 95% level. This appears to suggest that while leaflets alone are ineffective, the leaflet and canvass visit combination has strong effects on voter turnout. Though this is only the case among the full sample. Among subjects in the non-postal voter household experiment, the leaflet only treatment increased turnout by 4.3 percentage points. The effect of the leaflet plus canvass visit is larger—4.9 percentage points. Among postal voter households each treatment had negative and statistically insignificant effects.

Canvassing experiments typically report the Complier Average Causal Effect (CACE) estimator, to account for one-sided non-compliance in the canvass treatment group. Across the pooled groups, the CACE is 6.2 percentage points, with a 95% confidence interval of −8.2 to 20.6.Footnote 8 The confidence intervals surrounding the CACEs for the postal voter and non-postal voter household experiments are also wide. Among postal voter households, the CACE is 16%, with a 95% confidence interval of −10.4 to 42.5. Among non-postal voter households, the CACE is 2.3%, with a 95% confidence interval of −15.1 to 19.7. The low contact rate (28%), as well as the sample size, contribute to the uncertainty surrounding the CACE of the canvass treatment. As such, reliable inferences of the additional effect of canvassing cannot be made with a meaningful amount of statistical power, which is also reflected in the lack of statistical significance of the effects.

The results suggest that the campaign affected postal voter households and non-postal voter households differently. Among non-postal voters, the effect of being assigned to receive some form of Liberal Democrat campaign contact (Model 1) was to increase turnout by 4.6 percentage points, significant at the 99% level. While postal voting households turned out at a far higher rate, the results suggest the Lib Dem campaign did not increase turnout. Among postal voting households, the campaign had a negative, but statistically insignificant, effect on voter turnout. Similarly, each treatment had negative, but insignificant effects on turnout.

Overall, the results show that the Lib Dem campaign had an effect on turnout that was not unsubstantial. Though, results suggest that this was driven largely by the effects among non-postal voter households rather than postal voting households, who turned out at a much higher rate anyway. Interestingly, the leaflet was enough to increase turnout among non-postal voting households, while the evidence surrounding the additional effect of canvass visits is limited given the low contact rate.

Discussion and conclusion

By providing new, United Kingdom-based experimental evidence on the effect of party campaigning on turnout, this paper has built on a limited literature of European partisan field experiments. The 3.6 percentage point ITT of campaign contact detected sits toward the higher end of similar studies. Indeed, United States (Green, McGrath and Aronow Reference Green, McGrath and Aronow2013) and European (Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons Reference Pons2018; Pons and Liegey Reference Pons and Liegey2018) studies often report null effects of partisan GOTV on turnout. The two published UK partisan experiments, similarly, have shown null (Foos and John Reference Foos and John2016) or modest (Foos and de Rooij Reference Foos and de Rooij2017) effects of leaflets and phone calls, respectively, on overall participation. Interestingly, while existing studies show that campaign contact boosts the party’s vote share (e.g., Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons Reference Pons2018; Pons and Liegey Reference Pons and Liegey2018), or effectively “shapes” the electorate in the party’s favor through differential mobilization (Foos and John Reference Foos and John2016), the campaign reported here failed to substantially help the party in question.

At the election, the party’s vote share increased by 2.2 percentage points (from 4.4% in 2013 to 6.6%Footnote 9 ) but represented a modest performance in the context of the wider result in Thedwastre North. The UKIP vote collapsed, going from 25% in 2013 to not standing a candidate in 2017. Meanwhile, rival parties saw much larger increases in their vote share (+9.6 and +14.7 percentage points for the Green and Conservative candidates, respectively).Footnote 10 Given the small number of party supporters in the area, the result suggests that campaign contact may actually have mobilized rival supporters to turn out to vote against the party. Evidence of “counter-mobilization” has been detected elsewhere by Arceneaux and Kolodny (Reference Arceneaux and Kolodny2009), who found that prochoice Republicans were more likely to vote against Democratic candidates after being informed that these candidates shared their prochoice views. Whether the results can be explained by the Lib Dems’ national position, or their minor status locally, is unclear. But by conducting an experiment in an area with few party supporters, this paper contributes to the debate on how partisan GOTV efforts influence voters in different electoral contexts. Given the findings, future research may want to provide further tests of which electoral contexts allow campaigns to positively shape the composition of the electorate (Foos and John Reference Foos and John2016), and which can encourage possible counter-mobilization. In the case of the latter, contacting entire streets or wards would be detrimental to parties of minor local status. Parties should, in such cases, either use canvassing data and previous results to target their efforts toward partisans only or dedicate resources to more favorable seats elsewhere.

By testing the effect of a party leaflet, this article builds on the scarce literature of European-based experiments that test impersonal, partisan campaign contact. The effect of the leaflet (2.8 percentage points among the full sample and 4.3 points among non-postal voters) sits toward the higher end of US partisan GOTV literature (e.g., Cardy Reference Cardy2005; Barton, Castillo and Petrie Reference Barton, Castillo and Petrie2013). The result is all the more surprising given that the leaflet itself contained no specific behavioral or social pressure cues that have shown powerful effects elsewhere (Nickerson Reference Nickerson2007; Green and Gerber Reference Green and Gerber2008). Like Foos and John’s (Reference Foos and John2016) study, the leaflet contained partisan messaging. In this study, the leaflet was larger and contained more photographs and issue information than the Conservative leaflet that produced no overall effect in Somerset (Foos and John Reference Foos and John2016),Footnote 11 which could have made it more noticeable among supporters and rival supporters. However, it would be wise to treat this effect as a potential outlier. Indeed, given the size, and with so few other European experiments testing the impact of party leaflets (studies so far have tested canvassing, for example, Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons and Liegey Reference Pons and Liegey2018; Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016; Cantoni and Pons Reference Cantoni and Pons2016; Pons Reference Pons2018; Nyman Reference Nyman2017, or phone calls, e.g., Foos and de Rooij Reference Foos and de Rooij2017), replication and further research into the effects of different designs and types of leaflets is merited.

A further contribution is to emerging European GOTV research on the effect of canvassing. That canvass visits in this study appeared to produce no noticeable additive effect supports the thesis that door-to-door canvassing tends to produce weaker effects in Europe (Bhatti et al. Reference Bhatti, Dahlgaard, Hedegaard and Hansen2016). This may not be particularly surprising, given that the script simply echoed standard party scripts used at election time, and those found to produce similarly weak turnout effects in the United Kingdom and Europe (Ramiro, Morales and Jiménez-Buedo Reference Ramiro, Morales and Jiménez-Buedo2012; Pons and Liegey Reference Pons and Liegey2018; Foos and John Reference Foos and John2016; Pons Reference Pons2018). However, given that this experiment, along with most of the experiments included in Bhatti et al.’s (2016) meta-study involved partisan canvassing, there remain questions over whether canvassing is simply weaker in Europe, or whether it is the partisan nature of such canvass visits that render them less effective. Though, given the low contact rate in this study and high degree of uncertainty around the effect, the result should be treated with caution. Without a third treatment group testing the impact of canvass visits alone, it is not possible to determine whether it was simply the initial partisan contact that prompted the effect, regardless of whether that contact was a canvass visit or a leaflet.

Finally, this paper builds on our understanding of how party campaigns affect different types of voters by showing that postal voters are less likely to be mobilized by party campaign contact than those who cast their votes in person. The results provide support for Arceneaux, Kousser, and Mullin (Reference Arceneaux, Kousser and Mullin2012) aggregate level findings and offer a qualification to existing evidence that shows high propensity voters are more likely to be mobilized by GOTV campaigns (Enos, Fowler and Vavreck Reference Enos, Fowler and Vavreck2014). It may be that there is a ceiling effect in the efficacy of contact among postal voters who turn out at rates of 70–85% in the United Kingdom already (Rallings and Thrasher Reference Rallings and Thrasher2014; Rallings and Thrasher Reference Rallings and Thrasher2015). While the sample sizes involved in the postal voter component of this experiment were limited, if replicated, the findings could have implications for the way campaigns distribute their resources. If a ceiling effect exists among postal voters, parties may want to focus their mobilization efforts on non-postal voters and persuasion appeals toward postal voters who are highly likely to turnout regardless. The characteristics of postal voters and their behavior during campaigns merit future research.Footnote 12 Especially as they represent a significant and growing proportion of votes cast at local and general elections, rising from 2.3% of all the votes cast at the 1997 general election to 16.4% in 2015 (Rallings and Thrasher Reference Rallings and Thrasher2015). In some parliamentary constituencies, over 50% of votes are now cast by post (Cracknell Reference Cracknell2014). The research design presented in this paper therefore provides a useful template for future experiments to compare effects between postal and non-postal voters.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/psrm.2018.39

Footnotes

1 Figure A1 in the Online Appendix shows the location of Thedwastre North within Suffolk County Council, and Table A1 shows the result from the previous local election in 2013.

2 As a result of this design, spillover between postal voters and non-postal voters in the households that contained both could not be ruled out. But in all such households that contained a mixture of postal voters and non-postal voters, the household was assigned to the postal voter experiment. The potential for spillover was small, as all households in the non-postal voter experiment contained non-postal voters only, and most subjects in the postal voter households were postal voters (76%). Further, no individuals lived in a household with an individual assigned to another treatment or control group—postal voter or non-postal voter. Likewise, no households contained individual(s) in the postal voter experiment and individual(s) in the non-postal voter experiment. A note of the numbers of postal voters in the postal voter household groups: 1012 of the 1325 voters in the postal voter experiment were registered postal voters (396 of the 515 in the canvass group; 410 of the 541 in the leaflet group; and 206 of the 269 in the control group).

3 Several villages were excluded from the non-postal voter experiment as they were subject to a pilot experiment beforehand. To avoid any contamination of multiple treatments voters in these villages were removed from both the treatment and control groups (i.e. removed from the population altogether).

4 This is likely to have been due to a number of factors including the fact that much of the campaigning fell over the Easter period, when many people were away on holiday, and much of the door-knocking was carried out by two men in their mid-20s, which may have contributed to many people, particularly elderly voters, opting not to open their doors to strangers. In most cases, two or three canvass attempts were made to improve the success rate, all within three days of the delivery of the leaflet.

5 To verify the randomization statistically, I follow the procedure outlined by Gerber and Green (Reference Gerber and Green2012, 109) to test whether imbalances are larger than one would expect from chance alone. I run a multinomial logistic regression model to test whether assignment to treatment groups is significantly related to available individual-level covariate data. The results are reported in Tables A2 and A3 in the Online Appendix and show that all covariates taken together do not significantly predict assignment to treatment conditions, increasing confidence that the randomization process was successful.

6 The ITT is calculated by comparing the turnout rate (%) between those assigned to the control and treatment groups. The ITT therefore shows the effect of being assigned to a treatment group compared to a control group, and avoids the spuriousness associated with a turnout comparison of those successfully contacted and those not contacted (Gerber and Green Reference Gerber and Green2000; Green and Gerber 2015).

7 The pre-registration details and pre-analysis plan can be found on the EGAP website: http://egap.org/registration/2589

8 I followed the procedure laid out at length in Gerber and Green (Reference Gerber and Green2012). Estimating the treatment effect (tr), given the contact rate (cr), we can divide the ITT effect by the contact rate (cr), to give us the CACE. Thus: ITT/cr=CACE. This measure is a standard assessment of the effect sizes in GOTV field experiments given issues of non-compliance (i.e., not everyone in the treatment group(s) accepting their assigned treatment). Gerber and Green (Reference Gerber and Green2000) note that this is comparable to running a two-stage least squares regression of turnout on campaign contact, with randomization as an instrumental variable (Gerber and Green, Reference Gerber and Green2012).

9 The Labour candidate won a slightly lower share of the vote at the 2017 election. The full results of the 2017 local election in Thedwastre North are presented in Table A1 in the Online Appendix.

10 Indeed, other than the Lib Dems, only these two parties (Greens and Conservatives) made notable campaign efforts in Thedwastre North during the election. Both parties delivered (to my knowledge) one leaflet to most households in the area (see A4 and A5 in the Online Appendix). There is no account available to the researcher of which households the opposing parties targeted. But, given the random assignment of households to treatment conditions, we should anticipate that these other campaign efforts would be equally distributed between treatment and control groups. The 3.7 percentage point turnout effect can therefore be interpreted as the effect of Lib Dem campaigning only—in addition to any campaigning households were exposed to from other parties.

11 See Figure A4 in the Online Appendix for an image of the leaflet delivered in Foos and John’s study.

12 A demographic comparison of the postal voter households subsample with the full sample and the non-postal voter subsample is presented in the Online Appendix (Table A7).

References

Alvarez, M, Beckett, D Stewart, C III (2012) Voting Technology, Vote-by-Mail, and Residual Votes in California, 1990-2010. Political Research Quarterly 66(3), 658670.Google Scholar
Arceneaux, K Kolodny, R (2009) Educating the Least Informed: Group Endorsements in a Grassroots Campaign. American Journal of Political Science 53(4), 755770.Google Scholar
Arceneaux, K Nickerson, D (2009) Who is Mobilized to Vote? A Re-Analysis of 11 Field Experiments. American Journal of Political Science 53(1), 116.Google Scholar
Arceneaux, K, Kousser, T Mullin, M (2012) Get Out the Vote-By-Mail? A Randomized Field Experiment Testing the Effect of Mobilization in Traditional and Vote-By-Mail Precincts. Political Research Quarterly 65(4), 882894.Google Scholar
Barton, J, Castillo, M Petrie, R (2013) What Persuades Voters? A Field Experiment on Political Campaigning. The Economic Journal 124, 293326.Google Scholar
BBC (2016) EU Referendum: Local Results. Available at http://www.bbc.co.uk/news/politics/eu_referendum/results/local/m, accessed 16 October 2017.Google Scholar
Bhatti, Y, Dahlgaard, JO, Hedegaard, JH Hansen, KM (2016) Is Door-to-Door Canvassing Effective in Europe? Evidence from a Meta-Study Across Six European Countries. British Journal of Political Science, 112.Google Scholar
Bhatti, Y, Dahlgaard, JO, Hedegaard, JH Hansen, KM (2017) Can Governments Use Get Out the Vote Letters to Solve Europe’s Turnout Crisis? Evidence from a Field Experiment. West European Politics 41(1), 240260.Google Scholar
Broockman, DE Kalla, J (2016) Durably Reducing Transphobia: A Field Experiment on Door-to-Door Canvassing. Science 352(6282), 220224.Google Scholar
Cantoni, E Pons, V (2016) Do Interactions With Candidates Increase Voter Support and Participation? Experimental Evidence from Italy. Working Paper No. 16-080. Harvard Business School, Cambridge, MA.Google Scholar
Cardy, E (2005) An Experimental Field Study on the GOTV and Persuasion Effects of Partisan Direct Mail and Phone Calls. The ANNALS of the American Academy of Political and Social Science 601(28), 2840.Google Scholar
Cracknell, R (2014) ‘First Class the Post’: The Rise of Postal Voting. House of Commons Library. Available at https://commonslibrary.parliament.uk/parliament-and-elections/elections-elections/postal-voting-in-the-uk/ accessed 5 September 2018.Google Scholar
Cutts, D Fieldhouse, E (2009) What Small Spatial Scales are Relevant as Electoral Contexts for Individual Voters? The Importance of the Household on Turnout at the 2001 General Election. American Journal of Political Science 53(3), 726739.Google Scholar
Dale, A Strauss, A (2009) Don’t Forget to Vote: Text Message Reminders as a Mobilization Tool. American Journal of Political Science 53(4), 787804.Google Scholar
Electoral Commission (2013) Local Elections May 2013. The Electoral Commission. Available at https://www.lgcplus.com/Journals/2013/07/15/f/b/l/Local-Elections-May-2013-report.pdf, accessed 16 October 2017.Google Scholar
Enos, R, Fowler, A Vavreck, L (2014) Increasing Inequality: The Effect of GOTV Mobilization on the Composition of the Electorate. Journal of Politics 76(1), 273288.Google Scholar
Fieldhouse, E Cutts, D (2016) Shared Partisanship, Household Norms and Turnout: Testing a Relational Theory of Electoral Participation. British Journal of Political Science 48(3), 117.Google Scholar
Fieldhouse, E, Cutts, D, Widdop, P John, P (2013) Do Impersonal Mobilisation Methods Work? Evidence from a Nationwide Get-Out-the-Vote Experiment in England. Electoral Studies 32(1), 113123.Google Scholar
Fisher, J, Cutts, D, Fieldhouse, E Rottweiler, B (2016) The Evolution of District-Level Campaigning in Britain: The Resilience of Traditional Campaigning? Paper presented at the EPOP Annual Conference, Kent, 9 September. Available at https://blogs.kent.ac.uk/epop/files/2016/07/Fisher-Cutts-Fieldhouse-and-Rottweiler.pdf, accessed 5 September 2018.Google Scholar
Foos, F de Rooij, E (2017) The Role of Partisan Cues in Voter Mobilization Campaigns: Evidence from a Randomized Field Experiment. Electoral Studies 45(1), 6374.Google Scholar
Foos, F John, P (2016) Parties are No Civic Charities: Voter Contact and the Changing Partisan Composition of the Electorate. Political Science Research and Methods 6(2), 283298.Google Scholar
Gerber, A Green, D (2000) The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment. The American Political Science Review 94(3), 653663.Google Scholar
Gerber, A Green, D (2012) Field Experiments: Design, Analysis and Interpretation. New York: WW Norton.Google Scholar
Gerber, A Green, D (2017) Field Experiments on Voter Mobilization: An Overview of a Burgeoning Literature. In A. V. Benerjee and E. Duo (eds), Handbook of Economic Field Experiments, 395438. Oxford: Elsevier.Google Scholar
Green D and Gerber A (2004). Get Out The Vote: How to Increase Voter Turnout. Washington, DC: Brookings Institution Press. Google Scholar
Green, D Gerber, A (2008) Get Out the Vote: How to Increase Voter Turnout. 2nd ed. Washington, DC: Brookings Institution Press.Google Scholar
Green D and Gerber A (2015). Get Out The Vote: How to Increase Voter Turnout. 3rd ed. Washington DC: Brookings Institution Press. Google Scholar
Green, D, McGrath, MC Aronow, PM (2013) Field Experiments and the Study of Voter Turnout. Journal of Elections, Public Opinion & Parties 23(1), 2748.Google Scholar
Green, D, Palmquist, B Schickler, E (2002) Partisan Hearts and Minds: Political Parties and the Social Identities of Voters. New Haven, CT: Yale University Press.Google Scholar
John, P (2017) Fields Experiments on Political Behaviour. USA: Oxford Research Encyclopaedia, Politics (politics.oxfordre.com), New York: Oxford University Press.Google Scholar
John, P Brannan, T (2008) How Different are Telephoning and Canvassing Results from a Get Out The Vote Field Experiment in the British 2005 General Election? British Journal of Political Science 38(3), 565574.Google Scholar
Mattes, K Milazzo, C (2014) Pretty Faces, Marginal Races: Predicting Election Outcomes Using Trait Assessments of British Parliamentary Candidates. Electoral Studies 34, 177189.Google Scholar
McNulty, JE (2005) Phone-Based GOTV - What’s on the Line? Field Experiments with Varied Partisan Components, 2002-2003. The ANNALS of the American Academy of Political and Social Science 601(1), 4165.Google Scholar
Mondak, J (1993) Public Opinion and Heuristic Processing of Source Cues. Political Behavior 15(2), 167192.Google Scholar
Nickerson, D (2007) Quality is Job One: Volunteer and Professional Phone Calls. American Journal of Political Science 51(2), 269282.Google Scholar
Nickerson, D (2008) Is Voting Contagious? Evidence from Two Field Experiments. American Political Science Review 102, 4957.Google Scholar
Nickerson, D, Friedrichs, RD King, DC (2006) Partisan Mobilization Campaigns in the Field: Results from a Statewide Turnout Experiment in Michigan. Political Research Quarterly 59(1), 8597.Google Scholar
Nielsen, JH (2014) Why Use Experiments in EU Studies? Comparative European Politics, 14(5), 626–644.Google Scholar
Nyman, P (2017) Door-to-Door Canvassing in the European Elections: Evidence from a Swedish Field Experiment. Electoral Studies 45, 110118.Google Scholar
Panagopoulos, C (2009) Partisan and Non-Partisan Message Content and Voter Mobilization: Field Experimental Evidence. Political Research Quarterly 62(1), 7076.Google Scholar
Pons, V (2018) Will a Five-Minute Discussion Change Your Mind? A Countrywide Experiment on Voter Choice in France. American Economic Review 108(6), 1322–1363.Google Scholar
Pons, V Liegey, G (2018) Increasing the Electoral Participation of Immigrants: Experimental Evidence from France. The Economic Journal, https://doi.org/10.1111/ecoj.12584.Google Scholar
Rallings, C Thrasher, M (2014) Local Elections in England May 2014. The Electoral Commission. Available at https://www.electoralcommission.org.uk/__data/assets/pdf_file/0003/175062/Local-elections-2014-Electoral-data-report.pdf, accessed 16 October, 2017.Google Scholar
Rallings, C Thrasher, M (2015) The 2015 General Election: Aspects of Participation and Administration. The Electoral Commission. Available at http://www.electoralcommission.org.uk/__data/assets/pdf_file/0008/191861/Plymouth-UKPGE-electoral-data-report-final-WEB.pdf, accessed: 17 December 2017.Google Scholar
Ramiro, L, Morales, L Jiménez-Buedo, M (2012) Effects of Party Mobilization on Electoral Results. An Experimental Study of the 2011 Spanish Local Election. Paper Prepared for the 2012 International Political Science Association World Congress of Political Science, Madrid, 8–12 July.Google Scholar
Sniderman, P, Richard, M, Philippe, AB Tetlock, E (1991) Reasoning and Choice: Explorations in Political Psychology. Cambridge: Cambridge University Press.Google Scholar
Figure 0

Table 1 Effect of COMMON Get Out The Vote (GOTV) Tactics (Green, McGrath and Aronow 2013)

Figure 1

Fig. 1 Field experiment research design process Note: The relative sample sizes of the different groups were determined based on how many households we (the researcher and the local party), thought we could realistically contact. This was based on the timeframe and the personnel available at different times during the campaign. We wanted to ensure treatments were carried out two weeks prior to postal votes being sent out, and polling day, for postal voter households and non-postal voter households, respectively. We knew that there were fewer postal voter households than non-postal voter households, but we had approximately the same amount of time to contact each. Therefore, we decided to assign a larger proportion of postal voter households, which we were contacting during the first half of the campaign, to the treatment groups (534 out of 676 households, compared to 1179 out of 2695 for non-postal voter households).

Figure 2

Fig. 2 Lib Dem leaflet

Figure 3

Table 2 Sample Size and Contact Rates by Treatment Group

Figure 4

Table 3 Balance of Pre-Treatment Covariates Between Assignment Groups (% (n))

Figure 5

Fig. 3 Covariate-Adjusted Predicted Intent-To-Treat Effects of Treatments on Voter Turnout Note: the solid lines represent 95% confidence intervals.

Figure 6

Table 4 Experiment Results

Supplementary material: PDF

Townsley supplementary material

Appendix

Download Townsley supplementary material(PDF)
PDF 1.3 MB
Supplementary material: Link

Townsley Dataset

Link