Skip to main content Accessibility help
×
Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-06T13:06:15.081Z Has data issue: false hasContentIssue false

9 - Field Experiments in Political Science

Published online by Cambridge University Press:  05 June 2012

Alan S. Gerber
Affiliation:
Yale University
James N. Druckman
Affiliation:
Northwestern University, Illinois
Donald P. Greene
Affiliation:
Yale University, Connecticut
James H. Kuklinski
Affiliation:
University of Illinois, Urbana-Champaign
Arthur Lupia
Affiliation:
University of Michigan, Ann Arbor
Get access

Summary

After a period of almost total absence in political science, field experimentation has become a common research design. In this chapter, I discuss some of the reasons for the increasing use of field experiments. Several chapters in this volume provide comprehensive introductions to specific experimental techniques and detailed reviews of the now extensive field experimental literatures in multiple areas. This chapter does not duplicate these contributions, but instead provides background, arguments, opinions, and speculations. I begin by defining field experiments in Section 1. In Section 2, I discuss the intellectual context for the emergence of field experimentation in political science, beginning with the recent revival of field experimentation in studies of voter turnout. In Section 3, I describe the statistical properties of field experiments and explain how the approach addresses many of the common methodological deficiencies identified in earlier observational research on campaign effects and voter participation. Section 4 reviews the range of applications of field experimentation. In Section 5, I answer several frequently asked questions about the limitations and weaknesses of field experimentation. In Section 6, I briefly discuss some challenges that field experimentation faces as it becomes a more frequently employed methodological approach in political science. This includes a discussion of the external validity of field experimental results and consideration of how difficulties related to replication and bias in experimental reporting might affect the development of field experiment literatures.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abramowitz, Alan I. 1988. “Explaining Senate Election Outcomes.” American Political Science Review 82: 385–403.CrossRefGoogle Scholar
Achen, Christopher H., and Bartels, Larry M.. 2004. “Blind Retrospection: Electoral Responses to Droughts, Flu, and Shark Attacks.” Estudio/Working Paper 2004/199.
Adams, William C., and Smith, Dennis J.. 1980. “Effects of Telephone Canvassing on Turnout and Preferences: A Field Experiment.” Public Opinion Quarterly 44: 389–95.CrossRefGoogle Scholar
Addonizio, Elizabeth, Green, Donald, and Glaser, James M.. 2007. “Putting the Party Back into Politics: An Experiment Testing Whether Election Day Festivals Increase Voter Turnout.” PS: Political Science & Politics 40: 721–27.Google Scholar
Angrist, Joshua D. 1990. “Lifetime Earnings and the Vietnam Era Draft Lottery: Evidence from Social Security Administrative Records.” American Economic Review 80: 313–36.Google Scholar
Angrist, Joshua D., Imbens, Guido, and Rubin, Donald B.. 1996. “Identification of Causal Effects Using Instrumental Variables.” Journal of the American Statistical Association 91: 444–72.CrossRefGoogle Scholar
Angrist, Joshua D., and Krueger, Alan B.. 1991. “Does Compulsory School Attendance Affect Schooling and Earnings?” Quarterly Journal of Economics 106: 979–1014.CrossRefGoogle Scholar
Angrist, Joshua D., and Pischke, Jorn-Steffen. 2009. Mostly Harmless Econometrics: An Empiricist's Companion. Princeton, NJ: Princeton University Press.Google Scholar
Ansolabehere, Stephen D., and Gerber, Alan S.. 1994. “The Mismeasure of Campaign Spending: Evidence from the 1990 U.S. House Elections.” Journal of Politics 56: 1106–18.CrossRefGoogle Scholar
Ansolabehere, Stephen D., and Iyengar, Shanto. 1996. Going Negative: How Political Advertising Divides and Shrinks the American Electorate. New York: Free Press.Google Scholar
Ansolabehere, Stephen D., and Snyder, James M.. 1996. “Money, Elections, and Candidate Quality.” Typescript, Massachusetts Institute of Technology.
Ansolabehere, Stephen D., and Stewart, Charles III. 2005. “Residual Votes Attributable to Technology.” Journal of Politics 67: 365–89.CrossRefGoogle Scholar
Arceneaux, Kevin, Gerber, Alan S., and Green, Donald P.. 2006. “Comparing Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment.” Political Analysis 14: 1–36.CrossRefGoogle Scholar
Arceneaux, Kevin, and Nickerson, David. 2009. “Who Is Mobilized to Vote? A Re-Analysis of Eleven Randomized Field Experiments.” American Journal of Political Science 53: 1–16.CrossRefGoogle Scholar
Bergan, Daniel E. 2009. “Does Grassroots Lobbying Work?: A Field Experiment Measuring the Effects of an E-Mail Lobbying Campaign on Legislative Behavior.” American Politics Research 37: 327–52.CrossRefGoogle Scholar
Berger, Jonah, Meredith, Marc, and Wheeler, S. Christian. 2008. “Contextual Priming: Where People Vote Affects How They Vote.” Proceedings of the National Academy of Sciences 105: 8846–49.CrossRefGoogle ScholarPubMed
Brady, Henry E., and McNulty, John E.. 2004. “The Costs of Voting: Evidence from a Natural Experiment.” Paper presented at the annual meeting of the Society for Political Methodology, Palo Alto, CA.
Butler, Daniel M., and Nickerson, David W.. 2009. “Are Legislators Responsive to Public Opinion? Results from a Field Experiment.” Typescript, Yale University.
Chalmers, Iain. 2003. “Trying To Do More Good Than Harm in Policy and Practice: The Role of Rigorous, Transparent, Up-to-Date Evaluations.” Annals of the American Academy of Political and Social Science 589: 22–40.CrossRefGoogle Scholar
Chin, Michelle L., Bond, Jon R., and Geva, Nehemia. 2000. “A Foot in the Door: An Experimental Study of PAC and Constituency Effects on Access.” Journal of Politics 62: 534–49.CrossRefGoogle Scholar
Dale, Allison, and Strauss, Aaron. 2007. “Mobilizing the Mobiles: How Text Messaging Can Boost Youth Voter Turnout.” Working paper, University of Michigan.
Davenport, Tiffany C., Gerber, Alan S., and Green, Donald P.. 2010. “Field Experiments and the Study of Political Behavior.” In The Oxford Handbook of American Elections and Political Behavior, ed. Leighley, Jan E.. New York: Oxford University Press, 69–88.Google Scholar
Deaton, Angus S. 2009. “Instruments of Development: Randomization in the Tropics, and the Search for the Elusive Keys to Economic Development.” NBER Working Paper No. 14690.
Eldersveld, Samuel J. 1956. “Experimental Propaganda Techniques and Voting Behavior.” American Political Science Review 50: 154–65.CrossRefGoogle Scholar
Eldersveld, Samuel J., and Dodge, Richard W.. 1954. “Personal Contact or Mail Propaganda? An Experiment in Voting Turnout and Attitude Change.” In Public Opinion and Propaganda, ed. Katz, Daniel. New York: Holt, Rinehart and Winston, 532–42.Google Scholar
Erikson, Robert S., and Palfrey, Thomas R.. 2000. “Equilibria in Campaign Spending Games: Theory and Data.” American Political Science Review 94: 595–609.CrossRefGoogle Scholar
Gerber, Alan S. 1998. “Estimating the Effect of Campaign Spending on Senate Election Outcomes Using Instrumental Variables.” American Political Science Review 92: 401–11.CrossRefGoogle Scholar
Gerber, Alan S. 2004. “Does Campaign Spending Work?: Field Experiments Provide Evidence and Suggest New Theory.” American Behavioral Scientist 47: 541–74.CrossRefGoogle Scholar
Gerber, Alan S. in press. “New Directions in the Study of Voter Mobilization: Combining Psychology and Field Experimentation.”
Gerber, Alan S., and Doherty, David. 2009. “Can Campaign Effects Be Accurately Measured Using Surveys?: Evidence from a Field Experiment.” Typescript, Yale University.
Gerber, Alan S., Doherty, David, and Dowling, Conor M.. 2009. “Developing a Checklist for Reporting the Design and Results of Social Science Experiments.” Typescript, Yale University.
Gerber, Alan S., Gimpel, James G., Green, Donald P., and Shaw, Daron R.. in press. “The Size and Duration of Campaign Television Advertising Effects: Results from a Large-Scale Randomized Experiment.” American Political Science Review.
Gerber, Alan S., and Green, Donald P.. 2000. “The Effects of Canvassing, Direct Mail, and Telephone Contact on Voter Turnout: A Field Experiment.” American Political Science Review 94: 653–63.CrossRefGoogle Scholar
Gerber, Alan S., and Green, Donald P.. 2008. “Field Experiments and Natural Experiments.” In Oxford Handbook of Political Methodology, eds. Box-Steffensmeier, Janet M., Brady, Henry E., and Collier, David. New York: Oxford University Press, 357–81.Google Scholar
Gerber, Alan S., and Green, Donald P.. 2011. Field Experiments: Design, Analysis, and Interpretation. Unpublished Manuscript, Yale University.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Kaplan, Edward H.. 2004. “The Illusion of Learning from Observational Research.” In Problems and Methods in the Study of Politics, eds. Shapiro, Ian, Smith, Rogers, and Massoud, Tarek. New York: Cambridge University Press, 251–73.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Larimer, Christopher W.. 2008. “Social Pressure and Voter Turnout: Evidence from a Large-Scale Field Experiment.” American Political Science Review 102: 33–48.CrossRefGoogle Scholar
Gerber, Alan S., Green, Donald P., and Nickerson, David W.. 2001. “Testing for Publication Bias in Political Science.” Political Analysis 9: 385–92.CrossRefGoogle Scholar
Gerber, Alan S., Huber, Gregory A., and Washington, Ebonya. in press. “Party Affiliation, Partisanship, and Political Beliefs: A Field Experiment.” American Political Science Review.
Gerber, Alan S., Karlan, Dean, and Bergan, Daniel. 2009. “Does the Media Matter? A Field Experiment Measuring the Effect of Newspapers on Voting Behavior and Political Opinions.” American Economic Journal: Applied Economics 1: 35–52.Google Scholar
Gerber, Alan S., and Yamada, Kyohei. 2008. “Field Experiment, Politics, and Culture: Testing Social Psychological Theories Regarding Social Norms Using a Field Experiment in Japan.” Working paper, ISPS Yale University.
Gosnell, Harold F. 1927. Getting-Out-the-Vote: An Experiment in the Stimulation of Voting. Chicago: The University of Chicago Press.Google Scholar
Green, Donald P., and Gerber, Alan S.. 2004. Get Out the Vote: How to Increase Voter Turnout. Washington, DC: Brookings Institution Press.Google Scholar
Green, Donald P., and Gerber, Alan S.. 2008. Get Out the Vote: How to Increase Voter Turnout. 2nd ed. Washington, DC: Brookings Institution Press.Google Scholar
Green, Donald P., and Krasno, Jonathan S.. 1988. “Salvation for the Spendthrift Incumbent: Reestimating the Effects of Campaign Spending in House Elections.” American Journal of Political Science 32: 884–907.CrossRefGoogle Scholar
Guan, Mei, and Green, Donald P.. 2006. “Non-Coercive Mobilization in State-Controlled Elections: An Experimental Study in Beijing.” Comparative Political Studies 39: 1175–93.CrossRefGoogle Scholar
Habyarimana, James, Humphreys, Macartan, Posner, Dan, and Weinstein, Jeremy. 2007. “Why Does Ethnic Diversity Undermine Public Goods Provision? An Experimental Approach.” American Political Science Review 101: 709–25.CrossRefGoogle Scholar
Harrison, Glenn W., and List, John A.. 2004. “Field Experiments.” Journal of Economic Literature 42: 1009–55.CrossRefGoogle Scholar
Healy, Andrew J., Malhotra, Neil, and Mo, Cecilia Hyunjung. 2009. “Do Irrelevant Events Affect Voters' Decisions? Implications for Retrospective Voting.” Stanford Graduate School of Business Working Paper No. 2034.
Humphreys, Macartan, and Weinstein, Jeremy M.. 2007. “Policing Politicians: Citizen Empowerment and Political Accountability in Africa.” Paper presented at the annual meeting of the American Political Science Association, Chicago.
Humphreys, Macartan, and Weinstein, Jeremy. 2009. “Field Experiments and the Political Economy of Development.” Annual Review of Political Science 12: 367–78.CrossRefGoogle Scholar
Hyde, Susan D. 2010. “Experimenting in Democracy Promotion: International Observers and the 2004 Presidential Elections in Indonesia.” Perspectives on Politics 8: 511–27.CrossRefGoogle Scholar
Imbens, Guido W. 2009. “Better Late Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009).” National Bureau of Economic Research Working Paper No. 14896.
Jacobson, Gary C. 1978. “The Effects of Campaign Spending in Congressional Elections.” American Political Science Review 72: 469–91.CrossRefGoogle Scholar
Jacobson, Gary C. 1985. “Money and Votes Reconsidered: Congressional Elections, 1972–1982.” Public Choice 47: 7–62.CrossRefGoogle Scholar
Jacobson, Gary C. 1990. “The Effects of Campaign Spending in House Elections: New Evidence for Old Arguments.” American Journal of Political Science 34: 334–62.CrossRefGoogle Scholar
Jacobson, Gary C. 1998. The Politics of Congressional Elections. New York: Longman.Google Scholar
John, Peter, and Brannan, Tessa. 2008. “How Different Are Telephoning and Canvassing? Results from a ‘Get Out the Vote’ Field Experiment in the British 2005 General Election.” British Journal of Political Science 38: 565–74.CrossRefGoogle Scholar
Knack, Steve. 1994. “Does Rain Help the Republicans? Theory and Evidence on Turnout and the Vote.” Public Choice 79: 187–209.CrossRefGoogle Scholar
LaLonde, Robert J. 1986. “Evaluating the Econometric Evaluations of Training Programs with Experimental Data.” American Economic Review 76: 604–20.Google Scholar
Levitt, Steven D. 1994. “Using Repeat Challengers to Estimate the Effect of Campaign Spending on Election Outcomes in the U.S. House.” Journal of Political Economy 102: 777–98.CrossRefGoogle Scholar
Loudon, Irvine. 2000. The Tragedy of Childbed Fever. Oxford: Oxford University Press.CrossRefGoogle Scholar
Medical Research Council, Streptomycin in Tuberculosis Trials Committee. 1948. “Streptomycin Treatment for Pulmonary Tuberculosis.” British Medical Journal 2: 769–82.CrossRefGoogle Scholar
Michelson, Melissa R. 2003. “Getting Out the Latino Vote: How Door-to-Door Canvassing Influences Voter Turnout in Rural Central California.” Political Behavior 25: 247–63.CrossRefGoogle Scholar
Michelson, Melissa R., Bedolla, Lisa García, and McConnell, Margaret A.. 2009. “Heeding the Call: The Effect of Targeted Two-Round Phonebanks on Voter Turnout.” Journal of Politics 71: 1549–63.CrossRefGoogle Scholar
Miller, Joanne M., and Krosnick, Jon A.. 1998. “The Impact of Candidate Name Order on Election Outcomes.” Public Opinion Quarterly 62: 291–330.CrossRefGoogle Scholar
Miller, Roy E., Bositis, David A., and Baer, Denise L.. 1981. “Stimulating Voter Turnout in a Primary: Field Experiment with a Precinct Committeeman.” International Political Science Review 2: 445–60.CrossRefGoogle Scholar
Nickerson, David W. 2008. “Is Voting Contagious? Evidence from Two Field Experiments.” American Political Science Review 102: 49–57.CrossRefGoogle Scholar
Olken, Benjamin. 2010. “Direct Democracy and Local Public Goods: Evidence from a Field Experiment in Indonesia.” American Political Science Review 104: 243–67.CrossRefGoogle Scholar
Paluck, Elizabeth Levy, and Green, Donald P.. 2009. “Deference, Dissent, and Dispute Resolution: An Experimental Intervention Using Mass Media to Change Norms and Behavior in Rwanda.” American Political Science Review 103: 622–44.CrossRefGoogle Scholar
Panagopoulos, Costas, and Green, Donald P.. 2008. “Field Experiments Testing the Impact of Radio Advertisements on Electoral Competition.” American Journal of Political Science 52: 156–68.CrossRefGoogle Scholar
,Pew Research Center for the People and the Press. 1998. “Possible Consequences of Non-Response for Pre-Election Surveys: Race and Reluctant Respondents.” Survey Report, May 16. Retrieved from http://people-press.org/report/89/possible-consequences-of-non-response-for-pre-election-surveys (November 1, 2010).
Posner, Richard A. 2004. Catastrophe: Risk and Response. Oxford: Oxford University Press.Google Scholar
Rosenstone, Steven J., and Hansen, John Mark. 1993. Mobilization, Participation, and Democracy in America. New York: MacMillan.Google Scholar
Rubin, Donald B. 1978. “Bayesian Inference for Causal Effects: The Role of Randomization.” The Annals of Statistics 6: 34–58.CrossRefGoogle Scholar
Rubin, Donald B. 1990. “Formal Modes of Statistical Inference for Causal Effects.” Journal of Statistical Planning and Inference 25: 279–92.CrossRefGoogle Scholar
Taubes, Gary. 1993. Bad Science: The Short Life and Weird Times of Cold Fusion. New York: Random House.Google Scholar
Vavreck, Lynn. 2007. “The Exaggerated Effects of Advertising on Turnout: The Dangers of Self-Reports.” Quarterly Journal of Political Science 2: 287–305.CrossRefGoogle Scholar
Verba, Sidney, Schlozman, Kay Lehman, and Brady, Henry E.. 1995. Voice and Equality: Civic Voluntarism in American Politics. Cambridge, MA: Harvard University Press.Google Scholar
Wantchekon, Leonard. 2003. “Clientelism and Voting Behavior: Evidence from a Field Experiment in Benin.” World Politics 55: 399–422.CrossRefGoogle Scholar
Whitney, Simon N., and Schneider, Carl E.. 2010. “A Method to Estimate the Cost in Lives of Ethics Board Review of Biomedical Research.” Paper presented at the “Is Medical Ethics Really in the Best Interest of the Patient?” Conference, Uppsala, Sweden.

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×