Hostname: page-component-586b7cd67f-tf8b9 Total loading time: 0 Render date: 2024-11-24T15:54:16.788Z Has data issue: false hasContentIssue false
  • English
  • Français

Blocking for Sequential Political Experiments

Published online by Cambridge University Press:  04 January 2017

Ryan T. Moore  and
Sally A. Moore
Ryan T. Moore*
Affiliation:
Department of Political Science, Washington University in St. Louis, 241 Seigle Hall, Campus Box 1063, One Brookings Drive, St. Louis, MO 63130
Sally A. Moore
Affiliation:
VA Puget Sound Health Care System—Seattle Division, University of Washington, Department of Psychiatry and Behavioral Sciences, and Evidence-Based Treatment Centers of Seattle, 1200 Fifth Ave, Suite 800, Seattle, WA 98101
*
e-mail: [email protected] (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

In typical political experiments, researchers randomize a set of households, precincts, or individuals to treatments all at once, and characteristics of all units are known at the time of randomization. However, in many other experiments, subjects “trickle in” to be randomized to treatment conditions, usually via complete randomization. To take advantage of the rich background data that researchers often have (but underutilize) in these experiments, we develop methods that use continuous covariates to assign treatments sequentially. We build on biased coin and minimization procedures for discrete covariates and demonstrate that our methods outperform complete randomization, producing better covariate balance in simulated data. We then describe how we selected and deployed a sequential blocking method in a clinical trial and demonstrate the advantages of our having done so. Further, we show how that method would have performed in two larger sequential political trials. Finally, we compare causal effect estimates from differences in means, augmented inverse propensity weighted estimators, and randomization test inversion.

Type
Research Article
Information
Political Analysis , Volume 21 , Issue 4 , Autumn 2013 , pp. 507 - 523
Copyright
Copyright © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors' note: We thank Nicholas Beauchamp, Jens Hainmueller, Kosuke Imai, Rebecca Morton, Kevin Quinn, and the participants in EGAP 6 and ICHPS 9 for helpful suggestions. The replication archive is available at Moore and Moore (2013). Supplementary materials for the article are available on the Political Analysis Web site.

References

Atkinson, Anthony C. 2003. The distribution of loss in two-treatment biased-coin designs. Biometrics 4: 179–93.Google Scholar
Ball, Frank G., Smith, Adrian F. M., and Verdinelli, Isabella. 1993. Biased coin designs with a Bayesian bias. Journal of Statistical Planning and Inference 34: 403–21.Google Scholar
Bowers, Jake. 2011. Making effects manifest in randomized experiments. In Cambridge handbook of experimental political science, eds. Druckman, James N., Green, Donald P., Kuklinski, James H., and Lupia, Arthur, 459–80. New York, NY: Cambridge University Press.Google Scholar
Braucht, G. Nicholas, and Reichardt, Charles S. 1993. A computerized approach to trickle-process, random assignment. Evaluation Review 17: 79–90.Google Scholar
Chong, Dennis, and Druckman, James N. 2007. Framing public opinion in competitive democracies. American Political Science Review 101: 637–55.Google Scholar
Chow, Shein-Chung, and Chang, Mark. 2007. Adaptive design methods in clinical trials. Boca Raton, FL: Chapman & Hall.Google Scholar
Cobb, Rachael V., James Greiner, D., and Quinn, Kevin M. 2011. Can voter ID laws be administered in a race-neutral manner? Evidence from the city of Boston in 2008. Quarterly Journal of Political Science 6: 1–33.Google Scholar
Duflo, Esther, Glennerster, Rachel, and Kremer, Michael. 2008. Using randomization in development economics research: A toolkit. In Handbook of development economics, ed. Paul Schultz, T., Vol. 4, 3895–962. Amsterdam: Elsevier, B.V. Google Scholar
Efron, Bradley. 1971. Forcing a sequential experiment to be balanced. Biometrika 58: 403–17.Google Scholar
Hansen, Ben B., and Bowers, Jake. 2008. Covariate balance in simple, stratified, and clustered comparative studies. Statistical Science 23: 219–36.Google Scholar
Harrington, David P. 2000. The randomized clinical trial. Journal of the American Statistical Association 95: 312–15.CrossRefGoogle Scholar
Ho, Daniel E., and Imai, Kosuke. 2006. Randomization inference with natural experiments: An analysis of ballot effects in the 2003 California recall election. Journal of the American Statistical Association 101: 888–900.Google Scholar
Horiuchi, Yusaku, Imai, Kosuke, and Taniguchi, Naoko. 2007. Designing and analyzing randomized experiments: Application to a Japanese election survey experiment. American Journal of Political Science 51: 669–87.Google Scholar
Imai, Kosuke, King, Gary, and Nall, Clayton. 2009. The essential role of pair-matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation. Statistical Science 24: 29–53.Google Scholar
Kalish, Leslie A., and Begg, Colin B. 1985. Treatment allocation methods in clinical trials: A review. Statistics in Medicine 4: 129–44.CrossRefGoogle ScholarPubMed
King, Gary, Gakidou, Emmanuela, Ravishankar, Nirmala, Moore, Ryan T., Lakin, Jason, Vargas, Manett, María Téllez-Rojo, Martha, Eugenio Hernández Ávila, Juan, Hernández Ávila, Mauricio, and Hernández Llamas, Héctor. 2007. A “politically robust” experimental design for public policy evaluation, with application to the Mexican universal health insurance program. Journal of Policy Analysis and Management 26: 479–509.Google Scholar
Lachin, John M. 1988. Properties of simple randomization in clinical trials. Controlled Clinical Trials 9: 312–26.Google Scholar
Lachin, John M., Matts, John P., and Wei, L. J. 1988. Randomization in clinical trials: Conclusions and recommendations. Controlled Clinical Trials 9: 365–74.Google Scholar
Love, Thomas E., Cebul, Randall D., Einstadter, Douglas, Jain, Anil K., Miller, Holly, Martin Harris, C., Greco, Peter J., Husak, Scott S., and Dawson, Neal V. 2008. Electronic medical record-assisted design of a cluster-randomized trial to improve diabetes care and outcomes. Journal of General Internal Medicine 23: 383–91.Google Scholar
Malhotra, Neil, and Kuo, Alexander G. 2008. Attributing blame: The public's response to hurricane Katrina. Journal of Politics 70: 120–35.Google Scholar
Moore, Ryan T. 2012. Multivariate continuous blocking to improve political science experiments. Political Analysis 20: 460–79.Google Scholar
Moore, Ryan T., and Moore, Sally A. 2013. Replication data for: Blocking for sequential political experiments. http://hdl.handle.net/1902.1/21042. IQSS Dataverse Network, V1.Google Scholar
Moore, Sally A., Moore, Ryan T., and Simpson, Tracy L. In preparation 2013. The effects of practicing specific autobiographical memory retrieval in veterans with PTSD.Google Scholar
Morgan, Kari Lock, and Rubin, Donald B. 2012. Rerandomization to improve covariate balance in experiments. Annals of Statistics 40: 1263–82.Google Scholar
Morton, Rebecca B., and Williams, Kenneth C. 2010. Experimental political science and the study of causality: From nature to the lab. New York: Cambridge University Press.CrossRefGoogle Scholar
Pocock, Stuart J., and Simon, Richard. 1975. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics 31: 103–15.Google Scholar
R Core Team. 2013. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
Rosenbaum, Paul R. 2002. Observational studies. 2nd ed. New York: Springer.Google Scholar
Rosenbaum, Paul R. 2010. Design of observational studies. New York: Springer.Google Scholar
Rosenbaum, Paul R., and Rubin, Donald B. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70: 41–55.Google Scholar
Rosenberger, William F., and Lachin, John M. 2002. Randomization in clinical trials: Theory and practice. Hoboken, NJ: Wiley.Google Scholar
Rosenberger, William F., and Sverdlov, Oleksandr. 2008. Handling covariates in the design of clinical trials. Statistical Science 23: 404–19.Google Scholar
Rubin, Donald B. 2001. Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services and Outcomes Research Methodology 2: 169–88.Google Scholar
Schafer, Joseph L., and Kang, Joseph. 2008. Average causal effects from nonrandomized studies: A practical guide and simulated example. Psychological Methods 13: 279–313.Google Scholar
Schulz, Kenneth F., and Grimes, David A. 2002a. Generation of allocation sequences in randomized trials: Chance, not choice. The Lancet 359: 515–19.Google Scholar
Schulz, Kenneth F., and Grimes, David A. 2002b. Unequal group sizes in randomised trials: Guarding against guessing. The Lancet 359: 966–70.Google Scholar
Tsiatis, Anastasios A. 2006. Semiparametric theory and missing data. New York: Springer.Google Scholar
Whitehead, John. 1997. The design and analysis of sequential clinical trials. New York: Wiley.Google Scholar
Zelen, Marvin. 1969. Play the winner rule and the controlled clinical trial. Journal of the American Statistical Association 64: 131–46.Google Scholar
Zelen, Marvin. 1974. The randomization and stratification of patients to clinical trials. Journal of Chronic Disease 27: 365–75.Google Scholar
Supplementary material: PDF

Moore and Moore supplementary material

Supplementary Material

Download Moore and Moore supplementary material(PDF)
PDF 275 KB