Hostname: page-component-745bb68f8f-s22k5 Total loading time: 0 Render date: 2025-02-05T01:40:09.100Z Has data issue: false hasContentIssue false
  • English
  • Français

Why Experimenters Might Not Always Want to Randomize, and What They Could Do Instead

Published online by Cambridge University Press:  04 January 2017

Maximilian Kasy
Maximilian Kasy*
Affiliation:
Department of Economics, Harvard University, 1805 Cambridge Street, Cambridge, MA 02138, USA
*
e-mail: [email protected] (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

Suppose that an experimenter has collected a sample as well as baseline information about the units in the sample. How should she allocate treatments to the units in this sample? We argue that the answer does not involve randomization if we think of experimental design as a statistical decision problem. If, for instance, the experimenter is interested in estimating the average treatment effect and evaluates an estimate in terms of the squared error, then she should minimize the expected mean squared error (MSE) through choice of a treatment assignment. We provide explicit expressions for the expected MSE that lead to easily implementable procedures for experimental design.

Type
Articles
Information
Political Analysis , Volume 24 , Issue 3 , Summer 2016 , pp. 324 - 338
Copyright
Copyright © The Author 2016. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Author's note: I thank Alberto Abadie, Ivan Canay, Gary Chamberlain, Raj Chetty, Nathaniel Hendren, Guido Imbens, Larry Katz, Gary King, Michael Kremer, and Don Rubin, as well as seminar participants at the Harvard development retreat; the Harvard Labor Economics Workshop; the Harvard Quantitative Issues in Cancer Research Seminar; the Harvard Applied Statistics Seminar; UT Austin, Princeton, Columbia, and Northwestern Econometrics Seminars; at RAND; and at the 2013 CEME Conference at Stanford for helpful discussions. Replication data are available on the Harvard Dataverse at http://dx.doi.org/10.7910/DVN/I5KCWI. See Kasy (2016). Supplementary materials for this article are available on the Political Analysis Web site.

References

Berger, J. 1985. Statistical decision theory and Bayesian inference. New York: Springer.Google Scholar
Blattman, C., Hartman, A. C., and Blair, R. A. 2014. How to promote order and property rights under weak rule of law? An experiment in changing dispute resolution behavior through community education. American Political Science Review 108:100–120.Google Scholar
Findley, M. G., Nielson, D. L., and Sharman, J. 2015. Causes of noncompliance with international law: A field experiment on anonymous incorporation. American Journal of Political Science 59(1):146–61.Google Scholar
Kalla, J. L., and Broockman, D. E. 2015. Campaign contributions facilitate access to congressional officials: A randomized field experiment. American Journal of Political Science. “http://sfx.hul.harvard.edu/hvd?__char_set=utf8&id=doi:10.1111/ajps.12180&sid=libx%3Ahul.harvard&genre=article” doi: 10.1111/ajps.12180Google Scholar
Kasy, M. 2014. Using data to inform policy. Working Paper.Google Scholar
Kasy, M. 2016. Matlab implementation for: Why experimenters might not always want to randomize, and what they could do instead, Harvard Dataverse . http://dx.doi.org/10.7910/DVN/I5KCWI.CrossRefGoogle Scholar
Keele, L. 2015. The statistics of causal inference: A view from political methodology. Political Analysis. doi: 10.1093/pan/mpv007.Google Scholar
Moore, R. T. 2012. Multivariate continuous blocking to improve political science experiments. Political Analysis 20(4):460–79.Google Scholar
Morgan, K. L., and Rubin, D. B. 2012. Rerandomization to improve covariate balance in experiments. Annals of Statistics 40(2):1263–82.Google Scholar
Nyhan, B., and Reifler, J. 2014. The effect of fact-checking on elites: A field experiment on U. S. state legislators. American Journal of Political Science.Google Scholar
Supplementary material: PDF

Kasy supplementary material

Appendix

Download Kasy supplementary material(PDF)
PDF 223.3 KB