Hostname: page-component-586b7cd67f-2plfb Total loading time: 0 Render date: 2024-11-28T16:10:20.992Z Has data issue: false hasContentIssue false
  • English
  • Français

Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

Published online by Cambridge University Press:  10 November 2011

KOSUKE IMAI ,
LUKE KEELE ,
DUSTIN TINGLEY  and
TEPPEI YAMAMOTO
KOSUKE IMAI*
Affiliation:
Princeton University
LUKE KEELE*
Affiliation:
Pennsylvania State University
DUSTIN TINGLEY*
Affiliation:
Harvard University
TEPPEI YAMAMOTO*
Affiliation:
Massachusetts Institute of Technology
*
Kosuke Imai is Associate Professor, Department of Politics, Princeton University, Corwin Hall 036, Princeton NJ 08544 ([email protected]).
Luke Keele is Assistant Professor, Department of Political Science, Pennsylvania State University, 211 Pond Lab, University Park, PA 16802 ([email protected]).
Dustin Tingley is Assistant Professor, Department of Government, Harvard University, 1737 Cambridge Street, CGIS Knafel Building 208, Cambridge MA 02138 ([email protected]).
Teppei Yamamoto is Assistant Professor, Department of Political Science, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139 ([email protected]).
Get access Rights & Permissions [Opens in a new window]

Abstract

Identifying causal mechanisms is a fundamental goal of social science. Researchers seek to study not only whether one variable affects another but also how such a causal relationship arises. Yet commonly used statistical methods for identifying causal mechanisms rely upon untestable assumptions and are often inappropriate even under those assumptions. Randomizing treatment and intermediate variables is also insufficient. Despite these difficulties, the study of causal mechanisms is too important to abandon. We make three contributions to improve research on causal mechanisms. First, we present a minimum set of assumptions required under standard designs of experimental and observational studies and develop a general algorithm for estimating causal mediation effects. Second, we provide a method for assessing the sensitivity of conclusions to potential violations of a key assumption. Third, we offer alternative research designs for identifying causal mechanisms under weaker assumptions. The proposed approach is illustrated using media framing experiments and incumbency advantage studies.

Type
Research Article
Information
American Political Science Review , Volume 105 , Issue 4 , November 2011 , pp. 765 - 789
Copyright
Copyright © American Political Science Association 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Albert, J. 2008. “Mediation Analysis via Potential Outcomes Models.” Statistics in Medicine 27: 1282–1304.CrossRefGoogle ScholarPubMed
Angrist, J. D., Imbens, G. W., and Rubin, D. B.. 1996. “Identification of Causal Effects Using Instrumental Variables (with Discussion).” Journal of the American Statistical Association 91 (434): 444–55.CrossRefGoogle Scholar
Ansolabehere, S., Snowberg, E. C., and Snyder, J. M.. 2006. “Television and the Incumbency Advantage in U.S. Elections.” Legislative Studies Quarterly 31 (4): 469–90.CrossRefGoogle Scholar
Ansolabehere, S., Snyder, J. M., and Stewart, C.. 2000. “Old Voters, New Voters, and the Personal Vote: Using Redistricting to Measure the Incumbency Advantage.” American Journal of Political Science 44 (1): 17–34.CrossRefGoogle Scholar
Baron, R. M., and Kenny, D. A.. 1986. “The Moderator–Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations.” Journal of Personality and Social Psychology 51 (6): 1173–82.CrossRefGoogle ScholarPubMed
Bartels, L. M. 1993. “Messages Received: The Political Impact of Media Exposure.” American Political Science Review 87 (2): 267–85.CrossRefGoogle Scholar
Bertrand, M., and Mullainathan, S.. 2004. “Are Emily and Greg More Employable Than Lakisha and Jamal? A Field Experiment on Labor Market Discrimination.” American Economic Review 94 (4): 991–1013.CrossRefGoogle Scholar
Blattman, C. 2009. “From Violence to Voting: War and Political Participation in Uganda.” American Political Science Review 103 (2): 231–47.CrossRefGoogle Scholar
Brader, T., Valentino, N. A., and Suhay, E.. 2008. “What Triggers Public Opposition to Immigration? Anxiety, Group Cues, and Immigration.” American Journal of Political Science 52 (4): 959–78.CrossRefGoogle Scholar
Brady, H. E., and Collier, D.. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, MD: Rowman and Littlefield.Google Scholar
Bullock, J., Green, D., and Ha, S.. 2010. “Yes, But What's the Mechanism? (Don't Expect an Easy Answer).” Journal of Personality and Social Psychology 98 (4): 550–58.CrossRefGoogle ScholarPubMed
Chong, D., and Druckman, J.. 2007. “Framing Theory.” Annual Review of Political Science 10: 103–26.CrossRefGoogle Scholar
Cnudde, C. F., and McCrone, D. J.. 1966. “The Linkage between Constituency Attitudes and Congressional Voting Behavior: A Causal Model.” American Political Science Review 60 (1): 66–72.CrossRefGoogle Scholar
Cole, S. R., and Frangakis, C. E.. 2009. “The Consistency Statement in Causal Inference: A Definition or Assumption?” Epidemiology 20 (1): 3–5.CrossRefGoogle ScholarPubMed
Collier, D., Brady, H. E., and Seawright, J.. 2004. “Source of Leverage in Causal Inference: Toward an Alternative View of Methodology.” In Rethinking Social Inquiry:Diverse Tools, Shared Standards eds. Brady, H. and Collier, D., Berkeley, CA: Rowman and Littlefield.Google Scholar
Cox, G. W., and Katz, J. N.. 1996. “Why Did the Incumbency Advantage in U.S. House Elections Grow?” American Journal of Political Science 40 (2): 478–97.CrossRefGoogle Scholar
Deaton, A. 2010a. “Instruments, Randomization, and Learning about Development.” Journal of Economic Literature 48 (2): 424–55.CrossRefGoogle Scholar
Deaton, A. 2010b. “Understanding the Mechanisms of Economic Development.” Journal of Economic Perspectives 24 (3): 3–16.CrossRefGoogle Scholar
Druckman, J. 2005. “Media Matter: How Newspapers and Television News Cover Campaigns and Influence Voters.” American Political Science Review 22: 463–81.Google Scholar
Erikson, R. S., and Palfrey, T. R.. 1998. “Campaign Spending and Incumbency: An Alternative Simultaneous Equations Approach.” Journal of Politics 60 (2): 355–73.CrossRefGoogle Scholar
Gadarian, S. K. 2010. “The Politics of Threat: How Terrorism News Shapes Foreign Policy Attitudes.” Journal of Politics 72 (2): 469–83.CrossRefGoogle Scholar
Gelman, A., and King, G.. 1990. “Estimating Incumbency Advantage without Bias.” American Journal of Political Science 34 (4): 1142–64.CrossRefGoogle Scholar
Gerber, A. 1998. “Estimating the Effect of Campaign Spending on Senate Election Outcomes Using Instrumental Variables.” American Political Science Review 92 (2): 401–11.CrossRefGoogle Scholar
Glynn, A. N. 2010. “The Product and Difference Fallacies for Indirect Effects.” Department of Government, Harvard University. Unpublished manuscript, Mimeo.Google Scholar
Green, D. P., Ha, S. E., and Bullock, J. G.. 2010. “Enough Already about Black Box Experiments: Studying Mediation Is More Difficult Than Most Scholars Suppose.” Annals of the American Academy of Political and Social Sciences 628 (1): 200–08.CrossRefGoogle Scholar
Gross, J. J., and Levenson, R. W.. 1995. “Eliciting Emotions Using Films.” Cognition and Emotion 9 (1): 87–108.CrossRefGoogle Scholar
Haavelmo, T. 1943. “The Statistical Implications of a System of Simultaneous Equations.” Econometrica 11: 1–12.CrossRefGoogle Scholar
Heckman, J. J., and Smith, J. A.. 1995. “Assessing the Case for Social Experiments.” Journal of Economic Perspectives 9 (2): 85–110.CrossRefGoogle Scholar
Hetherington, M. J. 2001. “Resurgent Mass Partisanship: The Role of Elite Polarization.” American Political Science Review 95 (3): 619–31.CrossRefGoogle Scholar
Ho, D. E., Imai, K., King, G., and Stuart, E. A.. 2007. “Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference.” Political Analysis 15 (3): 199–236.CrossRefGoogle Scholar
Holland, P. W. 1986. “Statistics and Causal Inference.” Journal of the American Statistical Association 81: 945–60.CrossRefGoogle Scholar
Holland, P. W. 1988. “Causal Inference, Path Analysis, and Recursive Structural Equations Models.” Sociological Methodology 18: 449–84.CrossRefGoogle Scholar
Horiuchi, Y., Imai, K., and Taniguchi, N.. 2007. “Designing and Analyzing Randomized Experiments: Application to a Japanese Election Survey Experiment.” American Journal of Political Science 51 (3): 669–87.CrossRefGoogle Scholar
Imai, K., Keele, L., and Tingley, D.. 2010. “A General Approach to Causal Mediation Analysis.” Psychological Methods 15 (4): 309–34.CrossRefGoogle ScholarPubMed
Imai, K., Keele, L., Tingley, D. and Yamamoto, T.. 2010. “Causal Mediation Analysis Using R”. In Advances in Social Science Research Using R, ed. Vinod, H. D., Lecture Notes in Statistics. Springer-verlag: New York, 129–54.CrossRefGoogle Scholar
Imai, K., Keele, L., Tingley, D., and Yamamoto, T.. 2011. “Replication Data for: Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies.” The Dataverse Network. hdl:1902.1/16467 (accessed September 1, 2011).Google Scholar
Imai, K., Keele, L., and Yamamoto, T.. 2010. “Identification, Inference, and Sensitivity Analysis for Causal Mediation Effects.” Statistical Science 25 (1): 51–71.CrossRefGoogle Scholar
Imai, K., King, G., and Stuart, E. A.. 2008. “Misunderstandings among Experimentalists and Observationalists about Causal Inference.” Journal of the Royal Statistical Society, Series A (Statistics in Society) 171 (2): 481–502.CrossRefGoogle Scholar
Imai, K., and Tingley, D.. N.d. “A Statistical Method for Empirical Testing of Competing Theories.” American Journal of Political Science. Forthcoming.Google Scholar
Imai, K., Tingley, D., and Yamamoto, T.. N.d. “Experimental Designs for Identifying Causal Mechanisms.” (With discussions). Journal of the Royal Statistical Society, Series A (Statistics in Society). Forthcoming.Google Scholar
Imai, K., and Yamamoto, T.. 2010. “Causal Inference with Differential Measurement Error: Nonparametric Identification and Sensitivity Analysis.” American Journal of Political Science 54 (2): 543–60.CrossRefGoogle Scholar
Imai, K., and Yamamoto, T.. 2011. “Sensitivity Analysis for Causal Mediation Effects under Alternative Exogeneity Assumptions.” http://imai.princeton.edu/research/medsens.html. (accessed September 1, 2011).CrossRefGoogle Scholar
Isbell, L., and Ottati, V.. 2002. “The Emotional Voter.” In The Social Psychology of Politics, ed. Ottati, V., New York: Kluwer, 55–74.CrossRefGoogle Scholar
Jacobson, G. C. 1987. “The Politics of Congressional Elections. Boston: Little, Brown.Google Scholar
Jo, B. 2008. “Causal Inference in Randomized Experiments with Mediational Processes.” Psychological Methods 13 (4): 314–36.CrossRefGoogle ScholarPubMed
Jost, J. T., Napier, J. L., Thorisdottir, H., Gosling, S. D., Palfai, T. P., and Ostafin, B.. 2007. “Are Needs to Manage Uncertainty and Threat Associated With Political Conservatism or Ideological Extremity?” Personality and Social Psychology Bulletin 33 (7): 989–1007.CrossRefGoogle ScholarPubMed
Kinder, D. R. and Sanders, L.. 1996. Divided by Color: Racial Politics and Democratic Ideals. Chicago: University of Chicago Press.Google Scholar
King, G., Keohane, R. O., and Verba, S.. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
King, G., Tomz, M., and Wittenberg, J.. 2000. “Making the Most of Statistical Analyses: Improving Interpretation and Presentation.” American Journal of Political Science 44: 341–55.CrossRefGoogle Scholar
Levitt, S. D. and Wolfram, C. D.. 1997. “Decomposing the Sources of Incumbency Advantage in the U.S. House.” Legislative Studies Quarterly 22 (1): 45–60.CrossRefGoogle Scholar
MacKinnon, D. 2008. Introduction to Statistical Mediation Analysis. New York: Routledge.Google Scholar
MacKinnon, D., Lockwood, C., Brown, C., Wang, W., and Hoffman, J.. 2007. “The Intermediate Endpoint Effect in Logistic and Probit Regression.” Clinical Trials 4: 499–513.CrossRefGoogle ScholarPubMed
Manski, C. F. 2007. Identification for Prediction and Decision. Cambridge, MA: Harvard University Press.Google Scholar
Miller, J. M., and Krosnick, J. A.. 2000. “News Media Impact on the Ingredients of Presidential Evaluations: Politically Knowledgeable Citizens Are Guided by a Trusted Source.” American Journal of Political Science 44 (2): 301–15.CrossRefGoogle Scholar
Miller, W. E., and Stokes, D. W.. 1963. “Constituency Influence in Congress.” American Political Science Review 57 (1): 45–46.CrossRefGoogle Scholar
Nelson, T. E., Clawson, R. A., and Oxley, Z. M.. 1997. “Media Framing of a Civil Liberties Conflict and Its Effect on Tolerance.” American Political Science Review 91 (3): 567–83.CrossRefGoogle Scholar
Nelson, T. E., and Kinder, D. R.. 1996. “Issue Frames and Group-centrism in American Public Opinion.” The Journal of Politics 58 (4): 1055–78.CrossRefGoogle Scholar
Neyman, J. [1923] 1990. “On the Application of Probability Theory to Agricultural Experiments: Essay on Principles, Section 9.” Statistical Science 5: 465–80.Google Scholar
Olsson, A., Ebert, J. P., Banaji, M. R., and Phelps, E. A.. 2005. “The Role of Social Groups in the Persistence of Learned Fear.” Science 309 (5735): 785–87.CrossRefGoogle ScholarPubMed
Oxley, D. R., Smith, K. B., Alford, J. R., Hibbing, M. V., Miller, J. L., Scalora, M., Hatemi, P. K., and Hibbing, J. R.. 2008. “Political Attitudes Vary with Physiological Traits.” Science 321 (5896): 1667–70.CrossRefGoogle ScholarPubMed
Pearl, J. 2001. “Direct and Indirect Effects.” In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence, eds. Breese, Jack S. and Koller, Daphne. San Francisco: Morgan Kaufmann, 411–20.Google Scholar
Pearl, J. N.d. “The Causal Mediation Formula: A Guide to the Assessment of Pathways and Mechanisms.” Prevention Science. Forthcoming.Google Scholar
Petersen, M. L., Sinisi, S. E., and van der Laan, M. J.. 2006. “Estimation of Direct Causal Effects.” Epidemiology 17 (3): 276–84.CrossRefGoogle ScholarPubMed
Prior, M. 2006. “The Incumbent in the Living Room: The Rise of Television and the Incumbency Advantage in U.S. House Elections.” Journal of Politics 68 (3): 657–73.CrossRefGoogle Scholar
Robins, J. M. 2003. “Semantics of Causal DAG Models and the Identification of Direct and Indirect Effects.” In Highly Structured Stochastic Systems, eds. Green, P. J., Hjort, N. L., and Richardson, S., Oxford: Oxford University Press, 70–81.CrossRefGoogle Scholar
Robins, J. M., and Greenland, S.. 1992. “Identifiability and Exchangeability for Direct and Indirect Effects.” Epidemiology 3 (2): 143–55.CrossRefGoogle ScholarPubMed
Robins, J. M., and Richardson, T.. 2010. “Alternative Graphical Causal Models and the Identification of Direct Effects. In Causality and Psychopathology: Finding the Determinants of Disorders and Their Cures, eds. Shrout, P., Keyes, K., and Omstein, K., Oxford: Oxford University Press, 103–58.Google Scholar
Rosenbaum, P. R. 2002a. “Covariance Adjustment in Randomized Experiments and Observational Studies: Rejoinder.” Statistical Science 17 (3): 321–27.CrossRefGoogle Scholar
Rosenbaum, P. R. 2002b. “Covariance Adjustment in Randomized Experiments and Observational Studies (with Discussion).” Statistical Science 17 (3): 286–327.CrossRefGoogle Scholar
Rubin, D. B. 1974. “Estimating Causal Effects of Treatments in Randomized and Non-randomized Studies.” Journal of Educational Psychology 66: 688–701.CrossRefGoogle Scholar
Shadish, W. R., Cook, T. D., and Campbell, D. T.. 2001. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin.Google Scholar
Sjölander, A. 2009. “Bounds on Natural Direct Effects in the Presence of Confounded Intermediate Variables.” Statistics in Medicine 28 (4): 558–71.CrossRefGoogle ScholarPubMed
Sobel, M. E. 1982. “Asymptotic Confidence Intervals for Indirect Effects in Structural Equation Models.” Sociological Methodology 13: 290–321.CrossRefGoogle Scholar
Sobel, M. E. 2008. “Identification of Causal Parameters in Randomized Studies with Mediating Variables.” Journal of Educational and Behavioral Statistics 33 (2): 230–51.CrossRefGoogle Scholar
Spencer, S., Zanna, M., and Fong, G.. 2005. “Establishing a Causal Chain: Why Experiments Are Often More Effective Than Mediational Analyses in Examining Psychological Processes.” Journal of Personality and Social Psychology 89 (6): 845–51.CrossRefGoogle ScholarPubMed
Tiedens, L. Z. and Linton, S.. 2001. “Judgment under Emotional Certainty and Uncertainty: The Effects of Specific Emotions on Information Processing.” Journal of Personality and Social Psychology 81 (6): 973–88.CrossRefGoogle ScholarPubMed
Tomz, M. and Houweling, R. P. van. 2009. “The Electoral Implications of Candidate Ambiguity.” American Political Science Review 103 (1): 83–98.CrossRefGoogle Scholar
VanderWeele, T. J. 2009. “Marginal Structural Models for the Estimation of Direct and Indirect Effects.” Epidemiology 20 (1): 18–26.CrossRefGoogle ScholarPubMed
VanderWeele, T. J. and Robins, J. M.. 2009. “Minimal Sufficient Causation and Directed Acyclic Graphs.” Annals of Statistics 37 (3): 1437–65.CrossRefGoogle Scholar
Submit a response

Comments

No Comments have been published for this article.