Hostname: page-component-586b7cd67f-l7hp2 Total loading time: 0 Render date: 2024-11-24T08:46:02.663Z Has data issue: false hasContentIssue false
  • English
  • Français

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA

Published online by Cambridge University Press:  04 January 2017

Chris Krogslund ,
Donghyun Danny Choi  and
Mathias Poertner
Chris Krogslund*
Affiliation:
Travers Department of Political Science, UC Berkeley
Donghyun Danny Choi
Affiliation:
Travers Department of Political Science, UC Berkeley email: [email protected]
Mathias Poertner
Affiliation:
Travers Department of Political Science, UC Berkeley email: [email protected]
*
email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Scholars have increasingly turned to fuzzy set Qualitative Comparative Analysis (fsQCA) to conduct small- and medium-N studies, arguing that it combines the most desired elements of variable-oriented and case-oriented research. This article demonstrates, however, that fsQCA is an extraordinarily sensitive method whose results are worryingly susceptible to minor parametric and model specification changes. We make two specific claims. First, the causal conditions identified by fsQCA as being sufficient for an outcome to occur are highly contingent upon the values of several key parameters selected by the user. Second, fsQCA results are subject to marked confirmation bias. Given its tendency toward finding complex connections between variables, the method is highly likely to identify as sufficient for an outcome causal combinations containing even randomly generated variables. To support these arguments, we replicate three articles utilizing fsQCA and conduct sensitivity analyses and Monte Carlo simulations to assess the impact of small changes in parameter values and the method's built-in confirmation bias on the overall conclusions about sufficient conditions.

Type
Research Article
Information
Political Analysis , Volume 23 , Issue 1 , Winter 2015 , pp. 21 - 41
Copyright
Copyright © The Author 2014. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors' note: Supplementary materials for this article are available on the Political Analysis Web site. (Chris Krogslund, Donghyun Danny Choi, and Mathias Poertner, 2014, “Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA”, http://dx.doi.org/10.7910/DVN/27100 Dataverse [Distributor] V1 [Version]). We thank the editors, the anonymous reviewers, Henry Brady, Ruth Berins Collier, Thad Dunning, Zachary Elkins, Marcus Kurtz, Katherine Michel, Robert Mickey, Gerardo Munck, Jack Paine, Slim Pickens, Roxanna Ramzipoor, Ingo Rohlfing, Jason Seawright, Laura Stoker, Sean Tanner, Alrik Thiem, Alison Varney, and Sherry Zaks for their very helpful comments. A special note of thanks goes out to David Collier, who was instrumental in shaping this project.

References

Achen, C. 2005. Two cheers for Charles Ragin. Studies in Comparative International Development 40(1): 27–32.Google Scholar
Ahn, S.-H., and Lee, S. S.-Y. 2012. Explaining Korean welfare state development with new empirical data and methods. Asian Social Work and Policy Review 6(2): 67–85.Google Scholar
Brady, H. E., and Collier, D. 2010. Rethinking social inquiry: Diverse tools, shared standards. Lanham, MD: Rowman and Littlefield.Google Scholar
Freedman, D. 2010. Statistical models and causal inference: A dialogue with the social sciences. New York: Cambridge University Press.Google Scholar
Gerring, J. 2011. Social science methodology: A criterial framework. New York: Cambridge University Press.Google Scholar
Glaesser, J., and Cooper, B. 2014. Exploring the consequences of a recalibration of causal conditions when assessing sufficiency with fuzzy set QCA. International Journal of Social Research Methodology 17(4): 1–15.Google Scholar
Huber, E., Ragin, C., Stephens, J. D., Brady, D., and Beckfield, J. 2004. Comparative welfare states data set. Northwestern University, University of North Carolina, Duke University, and Indiana University.Google Scholar
Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–65.Google Scholar
King, G., Keohane, R. O., and Verba, S. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.CrossRefGoogle Scholar
Koenig-Archibugi, M. 2004. Explaining government preferences for institutional change in EU foreign policy and security policy. International Organization 58(1): 137–74.Google Scholar
Krogslund, C., Choi, D. D., and Poertner, M. 2014. Replication data for: Fuzzy sets on shaky ground: Parameter sensitivity and confirmation bias in fsQCA. Available at http://dx.doi.org/10.7910/DVN/27100 (accessed October 13, 2014).Google Scholar
Lucas, S. R., and Szatrowski, A. 2014. Qualitative comparative analysis in critical perspective. Sociological Methodology 44:1–79.Google Scholar
Maggetti, M., and Levi-Faur, D. 2013. Dealing with errors in QCA. Political Research Quarterly 66(1): 199–204.Google Scholar
Ragin, C. 1987. The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley: University of California Press.Google Scholar
Ragin, C. 2000. Fuzzy set social science. Chicago: University of Chicago Press.Google Scholar
Samford, S. 2010. Averting disruption and reversal: Reassessing the logic of rapid trade reform in Latin America. Politics and Society 38(3): 373–407.CrossRefGoogle Scholar
Schneider, C. Q., and Wagemann, C. 2012. Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis (QCA). Cambridge, UK: Cambridge University Press.Google Scholar
Seawright, J. 2005a. Assumptions, causal inferences, and the goals of QCA. Studies in Comparative International Development 40(1): 39–42.Google Scholar
Seawright, J. 2005b. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative International Development 40(1): 3–26.Google Scholar
Skaaning, S.-E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Social Methods Research 40(2): 391–408.Google Scholar
Thiem, A. 2014. Membership function sensitivity of descriptive statistics in fuzzy-set relations. International Journal of Social Research Methodology 17(6): 625–42.Google Scholar
Thiem, A., and Duşa, A. 2013. QCA: A package for qualitative comparative analysis. R Journal 5(1): 87–97.CrossRefGoogle Scholar
Supplementary material: File

Krogslund et al. supplementary material

Supplementary Material

Download Krogslund et al. supplementary material(File)
File 1.1 MB