Skip to main content
×
×
Home

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA

  • Chris Krogslund (a1), Donghyun Danny Choi (a2) and Mathias Poertner (a3)
Abstract

Scholars have increasingly turned to fuzzy set Qualitative Comparative Analysis (fsQCA) to conduct small- and medium-N studies, arguing that it combines the most desired elements of variable-oriented and case-oriented research. This article demonstrates, however, that fsQCA is an extraordinarily sensitive method whose results are worryingly susceptible to minor parametric and model specification changes. We make two specific claims. First, the causal conditions identified by fsQCA as being sufficient for an outcome to occur are highly contingent upon the values of several key parameters selected by the user. Second, fsQCA results are subject to marked confirmation bias. Given its tendency toward finding complex connections between variables, the method is highly likely to identify as sufficient for an outcome causal combinations containing even randomly generated variables. To support these arguments, we replicate three articles utilizing fsQCA and conduct sensitivity analyses and Monte Carlo simulations to assess the impact of small changes in parameter values and the method's built-in confirmation bias on the overall conclusions about sufficient conditions.

Copyright
Corresponding author
email: ckrogslund@berkeley.edu
Footnotes
Hide All

Authors' note: Supplementary materials for this article are available on the Political Analysis Web site. (Chris Krogslund, Donghyun Danny Choi, and Mathias Poertner, 2014, “Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA”, http://dx.doi.org/10.7910/DVN/27100 Dataverse [Distributor] V1 [Version]). We thank the editors, the anonymous reviewers, Henry Brady, Ruth Berins Collier, Thad Dunning, Zachary Elkins, Marcus Kurtz, Katherine Michel, Robert Mickey, Gerardo Munck, Jack Paine, Slim Pickens, Roxanna Ramzipoor, Ingo Rohlfing, Jason Seawright, Laura Stoker, Sean Tanner, Alrik Thiem, Alison Varney, and Sherry Zaks for their very helpful comments. A special note of thanks goes out to David Collier, who was instrumental in shaping this project.

Footnotes
References
Hide All
Achen, C. 2005. Two cheers for Charles Ragin. Studies in Comparative International Development 40(1): 2732.
Ahn, S.-H., and Lee, S. S.-Y. 2012. Explaining Korean welfare state development with new empirical data and methods. Asian Social Work and Policy Review 6(2): 6785.
Brady, H. E., and Collier, D. 2010. Rethinking social inquiry: Diverse tools, shared standards. Lanham, MD: Rowman and Littlefield.
Freedman, D. 2010. Statistical models and causal inference: A dialogue with the social sciences. New York: Cambridge University Press.
Gerring, J. 2011. Social science methodology: A criterial framework. New York: Cambridge University Press.
Glaesser, J., and Cooper, B. 2014. Exploring the consequences of a recalibration of causal conditions when assessing sufficiency with fuzzy set QCA. International Journal of Social Research Methodology 17(4): 115.
Huber, E., Ragin, C., Stephens, J. D., Brady, D., and Beckfield, J. 2004. Comparative welfare states data set. Northwestern University, University of North Carolina, Duke University, and Indiana University.
Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–65.
King, G., Keohane, R. O., and Verba, S. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.
Koenig-Archibugi, M. 2004. Explaining government preferences for institutional change in EU foreign policy and security policy. International Organization 58(1): 137–74.
Krogslund, C., Choi, D. D., and Poertner, M. 2014. Replication data for: Fuzzy sets on shaky ground: Parameter sensitivity and confirmation bias in fsQCA. Available at http://dx.doi.org/10.7910/DVN/27100 (accessed October 13, 2014).
Lucas, S. R., and Szatrowski, A. 2014. Qualitative comparative analysis in critical perspective. Sociological Methodology 44:179.
Maggetti, M., and Levi-Faur, D. 2013. Dealing with errors in QCA. Political Research Quarterly 66(1): 199204.
Ragin, C. 1987. The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley: University of California Press.
Ragin, C. 2000. Fuzzy set social science. Chicago: University of Chicago Press.
Samford, S. 2010. Averting disruption and reversal: Reassessing the logic of rapid trade reform in Latin America. Politics and Society 38(3): 373407.
Schneider, C. Q., and Wagemann, C. 2012. Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis (QCA). Cambridge, UK: Cambridge University Press.
Seawright, J. 2005a. Assumptions, causal inferences, and the goals of QCA. Studies in Comparative International Development 40(1): 3942.
Seawright, J. 2005b. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative International Development 40(1): 326.
Skaaning, S.-E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Social Methods Research 40(2): 391408.
Thiem, A. 2014. Membership function sensitivity of descriptive statistics in fuzzy-set relations. International Journal of Social Research Methodology 17(6): 625–42.
Thiem, A., and Duşa, A. 2013. QCA: A package for qualitative comparative analysis. R Journal 5(1): 8797.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax
Type Description Title
UNKNOWN
Supplementary materials

Krogslund et al. supplementary material
Supplementary Material

 Unknown (1.1 MB)
1.1 MB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed