Skip to main content Accessibility help
×
Home
Hostname: page-component-65dc7cd545-x46dj Total loading time: 0.178 Render date: 2021-07-25T00:21:16.524Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA

Published online by Cambridge University Press:  04 January 2017

Chris Krogslund
Affiliation:
Travers Department of Political Science, UC Berkeley
Donghyun Danny Choi
Affiliation:
Travers Department of Political Science, UC Berkeley email: dhchoi@berkeley.edu
Mathias Poertner
Affiliation:
Travers Department of Political Science, UC Berkeley email: mathias.poertner@berkeley.edu

Abstract

Scholars have increasingly turned to fuzzy set Qualitative Comparative Analysis (fsQCA) to conduct small- and medium-N studies, arguing that it combines the most desired elements of variable-oriented and case-oriented research. This article demonstrates, however, that fsQCA is an extraordinarily sensitive method whose results are worryingly susceptible to minor parametric and model specification changes. We make two specific claims. First, the causal conditions identified by fsQCA as being sufficient for an outcome to occur are highly contingent upon the values of several key parameters selected by the user. Second, fsQCA results are subject to marked confirmation bias. Given its tendency toward finding complex connections between variables, the method is highly likely to identify as sufficient for an outcome causal combinations containing even randomly generated variables. To support these arguments, we replicate three articles utilizing fsQCA and conduct sensitivity analyses and Monte Carlo simulations to assess the impact of small changes in parameter values and the method's built-in confirmation bias on the overall conclusions about sufficient conditions.

Type
Research Article
Copyright
Copyright © The Author 2014. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below.

Footnotes

Authors' note: Supplementary materials for this article are available on the Political Analysis Web site. (Chris Krogslund, Donghyun Danny Choi, and Mathias Poertner, 2014, “Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA”, http://dx.doi.org/10.7910/DVN/27100 Dataverse [Distributor] V1 [Version]). We thank the editors, the anonymous reviewers, Henry Brady, Ruth Berins Collier, Thad Dunning, Zachary Elkins, Marcus Kurtz, Katherine Michel, Robert Mickey, Gerardo Munck, Jack Paine, Slim Pickens, Roxanna Ramzipoor, Ingo Rohlfing, Jason Seawright, Laura Stoker, Sean Tanner, Alrik Thiem, Alison Varney, and Sherry Zaks for their very helpful comments. A special note of thanks goes out to David Collier, who was instrumental in shaping this project.

References

Achen, C. 2005. Two cheers for Charles Ragin. Studies in Comparative International Development 40(1): 2732.CrossRefGoogle Scholar
Ahn, S.-H., and Lee, S. S.-Y. 2012. Explaining Korean welfare state development with new empirical data and methods. Asian Social Work and Policy Review 6(2): 6785.CrossRefGoogle Scholar
Brady, H. E., and Collier, D. 2010. Rethinking social inquiry: Diverse tools, shared standards. Lanham, MD: Rowman and Littlefield.Google Scholar
Freedman, D. 2010. Statistical models and causal inference: A dialogue with the social sciences. New York: Cambridge University Press.Google Scholar
Gerring, J. 2011. Social science methodology: A criterial framework. New York: Cambridge University Press.CrossRefGoogle Scholar
Glaesser, J., and Cooper, B. 2014. Exploring the consequences of a recalibration of causal conditions when assessing sufficiency with fuzzy set QCA. International Journal of Social Research Methodology 17(4): 115.CrossRefGoogle Scholar
Huber, E., Ragin, C., Stephens, J. D., Brady, D., and Beckfield, J. 2004. Comparative welfare states data set. Northwestern University, University of North Carolina, Duke University, and Indiana University.Google Scholar
Hug, S. 2013. Qualitative comparative analysis: How inductive use and measurement error lead to problematic inference. Political Analysis 21(2): 252–65.CrossRefGoogle Scholar
King, G., Keohane, R. O., and Verba, S. 1994. Designing Social Inquiry. Princeton, NJ: Princeton University Press.Google Scholar
Koenig-Archibugi, M. 2004. Explaining government preferences for institutional change in EU foreign policy and security policy. International Organization 58(1): 137–74.CrossRefGoogle Scholar
Krogslund, C., Choi, D. D., and Poertner, M. 2014. Replication data for: Fuzzy sets on shaky ground: Parameter sensitivity and confirmation bias in fsQCA. Available at http://dx.doi.org/10.7910/DVN/27100 (accessed October 13, 2014).CrossRefGoogle Scholar
Lucas, S. R., and Szatrowski, A. 2014. Qualitative comparative analysis in critical perspective. Sociological Methodology 44:179.CrossRefGoogle Scholar
Maggetti, M., and Levi-Faur, D. 2013. Dealing with errors in QCA. Political Research Quarterly 66(1): 199204.Google Scholar
Ragin, C. 1987. The comparative method: Moving beyond qualitative and quantitative strategies. Berkeley: University of California Press.Google Scholar
Ragin, C. 2000. Fuzzy set social science. Chicago: University of Chicago Press.Google Scholar
Samford, S. 2010. Averting disruption and reversal: Reassessing the logic of rapid trade reform in Latin America. Politics and Society 38(3): 373407.CrossRefGoogle Scholar
Schneider, C. Q., and Wagemann, C. 2012. Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis (QCA). Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
Seawright, J. 2005a. Assumptions, causal inferences, and the goals of QCA. Studies in Comparative International Development 40(1): 3942.CrossRefGoogle Scholar
Seawright, J. 2005b. Qualitative comparative analysis vis-à-vis regression. Studies in Comparative International Development 40(1): 326.CrossRefGoogle Scholar
Skaaning, S.-E. 2011. Assessing the robustness of crisp-set and fuzzy-set QCA results. Social Methods Research 40(2): 391408.CrossRefGoogle Scholar
Thiem, A. 2014. Membership function sensitivity of descriptive statistics in fuzzy-set relations. International Journal of Social Research Methodology 17(6): 625–42.CrossRefGoogle Scholar
Thiem, A., and Duşa, A. 2013. QCA: A package for qualitative comparative analysis. R Journal 5(1): 8797.Google Scholar
Supplementary material: File

Krogslund et al. supplementary material

Supplementary Material

Download Krogslund et al. supplementary material(File)
File 1 MB
46
Cited by

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *