Skip to main content
×
×
Home

Mixing Methods: A Bayesian Approach

  • MACARTAN HUMPHREYS (a1) and ALAN M. JACOBS (a2)
Abstract

We develop an approach to multimethod research that generates joint learning from quantitative and qualitative evidence. The framework—Bayesian integration of quantitative and qualitative data (BIQQ)—allows researchers to draw causal inferences from combinations of correlational (cross-case) and process-level (within-case) observations, given prior beliefs about causal effects, assignment propensities, and the informativeness of different kinds of causal-process evidence. In addition to posterior estimates of causal effects, the framework yields updating on the analytical assumptions underlying correlational analysis and process tracing. We illustrate the BIQQ approach with two applications to substantive issues that have received significant quantitative and qualitative treatment in political science: the origins of electoral systems and the causes of civil war. Finally, we demonstrate how the framework can yield guidance on multimethod research design, presenting results on the optimal combinations of qualitative and quantitative data collection under different research conditions.

Copyright
Corresponding author
Macartan Humphreys is Professor, Department of Political Science, Columbia University, New York (mh2245@columbia.edu).
Alan M. Jacobs is Associate Professor, Department of Political Science, University of British Columbia, Vancouver, Canada (alan.jacobs@ubc.ca).
References
Hide All
Barton, Allen H., and Lazarsfeld, Paul F.. 1955. Some Functions of Qualitative Analysis in Social Research. Vol. 181. Bobbs Merrill Indianapolis, Indiana.
Beach, Derek, and Pedersen, Rasmus Brun. 2013. Process-Tracing Methods: Foundations and Guidelines. Ann Arbor: University of Michigan Press.
Beck, Nathaniel. 2010. “Causal Process ‘Observation’: Oxymoron or (Fine) Old Wine.” Political Analysis 18 (4): 499505.
Bennett, Andrew. 2008. “Process Tracing. A Bayesian Perspective.” In The Oxford Handbook of Political Methodology, eds. Box-Steffensmeier, Janet M., Brady, Henry E., and Collier, David. Oxford, UK: Oxford University Press.
Bennett, Andrew. 2010. “Process Tracing and Causal Inference.” In Rethinking Social Inquiry: Diverse Tools, Shared Standards, eds. Collier, David and Brady, Henry E.. Lanham, MD: Rowman & Littlefield, 207–20.
Bennett, Andrew. 2015. “Appendix.” In Process Tracing: From Metaphor to Analytic Tool, eds. Bennett, Andrew and Checkel, Jeffrey. New York: Cambridge University Press, 276–98.
Boix, Carles. 1999. “Setting the Rules of the Game: The Choice of Electoral Systems in Advanced Democracies.” The American Political Science Review 93 (3): 609–24.
Brady, H. E., and Collier, D.. 2004. Rethinking Social Inquiriy: Diverse Tools, Shared Standards. Lanham, MD: Rowman & Littlefield.
Chickering, David Maxwell, and Pearl, Judea. 1996. “A Clinician’s Tool for Analyzing Non-Compliance.” In Proceedings of the National Conference on Artificial Intelligence. Palo Alto, California: Association for the Advancement of Artificial Intelligence (AAAI), pp. 1269–76.
Collier, David. 2011. “Understanding Process Tracing.” PS: Political Science and Politics 44 (04): 823–30.
Collier, David, Brady, Henry E., and Seawright, Jason. 2004. “Sources of Leverage in Causal Inference: Toward an Alternative View of Methodology.” In Rethinking Social Inquiry: Diverse Tools, Shared Standards, edited by Collier, David and Brady, Henry E. Lanham, MD: Rowman & Littlefield, 229–66.
Collier, David, Brady, Henry E., and Seawright, Jason. 2010. “Sources of Leverage in Causal Inference: Toward an Alternative View of Methodology.” In Rethinking Social Inquiry: Diverse Tools, Shared Standards, eds. Collier, David and Brady, Henry E.. Lanham, MD: Rowman & Littlefield, 161200.
Collier, P., and Sambanis, N.. 2005. Understanding Civil War: Africa. Washington, D.C.: Stand Alone Series World Bank.
Collier, Paul, and Hoeffler, Anke. 2004. “Greed and Grievance in Civil War.” Oxford Economic Papers 56 (4): 563–95.
Creswell, J. W., and Garrett, Amanda L.. 2008. “The ‘Movement’ of Mixed Methods Research and the Role of Educators.” South African Journal of Education 28 (08/2008): 321–33.
Ericson, William A. 1969. “Subjective Bayesian Models in Sampling Finite Populations.” Journal of the Royal Statistical Society. Series B (Methodological) 31: 195224.
Fairfield, Tasha. 2013. “Going Where the Money Is: Strategies for Taxing Economic Elites in Unequal Democracies.” World Development 47 (221–236): 4257.
Freedman, David A. 2010. “On Types of Scientific Inquiry: The Role of Qualitative Reasoning.” In Rethinking Social Inquiry: Diverse Tools, Shared Standards, eds. Collier, David and Brady, Henry E.. Lanham, MD: Rowman & Littlefield, 251–73.
Gelman, Andrew, Carlin, John B., Stern, Hal S., Dunson, David B., Vehtari, Aki, and Rubin, Donald B.. 2013. Bayesian Data Analysis. Boca Raton, FL: CRC Press.
Gerber, Alan S., Green, Donald P., and Kaplan, Edward H.. 2004. “The Illusion of Learning from Observational Research.” In Problems and Methods in the Study of Politics, ed. Shapiro, Ian, Smith, Rogers M., and Masoud, Tarek E.. Cambridge, UK: Cambridge University Press.
Gerring, J. 2012. Social Science Methodology: A Unified Framework. Strategies for Social Inquiry. Cambridge, UK: Cambridge University Press.
Gill, Jeff, and Walker, Lee D.. 2005. “Elicited Priors for Bayesian Model Specifications in Political Science Research.” Journal of Politics 67 (3): 841–72.
Glynn, Adam N., Wakefield, Jon, Handcock, Mark S., and Richardson, Thomas S.. 2008. “Alleviating Linear Ecological Bias and Optimal Design with Subsample Data.” Journal of the Royal Statistical Society: Series A (Statistics in Society) 171 (1): 179202.
Glynn, Adam N., and Quinn, Kevin M.. 2011. “Why Process Matters for Causal Inference.” Political Analysis 19: 273–86.
Glynn, Adam N., and Ichino, Nahomi. 2014. “Using Qualitative Information to Improve Causal Inference.” American Journal of Political Science 59 (4): 1055–71.
Goertz, G., and Mahoney, J.. 2012. Tale of Two Cultures - Contrasting Qualitative and Quantitative. Princeton: University Press Group Limited.
Gordon, Sanford C., and Smith, Alastair. 2004. “Quantitative Leverage Through Qualitative Knowledge: Augmenting the Statistical Analysis of Complex Causes.” Political Analysis 12 (3): 233–55.
Hall, Peter A. 2003. “Aligning Ontology and Methodology in Comparative Research.” In Comparative Historical Analysis in the Social Sciences, ed. Mahoney, James and Rueschemeyer, Dietrich. Cambridge, UK and New York: Cambridge University Press.
Herron, Michael, and Quinn, Kevin. 2009. “A Careful Look at Modern Case Selection Methods.” Paper Presented at the 67th Annual Meeting of the Midwest Political Science Association.
Imbens, Guido W., and Rubin, Donald B.. 1997. “Bayesian Inference for Causal Effects in Randomized Experiments with Noncompliance.” The Annals of Statistics 25 (1): 305–27.
King, G., Keohane, R. O., and Verba, S.. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, NJ: Princeton University Press.
Kreuzer, Marcus. 2010. “Historical Knowledge and Quantitative Analysis: The Case of the Origins of Proportional Representation.” American Political Science Review 104 (5): 369–92.
Lengfelder, Christina. 2012. “Triangular Development Cooperation: How Emerging Powers Change the Landscape of Development Cooperation.” Ph.D. dissertation, Universidad Catolica de Chile.
Lieberman, E. S. 2003. Race and Regionalism in the Politics of Taxation in Brazil and South Africa. Cambridge Studies in Comparative Politics. Cambridge, UK: Cambridge University Press.
Lieberman, Evan S. 2005. “Nested Analysis as a Mixed-Method Strategy for Comparative Research.” American Political Science Review 99 (7): 435–52.
Mahoney, James. 2012. “The Logic of Process Tracing Tests in the Social Sciences.” Sociological Methods and Research 41 (4): 570–97.
Paluck, Elizabeth Levy. 2010. “The Promising Integration of Qualitative Methods and Field Experiments.” The ANNALS of the American Academy of Political and Social Science 628 (1): 5971.
Pearl, Judea. 2000. Causality: Models, Reasoning and Inference. Vol. 29. Cambridge, UK: Cambridge University Press.
Rohlfing, Ingo. 2012. Case Studies and Causal Inference: An Integrative Framework. Research Methods Series. New York: Palgrave Macmillan.
Ross, Michael L. 2004. “How do Natural Resources Influence Civil War? Evidence from Thirteen Cases.” International Organization 58 (01): 3567.
Rubin, Donald B. 1974. “Estimating Causal Effects of Treatments in Randomized and Nonrandomized Studies.” Journal of Educational Psychology 66: 688701.
Schlag, Karl H., Tremewan, James, and Van der Weele, Joel J.. 2013. “A Penny for Your Thoughts: A Survey of Methods for Eliciting Beliefs.” Experimental Economics: 1–34.
Seawright, Jason. ND. Multi-method Social Science: Combining Qualitative and Quantitative Tools. Cambridge, UK: Cambridge, Cambridge University Press.
Seawright, Jason, and Gerring, John. 2008. “Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options.” Political Research Quarterly 61 (2): 294308.
Stan Development Team. 2014. “RStan: the R interface to Stan, Version 2.5.0.” http://mc-stan.org/rstan.html.
Stokes, S.C. 2001. Mandates and Democracy: Neoliberalism by Surprise in Latin America. Cambridge Studies in Comparative Politics Cambridge University Press. Cambridge, UK: Cambridge University Press.
Swank, D. 2002. Global Capital, Political Institutions, and Policy Change in Developed Welfare States. Cambridge Studies in Comparative Politics Cambridge University Press. Cambridge, UK: Cambridge University Press.
Van Evera, Stephen. 1997. Guide to Methods for Students of Political Science. Ithaca, NY: Cornell University Press.
Western, Bruce, and Jackman, Simon. 1994. “Bayesian Inference for Comparative Research.” American Political Science Review 88 (6): 412–23.
White, Howard, and Philips, Daniel. 2012. “Addressing Attribution of Cause and Effect in Small N Impact Evaluations: Towards an Integrated Framework.” Technical Report 15 International Initiative for Impact Evaluation Working Papers. http://www.3ieimpact.org/media/filer/2012/06/29/working_paper_15.pdf.
Williamson, Jon. 2004. “Bayesian Nets and Causality: Philosophical and Computational Foundations.” Oxford, UK: Oxford University Press.
Young, Forrest W. 1981. “Quantitative Analysis of Qualitative Data.” Psychometrika 46 (4): 357–88.
Zaks, Sherry. 2013. “Relationships Among Rivals: Contending Hypotheses and the Logic of Process Tracing.” Paper prepared for the Short Course 1 on Multi-Method Research, 2012 Annual Meeting of the American Political Science Association, New Orleans, LA.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

American Political Science Review
  • ISSN: 0003-0554
  • EISSN: 1537-5943
  • URL: /core/journals/american-political-science-review
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Humphreys and Jacobs supplementary material
Humphreys and Jacobs supplementary material 1

 PDF (12.6 MB)
12.6 MB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 75
Total number of PDF views: 1184 *
Loading metrics...

Abstract views

Total abstract views: 2784 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 23rd June 2018. This data will be updated every 24 hours.