Hostname: page-component-848d4c4894-wzw2p Total loading time: 0 Render date: 2024-06-12T12:21:23.662Z Has data issue: false hasContentIssue false
  • English
  • Français

How Multiple Imputation Makes a Difference

Published online by Cambridge University Press:  04 January 2017

Ranjit Lall
Ranjit Lall*
Affiliation:
Department of Government, Harvard University, 1737 Cambridge Street, Cambridge, MA 02138
*
e-mail: ranjitlall@fas.harvard.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Political scientists increasingly recognize that multiple imputation represents a superior strategy for analyzing missing data to the widely used method of listwise deletion. However, there has been little systematic investigation of how multiple imputation affects existing empirical knowledge in the discipline. This article presents the first large-scale examination of the empirical effects of substituting multiple imputation for listwise deletion in political science. The examination focuses on research in the major subfield of comparative and international political economy (CIPE) as an illustrative example. Specifically, I use multiple imputation to reanalyze the results of almost every quantitative CIPE study published during a recent five-year period in International Organization and World Politics, two of the leading subfield journals in CIPE. The outcome is striking: in almost half of the studies, key results “disappear” (by conventional statistical standards) when reanalyzed.

Type
Articles
Information
Political Analysis , Volume 24 , Issue 4 , Autumn 2016 , pp. 414 - 433
Copyright
Copyright © The Author 2016. Published by Oxford University Press on behalf of the Society for Political Methodology 

Footnotes

Author's note: I am grateful to Anthony Atkinson, Jeffry Frieden, Adam Glynn, James Honaker, Gary King, Walter Mattli, Margaret Roberts, Beth Simmons, Arthur Spirling, and the editors and anonymous reviewers of Political Analysis for helpful comments and suggestions. I also thank Olivier Accominotti, Todd Allee, Ben Ansell, Lucio Baccaro, Carles Boix, Sarah Brooks, Asif Efrat, Sean Ehrlich, Lawrence Ezrow, Marc Flandreau, Alexandra Guisinger, Caroline Hartzell, Philip Keefer, Jeffrey Kucik, Marcus Kurtz, Christopher Meissner, Sonal Pandya, Clint Peinhardt, Krzysztof Pelc, Kristopher Ramsay, Diego Rei, David Rueda, David Singer, and Hugh Ward for generously sharing data with me. For replication materials, see Lall (2016). Supplementary materials for this article are available on the Political Analysis Web site.

References

Accominotti, Olivier, and Flandreau, Marc. 2008. Bilateral treaties and the most-favored-nation-clause: the myth of trade liberalization in the nineteenth century. World Politics 60(2):147–88.Google Scholar
Allison, Paul D. 2002. Missing data. Thousand Oaks, CA: Sage Publications.CrossRefGoogle Scholar
Allison, Paul D. 2005. Imputation of categorical variables with PROC MI. Paper 113–30, 30th Meeting of SAS Users Group International (SUGI 30). http://www2.sas.com/proceedings/sugi30/113-30.pdf.Google Scholar
Bodner, Todd E. 2008. What improves with increased missing data imputations? Structural Equation Modeling 15(4):651–75.Google Scholar
Collins, Linda M., Schafer, Joseph L, and Kam, Chi-Ming. 2001. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods 6(4):330–51.CrossRefGoogle ScholarPubMed
Cranmer, Skyler J., and Gill, Jeff. 2013. We have to be discrete about this: a non-parametric imputation technique for missing categorical data. British Journal of Political Science 43(2):425–49.Google Scholar
Dreher, Axel, and Gassebner, Martin. 2012. Do IMF and World Bank programs induce government crises? An empirical analysis. International Organization 66(2):329–58.CrossRefGoogle Scholar
Edwards, Martin S., Coolidge, Kelsey A., and Preston, Daria A. 2011. Who reveals? Transparency and the IMF's article IV consultations. Seton Hall University Working Paper Series. http://wp.peio.me/wp-content/uploads/2014/04/Conf5_Edwards-30.09.11.pdf.Google Scholar
Fearon, James D., and Laitin, David D. 2003. Ethnicity, insurgency, and civil war. American Political Science Review 97(1):75–90.CrossRefGoogle Scholar
Gleditsch, Kristian Skrede. 2002. Expanded trade and GDP data. Journal of Conflict Resolution 46(5):712–24.Google Scholar
Graham, John W. 2009. Missing data analysis: Making it work in the real world. Annual Review of Psychology 60:549–76.Google Scholar
Graham, John W., Olchowski, Allison E., and Gilreath, Tamika D. 2007. How many imputations are really needed? Some practical clarifications of multiple imputation theory. Prevention Science 8(3):206–13.CrossRefGoogle ScholarPubMed
Graham, John W., Hofer, Scott M., and MacKinnon, David P. 1996. Maximizing the usefulness of data obtained with planned missing value patterns: an application of maximum likelihood procedures. Multivariate Behavioral Research 31(2):197–218.Google Scholar
Guisinger, Alexandra, and Singer, David A. 2010. Exchange rate proclamations and inflation-fighting credibility. International Organization 64(Spring):313–37.Google Scholar
Hardt, Jochen, Herke, Max, and Leonhart, Rainer. 2012. Auxiliary variables in multiple imputation in regression with missing X: a warning against including too many in small sample research. BMC Medical Research Methodology 12(1):184–97.Google Scholar
Hollyer, James R., Peter Rosendorff, B., and Raymond Vreeland, James. 2011. Democracy and transparency. Journal of Politics 73(4):1191–205.Google Scholar
Honaker, James, King, Gary, and Blackwell, Matthew. 2011. Amelia II: a program for missing data. Journal of Statistical Software 45(7):1–47.Google Scholar
Horton, Nicholas J., Lipsitz, Stuart R., and Parzen, Michael. 2003. A potential for bias when rounding in multiple imputation. The American Statistician 57(4):229–32.Google Scholar
Houle, Christian. 2009. Inequality and democracy: why inequality harms consolidation but does not affect democratization. World Politics 61(4):589–622.Google Scholar
Keefer, Philip. 2007. Elections, special interests, and financial crisis. International Organization 61(3):607–41.CrossRefGoogle Scholar
King, Gary, Honaker, James, Joseph, Anne, and Scheve, Kenneth. 2001. Analyzing incomplete political science data: an alternative algorithm for multiple imputation. American Political Science Review 95(1):49–69.Google Scholar
Kropko, Jonathan, Goodrich, Ben, Gelman, Andrew, and Hill, Jennifer. 2014. Multiple imputation for continuous and categorical data: comparing joint multivariate normal and conditional approaches. Political Analysis 22(4):497–219.Google Scholar
Kurtz, Marcus J., and Brooks, Sarah M. 2008. Embedding neoliberal reform in Latin America. World Politics 60(2):231–80.CrossRefGoogle Scholar
Lall, Ranjit. Forthcoming. The missing dimension of the political resource curse debate. Comparative Political Studies.Google Scholar
Lall, Ranjit. 2016. Replication data for: how multiple imputation makes a difference. http://dx.doi.org/10.7910/DVN/CRLKIF, Harvard Dataverse.Google Scholar
Little, Roderick J. A. 1988. A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association 83(404):1198–202.Google Scholar
Little, Roderick J. A. 1993. Pattern-mixture models for multivariate incomplete data. Journal of the American Statistical Association 88(421):125–34.Google Scholar
Little, Roderick J.A., and Rubin, Donald. 1987. Statistical analysis with missing data. New York: Wiley.Google Scholar
Little, Roderick J.A., and Rubin, Donald. 2002. Statistical analysis with missing data, 2nd ed. Hoboken, NJ: Wiley.Google Scholar
Meng, Xiao-Li. 1994. Multiple-imputation inferences with uncongenial sources of input. Statistical Science 9(4):538–58.Google Scholar
Morrison, Kevin M. 2009. Oil, nontax revenue, and the redistributional foundations of regime stability. International Organization 63(1):107–38.Google Scholar
Obinger, Herbert, and Schmitt, Carina. 2011. Guns and butter? Regime competition and the welfare state during the Cold War. World Politics 63(2):246–70.Google Scholar
Ross, Michael. 2006. Is democracy good for the poor? American Journal of Political Science 50(4):860–74.Google Scholar
Ross, Michael L. 2004. What do we know about natural resources and civil war? Journal of Peace Research 41(3):337–56.Google Scholar
Rubin, Donald B. 1976. Inference and missing data (with discussion). Biometrika 63:581–92.Google Scholar
Rubin, Donald B. 1977. Formalizing subjective notions about the effect of nonrespondents in sample surveys. Journal of the American Statistical Association 72(359):538–43.Google Scholar
Rubin, Donald B. 1987. Multiple imputation for nonresponse in surveys. New York: John Wiley & Sons.Google Scholar
Rubin, Donald, and Schenker, Nathaniel. 1986. Multiple imputation for interval estimation from single random samples with ignorable nonresponse. Journal of the American Statistical Association 81(394):366–74.Google Scholar
Schafer, Joseph L. 1997. Analysis of incomplete multivariate data. London: Chapman and Hall.Google Scholar
Scheve, Kenneth, and Stasavage, David. 2009. Institutions, partisanship, and inequality in the long run. World Politics 61(2):215–53.Google Scholar
van Buuren, Stef. 2012. Flexible imputation of missing data. Boca Raton, FL: Taylor and Francis.Google Scholar
von Hippel, Paul T. 2009. How to impute squares, interactions, and other transformed variables. Sociological Methodology 39(1):265–91.Google Scholar
von Hippel, Paul T. 2013. Should a normal imputation model be modified to impute skewed variables? Sociological Methods and Research 42(1):105–38.Google Scholar
White, Ian R., Royston, Patrick, and Wood, Angela M. 2011. Multiple imputation using chained equations: issues and guidance for practice. Statistics in Medicine 30(4):377–99.Google Scholar
Supplementary material: PDF

Lall Supplementary Material

Appendix

Download Lall Supplementary Material(PDF)
PDF 187.5 KB