Hostname: page-component-76fb5796d-zzh7m Total loading time: 0 Render date: 2024-04-26T08:25:42.479Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Model Averaging: Theoretical Developments and Practical Applications

Published online by Cambridge University Press:  04 January 2017

Jacob M. Montgomery  and
Brendan Nyhan
Jacob M. Montgomery
Affiliation:
Department of Political Science, Duke University, 326 Perkins Library, Box 90204, Durham, NC 27708. e-mail: jmm61@duke.edu
Brendan Nyhan*
Affiliation:
School of Public Health, University of Michigan, 1420 Washington Heights, M2208 SPH-II, Ann Arbor, MI 48109
*
e-mail: bnyhan@umich.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

Political science researchers typically conduct an idiosyncratic search of possible model configurations and then present a single specification to readers. This approach systematically understates the uncertainty of our results, generates fragile model specifications, and leads to the estimation of bloated models with too many control variables. Bayesian model averaging (BMA) offers a systematic method for analyzing specification uncertainty and checking the robustness of one's results to alternative model specifications, but it has not come into wide usage within the discipline. In this paper, we introduce important recent developments in BMA and show how they enable a different approach to using the technique in applied social science research. We illustrate the methodology by reanalyzing data from three recent studies using BMA software we have modified to respect statistical conventions within political science.

Type
Research Article
Information
Political Analysis , Volume 18 , Issue 2 , Spring 2010 , pp. 245 - 270
Copyright
Copyright © The Author 2010. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Authors' note: A poster based on an earlier version of this paper was presented at the Society for Political Methodology Summer Conference, State College, PA, July 18–21, 2007. We thank James Adams, Benjamin G. Bishin, David W. Brady, Brandice Canes-Wrone, John F. Cogan, Jay K. Dow, James D. Fearon, and David D. Laitin for sharing their data and providing assistance with our replications of their work. We also thank John H. Aldrich, Michael C. Brady, Merlise Clyde, Josh Cutler, Scott de Marchi, Andrew Gelman, Daniel J. Lee, Efrén O. Pérez, Jill Rickershauser, David Sparks, Michael W. Tofias, T. Camber Warren, the editors, and two anonymous reviewers for helpful comments. All remaining errors are, of course, our own. Replication materials are available on the Political Analysis Web site.

References

Achen, Christopher H. 2005. Let's put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science 22: 327–39.Google Scholar
Adams, James, Bishin, Benjamin G., and Dow, Jay K. 2004. Representation in congressional campaigns: Evidence for discounting/directional voting in U.S. Senate elections. Journal of Politics 66: 348–73.Google Scholar
Adams, James, and Merrill, Samuel. 1999. Modeling party strategies and policy representation in multiparty elections: Why are strategies so extreme? American Journal of Political Science 43: 765–91.Google Scholar
Adler, E. Scott. n.d. Congressional district data file. Unpublished data-set, Boulder, CO: University of Colorado.Google Scholar
Akaike, Hirotugu. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19: 716–23.Google Scholar
Bartels, Larry M. 1997. Specification uncertainty and model averaging. American Journal of Political Science 41: 641–74.Google Scholar
Bartels, Larry M. 1998. Posterior distributions from model averaging: A clarification. Political Methodologist 8: 17–9.Google Scholar
Bartels, Larry M., and Zaller, John. 2001. Presidential vote models: A recount. PS: Political Science and Politics 34: 8–20.Google Scholar
Brambor, Thomas, Clark, William Roberts, and Golder, Matthew. 2006. Understanding interaction models: Improving empirical analyses. Political Analysis 14: 63–82.Google Scholar
Braumoeller, Bear. 2004. Hypothesis testing and multiplicative interaction terms. International Organization 58: 807–20.CrossRefGoogle Scholar
Canes-Wrone, Brandice, Brady, David W., and Cogan, John F. 2002. Out of step, out of office: Electoral accountability and House members’ voting. American Political Science Review 96: 127–40.Google Scholar
Clarke, Kevin A. 2001. Testing nonnested models of international relations: Reevaluating realism. American Journal of Political Science 45: 724–44.Google Scholar
Clarke, Kevin A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22: 341–52.CrossRefGoogle Scholar
Clyde, Merlise. 1999. ‘Bayesian model averaging: A tutorial’: Comment. Statistical Science 14: 401–4.Google Scholar
Clyde, Merlise. 2003. Model averaging. In Subjective and objective Bayesian statistics. 2nd ed., Chap. 13, ed. James Press, S., 320–35. Hoboken, NJ: Wiley-Interscience.Google Scholar
Clyde, Merlise. (with contributions from Littman, Michael). 2009. BAS: Bayesian model averaging using Bayesian adaptive sampling. R Package Version 0.45. http://cran.r-project.org/package=BAS.Google Scholar
Clyde, Merlise, and George, Edward I. 2004. Model uncertainty. Statistical Science 19: 81–94.Google Scholar
Clyde, Merlise, Ghosh, Joyee, and Littman, Michael. 2009. Bayesian adaptive sampling for variable selection. Unpublished manuscript, Department of Statistical Science Discussion Paper, Duke University.Google Scholar
Draper, David. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society, Series B (Methodological) 57: 45–97.Google Scholar
Elazar, Daniel J. 1972. American federalism: A view from the states. 2nd ed. New York: Crowell.Google Scholar
Erikson, Robert S., Bafumi, Joseph, and Wilson, Bret. 2001. Was the 2000 presidential election predictable? PS: Political Science and Politics 34: 815–19.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1993. Statehouse democracy: Public opinion and policy in the American states. New York: Cambridge University Press.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1997. Too many variables? A comment on Bartels’ model-averaging proposal. Paper presented at the 1997 Political Methodology Conference, Columbus, Ohio.Google Scholar
Fearon, James D., and Laitin, David D. 2003a. Additional tables for ‘ethnicity, insurgency, and civil war’. Unpublished manuscript, Stanford University.Google Scholar
Fearon, James D., and Laitin, David D. 2003b. Ethnicity, insurgency, and civil war. American Political Science Review 97: 75–90.Google Scholar
Fernandez, Carmen, Ley, Eduardo, and Steel, Mark F. J. 2001. Model uncertainty in cross-country growth regressions. Journal of Applied Econometrics 16: 563–76.Google Scholar
Geer, John, and Lau, Richard R. 2006. Filling in the blanks: A new method for estimating campaign effects. British Journal of Political Science 36: 269–90.Google Scholar
Gelman, Andrew, and Rubin, Donald B. 1995. Avoiding model selection in Bayesian social research. Sociological Methodology 25: 165–73.Google Scholar
Gerber, Alan, and Malhotra, Neil. 2008. Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Quarterly Journal of Political Science 3: 313–26.Google Scholar
Gill, Jeff. 1999. The insignificance of null hypothesis significance testing. Political Research Quarterly 52: 647–74.CrossRefGoogle Scholar
Gill, Jeff. 2004. Introduction to the special issue. Political Analysis 12: 323–37.Google Scholar
Griffin, John, and Newman, Brian. 2009. Assessing accountability. Paper presented at the annual meeting of the Midwest Political Science Association, Chicago, IL.Google Scholar
Harrell, Frank E. 2001. Regression modeling strategies. New York: Springer.Google Scholar
Ho, Daniel E., Imai, Kosuke, King, Gary, and Stuart, Elizabeth A. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15: 199–236.Google Scholar
Hoeting, Jennifer A., Madigan, David, Raftery, Adrian E., and Volinsky, Christopher T. 1999. Bayesian model averaging: A tutorial. Statistical Science 14: 382–401.Google Scholar
Imai, Kosuke, and King, Gary. 2004. Did illegal overseas absentee ballots decide the 2000 U.S. presidential election? Perspectives on Politics 2: 537–49.Google Scholar
Iversen, Torben. 1994. Political leadership and representation in west European democracies: A test of three models of voting. American Journal of Political Science 38: 45–74.Google Scholar
Jackman, Robert W. 1987. The politics of economic growth in the industrial democracies, 1974–80: Leftist strength or North Sea oil? Journal of Politics 49: 242–56.Google Scholar
Jeffreys, Harold. 1935. Some tests of significance, treated by the theory of probability. Proceedings of the Cambridge Philosophical Society 31: 203–22.Google Scholar
Jeffreys, Harold. 1961. Theory of probability. 3rd ed. Oxford: Oxford University Press.Google Scholar
Kass, Robert E., and Raftery, Adrian E. 1995. Bayes factors. Journal of the American Statistical Association 90: 773–95.Google Scholar
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14: 131–59.Google Scholar
Kuha, Jouni. 2004. AIC and BIC: Comparisons of assumptions and performance. Sociological Methods Research 33: 188–229.Google Scholar
Lange, Peter, and Garrett, Geoffrey. 1985. The politics of growth: Strategic interaction and economic performance in the advanced industrial democracies, 1974–1980. Journal of Politics 47: 792–827.CrossRefGoogle Scholar
Liang, Feng, Paulo, Rui, Molina, German, Clyde, Merlise A., and Berger, Jim O. 2008. Mixtures of g-priors for Bayesian variable selection. Journal of the American Statistical Association 103: 410–23.Google Scholar
Madigan, David, and Raftery, Adrian E. 1994. Model selection and accounting for model uncertainty in graphical models using Occam's Window. Journal of the American Statistical Association 89: 1535–46.Google Scholar
Merrill, Samuel, and Grofman, Bernard. 1999. A unified theory of voting: Directional and proximity spatial models. New York: Cambridge University Press.Google Scholar
Montgomery, Jacob, and Nyhan, Brendan. 2008. Bayesian model averaging: Theoretical developments and practical applications. Working paper, Society for Political Methodology.Google Scholar
Morales, Knashawn H., Ibrahim, Joseph G., Chen, Chien-Jen, and Ryan, Louise M. 2006. Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. Journal of the American Statistical Association 101: 9–17.Google Scholar
Pang, Xun, and Gill, Jeff. 2009. Spike and slab prior distributions for simultaneous Bayesian hypothesis testing, model selection, and prediction, of nonlinear outcomes. Unpublished manuscript, Washington University in St. Louis.Google Scholar
Poole, Keith T., and Rosenthal, Howard. 1997. Congress: A political-economic history of roll call voting. New York: Oxford University Press.Google Scholar
Poole, Keith T., and Rosenthal, Howard. 2007. Ideology and congress. New Brunswick, NJ: Transaction Publishers.Google Scholar
Raftery, Adrian E. 1995. Bayesian model selection in social research. Sociological Methodology 25: 111–63.Google Scholar
Raftery, Adrian E., Hoeting, Jennifer, Volinsky, Chris T., Painter, Ian S., and Yeung, Ka Yee. 2009. BMA: Bayesian model averaging. R package version 3.12. http://cran.r-project.org/package=BMA.Google Scholar
Raftery, Adrian E., Painter, Ian S., and Volinsky, Christopher T. 2005. BMA: An R package for Bayesian model averaging. R News 5: 2–8.Google Scholar
Weakliem, David L. 1999. A critique of the Bayesian information criterion for model selection. Sociological Methods & Research 27: 359–97.Google Scholar
Wintle, B.A., McCarthy, M.A., Volinsky, C. T., and Kavanagh, R. P. 2003. The use of Bayesian model averaging to better represent uncertainty in ecological models. Conservation Biology 17: 1579–90.Google Scholar
Yeung, Ka Yee, Bumgarner, Roger E., and Raftery, Adrian E. 2005. Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21: 2394–402.Google Scholar
Zaller, John R. 2004. Floating voters in U.S. presidential elections, 1948–2000. In Studies in public opinion: Attitudes, nonattitudes, measurement error, and change, eds. Saris, Willem and Sniderman, Paul M., 166–214. Princeton, NJ: Princeton University Press.Google Scholar
Zellner, Arnold. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian inference and decision techniques: Essays in honor of Burno de Finetti, eds. Goel, Prem K. and Zellner, Arnold, 233–43. North-Holland, The Netherlands: Elsevier.Google Scholar
Zellner, Arnold, and Siow, Aloysius. 1980. Posterior odds ratios for selected hypotheses. In Bayesian statistics, eds. Bernardo, J. M., DeGroot, M. H., Lindley, D. V., and Smith, A. F. M. Valencia, Spain: University Press.Google Scholar