Skip to main content Accessibility help
×
Home

Bayesian Model Averaging: Theoretical Developments and Practical Applications

  • Jacob M. Montgomery (a1) and Brendan Nyhan (a2)

Abstract

Political science researchers typically conduct an idiosyncratic search of possible model configurations and then present a single specification to readers. This approach systematically understates the uncertainty of our results, generates fragile model specifications, and leads to the estimation of bloated models with too many control variables. Bayesian model averaging (BMA) offers a systematic method for analyzing specification uncertainty and checking the robustness of one's results to alternative model specifications, but it has not come into wide usage within the discipline. In this paper, we introduce important recent developments in BMA and show how they enable a different approach to using the technique in applied social science research. We illustrate the methodology by reanalyzing data from three recent studies using BMA software we have modified to respect statistical conventions within political science.

Copyright

Corresponding author

e-mail: bnyhan@umich.edu (corresponding author)

Footnotes

Hide All

Authors' note: A poster based on an earlier version of this paper was presented at the Society for Political Methodology Summer Conference, State College, PA, July 18–21, 2007. We thank James Adams, Benjamin G. Bishin, David W. Brady, Brandice Canes-Wrone, John F. Cogan, Jay K. Dow, James D. Fearon, and David D. Laitin for sharing their data and providing assistance with our replications of their work. We also thank John H. Aldrich, Michael C. Brady, Merlise Clyde, Josh Cutler, Scott de Marchi, Andrew Gelman, Daniel J. Lee, Efrén O. Pérez, Jill Rickershauser, David Sparks, Michael W. Tofias, T. Camber Warren, the editors, and two anonymous reviewers for helpful comments. All remaining errors are, of course, our own. Replication materials are available on the Political Analysis Web site.

Footnotes

References

Hide All
Achen, Christopher H. 2005. Let's put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science 22: 327–39.
Adams, James, Bishin, Benjamin G., and Dow, Jay K. 2004. Representation in congressional campaigns: Evidence for discounting/directional voting in U.S. Senate elections. Journal of Politics 66: 348–73.
Adams, James, and Merrill, Samuel. 1999. Modeling party strategies and policy representation in multiparty elections: Why are strategies so extreme? American Journal of Political Science 43: 765–91.
Adler, E. Scott. n.d. Congressional district data file. Unpublished data-set, Boulder, CO: University of Colorado.
Akaike, Hirotugu. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19: 716–23.
Bartels, Larry M. 1997. Specification uncertainty and model averaging. American Journal of Political Science 41: 641–74.
Bartels, Larry M. 1998. Posterior distributions from model averaging: A clarification. Political Methodologist 8: 17–9.
Bartels, Larry M., and Zaller, John. 2001. Presidential vote models: A recount. PS: Political Science and Politics 34: 820.
Brambor, Thomas, Clark, William Roberts, and Golder, Matthew. 2006. Understanding interaction models: Improving empirical analyses. Political Analysis 14: 6382.
Braumoeller, Bear. 2004. Hypothesis testing and multiplicative interaction terms. International Organization 58: 807–20.
Canes-Wrone, Brandice, Brady, David W., and Cogan, John F. 2002. Out of step, out of office: Electoral accountability and House members’ voting. American Political Science Review 96: 127–40.
Clarke, Kevin A. 2001. Testing nonnested models of international relations: Reevaluating realism. American Journal of Political Science 45: 724–44.
Clarke, Kevin A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22: 341–52.
Clyde, Merlise. 1999. ‘Bayesian model averaging: A tutorial’: Comment. Statistical Science 14: 401–4.
Clyde, Merlise. 2003. Model averaging. In Subjective and objective Bayesian statistics. 2nd ed., Chap. 13, ed. James Press, S., 320–35. Hoboken, NJ: Wiley-Interscience.
Clyde, Merlise. (with contributions from Littman, Michael). 2009. BAS: Bayesian model averaging using Bayesian adaptive sampling. R Package Version 0.45. http://cran.r-project.org/package=BAS.
Clyde, Merlise, and George, Edward I. 2004. Model uncertainty. Statistical Science 19: 8194.
Clyde, Merlise, Ghosh, Joyee, and Littman, Michael. 2009. Bayesian adaptive sampling for variable selection. Unpublished manuscript, Department of Statistical Science Discussion Paper, Duke University.
Draper, David. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society, Series B (Methodological) 57: 4597.
Elazar, Daniel J. 1972. American federalism: A view from the states. 2nd ed. New York: Crowell.
Erikson, Robert S., Bafumi, Joseph, and Wilson, Bret. 2001. Was the 2000 presidential election predictable? PS: Political Science and Politics 34: 815–19.
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1993. Statehouse democracy: Public opinion and policy in the American states. New York: Cambridge University Press.
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1997. Too many variables? A comment on Bartels’ model-averaging proposal. Paper presented at the 1997 Political Methodology Conference, Columbus, Ohio.
Fearon, James D., and Laitin, David D. 2003a. Additional tables for ‘ethnicity, insurgency, and civil war’. Unpublished manuscript, Stanford University.
Fearon, James D., and Laitin, David D. 2003b. Ethnicity, insurgency, and civil war. American Political Science Review 97: 7590.
Fernandez, Carmen, Ley, Eduardo, and Steel, Mark F. J. 2001. Model uncertainty in cross-country growth regressions. Journal of Applied Econometrics 16: 563–76.
Geer, John, and Lau, Richard R. 2006. Filling in the blanks: A new method for estimating campaign effects. British Journal of Political Science 36: 269–90.
Gelman, Andrew, and Rubin, Donald B. 1995. Avoiding model selection in Bayesian social research. Sociological Methodology 25: 165–73.
Gerber, Alan, and Malhotra, Neil. 2008. Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Quarterly Journal of Political Science 3: 313–26.
Gill, Jeff. 1999. The insignificance of null hypothesis significance testing. Political Research Quarterly 52: 647–74.
Gill, Jeff. 2004. Introduction to the special issue. Political Analysis 12: 323–37.
Griffin, John, and Newman, Brian. 2009. Assessing accountability. Paper presented at the annual meeting of the Midwest Political Science Association, Chicago, IL.
Harrell, Frank E. 2001. Regression modeling strategies. New York: Springer.
Ho, Daniel E., Imai, Kosuke, King, Gary, and Stuart, Elizabeth A. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15: 199236.
Hoeting, Jennifer A., Madigan, David, Raftery, Adrian E., and Volinsky, Christopher T. 1999. Bayesian model averaging: A tutorial. Statistical Science 14: 382401.
Imai, Kosuke, and King, Gary. 2004. Did illegal overseas absentee ballots decide the 2000 U.S. presidential election? Perspectives on Politics 2: 537–49.
Iversen, Torben. 1994. Political leadership and representation in west European democracies: A test of three models of voting. American Journal of Political Science 38: 4574.
Jackman, Robert W. 1987. The politics of economic growth in the industrial democracies, 1974–80: Leftist strength or North Sea oil? Journal of Politics 49: 242–56.
Jeffreys, Harold. 1935. Some tests of significance, treated by the theory of probability. Proceedings of the Cambridge Philosophical Society 31: 203–22.
Jeffreys, Harold. 1961. Theory of probability. 3rd ed. Oxford: Oxford University Press.
Kass, Robert E., and Raftery, Adrian E. 1995. Bayes factors. Journal of the American Statistical Association 90: 773–95.
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14: 131–59.
Kuha, Jouni. 2004. AIC and BIC: Comparisons of assumptions and performance. Sociological Methods Research 33: 188229.
Lange, Peter, and Garrett, Geoffrey. 1985. The politics of growth: Strategic interaction and economic performance in the advanced industrial democracies, 1974–1980. Journal of Politics 47: 792827.
Liang, Feng, Paulo, Rui, Molina, German, Clyde, Merlise A., and Berger, Jim O. 2008. Mixtures of g-priors for Bayesian variable selection. Journal of the American Statistical Association 103: 410–23.
Madigan, David, and Raftery, Adrian E. 1994. Model selection and accounting for model uncertainty in graphical models using Occam's Window. Journal of the American Statistical Association 89: 1535–46.
Merrill, Samuel, and Grofman, Bernard. 1999. A unified theory of voting: Directional and proximity spatial models. New York: Cambridge University Press.
Montgomery, Jacob, and Nyhan, Brendan. 2008. Bayesian model averaging: Theoretical developments and practical applications. Working paper, Society for Political Methodology.
Morales, Knashawn H., Ibrahim, Joseph G., Chen, Chien-Jen, and Ryan, Louise M. 2006. Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. Journal of the American Statistical Association 101: 917.
Pang, Xun, and Gill, Jeff. 2009. Spike and slab prior distributions for simultaneous Bayesian hypothesis testing, model selection, and prediction, of nonlinear outcomes. Unpublished manuscript, Washington University in St. Louis.
Poole, Keith T., and Rosenthal, Howard. 1997. Congress: A political-economic history of roll call voting. New York: Oxford University Press.
Poole, Keith T., and Rosenthal, Howard. 2007. Ideology and congress. New Brunswick, NJ: Transaction Publishers.
Raftery, Adrian E. 1995. Bayesian model selection in social research. Sociological Methodology 25: 111–63.
Raftery, Adrian E., Hoeting, Jennifer, Volinsky, Chris T., Painter, Ian S., and Yeung, Ka Yee. 2009. BMA: Bayesian model averaging. R package version 3.12. http://cran.r-project.org/package=BMA.
Raftery, Adrian E., Painter, Ian S., and Volinsky, Christopher T. 2005. BMA: An R package for Bayesian model averaging. R News 5: 28.
Weakliem, David L. 1999. A critique of the Bayesian information criterion for model selection. Sociological Methods & Research 27: 359–97.
Wintle, B.A., McCarthy, M.A., Volinsky, C. T., and Kavanagh, R. P. 2003. The use of Bayesian model averaging to better represent uncertainty in ecological models. Conservation Biology 17: 1579–90.
Yeung, Ka Yee, Bumgarner, Roger E., and Raftery, Adrian E. 2005. Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21: 2394–402.
Zaller, John R. 2004. Floating voters in U.S. presidential elections, 1948–2000. In Studies in public opinion: Attitudes, nonattitudes, measurement error, and change, eds. Saris, Willem and Sniderman, Paul M., 166214. Princeton, NJ: Princeton University Press.
Zellner, Arnold. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian inference and decision techniques: Essays in honor of Burno de Finetti, eds. Goel, Prem K. and Zellner, Arnold, 233–43. North-Holland, The Netherlands: Elsevier.
Zellner, Arnold, and Siow, Aloysius. 1980. Posterior odds ratios for selected hypotheses. In Bayesian statistics, eds. Bernardo, J. M., DeGroot, M. H., Lindley, D. V., and Smith, A. F. M. Valencia, Spain: University Press.
MathJax
MathJax is a JavaScript display engine for mathematics. For more information see http://www.mathjax.org.

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed