Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Home
  • Home
EnglishFrançais
Political Analysis Political Analysis

Article contents

Bayesian Model Averaging: Theoretical Developments and Practical Applications

Published online by Cambridge University Press:  04 January 2017

Jacob M. Montgomery  and
Brendan Nyhan
Jacob M. Montgomery
Affiliation:
Department of Political Science, Duke University, 326 Perkins Library, Box 90204, Durham, NC 27708. e-mail: jmm61@duke.edu
Brendan Nyhan
Affiliation:
School of Public Health, University of Michigan, 1420 Washington Heights, M2208 SPH-II, Ann Arbor, MI 48109
Corresponding
E-mail address:
jmm61@duke.edu
bnyhan@umich.edu
Get access
Rights & Permissions[Opens in a new window]

Abstract

Political science researchers typically conduct an idiosyncratic search of possible model configurations and then present a single specification to readers. This approach systematically understates the uncertainty of our results, generates fragile model specifications, and leads to the estimation of bloated models with too many control variables. Bayesian model averaging (BMA) offers a systematic method for analyzing specification uncertainty and checking the robustness of one's results to alternative model specifications, but it has not come into wide usage within the discipline. In this paper, we introduce important recent developments in BMA and show how they enable a different approach to using the technique in applied social science research. We illustrate the methodology by reanalyzing data from three recent studies using BMA software we have modified to respect statistical conventions within political science.

Type
Research Article
Information
Political Analysis , Volume 18 , Issue 2 , Spring 2010 , pp. 245 - 270
Copyright
Copyright © The Author 2010. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below.
If you should have access and can't see this content please contact technical support.

Footnotes

Authors' note: A poster based on an earlier version of this paper was presented at the Society for Political Methodology Summer Conference, State College, PA, July 18–21, 2007. We thank James Adams, Benjamin G. Bishin, David W. Brady, Brandice Canes-Wrone, John F. Cogan, Jay K. Dow, James D. Fearon, and David D. Laitin for sharing their data and providing assistance with our replications of their work. We also thank John H. Aldrich, Michael C. Brady, Merlise Clyde, Josh Cutler, Scott de Marchi, Andrew Gelman, Daniel J. Lee, Efrén O. Pérez, Jill Rickershauser, David Sparks, Michael W. Tofias, T. Camber Warren, the editors, and two anonymous reviewers for helpful comments. All remaining errors are, of course, our own. Replication materials are available on the Political Analysis Web site.

References

Achen, Christopher H. 2005. Let's put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science 22: 327–39.CrossRefGoogle Scholar
Adams, James, Bishin, Benjamin G., and Dow, Jay K. 2004. Representation in congressional campaigns: Evidence for discounting/directional voting in U.S. Senate elections. Journal of Politics 66: 348–73.CrossRefGoogle Scholar
Adams, James, and Merrill, Samuel. 1999. Modeling party strategies and policy representation in multiparty elections: Why are strategies so extreme? American Journal of Political Science 43: 765–91.CrossRefGoogle Scholar
Adler, E. Scott. n.d. Congressional district data file. Unpublished data-set, Boulder, CO: University of Colorado.Google Scholar
Akaike, Hirotugu. 1974. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19: 716–23.CrossRefGoogle Scholar
Bartels, Larry M. 1997. Specification uncertainty and model averaging. American Journal of Political Science 41: 641–74.CrossRefGoogle Scholar
Bartels, Larry M. 1998. Posterior distributions from model averaging: A clarification. Political Methodologist 8: 17–9.Google Scholar
Bartels, Larry M., and Zaller, John. 2001. Presidential vote models: A recount. PS: Political Science and Politics 34: 8–20.Google Scholar
Brambor, Thomas, Clark, William Roberts, and Golder, Matthew. 2006. Understanding interaction models: Improving empirical analyses. Political Analysis 14: 63–82.CrossRefGoogle Scholar
Braumoeller, Bear. 2004. Hypothesis testing and multiplicative interaction terms. International Organization 58: 807–20.CrossRefGoogle Scholar
Canes-Wrone, Brandice, Brady, David W., and Cogan, John F. 2002. Out of step, out of office: Electoral accountability and House members’ voting. American Political Science Review 96: 127–40.CrossRefGoogle Scholar
Clarke, Kevin A. 2001. Testing nonnested models of international relations: Reevaluating realism. American Journal of Political Science 45: 724–44.CrossRefGoogle Scholar
Clarke, Kevin A. 2005. The phantom menace: Omitted variable bias in econometric research. Conflict Management and Peace Science 22: 341–52.CrossRefGoogle Scholar
Clyde, Merlise. 1999. ‘Bayesian model averaging: A tutorial’: Comment. Statistical Science 14: 401–4.Google Scholar
Clyde, Merlise. 2003. Model averaging. In Subjective and objective Bayesian statistics. 2nd ed., Chap. 13, ed. James Press, S., 320–35. Hoboken, NJ: Wiley-Interscience.Google Scholar
Clyde, Merlise. (with contributions from Littman, Michael). 2009. BAS: Bayesian model averaging using Bayesian adaptive sampling. R Package Version 0.45. http://cran.r-project.org/package=BAS.Google Scholar
Clyde, Merlise, and George, Edward I. 2004. Model uncertainty. Statistical Science 19: 81–94.Google Scholar
Clyde, Merlise, Ghosh, Joyee, and Littman, Michael. 2009. Bayesian adaptive sampling for variable selection. Unpublished manuscript, Department of Statistical Science Discussion Paper, Duke University.Google Scholar
Draper, David. 1995. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society, Series B (Methodological) 57: 45–97.Google Scholar
Elazar, Daniel J. 1972. American federalism: A view from the states. 2nd ed. New York: Crowell.Google Scholar
Erikson, Robert S., Bafumi, Joseph, and Wilson, Bret. 2001. Was the 2000 presidential election predictable? PS: Political Science and Politics 34: 815–19.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1993. Statehouse democracy: Public opinion and policy in the American states. New York: Cambridge University Press.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1997. Too many variables? A comment on Bartels’ model-averaging proposal. Paper presented at the 1997 Political Methodology Conference, Columbus, Ohio.Google Scholar
Fearon, James D., and Laitin, David D. 2003a. Additional tables for ‘ethnicity, insurgency, and civil war’. Unpublished manuscript, Stanford University.Google Scholar
Fearon, James D., and Laitin, David D. 2003b. Ethnicity, insurgency, and civil war. American Political Science Review 97: 75–90.CrossRefGoogle Scholar
Fernandez, Carmen, Ley, Eduardo, and Steel, Mark F. J. 2001. Model uncertainty in cross-country growth regressions. Journal of Applied Econometrics 16: 563–76.CrossRefGoogle Scholar
Geer, John, and Lau, Richard R. 2006. Filling in the blanks: A new method for estimating campaign effects. British Journal of Political Science 36: 269–90.CrossRefGoogle Scholar
Gelman, Andrew, and Rubin, Donald B. 1995. Avoiding model selection in Bayesian social research. Sociological Methodology 25: 165–73.CrossRefGoogle Scholar
Gerber, Alan, and Malhotra, Neil. 2008. Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Quarterly Journal of Political Science 3: 313–26.CrossRefGoogle Scholar
Gill, Jeff. 1999. The insignificance of null hypothesis significance testing. Political Research Quarterly 52: 647–74.CrossRefGoogle Scholar
Gill, Jeff. 2004. Introduction to the special issue. Political Analysis 12: 323–37.CrossRefGoogle Scholar
Griffin, John, and Newman, Brian. 2009. Assessing accountability. Paper presented at the annual meeting of the Midwest Political Science Association, Chicago, IL.Google Scholar
Harrell, Frank E. 2001. Regression modeling strategies. New York: Springer.CrossRefGoogle Scholar
Ho, Daniel E., Imai, Kosuke, King, Gary, and Stuart, Elizabeth A. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis 15: 199–236.CrossRefGoogle Scholar
Hoeting, Jennifer A., Madigan, David, Raftery, Adrian E., and Volinsky, Christopher T. 1999. Bayesian model averaging: A tutorial. Statistical Science 14: 382–401.Google Scholar
Imai, Kosuke, and King, Gary. 2004. Did illegal overseas absentee ballots decide the 2000 U.S. presidential election? Perspectives on Politics 2: 537–49.CrossRefGoogle Scholar
Iversen, Torben. 1994. Political leadership and representation in west European democracies: A test of three models of voting. American Journal of Political Science 38: 45–74.CrossRefGoogle Scholar
Jackman, Robert W. 1987. The politics of economic growth in the industrial democracies, 1974–80: Leftist strength or North Sea oil? Journal of Politics 49: 242–56.CrossRefGoogle Scholar
Jeffreys, Harold. 1935. Some tests of significance, treated by the theory of probability. Proceedings of the Cambridge Philosophical Society 31: 203–22.CrossRefGoogle Scholar
Jeffreys, Harold. 1961. Theory of probability. 3rd ed. Oxford: Oxford University Press.Google Scholar
Kass, Robert E., and Raftery, Adrian E. 1995. Bayes factors. Journal of the American Statistical Association 90: 773–95.CrossRefGoogle Scholar
King, Gary, and Zeng, Langche. 2006. The dangers of extreme counterfactuals. Political Analysis 14: 131–59.CrossRefGoogle Scholar
Kuha, Jouni. 2004. AIC and BIC: Comparisons of assumptions and performance. Sociological Methods Research 33: 188–229.CrossRefGoogle Scholar
Lange, Peter, and Garrett, Geoffrey. 1985. The politics of growth: Strategic interaction and economic performance in the advanced industrial democracies, 1974–1980. Journal of Politics 47: 792–827.CrossRefGoogle Scholar
Liang, Feng, Paulo, Rui, Molina, German, Clyde, Merlise A., and Berger, Jim O. 2008. Mixtures of g-priors for Bayesian variable selection. Journal of the American Statistical Association 103: 410–23.CrossRefGoogle Scholar
Madigan, David, and Raftery, Adrian E. 1994. Model selection and accounting for model uncertainty in graphical models using Occam's Window. Journal of the American Statistical Association 89: 1535–46.CrossRefGoogle Scholar
Merrill, Samuel, and Grofman, Bernard. 1999. A unified theory of voting: Directional and proximity spatial models. New York: Cambridge University Press.CrossRefGoogle Scholar
Montgomery, Jacob, and Nyhan, Brendan. 2008. Bayesian model averaging: Theoretical developments and practical applications. Working paper, Society for Political Methodology.Google Scholar
Morales, Knashawn H., Ibrahim, Joseph G., Chen, Chien-Jen, and Ryan, Louise M. 2006. Bayesian model averaging with applications to benchmark dose estimation for arsenic in drinking water. Journal of the American Statistical Association 101: 9–17.CrossRefGoogle Scholar
Pang, Xun, and Gill, Jeff. 2009. Spike and slab prior distributions for simultaneous Bayesian hypothesis testing, model selection, and prediction, of nonlinear outcomes. Unpublished manuscript, Washington University in St. Louis.Google Scholar
Poole, Keith T., and Rosenthal, Howard. 1997. Congress: A political-economic history of roll call voting. New York: Oxford University Press.Google Scholar
Poole, Keith T., and Rosenthal, Howard. 2007. Ideology and congress. New Brunswick, NJ: Transaction Publishers.Google Scholar
Raftery, Adrian E. 1995. Bayesian model selection in social research. Sociological Methodology 25: 111–63.Google Scholar
Raftery, Adrian E., Hoeting, Jennifer, Volinsky, Chris T., Painter, Ian S., and Yeung, Ka Yee. 2009. BMA: Bayesian model averaging. R package version 3.12. http://cran.r-project.org/package=BMA.Google Scholar
Raftery, Adrian E., Painter, Ian S., and Volinsky, Christopher T. 2005. BMA: An R package for Bayesian model averaging. R News 5: 2–8.Google Scholar
Weakliem, David L. 1999. A critique of the Bayesian information criterion for model selection. Sociological Methods & Research 27: 359–97.CrossRefGoogle Scholar
Wintle, B.A., McCarthy, M.A., Volinsky, C. T., and Kavanagh, R. P. 2003. The use of Bayesian model averaging to better represent uncertainty in ecological models. Conservation Biology 17: 1579–90.CrossRefGoogle Scholar
Yeung, Ka Yee, Bumgarner, Roger E., and Raftery, Adrian E. 2005. Bayesian model averaging: development of an improved multi-class, gene selection and classification tool for microarray data. Bioinformatics 21: 2394–402.CrossRefGoogle ScholarPubMed
Zaller, John R. 2004. Floating voters in U.S. presidential elections, 1948–2000. In Studies in public opinion: Attitudes, nonattitudes, measurement error, and change, eds. Saris, Willem and Sniderman, Paul M., 166–214. Princeton, NJ: Princeton University Press.Google Scholar
Zellner, Arnold. 1986. On assessing prior distributions and Bayesian regression analysis with g-prior distributions. In Bayesian inference and decision techniques: Essays in honor of Burno de Finetti, eds. Goel, Prem K. and Zellner, Arnold, 233–43. North-Holland, The Netherlands: Elsevier.Google Scholar
Zellner, Arnold, and Siow, Aloysius. 1980. Posterior odds ratios for selected hypotheses. In Bayesian statistics, eds. Bernardo, J. M., DeGroot, M. H., Lindley, D. V., and Smith, A. F. M. Valencia, Spain: University Press.Google Scholar

Altmetric attention score

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 0
Total number of PDF views: 233 *
View data table for this chart

* Views captured on Cambridge Core between 04th January 2017 - 27th January 2021. This data will be updated every 24 hours.

72 Cited by
Hostname: page-component-898fc554b-54xgk Total loading time: 0.367 Render date: 2021-01-27T20:13:19.650Z Query parameters: { "hasAccess": "0", "openAccess": "0", "isLogged": "0", "lang": "en" } Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": false, "newCiteModal": false }

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Bayesian Model Averaging: Theoretical Developments and Practical Applications
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Bayesian Model Averaging: Theoretical Developments and Practical Applications
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Bayesian Model Averaging: Theoretical Developments and Practical Applications
Available formats
×
×

Reply to: Submit a response

Your details

Conflicting interests

Do you have any conflicting interests? *