Skip to main content
×
Home
    • Aa
    • Aa

A New Multinomial Accuracy Measure for Polling Bias

  • Kai Arzheimer (a1) and Jocelyn Evans (a2)
Abstract

In this article, we propose a polling accuracy measure for multi-party elections based on a generalization of Martin, Traugott, and Kennedy's two-party predictive accuracy index. Treating polls as random samples of a voting population, we first estimate an intercept only multinomial logit model to provide proportionate odds measures of each party's share of the vote, and thereby both unweighted and weighted averages of these values as a summary index for poll accuracy. We then propose measures for significance testing, and run a series of simulations to assess possible bias from the resulting folded normal distribution across different sample sizes, finding that bias is small even for polls with small samples. We apply our measure to the 2012 French presidential election polls to demonstrate its applicability in tracking overall polling performance across time and polling organizations. Finally, we demonstrate the practical value of our measure by using it as a dependent variable in an explanatory model of polling accuracy, testing the different possible sources of bias in the French data.

Copyright
Corresponding author
e-mail: kai.arzheimer@googlemail.com (corresponding author)
Footnotes
Hide All

Authors' note: We are grateful to two anonymous reviewers and to the editors of Political Analysis for their extremely helpful comments and suggestions. We would also like to thank Gilles Ivaldi (CNRS/University of Nice) for providing data on the French election surveys. Replication materials can be found in Arzheimer and Evans (2013). surveybias.ado, a Stata add-on that facilitates the computation of B and Bw , is available from the Boston College Statistical Software Components (SSC) archive. Supplementary materials for this article are available on the Political Analysis Web site.

Footnotes
Linked references
Hide All

This list contains references from the content that can be linked to their source. For a full set of references and notes please see the PDF or HTML where available.

Alan Agresti . 2002. Categorical data analysis. 2nd ed. Hoboken, NJ: John Wiley.

Alan Agresti . 2002. Categorical data analysis. 2nd ed. Hoboken, NJ: John Wiley.

R. Michael Alvarez , and Jonathan Nagler . 1998. When politics and models collide: Estimating models of multiparty elections. American Journal of Political Science 42: 5596.

R. Michael Alvarez , and Jonathan Nagler . 1998. When politics and models collide: Estimating models of multiparty elections. American Journal of Political Science 42: 5596.

Mario Callegaro , and Giancarlo Gasperoni . 2008. Accuracy of pre-election polls for the 2006 Italian parliamentary election: Too close to call. International Journal of Public Opinion Research 20(2): 148–70.

Mario Callegaro , and Giancarlo Gasperoni . 2008. Accuracy of pre-election polls for the 2006 Italian parliamentary election: Too close to call. International Journal of Public Opinion Research 20(2): 148–70.

Simon Cheng , and J. Scott Long 2007. Testing for IIA in the multinomial logit model. Sociological Methods & Research 35(4): 583600.

Simon Cheng , and J. Scott Long 2007. Testing for IIA in the multinomial logit model. Sociological Methods & Research 35(4): 583600.

Jay K. Dow , and James W. Endersby 2004. Multinomial probit and multinomial logit: A comparison of choice models for voting research. Electoral Studies 23: 107–22.

Jay K. Dow , and James W. Endersby 2004. Multinomial probit and multinomial logit: A comparison of choice models for voting research. Electoral Studies 23: 107–22.

Claire Durand . 2008. The polls of the 2007 French presidential campaign: Were lessons learned from the 2002 catastrophe? International Journal of Public Opinion Research 20(3): 275–98.

Claire Durand . 2008. The polls of the 2007 French presidential campaign: Were lessons learned from the 2002 catastrophe? International Journal of Public Opinion Research 20(3): 275–98.

Stephen D. Fisher , Robert Ford , Will Jennings , Mark Pickup , and Christopher Wlezien . 2011. From polls to votes to seats: Forecasting the 2010 British general election. Electoral Studies 30(2): 250–57.

Stephen D. Fisher , Robert Ford , Will Jennings , Mark Pickup , and Christopher Wlezien . 2011. From polls to votes to seats: Forecasting the 2010 British general election. Electoral Studies 30(2): 250–57.

Simon Jackman . 2005. Pooling the polls over an election campaign. Australian Journal of Political Science 40(4): 499517.

Simon Jackman . 2005. Pooling the polls over an election campaign. Australian Journal of Political Science 40(4): 499517.

Elizabeth A. Martin , Michael W. Traugott , and Courtney Kennedy . 2005. A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly 69(3): 342–69.

Elizabeth A. Martin , Michael W. Traugott , and Courtney Kennedy . 2005. A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly 69(3): 342–69.

Iain McLean . 1995. Independence of irrelevant alternatives before Arrow. Mathematical Social Sciences 30(2): 107–26.

Iain McLean . 1995. Independence of irrelevant alternatives before Arrow. Mathematical Social Sciences 30(2): 107–26.

Karl Pearson . 1897. Mathematical contributions to the theory of evolution: On a form of spurious correlation which may arise when indices are used in the measurements of organs. Proceedings of the Royal Society of London 60: 489502.

Karl Pearson . 1897. Mathematical contributions to the theory of evolution: On a form of spurious correlation which may arise when indices are used in the measurements of organs. Proceedings of the Royal Society of London 60: 489502.

Paramesh Ray . 1973. Independence of irrelevant alternatives. Econometrica 41(5): 987–91.

Paramesh Ray . 1973. Independence of irrelevant alternatives. Econometrica 41(5): 987–91.

Patrick Royston , and Willi Sauerbrei . 2008. Multivariable model-building: A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester, UK: Wiley.

Patrick Royston , and Willi Sauerbrei . 2008. Multivariable model-building: A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester, UK: Wiley.

Lena-Maria Schaffer , and Gerald Schneider . 2005. Die Prognosegte von Wahlbörsen und Meinungsumfragen zur Bundestagswahl 2005. Politische Vierteljahresschrift 46(4): 674–81.

Lena-Maria Schaffer , and Gerald Schneider . 2005. Die Prognosegte von Wahlbörsen und Meinungsumfragen zur Bundestagswahl 2005. Politische Vierteljahresschrift 46(4): 674–81.

Kenneth Train . 2009. Discrete choice methods with simulation. 2nd ed. Cambridge, UK: Cambridge University Press.

Kenneth Train . 2009. Discrete choice methods with simulation. 2nd ed. Cambridge, UK: Cambridge University Press.

Guy D. Whitten , and Harvey D. Palmer 1996. Heightening comparativists' concern for model choice: Voting behavior in Great Britain and the Netherlands. American Journal of Political Science 40(1): 231–60.

Guy D. Whitten , and Harvey D. Palmer 1996. Heightening comparativists' concern for model choice: Voting behavior in Great Britain and the Netherlands. American Journal of Political Science 40(1): 231–60.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
MathJax
Type Description Title
UNKNOWN
Supplementary Materials

Arzheimer and Evans supplementary material
Supplementary Material

 Unknown (23 KB)
23 KB

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 18 *
Loading metrics...

Abstract views

Total abstract views: 94 *
Loading metrics...

* Views captured on Cambridge Core between 4th January 2017 - 20th September 2017. This data will be updated every 24 hours.