Skip to main content Accessibility help
×
Home
Hostname: page-component-59df476f6b-6nvld Total loading time: 0.219 Render date: 2021-05-17T20:55:59.999Z Has data issue: false Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true }

A New Multinomial Accuracy Measure for Polling Bias

Published online by Cambridge University Press:  04 January 2017

Kai Arzheimer
Affiliation:
University of Mainz, Dept. of Political Science, Colonel-Kleinmann-Weg 2, Mainz, Germany
Jocelyn Evans
Affiliation:
University of Leeds, POLIS, Leeds LS2 9JT, United Kingdom e-mail: j.a.j.evans@leeds.ac.uk
Corresponding
Rights & Permissions[Opens in a new window]

Abstract

In this article, we propose a polling accuracy measure for multi-party elections based on a generalization of Martin, Traugott, and Kennedy's two-party predictive accuracy index. Treating polls as random samples of a voting population, we first estimate an intercept only multinomial logit model to provide proportionate odds measures of each party's share of the vote, and thereby both unweighted and weighted averages of these values as a summary index for poll accuracy. We then propose measures for significance testing, and run a series of simulations to assess possible bias from the resulting folded normal distribution across different sample sizes, finding that bias is small even for polls with small samples. We apply our measure to the 2012 French presidential election polls to demonstrate its applicability in tracking overall polling performance across time and polling organizations. Finally, we demonstrate the practical value of our measure by using it as a dependent variable in an explanatory model of polling accuracy, testing the different possible sources of bias in the French data.

Type
Research Article
Copyright
Copyright © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology 

Footnotes

Authors' note: We are grateful to two anonymous reviewers and to the editors of Political Analysis for their extremely helpful comments and suggestions. We would also like to thank Gilles Ivaldi (CNRS/University of Nice) for providing data on the French election surveys. Replication materials can be found in Arzheimer and Evans (2013). surveybias.ado, a Stata add-on that facilitates the computation of B and Bw , is available from the Boston College Statistical Software Components (SSC) archive. Supplementary materials for this article are available on the Political Analysis Web site.

References

Agresti, Alan. 2002. Categorical data analysis. 2nd ed. Hoboken, NJ: John Wiley.CrossRefGoogle Scholar
Aitchison, John. 1982. The statistical analysis of compositional data (with discussion). Journal of the Royal Statistical Society, Series B 44(2): 139–77.Google Scholar
Alvarez, R. Michael, and Nagler, Jonathan. 1998. When politics and models collide: Estimating models of multiparty elections. American Journal of Political Science 42: 5596.CrossRefGoogle Scholar
Arzheimer, Kai, and Evans, Jocelyn. 2013. Replication data for: A new multinomial accuracy measure for polling bias. IQSS Dataverse Network/Distributor V1 [Version]. http://hdl.handle.net/1902.1/21603 (accessed August 6, 2013).Google Scholar
Bacon-Shone, John. 2011. A short history of compositional data analysis. In Compositional data analysis: Theory and applications, eds. Pawlowsky-Glahn, Vera and Buccianti, Antonella, 111. Chichester, UK: Wiley.Google Scholar
Callegaro, Mario, and Gasperoni, Giancarlo. 2008. Accuracy of pre-election polls for the 2006 Italian parliamentary election: Too close to call. International Journal of Public Opinion Research 20(2): 148–70.CrossRefGoogle Scholar
Cheng, Simon, and Scott Long, J. 2007. Testing for IIA in the multinomial logit model. Sociological Methods & Research 35(4): 583600.CrossRefGoogle Scholar
Dow, Jay K., and Endersby, James W. 2004. Multinomial probit and multinomial logit: A comparison of choice models for voting research. Electoral Studies 23: 107–22.CrossRefGoogle Scholar
Durand, Claire. 2008. The polls of the 2007 French presidential campaign: Were lessons learned from the 2002 catastrophe? International Journal of Public Opinion Research 20(3): 275–98.CrossRefGoogle Scholar
Evans, Jocelyn. 2012. The sound foundations of a socialist victory. Renewal 20 (2–3): 123–28.Google Scholar
Fisher, Stephen D., Ford, Robert, Jennings, Will, Pickup, Mark, and Wlezien, Christopher. 2011. From polls to votes to seats: Forecasting the 2010 British general election. Electoral Studies 30(2): 250–57.CrossRefGoogle Scholar
Jackman, Simon. 2005. Pooling the polls over an election campaign. Australian Journal of Political Science 40(4): 499517.CrossRefGoogle Scholar
Kropko, Jonathan. 2010. A comparison of three discrete choice estimators (unpublished dissertation chapter). University of North Carolina-Chapel Hill. http://www.unc.edu/∼kropko/paper1.pdf (accessed August 6, 2013).Google Scholar
Laakso, Markku, and Taagepera, Rein. 1979. “Effective” number of parties: A measure with application to West Europe. Comparative Political Studies 12(1): 327.CrossRefGoogle Scholar
Long, J. Scott. 1997. Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.Google Scholar
Luce, R. Duncan. 1959. Individual choice behavior. A theoretical analysis. New York: John Wiley & Sons.Google Scholar
Martin, Elizabeth A., Traugott, Michael W., and Kennedy, Courtney. 2005. A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly 69(3): 342–69.CrossRefGoogle Scholar
McFadden, Daniel. 1973. Conditional logit analysis of qualitative choice behaviour. In Frontiers of Econometrics, ed. Zarembka, Paul, 105–42. New York: Academic Press.Google Scholar
McLean, Iain. 1995. Independence of irrelevant alternatives before Arrow. Mathematical Social Sciences 30(2): 107–26.CrossRefGoogle Scholar
Mosteller, Frederick, Hyman, Herbert, McCarthy, Philip, Marks, Eli, and Truman, David. 1949. The pre-election polls of 1948: Report to the committee on analysis of pre-election polls and forecasts. New York: Social Science Research Council.Google Scholar
Pawlowsky-Glahn, Vera, and Buccianti, Antonella, eds. 2011. Compositional data analysis: Theory and applications. Chichester, UK: Wiley.CrossRefGoogle Scholar
Pearson, Karl. 1897. Mathematical contributions to the theory of evolution: On a form of spurious correlation which may arise when indices are used in the measurements of organs. Proceedings of the Royal Society of London 60: 489502.CrossRefGoogle Scholar
Ray, Paramesh. 1973. Independence of irrelevant alternatives. Econometrica 41(5): 987–91.CrossRefGoogle Scholar
Royston, Patrick, and Sauerbrei, Willi. 2008. Multivariable model-building: A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester, UK: Wiley.CrossRefGoogle Scholar
Schaffer, Lena-Maria, and Schneider, Gerald. 2005. Die Prognosegte von Wahlbörsen und Meinungsumfragen zur Bundestagswahl 2005. Politische Vierteljahresschrift 46(4): 674–81.CrossRefGoogle Scholar
Train, Kenneth. 2009. Discrete choice methods with simulation. 2nd ed. Cambridge, UK: Cambridge University Press.CrossRefGoogle Scholar
Whitten, Guy D., and Palmer, Harvey D. 1996. Heightening comparativists' concern for model choice: Voting behavior in Great Britain and the Netherlands. American Journal of Political Science 40(1): 231–60.CrossRefGoogle Scholar
Supplementary material: File

Arzheimer and Evans supplementary material

Supplementary Material

Download Arzheimer and Evans supplementary material(File)
File 23 KB
You have Access

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

A New Multinomial Accuracy Measure for Polling Bias
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

A New Multinomial Accuracy Measure for Polling Bias
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

A New Multinomial Accuracy Measure for Polling Bias
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *