Skip to main content

Scoring from Contests

  • Keith E. Schnakenberg (a1) and Elizabeth Maggie Penn (a2)

This article presents a new model for scoring alternatives from “contest” outcomes. The model is a generalization of the method of paired comparison to accommodate comparisons between arbitrarily sized sets of alternatives in which outcomes are any division of a fixed prize. Our approach is also applicable to contests between varying quantities of alternatives. We prove that under a reasonable condition on the comparability of alternatives, there exists a unique collection of scores that produces accurate estimates of the overall performance of each alternative and satisfies a well-known axiom regarding choice probabilities. We apply the method to several problems in which varying choice sets and continuous outcomes may create problems for standard scoring methods. These problems include measuring centrality in network data and the scoring of political candidates via a “feeling thermometer.” In the latter case, we also use the method to uncover and solve a potential difficulty with common methods of rescaling thermometer data to account for issues of interpersonal comparability.

Corresponding author
e-mail: (corresponding author)
Hide All

Authors' note: We are grateful for the helpful comments of David Darmofal, Mark Fey, Holger Kern, John Patty, and especially Stephane Wolton, along with seminar participants at Washington University, the Harris School of Public Policy, the University of South Carolina, the Stanford Graduate School of Business, the University of Maryland, and the University of Pittsburgh. This research was supported by NIH Grant (#1RC4LM010958-01), and we are particularly indebted to William Shannon, Elena Deych, Skye Buckner-Petty, and Berkley Shands at the Washington University School of Medicine for their support. Replication materials for this article are available from the Political Analysis Dataverse at

Hide All
Aitchison, John. 1986. The statistical analysis of compositional data. Monographs on Statistics and Applied Probability. London: Chapman & Hall.
Aitchison, J., and Egozcue, J. J. 2005. Compositional data analysis: Where are we and where should we be heading? Mathematical Geology 37(7): 829–50.
Alvarez, R. Michael, and Nagler, Jonathan. 1995. Economics, issues and the Perot candidacy: Voter choice in the 1992 presidential election. American Journal of Political Science 39(3): 714–44.
Alvarez, R. Michael, and Nagler, Jonathan. 1998. Economics, entitlements, and social issues: Voter choice in the 1996 presidential election. American Journal of Political Science 42(4): 1349–63.
Alvarez, R. Michael, and Nagler, Jonathan. 2000. A new approach for modelling strategic voting in multiparty elections. British Journal of Political Science 30: 5775.
Bechtel, Michael, and Scheve, Kenneth. 2012. Public support for global climate cooperation. Working paper.
Bradley, Ralph Allan, and Terry, Milton E. 1952. Rank analysis of incomplete block designs: I. The method of paired comparisons. Biometrika 39 (3/4): 324–45.
Clark, Derek J., and Riis, Christian. 1998. Contest success functions: An extension. Economic Theory 11(1): 201–4.
Davidson, Roger R. 1970. On extending the Bradley-Terry model to accommodate ties in paired comparison experiments. Journal of the American Statistical Association 65(329): 317–28.
Debreu, Gerard. 1960. Review of RD Luce, individual choice behavior: A theoretical analysis. American Economic Review 50(1): 186–88.
Efron, B. 1982. The jackknife, the bootstrap and other resampling plans. Vol. 38 of CBMS-NSF Regional Conference Series in Applied Mathematics. Philadelphia: SIAM.
Ford, L. R. Jr 1957. Solution of a ranking problem from binary comparisons. American Mathematical Monthly 64(8): 2833.
Fowler, J. H., Johnson, T. R., Spriggs, J. F. II, Jeon, S., and Paul, J. 2007. Network analysis and the law: Measuring the legal importance of Supreme Court precedents. Political Analysis 15(3): 324–46.
Glasgow, Garrett. 2001. Mixed logit models for multiparty elections. Political Analysis 9(2): 116–36.
Greene, William H. 2008. Econometric analysis. 6th ed. Upper Saddle River, NJ: Prentice Hall.
Jech, Thomas. 1983. The ranking of incomplete tournaments: A mathematician's guide to popular sports. American Mathematical Monthly 90(4): 246–64, 265–66.
Jost, John T., Federico, Christopher M., and Napier, Jaime L. N.d. Forthcoming.
Katz, Jonathan, and King, Gary. 1999. A statistical model for multiparty electoral data. American Political Science Review 93: 1532.
Laslier, Benoit, and Laslier, Jean-François. 2013. Reinforcement learning from comparisons: Three alternatives is enough, two is not. Working paper.
Laslier, Jean-François. 1997. Tournament solutions and majority voting. Studies in Economic Theory Series. Berlin: Springer.
Luce, R. Duncan. 1959. Individual choice behavior: A theoretical analysis. New York: Wiley.
Malhotra, Neil, and Margalit, Yotam. 2010. Short-term communication effects or longstanding dispositions? The publics response to the financial crisis of 2008. Journal of Politics 72: 852–67.
McFadden, Daniel. 1974. Conditional logit analysis of qualitative choice behavior. In Frontiers in econometrics, ed. Zarembka, P., 105–42. New York: Academic Press.
Page, S. E. 2007. The difference: How the power of diversity creates better groups, firms, schools, and societies. Princeton, NJ: Princeton University Press.
Patty, John W., Penn, Elizabeth M., and Schnakenberg, Keith E. 2013. Measuring the latent quality of precedent: Scoring vertices in a network. In Advances in political economy: Institutions, modelling and empirical analysis, eds. Kselman, Daniel, Caballero, Gonzalo, and Schofield, Norman. Berlin: Springer.
Pendergrass, Robert N., and Bradley, Ralph A. 1960. Ranking in triple comparisons. In Contributions to probability and statistics, eds. Olkin, O., Ghurye, S. G., Hoeffding, W., Madow, W. G., and Mann, H. B., 331–51. Stanford, CA: Stanford University Press.
Rubinstein, Ariel. 1980. Ranking the participants in a tournament. SIAM Journal on Applied Mathematics 38(1): 108–11.
Schnakenberg, Keith. 2013. Replication data for: Replication materials for “Scoring from Contests”. IQSS Dataverse Network. V1.
Stob, Michael. 1984. A Supplement to “A Mathematician's Guide to Popular Sports.” American Mathematical Monthly 91(5): 277–82.
Tversky, Amos. 1972. Elimination by aspects: A theory of choice. Psychological Review 79(4): 281.
Varadhan, Ravi, and Gilbert, Paul. 2009. BB: An R package for solving a large system of nonlinear equations and for optimizing a high-dimensional nonlinear objective function. Journal of Statistical Software 32(4): 126.
Winter, Nicholas, and Berinsky, Adam. 1999. What's your temperature? Thermometer ratings and political analysis. Working paper.
Yamamoto, Teppei. 2011. A multinomial response model for varying choice sets, with application to partially contested multiparty elections. Working paper.
Zermelo, Ernst. 1929. Die berechnung der turnier-ergebnisse als ein maximumproblem der wahrscheinlichkeitsrechnung. Mathematische Zeitschrift 29: 346–60.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed