18 results
10 - Ranking Methods
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 563-616
-
- Chapter
- Export citation
-
Summary
In this chapter we consider the problem of ranking alternatives using as input a sequence of rankings of subsets of the alternatives. This problem arises in many situations including that of finding a global ranking of teams in a competition based on observed outcomes in contests involving subsets of teams, e.g., pairs of teams in many popular sports competitions. We may also think of rank aggregation where the goal is to find a consensus ranking for a set of input rankings from different sources, e.g., computing an aggregate ranking given as input a list of top-ranked search results by different search engines. The rank aggregation problem also accommodates the problem of identifying a ground-truth ranking based on noisy input judgments by a panel of experts. For example, such a problem arises in classification of objects that is now commonly performed by less-than-expert workers in paid-labor crowdsourcing online platforms.
We shall first consider the problem of rank aggregation where given a set of input rankings of a set of alternatives, the goal is to find an aggregate ranking of the alternatives that minimizes a given loss function. This optimization-based approach can be seen as finding a global ranking that minimizes the extent of disagreement with the input set of rankings. Specifically, we shall consider the problem of finding an aggregate ranking of alternatives that minimizes the sum of distances to individual input rankings of the alternatives. We shall see that some well-known distances are within a constant factor of each other, e.g., the well-known Kendall's τ and Spearman's Footrule distances. We shall consider the well-known Kemeny rank aggregation where the goal is to find an aggregate ranking of alternatives that minimizes the total number of disagreements of ranking of pairs of alternatives with respect to the input rankings of alternatives. This problem is known to be NP hard. We shall discuss some simple algorithms that guarantee a constant-factor approximation. We shall then consider the problem known as the minimum feedback arc set in tournaments, where the input is a tournament graph defined as a directed graph where there is exactly one directed edge between each pair of vertices, and the goal is to find a linear ordering of vertices that minimizes the number of backward edges.
Preface
-
- By Milan Vojnović, University of Cambridge
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp xiii-xviii
-
- Chapter
- Export citation
-
Summary
Contests are systems in which participants, whom I refer to as players, invest efforts in order to win one or more prizes. A distinctive feature of a contest is that each player invests effort but may not be awarded a prize. This makes the area of contest design a subset of auction theory where the aim is to design an auction that achieves a desired goal without necessarily restricting the design to one in which everybody pays. The area is also different from that of mechanism design where the goal is to design a mechanism that optimizes a given objective subject to the constraint that the mechanism is truthful, i.e., players truthfully report their private information. In general, no such constraint is imposed for a contest design problem, and in fact, many contest designs are non-truthful. Another important feature of a contest is that contestants are rewarded with respect to their relative performance, e.g., allocating an award to the best performing player or based on the rank of individual production outputs. This is different from traditional compensation schemes based on some estimate of absolute performance output. The theory of contest design has been developed over the last hundred years or so; in the early days it was predominantly studied in the areas of statistics, political economy and public choice, and the research was motivated by the need to understand and study various competitions, such as sport competitions, rent-seeking, lobbying, conflicts, arm races, R&D competitions, and, more recently, online marketplaces and resource allocation mechanisms. The development of the theory and experimental evaluation have been especially advanced over recent years in the areas of theoretical computer science and management sciences, fueled by the needs of various applications in the context of Internet online services. Here we find a wide variety of contests offering either monetary rewards or reputation. For example, soliciting solutions to tasks through open calls to large communities, so-called crowdsourcing, has emerged as a method of choice for solving a wide range of tasks, including web design, software development, algorithmic and data mining challenges, and various other tasks that require human intelligence.
This book was written to provide an exposition of some of the central concepts in contest design. It should be accessible to any senior-level undergraduate and graduate student equipped with a basic knowledge of mathematics and probability theory.
9 - Rating Systems
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 501-562
-
- Chapter
- Export citation
-
Summary
In this chapter we study fundamental principles that underlie the design of rating systems for rating of players’ skills based on observed contest outcomes. Such rating systems have traditionally been used in the context of sports competitions. A canonical example is the rating of players’ skills in the game of chess, but rating systems have also been used in other sports; for example, for rating individual players’ strengths in the games of tennis and table tennis and for rating teams’ strengths in the games of football, basketball, and baseball. The use of rating systems has also played an important role in the context of online services. For example, rating of coders’ algorithmic and coding skills has been in use in popular competition-based crowdsourcing software-development platforms such as TopCoder. Another example is the rating of players’ skills in popular online multi-player gaming platforms such as Xbox Live. The rating systems are used for various purposes such as determining which players or teams of players qualify to participate in a tournament, seeding of tournaments, and creation of leaderboards. The use of rating systems may stimulate competition among players and general interest in a contest. The use of rating systems for matchmaking that biases competitions to be among similarly skilled players may stimulate the participation and contribution of players and increase the interest of spectators. Rating systems can also be used for prediction of contest outcomes, which is of particular interest in the context of betting services. The ratings of players’ skills can also be used as performance indicators for hiring and assigning work to skillful workers.
One of the main challenges for the design of a rating system is to accurately estimate players’ skills based on sparse input data that contains information about contest outcomes. In many situations in practice, only a small portion of all distinct pairs of players face each other in a contest. In Figure 9.1, the input data sparsity is illustrated for the case of TopCoder competitions. Although the designs of some popular existing rating systems differ from each other in their details, we shall see that they all share a few fundamental design principles.
4 - Smooth Allocation of Prizes
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 157-222
-
- Chapter
- Export citation
-
Summary
In this chapter we consider a class of contests where a prize is allocated to players according to an allocation mechanism that is a smooth function of invested efforts, with one exception: the corner case in which none of the players invest efforts. The smooth allocation mechanism differs from the rank-order allocation of prizes considered in previous two chapters where the allocation is according to a discontinuous function of effort investments. Smooth allocation of prizes may occur not only because of factors such as stochastic production, where individual production outputs depend on the invested efforts, but also because of exogenous random effects, or imperfect discrimination, where the ranking of players is according to some noisy observations of individual production outputs. As a result of such random effects, the probability of winning a prize may well end up being a smooth function of invested efforts. A smooth allocation of prizes may have desirable properties and for this reason may be imposed by the contest design. For example, one of the key features of the smooth allocation of prizes is that the best performing player may not be allocated the prize with some probability, which may intensify the competition and, as a result, elicit larger effort investments. Our overarching goal in this chapter is to characterize strategic behavior in contests under the smooth allocation of prizes and evaluate properties of interest for particular forms of contest success functions with respect to induced efforts and social efficiency. We present a set of axioms and some probabilistic justifications that serve as a motivation for particular forms of smooth prize allocation. This puts in the spotlight a contest success function that admits a general-logit form, which allocates the prize in proportion to increasing functions of individual effort investments. The general-logit function accommodates several interesting and well-studied special cases such as proportional allocation, where the prize is allocated in proportion to individual efforts, or more generally, the ratio form where the prize is allocated in proportion to a power function of the invested effort. At the end of the chapter, we discuss contest success functions of difference form where the contest success function is a function of the difference of individual efforts.
7 - Sequential Contests
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 359-442
-
- Chapter
- Export citation
-
Summary
In this chapter we consider contests among two or more players that proceed through multiple rounds and end either in a fixed number of rounds or as soon as a termination criteria is fulfilled; for example, as soon one of the players achieves a given point difference or the contest owner acquires a given quality of the production output. The players are awarded according to a prize allocation mechanism that is based on their effort investments. We shall consider prize allocation mechanisms that either award a prize at the end of the contest or some amount of that prize at the end of each round of the contest. We shall consider contests under different assumptions on the structure of the effort investments over rounds, including those in which each player invests effort in one round according to a given order of play or strategic decisions by the players, and contests in which each player can invest effort in each round having observed the efforts invested in earlier rounds. Sequential moves may either be imposed by the contest design, may endogenously arise as a strategic equilibrium, or may occur because players “psyche themselves up” in a contest. We shall consider such models of contests formulated as extensive form games and study the properties of subgame perfect Nash equilibria. These games differ with respect to the information available to the players about the abilities of other players; we shall consider both games with complete and incomplete information.
There are many contests in practice that are based on sequential effort investments. Traditional examples include R&D patent races in which individual firms compete in filing a larger number of patents than other firms, advertising campaigns in which firms try to maintain or increase their market shares at the expense of other firms through promotional competitions, political races in which candidates confront each other in a sequence of speeches, court trials in which it is customary for the plaintiff to present evidence prior to the defense lawyers and both sides make their final speeches in the same sequential order, and sport competitions such as team sports, gymnastic tournaments, and tennis matches. There are also numerous examples of sequential contests in the context of online services.
Index
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 713-717
-
- Chapter
- Export citation
2 - Standard All-Pay Contest
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 41-94
-
- Chapter
- Export citation
11 - Appendix
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 617-680
-
- Chapter
- Export citation
-
Summary
In the Appendix we define various mathematical concepts and state some of the theorems that are invoked at various places in the book. Most of the theorems are accompanied by proofs, with a few exceptions in which case we refer to the relevant literature.
Section 11.1 introduces the basic concepts of relations and orderings, sets, convex functions and optimization, the envelope theorem, some functional equations, and fixed-point theorems. The concept of a partial order is used in particular in Chapter 9 and Chapter 10. The convex optimization and the envelope theorem are used in Chapter 4. The Cauchy functional equations appear in the proofs of Theorem 4.2 and Theorem 10.15, and a functional equation related to trigonometric equations appears in the proof of Theorem 9.13. The fixed-point theorems are invoked in Section 11.3.
Section 11.2 covers some elements of probability and statistics including order statistics, distributions on a simplex, and Gaussian distributions. The order statistics are used throughout this book, but perhaps most prominently in Chapter 3. The distributions on a simplex are used in Chapter 5 to establish the existence of mixed-strategy Nash equilibria for the Colonel Blotto games. Some properties of Gaussian distributions are used in the context of approximate Bayesian inference for rating systems in Chapter 9.
In Section 11.3 we cover some special normal form games including concave, potential, and smooth games. The concept of a concave game that we discuss in Section 11.3.1 appears at several places in Chapter 4 and Chapter 6. In particular, we state and prove Rosen's theorem (Theorem 11.51) on the existence of a pure-strategy Nash equilibrium for concave games. The concept of a potential game, the existence of a pure-strategy Nash equilibrium for potential games, and conditions for a normal form game to be a potential game are discussed in Section 11.3.2. Some of these results are used in Chapter 5 to establish the existence of a pure-strategy Nash equilibrium of a normal form game that models a system of simultaneous contests, and in Chapter 6 for the utility sharing games with convex utility of production functions. The concept of a smooth game, different variants of smooth games, and the price of anarchy bounds that hold for smooth games are discussed in Section 11.3.3.
Frontmatter
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp i-iv
-
- Chapter
- Export citation
6 - Utility Sharing and Welfare
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 301-358
-
- Chapter
- Export citation
-
Summary
In this chapter we consider systems that consist of individuals who generate some type of production outputs and are awarded shares of the resulting utility of production according to a utility sharing mechanism. Think of systems in which individuals make contributions to one or several projects that for each project amount to some value of utility of production. Contributors to each project are awarded according to a utility sharing mechanism that determines how the utility of production of each project is shared among those who contributed to it. Such systems arise in many real-life situations. For example, in the context of online services, users may contribute to activities such as online content creation or software development and may be awarded credits for their contributions in various kinds such as monetary payments, attention, and reputation. Another example is that of scientific collaborations where scientists work jointly on research projects and receive credits for the impact of their research results. In this case, the value of credit received may depend on some measure of the impact of the work on society and how much each individual contributed toward the success of the project. It is natural in such scenarios to consider strategic individuals who aim to selfishly maximize their individual payoffs, both in non-cooperative and cooperative strategic settings. A central question of interest here is that of the efficiency with respect to the social utility of production in environments where individual objectives are not necessarily aligned with the social objective. We are especially interested in evaluating the efficiency of utility sharing mechanisms that are simple and commonly deployed in practice; for example, sharing of the produced utility of production according to equal shares, sharing that is proportional to individual contributions, or sharing according to fixed shares depending on the ranks of individual contributions.
The class of contests considered in the present chapter differs from those in previous chapters in that prizes are shares of the utility of production, which is a function of effort investments, rather than shares of a fixed prize purse. Another important distinction is that in the present context we also consider collaborative environments where it is natural for players to form coalitions. This asks us to go beyond the solution concepts from non-cooperative game theory to consider those that account for strategic cooperation among players.
5 - Simultaneous Contests
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 223-300
-
- Chapter
- Export citation
-
Summary
In this chapter we consider normal form games that consist of a set of one or more contests each offering a prize of a certain value and a set of two or more players who simultaneously invest efforts across the set of available contests. We consider strategic players who aim at selfishly maximizing their individual payoffs. The payoff of each player is assumed to be quasi-linear in the total value of prizes won across different contests and the incurred cost of production. The values of prizes are allowed to assume arbitrary positive values, except when we consider the case of contests with identical values of prizes. The existence of multiple available contests provides players with alternative options for effort investment. From the perspective of any given contest, this provides players with outside options that may significantly affect the effort investments directed into the given contest.
The type of normal form games that we study in this chapter serves as a natural model of the competition-based crowdsourcing services that solicit contributions to projects from online communities through contests. In such crowdsourcing services there are typically several open contests at any given time, sometimes as many as in the order of hundreds. Each contest awards one or more prizes to the winning solutions selected from the set of solutions submitted to this contest. This selection is made according to a set of contest rules, which are public information, or at a discretion of a contest owner who identifies one or more best-quality submissions according to a criteria. Some of the competition-based crowdsourcing services allow workers to choose to participate in any of the open contests. Such a design rests on a premise that each individual worker may be in a best position to appreciate his or her ability to perform well in any given contest based on specification of the underlying project requirements and some prior sense about the competition. However, such an assignment of projects to workers may result in inefficiencies due to non-cooperative strategic behavior. Some projects may attract many while others may only attract a few workers.
References
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 681-708
-
- Chapter
- Export citation
1 - Introduction and Preview
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 1-40
-
- Chapter
- Export citation
-
Summary
This book synthesizes what one may refer to as contest theory, understood in a broad sense to encompass scientific methods and theories for the better understanding and informed design of contests. Its goal is to provide a contest designer with a set of theoretical results and methods that can be used for the design of contests. An ambitious aspiration is to provide a toolkit for a contest designer of a similar kind to what control theory offers to engineers for the design of control systems. This is, undoubtedly, a challenging task, primarily because of the complexity of user behavior and incentives that play a key role in most of the systems of concern. This book covers a wide range of models developed in different areas of science including computer science, economics, and statistics.
Generally speaking, we refer to contests as situations in which individuals invest efforts toward winning one or more prizes, those investments of efforts are costly and irreversible, and prizes are allocated based on the relative values of efforts. A prize is understood in a broad sense to refer to a notion of value that is general enough to include not only monetary prizes but also social reputation and gratitude. How to allocate a prize purse to competitors in a contest was studied as early as 1902 by Galton, who reasoned about the question, “what is the most suitable proportion between the values of first and second prizes?” assuming a statistical model according to which individual production outputs are independent and identically distributed random variables with a given distribution. An economist's approach is to assume that contestants are rational players who strategically invest efforts with a selfish goal of maximizing their individual payoffs, which combine in some way the value of winning a prize and the cost of production. The study of a contest as a game using the framework of game theory allows us to reason about properties that arise in a strategic equilibrium. The design of a contest needs to ensure that proper incentives are put in place to achieve a desired objective. Commonly studied objectives include the total effort invested by contestants, the maximum individual effort over all contestants, and the social welfare defined as the value of the prizes to those who win them.
8 - Tournaments
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 443-500
-
- Chapter
- Export citation
Contents
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp v-xii
-
- Chapter
- Export citation
3 - Rank-Order Allocation of Prizes
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 95-156
-
- Chapter
- Export citation
-
Summary
In this chapter we consider contests that award one or more placement prizes based on the rank of individual performance. Such contests are rather common. The number of placement prizes and how the prize purse is split over a given number of placement prizes vary widely from one contest to another. Perhaps the most common contest design is to award only the first place prize, thus rewarding only the best performing contestant. Another common practice is to award two prizes: the first place prize and the runner-up prize to the best performing and the second best performing contestant, respectively. Also common are designs with three placement prizes: the first place prize to the best performing player, the second place prize to the second best performing player, and the third place prize to the third best performing player. A case that also often arises in practice is a contest that offers one or more prizes of identical values. For example, such prizes can be positions in the next stage of a tournament, admissions to a school program, or research papers accepted for inclusion in a conference program. The rank-based allocation of prizes that is considered in this chapter can be seen as a generalization of that studied in Chapter 2, where the focus was on contests that award only the first place prize. One might expect that devoting some amount of a prize purse to the runner-up and perhaps also to other placement prizes would incentivize lower ability contestants to try harder and as a result yield overall higher performance.
Our goal in this chapter is to characterize strategic behavior in contests that award one or more placement prizes. We shall pay particular attention to identifying conditions under which it is optimal for a contest owner to offer only the first place prize and when it is better to split a prize purse across several placement prizes. There are two important factors here: the informational assumptions about abilities of players and the nature of production costs. We shall see that if players are ex-ante identical with respect to their abilities and the production of each player exhibits a weakly diminishing marginal cost of production, it is optimal for the contest owner to allocate the entire prize purse to the first place prize with respect to both the expected total effort and the expected maximum individual effort in an equilibrium.
Index to Notations
- Milan Vojnović, London School of Economics and Political Science
-
- Book:
- Contest Theory
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016, pp 709-712
-
- Chapter
- Export citation
Contest Theory
- Incentive Mechanisms and Ranking Methods
- Milan Vojnović
-
- Published online:
- 05 January 2016
- Print publication:
- 04 February 2016
-
Contests are prevalent in many areas, including sports, rent seeking, patent races, innovation inducement, labor markets, scientific projects, crowdsourcing and other online services, and allocation of computer system resources. This book provides unified, comprehensive coverage of contest theory as developed in economics, computer science, and statistics, with a focus on online services applications, allowing professionals, researchers and students to learn about the underlying theoretical principles and to test them in practice. The book sets contest design in a game-theoretic framework that can be used to model a wide-range of problems and efficiency measures such as total and individual output and social welfare, and offers insight into how the structure of prizes relates to desired contest design objectives. Methods for rating the skills and ranking of players are presented, as are proportional allocation and similar allocation mechanisms, simultaneous contests, sharing utility of productive activities, sequential contests, and tournaments.