In this chapter we consider the problem of ranking alternatives using as input a sequence of rankings of subsets of the alternatives. This problem arises in many situations including that of finding a global ranking of teams in a competition based on observed outcomes in contests involving subsets of teams, e.g., pairs of teams in many popular sports competitions. We may also think of rank aggregation where the goal is to find a consensus ranking for a set of input rankings from different sources, e.g., computing an aggregate ranking given as input a list of top-ranked search results by different search engines. The rank aggregation problem also accommodates the problem of identifying a ground-truth ranking based on noisy input judgments by a panel of experts. For example, such a problem arises in classification of objects that is now commonly performed by less-than-expert workers in paid-labor crowdsourcing online platforms.

We shall first consider the problem of rank aggregation where given a set of input rankings of a set of alternatives, the goal is to find an aggregate ranking of the alternatives that minimizes a given loss function. This optimization-based approach can be seen as finding a global ranking that minimizes the extent of disagreement with the input set of rankings. Specifically, we shall consider the problem of finding an aggregate ranking of alternatives that minimizes the sum of distances to individual input rankings of the alternatives. We shall see that some well-known distances are within a constant factor of each other, e.g., the well-known Kendall's τ and Spearman's Footrule distances. We shall consider the well-known Kemeny rank aggregation where the goal is to find an aggregate ranking of alternatives that minimizes the total number of disagreements of ranking of pairs of alternatives with respect to the input rankings of alternatives. This problem is known to be NP hard. We shall discuss some simple algorithms that guarantee a constant-factor approximation. We shall then consider the problem known as the minimum feedback arc set in tournaments, where the input is a tournament graph defined as a directed graph where there is exactly one directed edge between each pair of vertices, and the goal is to find a linear ordering of vertices that minimizes the number of backward edges.

Contests are systems in which participants, whom I refer to as players, invest efforts in order to win one or more prizes. A distinctive feature of a contest is that each player invests effort but may not be awarded a prize. This makes the area of contest design a subset of auction theory where the aim is to design an auction that achieves a desired goal without necessarily restricting the design to one in which everybody pays. The area is also different from that of mechanism design where the goal is to design a mechanism that optimizes a given objective subject to the constraint that the mechanism is truthful, i.e., players truthfully report their private information. In general, no such constraint is imposed for a contest design problem, and in fact, many contest designs are non-truthful. Another important feature of a contest is that contestants are rewarded with respect to their relative performance, e.g., allocating an award to the best performing player or based on the rank of individual production outputs. This is different from traditional compensation schemes based on some estimate of absolute performance output. The theory of contest design has been developed over the last hundred years or so; in the early days it was predominantly studied in the areas of statistics, political economy and public choice, and the research was motivated by the need to understand and study various competitions, such as sport competitions, rent-seeking, lobbying, conflicts, arm races, R&D competitions, and, more recently, online marketplaces and resource allocation mechanisms. The development of the theory and experimental evaluation have been especially advanced over recent years in the areas of theoretical computer science and management sciences, fueled by the needs of various applications in the context of Internet online services. Here we find a wide variety of contests offering either monetary rewards or reputation. For example, soliciting solutions to tasks through open calls to large communities, so-called crowdsourcing, has emerged as a method of choice for solving a wide range of tasks, including web design, software development, algorithmic and data mining challenges, and various other tasks that require human intelligence.

This book was written to provide an exposition of some of the central concepts in contest design. It should be accessible to any senior-level undergraduate and graduate student equipped with a basic knowledge of mathematics and probability theory.

In this chapter we study fundamental principles that underlie the design of rating systems for rating of players’ skills based on observed contest outcomes. Such rating systems have traditionally been used in the context of sports competitions. A canonical example is the rating of players’ skills in the game of chess, but rating systems have also been used in other sports; for example, for rating individual players’ strengths in the games of tennis and table tennis and for rating teams’ strengths in the games of football, basketball, and baseball. The use of rating systems has also played an important role in the context of online services. For example, rating of coders’ algorithmic and coding skills has been in use in popular competition-based crowdsourcing software-development platforms such as TopCoder. Another example is the rating of players’ skills in popular online multi-player gaming platforms such as Xbox Live. The rating systems are used for various purposes such as determining which players or teams of players qualify to participate in a tournament, seeding of tournaments, and creation of leaderboards. The use of rating systems may stimulate competition among players and general interest in a contest. The use of rating systems for matchmaking that biases competitions to be among similarly skilled players may stimulate the participation and contribution of players and increase the interest of spectators. Rating systems can also be used for prediction of contest outcomes, which is of particular interest in the context of betting services. The ratings of players’ skills can also be used as performance indicators for hiring and assigning work to skillful workers.

One of the main challenges for the design of a rating system is to accurately estimate players’ skills based on sparse input data that contains information about contest outcomes. In many situations in practice, only a small portion of all distinct pairs of players face each other in a contest. In Figure 9.1, the input data sparsity is illustrated for the case of TopCoder competitions. Although the designs of some popular existing rating systems differ from each other in their details, we shall see that they all share a few fundamental design principles.

In this chapter we consider a class of contests where a prize is allocated to players according to an allocation mechanism that is a smooth function of invested efforts, with one exception: the corner case in which none of the players invest efforts. The smooth allocation mechanism differs from the rank-order allocation of prizes considered in previous two chapters where the allocation is according to a discontinuous function of effort investments. Smooth allocation of prizes may occur not only because of factors such as stochastic production, where individual production outputs depend on the invested efforts, but also because of exogenous random effects, or imperfect discrimination, where the ranking of players is according to some noisy observations of individual production outputs. As a result of such random effects, the probability of winning a prize may well end up being a smooth function of invested efforts. A smooth allocation of prizes may have desirable properties and for this reason may be imposed by the contest design. For example, one of the key features of the smooth allocation of prizes is that the best performing player may not be allocated the prize with some probability, which may intensify the competition and, as a result, elicit larger effort investments. Our overarching goal in this chapter is to characterize strategic behavior in contests under the smooth allocation of prizes and evaluate properties of interest for particular forms of contest success functions with respect to induced efforts and social efficiency. We present a set of axioms and some probabilistic justifications that serve as a motivation for particular forms of smooth prize allocation. This puts in the spotlight a contest success function that admits a general-logit form, which allocates the prize in proportion to increasing functions of individual effort investments. The general-logit function accommodates several interesting and well-studied special cases such as proportional allocation, where the prize is allocated in proportion to individual efforts, or more generally, the ratio form where the prize is allocated in proportion to a power function of the invested effort. At the end of the chapter, we discuss contest success functions of difference form where the contest success function is a function of the difference of individual efforts.

In this chapter we consider contests among two or more players that proceed through multiple rounds and end either in a fixed number of rounds or as soon as a termination criteria is fulfilled; for example, as soon one of the players achieves a given point difference or the contest owner acquires a given quality of the production output. The players are awarded according to a prize allocation mechanism that is based on their effort investments. We shall consider prize allocation mechanisms that either award a prize at the end of the contest or some amount of that prize at the end of each round of the contest. We shall consider contests under different assumptions on the structure of the effort investments over rounds, including those in which each player invests effort in one round according to a given order of play or strategic decisions by the players, and contests in which each player can invest effort in each round having observed the efforts invested in earlier rounds. Sequential moves may either be imposed by the contest design, may endogenously arise as a strategic equilibrium, or may occur because players “psyche themselves up” in a contest. We shall consider such models of contests formulated as extensive form games and study the properties of subgame perfect Nash equilibria. These games differ with respect to the information available to the players about the abilities of other players; we shall consider both games with complete and incomplete information.

There are many contests in practice that are based on sequential effort investments. Traditional examples include R&D patent races in which individual firms compete in filing a larger number of patents than other firms, advertising campaigns in which firms try to maintain or increase their market shares at the expense of other firms through promotional competitions, political races in which candidates confront each other in a sequence of speeches, court trials in which it is customary for the plaintiff to present evidence prior to the defense lawyers and both sides make their final speeches in the same sequential order, and sport competitions such as team sports, gymnastic tournaments, and tennis matches. There are also numerous examples of sequential contests in the context of online services.

In the Appendix we define various mathematical concepts and state some of the theorems that are invoked at various places in the book. Most of the theorems are accompanied by proofs, with a few exceptions in which case we refer to the relevant literature.

Section 11.1 introduces the basic concepts of relations and orderings, sets, convex functions and optimization, the envelope theorem, some functional equations, and fixed-point theorems. The concept of a partial order is used in particular in Chapter 9 and Chapter 10. The convex optimization and the envelope theorem are used in Chapter 4. The Cauchy functional equations appear in the proofs of Theorem 4.2 and Theorem 10.15, and a functional equation related to trigonometric equations appears in the proof of Theorem 9.13. The fixed-point theorems are invoked in Section 11.3.

Section 11.2 covers some elements of probability and statistics including order statistics, distributions on a simplex, and Gaussian distributions. The order statistics are used throughout this book, but perhaps most prominently in Chapter 3. The distributions on a simplex are used in Chapter 5 to establish the existence of mixed-strategy Nash equilibria for the Colonel Blotto games. Some properties of Gaussian distributions are used in the context of approximate Bayesian inference for rating systems in Chapter 9.

In Section 11.3 we cover some special normal form games including concave, potential, and smooth games. The concept of a concave game that we discuss in Section 11.3.1 appears at several places in Chapter 4 and Chapter 6. In particular, we state and prove Rosen's theorem (Theorem 11.51) on the existence of a pure-strategy Nash equilibrium for concave games. The concept of a potential game, the existence of a pure-strategy Nash equilibrium for potential games, and conditions for a normal form game to be a potential game are discussed in Section 11.3.2. Some of these results are used in Chapter 5 to establish the existence of a pure-strategy Nash equilibrium of a normal form game that models a system of simultaneous contests, and in Chapter 6 for the utility sharing games with convex utility of production functions. The concept of a smooth game, different variants of smooth games, and the price of anarchy bounds that hold for smooth games are discussed in Section 11.3.3.