In this chapter we study fundamental principles that underlie the design of rating systems for rating of players’ skills based on observed contest outcomes. Such rating systems have traditionally been used in the context of sports competitions. A canonical example is the rating of players’ skills in the game of chess, but rating systems have also been used in other sports; for example, for rating individual players’ strengths in the games of tennis and table tennis and for rating teams’ strengths in the games of football, basketball, and baseball. The use of rating systems has also played an important role in the context of online services. For example, rating of coders’ algorithmic and coding skills has been in use in popular competition-based crowdsourcing software-development platforms such as TopCoder. Another example is the rating of players’ skills in popular online multi-player gaming platforms such as Xbox Live. The rating systems are used for various purposes such as determining which players or teams of players qualify to participate in a tournament, seeding of tournaments, and creation of leaderboards. The use of rating systems may stimulate competition among players and general interest in a contest. The use of rating systems for matchmaking that biases competitions to be among similarly skilled players may stimulate the participation and contribution of players and increase the interest of spectators. Rating systems can also be used for prediction of contest outcomes, which is of particular interest in the context of betting services. The ratings of players’ skills can also be used as performance indicators for hiring and assigning work to skillful workers.

One of the main challenges for the design of a rating system is to accurately estimate players’ skills based on sparse input data that contains information about contest outcomes. In many situations in practice, only a small portion of all distinct pairs of players face each other in a contest. In Figure 9.1, the input data sparsity is illustrated for the case of TopCoder competitions. Although the designs of some popular existing rating systems differ from each other in their details, we shall see that they all share a few fundamental design principles.