The differential games framework extends static non-cooperative continuous-kernel game theory into dynamic environments by adopting the tools, methods, and models of optimal-control theory. Optimal-control theory [58] has been developed to obtain the optimal solutions to planning problems that involve dynamic systems, where the state evolves over time under the influence of a control input (which is the instrument variable that is designed). Differential games can be viewed as extensions of optimal-control problems in two directions: (i) the evolution of the state is controlled not by one input but by multiple inputs, with each under the control of a different player, and (ii) the objective function is no longer a single one, with each player now having a possibly different objective function (payoff or cost), defined over time intervals of interest and relevance to the problem. The relative position of differential games in this landscape is captured in Table 5.1 [58]. Two main approaches that yield solutions to optimal-control problems are dynamic programming (introduced by Bellman) and the maximum principle (introduced by Pontryagin) [58]. The former leads to an optimal control that is a function of the state and time (closed-loop feedback control), whereas the latter leads to one that is a function only of the time and the initial state (open-loop control). These two approaches have also been adopted in differential games, where the common solution concepts of a differential game are again the Nash equilibrium and the Stackelberg equilibrium, for non-hierarchical and hierarchical structures, respectively.

Non-cooperative game theory is one of the most important branches of game theory, focusing on the study and analysis of competitive decision-making involving several players. It provides an analytical framework suited for characterizing the interactions and decision-making process involving several players with partially or totally conflicting interests over the outcome of a decision process which is affected by their actions. Examples of non-cooperative games are ubiquitous. In economics, firms operating in the same market compete over pricing strategies, market control, trading of goods, and the like. In wireless and communication networks, wireless nodes are involved in numerous non-cooperative scenarios such as allocation of resources, choices of frequencies or transmit power, packet forwarding, and interference management. Beyond economics and networking, non-cooperative game theory has made its impact over a broad range of disciplines such as biology, political science, sociology, and military tactics. In this chapter, we introduce non-cooperative game theory along with different types of games, while presenting underlying fundamental notions and key solution concepts.

Non-cooperative games: preliminaries

In this section, we introduce some preliminary concepts and terminology that pertain to non-cooperative game theory.

Introduction

A non-cooperative game involves a number of players having totally or partially conflicting interests in the outcome of a decision process. For example, consider a number of wireless nodes attempting to control their transmit power, given the interference generated by other nodes.

In the past two decades, cellular communication has witnessed a significant growth. Today, millions of mobile users utilize cellular phones worldwide. In essence, a cellular network is designed to provide a large number of users with access to wireless services over a large area. The basic architecture of a cellular network relies on dividing a large area (e.g., a city) into smaller areas, commonly referred to as cells. Each cell typically represents the coverage area of a single base station that is often located at the center of the cell. By dividing the network into cells, one can ensure a reliable coverage and access to wireless services. Despite the emergence of ad hoc networks with no infrastructure, the cellular architecture remains prevalent in the majority of existing or soon-to-be deployed wireless networks, because of its proven success. In fact, cellular communication has been the pillar architecture in key wireless systems, from traditional 2G systems such as GSM to 3G systems such as UMTS, and the emerging 4G and 5G systems. Beyond traditional macrocell-based networks (e.g., 3G and 4G networks), the use of small cells, covered by low-cost, low-power stations known as femtocell access points that can be overlaid with existing architectures, has recently become of central importance in the design of next-generation wireless networks. Thus, cellular technology is expected to remain as one of the most important paradigms in future wireless communication systems. In Chapter 2, we provided a comprehensive introduction to cellular communication, its key challenges, as well as its past and projected future evolution.

Cognitive radio [201] is a new paradigm for designing wireless communication systems which aims at enhancing the utilization of the radio-frequency spectrum. The motivation for cognitive radio arises from the scarcity of the available frequency spectrum. Emerging wireless applications will increase spectrum demand from mobile users. However, most of the available radio spectrum has been allocated to existing wireless systems, and only small portions of the radio spectrum can be licensed to new wireless applications. Nonetheless, a study in [143] by the spectrum policy task force (SPTF) of the Federal Communications Commission (FCC) shows that there are also many frequency bands which are only partly occupied or largely unoccupied. For example, spectrum bands allocated to cellular networks in the USA [325] reach their highest utilization during working hours, while they remain unoccupied from midnight until early morning.

The major factor that leads to inefficient use of radio spectrum is the spectrum licensing scheme itself. In traditional spectrum allocation, based on a command-and-control model, where radio spectrum allocated to licensed users (i.e., primary users) is not used, it cannot be utilized by unlicensed users (i.e., secondary users) or applications [87]. Because of this static and inflexible allocation, legacy wireless systems operate only on a dedicated spectrum band, and cannot adapt the transmission band according to the changing environment.

Brief introduction to the history of game theory

Game theory can be viewed as a branch of applied mathematics as well as of applied sciences. It has been used in the social sciences, most notably in economics, but has also penetrated into a variety of other disciplines such as political science, biology, computer science, philosophy, and, recently, wireless and communication networks. Even though game theory is a relatively young discipline, the ideas underlying it have appeared in various forms throughout history and in numerous sources, including the Bible, the Talmud, the works of Descartes and Sun Tzu, and the writings of Charles Darwin, and in the 1802 work Considérations sur la Théorie Mathématique du Jeu of André-Marie Ampère, who was influenced by the 1777 Essai d'Arithmétique Morale of Georges Louis Buffon. Nonetheless, the main basis of modern-day game theory can be considered an outgrowth of three seminal works:

• Augustin Cournot's Mathematical Principles of the Theory of Wealth in 1838, which gives an intuitive explanation of what would, over a century later, be formalized as the celebrated Nash equilibrium solution to non-cooperative games. Furthermore, Cournot's work provides an evolutionary or dynamic notion of the idea of a “best response,” i.e., situations in which a player chooses the best action given the actions of other players, this being so for all players.

• […]

A wireless network refers to a telecommunications network whose interconnections between nodes are implemented without the use of wires. Wireless networks have experienced unprecedented growth over the last few decades, and are expected to continue to evolve in the future. Seamless mobility and coverage ensure that various types of wireless connections can be made anytime, anywhere. In this chapter, we introduce some basic types of wireless networks and provide the reader with some necessary background on state-of-art development.

Wireless networks use electromagnetic waves, such as radio waves, for carrying information. Therefore, their performance is greatly affected by the randomly fluctuating wireless channels. To develop an understanding of channels, in Section 2.1 we will study the radio frequency band first, then the existing wireless channel models used for different network scenarios, and finally the interference channel.

There exist several wireless standards. We describe them according to the order of coverage area, starting with cellular wireless networks. In Section 2.2.1 we provide an overview of the key elements and technologies of the third-generation wireless cellular network standards. In particular, we discuss WCDMA, CDMA2000, TD/S CDMA, and 4G and beyond. WiMax, based on the IEEE 802.16 standard for wireless metropolitan area networks, is discussed in Section 2.2.2. A wireless local area network (WLAN) is a network in which a mobile user can connect to a local area network through a wireless connection.

The game models discussed in the preceding chapters were all built on the governing assumption that the players all have complete information on the elements of the game, particularly on the action spaces of all players and the players' payoff (or cost) functions, and that this is all common information to all players. However, in many situations, especially in a competitive environment, the a priori information available to a player may not be publicly available to other players. In particular, a player may not have complete information on his opponents' possible actions, strategies, and payoffs. For example, in the latter case, a player may not know the resulting payoff value for another player when all players have picked specific actions (or strategies). One way of addressing situations that arise as a result of such incompleteness of information is to formulate them as Bayesian games—the topic of this chapter. We first introduce this class of games in general terms, and then discuss applications in wireless communications and networking.

Overview of Bayesian games

Simple example

Let us consider the example of a game between two car companies responding to the possibility of a Clean Air Act, such as the one in 1990. If the Act is passed, then both car companies will be faced with the task of redesigning their cars, which will be costly. In order to prevent this from happening, they decide to start a lobbying campaign against the Act.

Evolutionary-game theory has been developed as a mathematical framework to study the interaction among rational biological agents in a population [152]. In evolutionary-game theory, the agent adapts (i.e., evolves) the chosen strategy based on its fitness (i.e., payoff). In this way, both static and dynamic behavior (e.g., equilibrium) of the game can be analyzed.

Evolutionary-game theory has the following advantages over the traditional non-cooperative game theory we have studied in the previous chapters:

• As we have seen, the Nash equilibrium is the most common solution concept for non-cooperative games. An N-tuple of strategies in an N-player game is said to be in Nash equilibrium if an agent (player) cannot improve his payoff by moving to another strategy, given that the other players stay with their strategies at Nash equilibrium. Specifically, the strategy of a player at Nash equilibrium is the best response to the strategies of the other players, again at Nash equilibrium. However, the Nash equilibrium is not necessarily efficient, as it would be possible for all players to benefit from a collective behavior. Also, there could be multiple Nash equilibria in a game, and if the agent is restricted to adopting only pure strategies, the Nash equilibrium may not exist. In this case, the solution of the evolutionary game (i.e., evolutionarily stable strategies (ESS) or evolutionary equilibrium) can serve as a refinement to the Nash equilibrium, especially when multiple Nash equilibria exist.

• […]

While non-cooperative game theory studies the strategic choices resulting from the interactions among competing players, cooperative game theory provides analytical tools to study the behavior of rational players when they cooperate. In this context, in a cooperative-game scenario, the players are allowed to form agreements among themselves that can impact the strategic choices of these players as well as their utilities. Cooperative games encompass two main branches: bargaining theory and coalitional games. The former describes the bargaining process between a set of players that need to agree on the terms of cooperation, while the latter describes the formation of cooperating groups of players, referred to as coalitions, that can strengthen the players' positions in a game. In this chapter, we examine the key characteristics, properties, and solution concepts of both branches of cooperative games as well as sample applications within wireless and communication networks.

Bargaining theory

Introduction

In economics, many problems involve a number of entities that are interested in reaching an agreement over a trade or the sharing of a resource but have a conflicting interest on how to reach this agreement and on the terms of the agreement. In this context, a bargaining situation is defined as a situation in which two (or more) players can mutually benefit from reaching a certain agreement but have conflicting interests on the terms of the agreement.

Recent advances in wireless communication have made possible the large-scale deployment of wireless networks, which consist of small, low-cost nodes with simple processing and networking capabilities. In order to reach the desired destination such as the data sink, transmissions depending on multiple hops are necessary. As a result, the optimization of routing is a critical problem that involves many aspects such as link quality, energy efficiency, and security. Moreover, the nodes may not be willing to fully cooperate. For example, from the node's perspective, forwarding the arriving packets consumes its limited battery power, so it may not be in the node's interest to forward all arriving packets. But doing so will adversely affect network connectivity. Hence, it is crucial to design a distributed-control mechanism that encourages cooperation among participating multi-hop nodes.

This chapter studies game-theoretic approaches to routing in multi-hop networks. We first introduce important models and examples of routing games. We provide two detailed examples, a repeated-routing game and a hierarchical-routing game for enforcing cooperation. Finally, we list other approaches from the literature.

Routing-game basics

A network is given by a directed graph G = (V, E), with vertex set V and edge set E (either directed or undirected). A set {(s1, d1), …, (sK, dK)} consists of source–destination vertex pairs, which we also call commodities. Each player is identified with one commodity. Different players can originate from different source vertices and pass information to different destination vertices.

IEEE 802.11 wireless local area networks (WLANs) have been widely deployed in many places for both residential and commercial use. The IEEE 802.11 standard supports two major configurations – i.e., the point – coordination function (PCF) and the distributed coordination function (DCF). With PCF, the transmission in the network is based on a central node (i.e., an access point). Client nodes listen to the channel and wait for the signal from the access point. Once permission is sent by the access point, the client node can start data transmission. On the other hand, with DCF, the nodes employ carrier-sense multiple-access with collision avoidance (CSMA/CA) for MAC protocol. Each node can transmit independently, based on the availability of the channel. In particular, with CSMA/CA, the nodes listen for the channel status. If the channel is busy, the node defers its transmission by waiting for a backoff period. If a node senses a channel is idle, it will wait for a certain period of time and start transmission. In this case, multiple nodes can start transmissions at the same time, which results in collision. The colliding nodes will wait for the backoff period, and then sense for transmission again. To avoid performance degradation arising from packet collision, the backoff period can be adjusted, according to a specific rule, on the basis of the congestion level in the network (e.g., the rate of packet collisions).

With the recent advances in telecommunications technologies, wireless networking has become ubiquitous because of the great demand created by pervasive mobile applications. The convergence of computing, communications, and media will allow users to communicate with each other and access any content at any time and at any place. Future wireless networks are envisioned to support various services such as high-speed access, telecommuting, interactive media, video conferencing, real-time Internet games, e-business ecosystems, smart homes, automated highways, and disaster relief. Yet many technical challenges remain to be addressed in order to make this wireless vision a reality. A critical issue is devising distributed and dynamic algorithms for ensuring a robust network operation in time-varying and heterogeneous environments. Therefore, in order to support tomorrow's wireless services, it is essential to develop efficient mechanisms that provide an optimal cost-resource-performance tradeoff and that constitute the basis for next-generation ubiquitous and autonomic wireless networks.

Game theory is a formal framework with a set of mathematical tools to study the complex interactions among interdependent rational players. For more than half a century, game theory has led to revolutionary changes in economics, and it has found a number of important applications in politics, sociology, psychology, communication, control, computing, and transportation, to list only a few. During the past decade, there has been a surge in research activities that employ game theory to model and analyze modern communication systems.

Cooperative communication has attracted significant recent attention as a transmission strategy for future wireless networks. It efficiently takes advantage of the broadcast nature of wireless networks to allow network nodes to share their messages and transmit cooperatively as a virtual antenna array, thus providing diversity that can significantly improve system performance. Cooperative communication can be applied in a variety of wireless systems and networks. In the research community, a considerable amount of work has been done in this area for networks such as cellular, WiFi, ad hoc/sensor networks, and ultra wideband (UWB). These ideas are also working their way into standards; e.g., the IEEE 802.16 (WiMAX) standards body for future broadband wireless access has established the 802.16j relay task group to incorporate cooperative relaying mechanisms into this technology. Most existing work on cooperative communication concentrates on the physical (PHY) and medium access control (MAC) layers of wireless networks, examining issues such as capacity improvement, power control, and relay selection. The impact on the higher layers, such as routing in the network layer, has not been fully investigated yet.

The merits of cooperative transmission at the physical layer have been well explored. However, the impact of cooperative transmission on the design of the higher layers is not well understood yet. Specifically, the issues for various layers are:

• Physical layer. Objectives include optimizing the capacity region, minimizing the bit error rate (BER), and improving the link quality by power control.

• […]

Communication networks such as the Internet are becoming more and more dependent on the interactions of intelligent devices that are capable of autonomously operating within a highly dynamic and rapidly changing environment. The dynamism and complexity of Internet networks is a consequence of their size, heterogeneity, traffic diversity, and decentralized nature. Next-generation communication networks such as the future Internet are envisioned to be self-organizing, self-configuring, self-protecting, and self-optimizing. The applications and services that make use of these networks will also grow in complexity and impose stringent constraints and demands on network design: increased quality-of-service requirements for routing data, content distribution based on peer-to-peer networks, advanced pricing, and congestion control mechanisms, etc.

While these challenges were initially perceived with the emergence of the Internet, they are now essential in the design of every current and future network. To efficiently analyze and study such Internet-like networks, there is a need for a rich analytical framework such as game theory, whose models and algorithms can capture the numerous challenges arising in current and emerging communication networks. The challenges in designing Internet networks differ from those of their wireless counterparts in several aspects. In general, one does not need to worry about the reliability of the communication channel, as in the wireless case. But because Internet networks are generally composed of heterogeneous nodes having different capabilities and communicating over long paths and routes, the network size and as well as heterogeneity the nodes' capabilities, for example, play a role that is more critical in the design of Internet networks than in the wireless case.