In the third part of this book, the third branch of modern game theory – sequential decision-making – is presented. The important components in sequential decision-making, such as network externality, information asymmetry, and user rationality, are presented and defined. The limitations of the existing approaches, such as social learning and multiarm bandit problems, are also presented.

The viability of cooperative communications depends on the willingness of users to help. Therefore, it is important to study incentive issues when designing such systems. In this chapter, we discuss a cooperation stimulation scheme for multiuser cooperative communications using an indirect reciprocity game. By introducing the notion of reputation and social norms, rational users who care about their future utility are incentivized to cooperate with others. Differently from existing works on reputation-based schemes that mainly rely on experimental verification, the effectiveness of the scheme is demonstrated in two steps. First, we conduct a steady-state analysis of the game and show that cooperating with users who have a good reputation can be sustained as an equilibrium when the cost-to-gain ratio is below a certain threshold. Then, by modeling the action spreading at transient states as an evolutionary game, we show that the equilibria we found in the steady-state analysis are stable and can be reached with proper initial conditions. Moreover, we introduce energy detection to handle the possible cheating behaviors of users and study its impact on the indirect reciprocity game.

A huge amount of information, created and forwarded by millions of people with various characteristics, propagates through online social networks every day. Understanding the mechanisms of information diffusion over social networks is critical to various applications, including online advertisements and website management. Differently from most existing works in this area, we investigate information diffusion from an evolutionary game-theoretic perspective and try to reveal the underlying principles dominating the complex information diffusion process over heterogeneous social networks. Modeling the interactions among the heterogeneous users as a graphical evolutionary game, we derive the evolutionary dynamics and the evolutionarily stable states (ESSs) of the diffusion. The different payoffs of the heterogeneous users lead to different diffusion dynamics and ESSs among them, in accordance with the heterogeneity observed in real-world data sets. The theoretical results are confirmed by simulations. We also test the theory on the Twitter hashtag data set. We observe that the evolutionary dynamics fit the data well and can predict future diffusion data.

Network service acquisition in a wireless environment requires the selection of a wireless access network. A key problem in wireless access network selection is studying rational strategies that consider negative network externality. In this chapter, we formulate the wireless network selection problem as a stochastic game with negative network externality and show that finding the optimal decision rule can be modeled as a multidimensional Markov decision process. A modified value-iteration algorithm is utilized to efficiently obtain the optimal decision rule with a simple threshold structure. We further investigate the mechanism design problem with incentive compatibility constraints, which force the networks to reveal truthful state information. The formulated problem is a mixed-integer programming problem that, in general, lacks an efficient solution. Exploiting the optimality of substructures, we introduce a dynamic programming algorithm that can optimally solve the problem in the two-network scenario. For the multinetwork scenario, the dynamic programming algorithm can outperform the heuristic greedy approach in polynomial-time complexity.

In a social network, agents are intelligent and have the capacity to make decisions so as to maximize their utility. They can either make wise decisions by taking advantages of other agents’ experiences through learning or make decisions earlier to avoid competition from huge crowds. Both of these effects – social learning and negative network externality – play important roles in the decision-making process of an agent. In this chapter, a new game called the Chinese restaurant game is introduced to formulate the social learning problem with negative network externality. Through analyzing the Chinese restaurant game, we derive the optimal strategy of each agent and provide a recursive method to achieve the optimal strategy. How social learning and negative network externality influence each other under various settings is studied through simulations. We also illustrate the spectrum access problem in cognitive radio networks as one application of the Chinese restaurant game. We find that the Chinese restaurant game-theoretic approach indeed helps users make better decisions and improves overall system performance.

Many spectrum sensing methods and dynamic access algorithms have been proposed to improve secondary users’ access opportunities. However, few of them have considered integrating the design of spectrum sensing and access algorithms together by taking into account the mutual influence between them. In this chapter, we focus on jointly analyzing the spectrum sensing and access problem. Due to their selfish nature, secondary users tend to act selfishly to access the channel without contributing to spectrum sensing. Moreover, they may employ out-of-equilibrium strategies because of the uncertainty of others’ strategies. To model the complicated interactions among secondary users, the joint spectrum sensing and access problem is formulated as an evolutionary game and the evolutionarily stable strategy (ESS) that no one will deviate from is studied. Furthermore, a distributed learning algorithm for the secondary users to converge to the ESS is introduced. Simulation results shows that the system can quickly converge to the ESS and such an ESS is robust to the sudden unfavorable deviations of the selfish secondary users.

Cooperation is a promising approach to simultaneously achieving efficient spectrum resource use and improving the quality of service of primary users in dynamic spectrum access networks. However, due to their selfish nature, how to stimulate secondary users to play cooperatively is an important issue. In this chapter, we discuss a reputation-based spectrum access framework where the cooperation stimulation problem is modeled as an indirect reciprocity game. In this game, secondary users choose how to help primary users relay information and gain reputation, based on which they can access a certain amount of vacant licensed channels in the future. By formulating a secondary user's decision-making as a Markov decision process, the optimal action rule can be obtained, according to which the secondary user will use maximal power to help the primary user relay data and thus greatly improve the primary user's quality of service as well as the spectrum utilization efficiency. Moreover, the uniqueness of the stationary reputation distribution is proved, and the conditions under which the optimal action rule is evolutionarily stable are theoretically derived.

The motivation of this book and necessary background knowledge of this book are provided. First, a brief introduction to competition and cooperation in wireless and social networks is provided, along with examples and a literature review. Then, the limitations of traditional game theory in this area are presented. Finally, the three branches of modern game theory – indirect reciprocity, evolutionary games, and sequential decision-making – will be briefly mentioned to illustrate their strengths for overcoming the highlighted limitations.

Deal selection on Groupon represents a typical social learning and decision-making process, where the quality of a deal is usually unknown to the customers. The customers must acquire this knowledge through social learning from other social media, such as reviews on Yelp. Additionally, the quality of a deal depends on both the state of the vendor and the decisions of other customers on Groupon. How social learning and network externality affect the decisions of customers in deal selection on Groupon is the main focus of this chapter. We develop a data-driven game-theoretic framework to understand rational deal selection behaviors across social media. The sufficient condition of the Nash equilibrium is identified. A value-iteration algorithm is utilized to find the optimal deal selection strategy. We utilize the Groupon–Yelp data set to analyze the deal selection game in a realistic setting. Finally, the performance of the social learning framework is evaluated using real data. The results suggest that customers make decisions in a rational way instead of following naive strategies, and there is still room to improve their decisions with assistance from a game-theoretic framework.

How information diffuses over social networks has attracted much attention from both industry and academics. Most of the existing works in this area are based on machine learning methods focusing on social network structure analysis and empirical data mining. However, the network users’ decisions, actions, and socioeconomic interactions are generally ignored in most existing works. In this chapter, we discuss an evolutionary game-theoretic framework to model the dynamic information diffusion process in social networks. Specifically, we derive the information diffusion dynamics in complete networks and uniform-degree and nonuniform-degree networks. We find that the dynamics of information diffusion over these three kinds of networks are scale-free and the same as each other when the network scale is sufficiently large. To verify the theoretical analysis, we perform simulations of the information diffusion over synthetic networks and real-world Facebook networks. Moreover, we conduct an experiment on the Twitter hashtag data set, which shows that the game-theoretic model well fits and predicts information diffusion over real social networks.

Distributed adaptive filtering has been considered to be an effective approach for data processing and estimation over distributed networks. Most existing algorithms focus on designing different information diffusion rules, regardless of the evolutionary characteristics of a distributed network. In this chapter, we study the adaptive network from the game-theoretic perspective and formulate the distributed adaptive filtering problem as a graphical evolutionary game. With this formulation, the nodes in the network are regarded as players and the local combiner of estimated information from different neighbors is regarded as a form of diverse strategy selection. We show that this graphical evolutionary game framework is very general and can unify the existing adaptive network algorithms. Based on this framework, as examples, two error-aware adaptive filtering algorithms are discussed. Moreover, we use graphical evolutionary game theory to analyze the information diffusion process over the adaptive networks and the evolutionarily stable strategy of the system. Finally, simulation results are shown to verify the effectiveness of the method discussed in this chapter.

The effectiveness of a decision may be uncertain due to the unknown system state. This uncertainty can be eliminated through learning from information sources, such as user-generated content or revealed actions. Nevertheless, user-generated content could be untrustworthy, since other agents may maliciously create misleading content for their selfish interests. Passively revealed actions are potentially more trustworthy and also easier to gather through simple observation. In this chapter, we introduce a game-theoretic framework – the hidden Chinese restaurant game (H-CRG) – to utilize the passively revealed actions in social learning process. We design grand information extraction, a novel Bayesian belief extraction process, to extract beliefs on hidden information directly from observed actions. The optimal policy is then analyzed in both centralized and game-theoretic approaches. We demonstrate how the H-CRG can be applied to the channel access problem in cognitive radio networks. The simulation results show that the equilibrium strategy derived in the H-CRG provides greater expected utilities for new users and maintains reasonably high social welfare.

In many social computing systems, users decide sequentially whether to participate or not and, if they participate, whether to create a piece of content directly (i.e. answering) or to rate existing content contributed by previous users (i.e. voting). We present in this chapter a game-theoretic model that formulates the sequential decision-making of strategic users under the presence of this answering–voting externality. We prove theoretically the existence and uniqueness of a pure strategy equilibrium. We show that there exist advantages for users with higher abilities and for answering earlier. Therefore, the equilibrium exhibits a threshold structure and the threshold for answering gradually increases as answers accumulate. To show the validness of the game-theoretic model, we analyze user behavior data collected from a popular question-and-answer site Stack Overflow and show that the main qualitative predictions of the game-theoretic model match up with observations made from the data. Finally, we formulate the system designer’s problem and abstract several design principles that could potentially guide the design of incentive mechanisms for social computing systems in practice.

The basics of game theory, which are necessary for understanding the rest of the book, are provided in this chapter. Specifically, typical game compoments, solution concepts, and their applications are explained.

Users may have multiple concurrent options regarding different objects/resources and their decisions usually negatively influence each other’s utility, which makes the sequential decision-making problem more challenging. In this chapter, we introduce an Indian buffet game to study how users in a dynamic system learn about the uncertain system state and make multiple concurrent decisions by not only considering their current myopic utility, but also the influence of subsequent users’ decisions. We analyze the Indian buffet game under two different scenarios: one of customers requesting multiple dishes without budget constraints and the other with budget constraints. In both cases, we design recursive best-response algorithms to find the subgame-perfect Nash equilibrium for customers and characterize special properties of the Nash equilibrium profile in a homogeneous setting. Moreover, we introduce a non-Bayesian social learning algorithm by which customers can learn the system state, and we theoretically prove its convergence. Finally, we conduct simulations to validate the effectiveness and efficiency of the Indian buffet game.

Data sharing is a critical step in implementing data fusion, and how to encourage sensors to share their data is an important issue. In this chapter, we discuss a reputation-based incentive framework where the data-sharing stimulation problem is modeled as an indirect reciprocity game. In this game, sensors choose how to report their results to the fusion center and gain reputation, based on which they can obtain certain benefits in the future. Taking the sensing and fusion accuracy into account, reputation distribution is introduced into the game, where we prove theoretically the Nash equilibrium of the game and its uniqueness. Furthermore, we apply the scheme to cooperative spectrum sensing. We show that, within an appropriate cost-to-gain ratio, the optimal strategy for the secondary users is to report when the average received energy is above a given threshold and to keep silent otherwise. Such an optimal strategy is also proved to be a desirable evolutionarily stable strategy.