In this chapter, we study the Age of Information (AoI) when the status updates of the underlying process of interest can be sampled at any time by the source node and are transmitted over an error-prone wireless channel. We assume the availability of perfect feedback that informs the transmitter about the success or failure of transmitted status updates and consider various retransmission strategies. More specifically, we study the scheduling of sampling and transmission of status updates in order to minimize the long-term average AoI at the destination under resource constraints. We assume that the underlying statistics of the system are not known, and hence, propose average-cost reinforcement learning algorithms for practical applications. Extensions of the results to a multiuser setting with multiple receivers and to an energy-harvesting source node are also presented, different reinforcement learning methods including deep Q Network (DQN) are exploited and their performances are demonstrated.

In this chapter, we study the value of information, a more comprehensive instrument than the age of information, for shaping the information flow in a networked control system subject to random processing delay. In addition, we establish a connection between these two instruments by presenting a condition under which the value of information is expressible in terms of the age of information. Nonetheless, we show that this condition is not achievable without a degradation in the performance of the system.

This chapter characterizes the average Age of Information (AoI) for the case of having multiple sources sharing a service facility with a single server. In particular, a simplified explanation of the SHS for AoI approach is provided to calculate the average age of updates of any source at the monitor. This approach is applied to various queueing systems including FCFS, M/M/1*, and M/M/1/2*, and the latter two with and without source priorities.

This chapter considers an application of age of information called AoCSI in which the channel states in a wireless network represent the information of interest and the goal is to maintain fresh estimates of these channel states at each node in the network. Rather than sampling some underlying time-varying process and propagating updates through a queue or graph, the AoCSI setting obtains direct updates of the channels as a by-product of wireless communication through standard physical layer channel estimation techniques. These CSI estimates are then disseminated through the network to provide global snapshots of the CSI to all of the nodes in the network. What makes the AoCSI setting unique is that disseminating some CSI updates and directly sampling/estimating other CSI occur simultaneously. Moreover, as illustrated in this chapter, there are inherent trade-offs on how much CSI should be disseminated in each transmission to minimize the average or maximum age.

Age-of-information (AoI) is used to characterize the freshness of information, and is critical for information monitoring, tracking, and control, which is typically required in many network applications, such as autonomous vehicles, virtual/augmented reality, and Internet-of-Things (IoT). Both inter-arrival times and delays of packets affect AoI performance, and thus traditional delay-efficient algorithms do not necessarily exhibit low AoI performance. This calls for “age-efficient” algorithm design in communication networks, which forms the focus of this chapter. In particular, we first discuss the recent advances in the age-efficient algorithm design for three different types of common network traffic: (i) elastic traffic (cf. Section 1.1): packets are allowed to be delivered without any specific deadline constraints; (ii) inelastic traffic (cf. Section 1.2): packets will be dropped if they are not delivered within a specific deadline; (iii) heterogeneous traffic (cf. Section 1.3): different packets may have different size. To facilitate our discussions, we explicitly consider the discrete-time model and emphasize the difference between age-efficient and delay-efficient algorithm design paradigms. Then, we examine “fresh” scheduling design for remote estimation with the goal of optimally balancing the trade-of between the estimation accuracy and the communication cost (cf. Section 1.4).

In this chapter, we consider a joint sampling and scheduling problem for optimizing data freshness in multisource systems. Data freshness is measured by a nondecreasing penalty function of Age of Information, where all sources have the same age-penalty function. Sources take turns to generate update samples, and forward them to their destinations one-by-one through a shared channel with random delay. There is a scheduler, that chooses the update order of the sources, and a sampler, that determines when a source should generate a new sample in its turn. We aim to find the optimal scheduler–sampler pairs that minimize the total-average age-penalty (Ta-AP). We start the chapter by providing a brief explanation of the sampling problem in the light of single–source networks, as well as some useful insights and applications on age of information and its penalty functions. Then, we move on to the multisource networks, where the problem becomes more challenging. We provide a detailed explanation of the model and the solution in this case. Finally, we conclude this chapter by providing an open question in this area and its inherent challenges.

While age of Information (AoI) has gained importance as a metric characterizing the freshness of information in information-update systems and time-critical applications, most previous studies on AoI have been theoretical. In this chapter, we compile a set of recent works reporting AoI measurements in real-life networks and experimental testbeds, and investigating practical issues such assynchronization, the role of various transport layer protocols, congestion control mechanisms, application of machine learning for adaptation to network conditions, and device-related bottlenecks such as limited processing power.

In this chapter, we discuss the relationship between Age of Information and signal estimation error in real-time signal sampling and reconstruction. Consider a remote estimation system, where samples of a scalar Gauss–Markov signal are taken at a source node and forwarded to a remote estimator through a channel that is modeled as a queue. The estimator reconstructs an estimate of the real-time signal value from causally received samples. The optimal sampling policy for minimizing the mean square estimation error is presented, in which a new sample is taken once the instantaneous estimation error exceeds a predetermined threshold. When the sampler has no knowledge of current and history signal values, the optimal sampling problem reduces to a problem for minimizing a nonlinear Age of Information metric. In the AoI-optimal sampling policy, a new sample is taken once the expected estimation error exceeds a threshold. The threshold can be computed by low-complexity algorithms and the insights behind these algorithms are provided. These optimal sampling results were established (i) for general service time distributions of the queueing server, (ii) for both stable and unstable scalar Gauss–Markov signals, and (iii) for sampling problems both with and without a sampling rate constraint.

This chapter explores Age of Information (AoI) in the context of the timely source coding problem. In most of the existing literature, service (transmission) times are based on a given distribution. In the timely source coding problem, by using source coding schemes, we design the transmission times of the status updates. We observe that the average age minimization problem is different than the traditional source coding problem, as the average age depends on both the first and the second moments of the codeword lengths. For the age minimization problem, we first consider a greedy source coding scheme where all realizations are encoded. For this source coding scheme, we find the age-optimal real-valued code word lengths. Then, we explore the highest k selective encoding scheme, where instead of encoding all realizations, we encode only the most probable k realizations. For each source encoding scheme, we first determine the average age expressions and then, for a given pmf, characterize the age-optimal k value, and find the corresponding age-optimal codeword lengths. Through numerical results, we show that selective encoding schemes achieve lower average age than encoding all realizations.

We continue our discussion of hidden Markov models (HMMs) and consider in this chapter the solution of decoding problems. Specifically, given a sequence of observations $\{y_n,n=1,2,\dots ,N\}$, we would like to devise mechanisms that allow us to estimate the underlying sequence of state or latent variables $\{z_n,n=1,2,\dots ,N\}$. That is, we would like to recover the state evolution that “most likely” explains the measurements. We already know how to perform decoding for the case of mixture models with independent observations by using (38.12a)–(38.12b). The solution is more challenging for HMMs because of the dependency among the states.

The various reinforcement learning algorithms described in the last two chapters rely on estimating state values, ${\upsilon }^{\pi }\left(s\right)$, or state–action values, $q^{\pi }(s,a)$, directly.

The mean-square-error (MSE) problem of estimating a random variable $\mathit{x}$ from observations of another random variable $\mathit{y}$ seeks a mapping $c\left(\mathit{y}\right)$ that solves