In this chapter, we consider the consistent estimation of system parameters involving random matrices with large dimensions. When it comes to estimation or statistical inference in signal processing, there often exists a large number of different methods proposed in the literature, most of which are usually based on a reference, simple, and robust method which has various limitations such as the Urkowitz's power detector [Urkowitz, 1967] that only assumes the additive white Gaussian noise (AWGN) model, or the multiple signal classification (MUSIC) algorithm [Schmidt, 1986] of Schmidt that suffers from undecidability issues when the signal to noise ratio reaches a critically low value. When performing statistical inference based on a limited number of large dimensional vector inputs, the main limitation is due to the fact that those legacy estimators are usually built under the assumption that the number of available observations is extremely large compared to the number of system parameters to identify. In modern signal processing applications, especially for large sensor networks, the estimators receive as inputs the M stacked N-dimensional observation vectors Y = [y(1),…,y(M)] ∈ ℂN × M of some observation vectors y(m) ∈ ℂ N at time m, M and N being of similar size, or even sometimes M being much smaller than N. Novel estimators that can cope with this large population size limitation are therefore required in place of the historical estimators.

In this chapter, we now address a quite different problem than the performance evaluation of data transmissions in large dimensional communication channel models. The present chapter, along with Chapter 17, deals with practical signal processing techniques to solve problems involving (possibly large dimensional) random matrix models. Specifically in this chapter, we will first address the question of signal sensing using multi-dimensional sensor arrays.

Cognitive radios and sensor networks

A renewed motivation for large dimensional signal sensing has been recently triggered by the cognitive radio incentive, which, according to some, may be thought of as the next information-theoretic revolution after the original work of Shannon [Shannon, 1948] and the introduction of multiple antenna systems by Foshini [Foschini and Gans, 1998] and Telatar [Telatar, 1999]. In addition to the theoretical expression of the point-to-point noisy channel capacity in [Shannon, 1948], Shannon made us realize that, in order to achieve high rate of information transfer, increasing the transmission bandwidth is largely preferred over increasing the transmission power. Therefore, to ensure high rate communications with a finite power budget, we have to consider frequency multiplexing. This constituted the first and most important revolution in modern telecommunications and most notably wireless communications, which led today to an almost complete saturation of all possible transmission frequencies. By “all possible,” we mean those frequencies that can effciently carry information (high frequencies tend to be rapidly attenuated when propagating in the atmosphere) and be adequately processed by analog and digital devices (again, high frequency carriers require expensive and sometimes even physically infeasible radio front-ends).

This last chapter of Part I introduces very recent mathematical advances of deep interest to the field of wireless communications, related to the limiting behavior of the extreme eigenvalues and of their corresponding eigenvectors. Again, the main objects which have been extensively studied in this respect are derivatives of the sample covariance matrix and of the information plus noise matrix.

This chapter will be divided into two sections, whose results emerge from two very different random matrix approaches. The first results, about the limiting extreme eigenvalues of the spiked models, unfold from the previous exact separation results described in Chapter 7. It will in particular be proved that in a sample covariance matrix model, when all population eigenvalues are equal but for the few largest ones, the l.s.d. of the sample covariance matrix is still the Marčenko–Pastur law, but a few eigenvalues may now be found outside the support of the l.s.d. The second set of results concerns mostly random matrix models with Gaussian entries, for which limiting results on the behavior of extreme eigenvalues are available. These results use very different approaches than those proposed so far, namely the theory of orthogonal polynomials and determinantal representations. This subject, which requires many additional tools, is briefly introduced in this chapter. For more information about these tools, see, e.g. the tutorial [Johnstone, 2006] or the book [Mehta, 2004].

Before concluding this book, we will briefly discuss open questions in random matrix theory. We will describe current research topics, results that are still unknown and that would be worth looking into. We also introduce briefly the replica methods, an alternative to the Stieltjes transform, and free probability methods proposed in this book which have been gaining a lot of interest lately, and will conclude with possible extensions of random matrix theory to a more elaborate time-varying (stochastic) mathematical framework.

From asymptotic results to finite dimensional studies

First, we recall the recent advances in random matrix theory, already introduced in this book, and which will be studied further in the near future. In Part I, we first explored limiting spectral distributions of some simple random matrix models. For instance, we characterized the l.s.d. of the sample covariance matrix model when the e.s.d. of the population covariance matrix converges weakly to some distribution function. We then observed that, for more practical system models, such as the sum of matrices with independent entries and left- and right-correlations, there may not exist a limiting spectral distribution, even when all deterministic matrices in the model do have a l.s.d. This led us to consider the deterministic equivalent approach instead of the l.s.d. approach. These deterministic equivalents have an outstanding advantage over l.s.d. and can be considered a major breakthrough for applied random matrix theory for the following reasons.

In this chapter, we consider both multiple access channels (MAC), which assume a certain number of users competing for the access to (i.e. to transmit data to) a single resource, and broadcast channels (BC), which assume the opposite scenario where a single transmitter multicasts data to multiple receivers.

The performance of multi-user communications can no longer be assessed from a single capacity parameter, as was the case for point-to-point communications. In a K-user MAC, we must evaluate what vectors (R1, …, RK) of rates, Ri being the data rate transmitted by user i, are achievable, in the sense that simultaneous reliable decoding of all data streams is possible at the receiver. Now, similar to single-user communications, where all rates R less than the capacity C are achievable and therefore define a rate set R = {R, R ≤ C}, for the multiple access channel, we define the multi-dimensional MAC rate region as the set ℛmac of all vectors (R1, …, Rk) such that reliable decoding is possible at the receiver if users 1, …,K transmit, respectively, at rate R1, …, Rk. Similarly, for the broadcast channel, we define the BC rate region ℛbc as the (closed) set of all vectors (R1, …, RK), Ri being now the information data rate received by user i, such that every user can reliably decode its data.