Introduction

The Power Spectral Density of a stochastic process tells us more about the SP than just its power. It tells us something about how this power is distributed among the different frequencies that the SP occupies. The purpose of this chapter is to clarify this statement and to derive the PSD of PAM signals. Most of this chapter is written informally with an emphasis on ideas and intuition as opposed to mathematical rigor. The mathematically-inclined readers will find precise statements of the key results of this chapter in Section 15.5. We emphasize that this chapter only deals with real continuous-time stochastic processes.

The classical definition of the PSD of continuous-time stochastic processes (Definition 25.7.2 ahead) is only applicable to wide-sense stationary stochastic processes, and PAM signals are not WSS. Consequently, we shall have to introduce a new concept, which we call the operational power spectral density, or the operationalPSD for short. This new concept is applicable to a large family of stochastic processes that includes most WSS processes and most PAM signals. For WSS stochastic processes, the operational PSD and the classical PSD coincide (Section 25.14). In addition to being more general, the operational PSD is more intuitive in that it clarifies the origin of the words “power spectral density.” Moreover, it gives an operational meaning to the concept.

Section 15.2 provides some motivation for our definition of the operational PSD. Readers in a rush to get to the definition can skip it and jump directly to Section 15.3. Section 15.4 derives the operational PSD of PAM signals, and Section 15.5 does so a bit more rigorously. Sections 15.6–15.8 expand on the operational PSD. Section 15.6 explores its relationship to the average autocovariance function; Section 15.7 discusses the effect of filtering on the operational PSD; and Section 15.8 explores the technical assumptions that are required for the power to equal the integral of the operational PSD over all the frequencies.

Without conceding a blemish in the first edition, I think I had best come clean and admit that I embarked on a second edition largely to adopt a more geometric approach to the detection of signals in white Gaussian noise. Equally rigorous, yet more intuitive, this approach is not only student-friendly, but also extends more easily to the detection problem with random parameters and to the radar problem.

The new approach is based on the projection of white Gaussian noise onto a finite-dimensional subspace (Section 25.15.2) and on the independence of this projection and the difference between noise and projection; see Theorem 25.15.6 and Theorem 25.15.7. The latter theorem allows for a simple proof of the sufficiency of the matched-filters’ outputs without the need to define sufficient statistics for continuous-time observables. The key idea is that—while the receiver cannot recover the observable from its projection onto the subspace spanned by the mean signals—it can mimic the performance of any receiver that bases its decision on the observable using three steps (Figure 26.1 on Page 623): use local randomness to generate an independent stochastic process whose law is equal to that of the difference between the noise and its projection; add this stochastic process to the projection; and feed the result to the original receiver.

But the new geometric approach was not the only impetus for a second edition. I also wanted to increase the book's scope. This edition contains new chapters on the radar problem (Chapter 30), the intersymbol interference (ISI) channel (Chapter 32), and on the mathematical preliminaries needed for its study (Chapter 31). The treatment of the radar problem is fairly standard with two twists: we characterize all achievable pairs of false-alarm and missed-detection probabilities (pFA, pMD) and not just those that are Pareto-optimal. Moreover, we show that when the observable has a density under both hypotheses, all achievable pairs can be achieved using deterministic decision rules.

As to ISI channels, I adopted the classic approach of matched filtering, discretetime noise whitening, and running the Viterbi Algorithm. I only allow (boundedinput/ bounded-output) stable whitening filters, i.e., filters whose impulse response is absolutely summable; others often only require that the impulse response be square summable.

Introduction

In layman's terms, a sufficient statistic for guessing M based on the observable Y is a random variable or a collection of random variables that contains all the information in Y that is relevant for guessing M. This is a particularly useful concept when the sufficient statistic is more concise than the observables. For example, if we observe the results of a thousand coin tosses Y 1, …, Y1000 and we wish to test whether the coin is fair or has a bias of 1/4, then a sufficient statistic turns out to be the number of “heads”among the outcomes Y 1, …, Y1000. Another example was encountered in Section 20.12. There the observable was a two-dimensional random vector, and the sufficient statistic summarized the information that was relevant for guessing H in a scalar random variable; see (20.69).

In this chapter we provide a formal definition of sufficient statistics in the multihypothesis setting and explore the concept in some detail. We shall see that our definition is compatible with Definition 20.12.2, which we gave for the binary case. We only address the case where the observations take value in the d-dimensional Euclidean space Rd . Also, we only treat the case of guessing among a finite number of alternatives. We thus consider a finite set of messages

whereM ≥ 2, and we assume that associated with each message is a density on Rd , i.e., a nonnegative Borel measurable function that integrates to one.

The concept of sufficient statistics is defined for the family of densities

it is unrelated to a prior. But when we wish to use it in the context of hypothesis testing we need to introduce a probabilistic setting. If, in addition to the family, we introduce a prior, then we can discuss the pair (M, Y), where Pr[M = m] = πm , and where, conditionally on M = m, the distribution of Y is of density.