After studying this chapter you will be able to

• define random variables and signals;

• describe a random variable by the cumulative distribution function and by the probability density function;

• compute the expected value, mean, variance, standard deviation, correlation, and covariance of a random variable;

• define a Gaussian random signal;

• define independent and identically distributed (IID) signals;

• describe the concepts of stationarity, wide-sense stationarity, and ergodicity;

• compute the power spectrum and the cross-spectrum;

• relate the input and output spectra of an LTI system;

• describe the stochastic properties of linear least-squares estimates and weighted linear least-squares estimates;

• solve the stochastic linear least-squares problem; and

• describe the concepts of unbiased, minimum-variance, and maximum-likelihood estimates.

Introduction

In Chapter 3 the response of an LTI system to various deterministic signals, such as a step input, was considered. A characteristic of a deterministic signal or sequence is that it can be reproduced exactly. On the other hand, a random signal, or a sequence of random variables, cannot be exactly reproduced. The randomness or unpredictability of the value of a certain variable in a modeling context arises generally from the limitations of the modeler in predicting a measured value by applying the “laws of Nature.” These limitations can be a consequence of the limits of scientific knowledge or of the desire of the modeler to work with models of low complexity. Measurements, in particular, introduce an unpredictable part because of their finite accuracy.