Signals and Functional Spaces

The set of signals for the communication engineer corresponds to an infinite-dimensional functional space for the mathematician. This can be viewed as a vector space on which norm (i.e., length), inner product (i.e., angle), and limits can be defined. The engineering problem of determining the effective dimension of the signals’ space then falls in the mathematical framework of approximation theory that is concerned with finding a sequence of functions with desirable properties whose linear combination best approximates a given limit function.

Approximation is a well-studied problem in analysis. In terms of abstract Hilbert spaces, the problem is to determine what functions can asymptotically generate a given Hilbert space in the sense that the closure of their span (i.e., finite linear combinations and limits of those) is the whole space. Such a set of functions is called a basis, and its cardinality, taken in a suitable limiting sense, is the Hilbert dimension of the space.

Various differential equations arising in physics have orthogonal solutions that can be interpreted as bases of Hilbert spaces. One example is the solution of the wave equation leading to the prolate spheroidal wave functions examined in the previous chapter. Another notable example arises in quantum mechanics in the context of the Schrödinger differential equation.

In this chapter, we describe the connection between physical properties, such as the energy concentration of a wave function, and the mathematics of Hilbert spaces, showing that Slepian's concentration problem is a special case of the eigenvalue problem arising from the spectral decomposition of a self-adjoint operator on a Hilbert space. It turns out that this decomposition provides the optimal approximation for any function in the space. The effective dimension, or degrees of freedom, of the space is then defined as the cardinality of such an optimal representation.