Introduction

In this chapter we consider the optimization of scalar filters for single-input single-output (SISO) channels. A number of optimization problems which arise in different contexts will be considered. In Sec. 10.2 we begin with the digital communication system of Fig. 10.1 for a fixed channel H(jω). We consider the optimization of the continuous-time prefilter (transmitted pulse shape) and postfilter (equalizer) to minimize the mean square reconstruction error under the zero-forcing condition on the product F(jω)H(jω)G(jω). The zero-forcing condition does not uniquely determine the above product. It will be shown that the optimal product (under the zero-forcing condition) is the so-called optimal compaction filter of the channel (Sec. 10.2.3). Usually the filters that result from the above optimization problem are ideal, unrealizable, filters and can only be approximated. The equivalent digital channel therefore requires further equalization. In Sec. 10.3 we consider the problem of jointly optimizing a digital prefilter–postfilter pair to minimize the mean square error. Both the zero-forcing and the non-ZF situation are considered.

Section 10.4 revisits Fig. 10.1 for an arbitrary channel H(jω) from a more general viewpoint and formulates some general conditions on the filters F(jω) and G(jω) for optimality. The most general forms of the postfilter and prefilter for optimality are established. These forms were first derived by Ericson [1971, 1973]. Using these results we can argue that the optimization of the continuoustime filters in Fig. 10.1 can always be reformulated as the optimization of a digital prefilter–postfilter pair.

Digital communication systems have been studied for many decades, and they have become an integral part of the technological world we live in. Many excellent books in recent years have told the story of this communication revolution, and have explained in considerable depth the theory and applications. Since the late 1990s particularly, there have been a number of significant contributions to digital communications from the signal processing community. This book presents a number of these recent developments, with emphasis on the use of filter bank precoders and equalizers. Optimization of these systems will be one of the main themes in this book. Both multiple-input multiple-output (MIMO) systems and single-input single-output (SISO) systems will be considered.

The book is divided into four parts. Part 1 contains introductory material on digital communication systems and signal processing aspects. In Part 2 we discuss the optimization of transceivers, with emphasis on MIMO channels. Part 3 provides mathematical background material for optimization of transceivers. This part can be used as a reference, and will be useful for readers wishing to pursue more detailed literature on optimization. Part 4 contains eight appendices on commonly used material such as matrix theory, Wiener filtering, and so forth. Thus, while it is assumed that the reader has some exposure to digital communications and signal processing at the introductory level, there is plenty of review material at the introductory level (Part 1) and at the advanced level (Parts 3 and 4).