Error-control codes are now in widespread use in many applications such as communication systems, magnetic recording systems, and optical recording systems. The compact disk and the digital video disk are two familiar examples of such applications.

We shall discuss only block codes for error control. A block code for error control is a set of n-tuples in some finite alphabet, usually the finite field GF(q). The reason for choosing a field as the alphabet is to have a rich arithmetic structure so that practical codes can be constructed and encoders and decoders can be designed as computational algorithms. The most popular block codes are linear. This means that the componentwise sum of two codewords is a codeword, and any scalar multiple of a codeword is a codeword. So that a large number of errors can be corrected, it is desirable that codewords be very dissimilar from each other. This dissimilarity will be measured by the Hamming distance.

The most important class of block codes, the Reed–Solomon codes, will be described as an exercise in the complexity of sequences and of Fourier transform theory. Another important class of block codes, the BCH codes, will be described as a class of subcodes of the Reed–Solomon codes, all of whose components lie in a subfield. The BCH codes and the Reed–Solomon codes are examples of cyclic codes, which themselves form a subclass of the class of linear block codes.

An array, υ = [υi′i″], defined as a doubly indexed set of elements from a given alphabet, was introduced in Chapter 5. There we studied the relationship between the two-dimensional array υ and its two-dimensional Fourier transform V. In this chapter, further properties of arrays will be developed by drawing material from the subject of commutative algebra, but enriching this material for our purposes and presenting some of it from an unconventional point of view.

The two-dimensional array υ can be represented by the bivariate polynomial υ(x, y), so we can study arrays by studying bivariate polynomials, which is the theme of this chapter. The polynomial notation provides us with a convenient way to describe an array. Many important computations involving arrays can be described in terms of the addition, subtraction, multiplication, and division of bivariate polynomials. Although n-dimensional arrays also can be studied as n-variate polynomials, in this book we shall treat only two-dimensional arrays and bivariate polynomials.

As the chapter develops, it will turn heavily toward the study of ideals, zeros of ideals, and the relationship between the number of zeros of an ideal and the degrees of the polynomials in any set of polynomials that generates the ideal. A well known statement of this kind is Bézout's theorem, which bounds the number of zeros of an ideal generated by two polynomials.

An arrayυ = [υi′i″] is a doubly indexed set of elements from any given alphabet. The alphabet may be a field F, and this is the case in which we are interested. We will be particularly interested in arrays over the finite field GF(q). An array is a natural generalization of a sequence; we may refer to an array as a two-dimensional sequence or, with some risk of confusion, as a two-dimensional vector.

An array may be finite or infinite. We are interested in finite n′ by n″ arrays, and in those infinite arrays [υi′i″] that are indexed by nonnegative integer values of the indices i′ and i″. An infinite array is periodic if integers n′ and n″ exist such that υi′+n′,i″+n″ = υi′i″. Any finite array can be made into a doubly periodic infinite array by periodically replicating it on both axes.

The notion of an array leads naturally to the notion of a bivariate polynomial; the elements of the array υ are the coefficients of the bivariate polynomial υ(x, y). Accordingly, we take the opportunity in this chapter to introduce bivariate polynomials and some of their basic properties. The multiplication of bivariate polynomials is closely related to the two-dimensional convolution of arrays. Moreover, the evaluation of bivariate polynomials, especially bivariate polynomials over a finite field, is closely related to the two-dimensional Fourier transform.

In contrast to the class of Reed–Solomon codes, which was introduced by engineers, the class of hermitian codes was introduced by mathematicians as an example of an important class of algebraic geometry codes. In this chapter, we shall reintroduce hermitian codes as they might have appeared had they been discovered by the engineering community. Some additional insights will be exposed by this alternative formulation. In particular, we will shift our emphasis from the notion of punctured codes on curves to the notion of shortened codes on curves. We then give constructions of hermitian codes as quasi-cyclic codes and as linear combinations of Reed–Solomon codes akin to the Turyn construction. Much of the structure of hermitian codes stands out quite clearly when a code is restricted to the bicyclic plane (or torus), thereby forming an epicyclic hermitian code. If one takes the view that the cyclic form is the more fundamental form of the Reed–Solomon code, then perhaps one should take the parallel view that the epicyclic form is the more fundamental form of the hermitian code. In particular, we shall see that, for the epicyclic form of an hermitian code, there is no difference between a punctured code and a shortened code. This is important because the punctured code is compatible with encoding and the shortened code is compatible with decoding. In Section 11.2, we shall provide a method for the direct construction of shortened epicyclic hermitian codes.

The binary erasure channel (BEC) is perhaps the simplest non-trivial channel model imaginable. It was introduced by Elias as a toy example in 1954. The emergence of the Internet promoted the erasure channel into the class of “real-world” channels. Indeed, erasure channels can be used to model data networks, where packets either arrive correctly or are lost due to buffer overflows or excessive delays.

A priori, one might well doubt that studying the BEC will significantly advance our understanding of the general case. Quite surprisingly, however, most properties and statements that we encounter in our investigation of the BEC hold in much greater generality. Thus, the effort invested in fully understanding the BEC case will reap substantial dividends later on.

You do not need to read the whole chapter to know what iterative decoding for the BEC is about. The core of the material is contained in Sections 3.1–3.14 as well as 3.24. The remaining sections concern either more specialized or less accessible topics. They can be read in almost any order.

CHANNEL MODEL

Erasure channels model situations where information may be lost but is never corrupted. The BEC captures erasure in the simplest form: single bits are transmitted and either received correctly or known to be lost. The decoding problem is to find the values of the bits given the locations of the erasures and the non-erased part of the codeword. Figure 3.1 depicts the BEC(∊).

This book is all about iterative channel decoding. Two other names which are often used to identify the same area are probabilistic coding and codes on graphs. Iterative decoding was originally conceived by Gallager in his remarkable Ph.D. thesis of 1960. Gallager's work was, evidently, far ahead of its time. Limitations in computational resources in the 1960s were such that the power of his approach could not be fully demonstrated, let alone developed. Consequently, iterative decoding attracted only passing interest and slipped into a long dormancy. It was rediscovered by Berrou, Glavieux, and Thitimajshima in 1993 in the form of turbo codes, and then independently in the mid 1990s by MacKay and Neal, Sipser and Spielman, as well as Luby, Mitzenmacher, Shokrollahi, Spielman, and Stemann in a form much closer to Gallager's original construction. Iterative techniques have subsequently had a strong impact on coding theory and practice and, more generally, on the whole of communications.

The title Modern Coding Theory is clearly a hyperbole. There have been several other important recent developments in coding theory. To mention one prominent example: Sudan's algorithm and the Guruswami-Sudan improvement for list decoding of Reed-Solomon codes and their extension to soft-decision decoding have sparked new life into this otherwise mature subject. So what is our excuse? Iterative methods and their theory are strongly tied to advances in current computing technology and they are therefore inherently modern. They have also brought about a break with the past.

We now look at a select list of applications beyond the simple model of binary memoryless symmetric channels. We formalize each problem and point out how the system can be analyzed. Rather than discussing applications in their full generality, we limit ourselves to interesting special cases. In the same spirit, we do not present highly tuned solutions but explain how each system can be optimized. This keeps the exposition simple. The generalizations are quite routine. Since the Forney-style factor graph (FSFG) of a large system is the composition of the individual FSFGs of its components, it suffices for the most part to study those components in isolation. Real transmission scenarios typically involve combinations of the various components discussed in the following.

Each example introduces one new ingredient. A simple model of a fading channel is discussed in Section 5.1. We next discuss the prototypical asymmetric channel (the so-called Z channel) in Section 5.2. We then turn in Section 5.3 to an information-theoretic application of factor graphs – computing information rates of channels with memory. We also discuss how to code over channels with memory. In Section 5.4 we see how to construct systems with high spectral efficiency from simple binary ones. Very similar in spirit is the discussion on multiple-access channels in Section 5.5.

We now broaden our scope of channels from the binary erasure channel (BEC) to the class of binary memoryless symmetric (BMS) channels. Many concepts encountered during our study of the BEC still apply and reappear suitably generalized. It might at first seem that this expanded class of channels is still restricted and special and that we are only covering a small portion of the large volume of channels encountered in practice. Actually, however, a wide range of situations can be dealt with rather straightforwardly once we have mastered BMS channels. One should therefore view the following as part of the foundation upon which much of communications rests.

Sections 4.2–4.10 recapitulate Sections 3.5–3.6, and 3.8–3.14 in this more general setting and these sections form the core material. The remaining sections can be read in essentially any order. They contain either more advanced topics or less accessible material.

General BMS channels are more mathematically challenging than the BEC. Section 4.1 summarizes the necessary prerequisites. Our advice: quickly skim it so you know what material it contains but do not study it in detail. At any point later, when the need arises, you can return to fill in gaps.