In this chapter we shall turn our attention to the implementation of encoders and decoders. Elementary encoders and decoders will be based on shift-register circuits; more advanced encoders and decoders will be based on hardware circuits and software routines for Galois-field arithmetic.

Digital logic circuits can be organized easily into shift-register circuits that mimic the cyclic shifts and polynomial arithmetic used in the description of cyclic codes. Consequently, the structure of cyclic codes is closely related to the structure of shift-register circuits. These circuits are particularly well-suited to the implementation of many encoding and decoding procedures and often take the form of filters. In fact, many algorithms for decoding simple cyclic codes can be described most easily by using the symbolism of shift-register circuits. Studying such decoders for simple codes is worthwhile for its own sake, but it is also a good way to build up the insights and techniques that will be useful for designing decoders for large codes.

Logic circuits for finite-field arithmetic

Logic circuits are easily designed to execute the arithmetic of Galois fields, especially if q is a power of 2. We shall need circuit elements to store field elements and to perform arithmetic in the finite field.

A shift register, as shown in Figure 8.1, is a string of storage devices called stages. Each stage contains one element of GF(q). The symbol contained in each storage device is displayed on an output line leaving that stage.

A profusion and variety of communication systems, which carry massive amounts of digital data between terminals and data users of many kinds, exist today. Alongside these communication systems are many different magnetic tape storage systems, and magnetic and optical disk storage systems. The received signal in any communication or recording system is always contaminated by thermal noise and, in practice, may also be contaminated by various kinds of defects, nongaussian noise, burst noise, interference, fading, dispersion, cross talk, and packet loss. The communication system or storage system must transmit its data with very high reliability in the presence of these channel impairments. Bit error rates as small as one bit error in 1012 bits (or even smaller) are routinely specified.

Primitive communication and storage systems may seek to keep bit error rates small by the simple expedient of transmitting high signal power or by repeating the message. These simplistic techniques may be adequate if the required bit error rate is not too stringent, or if the data rate is low, and if errors are caused by noise rather than by defects or interference. Such systems, however, buy performance with the least expendable resources: Power and bandwidth.

In contrast, modern communication and storage systems obtain high performance via the use of elaborate message structures with complex cross-checks built into the waveform. The advantage of these modern communication waveforms is that high data rates can be reliably transmitted while keeping the transmitted power and spectral bandwidth small.

This book is a second edition of my 1983 book Theory and Practice of Error Control Codes. Some chapters from that earlier book reappear here with minor changes. Most chapters, however, have been completely rewritten. Some old topics have been removed, and some new topics have been inserted.

During the two decades since the publication of that first edition, error-control codes have become commonplace in communications and storage equipment. Many such communication and storage devices, including the compact disk, that are now in general use could not exist, or would be much more primitive, if it were not for the subject matter covered by that first edition and repeated in this edition.

The second edition retains the original purpose of the first edition. It is a rigorous, introductory book to the subject of algebraic codes for data transmission. In fact, this phrase, “algebraic codes for data transmission,” has been chosen as the title of the second edition because it reflects a more modern perspective on the subject.

Standing alongside the class of algebraic codes that is the subject of this book is another important class of codes, the class of nonalgebraic codes for data transmission. That rapidly developing branch of the subject, which is briefly treated in Chapter 11 of this edition, deserves a book of its own; this may soon appear now that the topic is reaching a more mature form.

A digital communication system may transmit messages consisting of thousands or even millions of bits. While one can always break a long message into short blocks for encoding, in principle, a single, long block code will give better performance because it will protect against both error patterns in which the errors are clustered and error patterns in which the errors are scattered throughout the message. Therefore, there are many occasions where good codes of very long blocklength can be used.

Although short binary cyclic codes can be quite good, the known long binary cyclic codes have a small minimum distance. Codes of large blocklength with a much larger minimum distance do in principle exist, though we know very little about these more powerful codes, nor do we know how to find them. Despite more than fifty years of intense effort, codes of large blocklength and large minimum distance, both binary and nonbinary, still elude us. Even if such codes were found, it may be that their decoding would be too complex. Accordingly, the most successful constructions for codes of large blocklength for many applications combine codes of small blocklength into more elaborate structures. We call such structures composite codes. The elementary codes from which composite codes are formed can then be called basic codes.

The theory of algebraic codes draws much of its strength from the structure and properties of finite fields. In earlier chapters these properties were used to develop powerful codes and efficient decoding algorithms. The properties of those codes are tightly coupled to the properties of the fields, and the decoding algorithms use the full structure of the field to realize computational efficiency. Algebraic codes are attractive, not necessarily because of their performance, but because their structure is closely entwined with the rich structure of finite fields, a structure that underlies practical encoder and decoder implementations.

However, not every code is well-suited to the powerful algebraic decoding algorithms that are now known and not every physical channel is well-described by the mathematical structure of a finite field. Moreover, a physical channel is usually a waveform channel that transmits real or complex functions of continuous time. The methods of modulation and demodulation allow the continuous channel to be treated as a discrete channel, transmitting real or complex sequences indexed by discrete time. The sequence of codeword symbols from a finite field must be mapped into the real or complex field to form a sequence of real or complex inputs to the channel. The sequence of real or complex outputs from the channel may be mapped back to the finite field but the map takes many values to one and information is lost.

An error-control code must be judged not only by its rate and minimum distance, but also by whether a decoder can be built for it economically. Usually, there are many ways to decode a given code. A designer can choose among several decoding algorithms and their variations, and must be familiar with all of them so that the best one for a particular application can be chosen. The choice will depend not only on the code parameters, such as blocklength and minimum distance, but also on how the implementation is to be divided between hardware and software, on the required decoding speed, and even on the economics of available circuit components.

In this chapter we shall broaden our collection of decoding techniques by working in the frequency domain. Included here are techniques for decoding in the presence of both erasures and errors, and techniques for decoding beyond the designed distance of a code.

Spectral estimation in a finite field

In Section 6.6. we studied the task of decoding BCH codes, including Reed–Solomon codes, up to the designed distance. This task will now be reinterpreted from the point of view of the Fourier transform. Using the terminology of the frequency domain, we shall develop alternative decoding procedures. These procedures will take the form of spectral estimation in a finite field.

In any engineering discipline, one looks for methods of solving a given class of problems and then for methods of quantifying the performance and complexity of those solutions. Accordingly, in this chapter, we shall study performance in terms of the composition structure of block codes, and the probabilities of decoding error and of decoding failure.

Later, after satisfactory methods are in hand, one turns to questions of optimality. Are these methods the best methods, and if not, in what ways and by how much are they inferior? To answer such questions, one needs to know how good are the known codes and how good are the best possible codes. Generally, we cannot answer either of these questions satisfactorily. Although for any given rate and blocklength, not too small, the best possible code is not known, a number of bounds are known – bounds beyond which no codes can exist, and bounds within which codes are sure to exist. We shall give some such bounds on the best possible codes later in this chapter. Knowledge of these bounds can deepen our understanding of the subject.

Weight distributions of block codes

If a linear block code has a minimum distance dmin, then we know that at least one codeword of weight dmin exists. Sometimes, we are not content with this single piece of information; we wish to know how many codewords have weight dmin, and what are the weights of the other codewords.

The most powerful and important ideas of coding theory are based on the arithmetic systems of Galois fields. Since these arithmetic systems may be unfamiliar to many, we must develop a background in this branch of mathematics before we can proceed with the study of coding theory.

In this chapter, we return to the development of the structure of Galois fields begun in Chapter 2. There we introduced the definition of a field, but did not develop procedures for actually constructing Galois fields in terms of their addition and multiplication tables. In this chapter, we shall develop such procedures. Galois fields will be studied by means of two constructions: one based on the integer ring and one based on polynomial rings. Because the integer ring and the polynomial rings have many properties in common we will find that the two constructions are very similar. Later, after the constructions of Galois fields are studied, we shall prove that all finite fields can be constructed in this way.

The integer ring

The set of integers (positive, negative, and zero) forms a ring under the usual operations of addition and multiplication. This ring is conventionally denoted by the label Z. We shall study the structure of the integer ring in this section.

Applications of the discrete Fourier transform in the complex field occur throughout the subject of signal processing. Fourier transforms also exist in the Galois field GF(q) and can play an important role in the study and processing of GF(q)-valued signals, that is, of codewords. By using the Fourier transform, the ideas of coding theory can be described in a setting that is much closer to the methods of signal processing. Cyclic codes can be defined as codes whose codewords have certain specified spectral components equal to zero.

In this chapter we shall study cyclic codes in the setting of the Fourier transform. In the next chapter, we shall study decoding algorithms for cyclic codes. The most important classes of cyclic codes studied in this chapter are the Reed–Solomon codes and their subcodes, the Bose–Chaudhuri–Hocquenghem (BCH) codes.

The BCH codes form a large class of multiple-error-correcting codes that occupy a prominent place in the theory and practice of error correction. This prominence is due to at least four reasons. (1) Provided the blocklength is not excessive, there are good codes in this class (but generally, not the best of known codes). (2) Relatively simple and implementable encoding and decoding techniques are known (although, if simplicity were the only consideration, other codes may be preferable). (3) The popular and powerful class of nonbinary BCH codes known as Reed–Solomon codes has a certain strong optimality property, and these codes have a well-understood distance structure.

Majority decoding is a method of decoding by voting that is simple to implement and is extremely fast. However, only a small class of codes can be majority decoded, and usually these codes are not as good as other codes. Because code performance is usually more important than decoder simplicity, majority decoding is not important in most applications. Nevertheless, the theory of majority-decodable codes provides another well-developed view of the subject of error-control codes. The topic of majority decoding has connections with combinatorics and with the study of finite geometries.

Most known codes that are suitable for majority decoding are cyclic codes or extended cyclic codes. For these codes, the majority decoders can always be implemented as Meggitt decoders and characterized by an especially simple logic tree for examining the syndromes. Thus one can take the pragmatic view and define majority-decodable codes as those cyclic codes for which the Meggitt decoder can be put in a standard simple form. But in order to find these codes, we must travel a winding road.

Reed–Muller codes

Reed–Muller codes are a class of linear codes over GF(2) that are easy to describe and have an elegant structure. They are an example of a code that can be decoded by majority decoding. For these reasons, the Reed–Muller codes are important even though, with some exceptions, their minimum distances are not noteworthy. Each Reed–Muller code can be punctured to give one of the cyclic Reed–Muller codes that were studied in Section 6.7.

Cyclic codes are a subclass of the class of linear codes obtained by imposing an additional strong structural requirement on the codes. Because of this structure, the search for error-control codes has been most successful within the class of cyclic codes. Here the theory of Galois fields has been used as a mathematical searchlight to spot the good codes. Outside the class of cyclic codes, the theory of Galois fields casts a dimmer light. Most of what has been accomplished builds on the ideas developed for cyclic codes.

The cyclic property in itself is not important and cyclic codes do not necessarily have a large minimum distance. Cyclic codes are introduced because their structure is closely related to the strong structure of the Galois fields. This is significant because the underlying Galois-field description of a cyclic code leads to encoding and decoding procedures that are algorithmic and computationally efficient. Algorithmic techniques have important practical applications, in contrast to the tabular decoding techniques that are used for arbitrary linear codes.

This chapter gives a leisurely introduction to cyclic codes as a special class of linear codes. An alternative approach to the topic of cyclic codes, based on the Fourier transform, is given in Chapter 6.

The search for good data-transmission codes has relied, to a large extent, on the powerful and beautiful structures of modern algebra. Many important codes, based on the mathematical structures known as Galois fields, have been discovered. Further, this algebraic framework provides the necessary tools with which to design encoders and decoders. This chapter and Chapter 4 are devoted to developing those topics in algebra that are significant to the theory of data-transmission codes. The treatment is rigorous, but it is limited to material that will be used in later chapters.

Fields of characteristic two

Real numbers form a familiar set of mathematical objects that can be added, subtracted, multiplied, and divided. Similarly, complex numbers form a set of objects that can be added, subtracted, multiplied, and divided. Both of these arithmetic systems are of fundamental importance in engineering disciplines. We will need to develop other, less familiar, arithmetic systems that are useful in the study of data-transmission codes. These new arithmetic systems consist of sets together with operations on the elements of the sets. We shall call the operations “addition,” “subtraction,” “multiplication,” and “division,” although they need not be the same operations as those of elementary arithmetic.

Modern algebraic theory classifies the many arithmetic systems it studies according to their mathematical strength. Later in this chapter, these classifications will be defined formally. For now, we have the following loose definitions.

1. Abelian group. A set of mathematical objects that can be “added” and “subtracted.”

2. Ring. A set of mathematical objects that can be “added,” “subtracted,” and “multiplied.”

3. […]

To encode an infinite stream of data symbols with a block code, the datastream is broken into blocks, each of k data symbols, called datawords. The block code encodes each block of k data symbols into a block of n code symbols, called a codeword. The codewords are concatenated to form an infinite stream of code symbols.

In contrast to a block code is a trellis code. A trellis code also encodes a stream of data symbols into a stream of code symbols. A trellis code divides the datastream into blocks of length k, called dataframes, which are usually much smaller and are encoded into blocks of length n, called codeframes. In both cases, block codes and trellis codes, the datastream is broken into a sequence of blocks or frames, as is the codestream. When using a block code, a single block of the codestream depends only on a single block of the datastream, whereas when using a trellis code, a single frame of the codestream depends on multiple frames of the datastream.

The most important trellis codes are those known as convolutional codes. Convolutional codes are trellis codes that satisfy certain additional linearity and time-invariance properties. Although we introduce the general notion of a trellis code, we will be concerned mostly with the special class of convolutional codes.