In this chapter we shall turn our attention to the implementation of encoders and decoders. Elementary encoders and decoders will be based on shift-register circuits; more advanced encoders and decoders will be based on hardware circuits and software routines for Galois-field arithmetic.

Digital logic circuits can be organized easily into shift-register circuits that mimic the cyclic shifts and polynomial arithmetic used in the description of cyclic codes. Consequently, the structure of cyclic codes is closely related to the structure of shift-register circuits. These circuits are particularly well-suited to the implementation of many encoding and decoding procedures and often take the form of filters. In fact, many algorithms for decoding simple cyclic codes can be described most easily by using the symbolism of shift-register circuits. Studying such decoders for simple codes is worthwhile for its own sake, but it is also a good way to build up the insights and techniques that will be useful for designing decoders for large codes.

Logic circuits for finite-field arithmetic

Logic circuits are easily designed to execute the arithmetic of Galois fields, especially if q is a power of 2. We shall need circuit elements to store field elements and to perform arithmetic in the finite field.

A shift register, as shown in Figure 8.1, is a string of storage devices called stages. Each stage contains one element of GF(q). The symbol contained in each storage device is displayed on an output line leaving that stage.

A profusion and variety of communication systems, which carry massive amounts of digital data between terminals and data users of many kinds, exist today. Alongside these communication systems are many different magnetic tape storage systems, and magnetic and optical disk storage systems. The received signal in any communication or recording system is always contaminated by thermal noise and, in practice, may also be contaminated by various kinds of defects, nongaussian noise, burst noise, interference, fading, dispersion, cross talk, and packet loss. The communication system or storage system must transmit its data with very high reliability in the presence of these channel impairments. Bit error rates as small as one bit error in 1012 bits (or even smaller) are routinely specified.

Primitive communication and storage systems may seek to keep bit error rates small by the simple expedient of transmitting high signal power or by repeating the message. These simplistic techniques may be adequate if the required bit error rate is not too stringent, or if the data rate is low, and if errors are caused by noise rather than by defects or interference. Such systems, however, buy performance with the least expendable resources: Power and bandwidth.

In contrast, modern communication and storage systems obtain high performance via the use of elaborate message structures with complex cross-checks built into the waveform. The advantage of these modern communication waveforms is that high data rates can be reliably transmitted while keeping the transmitted power and spectral bandwidth small.

This book is a second edition of my 1983 book Theory and Practice of Error Control Codes. Some chapters from that earlier book reappear here with minor changes. Most chapters, however, have been completely rewritten. Some old topics have been removed, and some new topics have been inserted.

During the two decades since the publication of that first edition, error-control codes have become commonplace in communications and storage equipment. Many such communication and storage devices, including the compact disk, that are now in general use could not exist, or would be much more primitive, if it were not for the subject matter covered by that first edition and repeated in this edition.

The second edition retains the original purpose of the first edition. It is a rigorous, introductory book to the subject of algebraic codes for data transmission. In fact, this phrase, “algebraic codes for data transmission,” has been chosen as the title of the second edition because it reflects a more modern perspective on the subject.

Standing alongside the class of algebraic codes that is the subject of this book is another important class of codes, the class of nonalgebraic codes for data transmission. That rapidly developing branch of the subject, which is briefly treated in Chapter 11 of this edition, deserves a book of its own; this may soon appear now that the topic is reaching a more mature form.

A digital communication system may transmit messages consisting of thousands or even millions of bits. While one can always break a long message into short blocks for encoding, in principle, a single, long block code will give better performance because it will protect against both error patterns in which the errors are clustered and error patterns in which the errors are scattered throughout the message. Therefore, there are many occasions where good codes of very long blocklength can be used.

Although short binary cyclic codes can be quite good, the known long binary cyclic codes have a small minimum distance. Codes of large blocklength with a much larger minimum distance do in principle exist, though we know very little about these more powerful codes, nor do we know how to find them. Despite more than fifty years of intense effort, codes of large blocklength and large minimum distance, both binary and nonbinary, still elude us. Even if such codes were found, it may be that their decoding would be too complex. Accordingly, the most successful constructions for codes of large blocklength for many applications combine codes of small blocklength into more elaborate structures. We call such structures composite codes. The elementary codes from which composite codes are formed can then be called basic codes.

The theory of algebraic codes draws much of its strength from the structure and properties of finite fields. In earlier chapters these properties were used to develop powerful codes and efficient decoding algorithms. The properties of those codes are tightly coupled to the properties of the fields, and the decoding algorithms use the full structure of the field to realize computational efficiency. Algebraic codes are attractive, not necessarily because of their performance, but because their structure is closely entwined with the rich structure of finite fields, a structure that underlies practical encoder and decoder implementations.

However, not every code is well-suited to the powerful algebraic decoding algorithms that are now known and not every physical channel is well-described by the mathematical structure of a finite field. Moreover, a physical channel is usually a waveform channel that transmits real or complex functions of continuous time. The methods of modulation and demodulation allow the continuous channel to be treated as a discrete channel, transmitting real or complex sequences indexed by discrete time. The sequence of codeword symbols from a finite field must be mapped into the real or complex field to form a sequence of real or complex inputs to the channel. The sequence of real or complex outputs from the channel may be mapped back to the finite field but the map takes many values to one and information is lost.

An error-control code must be judged not only by its rate and minimum distance, but also by whether a decoder can be built for it economically. Usually, there are many ways to decode a given code. A designer can choose among several decoding algorithms and their variations, and must be familiar with all of them so that the best one for a particular application can be chosen. The choice will depend not only on the code parameters, such as blocklength and minimum distance, but also on how the implementation is to be divided between hardware and software, on the required decoding speed, and even on the economics of available circuit components.

In this chapter we shall broaden our collection of decoding techniques by working in the frequency domain. Included here are techniques for decoding in the presence of both erasures and errors, and techniques for decoding beyond the designed distance of a code.

Spectral estimation in a finite field

In Section 6.6. we studied the task of decoding BCH codes, including Reed–Solomon codes, up to the designed distance. This task will now be reinterpreted from the point of view of the Fourier transform. Using the terminology of the frequency domain, we shall develop alternative decoding procedures. These procedures will take the form of spectral estimation in a finite field.

In any engineering discipline, one looks for methods of solving a given class of problems and then for methods of quantifying the performance and complexity of those solutions. Accordingly, in this chapter, we shall study performance in terms of the composition structure of block codes, and the probabilities of decoding error and of decoding failure.

Later, after satisfactory methods are in hand, one turns to questions of optimality. Are these methods the best methods, and if not, in what ways and by how much are they inferior? To answer such questions, one needs to know how good are the known codes and how good are the best possible codes. Generally, we cannot answer either of these questions satisfactorily. Although for any given rate and blocklength, not too small, the best possible code is not known, a number of bounds are known – bounds beyond which no codes can exist, and bounds within which codes are sure to exist. We shall give some such bounds on the best possible codes later in this chapter. Knowledge of these bounds can deepen our understanding of the subject.

Weight distributions of block codes

If a linear block code has a minimum distance dmin, then we know that at least one codeword of weight dmin exists. Sometimes, we are not content with this single piece of information; we wish to know how many codewords have weight dmin, and what are the weights of the other codewords.

The most powerful and important ideas of coding theory are based on the arithmetic systems of Galois fields. Since these arithmetic systems may be unfamiliar to many, we must develop a background in this branch of mathematics before we can proceed with the study of coding theory.

In this chapter, we return to the development of the structure of Galois fields begun in Chapter 2. There we introduced the definition of a field, but did not develop procedures for actually constructing Galois fields in terms of their addition and multiplication tables. In this chapter, we shall develop such procedures. Galois fields will be studied by means of two constructions: one based on the integer ring and one based on polynomial rings. Because the integer ring and the polynomial rings have many properties in common we will find that the two constructions are very similar. Later, after the constructions of Galois fields are studied, we shall prove that all finite fields can be constructed in this way.

The integer ring

The set of integers (positive, negative, and zero) forms a ring under the usual operations of addition and multiplication. This ring is conventionally denoted by the label Z. We shall study the structure of the integer ring in this section.

Applications of the discrete Fourier transform in the complex field occur throughout the subject of signal processing. Fourier transforms also exist in the Galois field GF(q) and can play an important role in the study and processing of GF(q)-valued signals, that is, of codewords. By using the Fourier transform, the ideas of coding theory can be described in a setting that is much closer to the methods of signal processing. Cyclic codes can be defined as codes whose codewords have certain specified spectral components equal to zero.

In this chapter we shall study cyclic codes in the setting of the Fourier transform. In the next chapter, we shall study decoding algorithms for cyclic codes. The most important classes of cyclic codes studied in this chapter are the Reed–Solomon codes and their subcodes, the Bose–Chaudhuri–Hocquenghem (BCH) codes.

The BCH codes form a large class of multiple-error-correcting codes that occupy a prominent place in the theory and practice of error correction. This prominence is due to at least four reasons. (1) Provided the blocklength is not excessive, there are good codes in this class (but generally, not the best of known codes). (2) Relatively simple and implementable encoding and decoding techniques are known (although, if simplicity were the only consideration, other codes may be preferable). (3) The popular and powerful class of nonbinary BCH codes known as Reed–Solomon codes has a certain strong optimality property, and these codes have a well-understood distance structure.