In 1948 Claude Shannon published a landmark paper “A mathematical theory of communication” that signified the beginning of both information theory and coding theory. Given a communication channel which may corrupt information sent over it, Shannon identified a number called the capacity of the channel and proved that arbitrarily reliable communication is possible at any rate below the channel capacity. For example, when transmitting images of planets from deep space, it is impractical to retransmit the images. Hence if portions of the data giving the images are altered, due to noise arising in the transmission, the data may prove useless. Shannon's results guarantee that the data can be encoded before transmission so that the altered data can be decoded to the specified degree of accuracy. Examples of other communication channels include magnetic storage devices, compact discs, and any kind of electronic communication device such as cellular telephones.

The common feature of communication channels is that information is emanating from a source and is sent over the channel to a receiver at the other end. For instance in deep space communication, the message source is the satellite, the channel is outer space together with the hardware that sends and receives the data, and the receiver is the ground station on Earth. (Of course, messages travel from Earth to the satellite as well.) For the compact disc, the message is the voice, music, or data to be placed on the disc, the channel is the disc itself, and the receiver is the listener.

Coding theory originated with the 1948 publication of the paper “A mathematical theory of communication” by Claude Shannon. For the past half century, coding theory has grown into a discipline intersecting mathematics and engineering with applications to almost every area of communication such as satellite and cellular telephone transmission, compact disc recording, and data storage.

During the 50th anniversary year of Shannon's seminal paper, the two volume Handbook of Coding Theory, edited by the authors of the current text, was published by Elsevier Science. That Handbook, with contributions from 33 authors, covers a wide range of topics at the frontiers of research. As editors of the Handbook, we felt it would be appropriate to produce a textbook that could serve in part as a bridge to the Handbook. This textbook is intended to be an in-depth introduction to coding theory from both a mathematical and engineering viewpoint suitable either for the classroom or for individual study. Several of the topics are classical, while others cover current subjects that appear only in specialized books and journal publications. We hope that the presentation in this book, with its numerous examples and exercises, will serve as a lucid introduction that will enable readers to pursue some of the many themes of coding theory.

Fundamentals of Error-Correcting Codes is a largely self-contained textbook suitable for advanced undergraduate students and graduate students at any level.

In this chapter we examine the properties of the binary and ternary Golay codes, the hexacode, and the Pless symmetry codes. The Golay codes and the hexacode have similar properties while the Pless symmetry codes generalize the extended ternary Golay code. We conclude the chapter with a section showing some of the connections between these codes and lattices.

The binary Golay codes

In this section we examine in more detail the binary Golay codes of lengths 23 and 24. We have established the existence of a and a binary code in Section 1.9.1. Recall that our original construction of the extended binary Golay code used a bordered reverse circulant generator matrix, and the code was obtained by puncturing. Since then we have given different constructions of these codes, both of which were claimed to be unique codes of their length, dimension, and minimum distance. We first establish this uniqueness.

Uniqueness of the binary Golay codes

Throughout this section let C be a (possibly nonlinear) binary code of length 23 and minimum distance 7 containing M ≥ 212 codewords, one of which is 0. In order to prove the uniqueness of C, we first show it has exactly 212 codewords and is perfect. We then show it has a uniquely determined weight distribution and is in fact linear. This proof of linearity follows along the lines indicated by.

In this chapter we shall turn our attention to the implementation of encoders and decoders. Elementary encoders and decoders will be based on shift-register circuits; more advanced encoders and decoders will be based on hardware circuits and software routines for Galois-field arithmetic.

Digital logic circuits can be organized easily into shift-register circuits that mimic the cyclic shifts and polynomial arithmetic used in the description of cyclic codes. Consequently, the structure of cyclic codes is closely related to the structure of shift-register circuits. These circuits are particularly well-suited to the implementation of many encoding and decoding procedures and often take the form of filters. In fact, many algorithms for decoding simple cyclic codes can be described most easily by using the symbolism of shift-register circuits. Studying such decoders for simple codes is worthwhile for its own sake, but it is also a good way to build up the insights and techniques that will be useful for designing decoders for large codes.

Logic circuits for finite-field arithmetic

Logic circuits are easily designed to execute the arithmetic of Galois fields, especially if q is a power of 2. We shall need circuit elements to store field elements and to perform arithmetic in the finite field.

A shift register, as shown in Figure 8.1, is a string of storage devices called stages. Each stage contains one element of GF(q). The symbol contained in each storage device is displayed on an output line leaving that stage.

A profusion and variety of communication systems, which carry massive amounts of digital data between terminals and data users of many kinds, exist today. Alongside these communication systems are many different magnetic tape storage systems, and magnetic and optical disk storage systems. The received signal in any communication or recording system is always contaminated by thermal noise and, in practice, may also be contaminated by various kinds of defects, nongaussian noise, burst noise, interference, fading, dispersion, cross talk, and packet loss. The communication system or storage system must transmit its data with very high reliability in the presence of these channel impairments. Bit error rates as small as one bit error in 1012 bits (or even smaller) are routinely specified.

Primitive communication and storage systems may seek to keep bit error rates small by the simple expedient of transmitting high signal power or by repeating the message. These simplistic techniques may be adequate if the required bit error rate is not too stringent, or if the data rate is low, and if errors are caused by noise rather than by defects or interference. Such systems, however, buy performance with the least expendable resources: Power and bandwidth.

In contrast, modern communication and storage systems obtain high performance via the use of elaborate message structures with complex cross-checks built into the waveform. The advantage of these modern communication waveforms is that high data rates can be reliably transmitted while keeping the transmitted power and spectral bandwidth small.

This book is a second edition of my 1983 book Theory and Practice of Error Control Codes. Some chapters from that earlier book reappear here with minor changes. Most chapters, however, have been completely rewritten. Some old topics have been removed, and some new topics have been inserted.

During the two decades since the publication of that first edition, error-control codes have become commonplace in communications and storage equipment. Many such communication and storage devices, including the compact disk, that are now in general use could not exist, or would be much more primitive, if it were not for the subject matter covered by that first edition and repeated in this edition.

The second edition retains the original purpose of the first edition. It is a rigorous, introductory book to the subject of algebraic codes for data transmission. In fact, this phrase, “algebraic codes for data transmission,” has been chosen as the title of the second edition because it reflects a more modern perspective on the subject.

Standing alongside the class of algebraic codes that is the subject of this book is another important class of codes, the class of nonalgebraic codes for data transmission. That rapidly developing branch of the subject, which is briefly treated in Chapter 11 of this edition, deserves a book of its own; this may soon appear now that the topic is reaching a more mature form.

A digital communication system may transmit messages consisting of thousands or even millions of bits. While one can always break a long message into short blocks for encoding, in principle, a single, long block code will give better performance because it will protect against both error patterns in which the errors are clustered and error patterns in which the errors are scattered throughout the message. Therefore, there are many occasions where good codes of very long blocklength can be used.

Although short binary cyclic codes can be quite good, the known long binary cyclic codes have a small minimum distance. Codes of large blocklength with a much larger minimum distance do in principle exist, though we know very little about these more powerful codes, nor do we know how to find them. Despite more than fifty years of intense effort, codes of large blocklength and large minimum distance, both binary and nonbinary, still elude us. Even if such codes were found, it may be that their decoding would be too complex. Accordingly, the most successful constructions for codes of large blocklength for many applications combine codes of small blocklength into more elaborate structures. We call such structures composite codes. The elementary codes from which composite codes are formed can then be called basic codes.

The theory of algebraic codes draws much of its strength from the structure and properties of finite fields. In earlier chapters these properties were used to develop powerful codes and efficient decoding algorithms. The properties of those codes are tightly coupled to the properties of the fields, and the decoding algorithms use the full structure of the field to realize computational efficiency. Algebraic codes are attractive, not necessarily because of their performance, but because their structure is closely entwined with the rich structure of finite fields, a structure that underlies practical encoder and decoder implementations.

However, not every code is well-suited to the powerful algebraic decoding algorithms that are now known and not every physical channel is well-described by the mathematical structure of a finite field. Moreover, a physical channel is usually a waveform channel that transmits real or complex functions of continuous time. The methods of modulation and demodulation allow the continuous channel to be treated as a discrete channel, transmitting real or complex sequences indexed by discrete time. The sequence of codeword symbols from a finite field must be mapped into the real or complex field to form a sequence of real or complex inputs to the channel. The sequence of real or complex outputs from the channel may be mapped back to the finite field but the map takes many values to one and information is lost.

An error-control code must be judged not only by its rate and minimum distance, but also by whether a decoder can be built for it economically. Usually, there are many ways to decode a given code. A designer can choose among several decoding algorithms and their variations, and must be familiar with all of them so that the best one for a particular application can be chosen. The choice will depend not only on the code parameters, such as blocklength and minimum distance, but also on how the implementation is to be divided between hardware and software, on the required decoding speed, and even on the economics of available circuit components.

In this chapter we shall broaden our collection of decoding techniques by working in the frequency domain. Included here are techniques for decoding in the presence of both erasures and errors, and techniques for decoding beyond the designed distance of a code.

Spectral estimation in a finite field

In Section 6.6. we studied the task of decoding BCH codes, including Reed–Solomon codes, up to the designed distance. This task will now be reinterpreted from the point of view of the Fourier transform. Using the terminology of the frequency domain, we shall develop alternative decoding procedures. These procedures will take the form of spectral estimation in a finite field.

In any engineering discipline, one looks for methods of solving a given class of problems and then for methods of quantifying the performance and complexity of those solutions. Accordingly, in this chapter, we shall study performance in terms of the composition structure of block codes, and the probabilities of decoding error and of decoding failure.

Later, after satisfactory methods are in hand, one turns to questions of optimality. Are these methods the best methods, and if not, in what ways and by how much are they inferior? To answer such questions, one needs to know how good are the known codes and how good are the best possible codes. Generally, we cannot answer either of these questions satisfactorily. Although for any given rate and blocklength, not too small, the best possible code is not known, a number of bounds are known – bounds beyond which no codes can exist, and bounds within which codes are sure to exist. We shall give some such bounds on the best possible codes later in this chapter. Knowledge of these bounds can deepen our understanding of the subject.

Weight distributions of block codes

If a linear block code has a minimum distance dmin, then we know that at least one codeword of weight dmin exists. Sometimes, we are not content with this single piece of information; we wish to know how many codewords have weight dmin, and what are the weights of the other codewords.

The most powerful and important ideas of coding theory are based on the arithmetic systems of Galois fields. Since these arithmetic systems may be unfamiliar to many, we must develop a background in this branch of mathematics before we can proceed with the study of coding theory.

In this chapter, we return to the development of the structure of Galois fields begun in Chapter 2. There we introduced the definition of a field, but did not develop procedures for actually constructing Galois fields in terms of their addition and multiplication tables. In this chapter, we shall develop such procedures. Galois fields will be studied by means of two constructions: one based on the integer ring and one based on polynomial rings. Because the integer ring and the polynomial rings have many properties in common we will find that the two constructions are very similar. Later, after the constructions of Galois fields are studied, we shall prove that all finite fields can be constructed in this way.

The integer ring

The set of integers (positive, negative, and zero) forms a ring under the usual operations of addition and multiplication. This ring is conventionally denoted by the label Z. We shall study the structure of the integer ring in this section.