The aim in this first chapter is to represent a message in as efficient or economical a way as possible, subject to the requirements of the devices that are to deal with it. For instance, computer memory stores information in binary form, essentially as strings of 0s and 1s. Everyone knows that English text contains far fewer letters q or j than e or t. So it is common sense to represent e and t in binary by shorter strings than are used for q and j. It is that common-sense idea that we shall elaborate in this chapter.

We do not consider at this stage any devices that corrupt messages or data. There is no error creation, so no need for error detection or correction. We are thus doing noiseless coding, and decoding. In later chapters we meet ‘noisy’ channels, that introduce occasional errors into messages, and will consider how to protect our messages against them. This will not make what we do in this chapter unnecessary, for we can employ coding and decoding for error correction as well as the noiseless coding and decoding to be met with here.

The first mathematical idea we shall consider about noiseless coding — beyond just setting up notation, though that carries ideas along with it — is that codes should be decipherable. We shall, naturally, insist on that! The mathematical expression of the idea, the Kraft inequality, limits how little code you can get away with to encode your messages. Under this limitation you still have much choice of code, and need therefore a criterion of what makes a code optimal. Now the problem is not to encode a single message, but to set up the method of encoding an indefinitely long stream, stretching into the future, of messages with similar characteristics. The likely characteristics of those prospective messages have to be specified probabilistically. That is, there is a message ‘source’ whose future output from the point of view of having to code it, is random, following a particular probability distribution or distributions which can be ascertained from the physical set-up or estimated statistically.

In this chapter we introduce Coding Theory. This topic, also known as the theory of error-correcting codes, has its origin in communication theory. Applications are concerned with several situations in which ‘coded’ messages are transmitted over a so-called noisy channel that has the effect that symbols in ‘words’ of the message are sometimes changed to other symbols of the ‘alphabet’. The system is designed in such a way that the most likely error-patterns (at the receiver end) can be recognized and corrected. In this book these practical applications are of no concern. During the development of the discipline of coding theory it turned out that several results from design theory could be used to construct ‘good’ codes. Later, theorems from coding theory contributed considerably to design theory. These connections are what interests us here and therefore the subject will be introduced as an (abstract) area of mathematics.

In coding theory one considers a. set F of q distinct symbols which is called the alphabet. In practice q is generally 2 and F = F2. In most of the theory one takes q = pr (p prime) and F = Fq. The code is called a q-ary code (binary for q = 2, ternary for q = 3).

Using the symbols of F, one forms all n-tuples, that is, Fn, and calls these n-tuples words and n the word length. If F = Fq, we shall denote the set of all words by Fnq and interpret this as n,-dimensional vector space over the field F. Sometimes we omit the index and speak of the space Fn.