This chapter is about coding information, which is the art of packaging and formatting information into meaningful codewords. Such codewords are meant to be recognized by computing machines for efficient processing or by human beings for practical understanding. The number of possible codes and corresponding codewords is infinite, just like the number of events to which information can be associated, in Shannon's meaning. This is the point where information theory will start revealing its elegance and power. We will learn that codes can be characterized by a certain efficiency, which implies that some codes are more efficient than others. This will lead us to a description of the first of Shannon's theorems, concerning source coding. As we shall see, coding is a rich subject, with many practical consequences and applications; in particular in the way we efficiently communicate information. We will first start our exploration of information coding with numbers and then with language, which conveys some background and flavor as a preparation to approach the more formal theory leading to the abstract concept of code optimality.

Coding numbers

Consider a source made of N different events. We can label the events through a set of numbers ranging from 1 to N, which constitute a basic source code. This code represents one out of N! different possibilities. In the code, each of the numbers represents a codeword.

This chapter describes the principle of compression in quantum communication channels. The underlying concept is that it is possible to convey “faithfully” a quantum message with a large number of qubits, while transmitting a compressed version of this message with a reduced number of qubits through the channel. Beyond the mere notion of fidelity, which characterizes the quality of quantum message transmission, the description brings the new concept of typicality in the space defined by all possible “quantum codewords.” The theorem of Schumacher's quantum compression states that for a qubit source with von Neumann entropy S, the message compression factor R has S − ε for the lower bound, where ε is any nonnegative parameter that can be made arbitrarily small for sufficiently long messages (hence, R ≈ S is the best possible compression factor). An original graphical and numerical illustration of the effect of Schumacher's quantum compression and the evolution of the typical quantum-codeword subspace with increasing message length is provided.

Quantum data compression and fidelity

In this chapter, we have reached the stage where it is possible to start addressing the issues that are central to information theory, namely, “How efficiently can we code information in a quantum communication channel?” both in terms of economy of means – the concept of data compression – and accuracy of transmission – the concept of message integrity or minimal data error, referred to here as fidelity.

In effect, the concept of information is obvious to anyone in life. Yet the word captures so much that we may doubt that any real definition satisfactory to a large majority of either educated or lay people may ever exist. Etymology may then help to give the word some skeleton. Information comes from the Latin informatio and the related verb informare meaning: to conceive, to explain, to sketch, to make something understood or known, to get someone knowledgeable about something. Thus, informatio is the action and art of shaping or packaging this piece of knowledge into some sensible form, hopefully complete, intelligible, and unambiguous to the recipient.

With this background in mind, we can conceive of information as taking different forms: a sensory input, an identification pattern, a game or process rule, a set of facts or instructions meant to guide choices or actions, a record for future reference, a message for immediate feedback. So information is diversified and conceptually intractable. Let us clarify here from the inception and quite frankly: a theory of information is unable to tell what information actually is or may represent in terms of objective value to any of its recipients! As we shall learn through this series of chapters, however, it is possible to measure information scientifically. The information measure does not concern value or usefulness of information, which remains the ultimate recipient's paradigm.

This chapter is concerned with the measure of information contained in qubits. This can be done only through quantum measurement, an operation that has no counterpart in the classical domain. I shall first describe in detail the case of single qubit measurements, which shows under which measurement conditions “classical” bits can be retrieved. Next, we consider the measurements of higher-order or n-qubits. Particular attention is given to the Einstein–Podolsky–Rosen (EPR) or Bell states, which, unlike other joint tensor states, are shown being entangled. The various single-qubit measurement outcomes from the EPR–Bell states illustrate an effect of causality in the information concerning the other qubit. We then focus on the technique of Bell measurement, which makes it possible to know which Bell state is being measured, yielding two classical bits as the outcome. The property of EPR–Bell state entanglement is exploited in the principle of quantum superdense coding, which makes it possible to transmit classical bits at twice the classical rate, namely through the generation and measurement of a single qubit. Another key application concerns quantum teleportation. It consists of the transmission of quantum states over arbitrary distances, by means of a common EPR–Bell state resource shared by the two channel ends. While quantum teleportation of a qubit is instantaneous, owing to the effect of quantum-state collapse, it is shown that its completion does require the communication of two classical bits, which is itself limited by the speed of light.

This chapter will take us into a world very different from all that we have seen so far concerning Shannon's information theory. As we shall see, it is a strange world made of virtual computers (universal Turing machines) and abstract axioms that can be demonstrated without mathematics merely by the force of logic, as well as relatively involved formalism. If the mere evocation of Shannon, of information theory, or of entropy may raise eyebrows in one's professional circle, how much more so that of Kolmogorov complexity! This chapter will remove some of the mystery surrounding “complexity,” also called “algorithmic entropy,” without pretending to uncover it all. Why address such a subject right here, in the middle of our description of Shannon's information theory? Because, as we shall see, algorithmic entropy and Shannon entropy meet conceptually at some point, to the extent of being asymptotically bounded, even if they come from totally uncorrelated basic assumptions! This remarkable convergence between fields must make integral part of our IT culture, even if this chapter will only provide a flavor. It may be perceived as being somewhat more difficult or demanding than the preceding chapters, but the extra investment, as we believe, is well worth it. In any case, this chapter can be revisited later on, should the reader prefer to keep focused on Shannon's theory and move directly to the next stage, without venturing into the intriguing sidetracks of algorithmic information theory.

This chapter introduces the notion of noisy quantum channels, and the different types of “quantum noise” that affect qubit messages passed through such channels. The main types of noisy channel reviewed here are the depolarizing, bit-flip, phase-flip, and bit-phase-flip channels. Then the quantum channel capacity χ is defined through the Holevo–Schumacher–Westmoreland (HSW) theorem. Such a theorem can conceptually be viewed as the elegant quantum counterpart of Shannon's (noisy) channel coding theorem, which was described in Chapter 13. Here, I shall not venture into the complex proof of the HSW theorem but only provide a background illustrating the similarity with its classical counterpart. The resemblance with the channel capacity χ and the Holevo bound, as described in Chapter 21, and with the classical mutual information H(X; Y), as described in Chapter 5, are both discussed. For advanced reference, a hint is provided as to the meaning of the still not fully explored concept of quantum coherent information. Several examples of quantum channel capacity, derived from direct applications of the HSW theorem, along with the solution of the maximization problem, are provided.

Noisy quantum channels

The notion of “noisiness” in a classical communication channel was first introduced in Chapter 12, when describing channel entropy. Such a channel can be viewed schematically as a probabilistic relation between two random sources, X for the originator, and Y for the recipient.

This chapter makes us walk a few preliminary, but decisive, steps towards quantum information theory (QIT), which will be the focus of the rest of this book. Here, we shall remain in the classical world, yet getting a hint that it is possible to think of a different world where computations may be reversible, namely, without any loss of information. One key realization through this paradigm shift is that “information is physical.” As we shall see, such a nonintuitive and striking conclusion actually results from the age-long paradox of Maxwell's demon in thermodynamics, which eventually found an elegant conclusion in Landauer's principle. This principle states that the erasure of a single bit of information requires one to provide an energy that is proportional to log 2, which, as we know from Shannon's theory, is the measure of information and also the entropy of a two-level system with a uniformly distributed source. This consideration brings up the issue of irreversible computation. Logic gates, used at the heart of the CPU in modern computers, are based on such computation irreversibility. I shall then describe the computers' von Newman's architecture, the intimate workings of the ALU processing network, and the elementary logic gates on which the ALU is based. This will also provide some basics of Boolean logic, expanding on Chapter 1, which is the key to the following logic-gate concepts.