WHY YOU SHOULD READ THIS BOOK

The technology of communication and computing advanced at a breathtaking pace in the 20th century, especially in the second half. A significant part of this advance in communication began some 60 years ago when Shannon published his seminal paper “A Mathematical Theory of Communication.” In that paper Shannon framed and posed a fundamental question: how can we efficiently and reliably transmit information? Shannon also gave a basic answer: coding can do it. Since that time the problem of finding practical coding schemes that approach the fundamental limits established by Shannon has been at the heart of information theory and communications. Recently, significant advances have taken place that bring us close to answering this question. Perhaps, at least in a practical sense, the question has been answered. This book is about that answer.

The advance came with a fundamental paradigm shift in the area of coding that took place in the early 1990s. In Modern Coding Theory, codes are viewed as large complex systems described by random sparse graphical models, and encoding as well as decoding are accomplished by efficient local algorithms. The local interactions of the codebits are simple but the overall code is nevertheless complex (and so sufficiently powerful to allow reliable communication) because of the large number of interactions. The idea of random codes is in the spirit of Shannon's original formulation. What is new is the sparseness of the description and the local nature of the algorithms.

Density evolution plays a fundamental role in the analysis of iterative systems; it is also a valuable tool in the design of such systems. Actual computation of density evolution for low-density parity-check (LDPC) codes requires an algorithmic implementation. Finding an efficient such implementation is a challenging problem. In this section we show how this can be done.

When implementing density evolution one may or may not assume symmetry of the densities. Working directly in the space of symmetric distributions yields the most efficient implementations. The interaction of the symmetry with the practical constraints of finite support typically leads to optimistic predictions of the thresh-old. On the other hand, sometimes one is interested specifically in consequences of non-symmetry. Moreover, by allowing densities to be non-symmetric one can compute density evolution for a message-passing decoder which corresponds to a quantized version of belief propagation. Since belief propagation is optimal, thresholds computed this way are lower bounds on the true belief propagation threshold. We will not assume strict symmetry in general, but we will assume that densities are “nearly” symmetric.

We use the notation * to denote standard convolution over ℝ, ℤ, or ℤ/N ℤ – the ring of integers modulo N. Variable-node domain convolution, which we have denoted by *, is standard convolution but we shall use ⊛ to emphasize computational aspects.

In practical LDPC design one is invariably interested in achieving the best possible performance. Although we have used the framework of irregular low-density parity-check (LDPC) ensembles throughout this book, the notion of “irregularity” we have employed is not the most general possible and the best performing structures are not properly captured in that framework.

We start by presenting in Section 7.1 a generalization of irregular LDPC ensembles called multi-edge-type LDPC ensembles. In principle, any of the ensembles suggested to date can be represented in this form and we demonstrate this discussing a few examples.

In Section 7.2 we review the theory surrounding these ensembles. It is largely the same as for the standard irregular case, which is why we have avoided using this more complex notion so far. The multi-edge-type generalization enjoys several advantages. With multi-edge-type LDPC codes one can achieve better performance at lower complexity. The generalization is especially useful under extreme conditions where standard irregular LDPC codes do not fare so well. Examples of these conditions include very low code rates, high rate codes that target very low bit error rates, and codes used in conjunction with turbo equalization schemes.

We discuss in Section 7.3 an alternative and complementary way of describing structured ensembles. These ensembles are the result of lifting a very small graph to a large graph by first replicating the structure of the small graph a large number of times and then by choosing the connections between these copies in a random way.

Asserting a specific property about an individual code is typically a hard task. To the contrary, it is often easy to show that most codes in a properly chosen ensemble possess this property. In the realm of classical coding theory an important such instance is the minimum distance of a code. For Elias's generator ensemble G a few lines suffice to show that with high probability an element chosen uniformly at random has a relative minimum distance of at least δGV, where δGV is the Gilbert-Varshamov distance discussed on page 7. But as mentioned on page 33, it is known that the corresponding decision problem – whether a given code has relative minimum distance at least δGV – is NP-complete.

We encounter a similar situation in the realm of message-passing decoding. The whole analysis rests on the investigation of ensembles of codes and, therefore, concentration theorems which assert that most codes in this ensemble behave close to the ensemble average are at the center of the theory. There is one big difference though which makes concentrations theorems invaluable for message-passing decoding, whereas in the classical setting they only play a marginal role. The main obstacle which we encounter in classical coding is that a random code (in G) is unlikely to have an efficient decoding algorithm, and, therefore, a random code is unlikely to be of much practical value. In the message-passing world the choice of the ensemble (e.g., LDPC, turbo, multi-edge, etc.) guarantees that every element can be decoded with equal ease.

Modulation refers to the representation of digital information in terms of analog waveforms that can be transmitted over physical channels. A simple example is depicted in Figure 2.1, where a sequence of bits is translated into a waveform. The original information may be in the form of bits taking the values 0 and 1. These bits are translated into symbols using a bit-to-symbol map, which in this case could be as simple as mapping the bit 0 to the symbol +1, and the bit 1 to the symbol −1. These symbols are then mapped to an analog waveform by multiplying with translates of a transmit waveform (a rectangular pulse in the example shown): this is an example of linear modulation, to be discussed in detail in Section 2.5. For the bit-to-symbol map just described, the bitstream encoded into the analog waveform shown in Figure 2.1 is 01100010100.

While a rectangular timelimited transmit waveform is shown in the example of Figure 2.1, in practice, the analog waveforms employed for modulation are often constrained in the frequency domain. Such constraints arise either from the physical characteristics of the communication medium, or from external factors such as government regulation of spectrum usage. Thus, we typically classify channels, and the signals transmitted over them, in terms of the frequency bands they occupy. In this chapter, we discuss some important modulation techniques, after first reviewing some basic concepts regarding frequency domain characterization of signals and systems.

Freedom from wires is an attractive, and often indispensable, feature for many communication applications. Examples of wireless communication include radio and television broadcast, point-to-point microwave links, cellular communications, and wireless local area networks (WLANs). Increasing integration of transceiver functionality using DSP-centric design has driven down implementation costs, and has led to explosive growth in consumer and enterprise applications of wireless, especially cellular telephony and WLANs.

While the focus of this chapter is on wireless link design, we comment briefly on some system design issues in this introductory section. In terms of system design, a key difference between wireless and wireline communication is that wireless is a broadcast medium. That is, users “close enough” to each other can “hear,” and potentially interfere with, each other. Thus, appropriate resource sharing mechanisms must be put in place if multiple users are to co-exist in a particular frequency band. The wireless channel can be shared among multiple users using several different approaches. One possibility is to eliminate potential interference by assigning different frequency channels to different users; this is termed frequency division multiple access (FDMA). Similarly, we can assign different time slots to different users; this is termed time division multiple access (TDMA). If we use orthogonal multiple access such as FDMA or TDMA, then we can focus on single-user wireless link design. However, there are also nonorthogonal forms of multiple access, in which different users can signal at the same time over the same frequency band.

In this chapter, we provide an introduction to some commonly used channel coding techniques. The key idea of channel coding is to introduce redundancy in the transmitted signal so as to enable recovery from channel impairments such as errors and erasures. We know from the previous chapter that, for any given set of channel conditions, there exists a Shannon capacity, or maximum rate of reliable transmission. Such Shannon-theoretic limits provide the ultimate benchmark for channel code design. A large number of error control techniques are available to the modern communication system designer, and in this chapter, we provide a glimpse of a small subset of these. Our emphasis is on convolutional codes, which have been a workhorse of communication link design for many decades, and turbo-like codes, which have revolutionized communication systems by enabling implementable designs that approach Shannon capacity for a variety of channel models.

Map of this chapter We begin in Section 7.1 with binary convolutional codes. We introduce the trellis representation and the Viterbi algorithm for ML decoding, and develop performance analysis techniques. The structure of the memory introduced by a convolutional code is similar to that introduced by a dispersive channel. Thus, the techniques are similar to (but simpler than) those developed for MLSE for channel equalization in Chapter 5. Concatenation of convolutional codes leads to turbo codes, which are iteratively decoded by exchanging soft information between the component convolutional decoders.

We define communication as information transfer between different points in space or time, where the term information is loosely employed to cover standard formats that we are all familiar with, such as voice, audio, video, data files, web pages, etc. Examples of communication between two points in space include a telephone conversation, accessing an Internet website from our home or office computer, or tuning in to a TV or radio station. Examples of communication between two points in time include accessing a storage device, such as a record, CD, DVD, or hard drive. In the preceding examples, the information transferred is directly available for human consumption. However, there are many other communication systems, which we do not directly experience, but which form a crucial part of the infrastructure that we rely upon in our daily lives. Examples include high-speed packet transfer between routers on the Internet, inter- and intra-chip communication in integrated circuits, the connections between computers and computer peripherals (such as keyboards and printers), and control signals in communication networks.

In digital communication, the information being transferred is represented in digital form, most commonly as binary digits, or bits. This is in contrast to analog information, which takes on a continuum of values. Most communication systems used for transferring information today are either digital, or are being converted from analog to digital.

The field of digital communication has evolved rapidly in the past few decades, with commercial applications proliferating in wireline communication networks (e.g., digital subscriber loop, cable, fiber optics), wireless communication (e.g., cell phones and wireless local area networks), and storage media (e.g., compact discs, hard drives). The typical undergraduate and graduate student is drawn to the field because of these applications, but is often intimidated by the mathematical background necessary to understand communication theory. A good lecturer in digital communication alleviates this fear by means of examples, and covers only the concepts that directly impact the applications being studied. The purpose of this text is to provide such a lecture style exposition to provide an accessible, yet rigorous, introduction to the subject of digital communication. This book is also suitable for self-study by practitioners who wish to brush up on fundamental concepts.

The book can be used as a basis for one course, or a two course sequence, in digital communication. The following topics are covered: complex baseband representation of signals and noise (and its relation to modern transceiver implementation); modulation (emphasizing linear modulation); demodulation (starting from detection theory basics); communication over dispersive channels, including equalization and multicarrier modulation; computation of performance benchmarks using information theory; basics of modern coding strategies (including convolutional codes and turbo-like codes); and introduction to wireless communication. The choice of material reflects my personal bias, but the concepts covered represent a large subset of the tricks of the trade.

In Chapter 3, we established a framework for demodulation over AWGN channels under the assumption that the receiver knows and can reproduce the noiseless received signal for each possible transmitted signal. These provide “templates” against which we can compare the noisy received signal (using correlation), and thereby make inferences about the likelihood of each possible transmitted signal. Before the receiver can arrive at these templates, however, it must estimate unknown parameters such as the amplitude, frequency and phase shifts induced by the channel. We discuss synchronization techniques for obtaining such estimates in this chapter. Alternatively, the receiver might fold in implicit estimation of these parameters, or average over the possible values taken by these parameters, in the design of the demodulator. Noncoherent demodulation, discussed in detail in this chapter, is an example of such an approach to dealing with unknown channel phase. Noncoherent communication is employed when carrier synchronization is not available (e.g., because of considerations of implementation cost or complexity, or because the channel induces a difficult-to-track time-varying phase, as for wireless mobile channels). Noncoherent processing is also an important component of many synchronization algorithms (e.g., for timing synchronization, which often takes place prior to carrier synchronization).

Since there are many variations in individual (and typically proprietary) implementations of synchronization and demodulation algorithms, the focus here is on developing basic principles, and on providing some simple examples of how these principles might be applied.

In this chapter, we develop channel equalization techniques for handling the intersymbol interference (ISI) incurred by a linearly modulated signal that goes through a dispersive channel. The principles behind these techniques also apply to dealing with interference from other users, which, depending on the application, may be referred to as co-channel interference, multiple-access interference, multiuser interference, or crosstalk. Indeed, we revisit some of these techniques in Chapter 8 when we briefly discuss multiuser detection. More generally, there is great commonality between receiver techniques for efficiently accounting for memory, whether it is introduced by nature, as considered in this chapter, or by design, as in the channel coding schemes considered in Chapter 7. Thus, the optimum receiver for ISI channels (in which the received signal is a convolution of the transmitted signal with the channel impulse response) uses the same Viterbi algorithm as the optimum receiver for convolutional codes (in which the encoded data are a convolution of the information stream with the code “impulse response”) in Chapter 7.

The techniques developed in this chapter apply to single-carrier systems in which data are sent using linear modulation. An alternative technique for handling dispersive channels, discussed in Chapter 8, is the use of multicarrier modulation, or orthogonal frequency division multiplexing (OFDM). Roughly speaking, OFDM, or multicarrier modulation, transforms a system with memory into a memoryless system in the frequency domain, by decomposing the channel into parallel narrowband subchannels, each of which sees a scalar channel gain.

Information theory (often termed Shannon theory in honor of its founder, Claude Shannon) provides fundamental benchmarks against which a communication system design can be compared. Given a channel model and transmission constraints (e.g., on power), information theory enables us to compute, at least in principle, the highest rate at which reliable communication over the channel is possible. This rate is called the channel capacity.

Once channel capacity is computed for a particular set of system parameters, it is the task of the communication link designer to devise coding and modulation strategies that approach this capacity. After 50 years of effort since Shannon's seminal work, it is now safe to say that this goal has been accomplished for some of the most common channel models. The proofs of the fundamental theorems of information theory indicate that Shannon limits can be achieved by random code constructions using very large block lengths. While this appeared to be computationally infeasible in terms of both encoding and decoding, the invention of turbo codes by Berrou et al. in 1993 provided implementable mechanisms for achieving just this. Turbo codes are random-looking codes obtained from easy-to-encode convolutional codes, which can be decoded efficiently using iterative decoding techniques instead of ML decoding (which is computationally infeasible for such constructions). Since then, a host of “turbo-like” coded modulation strategies have been proposed, including rediscovery of the low density parity check (LDPC) codes invented by Gallager in the 1960s.