This chapter covers digital information sources in some depth. It provides intuition on the information content of a digital source and introduces the notion of redundancy. As a simple but important example, discrete memoryless sources are described. The concept of entropy is defined as a measure of the information content of a digital information source. The properties of entropy are studied, and the source-coding theorem for a discrete memoryless source is given. In the second part of the chapter, practical data compression algorithms are studied. Specifically, Huffman coding, which is an optimal data-compression algorithm when the source statistics are known, and Lempel–Ziv (LZ) and Lempel–Ziv–Welch (LZW) coding schemes, which are universal compression algorithms (not requiring the source statistics), are detailed.

The chapter is devoted to several methods of the practically important area of conservative text compression. The first eight problems concern different types of text compression: Burrows- Wheeler transform, Lempel-Ziv coding, Huffman coding and Run-length encoding. They are completely different but used in common situations. In particular there is a problem about efficient arithmetic operations on really large numbers given in their run-length encoding. In the same spirit several unrelated but interesting problems are presented about compacted automata representing all factors of special words, about pattern-matching in compressed words, about compressing suffix arrays and about the compression ratio of greedy superstrings. All these problems show how to deal with large data using compression.