Introduction

In this chapter we present some of the basic concepts of information theory. The situations we have in mind involve the exchange of information through transmission and storage media designed for that purpose. The information is represented in digital formats using symbols and alphabets taken in a very broad sense. The deliberate choice of the way information is represented, often referred to as coding, is an essential aspect of the theory, and for many results it will be assumed that the user is free to choose a suitable code.

We present the classical results of information theory, which were originally developed as a model of communication as it takes place over a telephone circuit or a similar connection. However, we shall pay particular attention to two-dimensional (2-D) applications, and many examples are chosen from these areas. A huge amount of information is exchanged in formats that are essentially 2-D, namely web-pages, graphic material, etc. Such forms of communication typically have an extremely complex structure. The term media is often used to indicate the structure of the message as well as the surrounding organization. Information theory is relevant for understanding the possibilities and limitation of many aspects of 2-D media and one should not expect to be able to model and analyze all aspects within a single approach.

Entropy of discrete sources

A discrete information source is a device or process that outputs symbols at discrete instances from some finite alphabet A = {x1, x2, …, xr}.

This Appendix contains the following graph tables:

A1. The spectra and characteristic polynomials of the adjacency matrix, Seidel matrix, Laplacian and signless Laplacian for connected graphs with at most 5 vertices;

A2. The eigenvalues, angles and main angles of connected graphs with 2 to 5 vertices;

A3. The spectra and characteristic polynomials of the adjacency matrix for connected graphs with 6 vertices;

A4. The spectra and characteristic polynomials of the adjacency matrix for trees with at most 9 vertices;

A5. The spectra and characteristic polynomials of the adjacency matrix for cubic graphs with at most 12 vertices.

In Tables A1 and A2, the graphs are given in the same order as in Table 1 in the Appendix of [CvDSa]. In Table A1, the spectra and coefficients for the characteristic polynomials with respect to the adjacency matrix, Laplacian, signless Laplacian and Seidel matrix, appear in consecutive lines. Table A2, which is taken from [CvPe2], was also published in [CvRS3]. This table contains, for each graph, the eigenvalues (first line), the main angles (second line) and the vertex angle sequences, with vertices labelled as in the diagrams alongside. Vertices of graphs in Table A2 are ordered in such a way that the corresponding vertex angle sequences are in lexicographical order. Since similar vertices have the same angle sequence, just one sequence is given for each orbit.

In Chapters 3 and 4 we have concentrated on the relation between the structure and spectrum of a graph. Here we discuss the connection between structure and a single eigenvalue, and for this the central notion is that of a star complement. In Section 5.1 we define star complements both geometrically and algebraically, and note their basic properties. In Section 5.2 we illustrate a technique for constructing and characterizing graphs by star complements. In Section 5.3 we use star complements to obtain sharp upper bounds on the multiplicity of an eigenvalue different from −1 or 0 in an arbitrary graph, and in a regular graph. In Section 5.4 we describe how star complements can be used to determine the graphs with least eigenvalue −2, and in Section 5.5 we investigate the role of certain star complements in generalized line graphs.

Star complements

Let G be a graph with vertex set V(G) ={1, …, n} and adjacency matrix A. Let {e1, …, en} be the standard orthonormal basis of IRn and let P be the matrix which represents the orthogonal projection of IRn onto the eigenspace ε(μ) of A with respect to {e1, …, en}. Since ε(μ) is spanned by the vectors P ej (j =1, …, n) there exists X ⊆V(G) such that the vectors P ej (j ∈ X) form a basis for ε(μ). Such a subset X of V(G) is called a star set for μ in G.

This chapter treats source coding, with a focus on the basics of lossless coding of image and graphic data. Data compression and image coding are widely used today when transmitting and storing data. Examples are transmission of images on the Internet and storage of images, audio files, and video on CDs or DVDs.

In Chapter 1 source coding of discrete sources generating independent symbols was introduced, specifically in the form of Huffman codes and arithmetic coding. In Chapter 2 some source models were considered. In this chapter we treat source coding of statistically dependent data. Furthermore, for real-world data the model is unknown, so this chapter also deals with issues concerning efficiently estimating model parameters and possibly the model order for source coding. Actually, the standard notion of a model may be abandoned altogether and the goal may be to code a single individual sequence, i.e. independently of other sequences and their statistics.

First we consider how to code the source data in relation to the models previously defined. In general terms, the model is represented by a context defining a dependency on the past data.

We shall refer to the approach as context-adaptive coding. The context may readily be defined in two or more dimensions. Doing so naturally leads to source coding of images. This forms the basis of the binary image-coding standards originally developed for fax coding and today used in a number of applications including the widely used PDF (Portable Document Format) format.

Introduction

Reed–Solomon codes are error-correcting codes defined over large alphabets. They were among the early constructions of good codes (1959), and are now one of the most important classes of error-correcting codes for many applications. At the same time these codes constitute a natural starting point for studying algebraic coding theory, i.e. methods of correcting errors by solving systems of equations.

Finite fields

To describe the codes and the decoding methods, the symbol alphabet is given a structure that allows computations similar to those used for rational numbers. The structure is that of a field. In a field there are two compositions, addition and multiplication, satisfying the usual associative and distributive rules. The compositions have neutral elements 0 and 1, every element has an additive inverse (a negative), and nonzero elements have a multiplicative inverse.

Well-known examples of fields include the rational, the real, and the complex numbers. The integers are not a field because only ±1 have multiplicative inverses. However, there are also fields with a finite number of elements, and we actually used the binary field in the previous chapter to construct binary codes. A finite field with q elements is referred to as F(q). Having the alphabet given this structure allows us to use concepts of matrices, vector spaces, and polynomials, which concepts are essential to the construction of codes and decoding algorithms.

The simplest examples of finite fields are sets of integers, [0, 1, 2, …, p − 1], with addition and multiplication modulo p.

Shannon's paper from 1948, which presented information theory in a way that already included most of the fundamental concepts, helped bring about a fundamental change in electronic communication. Today digital formats have almost entirely replaced earlier forms of signaling, and coding has become a universal feature of communication and data storage.

Information theory has developed into a sophisticated mathematical discipline, and at the same time it has almost disappeared from textbooks on digital communication and coding methods. This book is a result of the authors' desire to teach information theory to students of electrical engineering and computer science, and to demonstrate its continued relevance. We have also chosen to mix source coding and error-correcting codes, since both are components of the systems we focus on.

Early attempts to apply information-theory concepts to a broad range of subjects met with limited success. The development of the subject has mostly been fuelled by the advances in design of transmitters and receivers for digital transmission such as modern design and other related applications. However, more recently the extensive use of digitized graphics has made possible a vast range of applications, and we have chosen to draw most of our examples from this area.

The first five chapters of the book can be used for a one-semester course at the advanced-undergraduate or beginning-graduate level. Chapter 6 serves as a transition from the basic subjects to the more complex environments covered by current standards.

This book has been written primarily as an introductory text for graduate students interested in algebraic graph theory and related areas. It is also intended to be of use to mathematicians working in graph theory and combinatorics, to chemists who are interested in quantum chemistry, and in part to physicists, computer scientists and electrical engineers using the theory of graph spectra in their work. The book is almost entirely self-contained; only a little familiarity with graph theory and linear algebra is assumed.

In addition to more recent developments, the book includes an up-to-date treatment of most of the topics covered in Spectra of Graphs by D. Cvetković, M. Doob and H. Sachs [CvDSa], where spectral graph theory was characterized as follows:

In this chapter we discuss several instances of the following problem:

Given the spectrum, or some spectral characteristics of a graph, determine all graphs from a given class of graphs having the given spectrum, or the given spectral characteristics.

In some cases, the solution of such a problem can provide a characterization of a graph up to isomorphism (see Section 4.1). In other cases we can deduce structural details (see also Chapter 3). Non-isomorphic graphs with the same spectrum can arise as sporadic exceptions to characterization theorems or from general constructions. Accordingly, Section 4.2 is devoted to cospectral graphs; we include comments on their relation to the graph isomorphism problem, together with various examples and statistics. We also discuss the use of other graph invariants to strengthen distinguishing properties. In particular, in Section 4.3, we consider characterizations of graphs by eigenvalues and angles.

Spectral characterizations of certain classes of graphs

In this section we investigate graphs that are determined by their spectra. The three subsections are devoted to (i) elementary characterizations, (ii) characterizations of graphs with least eigenvalue -2, and (iii) characterizations of special types. In the case of (i), a graph is uniquely reconstructed from its spectrum, while in cases (ii) and (iii) various exceptions occur due to the existence of cospectral graphs.

Introduction

In the previous chapters we have presented the basic concepts of information theory, source coding, and channel coding. In Chapters 1−3 we have followed traditional information-theory terminology in distinguishing between sources, which produce information, and channels, which are used for transmission (or storage) of information. In many current forms of communication information passes through multiple steps of processing and assembly into composite structures. Since in such cases it can be difficult to make a distinction between sources and channels, we use the neutral term information medium to refer to structures, whether physical or conceptual, that are used for storing and delivering information. In short form the terms medium and media are used. The diverse forms of electronic media may serve as examples of the composite objects we have in mind and the range of meanings of the term. As a specific case one can think of a two-dimensional (2-D) barcode printed on an advertising display so that it can be read by a cell-phone camera and used as a way of accessing the website for the business.

In the case of highly structured composite objects we shall make no attempt to directly apply concepts like entropy or capacity. Instead we limit our applications of information theory tools to more well-defined components of such objects in digital form. The present chapter discusses how 2-D media can be described in the light of these concepts, and how the various tools can be used in such applications.

Introduction

This chapter describes fields of discrete symbols over finite alphabets A (with ∣A∣ symbols). Such fields can serve as models of graphics media or of other forms of two-dimensional (2-D) storage medium. By addressing the storage medium as a surface on which an image array may be written, the density may be increased. Besides optical storage, new storage techniques based on holography and nano-technology have been demonstrated. Two-dimensional barcodes have also been designed for coding small messages, but increasing the capacity compared with conventional barcodes, which carry information in one dimension only. The Datamatrix is an example of such a code. We focus on 2-D constrained coding for storage applications, but the models presented are more general and they may be used for other applications as well.

Two-dimensional fields with a finite constraint

The symbols are placed in a regular grid (commonly referred to as a lattice) and indexed a(i, j), where i, j are integers. The first index indicates rows and the symbols are spaced at equal intervals within the row. We usually consider rectangular grids on which the symbols are aligned in columns indicated by the second index. However, the symbols in row i + 1 can be shifted a fraction of a symbol interval relative to row i. In particular, a shift of half a symbol interval gives a hexagonal grid. The mutual dependency of symbols is often described in terms of neighbors.

This chapter is devoted to results which did not fit readily into earlier chapters. Section 8.1 is concerned with the behaviour of certain eigenvalues when a graph is modified, and with further bounds on the index of a graph. Section 8.2 deals with relations between the structure of a graph and the sign pattern of certain eigenvectors. Results from these first two sections enable us to give a general description of the connected graphs having maximal index or minimal least eigenvalue among those with a given number of vertices and edges. In Section 8.3 we discuss the reconstruction of the characteristic polynomial of a graph from the characteristic polynomials of its vertex-deleted subgraphs. In Section 8.4 we review what is known about graphs whose eigenvalues are integers.

More on graph eigenvalues

In this section we revisit two topics which have featured in previous chapters. The first topic concerns the relation between the spectrum of a graph G and the spectrum of some modification G′ of G. When the modification arises as a small structural alteration (such as the deletion or addition of an edge or vertex), the eigenvalues of G′ are generally small perturbations of those of G, and we say that G′ is a perturbation of G. In Subsection 8.1.1, we use algebraic arguments to establish some general rules which determine whether certain eigenvalues increase or decrease under particular graph perturbations.