Let V be a finite-dimensional real or complex vector space with a given norm ∥·∥. The closed ball of radius ∊ about x ∈ V is B∊(x) = {y ∈ V :∥ y − x ∥≤ ∊}; the corresponding open ball is B∊ (x) = {y ∈ V :∥ y − x ∥ < ∊}. A set S ⊆ V is open if, for each x ∈ S, there is an ∊ > 0 such that B∊(x) ⊆ S. A set S ⊆ V is closed if the complement of S in V is open. A set S ⊆ V is bounded if there is an r > 0 such that S ⊆ Br(0). Equivalently, a set S ⊆ V is closed if and only if the limit of any convergent (with respect to ∥·∥) sequence of points in S is itself in S; S is bounded if and only if it is contained in some ball of finite radius. A set S ⊆ V is compact if it is both closed and bounded.

For a given set S ⊆ V and a given real-valued function f defined on S, infx∈S f(x) and supx∈S f(x) need not be finite, and even if they are, there may or may not be points xmin and xmax in S such that f(xmin) = infx∈S f(x) and f(xmax) = supx∈S f(x), that is, f need not attain a maximum or minimum value on S.

Introduction

In this initial chapter we summarize many useful concepts and facts, some of which provide a foundation for the material in the rest of the book. Some of this material is included in a typical elementary course in linear algebra, but we also include additional useful items, even though they do not arise in our subsequent exposition. The reader may use this chapter as a reviewbefore beginning the main part of the book in Chapter 1; subsequently, it can serve as a convenient reference for notation and definitions that are encountered in later chapters. We assume that the reader is familiar with the basic concepts of linear algebra and with mechanical aspects of matrix manipulations, such as matrix multiplication and addition.

Vector spaces

A finite dimensional vector space is the fundamental setting for matrix analysis.

Scalar field. Underlying a vector space is its field, or set of scalars. For our purposes, that underlying field is typically the real numbers R or the complex numbers C (see Appendix A), but it could be the rational numbers, the integers modulo a specified prime number, or some other field. When the field is unspecified, we denote it by the symbol F. To qualify as a field, a set must be closed under two binary operations: “addition” and “multiplication.” Both operations must be associative and commutative, and each must have an identity element in the set; inverses must exist in the set for all elements under addition and for all elements except the additive identity under multiplication; multiplication must be distributive over addition.

The basic structure of the first edition has been preserved in the second because it remains congruent with the goal of writing “a book that would be a useful modern treatment of a broad range of topics… [that] may be used as an undergraduate or graduate text and as a self-contained reference for a variety of audiences.” The quotation is from the Preface to the First Edition, whose declaration of goals for the work remains unchanged.

What is different in the second edition?

The core role of canonical forms has been expanded as a unifying element in understanding similarity (complex, real, and simultaneous), unitary equivalence, unitary similarity, congruence, *congruence, unitary congruence, triangular equivalence, and other equivalence relations. More attention is paid to cases of equality in the many inequalities considered in the book. Block matrices are a ubiquitous feature of the exposition in the new edition.

Learning mathematics has never been a spectator sport, so the new edition continues to emphasize the value of exercises and problems for the active reader. Numerous 2-by-2 examples illustrate concepts throughout the book. Problem threads (some span several chapters) develop special topics as the foundation for them evolves in the text. For example, there are threads involving the adjugate matrix, the compound matrix, finite-dimensional quantum systems, the Loewner ellipsoid and the Loewner-John matrix, and normalizable matrices; see the index for page references for these threads. The first edition had about 690 problems; the second edition has more than 1,100. Many problems have hints; they may be found in an appendix that appears just before the index.

Introduction

The eigenvalues of a diagonal matrix are very easy to locate, and the eigenvalues of a matrix are continuous functions of the entries, so it is natural to ask whether one can say anything useful about the eigenvalues of a matrix that is “nearly diagonal” in the sense that its off-diagonal elements are dominated in some way by the main diagonal entries. Such matrices arise in practice: Large systems of linear equations resulting from numerical discretization of boundary value problems for elliptic partial differential equations are typically of this form.

In differential equations problems involving the long-term stability of an oscillating system, it can be important to know that all of the eigenvalues of a given matrix are in the left half-plane. In statistics or numerical analysis, one may want to show that all the eigenvalues of a given Hermitian matrix are positive. In this chapter, we describe simple criteria that are sufficient to ensure that the eigenvalues of a given matrix are included in sets such as a given half plane, disc, or ray.

All the eigenvalues of a matrix A are located in a disc in the complex plane centered at the origin that has radius ∥∣A∣∥, in which ∥∣·∣∥ is any matrix norm. Are there other, smaller, sets that are readily determined and either include or exclude the eigenvalues? We identify several such sets in this chapter.

Introduction

In Chapter 1, we made an initial study of similarity of A ∈ Mn via a general nonsingular matrix S, that is, the transformation A → S−1AS. For certain very special nonsingular matrices, called unitary matrices, the inverse of S has a simple form: S−1 = S*. Similarity via a unitary matrix U, A → U*AU, is not only conceptually simpler than general similarity (the conjugate transpose is much easier to compute than the inverse), but it also has superior stability properties in numerical computations. A fundamental property of unitary similarity is that every A ∈ Mn is unitarily similar to an upper triangular matrix whose diagonal entries are the eigenvalues of A. This triangular form can be further refined under general similarity; we study the latter in Chapter 3.

The transformation A → S*AS, in which S is nonsingular but not necessarily unitary, is called *congruence; we study it in Chapter 4. Notice that similarity by a unitary matrix is both a similarity and a *congruence.

For A ∈ Mn, m, the transformation A →U AV, in which U ∈ Mm and V ∈ Mnare both unitary, is called unitary equivalence. The upper triangular form achievable under unitary similarity can be greatly refined under unitary equivalence and generalized to rectangular matrices: Every A ∈ Mn, m is unitarily equivalent to a nonnegative diagonal matrix whose diagonal entries (the singular values of A) are of great importance.

Linear algebra and matrix theory have long been fundamental tools in mathematical disciplines as well as fertile fields for research in their own right. In this book, and in the companion volume, Topics in Matrix Analysis, we present classical and recent results of matrix analysis that have proved to be important to applied mathematics. The book may be used as an undergraduate or graduate text and as a self-contained reference for a variety of audiences. We assume background equivalent to a one-semester elementary linear algebra course and knowledge of rudimentary analytical concepts. We begin with the notions of eigenvalues and eigenvectors; no prior knowledge of these concepts is assumed.

Facts about matrices, beyond those found in an elementary linear algebra course, are necessary to understand virtually any area of mathematical science, whether it be differential equations; probability and statistics; optimization; or applications in theoretical and applied economics, the engineering disciplines, or operations research, to name only a few. But until recently, much of the necessary material has occurred sporadically (or not at all) in the undergraduate and graduate curricula. As interest in applied mathematics has grown and more courses have been devoted to advanced matrix theory, the need for a text offering a broad selection of topics has become more apparent, as has the need for a modern reference on the subject.

There are several well-loved classics in matrix theory, but they are not well suited for general classroom use, nor for systematic individual study. A lack of problems, applications, and motivation; an inadequate index; and a dated approach are among the difficulties confronting readers of some traditional references.