This chapter deals with approximation methods, mainly through the use of series. After a short discussion of approximation of known functions, we focus on approximately solving equations for unknown functions. One might wonder why anyone should bother with an approximate solution in favor of an exact solution. There are many justifications. Often physical systems are described by complicated equations with detailed exact solutions; the details of the solution may in fact obscure easy interpretation of results, rendering the solution to be of small aid in discerning trends or identifying the most important causal agents. A carefully crafted approximate solution will often yield a result that exposes the important driving physics and filters away extraneous features of the solution. Colloquially, one hopes for an approximate solution that segregates the so-called signal from the noise. This can aid the engineer greatly in building or reinforcing intuition and sometimes lead to a more efficient design and control strategy. In other cases, including those with practical importance, exact solutions are not available. In such cases, engineers often resort to numerically based approximation methods. Indeed, these methods have been established as an essential design tool; however, short of exhaustive parametric studies, it can be difficult to induce significant general insight from numerics alone. Numerical approximation is a broad topic and is not is studied here in any real detail; instead, we focus on analysis-based approximation methods. They do not work for all problems, but in those cases where they do, they are potent aids to the engineer as a predictive tool for design.

Often, though not always, approximation methods rely on some form of linearization to capture the behavior of some local nonlinearity. Such methods are useful in solving algebraic, differential, and integral equations. We begin with a consideration of Taylor series and the closely related Padé approximant. The class of methods we next consider, power series, employed already in Section 4.4 for solutions of ordinary differential equations, is formally exact in that an infinite number of terms can be obtained. Moreover, many such series can be shown to have absolute and uniform convergence properties as well as analytical estimates of errors incurred by truncation at a finite number of terms.

Linear algebra is part of the foundation of mathematics and has widespread usage in engineering. In this chapter, we specialize the linear analysis of Chapter 6 to finite-dimensional vector spaces in which the linear operator is a constant matrix. Many of the topics will be familiar, and some will likely be new. Considerable effort is spent defining terms and finding the best solution to systems of linear algebraic equations. As nearly all computational methods for solution of equations modeling physical systems rely on linear algebra, our expansive treatment is justified. Throughout the chapter, geometric interpretations are applied when appropriate. Some topics introduced in previous chapters are more fully explored, including matrices that effect rotation and reflection, projection matrices, eigenvalues and eigenvectors, and quadratic forms. New topics include a variety of matrix decompositions that are widely used in computational linear algebra. Of these the most important is the so-called singular value decomposition (SVD). We also give a matrix interpretation of two methods in wide use in engineering: (1) the least squares method and (2) the discrete Fourier transform. We close with a general strategy to find the best solution to linear algebra systems based on the SVD. In contrast to Chapter 6, we return in this chapter to Gibbs notation for vectors and matrices. Thus, matrices will be represented by uppercase bold-faced letters, such as A, and vectors by lowercase bold-faced letters, such as x.

Paradigm Problem

One of the most important problems in linear algebra lies in addressing the equation

A · x = b, (7.1)

where A is a known constant matrix, b is a known column vector, and x is an unknown column vector. We note the analog to linear differential equations with the general form of Eq. (4.1), Ly = f(x). Here the matrix A plays the role of the differential operator L, the vector x plays the rule of the function y, and the vector b plays the role of the forcing function f(x).