This chapter deals with approximation methods, mainly through the use of series. After a short discussion of approximation of known functions, we focus on approximately solving equations for unknown functions. One might wonder why anyone should bother with an approximate solution in favor of an exact solution. There are many justifications. Often physical systems are described by complicated equations with detailed exact solutions; the details of the solution may in fact obscure easy interpretation of results, rendering the solution to be of small aid in discerning trends or identifying the most important causal agents. A carefully crafted approximate solution will often yield a result that exposes the important driving physics and filters away extraneous features of the solution. Colloquially, one hopes for an approximate solution that segregates the so-called signal from the noise. This can aid the engineer greatly in building or reinforcing intuition and sometimes lead to a more efficient design and control strategy. In other cases, including those with practical importance, exact solutions are not available. In such cases, engineers often resort to numerically based approximation methods. Indeed, these methods have been established as an essential design tool; however, short of exhaustive parametric studies, it can be difficult to induce significant general insight from numerics alone. Numerical approximation is a broad topic and is not is studied here in any real detail; instead, we focus on analysis-based approximation methods. They do not work for all problems, but in those cases where they do, they are potent aids to the engineer as a predictive tool for design.

Often, though not always, approximation methods rely on some form of linearization to capture the behavior of some local nonlinearity. Such methods are useful in solving algebraic, differential, and integral equations. We begin with a consideration of Taylor series and the closely related Padé approximant. The class of methods we next consider, power series, employed already in Section 4.4 for solutions of ordinary differential equations, is formally exact in that an infinite number of terms can be obtained. Moreover, many such series can be shown to have absolute and uniform convergence properties as well as analytical estimates of errors incurred by truncation at a finite number of terms.

Linear algebra is part of the foundation of mathematics and has widespread usage in engineering. In this chapter, we specialize the linear analysis of Chapter 6 to finite-dimensional vector spaces in which the linear operator is a constant matrix. Many of the topics will be familiar, and some will likely be new. Considerable effort is spent defining terms and finding the best solution to systems of linear algebraic equations. As nearly all computational methods for solution of equations modeling physical systems rely on linear algebra, our expansive treatment is justified. Throughout the chapter, geometric interpretations are applied when appropriate. Some topics introduced in previous chapters are more fully explored, including matrices that effect rotation and reflection, projection matrices, eigenvalues and eigenvectors, and quadratic forms. New topics include a variety of matrix decompositions that are widely used in computational linear algebra. Of these the most important is the so-called singular value decomposition (SVD). We also give a matrix interpretation of two methods in wide use in engineering: (1) the least squares method and (2) the discrete Fourier transform. We close with a general strategy to find the best solution to linear algebra systems based on the SVD. In contrast to Chapter 6, we return in this chapter to Gibbs notation for vectors and matrices. Thus, matrices will be represented by uppercase bold-faced letters, such as A, and vectors by lowercase bold-faced letters, such as x.

Paradigm Problem

One of the most important problems in linear algebra lies in addressing the equation

A · x = b, (7.1)

where A is a known constant matrix, b is a known column vector, and x is an unknown column vector. We note the analog to linear differential equations with the general form of Eq. (4.1), Ly = f(x). Here the matrix A plays the role of the differential operator L, the vector x plays the rule of the function y, and the vector b plays the role of the forcing function f(x).

We consider in this chapter linear ordinary differential equations. We have already introduced first-order linear differential equations in Chapter 3. Here we are mainly concerned with equations that are second order or higher in a single dependent variable. We review several topics that are commonly covered in undergraduate mathematics, including complementary functions, particular solutions, the superposition principle, Sturm-Liouville equations, and resonance of a sinusoidally forced linear oscillator. We close with a discussion of linear difference equations. Strictly speaking, these are not differential equations, but they certainly arise in many discretized forms of linear differential equations, and their solution has analog to the solution of differential equations. Intrinsic in much of our discussion will be the notion of oscillation at a variety of frequencies. This lays the foundation of the exercise of seeking repetitive patterns, a topic of relevance in engineering. The chapter also introduces the important concept of representation of a function by infinite trigonometric and nontrigonometric Fourier series, as well as projection of a function onto a basis composed of a finite Fourier series. This motivates important abstractions that will be considered in detail in the later Chapter 6. Advanced topics such as Green's functions for particular solutions and discrete/continuous spectra are included as well. All of these topics have relevance in the wide assortment of engineering systems that are well modeled by linear systems. We will provide some focus on linear oscillators, such as found in mass-spring systems. Analogs abound and are too numerous to be delineated.

Linearity and Linear Independence

An ordinary differential equation can be written in the form

Ly = f(x),

where L is a known operator, y is an unknown function, and f(x) is a known function of the independent variable x. The equation is said to be homogeneous if f(x) = 0, giving, then,

Ly = 0.

This is the most common usage for the term homogeneous. The operator L can involve a combination of derivatives d/dx, d2/dx2, and so on.

In this chapter, we consider the time-like evolution of sets of state variables, a subject often called dynamical systems. We begin with a brief consideration of discrete dynamical systems known as iterated maps, which are a nonlinear extension of the difference equations posed in Section 4.9 or the finite difference method posed in Example 4.24. We also show some of the striking geometries that result from iterated maps known as fractals. Generally, however, we are concerned with systems that can be described by sets of ordinary differential equations, both linear and nonlinear. In that solution to nonlinear differential equations is usually done in discrete form, there is a strong connection to iterated maps. And we shall see that discrete and continuous dynamical systems share many features.

This final chapter appropriately coalesces many topics studied earlier: discrete systems, ordinary differential equations, perturbation analysis, linear algebra, and geometry. Its use in modeling physical engineering systems is widespread and comes in two main classes: (1) systems that are modeled by coupled ordinary differential equations and (2) systems that are modeled by one or more partial differential equations. Systems of the first type often arise in time-dependent problems with spatial homogeneity in which discrete entities interact. Systems of the second type generally involve time-evolution of interacting systems with spatial in homogeneity and reduce to large systems of ordinary differential equations following either (1) discretization of one or more independent variables or (2) projection of the dependent variable onto a finite basis in function space. The chapter closes with examples that draw problems from the domain of partial differential equations into the domain of ordinary differential equations and thus demonstrates how the methods of this book can be brought to bear on this critical area of mathematics of engineering systems.

Iterated Maps

Similar to the nonlinear difference equation introduced by Eq. (4.610), a map fn : ℝN → ℝN can be iterated to give a dynamical system of the form

This dynamical system is obviously not a differential equation.