Types of error

The term “error” is going to appear throughout this book in different contexts. The varieties of error we will be concerned with are:

• Experimental error. We may wish to calculate some function y(x1, …, xn), where the quantities xi are measured. Any such measurement has associated errors, and they will affect the accuracy of the calculated y.

• Roundoff error. Even if x were measured exactly, odds are it cannot be represented exactly in a digital computer. Consider π, which cannot be represented exactly in decimal form. We can write π ≈ 3.1416, by rounding the exact number to fit 5 decimal figures. Some roundoff error occurs in almost every calculation with real numbers, and controlling how strongly it impacts the final result of a calculation is always an important numerical consideration.

• Approximation error. Sometimes we want one thing but calculate another, intentionally, because the other is easier or has more favorable properties. For example, one might choose to represent a complicated function by its Taylor series. When substituting expressions that are not mathematically identical we introduce approximation error.

Experimental error is largely outside the scope of numerical treatment, and we'll assume here, with few exceptions, that it's just something we have to live with. Experimental error plays an important role in data fitting, which will be described at length in Chapter 8.

Data fitting can be viewed as a generalization of polynomial interpolation to the case where we have more data than is needed to construct a polynomial of specified degree.

C.F. Gauss claims to have first developed solutions to the least squares problem, and both Gaussian elimination and the Gauss-Seidel iterative method were developed to solve these problems [52, 79]. In fact, interest in least squares by Galileo predates Gauss by over 200 years – a comprehensive history and analysis is given by Harter [97]. In addition to Gauss’ contributions, the Jacobi iterative method [118] and the Cholesky decomposition method [13] were developed to solve least squares problems. Clearly, the least squares problem was (and continues to be) a problem of considerable importance. All these methods were applied to the normal equations, which represent an overdetermined system as a square and symmetric positive definite matrix. Despite the astounding historical importance of the normal equations, the argument will be made that you should not ever use them. Extensions of least squares to nonlinear problems, and linear problems with normal error, are described.

Least squares refers to a best fit in the L2 norm, and this is by far the most commonly used norm. However, other norms are important for certain applications. Covariance-weighting leads to minimization in the Mahalanobis norm. L1 is commonly used in financial modeling, and L∞ may be most suitable when the underlying error distribution is uniform, versus normal.

The ab initio determination of molecular structure is an example that will use:

1. • singular value decomposition (Section 3.4) in particular the Lowdin decomposition (8.13) encountered in data fitting;

2. • the QR method for eigenvalues and eigenvectors (Section 3.3);

3. • interpolation (Chapter 5);

4. • fixed point iteration (Chapter 6) with stabilization by damping (Section 8.4);

5. • data fitting (Chapter 8), which involves solutions of linear systems (Chapter 2);

6. • integration: generally (Chapter 9), and Gaussian integration in particular (Section 9.4, Problem 9.6, which requires numerical root finding (Section 6.6) to set up), and the use of recursions, e.g., (10.35); and

7. • optimization with the variable metric method (Section 7.4).

In addition, concerns about numerical error (Chapter 1) are omnipresent. We will encounter the error function (Problems 5.7 and 9.7), and lots of Gaussian integrals (equations (8.10), (11.8), and (11.29)). We will use a paradigm called the variational principle that is essentially what motivated the conjugate gradient algorithm (Section 4.1).

The chemical physics problem is described in Section 12.1, and the Hartree–Fock–Roothaan equations are derived. These determine the energy of a particular molecular configuration. The HFR equations rely on a number of integrals of Gaussian functions that are introduced in their simplest form in Section 12.2. With these simple formulas we determine the energy of the H2 molecule for prescribed geometry, demonstrating in Section 12.3 reasonable results compared to far more elaborate calculations. In Section 12.4 the more complex Gaussian integrals are addressed.