Numerical Methods for Unconstrained Optimization and Nonlinear Equations

Table of Contents

Preface
1. Introduction. Problems to be considered
Characteristics of 'real-world' problems
Finite-precision arithmetic and measurement of error
Exercises
2. Nonlinear Problems in One Variable. What is not possible
Newton's method for solving one equation in one unknown
Convergence of sequences of real numbers
Convergence of Newton's method
Globally convergent methods for solving one equation in one uknown
Methods when derivatives are unavailable
Minimization of a function of one variable
Exercises
3. Numerical Linear Algebra Background. Vector and matrix norms and orthogonality
Solving systems of linear equations—matrix factorizations
Errors in solving linear systems
Updating matrix factorizations
Eigenvalues and positive definiteness
Linear least squares
Exercises
4. Multivariable Calculus Background
Derivatives and multivariable models
Multivariable finite-difference derivatives
Necessary and sufficient conditions for unconstrained minimization
Exercises
5. Newton's Method for Nonlinear Equations and Unconstrained Minimization. Newton's method for systems of nonlinear equations
Local convergence of Newton's method
The Kantorovich and contractive mapping theorems
Finite-difference derivative methods for systems of nonlinear equations
Newton's method for unconstrained minimization
Finite difference derivative methods for unconstrained minimization
Exercises
6. Globally Convergent Modifications of Newton's Method. The quasi-Newton framework
Descent directions
Line searches
The model-trust region approach
Global methods for systems of nonlinear equations
Exercises
7. Stopping, Scaling, and Testing. Scaling
Stopping criteria
Testing
Exercises
8. Secant Methods for Systems of Nonlinear Equations. Broyden's method
Local convergence analysis of Broyden's method
Implementation of quasi-Newton algorithms using Broyden's update
Other secant updates for nonlinear equations
Exercises
9. Secant Methods for Unconstrained Minimization. The symmetric secant update of Powell
Symmetric positive definite secant updates
Local convergence of positive definite secant methods
Implementation of quasi-Newton algorithms using the positive definite secant update
Another convergence result for the positive definite secant method
Other secant updates for unconstrained minimization
Exercises
10. Nonlinear Least Squares. The nonlinear least-squares problem
Gauss-Newton-type methods
Full Newton-type methods
Other considerations in solving nonlinear least-squares problems
Exercises
11. Methods for Problems with Special Structure. The sparse finite-difference Newton method
Sparse secant methods
Deriving least-change secant updates
Analyzing least-change secant methods
Exercises
Appendix A. A Modular System of Algorithms for Unconstrained Minimization and Nonlinear Equations (by Robert Schnabel)
Appendix B. Test Problems (by Robert Schnabel)
References
Author Index
Subject Index.