This article reviews the application of various notions from the theory of dynamical systems to the analysis of numerical approximation of initial value problems over long-time intervals. Standard error estimates comparing individual trajectories are of no direct use in this context since the error constant typically grows like the exponential of the time interval under consideration.

Instead of comparing trajectories, the effect of discretization on various sets which are invariant under the evolution of the underlying differential equation is studied. Such invariant sets are crucial in determining long-time dynamics. The particular invariant sets which are studied are equilibrium points, together with their unstable manifolds and local phase portraits, periodic solutions, quasi-periodic solutions and strange attractors.

Particular attention is paid to the development of a unified theory and to the development of an existence theory for invariant sets of the underlying differential equation which may be used directly to construct an analogous existence theory (and hence a simple approximation theory) for the numerical method.

Many physically interesting problems involve propagation of free surfaces. Vortex-sheet roll-up in hydrodynamic instability, wave interactions on the ocean's free surface, the solidification problem for crystal growth and Hele-Shaw cells for pattern formation are some of the significant examples. These problems present a great challenge to physicists and applied mathematicians because the underlying problem is very singular. The physical solution is sensitive to small perturbations. Naïve discretisations may lead to numerical instabilities. Other numerical difficulties include singularity formation and possible change of topology in the moving free surfaces, and the severe time-stepping stability constraint due to the stiffness of high-order regularisation effects, such as surface tension.

This paper reviews some of the recent advances in developing stable and efficient numerical algorithms for solving free boundary-value problems arising from fluid dynamics and materials science. In particular, we will consider boundary integral methods and the level-set approach for water waves, general multi-fluid interfaces, Hele–Shaw cells, crystal growth and solidification. We will also consider the stabilising effect of surface tension and curvature regularisation. The issue of numerical stability and convergence will be discussed, and the related theoretical results for the continuum equations will be addressed. This paper is not intended to be a detailed survey and the discussion is limited by both the taste and expertise of the author.

Interior methods for optimization were widely used in the 1960s, primarily in the form of barrier methods. However, they were not seriously applied to linear programming because of the dominance of the simplex method. Barrier methods fell from favour during the 1970s for a variety of reasons, including their apparent inefficiency compared with the best available alternatives. In 1984, Karmarkar's announcement of a fast polynomial-time interior method for linear programming caused tremendous excitement in the field of optimization. A formal connection can be shown between his method and classical barrier methods, which have consequently undergone a renaissance in interest and popularity. Most papers published since 1984 have concentrated on issues of computational complexity in interior methods for linear programming. During the same period, implementations of interior methods have displayed great efficiency in solving many large linear programs of ever-increasing size. Interior methods have also been applied with notable success to nonlinear and combinatorial problems. This paper presents a self-contained survey of major themes in both classical material and recent developments related to the theory and practice of interior methods.

Parallel processing has made iterative methods an attractive alternative for solving large systems of initial value problems. Iterative methods for initial value problems have a history of more than a century, and in the works of Picard (1893) and Lindelöf (1894) they were given a firm theoretical basis. In particular, the superlinear convergence on finite intervals is included in Lindelöf (1894).

Let us think about ways to find both eigenvalues and eigenvectors of tridiagonal matrices. An important special case is the computation of singular values and singular vectors of bidiagonal matrices. The discussion is addressed both to specialists in matrix computation and to other scientists whose main interests lie elsewhere. The reason for hoping to communicate with two such diverse sets of readers at the same time is that the content of the survey, though of recent origin, is quite elementary and does not demand familiarity with much beyond triangular factorization and the Gram-Schmidt process for orthogonalizing a set of vectors. For some readers the survey will cover familiar territory but from a novel perspective. The justification for presenting these ideas is that they lead to new variations of current methods that run a lot faster while achieving greater accuracy.

The main ideas of path following by predictor–corrector and piecewise-linear methods, and their application in the direction of homotopy methods and nonlinear eigenvalue problems are reviewed. Further new applications to areas such as polynomial systems of equations, linear eigenvalue problems, interior methods for linear programming, parametric programming and complex bifurcation are surveyed. Complexity issues and available software are also discussed.

We consider the efficient evaluation of accurate radiation boundary conditions for time domain simulations of wave propagation on unbounded spatial domains. This issue has long been a primary stumbling block for the reliable solution of this important class of problems. In recent years, a number of new approaches have been introduced which have radically changed the situation. These include methods for the fast evaluation of the exact nonlocal operators in special geometries, novel sponge layers with reflectionless interfaces, and improved techniques for applying sequences of approximate conditions to higher order. For the primary isotropic, constant coefficient equations of wave theory, these new developments provide an essentially complete solution of the numerical radiation condition problem.

In this article we review the image reconstruction algorithms used in tomography. We restrict ourselves to the standard problems in the reconstruction of function from line or plane integrals as they occur in X-ray tomography, nuclear medicine, magnetic resonance imaging, and electron microscopy. Nonstandard situations, such as incomplete data, unknown orientations, local tomography, and discrete tomography are not dealt with. Nor do we treat nonlinear tomographic techniques such as impedance, ultrasound, and near-infrared imaging.

Differential algebraic equations (DAE) are special implicit ordinary differential equations (ODE)

where the partial Jacobian f′y(y, x, t) is singular for all values of its arguments.

This paper aims to give an overview and summary of numerical methods for the solution of stochastic differential equations. It covers discrete time strong and weak approximation methods that are suitable for different applications. A range of approaches and results is discussed within a unified framework. On the one hand, these methods can be interpreted as generalizing the well-developed theory on numerical analysis for deterministic ordinary differential equations. On the other hand they highlight the specific stochastic nature of the equations. In some cases these methods lead to completely new and challenging problems.

There is more to the computation of pseudospectra than the obvious algorithm of computing singular value decompositions on a grid and sending the results to a contour plotter. Other methods may be hundreds of times faster. The state of the art is reviewed, with emphasis on methods for dense matrices, and a Matlab code is given.

Since its popularization in the late 1970s, Sequential Quadratic Programming (SQP) has arguably become the most successful method for solving nonlinearly constrained optimization problems. As with most optimization methods, SQP is not a single algorithm, but rather a conceptual method from which numerous specific algorithms have evolved. Backed by a solid theoretical and computational foundation, both commercial and public-domain SQP algorithms have been developed and used to solve a remarkably large set of important practical problems. Recently large-scale versions have been devised and tested with promising results.

Many different procedures have been proposed for optimization calculations when first derivatives are not available. Further, several researchers have contributed to the subject, including some who wish to prove convergence theorems, and some who wish to make any reduction in the least calculated value of the objective function. There is not even a key idea that can be used as a foundation of a review, except for the problem itself, which is the adjustment of variables so that a function becomes least, where each value of the function is returned by a subroutine for each trial vector of variables. Therefore the paper is a collection of essays on particular strategies and algorithms, in order to consider the advantages, limitations and theory of several techniques. The subjects addressed are line search methods, the restriction of vectors of variables to discrete grids, the use of geometric simplices, conjugate direction procedures, trust region algorithms that form linear or quadratic approximations to the objective function, and simulated annealing. We study the main features of the methods themselves, instead of providing a catalogue of references to published work, because an understanding of these features may be very helpful to future research.

Among the iterative methods for solving large linear systems with a sparse (or, possibly, structured) nonsymmetric matrix, those that are based on the Lanczos process feature short recurrences for the generation of the Krylov space. This means low cost and low memory requirement. This review article introduces the reader not only to the basic forms of the Lanczos process and some of the related theory, but also describes in detail a number of solvers that are based on it, including those that are considered to be the most efficient ones. Possible breakdowns of the algorithms and ways to cure them by look-ahead are also discussed.

We survey general techniques and open problems in numerical linear algebra on parallel architectures. We first discuss basic principles of paralled processing, describing the costs of basic operations on parallel machines, including general principles for constructing efficient algorithms. We illustrate these principles using current architectures and software systems, and by showing how one would implement matrix multiplication. Then, we present direct and iterative algorithms for solving linear systems of equations, linear least squares problems, the symmetric eigenvalue problem, the nonsymmetric eigenvalue problem, and the singular value decomposition. We consider dense, band and sparse matrices.

Complexity theory of numerical analysis is the study of the number of arithmetic operations required to pass from the input to the output of a numerical problem.

We consider a system whose state is given by the solution y to a Partial Differential Equation (PDE) of evolution, and which contains control functions, denoted by v.

In this survey we discuss various approximation-theoretic problems that arise in the multilayer feedforward perceptron (MLP) model in neural networks. The MLP model is one of the more popular and practical of the many neural network models. Mathematically it is also one of the simpler models. Nonetheless the mathematics of this model is not well understood, and many of these problems are approximation-theoretic in character. Most of the research we will discuss is of very recent vintage. We will report on what has been done and on various unanswered questions. We will not be presenting practical (algorithmic) methods. We will, however, be exploring the capabilities and limitations of this model.