Starting from this chapter, we shall first describe various preconditioning techniques that are based on manipulation of a given matrix. These are classified into four categories: direct matrix extraction (or operator splitting type), inverse approximation (or inverse operator splitting type), multilevel Schur complements and multi-level operator splitting (multilevel methods).

The acoustic scattering modelling provides a typical example of utilizing a boundary element method to derive a dense matrix application as shown in Chapter 1. Such a physical problem is only a simple model of the full wave equations or the Maxell equations from electromagnetism. The challenges are:

1. (i) the underlying system is dense and non-Hermitian;

2. (ii) the kernel of a boundary integral operator is highly oscillatory for high wavenumbers, implying that a large linear system must be solved. The oscillation means that the fast multipole method and the fast wavelet methods are not immediately applicable.

This chapter reviews the recent work on using preconditioned iterative solvers for such linear systems arising from acoustic scattering modelling and points out the various challenges for future research work. We consider the following.

Matrix computing arises in the solution of almost all linear and nonlinear systems of equations. As the computer power upsurges and high resolution simulations are attempted, a method can reach its applicability limits quickly and hence there is a constant demand for new and fast matrix solvers. Preconditioning is the key to a successful iterative solver. It is the intention of this book to present a comprehensive exposition of the many useful preconditioning techniques.

Preconditioning equations mainly serve for an iterative method and are often solved by a direct solver (occasionally by another iterative solver). Therefore it is inevitable to address direct solution techniques for both sparse and dense matrices. While fast solvers are frequently associated with iterative solvers, for special problems, a direct solver can be competitive. Moreover, there are situations where preconditioning is also needed for a direct solution method. This clearly demonstrates the close relationship between a direct and an iterative method.

Parallel computing represents a major research direction for the future and offers the best and often the only solution to large-scale computational problems in today's technology. A book on fast solvers is incomplete without a discussion of this important topic. However, any incomplete description of the subject is of no use and there are already too many books available. Nevertheless, the author believes that too much emphasis has been put from a computer scientist's view (on parallelization) so that a beginner may feel either confused with various warnings and jargons of new phrases or intimated by the complexity of some published programs (algorithms) of well-known methods. Hence we choose to give complete details for a few selected examples that fall into the category of ‘embarrassingly parallelizable’ methods.

Therefore the purpose of this chapter is to convey two simple messages.

The research of inverse problems has become increasingly popular for two reasons:

1. (i) there is an urgent need to understand these problems and find adequate solution methods; and

2. (ii) the underlying mathematics is intriguingly nonlinear and is naturally posed as a challenge to mathematicians and engineers alike.

It is customary for an introduction to inverse problems of boundary value problems to discuss the somewhat unhelpful terms of ‘ill-posed problems’ or ‘improperly-posed problems’.

The linear system (1.1) may represent the result of discretization of a continuous (operator) problem over the finest grid that corresponds to a user required resolution. To solve such a system or to find an efficient preconditioner for it, it can be advantageous to set up a sequence of coarser grids with which much efficiency can be gained.

This sequence of coarser grids can be nested with each grid contained in all finer grids (in the traditional geometry-based multigrid methods), or non-nested with each grid contained only in the finest grid (in the various variants of the domain decomposition methods), or dynamically determined from the linear system alone in a purely algebraic way (the recent algebraic multigrid methods).

The electrical power network is a real life necessity all over the world; however, delivering the power supply stably while allowing various demand pattern changes and adjustments is an enormous challenge. Mathematically speaking, the beauty of such networks lies in their providing a challenging set of nonlinear differential-algebraic equations (DAEs) in the transient case and a set of nonlinear algebraic equations in the equilibrium case [438, 89, 292].

This chapter introduces the equilibrium equations, discusses some recent fast nonlinear methods for computing the fold bifurcation parameter and finally highlights the open challenge arisen from computing the Hopf bifurcation parameter, where one must solve a new system of size O(n2) for an original problem of size n. We shall consider the following.

1. Section 14.1 The model equations

2. Section 14.2 Fold bifurcation and arc-length continuation

3. Section 14.3 Hopf bifurcation and solutions

4. Section 14.4 Preconditioning issues

5. Section 14.5 Discussion of software and the supplied Mfiles

An iterative method for linear system Ax = b finds an infinite sequence of approximate solutions x(j) to the exact answer x*, each ideally with a decreased error, by using A repeatedly and without modifying it. The saving from using an iterative method lies in a hopefully early termination of the sequence as most practical applications are only interested in finding a solution x close enough to x*. Therefore, it almost goes without saying that the essence of an iterative method is fast convergence or at least convergence. When this is not possible for (1.1), we shall consider (1.2) with a suitable M.

This chapter will review a selection of iterative methods for later use as building blocks for preconditioner designs and testing.

This chapter will discuss the construction of Inverse Type preconditioners (or approximate inverse type) i.e. for equation (1.2)

MAx = Mb

and other types as shown on Page 3. Our first concern will be a theoretical one on characterizing A–1. It turns out that answering this concern reveals most underlying ideas of inverse type preconditioners. We shall present the following.

1. Section 5.1 How to characterize A–1 in terms of A

2. Section 5.2 Banded preconditioner

3. Section 5.3 Polynomial pk(A) preconditioners

4. Section 5.4 General and adaptive SPAI preconditioners

5. Section 5.5 AINV type preconditioner

6. Section 5.6 Multi-stage preconditioners

7. Section 5.7 The dual tolerance self-preconditioning method

8. Section 5.8 Mesh near neighbour preconditioners

9. Section 5.9 Numerical experiments

10. Section 5.10 Discussion of software and Mfile

In the usual FEM setting, Schur complement methods from Chapter 7 perform the best if there is some kind of ‘diagonal’ dominance. This chapter proposes two related and efficient iterative algorithms based on the wavelet formulation for solving an operator equation with conventional arithmetic. In the new wavelet setting, the stiffness matrix possesses the desirable properties suitable for using the Schur complements. The proposed algorithms utilize the Schur complements recursively; they only differ in how to use coarse levels to solve Schur complements equations. In the first algorithm, we precondition a Schur complement by using coarse levels while in the second we use approximate Schur complements to construct a preconditioner. We believe that our algorithms can be adapted to higher dimensional problems more easily than previous work in the subject.

The coupled matrix problems represent a vast class of scientific problems arising from discretization of either systems of PDE's or coupled PDE's and integral equations, among other applications such as the Karush–Kuhn–Tucker (KKT) matrices from nonlinear programming [273, 43]. The reader may be aware of the fact that many coupled (nonlinear) systems may be solved by Uzawa type algorithms [92, 153, 144, 115], i.e. all equations are ‘artificially’ decoupled and solved in turns. A famous example of this strategy is the SIMPLE algorithm widely used in computational fluid dynamics along with finite volume discretization [473, 334]. While there is much to do in designing better and more robust preconditioners for a single system such as (1.1), one major challenge in future research will be to solve the coupled problems many of which have only been tackled recently.

This chapter will first review the recent development on a general coupled system and then discuss some specific coupled problems. The latter samples come from a large range of challenging problems including elasticity, particle physics and electromagnetism. We shall discuss the following.