This chapter is devoted to the study of finite element methods in one and two dimensions. We begin by presenting the general theory of Galerkin methods and their analysis; in particular Galerkin orthogonality and Cea’s lemma are introduced in an abstract setting. Then the construction of finite element spaces, and their bases, in one dimension is detailed. The notions of mesh and hat basis functions are introduced here. The general theory of Galerkin approximations is then used to reduced the error analysis of finite element schemes to a question in approximation theory. The properties of the Lagrange interpolant in Sobolev spaces (in one dimension) then close the argument. Duality techniques, i.e. Nitsche’s trick are then used to obtain optimal error estimates in L2. The same ideas are presented, mostly without proof, for the finite element scheme in two dimensions.

Periodic differential equations and their approximation are the topic of this chapter. We discuss the application of classical finite difference schemes and their analysis in this setting. The spectral Galerkin method, that is, using trigonometric polynomials as basis is then discussed and we show its spectral accuracy. Finally the pseudo-spectral method is presented, its implementation via the DFT is discussed.

In this chapter we begin the study of best approximations. In this case we study the best (min) polynomial approximation in the uniform (max) norm. The existence a best approximating polynomial is first presented. The more subtle issue of uniqueness is then discussed. To show uniqueness the celebrated de la Vallee Poussin, and Chebyshev equi-oscillation theorems are presented. A first error estimate is then presented. The problems of interpolation, discussed in the previous chapter, and best approximation are then related via the Lebesgue constant. Chebyshev polynomials are then introduced, and their most relevant properties presented. Interpolation at Chebyshev nodes, and the mitigation of the Runge phenomenon are then discussed. Finally; Bernstein polynomials; moduli of continuity and smoothness; are detailed in order to study Weierstrass approximation theorem.

We prove the equivalence of the solution to a linear system of equations with an HPD matrix to the problem of quadratic minimization. With the help of this equivalence, we study the minimization of a quadratic energy and introduce gradient descent methods with exact and approximate line search, we study the preconditioned steepest descent method. We introduce the conjugate gradient method, with preconditioning, as a Galerkin approximation over Krylov subspaces and show its convergence. For systems with non HPD matrices we discuss the GMRES method.

We present the classical theory of linear iterative schemes for linear systems of equations: the Richardson, Jacobi, Gauss–Seidel, and relaxation methods are presented and analyzed. We introduce the Householder-John criterion for convergence of iterative schemes. The symmetrization and symmetric iterations are presented for the relaxation scheme. Some nonstationary methods, like minimal residuals, Chebyshev iterations, and minimal corrections are presented.

This chapter studies finite difference methods for elliptic problems. It begins with a rather lengthy and general discussion of grid domains, grid functions, finite difference operators, and their consistency. We then introduce the notion of stability of a finite difference scheme and Lax’s principle: a consistent and stable scheme is convergent. Then we apply all these notions to elliptic operators in one and two dimensions, with the main focus being the Laplacian. We show the discrete maximum principle, energy arguments and how these can be used to attain stability and convergence in various norms. For more general operators we introduce the notions of homogeneous schemes and upwinding. For operators in divergence form we provide an analysis via energy arguments. For non divergence form operators we analyze the monotonicity and comparison principles of the arising schemes.

We introduce Runge-Kutta methods and their Butcher tableau. We discuss necessary order conditions, and thoroughly analyze some two and three stage schemes. We then discuss the class of Runge-Kutta collocation methods and their consistency. In particular we present the class of Gauss-Legendre-Runge-Kutta methods and their order. Finally, we study how to approximate dissipative equations via so-called dissipative schemes: the M-matrix of a scheme and schemes of B(q) and C(q) types.

In this chapter we study finite difference schemes for parabolic partial differential equations. The notions of conditional and unconditional stability, and CFL condition are introduced to analyze the classical schemes for the heat equation. Different techniques, like maximum principles and energy arguments are presented to obtain stability in different norms. Then, we turn to the study of the pure initial value problem, the grand goal being to discuss the von Neumann stability analysis. To accomplish this we introduce the notions of Fourier-Z transform of grid functions and the symbol of a finite difference scheme. This allows us to state the von Neumann stability condition and prove that it is necessary and sufficient for stability. These notions are also used to present a covergence analysis that is somewhat different than the one presented in previous sections.

We study the solution of overdetermined systems of equations. Introduce weak, and in particular least squares solutions. For full rank systems, we show existence and uniqueness via the normal equations. We introduce projection matrices and the QR factorization. We discuss the computation of the QR factorization with the help of Householder reflectors. For rank defficient systems we prove the existence and uniqueness of a minimal norm least squares solution. We introduce the Moore-Penrose pseudoinverse, show how it relates to the SVD, and how it can be used to solve rank defficient systems.

This appendix collects a review of the calculus and analysis in one and several variables that the reader should be familiar with. Notions of convergence, continuity, differentiability and integrability are recalled here.

We present several facts about the natural transformations between vector spaces, and their representations via matrices. We introduce induced matrix norms, and the spectral decomposition of nondefective matrices

This chapter presents all the needed theoretical background regarding the initial value problem for a first order ordinary differential equation in finite dimensions. Local and global existence, uniqueness, and continuous dependence on data are presented. The discussion then turns to stability of solutions. We discuss the flow map and the Alekseev-Grobner Lemma. Dissipative equations. and a discussion of Lyapunov stability of fixed points conclude the chapter.