Many models in economics are formulated as dynamic optimization problems in which the objective functional is a sum (often discounted) of the instantaneous utilities derived over infinitely many periods and in which the system variables of adjacent periods are linked via intertemporal constraints. In this chapter, we present different solution methods for this class of problems which we refer to as ‘standard problems’.

We start in section 5.1 by describing the structure of the standard problem and by defining what we mean by its solution. After formulating the problem in primitive form (containing both state variables and control variables), we also present a reduced form that contains only state variables. In section 5.2, we discuss a solution method for the reduced form problem that is based on the Euler equation and the transversality condition. In section 5.3, we return to the primitive form of the dynamic optimization problem and show how it can be solved by a Lagrangian approach. Both approaches, the one based on the Euler equation and the transversality condition as well as the Lagrangian approach, are variational techniques that require differentiability assumptions. In section 5.4, we contrast these approaches to the technique of dynamic programming which can, in principle, be applied without any smoothness assumptions. Dynamic programming is a recursive solution technique, the heart of which is formed by the Bellman equation and the optimal value function. It is particularly powerful in the class of stationary discounted problems which will be discussed in section 5.5.

Model formulation and terminology

In the present section, we formulate a dynamic optimization problem that appears frequently in economics. As in part I of the book, we assume that the economic system under consideration is described by a finite set of system variables. In what follows, however, we shall distinguish between two kinds of system variables: state variables and control variables. The state variables describe the state of the system at the beginning of period t before the decision maker can make any choices.