The optimal control problem

Consider a fish stock which has some natural rate of growth and which is harvested. Too much harvesting could endanger the survival of the fish, too little and profits are forgone. Of course, harvesting takes place over time. The obvious question is: ‘what is the best harvesting rate, i.e., what is the optimal harvesting?’ The answer to this question requires an optimal path or trajectory to be identified. ‘Best’ itself requires us to specify a criterion by which to choose between alternative paths. Some policy implies there is a means to influence (control) the situation. If we take it that x(t) represents the state of the situation at time t and u(t) represents the control at time t, then the optimal control problem is to find a trajectory {x(t)} by choosing a set {u(t)} of controls so as to maximise or minimise some objective that has been set. There are a number of ways to solve such a control problem, of which the literature considers three:

1. (1) Calculus of variations

2. (2) Dynamic programming

3. (3) Maximum principle.

In this chapter we shall deal only with the third, which now is the dominant approach, especially in economics. This approach is based on the work of Pontryagin et al. (1962), and is therefore sometimes called the Pontryagin maximum principle.