Our study thus far points to the maximum principle as the fundamental principle of optimality and identifies the symplectic structure and the associated Hamiltonian formalism as the main theoretical ingredients required for its proper understanding. In this chapter we shall take that direction to its natural end and ultimately arrive at a geometric formulation of the maximum principle for optimal problems on arbitrary differentiable manifolds, rather than solely on ℝn as is customarily done in the literature on control theory. The geometric formulation of the maximum principle, essential for effective use of the principle for problems of mechanics and geometry, in a larger context illuminates the contribution of optimal control to the classic theory of Hamiltonian systems. Both of these points will become clearer in the next chapter, which deals with optimal problems on Lie groups.

This chapter begins with an initial formulation of the maximum principle for optimal problems defined on open subsets of ℝn. Rather than seeking the most general conditions under which the principle is valid, we shall follow the original presentation by Pontryagin et al. (1962). This level of generality is sufficient for the applications that follow and is at the same time relatively free of the technicalities that could obscure its geometric content.

The initial formulation is followed by a comparison between the maximum principle and other necessary conditions for optimality that emphasize the distinctions between strong and weak minima: The maximum principle, as an extension of the condition of Weierstrass, corresponds to the strong minimum, whereas the Euler-Lagrange equation corresponds to the weak minimum.

Minimizing the integral of a quadratic form over the trajectories of a linear control problem, known as the linear quadratic problem, was one of the earliest optimal-control problems (Kalman, 1960). Rather than limit our attention to the positive-definite case, as is usually done in the control-theory literature, we shall consider the most general situation for which the question is well posed. The minimal assumptions under which this problem is treated reveal a rich theory that derives from the classic heritage of the calculus of variations and yet is sufficiently distinctive to describe new phenomena outside the scope of the classic theory. As such, this class of problems is a natural starting point for optimal control theory.

This chapter contains a derivation of the “maximum principle” for this class of problems. The curves that satisfy the maximum principle are called extremal curves. The class of problems for which the Legendre condition holds is called “regular.” In the regular case, the maximum principle determines a single Hamiltonian, and the optimal solutions are the projections of the integral curves of the corresponding Hamiltonian vector field. The projections of these extremal curves remain optimal up to the first conjugate point.

Problems in the subclass for which the Legendre condition is not satisfied are called “singular.” For singular problems, the maximum principle determines an affine space of quadratic Hamiltonians and a space of linear constraints. The resolution of the corresponding constrained Hamiltonian system reveals a generalized optimal synthesis consisting of turnpike-type solutions. The complete description of these solutions makes use of higher-order Poisson brackets and is sufficiently complex to merit a separate chapter.

Having incorporated into optimal control theory the contributions made by the calculus of variations, mechanics, and geometry, we can begin to appreciate the exciting and challenging possibilities that this enriched subject offers back to geometry and applied mathematics. We have seen that the most remarkable equations of classic applied mathematics appear as only natural steps in the context of geometric control theory. In this chapter we shall again turn to the classic past to find inspiration for the future. We shall reconsider Kirchhoff's elastic problem in the Euclidean space E3 and his famous observation of 1859 that the elastic problem is “like” the heavy-top problem, known since then as the mechanical or kinetic analogue of the elastic problem.

Brilliant, but baffling, Kirchhoff's observation has had lasting influence on subsequent generations of mathematicians interested in the elastic problem who have sought to understand its solution through the analogy with the heavy-top problem. That view of the elastic problem partly explains why it has always been in the shadow of the heavy-top problem and is also partly responsible for its relative obscurity outside of the literature on elasticity.

Our treatment of variational problems on Lie groups in Chapters 12 and 13 shows clear connections between these two problems and provides natural interpretations for the famous observation of Kirchhoff: The heavy-top problem is “like” the elastic problem, and the analogy with the elastic problem illuminates its solutions, rather than the other way around as previously understood.

For the purposes of this book, a “control system” is any system of differential equations in which control functions appear as parameters. Our qualitative theory of control systems begins with the important geometric observation that each control determines a vector field, and therefore a control system can also be viewed as a family of vector fields parametrized by controls. A trajectory of such a system is a continuous curve made up of finitely many segments of integral curves of vector fields in the family.

This geometric view of control systems fits closely the theoretical framework of Sophus Lie for integration of differential equations and points to the non-commutativity of vector fields as a fundamental issue of control theory. The geometric context quickly reveals the Lie bracket as the basic theoretical tool, and the corresponding theory, known as geometric control theory, becomes a subject intimately connected with the structural properties of the enveloping Lie algebras and their integral manifolds. For this reason, our treatment of the subject begins with differentiable manifolds, rather than with ℝn as is customary in the control-theory literature.

As natural a beginning as it may seem, particularly to the reader already familiar with differential geometry, this point of view is a departure from the usual presentation of control theory, which traditionally has been confined either to linear theory and the use of linear algebra or to control systems in ℝn, with an emphasis on optimality. The absence of geometric considerations and explicit mention of the Lie bracket in this literature can be attributed to the historical development of the subject.

The process of transferring one state into another along a trajectory of a given differential system such that the time of transfer is minimal is known as the minimal-time problem, and it is one of the basic concerns of optimal control theory.

Minimal-time problems go back to the beginning of the calculus of variations. John Bernoulli's solution of the brachistochrone problem in 1697 was based on Fermat's principle of least time, which postulates that light traverses any medium in the least possible time. According to Goldstine's account (1980) of the history of the calculus of variations, Fermat announced that principle in 1662 in his collected works by saying that “nature operates by means and ways that are the easiest and fastest,” and he further differentiated that statement from the statement that “nature always acts along shortest paths” by citing an example from Galileo concerning the paths of particles moving under the action of gravity. Since then, time-optimal problems have remained important sources of inspiration during the growth of the calculus of variations.

In spite of the extensive literature on the subject, control theorists in the early 1950s believed that the classic theory did not adequately confront optimal problems that involved inequalities and was not applicable to problems of optimal control. Their early papers on time-optimal control problems paved the way to the maximum principle as a necessary condition for optimality.

This chapter begins with linear time-optimal problems and provides a selfcontained characterization of their time-optimal trajectories.

The differential systems that are encountered in applications are fundamentally analytic and therefore share a common geometric base. At the same time, such systems have extra-mathematical properties that differentiate one system from another and account for the particular features of their solutions. The theory of such systems is a blend of the general, shared by all Lie-determined systems, and the particular, due to other mathematical structures, and this recognition provides much insight and understanding, even for systems motivated by narrow practical considerations.

In this chapter we shall expand the theory of linear systems for systems with single inputs initiated in Chapter 4. In contrast to the traditional presentation of this theory, which relies entirely on the use of linear algebra and functional analysis, we shall develop the basic theory using the geometric tools introduced in Chapter 3. The geometric point of view will reveal that much of the theory of linear systems follows from considerations independent of the linear properties of the system and therefore extends to larger classes of systems.

Our selection of topics in this theory is motivated partly by the striking nature of the mathematical results and partly by their relevance to the second part of this book, dealing with optimality. Our treatment of linear systems begins with their controllability properties. We shall show that the main theorem has natural Lie-theory interpretations in terms of the Lie saturate of the system. Second, we shall show that a linear controllable system can be decoupled by means of linear feedback into a finite number of independent scalar linear differential equations. This result is known as Brunovsky's normal form.

Much like the unidentified hero figure in classic folk tales who, after having won his position by virtue of his achievements, in the end reveals a hidden but distinguished parentage, so optimal control theory, recognized initially as an engineering subject, reveals, as it reaches maturity, a distinct relationship to classic forebears: the calculus of variations, differential geometry, and mechanics. This distinctive character of optimal control theory can be traced to the mathematical problems of the subject in the mid-1950s dealing with inequality constraints. Faced with the practical, time-optimal control problems of that period, mathematicians and engineers looked to the calculus of variations for answers, but soon discovered, through the papers of Bellman et al. (1956), LaSalle (1954), and Bushaw (1958) on the bang-bang controls, that the answers to their problems were outside the scope of the classic theory and would require different mathematical tools. That realization initiated a search for new necessary conditions for optimality suitable for control problems. That search, further intensified by the space program and the race to the moon, eventually led, in 1959, to the “maximum principle,” which answered the practical needs of that period. So strongly was the maximum principle linked to control problems involving bounds on the controls that its significance in a larger context – as an important extension of Weierstrass's excess function in the calculus of variations – went unnoticed for a long time after its discovery.

The treatment here of optimal control problems combines the rich theory of the calculus of variations with control-theory innovations. I believe, as does Young (1969), that a control-theory point of view is more natural for the calculus of variations.