1 Introduction
The idea of turnpike in the Calculus of Variations or in Optimal Control describes the (usual) phenomenon which takes place when, in problems with finite but arbitrarily large time, the optimal solutions spend most of their time near a specific point. Even more, these solutions tend to be composed of three parts: an entry arc, the turnpike arc, and the leaving arc. The first and last ones are transitory arcs which take little time of the solution, whereas the middle arc (the turnpike) is a long arc which is essentially stationary, and tends to be exponentially near an equilibrium (see [Reference Trélat and Zuazua19]). Roughly speaking, in the long term, approximate solutions to problems having a turnpike are determined essentially by the integrand function of the objective functional, and are – again, essentially – independent of their endpoints and time interval.
Although the first works on the topic investigated specific problems arising in the context of economics and econometrics [Reference Dorfman, Samuelson and Solow7, Reference McKenzie18], today the turnpike property has become of interest in other areas [Reference Faulwasser, Korda, Jones and Bonvin9, Reference Grüne and Müller12]. Recent studies have proposed its use in applications varying from membrane-filtration systems [Reference Kalboussi, Rapaport, Bayen, Amar, Ellouze and Harmand15] to control of chemical reactors with uncertain models [Reference Faulwasser and Pannocchia10] or shape optimization in aircraft design [Reference Lance, Trelat and Zuazua17].
The property has also been noticed in Optimal Control Problems of almost any type: with/without terminal constraints [Reference Clarke4, Reference Trélat and Zuazua19]; with/without discounted cost functionals [Reference Gurman and Ukhin13, Reference Zaslavski24]; discrete-time problems with constraints [Reference Damm, Grüne, Stieler and Worthmann6, Reference Grüne and Müller12]; in a continuous-time problems without constraints [Reference Trélat and Zuazua19]; and so forth. Of course, no work on the turnpike property can omit referencing Zaslavski’s exhaustive studies, whose results and complete references are collected in [Reference Zaslavski21, Reference Zaslavski23].
From a practical point of view, the interest of the turnpike phenomenon arises from the fact that under this condition, the computation of (approximate) optimal trajectories in all areas of optimal control and variational problems becomes trivial for long enough time spans. In this sense, one of the first applications is [Reference Anderson and Kokotovic2], where a time-invariant linear quadratic optimal control problem is studied. They prove that the optimal trajectory is approximately composed of two solutions of two infinite-horizon optimal control problems. With
$x(0)$
fixed, the solution for the interval
$(0,+\infty )$
defines the part of the trajectory for the original problem from
$x=0$
to the turnpike. With
$x(T)$
fixed, the solution for the interval
$(-\infty ,T)$
defines the part of the trajectory of the original problem from the turnpike to
$t=T$
. The two parts are then pieced together and exhibit a similar transient behavior. Their approach is elementary and points out very clearly that the hyperbolicity phenomenon is the heart of the turnpike property.
Recently, in [Reference Trélat and Zuazua19], the authors investigate the relation between the turnpike property and numerical methods (direct and indirect) for a general nonlinear optimal control problem, without any specific assumption, and for very general terminal conditions. In the context of turnpike theorem, they provide a new method to ensure the successful initialization of numerical methods. Assuming that the Pontryagin maximum principle has been applied, the usual shooting method can be used. However, this is in general very hard to initialize. As a solution, they propose a variant: as the extremal is approximately known along the interval
$[\varepsilon ,T-\varepsilon ]$
, for some
$\varepsilon>0$
(i.e., the turnpike), but it is not at the endpoints
$t=0$
and
$t=T,$
the idea is choose some arbitrary point of
$[\varepsilon ,T-\varepsilon ]$
(for instance,
$t=T/2$
), and then integrate backward over
$[0,T/2]$
to get an approximation to
$x(0)$
, and forward over
$[T/2,T]$
to get the approximation to
$x(T)$
. The unknown value of
$x(T/2)$
must be adjusted, for instance, through a Newton method, so that transversality conditions are satisfied.
Even more recently, in the same spirit, in [Reference Faulwasser and Grüne8], the authors use the turnpike property in the numerical computation of optimal trajectories, splitting the optimization horizon at the turnpike. They proceed as follows: given the turnpike equilibrium
$x^{e}$
, the optimization horizon
$T>0$
, an initial condition
$x(0)$
, and a terminal condition
$x(T)$
, they compute an optimal trajectory
$x_{1}(\cdot )$
with finite horizon
$T_{1}<T$
and initial and terminal conditions
$x_{1}(0)=x(0)$
and
$x_{1}(T_{1})=x^{e}$
; and an optimal trajectory
$x_{2}(\cdot )$
with horizon
$T_{2}<T-T_{1}$
and initial and terminal conditions
$x_{2}(0)=x^{e}$
and
$x_{2}(T_{2})=x(T)$
. Finally, an approximation of the optimal trajectory is then obtained by concatenating the three arcs:
$x_{1}(t)$
,
$t\in \lbrack 0,T_{1}]$
;
$x^{e}$
,
$t\in \lbrack T_{1},T-T_{2}]$
; and
$x_{2}(t-T+T_{2})$
,
$t\in \lbrack T-T_{2},T].$
The resulting error can be estimated if the speed of convergence toward the turnpike is known (as in the case of exponential turnpike). They also use a second approach via Model Predictive Control which may have has some advantages.
To illustrate the turnpike and their methods, they consider a well-known harvest example [Reference Cliff and Vincent5], with both bilinear and quadratic objectives. Remarkably enough, the authors do not seem to notice that in the free-endpoint case, the leaving arc ends always at the same value of
$x(T)$
, regardless of
$x(0)$
and T. Something similar happens in [Reference Grüne11]: the author, who studies two examples of optimal investment problems, states literally: “Without any terminal constraints, all predictions end in
$x=2$
,” but does not delve into this happening. We shall see that this is a general property of turnpikes with free-endpoint solutions.
As a matter of fact, one of us had already noticed this in the previous paper [Reference Bayón, Garcia-Nieto, Garcia-Rubio, Otero, Suarez and Tasis3]. There a model of renewable resource exploitation in an open-access fishery [Reference Agnarsson, Arnason, Johannsdottir, Ravn-Jonsen, Sandal, Steinshamn and Vestergaard1], more detailed and general than [Reference Cliff and Vincent5], is studied. It was noticed that, without constraints on the terminal state (which force the trajectory to leave the turnpike), the solution spontaneously leaves the turnpike in order to reduce the cost of the overall trajectory, and the leaving arc always ends at the same value of
$x(T)$
, for all T.
In this note, we intend to settle the question of the entry and leaving arcs of the turnpike in the generic hyperbolic situation for variational problems. The key point was suggested in [Reference Bayón, Garcia-Nieto, Garcia-Rubio, Otero, Suarez and Tasis3] but not led to its natural consequence there. In short, and loosely speaking, our statement can be summarized as follows (for autonomous problems in
$\mathbb {R}$
): assume
$P=(x_P,\dot {x}_P)$
is a turnpike for a problem with initial condition
$x_0$
and free terminal condition, and let
$T(x,\dot {x})=0$
denote the equation giving the transversality condition. Then:
Statement: There is a function
$C(x,\dot {x})$
such that:
-
• The entry arc of the turnpike starts at
$$ \begin{align*}Q_{e}=\left\{ x=x_0\right\} \cap \left\{C(x,\dot{x})=C(x_P,\dot{x}_{P}) \right\}. \end{align*} $$
-
• The leaving arc of the turnpike ends at
$$ \begin{align*}Q_{l}=\left\{ T(x,\dot{x})=0\right\}\cap \left\{C(x,\dot{x})=C(x_P, \dot{x}_P) \right\}. \end{align*} $$
The function
$C(x,\dot {x})$
is well known to any practitioner: it is the function whose level sets are the extremals [Reference Kielhöfer16]. Certainly, the statement needs to be properly formalized, but its spirit should be clear to anyone familiar with the turnpike property. It is also more general (the problem may have both endpoints fixed, or none).
The main tool in our argument is to study the phase space of the plane vector field equivalent to the Euler equation associated with the variational problem. This vector field has very nice properties (among other things, its trajectories are both the extremals of the problem and the level sets of
$C(x,\dot {x})$
), and a direct application of the classical results on ordinary differential equations is enough to prove the statement.
The consequences of that result are straightforward: in order to determine the entry and leaving arcs, one only needs to know the intersection points between
$C(x,\dot {x})$
, the transversality condition, and/or the initial and final conditions (if any). Once these points are known, the entry arc can be computed by forward integration, and the leaving arc by backward integration, as the question has become an initial value problem at this point.
We hope this work provides a useful support for the study of long-term autonomous variational (and possibly control) problems near a steady state.
Our results are all straightforward consequences of the standard results on continuous dependence on parameters of solutions of ordinary differential equations, as well as the local structure of hyperbolic singularities. Despite this fact, we dedicate Section 3 to a thorough description of the geometric setting of the problem, with the aim of helping the reader understand the situation. We hope this is clearer, briefer, and simpler than a technical proof which would provide no insight and would be no more informative than what we provide.
After the formal statements in Section 4 and a suggestion for an approximate algorithm, we dedicate Section 5 to a hopefully illustrative example and Section 6 to numerical computations in it. A final section provides some remarks on the n-dimensional case.
2 Statement of the problem
Consider the autonomous variational problem in one dimension:
$$ \begin{align} \mathcal{P}\equiv \left\{ \begin{array}{l} \displaystyle\min \int_0^T F(x(t), \dot{x}(t))\,dt\,,\\[1em] x(0) = x_0\,, \end{array} \right. \end{align} $$
where F is a
$\mathcal {C}^2(\mathbb {R}^{2})$
function, and T is large enough. It is well known since [Reference Dorfman, Samuelson and Solow7] that many of these problems have a turnpike: a value
$x_{P}$
such that “most” solutions of (2.1) pass near it during a long period (i.e.,
$x(t)\simeq x_{P}$
and
$\dot {x}(t)\simeq 0$
for a “large” inner subinterval of
$[0,T]$
), for
$T\rightarrow \infty $
. Moreover, as Zaslavsky has proved [Reference Zaslavski22], there are also initial and final curves (the entry and leaving arcs)
$\gamma _e$
and
$\gamma _l$
such that, as
$T\rightarrow \infty $
, any solution of that problem is very near
$\gamma _e$
at the beginning, then near
$x_{P}$
, and finally, it is near
$\gamma _l$
in the end. Of course, all the terms between quotation marks can be properly defined [Reference Zaslavski24].
However, despite all the results around turnpikes, and as we have remarked in the Introduction, there is still no programmatic way to find their entry and exit arcs. The aim of this work is to explicitly show which curves these arcs are and how to compute them in the generic case.
3 Extremal curves, level sets, and hyperbolic saddles
Given problem (2.1), Euler’s equation
$$ \begin{align} \frac{\partial F(x(t),\dot{x}(t))}{\partial x} - \frac{d}{dt}\left( \frac{\partial F(x(t),\dot{x}(t))}{\partial \dot{x}} \right)=0 \end{align} $$
is best rewritten, after simplifying a common factor u, for our purposes, as a vector field, using x and u as subindices to indicate partial differentiation with respect to the first and second variables:
$$ \begin{align} \mathcal{E}\equiv\left\{ \begin{array}{l} \dot{x} = u\,,\\[3pt] \displaystyle \dot{u} = \frac{F_x - u F_{x u}}{F_{uu}}\,. \end{array} \right. \end{align} $$
This vector field might have singularities where
$F_{uu}=0$
(this implies, for instance, that if the problem is strictly convex in u, then there are no such singularities). The extremal curves (solutions to Euler’s equation) then coincide with the trajectories of
$\mathcal {E}$
. Moreover, as the problem is autonomous, it is well known (see, for instance, [Reference Kielhöfer16]) that the following function
$$ \begin{align} C(x,u) = F(x,u) - u F_u(x,u) \end{align} $$
is constant in the extremals. Thus, extremals, as one-dimensional manifolds, are the level sets of
$C(x,u)$
in
$\mathbb {R}^{2}$
, for the problem
$\mathcal {P}$
.
Let us work away from the points where
$F_{uu}=0$
, that is, we limit ourselves to an open set W where
$F_{uu}(x,u)\neq 0$
. Consider an equilibrium point P of
$\mathcal {E}$
, that is, a point with
$u=0, F_{x}-uF_{xu}=0$
. The linear part of
$\mathcal {E}$
is always of the form
$$ \begin{align*} L= \begin{pmatrix} 0 & \star_{1}\\ 1 & \star_2 \end{pmatrix}, \end{align*} $$
where the stars are unknown values. Except in degenerate cases, P is then either a center-focus (when both eigenvalues of L are complex), a node (both eigenvalues of L are real and have the same sign), or a hyperbolic saddle (real eigenvalues with different sign). Obviously, the nature of either center-foci or nodes prevents such a point from being a turnpike with entry and leaving arcs: if P is a center, trajectories turn around it, if it is a focus, then they either converge to it (so that P is not strictly speaking a turnpike) or move away from it (again, not a turnpike). For the same reasons as for foci, nodes cannot be turnpikes. Hence, turnpikes with entry and leaving arcs correspond, in the non-degenerate case, to hyperbolic saddles, as is well known.
Assume then that
$P=(x_P,u_P)$
is a hyperbolic saddle of
$\mathcal {E}$
, which by definition will have
$u_P=0$
(this is exactly what makes P a turnpike: near P, the velocity of
$\mathcal {E}$
tends to
$0$
and extremals spend “a long time” near P). On the other hand, we have
$F_x(x_P,u_P)-u_PF_{xu}(x_p,u_P)=0$
, which becomes at P just
$F_x(x_P,0)=0$
. As P is a hyperbolic saddle, there are two invariant manifolds adherent to P: the stable
$X_{s}$
and unstable
$X_{u}$
ones, meeting transversely at P (see Figure 1: the stable manifold “falls” toward P and the unstable one “goes away” from it). As these manifolds are unions of extremal curves (they are trajectories of
$\mathcal {E}$
), they correspond also to level curves of
$C(x,u)$
and, as P belongs to both, if we denote by
$M=X_s\cup X_{u}$
their union, they must necessarily have
$$ \begin{align*} M \equiv C(x,u) = C(x_P,0). \end{align*} $$
That is, the invariant set near P is given by
$C(x,u)=C(x_P,0)$
.
Hyperbolic saddle P and the open sets
$U_{ij}$
.
Near P, the set M can be divided into four different trajectories of
$\mathcal {E}$
:
$\gamma _s^1, \gamma _s^2$
, which are the two components of
$X_s\backslash \left \{ P \right \}$
and
$\gamma _{u}^1$
,
$\gamma _u^2$
for
$X_{u}\backslash \left \{ P \right \}$
. Any connected open set V containing P with sufficiently smooth border is divided by these four curves into four open subsets:
$U_{11}$
,
$U_{12}$
,
$U_{21}$
,
$U_{22}$
, each
$U_{ij}$
corresponding to the “angle” defined by
$\gamma _s^i$
and
$\gamma _u^j$
, in that order (see Figure 1).
The solutions of the variational problem
$\mathcal {P}$
are extremals (so they correspond to trajectories of
$\mathcal {E}$
) which verify the initial condition
$x(0)=x_0$
and also satisfy the transversality condition
$F_u(x(T),u(T))=0$
. The equation given by the trasnversality condition
$F_{u}(x,u)=0$
defines (usually) a curve in the
$(x,u)$
-plane. Figure 2 shows the “general” situation in which we find ourselves. The trajectory
$\gamma _{e}$
, part of the stable manifold, and
$\gamma _{l}$
, part of the unstable one, are the entry and leaving arcs, respectively.
Hyperbolic saddle P (turnpike), extremal (
$\gamma $
), and entry (
$\gamma _{e}$
) and leaving (
$\gamma _{l}$
) arcs. In yellow, the “slow” zone. As long as there are no singularities of
$\mathcal {E}$
in the cyan zone, the turnpike property holds inside it, and as
$T\rightarrow \infty $
, the corresponding extremal of
$\mathcal {P}$
approaches
$\gamma _{e}$
at the beginning and
$\gamma _{l}$
at the end. The entry arc starts at
$Q_{e}$
, and the leaving arc ends at
$Q_{l}$
.
The intersection points between the transversality condition
$F_u(x,u)=0$
and M are key in our statements. These points are the solutions of the system of equations:
$$ \begin{align*} \left\{ \begin{array}{l} F(x,u) -u F_u(x,u)= C(x_P,0)\,,\\ F_u(x,u) = 0\,, \end{array} \right. \end{align*} $$
which, after simplifying, becomes (see [Reference Bayón, Garcia-Nieto, Garcia-Rubio, Otero, Suarez and Tasis3], where this system of equations appeared for the first time):
$$ \begin{align} \left\{ \begin{array}{l} F(x,u) = C(x_P,0)\,,\\ F_u(x,u) = 0\,, \end{array} \right. \end{align} $$
In the problem
$\mathcal {P}$
, the initial value
$x(0)=x_0$
is set. Assume
$Q_{e}=(x_0,u_{e})$
belongs to
$x=x_{0}\cap \{C(x,u)=C(x_P,0)\}$
and to the stable manifold of P, and let
$Q=(x_{l},u_{l})$
be the solution of (3.4) in the unstable manifold (as in Figure 2) which is nearest to P. Under these assumptions, the Turnpike property happens relative to P (as in Figure 2) and extremals with
$x(0)=x_0$
start near
$Q_{e}=(x_{0},u_{e})$
, so that
$u(0)\rightarrow u_{e}$
, and finish near
$Q_{l}$
, so that
$x(T)\rightarrow x_{l}$
as
$T\rightarrow \infty $
.
If there existed a solution
$R=(x_{e}, u_{e})$
of (3.4), belonging to the stable manifold and satisfying the transversality condition (this case is not plotted in Figure 2), a dual argument using
$f(x,-\dot {x})$
shows that there are extremals of the variational problem with no initial or terminal condition:
$$ \begin{align} \mathcal{P}^{\prime}\equiv \min \int_0^T F(x(t), \dot{x}(t))\,dt \end{align} $$
with starting points
$x(0)\rightarrow x_{e}$
(and endpoints ending at
$Q_{l}$
as above).
This theoretical description which is just a qualitative expression of the well-known results on the continuous dependence of solutions of ODEs on the parameters, and of the local structure of hyperbolic singularities (see, for example, [Reference Ilyashenko and Yakovenko14]), is enough to prove our results, so that instead of proofs, we just refer to this section.
Our statements have two versions: one in which a solution
$\gamma $
of
$\mathcal {P}$
is already known, and one in which all depends just on the solutions of (3.4).
4 Statements of the results
As explained in the Introduction,
$F(x,u)$
is of class
$\mathcal {C}^2$
in
$\mathbb {R}^2$
, and all our statements are in an open set
$W\subset \mathbb {R}^{2}$
where
$F_{uu}(x,u)\neq 0$
. We shall make frequent reference to the vector field
$\mathcal {E}$
defined in (3.2).
Our first result assumes the existence of an extremal “sufficiently” near a hyperbolic turnpike.
Theorem 4.1 Let
$P=(x_P,u_P)$
be a hyperbolic saddle of
$\mathcal {E}$
, and let
$\gamma $
be an extremal of
$\mathcal {P}$
included in an open set
$U\subset W$
containing P which admits a subdivision
$U_{ij}$
for
$i,j\in \left \{ 1,2 \right \}$
as above. We assume the following conditions:
-
(1) The curve
$F_u(x,u)=0$
meets
$\gamma _u^1$
and
$\gamma $
transversely at the points
$Q=(x_{l},u_{l})$
and
$(x(T),u(T))$
. -
(2) That curve
$F_u(x,u)=0$
admits an injective parametrization near Q,
$\eta :[-1,1]\rightarrow \mathbb {R}^2$
with
$\eta (0)=Q$
,
$\eta (1)=(x(T),u(T))$
such that
$\eta $
is transverse to any extremal meeting it. -
(3) The extremals
$\gamma _s^1$
and
$\gamma $
meet the manifold
$x=x_0$
transversely at
$(x_0,u_{e})$
and
$(x_0,u_0)$
, respectively. -
(4) The open set V (delimited by
$\gamma ^1_s$
and
$\gamma _u^1$
,
$\eta $
, and the line
$x=x_0$
) contains no more singularities of
$\mathcal {E}$
.
Then:
-
(1) For any
$\overline {T}>T$
, the problem has an extremal
$$ \begin{align*} \mathcal{S}_{\overline{T}} \ \left\{ \begin{array}{l} \displaystyle\min \int_0^{\overline{T}} F(x(t), \dot{x}(t))\,dt\\[1em] x(0) = x_0 \end{array} \right. \end{align*} $$
$\gamma $
which is totally included in V.
-
(2) For any
$\epsilon>0$
, there is
$T_{\epsilon }>T$
,
$T_{\epsilon ,{e}}>0$
, and
$T_{\epsilon ,\infty }<T$
, such that for
$\overline {T}>T_{\epsilon }$
, the corresponding extremal
$\gamma _{\overline {T}}$
satisfies:-
• The metric distance between
$\gamma _{\overline {T}}:[0,T_{e}]\rightarrow \mathbb {R}^2$
and
$\gamma _{e}$
is less than
$\epsilon $
. -
• The metric distance between
$\gamma _{\overline {T}}:[T_{e}, T_{l}]\rightarrow \mathbb {R}^2$
and P is less than
$\epsilon $
. -
• The metric distance between
$\gamma _{\overline {T}}:[T_{l}, \overline {T}]\rightarrow \mathbb {R}^2$
and
$\gamma _{l}$
is less than
$\epsilon $
.
$T_{e}$
and
$T_{l}$
such that
$T_{l}-T_{e}\rightarrow \infty $
as
$\epsilon \rightarrow 0$
and
$T_l,T_e<K$
for some
$K<\infty $
.
-
Proof The first conclusion follows from the continuous dependence of solutions of an ordinary differential equation on the parameters (and from all the qualitative descriptions in Section 3). The second one follows also from the local structure of hyperbolic saddle singularities (see, for instance, [Reference Ilyashenko and Yakovenko14]).
Definition 4.1 The curve
$\gamma _{e}$
from
$x=x_0\cap \gamma _{e}$
to P is called the entry arc to the turnpike P. The curve
$\gamma _{l}$
from P to Q is called the leaving arc of the turnpike P.
The previous statement seems to require a lot from the equation. As it happens, most of the hypotheses are just technical and will hold in generic situations. On the other hand, we can also say a lot (locally) if we just know that the transversality condition meets
$\gamma _{l}$
transversely. The following result is again a straightforward consequence of the local structure of hyperbolic saddles and the continuous dependence on the parameters of solutions of ODEs. All the statements are inside
$W\subset F_{uu}(x,u)\neq 0$
.
Theorem 4.2 Let P be a hyperbolic saddle, and let
$\gamma _{l}\subset \gamma _u^1\cup \gamma _u^2$
be one of the components of the unstable manifold. Assume that the transversality condition
$Tr\equiv F_u(x,u)=0$
meets
$\gamma _{l}$
transversely at
$Q_{l}=(x_{l}, u_{l})$
and that there are no more intersection points between
$Q_l$
and P belonging to
$\gamma _{l}$
. Then there is
$\epsilon>0$
and a parametrization
$\tau :[-\epsilon ,\epsilon ]\rightarrow T_c$
with
$\tau (0)=Q_{l}$
such that the two extremals containing
$\tau (-\epsilon )$
and
$\tau (\epsilon )$
satisfy all the properties of Theorem 4.1. As a consequence, P is a turnpike for
$\mathcal {P}$
. Moreover, assume that
$\gamma _s^1$
is to the right of P and that
$\gamma _s^2$
is to its left. Assume, for simplicity, that
$Q_{l}$
is to the left of P (i.e.,
$x_{l}<x_P$
). Then:
-
(1) The point P is a turnpike for
$\mathcal {P}$
for
$x_0\in (x_P,x_P+\epsilon )$
, for some
$\epsilon>0$
,
$\gamma _{l}$
(from P to Q) is the leaving arc, and
$\gamma _s^1$
(from
$x=x_0$
to P) is the entry arc. -
(2) At the same time, P is a turnpike for
$\mathcal {P}$
for
$x_0\in (x_P-\epsilon ,x_P)$
for some
$\epsilon>0$
,
$\gamma _{l}$
(from P to Q) is the leaving arc, and
$\gamma _s^2$
(from
$x=x_0$
to P) is the entry arc.
Notice how there is a switch of entry arcs when the initial condition
$x_0$
changes from being “greater than
$x_{P}$
” to “less than
$x_P$
.” This is easy to see in Figure 2: if
$x_0$
is less than
$x_P$
, the extremals ending near Q must approach, at their beginning, the top-left separatrix for
$T\rightarrow \infty $
, instead of
$\gamma _{e}$
. Obviously, for the dual problem (final condition set but initial condition free), it is the leaving arc that changes.
Consider the problem (3.5) with no initial or final condition. Near a hyperbolic singularity P of
$\mathcal {E}$
, we may have a turnpike result if the system of equations (3.4) has two solutions near P. Again, everything is restricted to some open set
$W\subset F_{uu}(x,u)\neq 0$
.
Theorem 4.3 With the notations above, assume that P is a hyperbolic singularity of
$\mathcal {E}$
. If
$Q_{e}\in \gamma _s^{1}$
and
$Q_{l}\in \gamma _u^1$
are two solutions of (3.4) and there are no more solutions of (3.4) between
$Q_{e}$
and P and P and
$Q_{l}$
, then P is a turnpike for the problem (3.5). That is, for
$T\rightarrow \infty $
, there are extremals
$\gamma =(x(t),u(t))$
of (3.5) satisfying:
-
(1) The origin tends to
$Q_{e}$
:
$(x(0),u(0))\rightarrow Q_{e}$
. -
(2) The end tends to
$Q_{l}$
:
$(x(T),u(T))\rightarrow Q_{l}$
. -
(3) The curve
$\gamma $
approaches the part of
$\gamma _s^{1}$
between
$Q_{e}$
and P at the beginning (the entry arc). -
(4) The curve
$\gamma $
approaches the part of
$\gamma _u^{1}$
between P and
$Q_{l}$
at the end (the leaving arc).
Proof As previously, the proof is a straightforward application of the description in Section 3, the structure of hyperbolic singularities and the continuous dependence on parameters of solutions of ODEs.
Finally, consider the problem with fixed endpoints:
$$ \begin{align} \overline{\mathcal{P}}\equiv \left\{ \begin{array}{l} \displaystyle\min \int_0^T F(x(t), \dot{x}(t))\,dt,\\[1em] x(0) = x_0,\ x(T) = x_T. \end{array} \right. \end{align} $$
In this case, the statements hold regardless of the transversality condition.
Theorem 4.4 Assume that P is a hyperbolic singularity of
$\mathcal {E}$
. Assume that
$x=x_0$
meets
$\gamma _1^s$
at
$Q_{e}$
and that
$x=x_T$
meets
$\gamma _1^u$
at
$Q_{l}$
. Then P is a turnpike for the problem (4.1), and there is an open neighborhood V of P such that for
$T\rightarrow \infty $
, any extremal
$\gamma $
of (4.1) included in V satisfies statements (1)–(4) of Theorem 4.3.
Notice that
$Q_{e}$
and
$Q_{l}$
in previous results can be easily computed using the fact that
$\gamma _i^{s}$
and
$\gamma _i^u$
are level sets of
$C(x,u)$
. Thus, if they exist, then
$$ \begin{align*} Q_{e} \in \left\{ C(x,u)=C(P) \land x=x_0 \right\},\,\,\, Q_{l} \in \left\{ C(x,u)=C(P) \land x=x_T \right\}. \end{align*} $$
However, those solution sets might have more than one point, and one needs to verify which (if any) can be a candidate.
4.1 Suggestion for an approximate algorithm
From the discussions above, the following method is suggested to use the turnpike and the entry and leaving arcs as approximate extremals for
$T\gg 0$
. Specifically, for Problem (2.1) with
$x(0)$
fixed and
$x(T)$
free:
-
(1) State a tolerance
$\epsilon>0$
. -
(2) Find the possible turnpike
$P=(x_P,0)$
. This requires studying the phase space of (3.2), its singularities, and the separatrices
$\gamma _s^i$
and
$\gamma _u^i$
, for
$i=1,2$
. For this, one can just use the level set
$C(x,u)=C(x_P,0)$
. -
(3) Find the adequate
$Q_e$
and
$Q_l$
. As explained above,
$Q_e$
belongs to
$x=x_0$
and
$C(x,u)=C(x_P,0)$
, whereas
$Q_l$
is found using system (3.4). -
(4) From
$Q_e$
, compute the trajectory
$\gamma _{e}(t)$
of
$\mathcal {E}$
with
$\gamma _e(0)=Q_e$
and ending at
$|\gamma _e(T_e)-x_P|<\epsilon $
. This is an IVP integrated until some condition is met. -
(5) From
$Q_l$
, compute the trajectory
$\gamma _l(t)$
of
$\mathcal {E}$
with
$\gamma _l(T_{l})=Q_l$
and
$|\gamma _l(0)-x_P|<\epsilon $
. This is a backward IVP integrated until some condition is met.
After those computations, if
$T>T_e+T_l$
, then any extremal
$\gamma _T$
of (2.1) can be approximated by the turnpike as:
$$ \begin{align} \gamma_T(t) \simeq \left\{ \begin{array}{l} \gamma_e(t)\ \mathrm{if}\ t\in[0,T_{e})\,,\\ x_{P}\ \mathrm{if}\ t\in[T_e,T-T_l]\,,\\ \gamma_l(t)\ \mathrm{if}\ t\in(T-T_l,T]\,. \end{array} \right. \end{align} $$
5 An example: shallow lakes
In this section, we showcase the well-known shallow lakes model without discount (see, for instance, [Reference Wagener20] for the details), with a modified cost function to prevent, in our example, the issues with the logarithm. The problem to be solved is, initially, the Optimal Control problem with control variable v:
$$ \begin{align} \mathcal{P}\equiv \left\{ \begin{array}{l} \displaystyle\max \int_0^T \left(v^{2} - c x^{2}\right)\,dt\\[0.5em] \dot{x} = v - b x + r \displaystyle\frac{x^{2}}{x^2+1}\\ x(0) = x_0 \end{array} \right. \end{align} $$
with
$c, b, r$
positive constants. This is, in fact, a variational problem, as v can be expressed as a function of
$x, \dot {x}$
and there are no restrictions. Thus, we shall in fact study the variational problem
$$ \begin{align} \mathcal{P}\equiv \left\{ \begin{array}{l} \displaystyle\max \int_0^T F(x, \dot{x}) \,dt\\[0.8em] x(0) = x_0 \end{array} \right. \end{align} $$
with
$$ \begin{align*} F(x,\dot{x}) = b^2 x^2-\frac{2 b r x^3}{x^2+1}+2 b x \dot{x}-c x^2+\frac{r^2 x^4}{\left(x^2+1\right)^2}-\frac{2 r x^2 \dot{x}}{x^2+1}+\dot{x}^2, \end{align*} $$
and we shall set the value of the constants to
$r=1, c=0.1$
, and
$b=0.7$
. The Euler equation for this problem is, once divided by
$\dot {x}$
,
$$ \begin{align} \frac{1}{(1+x^2)^3}&\bigg( x^6 (-2 \ddot{x}-1.4)+x^4 (-6 \ddot{x}-5.6)\bigg.+ x^2 (-6 \ddot{x}-4.2)- \notag\\ &\qquad\qquad\qquad\qquad \bigg.(2 \ddot{x}+0.78 x^7+2.34 x^5+6.34 x^3+0.78 x \bigg) = 0. \end{align} $$
And, the vector field associated with this second-order equation is, in the
$(x,u)$
plane corresponding to
$(x,\dot {x})$
,
$$ \begin{align} \mathcal{E} \equiv \left\{ \begin{array}{l} \dot{x} = u\,,\\ \dot{u} = \displaystyle \frac{0.39 x \left(x^6-1.79487 x^5+3. x^4-7.17949 x^3+8.12821 x^2-5.38462 x+1\right)}{\left(x^2+1\right)^3}. \end{array} \right. \end{align} $$
The denominator in
$\dot {u}$
is never
$0$
, so that
$\mathcal {E}$
is well defined in all
$\mathbb {R}^{2}$
. The vector field
$\mathcal {E}$
has three singular points: two hyperbolic saddles
$P_1=(0,0)$
and
$P_2\simeq (1.5062,0)$
and a center/focus,
$O=(0.2747,0)$
. Figure 3 shows the structure of
$\mathcal {E}$
near its singularities (in red).
Stream lines of
$\mathcal {E}$
in the example. The red dots are its singularities, at
$u=0$
,
$x\in \left \{ 0,0.2747,1.5062 \right \}$
.
The function whose level sets are the extremals (the trajectories of
$\mathcal {E}$
) is
$$ \begin{align*} C(x,u) = x^2 \left(\frac{x^2}{\left(x^2+1\right)^2}-\frac{1.4 x}{x^2+1}+0.39\right)-u^2 \end{align*} $$
so that we need to focus our attention on the level sets
$$ \begin{align*} L_1 \equiv C(x,u) = C(P_1) = 0 \end{align*} $$
and
$$ \begin{align*} L_2 \equiv C(x,u) =C(P_2) = -0.097. \end{align*} $$
Finally, the transversality condition in this case is given by
$$ \begin{align*} Tr \equiv 2 u + 1.4 x - \frac{2 x^2}{1 + x^2}=0. \end{align*} $$
In Figure 4, we have plotted the sets
$L_1$
(cyan),
$L_2$
(yellow), and
$Tr$
(black). Notice how
$Tr \cap L_1$
is just the hyperbolic point
$P_1$
whereas
$Tr\cap L_2$
has two points, one above
$u=0$
and the other one below (both in green).
Hyperbolic structure of the example. The leftmost singularity is hyperbolic, but its level set (blue) meets the transversality condition only at the singularity. The level set of the rightmost singularity (yellow) meets the transversality condition twice (at the green dots).
Surprisingly enough, the transversality condition (in black in Figure 4) only meets the curve
$M_{1}\equiv C(x,u)=C(P_1)$
(in cyan) at
$(0,0)$
so that our results only apply to
$P_1$
in the fixed-endpoints versions (because
$Tr$
never meets
$M_1$
transversely).
Consider the hyperbolic saddle
$P_2$
. We are going to showcase the four turnpike possibilities for it under problem (5.2).
The transversality condition meets the (yellow) curve
$M_{2}=C(x,u)=C(P_2)$
at the (green) points
$Q_1\simeq (-0.9852,1.1822)$
and
$Q_2\simeq (0.9852,-1.971)$
. Clearly, the top-left and bottom-right parts of M are the stable manifolds, call them
$\gamma _s^1$
and
$\gamma _s^2$
, respectively, whereas
$\gamma _u^1$
is the bottom-left part.
5.1 Initial condition fixed. Change of entry arc and of turnpike
Recall that we have called:
$\gamma _s^1$
the top-left branch of
$M_2$
(in yellow in Figure 4) and
$\gamma _s^2$
the bottom-right branch (these are the stable trajectories and will give rise to the entry arcs). Also,
$\gamma _u^1$
is the bottom-left branch, and
$\gamma _u^{2}$
is the top-right one, which will give rise to the leaving arcs.
In this subsection, we are going to study the problem (5.1) (i.e., with initial condition but no end condition).
If
$x_0>1.5062$
, the extremals meet the transversality condition near
$Q_2\simeq (0.9852,-1.971)$
for
$T\rightarrow \infty $
, whatever the value of
$x_0$
. The entry arc to the turnpike
$P_{2}$
in this case is
$\gamma _2^s$
from
$x=x_0$
to
$P_{2}$
: this happens for any
$x_0>1.5062$
because the transversality condition does not meet
$M_2$
for
$x>1.5026$
.
However, the moment
$x_0$
is to the left of
$P_2$
, that is,
$x_0<1.5062$
, and the entry arc to turnpike P changes from
$\gamma _s^2$
to
$\gamma _s^1$
(which is above
$u=0$
). As
$x_{0}\rightarrow -0.9852$
(the x-coordinate of
$Q_1$
), the extremals approach
$\gamma _s^1$
. The problem with
$x_0=-0.9852$
has no solution because
$Q_1$
is the only intersection point between an extremal and the transversality condition (this is easily seen in Figure 4).
Finally, for
$x_0<-0.9852$
, the candidate extremals for problem
$\mathcal {P}_f$
for
$T\rightarrow \infty $
approach
$P_1=(0,0)$
, the intersection point of
$Tr$
and
$M_1$
(the black and cyan lines in Figure 4). Thus, there is an entry arc, from
$x=x_0\cap Tr$
to
$P_1$
, but the turnpike is never left, in this case.
5.2 Initial and final conditions fixed
When
$x(0)=x_0$
and
$x(T)=x_T$
are both fixed, the transversality condition plays no role and one needs only study the relation between these conditions and the hyperbolic singularities
$P_1$
and
$P_2$
. For the sake of simplicity, we are only going to show some cases. Let
$x_{P_1}=0$
denote the x-coordinate of
$P_1$
and
$x_{P_2}\simeq 1.5062$
the one of
$P_2$
. Of course, in order to have a turnpike behavior, there must be at least one singularity between
$x_0$
and
$x_T$
.
-
• When
$x_0>x_{P_2}>x_T$
, then
$P_2$
is a turnpike and the entry arc is
$\gamma _s^2\cap \{x=x_{0}\}$
, and the leaving arc is
$\gamma _u^1\cap \{x=x_T\}$
. -
• On the other hand, if
$x_0<x_{P_2}<x_T$
, then the situation reverses at
$P_{2}$
(we are “above
$u=0$
” and the arcs are now:
$\gamma _s^1$
the entry one from
$x=x_0$
to
$P_2$
and
$\gamma _u^2$
from
$P_2$
to
$x=x_T$
. -
• If
$x_0,x_T\in (x_{P_1},x_{P_2})$
(that is, both endpoints are between
$P_1$
and
$P_{2}$
), it is easy to realize that
$P_1$
is still a turnpike and the entry and leaving paths correspond to
$\gamma _s^1$
and
$\gamma _u^2$
, respectively (starting at
$x=x_0$
and ending at
$x=x_T$
, also). -
• When, say
$x_{0}<x_{P_1}$
and
$x_T<x_{P_2}$
, there are two candidate extremal curves for
$T\rightarrow \infty $
: one having a turnpike at
$P_1$
, and the other one at
$P_2$
; it is necessary here to discern the optimality by other methods (which we shall not do, as this is out of our aim). Obviously, each turnpike has his respective entry and leaving arcs (in this case,
$P_1$
arcs are the cyan unbounded curves to its left).
5.3 The free endpoint problem
Finally, the free endpoint problem requires the extremals to meet the transversality condition at
$x(0)$
and
$x(T)$
. In our case, only
$M_2$
meets
$Tr$
twice away from a singularity, whereas
$M_1\cap Tr=\{ P_{1} \}$
. As far as extremals go, the “constant curve”
$(x(t),u(t))=P_1$
for all
$t\in [0,T]$
is always a candidate trajectory (as it is an extremal which satisfies the transversality conditions). These have obviously constant cost
$F(P_1)\times T$
.
There is, however, a second possibility giving rise to a true turnpike: the solutions starting near
$Q_1$
below
$\gamma ^1_s$
, approaching
$P_2$
and ending near
$Q_2$
above
$\gamma _u^1$
. In this case, the entry arc is
$\gamma _s^1$
from
$Q_1$
to P and the leaving arc is
$\gamma _u^1$
from
$P_1$
to
$Q_2$
.
6 Simulations
In this section, we plot the simulations corresponding to some of the cases in Section 5. We have used a budget computer (Intel Core i5 with 16GB RAM) and Mathematica, with no excessive time used (the simulations can be run in several hours, the longest time taken by the very precise computation of the entry and leaving arcs and, unfortunately, the plotting commands, as the numerical solutions are interpolating functions and their evaluation is quite slow). We restrict ourselves, for the sake of brevity, to the initial problem (2.1) with
$x(0)=x_0=0.5$
and
$T\gg 0$
.
The above requires computing the turnpike entry arc starting at the point
$P_{e}=(0.5,u_e)$
, which is the solution of the first equation in (3.4) with
$x=0.5$
, that is,
$u_e$
is the solution of
$$ \begin{align} F(0.5, u) = C(P_2) = C(1.5062,0), \end{align} $$
giving
$u_e\simeq 0.30751221580$
. However, one needs to compute
$u_e$
with a huge precision in order to really obtain a fine approximation to the turnpike. In our computations, we used
$30$
values of precision when computing the solution of (6.1) (so that
$P_2$
was also computed with that precision).
We also need to compute the leaving arc of that turnpike, which requires knowing the point
$P_l=Q_2$
, solution of (3.4):
$$ \begin{align} \left\{ \begin{array}{l} F(x_{l}, u_{l}) = C(P_2)\,,\\ Tr(x_l,u_l) = 0\,, \end{array} \right. \end{align} $$
which gives, as indicated above,
$P_l\simeq (0.9852, -1.971)$
(with the same caveat regarding the precision).
Figure 5 contains the plot of the extremal
$x(t)$
(in blue) and its derivative
$u(t)=\dot {x}(t)$
(orange), corresponding to (2.1) with
$x_0=0.5$
and
$T=63$
. Overlain (in dashed lines) we have plotted the entry arc from
$t=0$
to
$t=24$
, and the leaving arc, from
$t=41$
to
$t=63$
. There is no noticeable difference.
Turnpike entry and leaving arcs compared to solution for
$T= 63$
.
Figure 6 shows, on the left, the difference between
$x(t)$
and the entry arc for the same
$T=63$
, and on the right, the difference between
$u(t)=\dot {x}(t)$
and the corresponding value on the turnpike, for
$t=T-22$
to T (where
$22$
is taken as a value where the x-value of the leaving arc is less than
$10^{-5}$
from the true turnpike
$P_2$
). Notice that the time is inverted in the latter plot because we have computed the leaving arc “backward.” The errors are, as can be seen, irrelevant to all purposes.
Absolute differences between entry (left) and leaving (right) arcs and the corresponding part of the solution for
$T=63$
. On the right, the time is reversed (from
$Q_l$
to P).
Finally, Figure 7 contains the plots of the different solutions
$x(t)$
for times T between
$51$
and
$56$
and for time
$T=63$
. The structure of the entry arc is essentially the same for all and all are, obviously, indistinguishable, whereas the leaving arc is also essentially equal but starts at different times.
Solutions for times between 51 and 56, and for
$T=63$
.
Figure 8 shows the difference between the corresponding entry and leaving arcs and the ones of the turnpike (where the cutting point is set as above).
Absolute difference between the solutions in Figure 7 and the entry (left) and leaving (right) arcs (time is reversed on the right).
7 Final remarks
Our aim in this paper is just to show, in the case of dimension
$1$
, which is the most graphical one and how to compute the entry and leaving arcs of the turnpike of an autonomous variational problem in order to settle this question. Of course, the generalization to variational problems in which the functional
$F(x_1,\dot {x}_{1},\ldots , x_k,\dot {x}_k)$
has “separated variables,” that is, problems with
$$ \begin{align*} \frac{\partial^2 F}{\partial u \partial v} = 0 \end{align*} $$
whenever
$u, v$
correspond to variables with different indices (i.e.,
$u\in \left \{ x_i,\dot {x}_{i} \right \}$
and
$v\in \left \{ x_j,\dot {x}_j \right \}$
with
$i\neq j$
), is straightforward, as the associated vector fields are defined by independent equations.
The most general autonomous case is, for the time being, inaccessible to us, but we hope the technique presented in this work may be useful to elucidate their solution.
Open access
