Introduction

I have spent 29 years of my life with INRIA. I had seen the beginning of the institute in 1967. I was appointed president of the institute in 1984 and I left it in 1996, after 12 years as president.

Gilles Kahn joined IRIA in the late 1960s and made his entire career in the institute. He passed away while he was president, after a courageous fight against a dreadful illness, which unfortunately did not leave him any chance. I knew him for more than 35 years and I have an accurate vision on the role he played in the development of the institute. More globally, I have seen his influence in the computer science community in France and in Europe. This article gives me the opportunity to understand what was behind such a leadership and to recall some of the challenges that INRIA has faced during more than three decades. What we can learn from the past and from the action of former great leaders is extremely helpful for the future.

This is probably the best way to be faithful to Gilles and to remain close to his thoughts.

Historical IRIA

Why IRIA?

INRIA was born as an evolution of IRIA.

IRIA was created in 1967 as a part of a set of decisions taken under the leadership of General de Gaulle. It was a time of bold decisions towards research at large, based on clear political goals.

Introduction

The stochastic maximum principle and dynamic programming are among the main methods stochastic control theory. It is also possible to develop such methods for partially observable systems. This is a priori expectable since the stochastic control problem with partial observation can be reduced to a stochastic control problem with complete observation, with the reservation that the system with full observation (to be controlled) is not finite dimensional, but infinite dimensional. With this remark in mind, we see that we are led to use the maximum principle or dynamic programming for an infinite dimensional system. The situation is very similar to that for systems governed by partial differential equations.

We shall not attempt to cover all possible cases in one theorem. Our approach will be to reduce the problem to the control with full observation of a stochastic PDE, namely the Zakai equation. This equation will be formulated as a differential equation in a Hilbert space, using variational techniques (see Section 4.7). In this framework, it is convenient to use variational techniques to derive necessary conditions of optimality. One advantage of this approach is that it is mostly analytic. On the other hand, the case of unbounded coefficients (for instance the linear quadratic case) is not easily covered in this formulation, without substantial technical transformations. However, the result is formally applicable without any difficulty.

The present developments improve previous results of mine (see Bensoussan 1983 and simplify the derivation. Besides the application to the LQG problem, we consider the situation of the separation principle and obtain it in a new way, through the stochastic maximum principle.

The problem of stochastic control of partially observable systems plays an important role in many applications. All real problems are in fact of this type, and deterministic control as well as stochastic control with full observation can only be approximations of the real world. This justifies the importance of having a theory as complete as possible, which can be used for numerical implementation.

In the first three chapters of this book we study problems which can be dealt with directly by algebraic manipulations, without using the complete theory. This is because the system and the observation have linear dynamics, and the cost is either quadratic or the exponential of a quadratic functional.

In Chapters 4 to 6, we present the theory of non linear filtering, which is the basic step in formulating the control problem adequately. The main difficulty, especially from the point of view of numerical applications, is that there are no statistics which are finite dimensional, and the basic object to be computed is the conditional probability. This is the solution of a stochastic partial differential equation (PDE) studied in Chapter 4. Chapters 5 and 6 are devoted to approximations, when perturbation methods are applicable, or to particular cases when simplification occurs, and sufficient statistics exist.

In Chapter 7, we study stochastic control problems with partial information, in an intermediate case, namely when the direct methods of Chapters 1, 2, 3 are not applicable yet the full theory is not necessary, either because finite dimensional sufficient statistics are available, or approximations are possible.

Introduction

In Chapter 2 (see comments) we have seen the concepts of certainty equivalence and the separation principle. Again the main idea is that an optimal control for a stochastic control problem with partial observation can be obtained by a feedback rule (deterministic of course) applied on the Kalman filter (the best estimate of the state). As we shall see in later chapters, this situation is by no means general. The ‘sufficient statistics’, to which one applies a feedback rule, are in general infinite dimensional (as will be seen it is the conditional probability distribution), and thus does not reduce to a single moment, the conditional mean. Some natural questions arise at this stage. Can we find examples in which the sufficient statistics are finite dimensional, for instance several moments (conditional mean and conditional variance)? Note that if this holds, then the dimension of the sufficient statistics, although finite, is larger than that of the state. In this chapter we will study a different situation. We shall meet a situation where the optimal control is given by feedback on sufficient statistics, which are finite dimensional, with a dimension which is the same as that of the state, but does not coincide with the conditional mean, the best estimate of the state. It is clearly a situation where the separation principle does not hold, but from a computational viewpoint offers the same simplicity as that when the separation principle does hold.

The situation that we consider here is naturally very specific again, and corresponds to linear dynamics, with linear observation, and a payoff which is the exponential of the standard quadratic functional.