So far we have considered only differential games in which the fundamentals (utility functions, system dynamics, initial state, etc.) do not contain any uncertainty. For some applications this is quite satisfactory, but for others it presents a severe limitation. We now take a look at stochastic optimal control problems and differential games in which some of the fundamentals involve random variables or stochastic processes. Uncertainty can be incorporated into our basic framework in many different ways, so we shall not give a complete treatment of this matter but concentrate on two forms of stochastic models which seem to be most useful in applications. The first is based on so-called piecewise deterministic processes whereas the second makes use of the Wiener process. The notion of piecewise deterministic processes captures the idea that uncertain changes in the system occur at discrete (but random) time instants. Between these so-called jump times, the system evolves in a deterministic way. The Wiener process, on the other hand, is used to model situations characterized by continuous stochastic noise.

Piecewise deterministic games

A piecewise deterministic control model

A piecewise deterministic process is a system which evolves in a deterministic way, except at certain jump times T1, T2, … at which the deterministic law of motion switches from one mode to another. Both the jump times Tl and the system modes which govern the motion between jump times are randomly selected.