Skip to main content Accessibility help

Optimal control of a class of piecewise deterministic processes

  • M. ANNUNZIATO (a1) and A. BORZÌ (a2)


A new control strategy for a class of piecewise deterministic processes (PDP) is presented. In this class, PDP stochastic processes consist of ordinary differential equations that are subject to random switches corresponding to a discrete Markov process. The proposed strategy aims at controlling the probability density function (PDF) of the PDP. The optimal control formulation is based on the hyperbolic Fokker–Planck system that governs the time evolution of the PDF of the PDP and on tracking objectives of terminal configuration with a target PDF. The corresponding optimization problems are formulated as a sequence of open-loop hyperbolic optimality systems following a model predictive control framework. These systems are discretized by first-order schemes that guarantee positivity and conservativeness of the numerical PDF solution. The effectiveness of the proposed computational control framework is validated considering PDP with dichotomic noise.



Hide All
[1]Amudevar, A. (2001) A dynamic programming algorithm for the optimal control of piecewise deterministic Markov processes. SIAM J. Control Optim. 40, 525539.
[2]Annunziato, M. (2002) Non-Gaussian equilibrium distributions arising from the Langevin equation. Phys. Rev. E 65, 21113 (1–6).
[3]Annunziato, M. (2007) A finite difference method for piecewise deterministic processes with memory. Math. Mod. Anal. 12, 157178.
[4]Annunziato, M. (2008) Analysis of upwind method for piecewise deterministic Markov processes. Comp. Meth. Appl. Math. 8, 320.
[5]Annunziato, M. (2012) On the action of a semi-Markov process on a system of ordinary differential equations. Math. Mod. Anal. 17, 650672.
[6]Annunziato, M. & Borzì, A. (2010) Optimal control of probability density functions of stochastic processes. Math. Mod. Anal. 15, 393407.
[7]Annunziato, M. & Borzì, A. (2013) A Fokker–Planck control framework for multidimensional stochastic processes. J. Comput. Appl. Math. 237, 487507.
[8]Annunziato, M., Grigolini, P. & West, B. J. (2001) Canonical and noncanonical equilibrium distribution. Phys. Rev. E 66, 011107 (1–13).
[9]Bäurle, N. & Rieder, U. (2009) MDP algorithms for portfolio optimization problems in pure jump markets. Finance Stoch. 13, 591611.
[10]Bect, J. (2010) A unifying formulation of the Fokker–Planck–Kolmogorov equation for general stochastic hybrid systems. Nonlinear Anal.: Hybrid Syst. 4, 357370.
[11]Bertsekas, D. P. (2005) Dynamic Programming and Optimal Control, Athena Scientific, Belmont, MA.
[12]Borzì, A. & Schulz, V. (2012) Computational optimization of systems governed by partial differential equations. SIAM Book Series on Computational Science and Engineering 08, SIAM, Philadelphia, PA.
[13]Cassandras, C. G. & Lygeros, J. (2007) Stochastic Hybrid Systems, CRC Press Taylor & Francis Group, Boca Raton, FL.
[14]Chiquet, J., Limnios, N. & Eid, M. (2009) Piecewise deterministic Markov processes applied to fatigue crack growth modelling. J. Stat. Plan. Inference (Special Issue), 139, 16571667.
[15]Choo, K. G., Teo, K. L. & Wu, Z. S. (1981/1982) On an optimal control problem involving first order hyperbolic systems with boundary controls. Numer. Funct. Anal. Optim., 4, 171190.
[16]Cocozza-Thivent, C., Eymard, R. & Mercier, S. (2006) A finite volume scheme for dynamic reliability models. IMA J. Numer. Anal. 26, 446471.
[17]Cocozza-Thivent, C., Eymard, R., Mercier, S. & Roussignol, M. (2006) Characterization of the marginal distributions of Markov processes used in dynamic reliability. J. Appl. Math. Stoch. Anal. article no. 92156, 118.
[18]Costa, O. L. V. & Dufour, F. (2003) On the Poisson equation for piecewise-deterministic Markov processes. SIAM J. Control Optim. 42, 9851001.
[19]Costa, O. L. V. & Dufour, F. (2010) Average continuous control of piecewise deterministic Markov processes. SIAM J. Control Optim. 48, 42624291.
[20]Cox, D. R. & Miller, H. D. (2001) The Theory of Stochastic Processes, Chapman & Hall CRC, Boca Raton, FL.
[21]Davis, M. H. A. (1984) Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J. R. Stat. Soc. B, 46, 353388.
[22]Dempster, M. A. H. & Ye, J. J. (1995) Impulse control of piecewise deterministic Markov processes. Ann. Appl. Probab. 5, 399423.
[23]Ebeling, W., Gudowska-Nowak, E. & Sokolov, I. M. (2008) On stochastic dynamics in physics—Remarks on history and terminology. Acta Phys. Pol., 39, 10031018.
[24]Evans, L. C. (2002) Partial Differential Equations, Graduate Studies in Mathematics, Vol. 19, American Mathematical Society, Providence, RI.
[25]Eymard, R., Mercier, S. & Prignet, A. (2008) An implicit finite volume scheme for a scalar hyperbolic problem with measure data related to piecewise deterministic Markov processes. J. Comput. Appl. Math. 222, 293323.
[26]Faggionato, A., Gabrielli, D. & Ribezzi Crivellari, M. (2009) Non-equilibrium thermodynamics of piecewise deterministic Markov processes J. Stat. Phys. 137, 259304.
[27]Filliger, R. & Hongler, M. O. (2004) Supersymmetry in random two-velocity processes. Physica. A 332, 141150.
[28]Gilbert, J. C. & Nocedal, J. (1992) Global convergence properties of conjugate gradient methods for optimization. SIAM J. Opt. 2, 2142.
[29]Grüne, L. & Pannek, J. (2011) Nonlinear Model Predictive Control, Theory and Algorithms, Communications and Control Engineering, Springer Verlag, London/Dordrecht/Heidelberg/New York.
[30]Gustafsson, M. & Holmgren, S. (2010) An implementation framework for solving high-dimensional PDEs on massively parallel computers. In: Kreiss, al. (editors), Numerical Mathematics and Advanced Applications 2009, Springer-Verlag, Berlin–Heidelberg, pp. 417424.
[31]Hillen, T. & Othmer, H. G. (2000) The diffusion limit of transport equations derived from velocity-jump processes. SIAM J. Appl. Math. 61, 751775.
[32]Horsthemke, H. (1999) Spatial instabilities in reaction random walks with direction-independent kinetics. Phys. Rev E 60, 26512663.
[33]Ito, K. & Kunisch, K. (1990) Receding horizon optimal control for infinite dimensional systems. ESAIM: Control Optim. Calculus Var. 35, 814824.
[34]Kittel, C. (2004) Elementary Statistical Physics, Dover Publications, New York/London.
[35]Lax, P. D. (2006) Hyperbolic Partial Differential Equations—Courant Lecture Notes in Mathematics, Courant Institute of Mathematical Sciences, American Mathematical Society, Providence, RI.
[36]Lions, J. L., (1971) Optimal Control of Systems Governed by Partial Differential Equations, Springer, Berlin.
[37]Magni, L., Raimondo, D. M. & Allgöwer, F. (2009) Nonlinear Model Predictive Control, Springer, Berlin.
[38]Maurer, H. & Zowe, J. (1979) First- and second-order necessary and sufficient optimality conditions for infinite-dimensional programming problems. Math. Program. 16, 98110.
[39]Mayne, D. Q. & Michalska, H. (1990) Receding horizon control for nonlinear systems. IEEE Trans. Aut. Control 35, 814–824.
[40]Mil'shtein, G. N. & Repin, Y. M. (1972) Action of a Markov process on a system of differential equations. Differ. Equ. (translated from Russian), 5, 10101019.
[41]Moresino, F., Pourtallier, O. & Tidball, M. (1999) Using Viscosity Solution for Approximation in Piecewise Deterministic Control Systems, Unité de recherche INRIA Sophia Antipolis: Rapport de recherche n. 3687.
[42]Morita, A. (1990) Free Brownian motion of a particle driven by a dichotomous random force. Phys. Rev A 41, 754760.
[43]Morton, K. W. & Mayers, D. F. (2005) Numerical Solution of Partial Differential Equations: An Introduction, Cambridge University Press, Cambridge, UK.
[44]Ou, Y. & Schuster, E. (2010) On the stability of receding horizon control of bilinear parabolic PDE systems. In: Proceedings of the 2010 IEEE Conference on Decision and Control, 15–17 December, 2010, Atlanta, Georgia.
[45]Pawula, R. F. (1977) The probability density and level-crossings of first-order non-linear systems driven by the random telegraph signal. Int. J. Control 25, 283292.
[46]Pawula, R. F. & Rice, O. (1986) On filtered binary processes. IEEE Trans. Inf. Th. IT-32, 6372.
[47]Primak, S., Kontorovich, V. & Lyandres, V. (2004) Stochastic Methods and Their Applications to Communications, John Wiley & Sons, Chichester.
[48]Risken, R. (1996) The Fokker–Planck Equation: Methods of Solution and Applications, Springer, Berlin.
[49]Shanno, D. F. (1978) Conjugate gradient methods with inexact searches. Math. Oper. Res. 3, 244256.
[50]Zubair, H.Bin, , Oosterlee, C. C. & Wienands, R. (2007) Multigrid for high-dimensional elliptic partial differential equations on non-equidistant grid. SIAM J. Sci. Comput. 29, 16131636.



Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed