Skip to main content Accessibility help
×
  • Cited by 33
Publisher:
Cambridge University Press
Online publication date:
December 2011
Print publication year:
2011
Online ISBN:
9781139003605

Book description

This comprehensive volume on ergodic control for diffusions highlights intuition alongside technical arguments. A concise account of Markov process theory is followed by a complete development of the fundamental issues and formalisms in control of diffusions. This then leads to a comprehensive treatment of ergodic control, a problem that straddles stochastic control and the ergodic theory of Markov processes. The interplay between the probabilistic and ergodic-theoretic aspects of the problem, notably the asymptotics of empirical measures on one hand, and the analytic aspects leading to a characterization of optimality via the associated Hamilton–Jacobi–Bellman equation on the other, is clearly revealed. The more abstract controlled martingale problem is also presented, in addition to many other related issues and models. Assuming only graduate-level probability and analysis, the authors develop the theory in a manner that makes it accessible to users in applied mathematics, engineering, finance and operations research.

Reviews

‘Assuming good knowledge in analysis, probability theory and stochastic processes, [this book provides] a careful and comprehensive treatment of ergodic control of diffusion processes.'

Kurt Marti Source: Zentralblatt MATH

Refine List

Actions for selected content:

Select all | Deselect all
  • View selected items
  • Export citations
  • Download PDF (zip)
  • Save to Kindle
  • Save to Dropbox
  • Save to Google Drive

Save Search

You can save your searches here and later view and run them again in "My saved searches".

Please provide a title, maximum of 40 characters.
×

Contents

References
[1] Agmon, S., Douglis, A., and Nirenberg, L. 1959. Estimates near the boundary for solutions of elliptic partial differential equations satisfying general boundary conditions. I. Comm. Pure Appl. Math., 12, 623–727.
[2] Allinger, D. F. and Mitter, S. K. 1980. New results on the innovations problem for nonlinear filtering. Stochastics, 4(4), 339–348.
[3] Anderson, E. J. and Nash, P. 1987. Linear Programming in Infinite-Dimensional Spaces. Wiley-Interscience Series in Discrete Mathematics and Optimization. Chichester: John Wiley & Sons.
[4] Arapostathis, A. and Borkar, V. S. 2010. Uniform recurrence properties of controlled diffusions and applications to optimal control. SIAM J. Control Optim., 48(7), 152–160.
[5] Arapostathis, A. and Ghosh, M. K. 2004 (Dec). Ergodic control of jump diffusions in ℝd under a near-monotone cost assumption. Pages 4140–4145 of:43rd IEEE Conference on Decision and Control, vol. 4.
[6] Arapostathis, A., Borkar, V. S., Fernández-Gaucherand, E., Ghosh, M. K., and Marcus, S. I. 1993. Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control Optim., 31(2), 282–344.
[7] Arapostathis, A., Ghosh, M. K., and Marcus, S. I. 1999. Harnack's inequality for cooperative, weakly coupled elliptic systems. Comm. Partial Differential Equations, 24, 1555–1571.
[8] Arisawa, M., and Lions, P.-L. 1998. On ergodic stochastic control. Comm. Partial Differential Equations, 23(11–12), 333–358.
[9] Arrow, K. J., Barankin, E. W., and Blackwell, D. 1953. Admissible points of convex sets. Pages 87–91 of: Contributions to the Theory of Games, vol. 2. Annals of Mathematics Studies, no. 28. Princeton, NJ: Princeton University Press.
[10] Bachelier, L. 2006. Louis Bachelier's Theory of Speculation: The Origins of Modern Finance. Princeton, NJ: Princeton University Press. Translated and with a commentary by Mark Davis and Alison Etheridge.
[11] Basak, G. K., Borkar, V. S., and Ghosh, M. K. 1997. Ergodic control of degenerate diffusions. Stochastic Anal. Appl., 15(1), 1–17.
[12] Bass, R. F. 1998. Diffusions and Elliptic Operators. Probability and its Applications. New York: Springer-Verlag.
[13] Beneš, V. E. 1970. Existence of optimal strategies based on specified information, for a class of stochastic decision problems. SIAM J. Control, 8, 179–188.
[14] Bensoussan, A. 1982. Stochastic Control by Functional Analysis Methods. Studies in Mathematics and its Applications, vol. 11. Amsterdam: North-Holland Publishing Co.
[15] Bensoussan, A. and Borkar, V. 1984. Ergodic control problem for one-dimensional diffusions with near-monotone cost. Systems Control Lett., 5(2), 127–133.
[16] Bensoussan, A. and Borkar, V. 1986. Corrections to: “Ergodic control problem for onedimensional diffusions with near-monotone cost” [Systems Control Lett. 5(1984), no. 2, 127–133]. Systems Control Lett., 7(3), 233–235.
[17] Bensoussan, A. and Frehse, J. 1992. On Bellman equations of ergodic control in Rn. J. Reine Angew. Math., 429, 125–160.
[18] Bensoussan, A. and Frehse, J. 2002. Ergodic control Bellman equation with Neumann boundary conditions. Pages 59–71 of: Stochastic Theory and Control (Lawrence, KS, 2001). Lecture Notes in Control and Inform. Sci., vol. 280. Berlin: Springer.
[19] Bertoin, J. 1996. Lévy Processes. Cambridge Tracts in Mathematics, vol. 121. Cambridge: Cambridge University Press.
[20] Bertsekas, D. P. and Shreve, S. E. 1978. Stochastic Optimal Control: The Discrete Time Case. New York: Academic Press.
[21] Bhatt, A. G. and Borkar, V. S. 1996. Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab., 24(3), 1531–1562.
[22] Bhatt, A. G. and Borkar, V. S. 2005. Existence of optimal Markov solutions for ergodic control of Markov processes. Sankhyā, 67(1), 1–18.
[23] Bhatt, A. G. and Karandikar, R. L. 1993. Invariant measures and evolution equations for Markov processes characterized via martingale problems. Ann. Probab., 21(4), 2246–2268.
[24] Billingsley, P. 1968. Convergence of Probability Measures. New York: John Wiley & Sons.
[25] Billingsley, P. 1995. Probability and Measure. Third edition. Wiley Series in Probability and Mathematical Statistics. New York: John Wiley & Sons.
[26] Bogachev, V. I., Krylov, N. V., and Röckner, M. 2001. On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions. Comm. Partial Differential Equations, 26(11–12), 2037–2080.
[27] Bogachev, V. I., Rökner, M., and Stannat, V. 2002. Uniqueness of solutions of elliptic equations and uniqueness of invariant measures of diffusions. Mat. Sb., 193(7), 3–36.
[28] Borkar, V. S. 1989a. Optimal Control of Diffusion Processes. Pitman Research Notes in Mathematics Series, vol. 203. Harlow: Longman Scientific & Technical.
[29] Borkar, V. S. 1989b. A topology for Markov controls. Appl. Math. Optim., 20(1), 55–62.
[30] Borkar, V. S. 1991. On extremal solutions to stochastic control problems. Appl. Math. Optim., 24(3), 317–330.
[31] Borkar, V. S. 1993. Controlled diffusions with constraints. II. J. Math. Anal. Appl., 176(2), 310–321.
[32] Borkar, V. S. 1995. Probability Theory: An Advanced Course. New York: Springer-Verlag.
[33] Borkar, V. S. 2003. Dynamic programming for ergodic control with partial observations. Stochastic Process. Appl., 103(2), 293–310.
[34] Borkar, V. S. and Budhiraja, A. 2004a. Ergodic control for constrained diffusions: characterization using HJB equations. SIAM J. Control Optim., 43(4), 1467–1492.
[35] Borkar, V. S. and Budhiraja, A. 2004b. A further remark on dynamic programming for partially observed Markov processes. Stochastic Process. Appl., 112(1), 79–93.
[36] Borkar, V. S. and Gaitsgory, V. 2007. Singular perturbations in ergodic control of diffusions. SIAM J. Control Optim., 46(5), 1562–1577.
[37] Borkar, V. S. and Ghosh, M. K. 1988. Ergodic control of multidimensional diffusions. I. The existence results. SIAM J. Control Optim., 26(1), 112–126.
[38] Borkar, V. S. and Ghosh, M. K. 1990a. Controlled diffusions with constraints. J. Math. Anal. Appl., 152(1), 88–108.
[39] Borkar, V. S. and Ghosh, M. K. 1990b. Ergodic control of multidimensional diffusions. II. Adaptive control. Appl. Math. Optim., 21(2), 191–220.
[40] Borkar, V. S. and Ghosh, M. K. 2003. Ergodic control of partially degenerate diffusions in a compact domain. Stochastics, 75(4), 221–231.
[41] Borkar, V. S. and Mitter, S. K. 2003. A note on stochastic dissipativeness. Pages 41– 49 of: Directions in mathematical systems theory and optimization. Lecture Notes in Control and Inform. Sci., vol. 286. Berlin: Springer.
[42] Busca, J. and Sirakov, B. 2004. Harnack type estimates for nonlinear elliptic systems and applications. Ann. Inst. H. Poincaré Anal. Non Linéaire, 21(5), 543–590.
[43] Chen, Y.-Z. and Wu, L.-C. 1998. Second Order Elliptic Equations and Elliptic Systems. Translations of Mathematical Monographs, vol. 174. Providence, RI: American Mathematical Society. Translated from the 1991 Chinese original by Bei Hu.
[44] Choquet, G. 1969. Lectures on Analysis. Vol. II: Representation Theory. Edited by Marsden, J., Lance, T. and Gelbart, S.. New York: W. A. Benjamin.
[45] Chung, K. L. 1982. Lectures from Markov Processes to BrownianMotion. Grundlehren der Mathematischen Wissenschaften, vol. 249. New York: Springer-Verlag.
[46] Crandall, M. G., Kocan, M., and Święch, A. 2000. Lp-theory for fully nonlinear uniformly parabolic equations. Comm. Partial Differential Equations, 25(11–12), 1997–2053.
[47] Dellacherie, C. and Meyer, P. 1978. Probabilities and Potential A. North-Holland Mathematics Studies, vol. 29. Amsterdam: North-Holland.
[48] Dubins, L. 1962. On extreme points of convex sets. J. Math. Anal. Appl., 5, 237–244.
[49] Dubins, L. and Freedman, D. 1964. Measurable sets of measures. Pacific J. Math., 14, 1211–1222.
[50] Dudley, R. M. 2002. Real Analysis and Probability. Cambridge Studies in Advanced Mathematics, vol. 74. Cambridge: Cambridge University Press.
[51] Dunford, N. and Schwartz, J. T. 1988. Linear Operators. Part I. Wiley Classics Library. New York: John Wiley & Sons.
[52] Dupuis, P. and Ishii, H. 1991. On Lipschitz continuity of the solution mapping to the Skorohod problem with applications. Stochastics, 35, 31–62.
[53] Dynkin, E. B. and Yushkevich, A. A. 1979. Controlled Markov Processes. Grundlehren der Mathematischen Wissenschaften, vol. 235. Berlin: Springer-Verlag.
[54] El Karoui, N., Nguyen, D. H., and Jeanblanc-Picqué, M. 1987. Compactification methods in the control of degenerate diffusions: existence of an optimal control. Stochastics, 20(3), 169–219.
[55] Ethier, S. N. and Kurtz, T. G. 1986. Markov Processes. Wiley Series in Probability and Mathematical Statistics: Probability and Mathematical Statistics. New York: John Wiley & Sons.
[56] Fabes, E. B. and Kenig, C. E. 1981. Examples of singular parabolic measures and singular transition probability densities. Duke Math. J., 48(4), 845–856.
[57] Feller, W. 1959. Non-Markovian processes with the semigroup property. Ann. Math. Statist., 30, 1252–1253.
[58] Fleming, W. H. and Rishel, R. W. 1975. Deterministic and Stochastic Optimal Control. Berlin: Springer-Verlag.
[59] Fleming, W. H. 1980. Measure-valued processes in the control of partially-observable stochastic systems. Appl. Math. Optim., 6(3), 271–285.
[60] Fleming, W. H., and Pardoux, E. 1982. Optimal control for partially observed diffusions. SIAM J. Control Optim., 20(2), 261–285.
[61] Freidlin, M. I. 1963. Diffusion processes with reflection and a directional derivative problem on a manifold with boundary. Theory Probab. Appl., 8(1), 75–83.
[62] Friedman, A. 1964. Partial Differential Equations of Parabolic Type. Englewood Cliffs, NJ: Prentice-Hall.
[63] Friedman, A. 2006. Stochastic Differential Equations and Applications. Mineola, NY: Dover Publications.
[64] Ghosh, M. K., Arapostathis, A., and Marcus, S. I. 1993. Optimal control of switching diffusions with application to flexible manufacturing systems. SIAM J. Control Optim., 31(5), 1183–1204.
[65] Ghosh, M. K., Arapostathis, A., and Marcus, S. I. 1997. Ergodic control of switching diffusions. SIAM J. Control Optim., 35(6), 1952–1988.
[66] Gikhman, I. I. and Skorokhod, A. V. 1969. Introduction to the Theory of Random Processes. Translated from the Russian by Scripta Technica, Inc. Philadelphia, PA: W. B. Saunders.
[67] Gilbarg, D. and Trudinger, N. S. 1983. Elliptic Partial Differential Equations of Second Order. Second edition. Grundlehren der Mathematischen Wissenschaften, vol. 224. Berlin: Springer-Verlag.
[68] Gyöngy, I. and Krylov, N. 1996. Existence of strong solutions for Itô's stochastic equations via approximations. Probab. Theory Related Fields, 105(2), 143–158.
[69] Has′minskiĭ, R. Z. 1960. Ergodic properties of recurrent diffusion processes and stabilization of the solution of the Cauchy problem for parabolic equations. Theory Probab. Appl., 5(2), 179–196.
[70] Has'minskiĭ, R. Z. 1980. Stochastic Stability of Differential Equations. The Netherlands: Sijthoff & Noordhoff.
[71] Haussmann, U. G. 1985. L'équation de Zakai et le problème séparè du contrôle optimal stochastique. Pages 37–62 of: Séminaire de Probabilités, XIX, 1983/84. Lecture Notes in Math., vol. 1123. Berlin: Springer.
[72] Haussmann, U. G. 1986. Existence of optimal Markovian controls for degenerate diffusions. Pages 171–186 of: Stochastic Differential Systems (Bad Honnef, 1985). Lecture Notes in Control and Information Science, vol. 78. Berlin: Springer.
[73] Hernández-Lerma, O. 1989. Adaptive Markov Control Processes. Applied Mathematical Sciences, vol. 79. New York: Springer-Verlag.
[74] Ikeda, N. and Watanabe, S. 1989. Stochastic Differential Equations and Diffusion Processes. Second edition. North-Holland Mathematical Library, vol. 24. Amsterdam: North-Holland Publishing.
[75] Kallenberg, L. C. M. 1983. Linear Programming and Finite Markovian Control Problems. Mathematical Centre Tracts, vol. 148. Amsterdam: Mathematisch Centrum.
[76] Karatzas, I. and Shreve, S. E. 1991. Brownian Motion and Stochastic Calculus. Second edition. Graduate Texts in Mathematics, vol. 113. New York: Springer-Verlag.
[77] Kogan, Ya. A. 1969. The optimal control of a non-stopping diffusion process with reflection. Theory Probab. Appl., 14(3), 496–502.
[78] Krylov, N. V. 1980. Controlled Diffusion Processes. Applications of Mathematics, vol. 14. New York: Springer-Verlag. Translated from the Russian by A. B. Aries.
[79] Krylov, N. V. 1995. Introduction to the Theory of Diffusion Processes. Translations of Mathematical Monographs, vol. 142. Providence, RI: American Mathematical Society.
[80] Kunita, H. 1981. Some extensions of Itô's formula. Pages 118–141 of: Seminar on Probability, XV (Univ. Strasbourg, Strasbourg, 1979/1980) (French). Lecture Notes in Math., vol. 850. Berlin: Springer.
[81] Kurtz, T. G. and Stockbridge, R. H. 1998. Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim., 36(2), 609–653.
[82] Kurtz, T. G. and Stockbridge, R. H. 2001. Stationary solutions and forward equations for controlled and singular martingale problems. Electron. J. Probab., 6, no. 17, 52 pp. (electronic).
[83] Ladyženskaja, O. A., Solonnikov, V. A., and Ural′ceva, N. N. 1967. Linear and Quasilinear Equations of Parabolic Type. Translated from the Russian by Smith, S.. Translations of Mathematical Monographs, Vol. 23. Providence, RI: American Mathematical Society.
[84] Lions, P.-L. 1983a. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. I. The dynamic programming principle and applications. Comm. Partial Differential Equations, 8(10), 1101–1174.
[85] Lions, P.-L. 1983b. Optimal control of diffusion processes and Hamilton–Jacobi–Bellman equations. II. Viscosity solutions and uniqueness. Comm. Partial Differential Equations, 8(11), 1229–1276.
[86] Lions, P. L. and Sznitman, A. S. 1984. Stochastic differential equations with reflecting boundary conditions. Comm. Pure Appl. Math., 37, 511–537.
[87] Liptser, R. S. and Shiryayev, A. N. 1977. Statistics of Random Processes. I. Applications of Mathematics, Vol. 5. New York: Springer-Verlag. Translated by A. B. Aries.
[88] Luenberger, D. G. 1967. Optimization by Vector Space Methods. New York: John Wiley & Sons.
[89] Menaldi, J.-L. and Robin, M. 1997. Ergodic control of reflected diffusions with jumps. Appl. Math. Optim., 35(2), 117–137.
[90] Menaldi, J.-L. and Robin, M. 1999. On optimal ergodic control of diffusions with jumps. Pages 439–456 of: Stochastic Analysis, Control, Optimization and Applications. Systems Control Found. Appl. Boston, MA: Birkhäuser Boston.
[91] Meyn, S. and Tweedie, R. L. 2009. Markov Chains and Stochastic Stability. Second edition. Cambridge: Cambridge University Press.
[92] Nadirashvili, N. 1997. Nonuniqueness in the martingale problem and the Dirichlet problem for uniformly elliptic operators. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 24(3), 537–549.
[93] Neveu, J. 1965. Mathematical Foundations of the Calculus of Probability. San Francisco, CA: Holden-Day.
[94] Parthasarathy, K. R. 1967. Probability Measures on Metric Spaces. Probability and Mathematical Statistics, No. 3. New York: Academic Press.
[95] Phelps, R. 1966. Lectures on Choquet's Theorem. New York: Van Nostrand.
[96] Portenko, N. I. 1990. Generalized Diffusion Processes. Translations of Mathematical Monographs, vol. 83. Providence, RI: American Mathematical Society. Translated from the Russian by H. H. McFaden.
[97] Puterman, M. I. 1994. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Hoboken, NJ: John Wiley & Sons.
[98] Rachev, S. T. 1991. Probability Metrics and the Stability of Stochastic Models. Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics. Chichester: John Wiley & Sons.
[99] Rishel, R. 1970. Necessary and sufficient dynamic programming conditions for continuous time stochastic control problem. SIAM J. Control, 8, 559–571.
[100] Robin, M. 1983. Long-term average cost control problems for continuous time Markov processes: a survey. Acta Appl. Math., 1(3), 281–299.
[101] Rockafellar, R. T. 1946. Convex Analysis. Princeton Mathematical Series, vol. 28. Princeton, NJ: Princeton University Press.
[102] Rogers, L. C. G., and Williams, D. 2000a. Diffusions, Markov Processes, and Martingales. Vol. 1. Cambridge Mathematical Library. Cambridge: Cambridge University Press.
[103] Rogers, L. C. G. and Williams, D. 2000b. Diffusions, Markov Processes, and Martingales. Vol. 2. Cambridge Mathematical Library. Cambridge: Cambridge University Press.
[104] Ross, S. M. 1970. Average cost semi-Markov decision processes. J. Appl. Probability, 7, 649–656.
[105] Rudin, W. 1973. Functional Analysis. New York: McGraw-Hill.
[106] Rugh, W. J. 1996. Linear System Theory. Second edition. Prentice Hall Information and System Sciences Series. Englewood Cliffs, NJ: Prentice Hall.
[107] Safonov, M. V. 1999. Nonuniqueness for second-order elliptic equations with measurable coefficients. SIAM J. Math. Anal., 30(4), 879–895 (electronic).
[108] Skorohod, A. V. 1989. Asymptotic Methods in the Theory of Stochastic Differential Equations. Translations of Mathematical Monographs, vol. 78. Providence, RI: American Mathematical Society.
[109] Stannat, W. 1999. (Nonsymmetric) Dirichlet operators on L1: existence, uniqueness and associated Markov processes. Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4), 28(1), 99–140.
[110] Stockbridge, R. H. 1989. Time-average control of martingale problems: the Hamilton–Jacobi–Bellman equation. Stochastics Stochastics Rep., 27(4), 249–260.
[111] Stockbridge, R. H. 1990a. Time-average control of martingale problems: a linear programming formulation. Ann. Probab., 18(1), 206–217.
[112] Stockbridge, R. H. 1990b. Time-average control of martingale problems: existence of a stationary solution. Ann. Probab., 18(1), 190–205.
[113] Striebel, C. 1984. Martingale methods for the optimal control of continuous time stochastic systems. Stoch. Process. Appl., 18, 324–347.
[114] Stroock, D. W. and Varadhan, S. R. S. 1971. Diffusion processes with boundary conditions. Comm. Pure Appl. Math., 24, 147–225.
[115] Stroock, D. W. and Varadhan, S. R. S. 1979. Multidimensional Diffusion Processes. Grundlehren der Mathematischen Wissenschaften, vol. 233. Berlin: Springer-Verlag.
[116] Veretennikov, A. Yu. 1980. Strong solutions and explicit formulas for solutions of stochastic integral equations. Mat. Sb. (N.S.), 111(153)(3), 434–452, 480.
[117] Veretennikov, A. Yu. 1987. On strong solutions of stochastic Itô equations with jumps. Theory Probab. Appl., 32(1), 148–152.
[118] Wagner, D. H. 1977. Survey of measurable selection theorems. SIAM J. Control Optim., 15, 859–903.
[119] Walters, P. 1982. An Introduction to Ergodic Theory. Graduate Texts in Mathematics, vol. 79. New York: Springer-Verlag.
[120] Willems, J. C. 1972. Dissipative dynamical systems. I. General theory. Arch. Rational Mech. Anal., 45, 321–351.
[121] Wong, E. 1971. Representation of martingales, quadratic variation and applications. SIAM J. Control, 9, 621–633.
[122] Wong, E. and Hajek, B. 1985. Stochastic Processes in Engineering Systems. Springer Texts in Electrical Engineering. New York: Springer-Verlag.
[123] Wu, W., Arapostathis, A., and Shakkottai, S. 2006. Optimal power allocation for a timevarying wireless channel under heavy-traffic approximation. IEEE Trans. Automat. Control, 51(4), 580–594.
[124] Xiong, J. 2008. An Introduction to Stochastic Filtering Theory. Oxford Graduate Texts in Mathematics, vol. 18. Oxford: Oxford University Press.
[125] Yosida, K. 1980. Functional Analysis. Sixth edition. Grundlehren der Mathematischen Wissenschaften, vol. 123. Berlin: Springer-Verlag.
[126] Young, L. C. 1969. Lectures on the Calculus of Variations and Optimal Control Theory. Foreword by Wendell Fleming, H.. Philadelphia: W. B. Saunders.
[127] Zvonkin, A. K. 1974. A transformation of the phase space of a diffusion process that will remove the drift. Mat. Sb. (N.S.), 93(135), 129–149, 152.

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Book summary page views

Total views: 0 *
Loading metrics...

* Views captured on Cambridge Core between #date#. This data will be updated every 24 hours.

Usage data cannot currently be displayed.