Hostname: page-component-76fb5796d-5g6vh Total loading time: 0 Render date: 2024-04-28T10:14:51.439Z Has data issue: false hasContentIssue false
  • English
  • Français

Nonlinear Markov chains with finite state space: invariant distributions and long-term behaviour

Published online by Cambridge University Press:  22 September 2022

Berenice Anne Neumann Open the ORCID record for Berenice Anne Neumann [Opens in a new window]
Berenice Anne Neumann*
Affiliation:
University of Trier
*
*Postal address: University of Trier, Department IV, Universitätsring 19, 54296 Trier, Germany. Email address: neumannb@uni-trier.de
Get access Rights & Permissions [Opens in a new window]

Abstract

Nonlinear Markov chains with finite state space were introduced by Kolokoltsov (Nonlinear Markov Processes and Kinetic Equations, 2010). The characteristic property of these processes is that the transition probabilities depend not only on the state, but also on the distribution of the process. Here we provide first results regarding their invariant distributions and long-term behaviour: we show that under a continuity assumption an invariant distribution exists and provide a sufficient criterion for the uniqueness of the invariant distribution. Moreover, we present examples of peculiar limit behaviour that cannot occur for classical linear Markov chains. Finally, we present for the case of small state spaces sufficient (and easy-to-verify) criteria for the ergodicity of the process.

Keywords

Nonlinear Markov chaininvariant distributionlong-term behaviourergodicity
Type
Original Article
Information
Journal of Applied Probability , Volume 60 , Issue 1 , March 2023 , pp. 30 - 44
Check for updates
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Asmussen, S. (2003). Applied Probability and Queues. Springer, New York.Google Scholar
Aubin, J.-P. and Cellina, A. (1984). Differential Inclusions: Set-Valued Maps and Viability Theory. Springer, Berlin, Heidelberg.CrossRefGoogle Scholar
Bather, J. A. (1963). Two non-linear birth and death processes. J. Austral. Math. Soc. 3, 104–116.CrossRefGoogle Scholar
Butkovsky, O. A. (2014). On ergodic properties of nonlinear Markov chains and stochastic McKean–Vlasov equations. Theory Prob. Appl. 58, 661–674.CrossRefGoogle Scholar
Carmona, R. and Delarue, F. (2015). Forward–backward stochastic differential equations and controlled McKean–Vlasov dynamics. Ann. Prob. 43, 2647–2700.CrossRefGoogle Scholar
Deimling, K. (1985). Nonlinear Functional Analysis. Springer, Berlin.CrossRefGoogle Scholar
Fernandes, M. L. and Zanolin, F. (1987). Remarks on strongly flow-invariant sets. J. Math. Anal. Appl. 128, 176–188.CrossRefGoogle Scholar
Iosifescu, M. (1980). Finite Markov Processes and Their Applications. John Wiley, Chichester.Google Scholar
Jordan, D. W. and Smith, P. (2005). Nonlinear Ordinary Differential Equations: An Introduction to Dynamical Systems. Oxford University Press.Google Scholar
Kolokoltsov, V. N. (2010). Nonlinear Markov Processes and Kinetic Equations. Cambridge University Press.CrossRefGoogle Scholar
Kolokoltsov, V. N. (2012). Nonlinear Markov games on a finite state space (mean-field and binary interactions). Internat. J. Statist. Prob. 1, 77–91.CrossRefGoogle Scholar
Kolokoltsov, V. N. and Malafeyev, O. A. (2019). Many Agent Games in Socio-economic Systems: Corruption, Inspection, Coalition Building, Network Growth, Security. Springer, Cham.CrossRefGoogle Scholar
McKean, H. P., Jr. (1966). A class of Markov processes associated with nonlinear parabolic equations. Proc. Nat. Acad. Sci. USA 56, 1907–1911.CrossRefGoogle ScholarPubMed
McShane, E. J. (1934). Extension of range of functions. Bull. Amer. Math. Soc. 40, 837–842.CrossRefGoogle Scholar
Neumann, B. A. (2019). Stationary equilibria of mean field games with finite state and action space : existence, computation, stability, and a myopic adjustment process. Doctoral Thesis, Universität Hamburg.Google Scholar
Neumann, B. A. (2020). Stationary equilibria of mean field games with finite state and action space. Dynam. Games Appl. 10, 845–871.CrossRefGoogle Scholar
Novozhilov, A. S., Karev, G. P. and Koonin, E. V. (2006). Biological applications of the theory of birth-and-death processes. Briefings Bioinformat. 7, 70–85.CrossRefGoogle ScholarPubMed
Pham, H. and Wei, X. (2017). Dynamic programming for optimal control of stochastic McKean–Vlasov dynamics. SIAM J. Control Optimization 55, 1069–1101.CrossRefGoogle Scholar
Resnick, S. I. (1992). Adventures in Stochastic Processes. Birkhäuser, Boston.Google Scholar
Saburov, M. (2016). Ergodicity of nonlinear Markov operators on the finite dimensional space. Nonlinear Anal. Theory Methods Appl. 143, 105–119.CrossRefGoogle Scholar
Sznitman, A.-S. (1991). Topics in propagation of chaos. In Ecole d’Eté de Probabilités de Saint-Flour XIX—1989, ed. P.-L. Hennequin, Springer, Berlin, Heidelberg, pp. 165–251.CrossRefGoogle Scholar
Teschl, G. (2012). Ordinary Differential Equations and Dynamical Systems. American Mathematical Society, Providence, RI.CrossRefGoogle Scholar
Walter, W. (1998). Ordinary Differential Equations. Springer, New York.CrossRefGoogle Scholar