Hostname: page-component-76fb5796d-r6qrq Total loading time: 0 Render date: 2024-04-30T07:08:31.773Z Has data issue: false hasContentIssue false
  • English
  • Français

The asymptotic tails of limit distributions of continuous-time Markov chains

Part of: Markov processes

Published online by Cambridge University Press:  06 October 2023

Chuang Xu Open the ORCID record for Chuang Xu [Opens in a new window] ,
Mads Christian Hansen  and
Carsten Wiuf
Chuang Xu*
Affiliation:
University of Hawai’i at Mānoa
Mads Christian Hansen*
Affiliation:
University of Copenhagen
Carsten Wiuf*
Affiliation:
University of Copenhagen
*
*Postal address: Department of Mathematics, University of Hawai’i at Mānoa, Honolulu, HI 96822, USA. Email address: chuangxu@hawaii.edu
**Postal address: Department of Mathematical Sciences, University of Copenhagen, Copenhagen, 2100, Denmark.
**Postal address: Department of Mathematical Sciences, University of Copenhagen, Copenhagen, 2100, Denmark.
Get access Rights & Permissions [Opens in a new window]

Abstract

This paper investigates tail asymptotics of stationary distributions and quasi-stationary distributions (QSDs) of continuous-time Markov chains on subsets of the non-negative integers. Based on the so-called flux-balance equation, we establish identities for stationary measures and QSDs, which we use to derive tail asymptotics. In particular, for continuous-time Markov chains with asymptotic power law transition rates, tail asymptotics for stationary distributions and QSDs are classified into three types using three easily computable parameters: (i) super-exponential distributions, (ii) exponential-tailed distributions, and (iii) sub-exponential distributions. Our approach to establish tail asymptotics of stationary distributions is different from the classical semimartingale approach, and we do not impose ergodicity or moment bound conditions. In particular, the results also hold for explosive Markov chains, for which multiple stationary distributions may exist. Furthermore, our results on tail asymptotics of QSDs seem new. We apply our results to biochemical reaction networks, a general single-cell stochastic gene expression model, an extended class of branching processes, and stochastic population processes with bursty reproduction, none of which are birth–death processes. Our approach, together with the identities, easily extends to discrete-time Markov chains.

Keywords

Discrete-time Markov chainstationary measuretail distributionquasi-stationary distributionstochastic reaction network

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 60J28: Applications of continuous-time Markov processes on discrete state spaces
Type
Original Article
Information
Advances in Applied Probability , First View , pp. 1 - 42
Check for updates
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of Applied Probability Trust

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abramowitz, M. and Stegun, I. A. (1972). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables. US Government Printing Office, Washington, DC.Google Scholar
Anderson, D. F., Craciun, G. and Kurtz, T. G. (2010). Product-form stationary distributions for deficiency zero chemical reaction networks. Bull. Math. Biol. 72, 1947–1970.CrossRefGoogle ScholarPubMed
Anderson, D. F. and Kurtz, T. G. (2011). Continuous time Markov chain models for chemical reaction networks. In Design and Analysis of Biomolecular Circuits: Engineering Approaches to Systems and Synthetic Biology, eds Koeppl, H., Setti, G., M. di Bernardo and D. Densmore, Springer, New York, pp. 3–42.CrossRefGoogle Scholar
Anderson, W. J. (1991). Continuous-Time Markov Chains: An Applications-Oriented Approach. Springer, New York.CrossRefGoogle Scholar
Asmussen, S. (1998). Subexponential asymptotics for stochastic processes: extremal behavior, stationary distributions and first passage probabilities. Ann. Appl. Prob. 8, 354–374.CrossRefGoogle Scholar
Aspandiiarov, S. and Iasnogorodski, R. (1999). Asymptotic behaviour of stationary distributions for countable Markov chains, with some applications. Bernoulli 5, 535–569.CrossRefGoogle Scholar
Be’er, S. and Assaf, M. (2016). Rare events in stochastic populations under bursty reproduction. J. Statist. Mech. 2016, article no. 113501.CrossRefGoogle Scholar
Bertsimas, D., Gamarnik, D. and Tsitsiklis, J. N. (2001). Performance of multiclass Markovian queueing networks via piecewise linear Lyapunov functions. Ann. Appl. Prob. 11, 1384–1428.CrossRefGoogle Scholar
Cappelletti, D. and Wiuf, C. (2016). Product-form Poisson-like distributions and complex balanced reaction systems. SIAM J. Appl. Math. 76, 411–432.CrossRefGoogle Scholar
Cavender, J. A. (1978). Quasi-stationary distributions of birth-and-death processes. Adv. Appl. Prob. 10, 570–586.CrossRefGoogle Scholar
Champagnat, N. and Villemonais, D. (2016). Exponential convergence to quasi-stationary distribution and Q-process. Prob. Theory Relat. Fields. 164, 243–283.CrossRefGoogle Scholar
Champagnat, N. and Villemonais, D. (2017). General criteria for the study of quasi-stationarity. Electron. J. Prob. 28, article no. 22.Google Scholar
Chen, R.-R. (1997). An extended class of time-continuous branching processes. J. Appl. Prob. 34, 14–23.CrossRefGoogle Scholar
Chowdhury, A. R., Chetty, M. and Evans, R. (2015). Stochastic S-system modeling of gene regulatory network. Cognitive Neurodynamics 9, 535–547.CrossRefGoogle ScholarPubMed
Collet, P., Martínez, S. and San Martín, J. (2013). Quasi-Stationary Distributions: Markov Chains, Diffusions and Dynamical Systems. Springer, Heidelberg.CrossRefGoogle Scholar
Denisov, D., Korshunov, D. and Wachtel, V. (2013). Potential analysis for positive recurrent Markov chains with asymptotically zero drift: power-type asymptotics. Stoch. Process. Appl. 123, 3027–3051.CrossRefGoogle Scholar
Denisov, D., Korshunov, D. and Wachtel, V. (2016). At the Edge of Criticality: Markov Chains with Asymptotically Zero Drift. Available at https://arxiv.org/abs/1612.01592.Google Scholar
Economou, A. (2005). Generalized product-form stationary distributions for Markov chains in random environments with queueing applications. Adv. Appl. Prob. 37, 185–211.CrossRefGoogle Scholar
Ewens, W. J. (2004). Mathematical Population Genetics. I. Theoretical Introduction. Springer, New York.CrossRefGoogle Scholar
Falk, J., Mendler, B. and Drossel, B. (2017). A minimal model of burst-noise induced bistability. PLOS ONE 12, article no. e0176410.CrossRefGoogle Scholar
Gardiner, C. W. (2009). Stochastic Methods: A Handbook for Physics, Chemistry and the Natural Sciences, 4th edn. Springer, Berlin.Google Scholar
Gelenbe, E. (1991). Product-form queueing networks with negative and positive customers. J. Appl. Prob. 28, 656–663.CrossRefGoogle Scholar
Gelenbe, E. and Mitrani, L. (2009). Analysis and Synthesis of Computer Systems, 2nd edn. Academic Press, London.Google Scholar
Gillespie, D. T. (1992). A rigorous derivation of the chemical master equation. Physica A 188, 404–425.CrossRefGoogle Scholar
Gupta, A., Briat, C. and Khammash, M. (2014). A scalable computational framework for establishing long-term behavior of stochastic reaction networks. PLOS Comput. Biol. 10, article no. e1003669.CrossRefGoogle Scholar
Hansen, M. C. (2018). Quasi-stationary Distributions in Stochastic Reaction Networks. Doctoral Thesis, University of Copenhagen.Google Scholar
Hansen, M. C. and Wiuf, C. (2020). Existence of a unique quasi-stationary distribution in stochastic reaction networks. Electron. J. Prob. 25, article no. 45.CrossRefGoogle Scholar
Hoessly, L. and Mazza, C. (2019). Stationary distributions and condensation in autocatalytic reaction networks. SIAM J. Appl. Math. 79, 1173–1196.CrossRefGoogle Scholar
Johnson, N. L., Kemp, A. W. and Kotz, S. (2005). Univariate Discrete Distributions, 3rd edn. John Wiley, New York.CrossRefGoogle Scholar
Kelly, F. P. (2011). Reversibility and Stochastic Networks. Cambridge University Press.Google Scholar
Liu, Y. and Zhao, Y. Q. (2011). Asymptotics of the invariant measure of a generalized Markov branching process. Stoch. Models 27, 251–271.CrossRefGoogle Scholar
Mairesse, J. and Nguyen, H.-T. (2010). Deficiency zero Petri nets and product form. Fund. Informaticae 105, 237–261.CrossRefGoogle Scholar
Maulik, K. and Zwart, B. (2006). Tail asymptotics of exponential functionals of Lévy processes. Stoch. Process. Appl. 116, 156–177.CrossRefGoogle Scholar
Méléard, S. and Villemonais, D. (2012). Quasi-stationary distributions and population processes. Prob. Surveys 9, 340–410.CrossRefGoogle Scholar
Menshikov, M. V. and Popov, S. Y. (1995). Exact power estimates for countable Markov chains. Markov Process. Relat. Fields 1, 57–78.Google Scholar
Meyn, S. P. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press, New York.CrossRefGoogle Scholar
Miller, R. G. (1963). Stationary equations in continuous time Markov chains. Trans. Amer. Math. Soc. 109, 35–44.Google Scholar
Neuts, M. F. (1984). Matrix-analytic methods in queuing theory. Europ. J. Operat. Res. 15, 2–12.CrossRefGoogle Scholar
Norris, J. R. (1998). Markov Chains. Cambridge University Press.Google Scholar
O’Neill, P. D. (2007). Constructing population processes with specified quasi-stationary distributions. Stoch. Models 23, 439–449.CrossRefGoogle Scholar
Pastor-Satorras, R., Castellano, C., Van Mieghem, P. and Vespignani, A. (2015). Epidemic processes in complex networks. Rev. Modern Phys. 87, 925–979.CrossRefGoogle Scholar
Ramaswami, V. (1988). A stable recursion for the steady state vector in Markov chains of M/G/1 type. Commun. Statist. Stoch. Models. 4, 183–188.CrossRefGoogle Scholar
Shahrezaei, V. and Swain, P. S. (2008). Analytical distributions for stochastic gene expression. Proc. Nat. Acad. Sci. USA 105, 17256–17261.CrossRefGoogle Scholar
Takine, T. (2004). Geometric and subexponential asymptotics of Markov chains of M/G/1 type. Math. Operat. Res. 29, 624–648.CrossRefGoogle Scholar
Van Doorn, E. A. (1991). Quasi-stationary distributions and convergence to quasi-stationarity of birth–death processes. Adv. Appl. Prob. 23, 683–700.CrossRefGoogle Scholar
Whittle, P. (1986). Systems in Stochastic Equilibrium. John Wiley, Chichester.Google Scholar
Xu, C., Hansen, M. C. and Wiuf, C. (2022). Structural classification of continuous time Markov chains with applications. Stochastics 94, 1003–1030.CrossRefGoogle Scholar
Xu, C., Hansen, M. C. and Wiuf, C. (2023). Full classification of dynamics for one-dimensional continuous time Markov chains with polynomial transition rates. Adv. Appl. Prob. 55, 321–355.CrossRefGoogle Scholar