Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-18T01:18:55.168Z Has data issue: false hasContentIssue false
  • English
  • Français

Stability and exponential convergence of continuous-time Markov chains

Part of: Basic linear algebra Markov processes Distribution theory - Probability Stability theory

Published online by Cambridge University Press:  14 July 2016

A. Yu. Mitrophanov
A. Yu. Mitrophanov*
Affiliation:
Saratov State University
*
∗Postal address: Faculty of Computer Science and Information Technology, Saratov State University, 83 Astrakhanskaya str., Saratov 410012, Russia. Email address: mitrophanovay@info.sgu.ru
Get access Rights & Permissions [Opens in a new window]

Abstract

For finite, homogeneous, continuous-time Markov chains having a unique stationary distribution, we derive perturbation bounds which demonstrate the connection between the sensitivity to perturbations and the rate of exponential convergence to stationarity. Our perturbation bounds substantially improve upon the known results. We also discuss convergence bounds for chains with diagonalizable generators and investigate the relationship between the rate of convergence and the sensitivity of the eigenvalues of the generator; special attention is given to reversible chains.

Keywords

Markov chainexponential convergenceperturbation boundssensitivity analysisergodicity coefficienteigenvaluespectral gap

MSC classification

Primary: 60J27: Continuous-time Markov processes on discrete state spaces
Secondary: 60J35: Transition functions, generators and resolvents 60E15: Inequalities; stochastic orderings 34D10: Perturbations 15A42: Inequalities involving eigenvalues and eigenvectors
Type
Research Papers
Information
Journal of Applied Probability , Volume 40 , Issue 4 , September 2003 , pp. 970 - 979
Copyright
Copyright © Applied Probability Trust 2003 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Andreev, D. B. et al. (2002). On ergodicity and stability estimates for some nonhomogeneous Markov chains. J. Math. Sci. (New York) 112, 4111–4118.Google Scholar
Arakelian, V. B., Wild, J. R., and Simonian, A. L. (1998). Investigation of stochastic fluctuations in the signal formation of microbiosensors. Biosensors Bioelectron. 13, 55–59.CrossRefGoogle Scholar
Ball, F. G., Milne, R. K., and Yeo, G. F. (2000). Stochastic models for systems of interacting ion channels. IMA J. Math. Appl. Med. Biol. 17, 263–293.CrossRefGoogle ScholarPubMed
Diaconis, P., and Saloff-Coste, L. (1996). Logarithmic Sobolev inequalities for finite Markov chains. Ann. Appl. Prob. 6, 695–750.Google Scholar
Diaconis, P., and Saloff-Coste, L. (1996). Nash inequalities for finite Markov chains. J. Theoret. Prob. 9, 459–510.Google Scholar
Diaconis, P., and Strook, D. (1991). Geometric bounds for eigenvalues of Markov chains. Ann. Appl. Prob. 1, 36–61.Google Scholar
Fill, J. A. (1991). Eigenvalue bounds on convergence to stationarity for nonreversible Markov chains, with an application to the exclusion process. Ann. Appl. Prob. 1, 62–87.CrossRefGoogle Scholar
Granovsky, B. L., and Zeifman, A. I. (1997). The decay function of nonhomogeneous birth—death processes, with application to mean-field models. Stoch. Process. Appl. 72, 105–120.Google Scholar
Granovsky, B. L., and Zeifman, A. I. (2000). Nonstationary Markovian queues. J. Math. Sci. (New York) 99, 1415–1438.Google Scholar
Granovsky, B. L., and Zeifman, A. I. (2000). The N-limit of spectral gap of a class of birth—death Markov chains. Appl. Stoch. Models Business Industry 16, 235–248.Google Scholar
Horn, R. A., and Johnson, C. R. (1985). Matrix Analysis. Cambridge University Press.CrossRefGoogle Scholar
Kartashov, N. V. (1986). Inequalities in theorems of ergodicity and stability for Markov chains with common state space. I. Theory Prob. Appl. 30, 247–259.Google Scholar
Kartashov, N. V. (1986). Inequalities in theorems of ergodicity and stability for Markov chains with common state space. II. Theory Prob. Appl. 30, 507–515.CrossRefGoogle Scholar
Peng, N. F. (1996). Spectral representations of the transition probability matrices for continuous time finite Markov chains. J. Appl. Prob. 33, 28–33.Google Scholar
Seneta, E. (1979). Coefficients of ergodicity: structure and applications. Adv. Appl. Prob. 11, 576–590.Google Scholar
Thoumine, O., and Meister, J.-J. (2000). A probabilistic model for ligand-cytoskeleton transmembrane adhesion: predicting the behaviour of microspheres on the surface of migrating cells. J. Theoret. Biol. 204, 381–392.CrossRefGoogle ScholarPubMed
Van Kampen, N. G. (1981). Stochastic Processes in Physics and Chemistry. North Holland, Amsterdam.Google Scholar
Zeifman, A. I. (1985). Stability for continuous-time nonhomogeneous Markov chains. In Stability Problems for Stochastic Models (Lecture Notes Math. 1155), eds Kalashnikov, V. V. and Zolotarev, V. M., Springer, Berlin, pp. 401-414.Google Scholar
Zeifman, A. I. (1995). Upper and lower bounds on the rate of convergence for nonhomogeneous birth and death processes. Stoch. Process. Appl. 59, 157–173.Google Scholar
Zeifman, A. I. (1998). Stability of birth-and-death processes. J. Math. Sci. (New York) 91, 3023–3031.Google Scholar
Zeifman, A. I., and Isaacson, D. L. (1994). On strong ergodicity for nonhomogeneous continuous-time Markov chains. Stoch. Process. Appl. 50, 263–273.CrossRefGoogle Scholar
Zheng, Q. (1998). Note on the non-homogeneous Prendiville process. Math. Biosci. 148, 1–5.CrossRefGoogle ScholarPubMed