Article contents
Perturbation analysis of inhomogeneous finite Markov chains
Published online by Cambridge University Press: 24 March 2016
Abstract
In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0 , t]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.
Keywords
MSC classification
- Type
- Research Article
- Information
- Copyright
- Copyright © Applied Probability Trust 2016
References
- 8
- Cited by