Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-23T16:08:46.566Z Has data issue: false hasContentIssue false
  • English
  • Français

Stationary distributions for piecewise-deterministic Markov processes

Published online by Cambridge University Press:  14 July 2016

O. L. V. Costa
O. L. V. Costa*
Affiliation:
Escola Politécnica da USP
*
∗Postal addres: Departamento de Engenharia de Eletricidade, Escola Politécnica da USP, 05508 São Paulo, Brazil.
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we show that the problem of existence and uniqueness of stationary distributions for piecewise-deterministic Markov processes (PDPs) is equivalent to the same problem for the associated Markov chain, so long as some mild conditions on the parameters of the PDP are satisfied. Our main result is the construction of an invertible mapping from the set of stationary distributions for the PDP to the set of stationary distributions for the Markov chain. Some sufficient conditions for existence are presented and an application to capacity expansion is given.

Keywords

EMBEDDED MARKOV CHAINEQUIVALENCE RESULTSCONDITIONS FOR EXISTENCE
Type
Research Papers
Information
Journal of Applied Probability , Volume 27 , Issue 1 , March 1990 , pp. 60 - 73
Copyright
Copyright © Applied Probability Trust 1990 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Research carried out while the author held an SERC Post-Doctoral Research Assistantship in the Department of Electrical Engineering at Imperial College, London.

References

[1] Asmussen, S. (1987) Applied Probability and Queues. Wiley, New York.Google Scholar
[2] Breiman, L. (1968) Probability. Addison-Wesley, Reading, Mass.Google Scholar
[3] Davis, M. H. A. (1984) Piecewise-deterministic Markov processes: A general class of non-diffusion stochastic models. J. R. Statist. Soc. B 46, 353–388.Google Scholar
[4] Davis, M. H. A., Dempster, M. A. H., Sethi, S. P. and Vermes, D. (1987) Optimal capacity expansion under uncertainty. Adv. Appl. Prob. 19, 156–176.Google Scholar
[5] Ethier, S. N. and Kurtz, T. G. (1986) Markov Processes; Characterization and Convergence. Wiley, New York.Google Scholar
[6] Hordijk, A. and Van Der Duyn Schouten, F. A. (1984) Discretization and weak convergence in Markov decision drift processes, Math. Operat. Res. 9, 112–141.Google Scholar
[7] Tweedie, R. L. (1975) Sufficient conditions for ergodicity and recurrence of Markov chains on a general state space. Stoch. Proc. Appl. 3, 385–403.Google Scholar
[8] Yushkevich, A. A. (1983) Continuous-time Markov decision processes with interventions. Stochastics 9, 235–274.Google Scholar