Skip to main content

Hitting-time and occupation-time bounds implied by drift analysis with applications

  • Bruce Hajek (a1)

Bounds of exponential type are derived for the first-hitting time and occupation times of a real-valued random sequence which has a uniform negative drift whenever the sequence is above a fixed level. The only other assumption on the random sequence is that the increments satisfy a uniform exponential decay condition. The bounds provide a flexible technique for proving stability of processes frequently encountered in the control of queues.

Two applications are given. First, exponential-type bounds are derived for a GI/G/1 queue when the service distribution is exponential type. Secondly, geometric ergodicity is established for a certain Markov chain in which arises in the decentralized control of a multi-access, packet-switched broadcast channel.

Corresponding author
Postal address: Coordinated Science Laboratory, University of Illinois, 1101 West Springfield Ave., Urbana, IL 61801, U.S.A.
Hide All

This work was supported by U.S. Navy contract N00014–80-C-0802.

Hide All
Breiman, L. (1968) Probability. Addison-Wesley, Reading, MA.
Cheong, C. K. and Heathcote, C. R. (1965) On the rate of convergence of waiting times. J. Austral. Math. Soc. 5, 365373.
Doob, J. L. (1953) Stochastic Processes. Wiley, New York.
Hajek, B. and Van Loon, T. (1982) Decentralized dynamic control of a multi-access broadcast channel. IEEE Trans. Automatic Control 27,
Kamae, T., Krengel, U., and O'Brien, G. L. (1977) Stochastic inequalities on partially ordered spaces. Ann. Prob. 5, 899912.
Kemeny, J. G., Snell, J. L. and Knapp, A. W. (1976) Denumberable Markov Chains. Springer-Verlag, New York.
Kingman, J. F. C. (1970) Inequalities in the theory of queues. J. R. Statist. Soc. B32, 102110.
Kingman, J. F. C. (1974) A martingale inequality in the theory of queues. Proc. Camb. Phil. Soc. 59, 359361.
Lai, T. L. (1974) Martingales and boundary crossing probabilities for Markov processes. Ann. Prob. 6, 11521167.
Lamperti, J. (1960) Criteria for the recurrence or transience of stochastic processes, I. J. Math. Anal. Appl. 1, 314330.
Lamperti, J. (1963) Criteria for stochastic processes II: passage-time moments. J. Math. Anal. Appl. 7, 127145.
Malysev, V. A. (1972) Classification of two-dimensional positive random walks and almost linear semimartingales. Soviet Math. 13, 136139.
Meyer, P. A. (1966) Probability and Potentials. Blaisdell, Waltham, MA. Massachusetts.
Miller, H. D. (1966) Geometric ergodicity in a class of denumerable Markov chains. Z. Wahrscheinlichkeitsth. 4, 354373.
Neuts, M. F. and Teugels, J. L. (1969) Exponential ergodicity of the M/G/1 queue. SIAM J. Appl. Math. 17, 921929.
Nummelin, E. and Tweedie, R. L. (1978) Geometric ergodicity and R-positivity for general Markov chains. Ann. Prob. 6, 404420.
Pakes, A. G. (1969) Some conditions for ergodicity and recurrence of Markov chains. Operat. Res. 17, 10581061.
Ross, S. M. (1974) Bounds on the delay distribution in GI/G/1 queues. J. Appl. Prob. 11, 417421.
Teugels, J. L. (1967) Exponential decay in renewal theorems. Bull. Soc. Math. Belg. 19, 259276.
Tuominen, P. and Tweedie, R. L. (1979) Exponential ergodicity in Markovian queueing and dam models. J. Appl. Prob. 16, 867880.
Tweedie, R. L. (1976) Criteria for classifying general Markov chains. Adv. Appl. Prob. 8, 737771.
Wald, A. (1944) On cumulative sums of random variables. Ann. Math. Statist. 15, 283296.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Advances in Applied Probability
  • ISSN: 0001-8678
  • EISSN: 1475-6064
  • URL: /core/journals/advances-in-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *