Skip to main content


  • Michael N. Katehakis (a1) and Laurens C. Smit (a2)

A class of Markov chains we call successively lumbaple is specified for which it is shown that the stationary probabilities can be obtained by successively computing the stationary probabilities of a propitiously constructed sequence of Markov chains. Each of the latter chains has a(typically much) smaller state space and this yields significant computational improvements. We discuss how the results for discrete time Markov chains extend to semi-Markov processes and continuous time Markov processes. Finally, we will study applications of successively lumbaple Markov chains to classical reliability and queueing models.

Hide All
1.Derman, C., Lieberman, G.J., & Ross, S.M. (1980). On the optimal assignment of servers and a repairman. Journal of Applied Probability 17: 577581.
2.Feinberg, B.N., & Chui, S.S. (1987). A method to calculate steady state distributions of large Markov chains. Operations Research 35: 282290.
3.Frostig, E. (1999). Jointly optimal allocation of a repairman and optimal control of service rate for machine repairman problem. European Journal of Operational Research 116: 274280.
4.Hooghiemstra, G., & Koole, G. (2000). On the convergence of the power series algorithm. Performance Evaluation 42: 2139.
5.Katehakis, M.N., & Derman, C. (1984). Optimal repair allocation in a series system. Mathematics of Operations Research 9: 615623.
6.Katehakis, M.N., & Derman, C. (1989). On the maintenance of systems composed of highly reliable components. Management Science 35(5): 551560.
7.Katehakis, M.N., & Melolidakis, C. (1995). On the optimal maintenance of systems and control of arrivals in queues. Stochastic Analysis and Applications 13: 137164.
8.Kemeny, J.G., & Snell, J.L. (1960). Finite Markov chains. Princeton, NJ: D. van Nostrand Company, Inc.
9.Kim, D.S., & Smith, R.L. (1989). An exact aggregation algorithm for a special class of markov chains. Technical Report. University of Michigan.
10.Kim, D.S., & Smith, R.L. (1990), An exact aggregation–disaggregation algortihm for mandatory set decomposable markovchains. Numerical Solution of Markov Chains 89104.
11.Koole, G.M., & Spieksma, F.M. (2001). On deviation matrices for birth–death processes. Probability in the Engineering and Informational Sciences 15: 239258.
12.Miranker, W.L., & Pan, V.Ya. (1980). Methods of aggregation. Linear Algebra and its Application 29: 231257.
13.Neuts, M.F. (1981). Matrix-geometric solutions in stochastic models. Baltimore: The Johns Hopkins University Press.
14.Righter, R. (1996). Optimal policies for scheduling repairs and allocating heterogeneous servers. Journal of Applied Probability 436547.
15.Ross, S.M. (1996). Stochastic processes. New York: John Wiley and Sons.
16.Schweitzer, P.J., Puterman, M.L., & Kindle, W.L. (1984). Iterative aggregation–disaggregation procedures for discounted semi-markov reward processes. Operations Research 33: 589605.
17.Ungureanu, V., Melamed, B., Katehakis, M.N., & Bradford, P.G. (2006). Deferred assignment scheduling in cluster-based servers. Cluster Computing 9: 5765.
18.Yap, V.B. (2009). Similar states in continuous time Markov chains. Journal of Applied Probability 46: 497506.
19.Zhang, Q., Mi, N., Riska, A., & Smirni, E. (2007), Performance-guided load (un) balancing under autocorrelated flows. IEEE Transactions on Parallel and Distributed Systems 652665.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Probability in the Engineering and Informational Sciences
  • ISSN: 0269-9648
  • EISSN: 1469-8951
  • URL: /core/journals/probability-in-the-engineering-and-informational-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 56 *
Loading metrics...

Abstract views

Total abstract views: 334 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 23rd July 2018. This data will be updated every 24 hours.