Hostname: page-component-54dcc4c588-r5qjk Total loading time: 0 Render date: 2025-09-30T06:37:27.410Z Has data issue: false hasContentIssue false
  • English
  • Français

A general markov decision method II: Applications

Published online by Cambridge University Press:  01 July 2016

G. De Leve ,
A. Federgruen  and
H. C. Tijms
G. De Leve
Affiliation:
Mathematisch Centrum, Amsterdam
A. Federgruen
Affiliation:
Mathematisch Centrum, Amsterdam
H. C. Tijms
Affiliation:
Mathematisch Centrum, Amsterdam
Get access Rights & Permissions [Opens in a new window]

Abstract

In a preceding paper [2] we have introduced a new approach for solving a wide class of Markov decision problems in which the state-space may be general and the system may be continuously controlled. The criterion is the average cost. This paper discusses two applications of this approach. The first application concerns a house-selling problem in which a constructor builds houses which may be sold at any stage of the construction and potential customers make offers depending on the stage of the construction. The second application considers an M/M/c queueing problem in which the number of operating servers can be controlled by turning servers on or off.

Keywords

MARKOV DECISION PROBLEMSAVERAGE COSTGENERAL STATE SPACECONTINUOUS CONTROLAPPLICATIONSHOUSE-SELLING PROBLEMM/M/C QUEUEING PROBLEM WITH VARIABLE NUMBER OF SERVERS

Information

Type
Research Article
Information
Advances in Applied Probability , Volume 9 , Issue 2 , June 1977 , pp. 316 - 335
Copyright
Copyright © Applied Probability Trust 1977 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

[1] Bell, C. E. (1975) Turning off a server with customers present: is this any way to run an M/M/c queue with removable servers? Opns. Res. 23, 571–574.CrossRefGoogle Scholar
[2] De Leve, G., Federgruen, A. and Tijms, H. C. (1977) A general Markov decision method, I: model and techniques. Adv. Appl. Prob. 9.Google Scholar
[3] Feller, W. (1966) An Introduction to Probability Theory and its Applications, Vol. 2. Wiley, New York.Google Scholar
[4] Howard, R. A. (1971) Dynamic Probabilistic Systems, Vol. II: Semi-Markov and Decision Processes. Wiley, New York.Google Scholar
[5] Lippman, S. A. (1973) A new technique in the optimization of exponential queueing systems. Working Paper No. 211, Western Management Science Institute, University of California, Los Angeles.CrossRefGoogle Scholar
[6] McGill, J. T. (1969) Optimal control of a queueing system with variable number of servers. Technical Report No. 2, Department of Operations Research, Stanford University.Google Scholar
[7] Miller, K. S. (1968) Linear Difference Equations. W. A. Benjamin Inc., New York.CrossRefGoogle Scholar
[8] Robin, M. (1975) Contrôle optimal de files d'attente. Rapport de Recherche No. 117, IRIA, Rocqencourt, France.Google Scholar
[9] Sobel, M. J. (1974) Optimal operation of queues. In Mathematical Methods in Queueing Theory, ed. Clarke, A. B. Lecture Notes in Economics and Mathematical Systems 98, Springer-Verlag, Berlin.Google Scholar
[10] Sobel, M. J. (1971) Production smoothing with stochastic demand II: infinite-horizon case. Management Sci. 17, 724–735.CrossRefGoogle Scholar
[11] Van Domselaar, V. and Hemker, P. W. (1975) Nonlinear parameter estimation in initial value problems. Mathematical Centre Report NW 18/75, Mathematisch Centrum, Amsterdam.Google Scholar