Hostname: page-component-89b8bd64d-b5k59 Total loading time: 0 Render date: 2026-05-08T02:10:00.546Z Has data issue: false hasContentIssue false
  • English
  • Français

OPTIMALLY REPLACING MULTIPLE SYSTEMS IN A SHARED ENVIRONMENT

Published online by Cambridge University Press:  10 May 2017

David T. Abdul-Malak  and
Jeffrey P. Kharoufeh
David T. Abdul-Malak
Affiliation:
Department of Industrial Engineering, University of Pittsburgh, 1025 Benedum Hall, 3700 O'Hara Street, Pittsburgh, PA 15261, USA E-mail: dta10@pitt.edu; jkharouf@pitt.edu
Jeffrey P. Kharoufeh
Affiliation:
Department of Industrial Engineering, University of Pittsburgh, 1025 Benedum Hall, 3700 O'Hara Street, Pittsburgh, PA 15261, USA E-mail: dta10@pitt.edu; jkharouf@pitt.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of optimally replacing multiple stochastically degrading systems using condition-based maintenance. Each system degrades continuously at a rate that is governed by the current state of the environment, and each fails once its own cumulative degradation threshold is reached. The objective is to minimize the sum of the expected total discounted setup, preventive replacement, reactive replacement, and downtime costs over an infinite horizon. For each environment state, we prove that the cost function is monotone nondecreasing in the cumulative degradation level. Additionally, under mild conditions, these monotonicity results are extended to the entire state space. In the case of a single system, we establish that monotone policies are optimal. The monotonicity results help facilitate a tractable, approximate model with state- and action-space transformations and a basis-function approximation of the action-value function. Our computational study demonstrates that high-quality, near-optimal policies are attainable and significantly outperform heuristic policies.

Keywords

Maintenance optimizationapproximate dynamic programming

Information

Type
Research Article
Information
Probability in the Engineering and Informational Sciences , Volume 32 , Issue 2 , April 2018 , pp. 179 - 206
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1. Barlow, R.E. & Proschan, F. (1965). Mathematical theory of reliability. New York, NY: John Wiley & Sons, Inc.Google Scholar
2. Bellman, R.E. (1957a). Dynamic programming. Princeton, NJ: Princeton University Press.Google Scholar
3. Bellman, R.E. (1957b). A Markovian decision process. Indiana University Mathematics Journal 6(4): 679–684.Google Scholar
4. Bertsekas, D.P. & Shreve, S.E. (1996). Stochastic optimal control: the discrete-time case. Belmont, MA: Athena Scientific.Google Scholar
5. Birkhoff, G. (1967). Lattice theory, 3rd ed. Providence, RI: American Mathematical Society.Google Scholar
6. Bouvard, K., Artus, S., Berenguer, C., & Cocquempot, V. (2011). Condition-based dynamic maintenance operataions planning and grouping: Application to commercial heavy vehicles. Reliability Engineering and System Safety 96(6): 601–610.Google Scholar
7. Castanier, B., Grall, A., & Berenguer, C. (2005). A condition-based maintenance policy with non-periodic inspections for a two-unit series system. Reliability Engineering and System Safety 87(1): 109–120.Google Scholar
8. Chen, N. & Tsui, K.L. (2013). Condition monitoring and remaining useful life prediction using degradation signals: Revisited. IIE Transactions 45(9): 939–952.Google Scholar
9. Cho, D.I. & Parlar, M.A. (1991). A survey of maintenance models for multi-unit systems. European Journal of Operational Research 51(1): 1–23.Google Scholar
10. Dekker, R. (1996). Applications of maintenance optimization models: A review and analysis. Reliability Engineering and System Safety 51: 229–240.Google Scholar
11. Dekker, R., Wildeman, R.E., & van der Duyn Schouten, F.A. (1997). A review of multi-component maintenance models with economic dependence. Mathematical Methods of Operations Research 45: 411–435.Google Scholar
12. Gebraeel, N.Z., Lawley, M.A., Li, R., & Ryan, J.K. (2005). Residual-life distributions from component degradation signals: A Bayesian approach. IIE Transactions 37(6): 543–557.Google Scholar
13. Geramifard, A., Walsh, T.J., Tellex, S., Chowdhary, G., Roy, N., & How, J.P. (2013). A tutorial on linear function approximators for dynamic programming and reinforcement learning. Foundations and Trends in Machine Learning 6(4): 375–451.Google Scholar
14. Howard, R.A. (1960). Dynamic programming and Markov processes. Cambridge, MA: The MIT Press.Google Scholar
15. Kharoufeh, J.P. & Cox, S.M. (2005). Stochastic models for degradation-based reliability. IIE Transactions 37(6): 533–542.Google Scholar
16. Kharoufeh, J.P., Finkelstein, D.E., & Mixon, D.G. (2006). Availability of periodically inspected systems with Markovian wear and shocks. Journal of Applied Probability 43(2): 303–317.Google Scholar
17. Kharoufeh, J.P. & Mixon, D.G. (2009). On a Markov-modulated shock and wear process. Naval Research Logistics 56(6): 563–576.Google Scholar
18. Kharoufeh, J.P., Solo, C., & Ulukus, M.Y. (2010). Semi-Markov models for degradation-based reliability. IIE Transactions 42(8): 599–612.Google Scholar
19. Ko, Y.M. & Byon, E. (2017). Condition-based joint maintenance optimization for a large-scale system with homogeneous units. IIE Transactions 49(5): 493–504.Google Scholar
20. Konidaris, G.D., Osentoski, S., & Thomas, P. (2011). Value function approximation in reinforcement learning using the Fourier basis. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, San Francisco, CA. The AAAI Press.Google Scholar
21. Marseguerra, M., Zio, E., & Podofillini, L. (2002). Condition-based maintenance optimization by means of genetic algorithms and Monte Carlo simulation. Reliability Engineering and System Safety 77(2): 151–165.Google Scholar
22. McCall, J.J. (1965). Maintenance policies for stochastically failing equipment: A survey. Management Science 11(5): 493–524.Google Scholar
23. Nicolai, R.P. & Dekker, R. (2008). Optimal maintenance of multi-component systems: A review. In Kobbacy, K. & Murthy, D. (eds.), Complex system maintenance handbook, London: Springer London, pp. 263–286.CrossRefGoogle Scholar
24. Pham, H. & Wang, H. (1996). Imperfect maintenance. European Journal of Operational Research 94(3): 425–438.Google Scholar
25. Pierskalla, W. & Voelker, J. (1976). A survey of maintenance models: The control and surveillance of deteriorating systems. Naval Research Logistics Quarterly 23(3): 353–388.CrossRefGoogle Scholar
26. Powell, W.B. (2007). Approximate dynamic programming: solving the curses of dimensionality. Hoboken, NJ: Wiley.Google Scholar
27. Puterman, M.L. (2005). Markov decision processes: discrete stochastic dynamic programming, 2nd ed. Hoboken, NJ: Wiley.Google Scholar
28. Rummery, G.A. & Niranjan, M. (1994). On-line Q-learning using connectionist systems. Technical report, Cambridge University Engineering Department.Google Scholar
29. Shaked, M. & Shanthikumar, J.G. (2007). Stochastic orders. New York, NY: Springer.Google Scholar
30. Sharma, A., Yadava, G.S., & Deshmukh, S.G. (2011). A literature review and future perspectives on maintenance optimization. Journal of Quality in Maintenance Engineering 17(1): 5–25.Google Scholar
31. Sherif, Y.S. & Smith, M.L. (1981). Optimal maintenance models for systems subject to failure – A review. Naval Research Logistics Quarterly 28(1): 47–74.Google Scholar
32. Singpurwalla, N. (1995). Survival in dynamic environments. Statistical Science 10(1): 86–103.Google Scholar
33. Sutton, R.S. & Barto, A.G. (1998). Reinforcement learning: an introduction. Cambridge, MA: The MIT Press.Google Scholar
34. Tian, Z. & Liao, H. (2011). Condition-based maintenance optimization for multi-component systems using proportional hazards model. Reliability Engineering and System Safety 96(5): 581–589.Google Scholar
35. Topkis, D.M. (1998). Supermodularity and complementarity. Princeton, NJ: Princeton University Press.Google Scholar
36. Ulukus, M.Y., Kharoufeh, J.P., & Maillart, L.M. (2012). Optimal replacement policies under environment-driven degradation. Probability in the Engineering and Informational Sciences 26(3): 405–424.Google Scholar
37. Valdez-Flores, C. & Feldman, R. (1989). A survey of preventive maintenance models for stochastically deteriorating single-unit systems. Naval Research Logistics 36(4): 419–446.Google Scholar
38. Wang, H. (2002). A survey of maintenance policies of deteriorating systems. European Journal of Operational Research 139(3): 469–489.Google Scholar
39. Yang, W., Tavner, P.J., Crabtree, C.J., Feng, Y., & Qiu, Y. (2012). Wind turbine condition monitoring: Technical and commercial challenges. Wind Energy 17(5): 673–693.Google Scholar
40. Ye, Z.S., Wang, Y., Tsui, K.L., & Pecht, M. (2013). Degradation data analysis using Wiener processes with measurement errors. IEEE Transactions on Reliability 62(4): 772–780.Google Scholar
41. Zhu, Q., Peng, H., & van Houtum, G. (2015). A condition-based maintenance policy for multi-component systems with a high maintenance setup cost. OR Spectrum 37: 1007–1035.Google Scholar