Hostname: page-component-78857b5c4d-wh9bc Total loading time: 0 Render date: 2023-08-04T00:23:51.102Z Has data issue: false Feature Flags: { "corePageComponentGetUserInfoFromSharedSession": true, "coreDisableEcommerce": false, "coreDisableSocialShare": false, "coreDisableEcommerceForArticlePurchase": false, "coreDisableEcommerceForBookPurchase": false, "coreDisableEcommerceForElementPurchase": false, "coreUseNewShare": false, "useRatesEcommerce": true } hasContentIssue false

OPTIMALLY REPLACING MULTIPLE SYSTEMS IN A SHARED ENVIRONMENT

Published online by Cambridge University Press:  10 May 2017

David T. Abdul-Malak
Affiliation:
Department of Industrial Engineering, University of Pittsburgh, 1025 Benedum Hall, 3700 O'Hara Street, Pittsburgh, PA 15261, USA E-mail: dta10@pitt.edu; jkharouf@pitt.edu
Jeffrey P. Kharoufeh
Affiliation:
Department of Industrial Engineering, University of Pittsburgh, 1025 Benedum Hall, 3700 O'Hara Street, Pittsburgh, PA 15261, USA E-mail: dta10@pitt.edu; jkharouf@pitt.edu

Abstract

We consider the problem of optimally replacing multiple stochastically degrading systems using condition-based maintenance. Each system degrades continuously at a rate that is governed by the current state of the environment, and each fails once its own cumulative degradation threshold is reached. The objective is to minimize the sum of the expected total discounted setup, preventive replacement, reactive replacement, and downtime costs over an infinite horizon. For each environment state, we prove that the cost function is monotone nondecreasing in the cumulative degradation level. Additionally, under mild conditions, these monotonicity results are extended to the entire state space. In the case of a single system, we establish that monotone policies are optimal. The monotonicity results help facilitate a tractable, approximate model with state- and action-space transformations and a basis-function approximation of the action-value function. Our computational study demonstrates that high-quality, near-optimal policies are attainable and significantly outperform heuristic policies.

Type
Research Article
Copyright
Copyright © Cambridge University Press 2017 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1. Barlow, R.E. & Proschan, F. (1965). Mathematical theory of reliability. New York, NY: John Wiley & Sons, Inc.Google Scholar
2. Bellman, R.E. (1957a). Dynamic programming. Princeton, NJ: Princeton University Press.Google ScholarPubMed
3. Bellman, R.E. (1957b). A Markovian decision process. Indiana University Mathematics Journal 6(4): 679684.CrossRefGoogle Scholar
4. Bertsekas, D.P. & Shreve, S.E. (1996). Stochastic optimal control: the discrete-time case. Belmont, MA: Athena Scientific.Google Scholar
5. Birkhoff, G. (1967). Lattice theory, 3rd ed. Providence, RI: American Mathematical Society.Google Scholar
6. Bouvard, K., Artus, S., Berenguer, C., & Cocquempot, V. (2011). Condition-based dynamic maintenance operataions planning and grouping: Application to commercial heavy vehicles. Reliability Engineering and System Safety 96(6): 601610.CrossRefGoogle Scholar
7. Castanier, B., Grall, A., & Berenguer, C. (2005). A condition-based maintenance policy with non-periodic inspections for a two-unit series system. Reliability Engineering and System Safety 87(1): 109120.CrossRefGoogle Scholar
8. Chen, N. & Tsui, K.L. (2013). Condition monitoring and remaining useful life prediction using degradation signals: Revisited. IIE Transactions 45(9): 939952.CrossRefGoogle Scholar
9. Cho, D.I. & Parlar, M.A. (1991). A survey of maintenance models for multi-unit systems. European Journal of Operational Research 51(1): 123.CrossRefGoogle Scholar
10. Dekker, R. (1996). Applications of maintenance optimization models: A review and analysis. Reliability Engineering and System Safety 51: 229240.CrossRefGoogle Scholar
11. Dekker, R., Wildeman, R.E., & van der Duyn Schouten, F.A. (1997). A review of multi-component maintenance models with economic dependence. Mathematical Methods of Operations Research 45: 411435.CrossRefGoogle Scholar
12. Gebraeel, N.Z., Lawley, M.A., Li, R., & Ryan, J.K. (2005). Residual-life distributions from component degradation signals: A Bayesian approach. IIE Transactions 37(6): 543557.CrossRefGoogle Scholar
13. Geramifard, A., Walsh, T.J., Tellex, S., Chowdhary, G., Roy, N., & How, J.P. (2013). A tutorial on linear function approximators for dynamic programming and reinforcement learning. Foundations and Trends in Machine Learning 6(4): 375451.CrossRefGoogle Scholar
14. Howard, R.A. (1960). Dynamic programming and Markov processes. Cambridge, MA: The MIT Press.Google Scholar
15. Kharoufeh, J.P. & Cox, S.M. (2005). Stochastic models for degradation-based reliability. IIE Transactions 37(6): 533542.CrossRefGoogle Scholar
16. Kharoufeh, J.P., Finkelstein, D.E., & Mixon, D.G. (2006). Availability of periodically inspected systems with Markovian wear and shocks. Journal of Applied Probability 43(2): 303317.CrossRefGoogle Scholar
17. Kharoufeh, J.P. & Mixon, D.G. (2009). On a Markov-modulated shock and wear process. Naval Research Logistics 56(6): 563576.CrossRefGoogle Scholar
18. Kharoufeh, J.P., Solo, C., & Ulukus, M.Y. (2010). Semi-Markov models for degradation-based reliability. IIE Transactions 42(8): 599612.CrossRefGoogle Scholar
19. Ko, Y.M. & Byon, E. (2017). Condition-based joint maintenance optimization for a large-scale system with homogeneous units. IIE Transactions 49(5): 493504.Google Scholar
20. Konidaris, G.D., Osentoski, S., & Thomas, P. (2011). Value function approximation in reinforcement learning using the Fourier basis. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, San Francisco, CA. The AAAI Press.Google Scholar
21. Marseguerra, M., Zio, E., & Podofillini, L. (2002). Condition-based maintenance optimization by means of genetic algorithms and Monte Carlo simulation. Reliability Engineering and System Safety 77(2): 151165.CrossRefGoogle Scholar
22. McCall, J.J. (1965). Maintenance policies for stochastically failing equipment: A survey. Management Science 11(5): 493524.CrossRefGoogle Scholar
23. Nicolai, R.P. & Dekker, R. (2008). Optimal maintenance of multi-component systems: A review. In Kobbacy, K. & Murthy, D. (eds.), Complex system maintenance handbook, London: Springer London, pp. 263286.CrossRefGoogle Scholar
24. Pham, H. & Wang, H. (1996). Imperfect maintenance. European Journal of Operational Research 94(3): 425438.CrossRefGoogle Scholar
25. Pierskalla, W. & Voelker, J. (1976). A survey of maintenance models: The control and surveillance of deteriorating systems. Naval Research Logistics Quarterly 23(3): 353388.CrossRefGoogle Scholar
26. Powell, W.B. (2007). Approximate dynamic programming: solving the curses of dimensionality. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
27. Puterman, M.L. (2005). Markov decision processes: discrete stochastic dynamic programming, 2nd ed. Hoboken, NJ: Wiley.Google Scholar
28. Rummery, G.A. & Niranjan, M. (1994). On-line Q-learning using connectionist systems. Technical report, Cambridge University Engineering Department.Google Scholar
29. Shaked, M. & Shanthikumar, J.G. (2007). Stochastic orders. New York, NY: Springer.CrossRefGoogle Scholar
30. Sharma, A., Yadava, G.S., & Deshmukh, S.G. (2011). A literature review and future perspectives on maintenance optimization. Journal of Quality in Maintenance Engineering 17(1): 525.CrossRefGoogle Scholar
31. Sherif, Y.S. & Smith, M.L. (1981). Optimal maintenance models for systems subject to failure – A review. Naval Research Logistics Quarterly 28(1): 4774.CrossRefGoogle Scholar
32. Singpurwalla, N. (1995). Survival in dynamic environments. Statistical Science 10(1): 86103.CrossRefGoogle Scholar
33. Sutton, R.S. & Barto, A.G. (1998). Reinforcement learning: an introduction. Cambridge, MA: The MIT Press.Google Scholar
34. Tian, Z. & Liao, H. (2011). Condition-based maintenance optimization for multi-component systems using proportional hazards model. Reliability Engineering and System Safety 96(5): 581589.CrossRefGoogle Scholar
35. Topkis, D.M. (1998). Supermodularity and complementarity. Princeton, NJ: Princeton University Press.Google Scholar
36. Ulukus, M.Y., Kharoufeh, J.P., & Maillart, L.M. (2012). Optimal replacement policies under environment-driven degradation. Probability in the Engineering and Informational Sciences 26(3): 405424.CrossRefGoogle Scholar
37. Valdez-Flores, C. & Feldman, R. (1989). A survey of preventive maintenance models for stochastically deteriorating single-unit systems. Naval Research Logistics 36(4): 419446.3.0.CO;2-5>CrossRefGoogle Scholar
38. Wang, H. (2002). A survey of maintenance policies of deteriorating systems. European Journal of Operational Research 139(3): 469489.CrossRefGoogle Scholar
39. Yang, W., Tavner, P.J., Crabtree, C.J., Feng, Y., & Qiu, Y. (2012). Wind turbine condition monitoring: Technical and commercial challenges. Wind Energy 17(5): 673693.CrossRefGoogle Scholar
40. Ye, Z.S., Wang, Y., Tsui, K.L., & Pecht, M. (2013). Degradation data analysis using Wiener processes with measurement errors. IEEE Transactions on Reliability 62(4): 772780.CrossRefGoogle Scholar
41. Zhu, Q., Peng, H., & van Houtum, G. (2015). A condition-based maintenance policy for multi-component systems with a high maintenance setup cost. OR Spectrum 37: 10071035.CrossRefGoogle Scholar