Skip to main content Accessibility help
×
Home

Ensemble of surrogates and cross-validation for rapid and accurate predictions using small data sets

  • Reza Alizadeh (a1), Liangyue Jia (a2), Anand Balu Nellippallil (a3), Guoxin Wang (a2), Jia Hao (a2), Janet K. Allen (a1) and Farrokh Mistree (a1)...

Abstract

In engineering design, surrogate models are often used instead of costly computer simulations. Typically, a single surrogate model is selected based on the previous experience. We observe, based on an analysis of the published literature, that fitting an ensemble of surrogates (EoS) based on cross-validation errors is more accurate but requires more computational time. In this paper, we propose a method to build an EoS that is both accurate and less computationally expensive. In the proposed method, the EoS is a weighted average surrogate of response surface models, kriging, and radial basis functions based on overall cross-validation error. We demonstrate that created EoS is accurate than individual surrogates even when fewer data points are used, so computationally efficient with relatively insensitive predictions. We demonstrate the use of an EoS using hot rod rolling as an example. Finally, we include a rule-based template which can be used for other problems with similar requirements, for example, the computational time, required accuracy, and the size of the data.

Copyright

Corresponding author

Author for correspondence: Guoxin Wang, E-mail: wangguoxin@bit.edu.cn

References

Hide All
Acar, E (2015) Effect of error metrics on optimum weight factor selection for ensemble of metamodels. Expert Systems with Applications 42, 27032709. doi:10.1016/j.eswa.2014.11.020
Acar, E and Rais-Rohani, M (2009) Ensemble of metamodels with optimized weight factors. Structural and Multidisciplinary Optimization 37, 279294.
Adhav, R, Samad, A and Kenyery, F (2015) Design optimization of electric centrifugal pump by multiple surrogate models. Paper Presented at the SPE Middle East Oil & Gas Show and Conference, Manama, Bahrain.
Alizadeh, R, Lund, PD, Beynaghi, A, Abolghasemi, M and Maknoon, R (2016) An integrated scenario-based robust planning approach for foresight and strategic management with application to energy industry. Technological Forecasting and Social Change 104, 162171. doi:10.1016/j.techfore.2015.11.030
Alizadeh, R, Allen, JK and Mistree, F (2019) Managing computational complexity using surrogate models: a critical review. Research in Engineering Design. Under Review.
Arias-Montano, A, Coello, CAC and Mezura-Montes, E (2012) Multi-objective airfoil shape optimization using a multiple-surrogate approach. Paper Presented at the IEEE Congress on Evolutionary Computation (CEC), Brisbane, Australia.
Babaei, M and Pan, I (2016) Performance comparison of several response surface surrogate models and ensemble methods for water injection optimization under uncertainty. Computers & Geosciences 91, 1932. doi:10.1016/j.cageo.2016.02.022
Badhurshah, R and Samad, A (2015) Multiple surrogate based optimization of a bidirectional impulse turbine for wave energy conversion. Renewable Energy 74, 749760. doi:10.1016/j.renene.2014.09.001
Basudhar, A (2012) Selection of anisotropic kernel parameters using multiple surrogate information. Paper Presented at the 12th AIAA Aviation Technology, Integration, and Operations (ATIO) Conference and 14th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference, Indianapolis, Indiana.
Bellary, SAI and Samad, A (2017) An alternative approach to surrogate averaging for a centrifugal impeller shape optimisation. International Journal of Computer Aided Engineering and Technology 9, 6283. doi:10.1504/ijcaet.2017.080769
Bellary, SAI, Adhav, R, Siddique, MH, Chon, B-H, Kenyery, F and Samad, A (2016) Application of computational fluid dynamics and surrogate-coupled evolutionary computing to enhance centrifugal-pump performance. Engineering Applications of Computational Fluid Mechanics 10, 171181. doi:10.1080/19942060.2015.1128359
Bellucci, JP and Bauer, KW Jr (2017) A Taylor series approach to the robust parameter design of computer simulations using kriging and radial basis function neural networks. International Journal of Quality Engineering and Technology 6, 137160.
Beynaghi, A, Moztarzadeh, F, Shahmardan, A, Alizadeh, R, Salimi, J and Mozafari, M (2016) Makespan minimization for batching work and rework process on a single facility with an aging effect: a hybrid meta-heuristic algorithm for sustainable production management. Journal of Intelligent Manufacturing 30, 3345. doi:10.1007/s10845-016-1223-0
Bhat, S, Viana, FAC, Lind, R and Haftka, R (2010) Control-oriented design using H-infinity synthesis and multiple surrogates. Paper Presented at the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Orlando, Florida.
Bhattacharjee, KS, Singh, HK, Ray, T and Branke, J (2016) Multiple surrogate assisted multiobjective optimization using improved pre-selection. Paper Presented at the 2016 IEEE Congress on Evolutionary Computation (CEC).
Bhattacharjee, KS, Singh, HK and Tapabrata, R (2018) Multiple surrogate assisted many-objective optimization for computationally expensive engineering design. ASME Journal of Mechanical Design 140, 051403.
Bishop, CM (1995) Neural Networks for Pattern Recognition. Birmingham, UK: Department of Computer Science and Applied Mathematics Aston University, Oxford University Press.
Bodnar, R and Hansen, S (1994) Effects of austenite grain size and cooling rate on Widmanstätten ferrite formation in low-alloy steels. Metallurgical and Materials Transactions A 25, 665675.
Chaudhuri, A and Haftka, RT (2014) Efficient global optimization with adaptive target setting. AIAA Journal 52, 15731578. doi:10.2514/1.J052930
Chaudhuri, A, Haftka, RT, Ifju, P, Chang, K, Tyler, C and Schmitz, T (2015) Experimental flapping wing optimization and uncertainty quantification using limited samples. Structural and Multidisciplinary Optimization 51, 957970. doi:10.1007/s00158-014-1184-x
Ezhilsabareesh, K, Rhee, SH and Samad, A (2018) Shape optimization of a bidirectional impulse turbine via surrogate models. Engineering Applications of Computational Fluid Mechanics 12, 112. doi:10.1080/19942060.2017.1330709
Goel, T, Haftka, RT, Shyy, W and Queipo, NV (2006) Ensemble of surrogates. Structural and Multidisciplinary Optimization 33, 199216. doi:10.1007/s00158-006-0051-9
Goel, T, Haftka, RT, Shyy, W and Queipo, NV (2007) Ensemble of surrogates. Structural and Multidisciplinary Optimization 33, 199216. doi:10.1007/s00158-006-0051-9
Habib, A, Kumar Singh, H and Ray, T (2017) A multiple surrogate assisted evolutionary algorithm for optimization involving iterative solvers. Engineering Optimization 49, 120. doi:10.1080/0305215X.2017.1401068
Jägle, E (2007) Modelling of Microstructural Banding During Transformations in Steel (Master of Philosophy Dissertation). University of Cambridge, Cambridge, UK.
Jones, S and Bhadeshia, H (1997) Kinetics of the simultaneous decomposition of austenite into several transformation products. Acta Materialia 45, 29112920.
Jones, SJ and Bhadeshia, HKDH (2017) Program STRUCTURE on the Materials Algorithm Project Web Site. Available at http://www.msm.cam.ac.uk/map/steel/programs/structure.html.
Kaleibari, SS, Beiragh, RG, Alizadeh, R and Solimanpur, M (2016) A framework for performance evaluation of energy supply chain by a compatible network data envelopment analysis model. Scientia Iranica. Transaction E, Industrial Engineering 23, 19041917.
Khademi, A, Ghorbani Renani, N, Mofarrahi, M, Rangraz Jeddi, A and Mohd Yusof, N (2013) The best location for speed bump installation using experimental design methodology. Promet - Traffic and Transportation 25, 565574. doi:10.7307/ptt.v25i6.1188
Kleijnen, JP (2017) Regression and Kriging metamodels with their experimental designs in simulation: a review. European Journal of Operational Research 256, 116.
Korda, AA, Mutoh, Y, Miyashita, Y, Sadasue, T and Mannan, S (2006) In situ observation of fatigue crack retardation in banded ferrite–pearlite microstructure due to crack branching. Scripta Materialia 54, 18351840.
Krauss, GB (2003) Solidification, segregation, and banding in carbon and alloy steels. Metallurgical and Materials Transactions A 34, 781792.
Lim, D, Ong, Y-S, Jin, Y and Sendhoff, B (2007) A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation. Paper Presented at the Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, England.
Liu, K, Tovar, A, Nutwell, E and Detwiler, D (2015) Thin-walled compliant mechanism component design assisted by machine learning and multiple surrogates. Paper Presented at the SAE 2015 World Congress & Exhibition, Detroit, Michigan.
Lv, Z, Zhao, J, Wang, W and Liu, Q (2018) A multiple surrogates based PSO algorithm. Artificial Intelligence Review, 122. doi:10.1007/s10462-017-9601-3
Mack, Y, Goel, T, Shyy, W, Haftka, R and Queipo, N (2005) Multiple surrogates for the shape optimization of bluff body-facilitated mixing. Paper Presented at the 43rd AIAA Aerospace Sciences Meeting and Exhibit, Reno, Nevada. doi:10.2514/6.2005-333
Mirjalili, S (2019) Evolutionary Radial Basis Function Networks. In Mirjalili, S (ed.), Evolutionary Algorithms and Neural Networks: Theory and Applications, Vol. 2. Cham: Springer International Publishing, pp. 105139.
Montgomery, DC (2017) Design and Analysis of Experiments. Hoboken, NJ: John Wiley & Sons.
Nellippallil, AB, Rangaraj, V, Gautham, BP, Singh, AK, Allen, JK and Mistree, F (2017) A goal-oriented, inverse decision-based design method to achieve the vertical and horizontal integration of models in a hot rod rolling process chain. Paper Presented at the ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Cleveland, Ohio, USA.
Nellippallil, AB, Rangaraj, V, Gautham, BP, Singh, AK, Allen, JK and Mistree, F (2018) An inverse, decision-based design method for integrated design exploration of materials, products and manufacturing processes. ASME Journal of Mechanical Design 140, 111403111417.
Qiu, Q, Li, B and Feng, P (2016) Optimal design of hydraulic excavator working device based on multiple surrogate models. Advances in Mechanical Engineering 8. doi:10.1177/1687814016647947
Razavi, S, Tolson, BA and Burn, DH (2012) Review of surrogate modeling in water resources. Water Resources Research 48, 132. doi:10.1029/2011WR011527
Samad, A, Kim, K-Y, Goel, T, Haftka, RT and Shyy, W (2006) Shape optimization of turbomachinery blade using multiple surrogate models. Paper Presented at the ASME 2006 2nd Joint U.S.-European Fluids Engineering Summer Meeting Collocated With the 14th International Conference on Nuclear Engineering, Miami, Florida, USA.
Samad, A, Lee, K-D, Kim, K-Y and Haftka, R (2007) Application of multiple-surrogate model to optimization of a dimpled channel. Paper Presented at the 7th World Congresses of Structural and Multidisciplinary Optimization, Seoul, Korea.
Shankar Bhattacharjee, K, Kumar Singh, H and Ray, T (2016) Multi-objective optimization with multiple spatially distributed surrogates. ASME Journal of Mechanical Design 138, 091401091410. doi:10.1115/1.4034035
Shi, R, Liu, L, Long, T and Liu, J (2016) An efficient ensemble of radial basis functions method based on quadratic programming. Engineering Optimization 48, 12021225. doi:10.1080/0305215X.2015.1100470
Shyy, W, Tucker, PK and Vaidyanathan, R (2001) Response surface and neural network techniques for rocket engine injector optimization. Journal of Propulsion and Power 17, 391401.
Song, X, Lv, L, Li, J, Sun, W and Zhang, J (2018) An advanced and robust ensemble surrogate model: extended adaptive hybrid functions. Journal of Mechanical Design 140, 041402041409. doi:10.1115/1.4039128
Spitzig, W (1983) Effect of sulfide inclusion morphology and pearlite banding on anisotropy of mechanical properties in normalized C-Mn steels. Metallurgical Transactions A 14, 271283.
Tomita, Y (1995) Effect of modified austemper on tensile properties of 0·52%C steel. Materials Science and Technology 11, 994997.
Viana, FAC and Haftka, RT (2008) Using multiple surrogates for minimization of the RMS Error in meta-modeling. Paper Presented at the ASME 2008 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Brooklyn, New York, USA.
Viana, F, Haftka, R, Steffen, JV, Butkewitsch, S and Leal, MF (2008) Optimal use of multiple surrogate for reduced RMS error in meta-model. Paper Presented at the NSF Engineering Research and Innovation Conference, Knoxville, Tennessee.
Viana, FA, Picheny, V and Haftka, RT (2009) Conservative prediction via safety margin: design through cross-validation and benefits of multiple surrogates. Paper Presented at the ASME 2009 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, San Diego, California, USA.
Viana, F, Haftka, R and Watson, L (2010) Why not run the efficient global optimization algorithm with multiple surrogates? Paper Presented at the 51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, Orlando, Florida.
Viana, FA, Haftka, RT and Watson, LT (2013) Efficient global optimization algorithm assisted by multiple surrogate techniques. Journal of Global Optimization 56, 669689.
Villanueva, D, Haftka, RT, Le Riche, R and Picard, G (2013) Locating multiple candidate designs with surrogate-based optimization. Paper Presented at the 10th World Congress on structural and multidisciplinary optimization, Orlando, Florida, USA.
Wang, H, Ye, F, Li, E and Li, G (2016) A comparative study of expected improvement-assisted global optimization with different surrogates. Engineering Optimization 48, 14321458. doi:10.1080/0305215X.2015.1115645
Xu, J and Zeger, SL (2001) The evaluation of multiple surrogate endpoints. Biometrics 57, 8187.
Yin, H, Fang, H, Wen, G, Gutowski, M and Xiao, Y (2018) On the ensemble of metamodels with multiple regional optimized weight factors. Structural and Multidisciplinary Optimization 58, 245263. doi:10.1007/s00158-017-1891-1
Zamani Sabzi, H, Abudu, S, Alizadeh, R, Soltanisehat, L, Dilekli, N and King, JP (2018) Integration of time series forecasting in a dynamic decision support system for multiple reservoir management to conserve water sources. Energy Sources, Part A: Recovery, Utilization, and Environmental Effects 40, 13981416. doi:10.1080/15567036.2018.1476934.
Zerpa, LE, Queipo, NV, Pintos, S and Salager, J-L (2005) An optimization methodology of alkaline–surfactant–polymer flooding processes using field scale numerical simulation and multiple surrogates. Journal of Petroleum Science and Engineering 47, 197208.
Zhou, Q, Wang, Y, Choi, S, Jiang, P, Shao, X, Hu, J and Shu, L (2018) A robust optimization approach based on multi-fidelity metamodel. Structural and Multidisciplinary Optimization 57, 775797.

Keywords

Ensemble of surrogates and cross-validation for rapid and accurate predictions using small data sets

  • Reza Alizadeh (a1), Liangyue Jia (a2), Anand Balu Nellippallil (a3), Guoxin Wang (a2), Jia Hao (a2), Janet K. Allen (a1) and Farrokh Mistree (a1)...

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed