Hostname: page-component-857557d7f7-h6shg Total loading time: 0 Render date: 2025-11-22T19:33:49.074Z Has data issue: false hasContentIssue false
  • English
  • Français

A Reinforcement Learning Approach to Predicting Human Design Actions Using a Data-Driven Reward Formulation

Published online by Cambridge University Press:  26 May 2022

M. H. Rahman ,
A. E. Bayrak  and
Z. Sha
M. H. Rahman
Affiliation:
University of Arkansas, United States of America
A. E. Bayrak
Affiliation:
Stevens Institute of Technology, United States of America
Z. Sha*
Affiliation:
The University of Texas at Austin, United States of America
*
zsha@austin.utexas.edu

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this paper, we develop a design agent based on reinforcement learning to mimic human design behaviours. A data-driven reward mechanism based on the Markov chain model is introduced so that it can reinforce prominent and beneficial design patterns. The method is implemented on a set of data collected from a solar system design problem. The result indicates that the agent provides higher prediction accuracy than the baseline Markov chain model. Several design strategies are also identified that differentiate high-performing designers from low-performing designers.

Keywords

artificial intelligence (AI)human behaviourdesign thinking

Information

Type
Article
Information
Proceedings of the Design Society , Volume 2: DESIGN2022 , May 2022 , pp. 1709 - 1718
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2022.

References

Bayrak, A.E. and Sha, Z. (2020), “Integrating Sequence Learning and Game Theory to Predict Design Decisions Under Competition”, Journal of Mechanical Design, Vol. 143 No. 5, available at: 10.1115/1.4048222.Google Scholar
Burnap, A., Ren, Y., Gerth, R., Papazoglou, G., Gonzalez, R. and Papalambros, P.Y. (2015), “When Crowdsourcing Fails: A Study of Expertise on Crowdsourced Design Evaluation”, Journal of Mechanical Design, Vol. 137 No. 3, available at: 10.1115/1.4029065.Google Scholar
Egan, P. and Cagan, J. (2016), “Human and computational approaches for design problem-solving”, Experimental Design Research, Springer, pp. 187–205.Google Scholar
Fuge, M., Peters, B. and Agogino, A. (2014), “Machine Learning Algorithms for Recommending Design Methods”, Journal of Mechanical Design, Vol. 136 No. 10, available at: 10.1115/1.4028102.CrossRefGoogle Scholar
Gensch, D.H. and Recker, W.W. (1979), “The Multinomial, Multiattribute Logit Choice Model”, Journal of Marketing Research, SAGE Publications Inc, Vol. 16 No. 1, pp. 124–132, available at: 10.2307/3150883Google Scholar
Gero, J.S. and Kannengiesser, U. (2014), “The Function-Behaviour-Structure Ontology of Design BT - An Anthology of Theories and Models of Design: Philosophy, Approaches and Empirical Explorations”, in Chakrabarti, A. and Blessing, L.T.M. (Eds.), , Springer London, London, pp. 263–283.Google Scholar
Jang, B., Kim, M., Harerimana, G. and Kim, J.W. (2019), “Q-Learning Algorithms: A Comprehensive Classification and Applications”, IEEE Access, Vol. 7, pp. 133653–133667.CrossRefGoogle Scholar
Kannengiesser, U., Gero, J.S., De Smet, C.M. and Peeters, J.A. (2009), “An ontology of computer-aided design”, Computer-Aided Design Research and Development, In Computer-Aided Design Research and Development. Hauppauge, NY, USA: Nova Science Publishers.Google Scholar
McComb, C., Cagan, J. and Kotovsky, K. (2016), “Drawing Inspiration From Human Design Teams for Better Search and Optimization: The Heterogeneous Simulated Annealing Teams Algorithm”, Journal of Mechanical Design, Vol. 138 No. 4, available at: 10.1115/1.4032810.CrossRefGoogle Scholar
Rahman, M.H., Gashler, M., Xie, C. and Sha, Z. (2018), “Automatic clustering of sequential design behaviors”, Proceedings of the ASME Design Engineering Technical Conference, Vol. 1B-2018, available at: 10.1115/DETC201886300.Google Scholar
Rahman, M.H., Schimpf, C., Xie, C. and Sha, Z. (2019), “A Computer-Aided Design Based Research Platform for Design Thinking Studies”, Journal of Mechanical Design, Vol. 141 No. 12.CrossRefGoogle Scholar
Rahman, M.H., Xie, C. and Sha, Z. (2021), “Predicting Sequential Design Decisions Using the Function-Behavior-Structure Design Process Model and Recurrent Neural Networks”, Journal of Mechanical Design, pp. 1–46, available at: 10.1115/1.4049971.Google Scholar
Rahman, M.H., Yuan, S., Xie, C. and Sha, Z. (2020), “Predicting human design decisions with deep recurrent neural network combining static and dynamic data”, Design Science, Cambridge University Press, Vol. 6, p. e15, available at: 10.1017/dsj.2020.12Google Scholar
Raina, A., McComb, C. and Cagan, J. (2019), “Learning to design from humans: Imitating human designers through deep learning”, Journal of Mechanical Design, Vol. 141 No. 11, available at: 10.1115/1.4044256CrossRefGoogle Scholar
Sexton, T. and Ren, M.Y. (2017), “Learning an Optimization Algorithm Through Human Design Iterations”, Journal of Mechanical Design, Vol. 139 No. 10, p. 101404, available at: 10.1115/1.4037344CrossRefGoogle Scholar
Sutton, R.S. and Barto, A.G. (2018), Reinforcement Learning: An Introduction, A Bradford Book, Cambridge, MA, USA.Google Scholar
Wu, H., Ghadami, A., Bayrak, A.E., Smereka, J.M. and Epureanu, B.I. (2021), “Impact of Heterogeneity and Risk Aversion on Task Allocation in Multi-Agent Teams”, IEEE Robotics and Automation Letters, Vol. 6 No. 4, pp. 7065–7072, available at: 10.1109/LRA.2021.3097259CrossRefGoogle Scholar
Xie, C., Schimpf, C., Chao, J., Nourian, S. and Massicotte, J. (2018), “Learning and teaching engineering design through modeling and simulation on a CAD platform”, Computer Applications in Engineering Education, Vol. 26 No. 4, pp. 824–840, available at 10.1002/cae.21920.Google Scholar