Hostname: page-component-857557d7f7-8wkb5 Total loading time: 0 Render date: 2025-11-21T23:57:12.562Z Has data issue: false hasContentIssue false
  • English
  • Français

Benchmarking AI design skills: insights from ChatGPT's participation in a prototyping hackathon

Published online by Cambridge University Press:  16 May 2024

Daniel Nygård Ege ,
Henrik H. Øvrebø ,
Vegar Stubberud ,
Martin Francis Berg ,
Martin Steinert  and
Håvard Vestad
Daniel Nygård Ege*
Affiliation:
Norwegian University of Science and Technology, Norway
Henrik H. Øvrebø
Affiliation:
Norwegian University of Science and Technology, Norway
Vegar Stubberud
Affiliation:
Norwegian University of Science and Technology, Norway
Martin Francis Berg
Affiliation:
Norwegian University of Science and Technology, Norway
Martin Steinert
Affiliation:
Norwegian University of Science and Technology, Norway
Håvard Vestad
Affiliation:
Norwegian University of Science and Technology, Norway
*
danieneg@stud.ntnu.no

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

This study provides insights into the capabilities and performance of generative AI, specifically ChatGPT, in engineering design. ChatGPT participated in a 48-hour hackathon by instructing two participants who acted out its instructions, successfully designing and prototyping a NERF dart launcher that finished second among six teams. The paper highlights the potential and limitations of generative AI as a tool for ideation, decision-making, and optimization in engineering tasks, demonstrating the practical applicability of generating viable design solutions under real-world constraints.

Keywords

prototypingChatGPTartificial intelligence (AI)engineering design

Information

Type
Artificial Intelligence and Data-Driven Design
Information
Proceedings of the Design Society , Volume 4: DESIGN 2024 , May 2024 , pp. 1999 - 2008
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2024.

References

Ahmad, A., Waseem, M., Liang, P., Fahmideh, M., Aktar, M.S., Mikkonen, T., 2023. Towards Human-Bot Collaborative Software Architecting with ChatGPT, in: Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering, EASE '23. Association for Computing Machinery, New York, NY, USA, pp. 279–285. https://doi.org/10.1145/3593434.3593468Google Scholar
Badini, S., Regondi, S., Frontoni, E., Pugliese, R., 2023. Assessing the capabilities of ChatGPT to improve additive manufacturing troubleshooting. Advanced Industrial and Engineering Polymer Research 6, 278–287. https://doi.org/10.1016/j.aiepr.2023.03.003CrossRefGoogle Scholar
Chakrabarti, A., Shea, K., Stone, R., Cagan, J., Campbell, M., Hernandez, N.V., Wood, K.L., 2011. Computer-Based Design Synthesis Research: An Overview. Journal of Computing and Information Science in Engineering 11. https://doi.org/10.1115/1.3593409CrossRefGoogle Scholar
Ege, D.N., Goudswaard, M., Gopsill, J., Steinert, M., Hicks, B., 2024a. What, how and when should i prototype? an empirical study of design team prototyping strategies at the idea challenge hackathon. Under review.CrossRefGoogle Scholar
Ege, D.N., Øvrebø, H., Stubberud, V., Berg, M.F., Elverum, C.W., Steinert, M., Vestad, H., 2024b. The Trolllabs Open Dataset. Available at SSRN: https://doi.org/10.2139/ssrn.4720643CrossRefGoogle Scholar
Filippi, S., 2023. Measuring the Impact of ChatGPT on Fostering Concept Generation in Innovative Product Design. Electronics 12, 3535. https://doi.org/10.3390/electronics12163535CrossRefGoogle Scholar
Flus, M., Hurst, A., 2021. Design at hackathons: new opportunities for design research. Des. Sci. 7, e4. https://doi.org/10.1017/dsj.2021.1Google Scholar
Giunta, L., Gopsill, J., Kent, L., Goudswaard, M., Snider, C., Hicks, B., 2022. Pro2Booth: Towards an Improved Tool for Capturing Prototypes and the Prototyping Process. Proc. Des. Soc. 2, 415–424. https://doi.org/10.1017/pds.2022.43CrossRefGoogle Scholar
Hwang, A.H.-C., 2022. Too Late to be Creative? AI-Empowered Tools in Creative Processes. CHI Conference on Human Factors in Computing Systems Extended Abstracts 1–9. https://doi.org/10.1145/3491101.3503549CrossRefGoogle Scholar
McCormack, J., Dorin, A., Innocent, T., 2004. Generative Design: A Paradigm for Design Research. DRS Biennial Conference Series.Google Scholar
Mountstephens, J., Teo, J., 2020. Progress and Challenges in Generative Product Design: A Review of Systems. Computers 9, 80. https://doi.org/10.3390/computers9040080CrossRefGoogle Scholar
Nelson, M.D., Goenner, B.L., Gale, B.K., 202Utilizinging ChatGPT to assist CAD design for microfluidic devices. Lab Chip 23, 3778–3784. https://doi.org/10.1039/D3LC00518FCrossRefGoogle Scholar
OpenAI, 2023. GPT-4 Technical Report. https://doi.org/10.48550/arXiv.2303.08774CrossRefGoogle Scholar
Purcell, A.T., Gero, J.S., 1996. Design and other types of fixation. Design Studies, Special Issue: Design Cognition and Computation 17, 363–383. https://doi.org/10.1016/S0142-694X(96)00023-3Google Scholar
Qadir, J., 2023. Engineering Education in the Era of ChatGPT: Promise and Pitfalls of Generative AI for Education, in: 2023 IEEE Global Engineering Education Conference (EDUCON). Presented at the 2023 IEEE Global Engineering Education Conference (EDUCON), pp. 1–9. https://doi.org/10.1109/EDUCON54358.2023.10125121CrossRefGoogle Scholar
Regenwetter, L., Nobari, A.H., Ahmed, F., 2022. Deep Generative Models in Engineering Design: A Review. Journal of Mechanical Design 144. https://doi.org/10.1115/1.4053859CrossRefGoogle Scholar
Sajja, R., Ramirez, C.E., Li, Z., Demiray, B.Z., Sermet, Y., Demir, I., 2024. Integrating Generative AI in Hackathons: Opportunities, Challenges, and Educational Implications. https://doi.org/10.48550/arXiv.2401.17434CrossRefGoogle Scholar
The Next Wave of Intelligent Design Automation - SPONSORED CONTENT FROM AUTODESK, 2018. Harvard Business Review.Google Scholar
Tholander, J., Jonsson, M., 2023. Design Ideation with AI - Sketching, Thinking and Talking with Generative Machine Learning Models, in: Proceedings of the 2023 ACM Designing Interactive Systems Conference, DIS '23. Association for Computing Machinery, New York, NY, USA, pp. 1930–1940. https://doi.org/10.1145/3563657.3596014Google Scholar
Thoring, K., Huettemann, S., Mueller, R.M., 2023. THE AUGMENTED DESIGNER: A RESEARCH AGENDA FOR GENERATIVE AI-ENABLED DESIGN. Proc. Des. Soc. 3, 3345–3354. https://doi.org/10.1017/pds.2023.335CrossRefGoogle Scholar
Thunström, A.O., 2022. We Asked GPT-3 to Write an Academic Paper about Itself--Then We Tried to Get It Published [WWW Document]. Scientific American. URL https://www.scientificamerican.com/article/we-asked-gpt-3-to-write-an-academic-paper-about-itself-mdash-then-we-tried-to-get-it-published/ (accessed 11.13.23).Google Scholar
Tiro, D., 2023. The Possibility of Applying ChatGPT (AI) for Calculations in Mechanical Engineering, in: Karabegovic, I., Kovačević, A., Mandzuka, S. (Eds.), New Technologies, Development and Application VI, Lecture Notes in Networks and Systems. Springer Nature Switzerland, Cham, pp. 313–320. https://doi.org/10.1007/978-3-031-31066-9_34Google Scholar
Vestad, H., Kriesi, C., Sl\aattsveen, K., Steinert, M., 2019. Observations on the Effects of Skill Transfer through Experience Sharing and In-Person Communication, in: Proceedings of the Design Society: International Conference on Engineering Design. Cambridge University Press, pp. 199–208. https://doi.org/10.1017/dsi.2019.23Google Scholar
Wang, X., Anwer, N., Dai, Y., Liu, A., 2023. ChatGPT for design, manufacturing, and education. Procedia CIRP, The 33rd CIRP Design Conference 119, 7–14. https://doi.org/10.1016/j.procir.2023.04.001CrossRefGoogle Scholar