Hostname: page-component-7f64f4797f-fz5kh Total loading time: 0 Render date: 2025-11-10T01:24:37.978Z Has data issue: false hasContentIssue false
  • English
  • Français

THE AUGMENTED DESIGNER: A RESEARCH AGENDA FOR GENERATIVE AI-ENABLED DESIGN

Published online by Cambridge University Press:  19 June 2023

Katja Thoring ,
Sebastian Huettemann  and
Roland M. Mueller
Katja Thoring*
Affiliation:
Technical University of Munich, Germany;
Sebastian Huettemann
Affiliation:
Berlin School of Economics and Law, Germany
Roland M. Mueller
Affiliation:
Berlin School of Economics and Law, Germany
*
Thoring, Katja, Technical University of Munich, Germany, katja.thoring@tum.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Generative AI algorithms that are able to generate creative output are progressing at tremendous speed. This paper presents a research agenda for Generative AI-based support for designers. We present examples of existing applications and thus illustrate the possible application space of Generative AI reflecting the current state of this technology. Furthermore, we provide a theoretical foundation for AI-supported design, based on a typology of design knowledge and the concept of evolutionary creativity. Both concepts are discussed in relation to the changing roles of AI and the human designer. The outlined research agenda presents 10 research opportunities for possible AI-support to augment the designer of the future. The results presented in this paper provide researchers with an introduction to and overview of Generative AI, as well as the theoretical understanding of potential implications for the future of the design discipline.

Keywords

Artificial intelligenceCreativityComputational design methodsDesign knowledgeGenerative AI

Information

Type
Article
Information
Proceedings of the Design Society , Volume 3: ICED23 , July 2023 , pp. 3345 - 3354
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press

References

AIVA, 2022. AIVA - The AI composing emotional soundtrack music, URL https://www.aiva.ai/ (accessed 12.5.22).Google Scholar
Artbreeder, 2022. Artbreeder, URL https://www.artbreeder.com/ (accessed 12.5.22).Google Scholar
Arvanitidis, G., Hansen, L.K., Hauberg, S., 2018. Latent Space Oddity: on the Curvature of Deep Generative Models, in: International Conference on Learning Representations.Google Scholar
Bahmani, S., Park, J.J., Paschalidou, D., Tang, H., Wetzstein, G., Guibas, L., Van Gool, L., Timofte, R., 2022. 3D-Aware Video Generation. arXiv:2206.14797.Google Scholar
Buonamici, F., Carfagni, M., Furferi, R., Volpe, Y., Governi, L., 2020. Generative Design: An Explorative Study. Computer-Aided Design and Applications, 18, 144–155.CrossRefGoogle Scholar
DALL-E 2, 2022. DALL-E 2 - a new AI system that can create realistic images and art from a description in natural language. OpenAI. URL https://openai.com/dall-e-2/ (accessed 12.5.22).Google Scholar
Gero, J.S., 2007. AI EDAM at 20: Artificial intelligence in designing. AI EDAM, 21, 17–18.Google Scholar
Gero, J.S., Kazakov, V.A., 1996. An Exploration-Based Evolutionary Model of a Generative Design Process. Computer-Aided Civil and Infrastructure Engineering, 11, 211–218.CrossRefGoogle Scholar
Ghose, S., Prevost, J.J., 2021. AutoFoley: Artificial Synthesis of Synchronized Sound Tracks for Silent Videos with Deep Learning. IEEE Transactions on Multimedia, 23, 1895–1907.CrossRefGoogle Scholar
GitHub, 2022. GitHub Copilot - Your AI pair programmer, GitHub. URL https://github.com/features/copilot (accessed 12.6.22).Google Scholar
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 2020. Generative adversarial networks. Communications of the ACM, 63, 139–144.CrossRefGoogle Scholar
Gunaratnam, D.J., Gero, J.S., 1994. Effect of representation on the performance of neural networks in structural engineering applications. Computer-Aided Civil and Infrastructure Engineering. 9, 97–108.CrossRefGoogle Scholar
Halperin, T., Hakim, H., Vantzos, O., Hochman, G., Benaim, N., Sassy, L., Kupchik, M., Bibi, O., Fried, O., 2021. Endless loops: detecting and animating periodic patterns in still images. ACM Transactions on Graphics, 40, 1–12.CrossRefGoogle Scholar
Hélie, S., Sun, R., 2010. Incubation, insight, and creative problem solving: a unified theory and a connectionist model. Psychological Review, 117, 994.CrossRefGoogle Scholar
Houde, S., Ross, S.I., Muller, M., Agarwal, M., Martinez, F., Richards, J., Talamadupula, K., Weisz, J.D., 2022. Opportunities for Generative AI in UX Modernization, Houde, Stephanie, et al. “Opportunities for Generative AI in UX Modernization.” Joint International Conference on Intelligent User Interfaces Workshops, 11.Google Scholar
Hwang, A.H.-C., 2022. Too Late to be Creative? AI-Empowered Tools in Creative Processes, in: CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI 22). ACM, New Orleans LA USA, pp. 1–9.Google Scholar
Jasper, 2022. Jasper - AI Copywriting & Content Generation for Teams, URL https://www.jasper.ai/ (accessed 12.5.22).Google Scholar
Karadag, I., Güzelci, O.Z., Alaçam, S., 2022. EDU-AI: a twofold machine learning model to support classroom layout generation. Construction Innovation.Google Scholar
Karpathy, A., Fei-Fei, L., 2015. Deep Visual-Semantic Alignments for Generating Image Descriptions. Proceedings of the IEEE conference on computer vision and pattern recognition.Google Scholar
Kicinger, R., Arciszewski, T., De Jong, K., 2005a. Evolutionary computation and structural design: A survey of the state-of-the-art. Computers & structures, 83, 1943–1978.CrossRefGoogle Scholar
Kicinger, R., Arciszewski, T., De Jong, K., 2005b. Parameterized versus generative representations in structural design: an empirical comparison, in: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation. pp. 2007–2014.CrossRefGoogle Scholar
Kingma, D.P., Welling, M., 2013. Auto-Encoding Variational Bayes. arXiv:1312.6114.Google Scholar
Kopf, J., Matzen, K., Alsisan, S., Quigley, O., Ge, F., Chong, Y., Patterson, J., Frahm, J.-M., Wu, S., Yu, M., Zhang, P., He, Z., Vajda, P., Saraf, A., Cohen, M., 2020. One Shot 3D Photography. ACM Transactions on Graphics, 39(4).CrossRefGoogle Scholar
McClelland, R.S., 2022. Generative design and digital manufacturing: using AI and robots to build lightweight instrument structures, in: Padilla-Vivanco, A., Johnson, R.B., Mahajan, V.N., Thibault, S. (Eds.), Current Developments in Lens Design and Optical Engineering, p. 37.Google Scholar
Melobytes, 2022. AI in the service of art, URL https://melobytes.com/en/ (accessed 12.5.22).Google Scholar
Midjourney, 2023. Midjourney. URL https://www.midjourney.com/ (accessed 4.3.23).Google Scholar
Müller, R.M., Thoring, K., 2011. Understanding artifact knowledge in design science: Prototypes and products as knowledge repositories, in: Proceedings of the 17th Americas Conference on Information Systems (AMCIS). Detroit, USA, pp. 1933–1941.Google Scholar
Nash, C., Carreira, J., Walker, J., Barr, I., Jaegle, A., Malinowski, M., Battaglia, P., 2022. Transframer: Arbitrary Frame Prediction with Generative Models. arXiv:2203.09494.Google Scholar
Nobari, A.H., Rashad, M.F., Ahmed, F., 2021. CreativeGAN: Editing Generative Adversarial Networks for Creative Design Synthesis. In: Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference.Google Scholar
NVIDIA, 2022a. NVIDIA Canvas: Harness The Power Of AI, NVIDIA. URL https://www.nvidia.com/en-us/studio/canvas/ (accessed 12.5.22).Google Scholar
NVIDIA, 2022b. Omniverse Audio2Face App, URL https://www.nvidia.com/en-us/omniverse/apps/audio2face/ (accessed 12.5.22).Google Scholar
OpenAI, 2023. GPT-4 Technical Report. https://cdn.openai.com/papers/gpt-4.pdf (accessed 3.4.23)Google Scholar
OpenAI, 2022. ChatGPT: Optimizing Language Models for Dialogue, OpenAI. URL https://openai.com/blog/chatgpt/ (accessed 12.5.22).Google Scholar
Passalis, N., Doropoulos, S., 2021. deepsing: Generating Sentiment-aware Visual Stories using Cross-modal Music Translation. Expert Systems with Applications, 164, 114059.CrossRefGoogle Scholar
Poole, B., Jain, A., Barron, J.T., Mildenhall, B., 2022. DreamFusion: Text-to-3D using 2D Diffusion. arXiv:2209.14988.Google Scholar
PromptBase, 2022. PromptBase, URL https://promptbase.com/ (accessed 12.5.22).Google Scholar
Studios, Replica, 2022. Replica Studios - Synthesize Voice AI and Natural Sounding Text-to-Speech, Replica Studio. URL https://replicastudios.com/ (accessed 12.5.22).Google Scholar
Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Müller, K.-R., 2019. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer Nature.CrossRefGoogle Scholar
Sarica, S., Song, B., Low, E., Luo, J., 2019. Engineering knowledge graph for keyword discovery in patent search, in: Proceedings of the International Conference on Engineering Design. Cambridge University Press, pp. 2249–2258.Google Scholar
Särmäkari, N., Vänskä, A., 2022. ‘Just hit a button!’ – fashion 4.0 designers as cyborgs, experimenting and designing with generative algorithms. International Journal of Fashion Design, Technology and Education, 15, 211–220.Google Scholar
Siemon, D., Elshan, E., de Vreede, T., Oeste-Reiß, S., de Vreede, G.-J., Ebel, P., 2022. Examining the Antecedents of Creative Collaboration with an AI Teammate. In: Proceedings of the International Conference on Information Systems.Google Scholar
Simonton, D.K., 1999a. Origins of genius: Darwinian perspectives on creativity. Oxford, New York.CrossRefGoogle Scholar
Simonton, D.K., 1999b. Creativity as blind variation and selective retention: Is the creative process Darwinian? Psychological Inquiry, 10, 309–328.Google Scholar
Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., Parikh, D., Gupta, S., Taigman, Y., 2022. Make-A-Video: Text-to-Video Generation without Text-Video Data. arXiv:2209.14792..Google Scholar
Sonix, 2022. Automatically convert audio and video to text: Fast, Accurate, & Affordable, Sonix. URL https://sonix.ai/ (accessed 12.5.22).Google Scholar
Sutherland, S., Egger, B., Tenenbaum, J., 2022. Building 3D Generative Models from Minimal Data. arXiv:2203.02554..CrossRefGoogle Scholar
Synthesia, 2022. AI Video Generation Platform, URL https://www.synthesia.io/ (accessed 12.5.22).Google Scholar
Thoring, K., Mueller, R.M., Desmet, P., Badke-Schaub, P., 2022. Toward a Unified Model of Design Knowledge. Design Issues 38, 17–32.CrossRefGoogle Scholar
Thoring, K., Müller, R.M., 2011. Understanding the creative mechanisms of design thinking: an evolutionary approach, in: Proceedings of the Second Conference on Creativity and Innovation in Design (DESIRE). Eindhoven, NL, pp. 137–147.CrossRefGoogle Scholar
Uizard, 2022. App & Web Design Made Easy | Powered By AI | Uizard, URL https://uizard.io/ (accessed 12.6.22).Google Scholar
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I., 2017. Attention Is All You Need. Presented at the 31st Conference on Neural Information Processing Systems (NIPS 2017).Google Scholar
Verganti, R., Vendraminelli, L., Iansiti, M., 2020. Innovation and Design in the Age of Artificial Intelligence. Journal of Product Innovation Management, 37, 212–227.CrossRefGoogle Scholar
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Liu, G., Tao, A., Kautz, J., Catanzaro, B., 2018. Video-to-Video Synthesis. arXiv:1808.06601.Google Scholar
WZRD, 2022. WZRD - Generating videos from images, URL https://wzrd.ai/ (accessed 12.5.22).Google Scholar
Yang, Xinyue, 2022. Scribbling Speech by Xinyue Yang - Experiments with Google, URL https://experiments.withgoogle.com/scribbling-speech (accessed 12.6.22).Google Scholar
Yi, K., Gan, C., Li, Y., Kohli, P., Wu, J., Torralba, A., Tenenbaum, J.B., 2020. CLEVRER: CoLlision Events for Video REpresentation and Reasoning. arXiv:1910.01442.Google Scholar
Zhao, C., Yang, J., Xiong, W., Li, J., 2021. Two Generative Design Methods of Hospital Operating Department Layouts Based on Healthcare Systematic Layout Planning and Generative Adversarial Network. Journal of Shanghai Jiaotong University (Science), 26, 103–115.CrossRefGoogle Scholar