Introduction
Creativity and innovation are quintessential during the conceptual design phase for solving problems effectively in an increasingly complex and technology-driven world. Generating many novel ideas quickly is not only valued but also expected from the designers (Bao et al., Reference Bao, Faas and Yang2018; Sankar and Sen, Reference Sankar and Sen2023). However, this creative ideation process can often be challenging for product or industrial designers. Novice designers face this even more daunting, especially in the early idea generation phase (Chen, Reference Chen2019). This is because the young designers lack experience and suffer from design fixation, mental block, and/or cognitive fatigue during ideation (Gonçalves and Badke-Schaub, Reference Gonçalves and Badke-Schaub2014; Bengtsson et al., Reference Bengtsson, Ruiz and Wever2019). We collectively refer to these cognitive barriers as ideation bottlenecks that hinder their ability to generate novel ideas (detailed in the following Section “Background work”). The use of the word “designer(s)” in this article from here on would refer to product/industrial designers.
Creativity is defined as the production of novel and useful ideas in any domain (Amabile et al., Reference Amabile, Conti, Coon, Lazenby and Herron1996). During ideation, designers are encouraged to engage in divergent thinking while also considering the practical utility of their ideas, which is crucial for addressing real-world challenges (David et al., Reference David, Cropley, Cropley, Kaufman and Baer2005; Dean, Reference Dean2006). Creative thinking in this context is an interplay between divergent and convergent thinking, where each cycle of convergence can lead to the emergence of new ideas, supporting a step-by-step approach to problem solving (Lai et al., Reference Lai, Sun and Yu2004). Ideation is a critical phase in the early stages of product design (Visser, Reference Visser2004), where the aim is to generate a diverse array of ideas, navigating through a “design space” that evolves from abstract concepts to tangible products (Bruno et al., Reference Bruno, Muzzupappa and Rizzuti2003; Chen, Reference Chen2019). It is the activity of generating, developing, and communicating abstract, ambiguous, and imprecise ideas. The activity typically starts with defining the product function and sub-functions, generating ideas for these components, and then integrating them to form a cohesive concept (Bryant et al., Reference Bryant, McAdams, Stone, Kurtoglu and Campbell2005).
In Cheng’s (Reference Cheng2016) study, the keyword-based online resource retrieval process is analyzed. It identified the importance of text in facilitating systematic ideation through the development of a high-level design language. Keywords guide a designer’s self-dialog and information search behaviors, involving word and image associations that contribute to generating creative solutions (Bryant et al., Reference Bryant, McAdams, Stone, Kurtoglu and Campbell2005; Cheng and Do, Reference Cheng and Do2011). There is evidence that designers exhibit thoughtful construction of appropriate linguistic queries for effective information retrieval (Cheng and Do, Reference Cheng and Do2011; Singh and Tomar, Reference Singh and Tomar2023). The ideation process benefits from external stimuli such as “language terms” (Liikkanen and Perttula, Reference Liikkanen and Perttula2010; Goldschmidt, Reference Goldschmidt2011; Sarkar and Chakrabarti, Reference Sarkar and Chakrabarti2011; Gonalves et al., Reference Gonalves, Cardoso and Badke-Schaub2012; Hao et al., Reference Hao, Zhao and Xue2017), enhancing the creative potential of design solutions.
Ideation techniques are integral to the design process and provide designers with a pathway to foster creativity. Over time, design researchers have developed and employed a wide range of ideation techniques comprising various methods and methodologies. Tables 1 and 2 below summarize some commonly used methods and some methodologies used during ideation along with their processes, cognitive principles involved, expected outcome, and their limitations (de Bono, Reference de Bono1970; Benyus, Reference Benyus1997; Börekçi, Reference Börekçi2015; de Bono, Reference de Bono1985; Brown, Reference Brown2009; Chakrabarti et al., Reference Chakrabarti, Siddharth, Dinakar, Panda, Palegar, Keshwani, Chakrabarti and Chakrabarti2017; Chou, Reference Chou2014; Eberle, Reference Eberle1971; Elsen et al., Reference Elsen, Häggman, Honda and Yang2012; Gonçalves and Badke-Schaub, Reference Gonçalves and Badke-Schaub2014; Gordon, Reference Gordon1961; Puccio et al., Reference Puccio, Klarman, Szalay and Glăveanu2022; Book reviews, 1982; Shah et al., Reference Shah, Kulkarni and Vargas-Hernandez2000; Srinivasan et al., Reference Srinivasan, Song, Luo, Subburaj, Elara, Blessing and Wood2023; Tsai, Reference Tsai2011; Tschimmel, Reference Tschimmel2012; Vargas Hernandez et al., Reference Vargas Hernandez, Schmidt and Okudan2012; Yang, Reference Yang2009; Zwicky, Reference Zwicky1969). Each of the methods listed in Table 1 and methodologies listed in Table 2 offers a unique approach to stimulate creativity based on the various cognitive principles listed in the tables and waits for innovative ideas to spur into the designer’s mind. These furnish designers with procedural structures through defined rules, guidelines, and/or procedures for generating ideas, albeit without direct involvement in the ideation process (passive) (Isaksen and Gaulin, Reference Isaksen and Gaulin2005). Moreover, it is mentioned in Bryant et al.’s (Reference Bryant, McAdams, Stone, Kurtoglu and Campbell2005) study that the traditional methodologies focus on stimulating creativity and providing inspiration, leaving designers on their own to create quality ideas. Although some methods, such as brainstorming and morphological analysis, are widely used, they rely on individual’s knowledge and experience (Stone and Wood, Reference Stone and Wood1999) – other ideation methodologies, such as bio-inspiration and analogous thinking, provide connections and inspiration. Traditional ideation approaches are based on established thinking paradigms where the human designer plays an active role. Some program-based ideations automate and integrate the advantages of different traditional ideation approaches, whereas some data-driven ideation (Chen, Reference Chen2019) relies exclusively on analyzing existing data obtained from design experiments.
Distinctive characteristics of some methods used for ideation
Distinctive characteristics of some methodologies used during ideation
A more general observation from Tables 1 and 2 is that the common limitation across these methods and methodologies is their heavy reliance on the designer’s expertise, knowledge base, and cognitive abilities. Most of these approaches are subjective and skill dependent, meaning their effectiveness is directly proportional to the designer’s skill. Additionally, these methods often leave the generation of novel ideas to chance, as they focus on frameworks or processes that guide thinking but do not consistently lead to the generation of innovative ideas. Many aim to explore the problem deeply, assuming that a better understanding of the underlying issues will spark creative solutions, yet this assumes the designer’s cognitive skills can bridge the gap from problem analysis to idea generation. Novice designers often lack the broad knowledge, abstract thinking, and analogical reasoning required to draw meaningful connections or refine raw thoughts. Without experience, they may struggle to use these methods effectively, leading to suboptimal outcomes. Therefore, for novice designers, who are in need of support during the early stages of conceptual design, the prerequisites and nuances of these techniques act as challenges and fail to actively aid and engage them in fruitful ideation.
An effective ideation phase should result in diverse potential designs because multiple variations increase the likelihood of finding novel and innovative solutions (Daly et al., Reference Daly, Yilmaz, Christian, Seifert and Gonzalez2012; He and Luo, Reference He and Luo2017). However, designers often fixate on specific design options early, limiting the variety of designs. This phenomenon is called design fixation, where designers adhere blindly to a limited group of ideas, negatively impacting creativity and reducing the diversity and quantity of generated design concepts (Hao et al., Reference Hao, Zhao and Xue2017). Knowledge and experience can contribute to design fixation, as our preconceived notions restrict the design thinking process. Novice designers may struggle to generate diverse concepts and often exhibit this phenomenon early in design. Experience and expertise play a significant role in creative idea generation, and designers progress through different levels of expertise, though this progression is not necessarily linear (Gonçalves and Badke-Schaub, Reference Gonçalves and Badke-Schaub2014). Thus, the current ideation methods can hinder creativity for experienced designers due to fixation, while novice designers may struggle for meaningful ideation due to a lack of sufficient knowledge and experience. This dual perspective underscores the complexity of balancing experience with creative freedom. Considering these constraints, a more proactive, engaging, and responsive tool is required to overcome the bottlenecks during ideation.
This article explores the potential for using artificial intelligence (AI) for ideation. Particularly, natural language processing (NLP), a subset of AI, has a set of algorithms called large language models (LLM) that has the ability to proactively engage with a human user by generating text responses for input queries. We propose a conversational AI (CAI) tool that could dynamically and contextually interact with designers. The proposed tool employs a state-of-the-art large language model called generative pretrained transformer (GPT), which was fine-tuned and used as a design chatbot to aid designers during idea generation. The purpose of this article is not to undermine the value of traditional ideation techniques, as they remain pivotal in design. Since the ideation bottleneck is a common and known phenomenon, especially among novice designers, it is prudent to explore the possibility of supporting the designers to sail through these phases with the support of an AI integrated tool. To provide such a solution, this study delves into the design, development, and use case of a conversational AI system to facilitate prolific, novel, and diverse idea generation. It acts as an expert omniscience in ideation, enabling a naturalized human-like conversation.
This research focuses on comparing the efficiency of the CAI-based ideation tool with traditional methods. The ideas generated by the proposed tool are compared against the traditional ideation methods, where the responsibility for idea generation lies with the designer. The shortcomings were identified from the comparison, and then the tool was redesigned to provide structured responses based on structured queries. Different examples of such a structured system for some design problems are also given to show the potential of the structured CAI (s-CAI) system. This article aims to explore the potential of CAI-enabled ideation, which we refer to as computer-generated ideation, to help novice designers overcome ideation bottlenecks, unlock their creative abilities, and provide innovative solutions.
Background work
Identification of ideation bottlenecks
In pursuing innovation through the ideation phase of product design, several impediments can hinder designers’ creativity and idea generation. We identify such impediments and categorize them into four primary types, which we term as ideation bottlenecks as summarized in Figure 1. These bottlenecks collectively identified from literature can stifle a designer in creative idea generation. The subsequent explanations of identified bottlenecks are defined by the authors to create a shared understanding within the scope of this study.
Types of ideation bottlenecks.
Lack of experience is a bottleneck where designers may struggle to generate valuable ideas due to insufficient exposure to varied design challenges or a limited knowledge base. This inexperience can lead to a paucity of ideas or the generation of ideas that lack depth and practical applicability, ultimately impacting the quality and novelty of the design solutions proposed.
Design fixation occurs when designers become anchored to their initial ideas. After conceiving a few preliminary ideas, there may be a reluctance or inability to diverge from these nascent solutions. This fixation not only restricts exploration, but also inhibits the consideration of alternative and potentially superior ideas, thereby limiting the scope of the design and its potential for innovation.
Cognitive fatigue is a state where the mental exertion of continuous ideation leads to a depletion of cognitive resources. This fatigue can manifest as a reduced ability to generate a breadth of diverse ideas. It can impede the designer’s ability to make connections between disparate ideas, resulting in a homogenization of ideas that fails to push the boundaries of the design solution space.
Mental block represents a bottleneck where designers find themselves unable to conceive novel ideas. This blockage is often characterized by a blank state of mind, where no new ideas emerge, or a cyclic return to previously discarded ideas. Such a block can be due to various factors, including stress, pressure to innovate, or oversaturation with the problem at hand, leading to a paralysis of the creative process.
Each of these bottlenecks presents a significant challenge in the ideation phase, requiring strategies and interventions tailored to address and overcome them. Recognizing these bottlenecks is the first step toward mitigating their effects and fostering a more fluid and dynamic ideation process.
Classification of ideation based on their characteristics
In the context of product design, the ideation phase is pivotal yet often encounters many cognitive and practical barriers (Butler and Roberto, Reference Butler and Roberto2018). Regarding creativity and ideation, various definitions exist, as discussed in the Introduction. Still, for the present context, we define a creative idea as “an assertive statement that describes ‘what’ part of the solution and needs to be both novel and diverse.” Assessing the degree of novelty in an idea involves measuring how much it deviates from existing solutions and emphasizes the significance of originality, whereas the diversity of ideas refers to how far they are situated in the design space from one another. In this section, the authors identify certain characteristics prevalent among the traditional ideation techniques, characterized by a reliance on self-established thought patterns and conventional thought-provoking techniques. We introduce characteristics of a new ideation methodology that encourages dynamic engagement and the proactive pursuit of novel solutions using technology. Examining the characteristics would help to better understand the ideation tools and help formulate the proposed ideation method as a novel idea generation tool.
Solo ideation
Novel ideas generated during ideation form the foundation for innovative and impactful products. Generating such novel ideas requires interaction between two parties, namely a designer and an ideation technique. However, many existing ideation techniques require the designers to work alone. This is what we refer to as solo ideation. When the ideation technique acts as an aid to the designer, providing a direction of thought for the designer to follow by themselves, then this type of solo ideation is what we term as passive ideation. Passive ideation is defined as one that enables only one-way communication, putting the burden on the designer’s mind to generate ideas, thereby limiting the potential of designers. The following are some of the limitations of passive ideation methods.
Lack of engagement: While valuable, brainstorming or mind-mapping techniques can become monotonous (Isaksen and Gaulin, Reference Isaksen and Gaulin2005; Castillo, Reference Castillo2013). This passive approach could hinder designers from fully exploring unconventional or breakthrough ideas, restricting their creativity. Thus, the traditional ideation methods lack the level of engagement necessary to inspire and ignite the creative process.
Limited collaboration: Many traditional ideation methods are primarily based on personal thought discovery. While individual thinking is essential, harnessing the collective intelligence of a diverse group can lead to more innovative concepts. Passive methods may not facilitate collaboration, resulting in missed opportunities for cross-pollination of ideas and fresh perspectives. Collaboration also requires discussion with experts and peers. Humans, with their diverse personalities, may hinder the generation of novel ideas as they can be biased toward their ideas (Fleury et al., Reference Fleury, Agnès, Cados, Denis-Lutard, Duchêne, Rigaud and Richir2020; Lefebvre and Camarda, Reference Lefebvre and Camarda2024).
Slow iteration: Passive ideation methods often require significant time investment, slowing the iterative design process. Waiting for individual ideas to be shared, analyzed, and consolidated can delay progress and hinder the exploration of alternative concepts. Designers need a more agile, active approach that allows for rapid idea generation, testing, and refinement.
Collaborative ideation
In any collaborative endeavor, there are multiple participating members, often with complementary competencies, who synergistically work toward a common goal. In the context of design, the members involve themselves in continuous interaction for clarification of tasks, resolution of cognitive conflicts, and proposition of new direction of thought, which eventually leads to the generation of an idea. This is what we refer to as collaborative ideation. When one of the members is a machine, such as a computer, that acts as an active contributor to the ideation process, then this type of collaborative ideation is what we term as active ideation. Active ideation is one that enables two-way, interactive, unbiased, adaptive engagement between designers and a machine. Following are some of the advantages of active ideation.
Enhanced engagement: With a higher level of engagement from the machine in the ideation process through textual stimuli, dynamic prompts, and immersive experiences, designers are encouraged to think more creatively, break free from conventional thinking, and explore novel solutions to design challenges.
Dynamic idea exploration: An active ideation machine would encourage designers to think beyond traditional boundaries and experiment with various ideas through multiple quick revisions. By providing the ability to have natural conversational dialogs with the machine, designers can explore different solutions and generate multiple design options that would not have been possible.
Data-driven insights: Active ideation machine would collect and analyze conversational data during the ideation process, providing designers with valuable insights into the effectiveness and feasibility of different ideas. By leveraging real-time answers, designers can make informed decisions, prioritize ideas, and identify emerging patterns, leading to more diverse and detailed ideas.
Potential of AI for ideation
Matching user requirements with novel solutions is a significant challenge due to the multidisciplinary knowledge required by the designer to create potential ideas. Generating many ideas and identifying the most valuable ones is beneficial to developing novel solutions in complex systems and competitive markets (Cai and Lin, Reference Cai and Lin2020). The ideation process has traditionally been a human-centric task, relying on the cognitive abilities of the designer. This approach has significant challenges due to its dependence on designers’ expertise and the above bottlenecks. With the use of computers in different phases of design, there has been a progressive shift toward integrating computer-aided tools into the creative ideation workflow as well.
Computer-aided ideation
Computer-aided ideation tools have existed for some time, offering a framework and environment where designers can explore, manipulate, and visualize ideas. These tools often incorporate databases of knowledge, templates for brainstorming, and mechanisms for capturing and categorizing ideas (Camburn et al., Reference Camburn, Raviselvam, Luo and Wood2020). Some of these tools, such as iDea of Ekströmer (Reference Ekströmer2019), Digital Brainstorming of Bryant et al. (Reference Bryant, McAdams, Stone, Kurtoglu and Campbell2005); Maaravi et al. (Reference Maaravi, Heller, Shoham, Mohar and Deutsch2021); Siegle (Reference Siegle2020), Idea-Inspire of Chakrabarti et al. (Reference Chakrabarti, Siddharth, Dinakar, Panda, Palegar, Keshwani, Chakrabarti and Chakrabarti2017), Bio-Inspire of Benyus (Reference Benyus1997), FuncSION of Pal et al. (Reference Pal, Liu and Chakrabarti2014), Co-storm of Zhang et al. (Reference Zhang, López-Parra, Chen and Tian2019), PANDA of Roderman and Tsatsoulis (Reference Roderman and Tsatsoulis1993), Web-enabled ideation of Beretta et al. (Reference Beretta, Björk and Magnusson2018), among other, have provided significant assistance in the ideation process, helping designers to organize thoughts, inspire creativity, and document the process. They augment the designer’s natural ideation capabilities by providing a digital space for exploration and documentation. However, the creative spark and the inception of novel ideas still originate from the human intellect, with the computer acting as a repository and mediator rather than a generator of ideas.
CAI for collaboration with computers
Let us imagine a situation where computers take a more active role in idea generation rather than being just a facilitator in the ideation process as in computer-aided ideation. The computer would generate ideas, suggest alternatives, and even challenge the designer’s assumptions, and the designer would play the role of a curator of what the computer proposed. That is, the designer’s expertise is utilized to evaluate the ideas and select the potential ones. This symbiotic relationship can be viewed as a Human–Computer Collaboration that capitalizes on the strengths of both computers and humans, combining raw computational power and intuitive judgment.
Recently, large language models (LLMs) from the generative AI domain have gained prominence for their ability to generate diverse forms of original text (Tholander and Jonsson, Reference Tholander and Jonsson2023). It is known that conversational AI (CAI) systems can be tailored to align with human values, as they are designed to emulate intelligent human agents (Tholander and Jonsson, Reference Tholander and Jonsson2023). A salient feature of popular CAI systems is their focus on natural language interaction through chat-based dialogs, resembling human-to-human interactions (Tholander and Jonsson, Reference Tholander and Jonsson2023; Song et al., Reference Song, Zhu and Luo2024). In addition, big data have opened up novel avenues for such CAI by harnessing the potential of large datasets as knowledge bases (Zhu and Luo, Reference Zhu and Luo2023). The authors believe that this ability of modern CAI systems to meaningfully respond to vague queries while exploiting big data resources makes them similar to human experts in creative problem solving.
Computer-generated ideation
Ideation in design can be conceptualized as a cognitive exploration that seeks to bridge the gap between the problem and solution spaces. Traditionally, this process relies on the designer’s ability to access and leverage their knowledge, experience, and expertise. Only a fraction of the designer’s knowledge, which is a small subset of the world’s collective knowledge, gets invoked at the crucial moment for idea generation. This inherent limitation can significantly constrain a designer’s capacity to generate novel, high-quality ideas within a given time. Moreover, the spontaneous invocation of pertinent knowledge is not within the designer’s volitional control, which can lead to cognitive stress and fatigue. This unpredictable nature of retrieval of a designer’s own knowledge underscores the challenges faced in conventional ideation techniques as detailed in Section “Introduction”. Therefore, we believe that timely access and harvesting of principles from a large repository of knowledge is critical for the formulation of creative solutions for difficult practical problems.
Computer-generated ideation using CAI includes the application of LLMs, which can interact with designers through natural language processing (NLP). These LLM models can understand context, generate coherent ideas, assist in the iterative refinement of ideas, and provide a rationale for their suggestions. The LLMs, such as conversational AI, possess two critical attributes that make them invaluable to the ideation process. First, they have access to a vast repository of knowledge far exceeding the scope of any individual designer’s memory. Second, given appropriate inputs (Prompts), they can rapidly generate coherent and contextually relevant sentences as outputs (Responses). The morphology of these sentences can resemble an idea. Thus, although LLMs lack intrinsic cognition and the ability to discriminate, their apparently intelligent responses can potentially be exploited fruitfully. The designer’s responsibility shifts toward critically analyzing and selecting the best ideas, harnessing the full potential of computational power for the generation of novel ideas.
Ideation through CAI
There are two interesting features that make the CAI systems appear to be intelligent:
-
(1) the ability to generate intellectually acceptable write-ups on a given topic and
-
(2) the ability to generate responses to subsequent queries that build upon the previous interactions.
This makes the interaction a coherent conversation on a given topic. Thus, if feature (1) is a description of an idea, then feature (2) could be constructed as an elaboration and clarification of that idea. In the following paragraphs, we elaborate on how these features of one such CAI system, GPT, can be customized and utilized for conversational design ideation. The techniques such as fine-tuning, prompt engineering, and transfer learning described next actually make GPT suitable for design. These techniques are applicable to other CAI systems that leverage any other LLM model as well. We believe that any other CAI system customized through these techniques can also produce relevant responses. However, due to time and resource constraints, this study was conducted using the latest GPT model only.
GPT – a conversational AI
The generative pretrained transformer (GPT) is a state-of-the-art large language model (LLM) developed by OpenAI. As a natural language model, it has been trained to predict the next word in each piece of text, enabling it to generate coherent and contextually relevant sentences (Ray, Reference Ray2023). The GPT-4 model by OpenAI has undergone a complex training regime that involves three core stages: unsupervised pretraining (USPT), supervised fine-tuning (SFT), and reinforcement learning from human feedback (RLHF). These stages collectively train GPT to produce human-like text while adhering to safety and ethical standards, balancing raw performance, and controlled behavior (Chang, Reference Chang2023). Due to its vast size and extensive training, the model can answer questions, write essays, summarize texts, and even translate languages – essentially any task that involves predicting the next word in a sentence (Chang and Chang, Reference Chang and Chang2023).
Characteristics of GPT as a potential tool in ideation
GPT models are pretrained on vast text corpora, enabling them to generate coherent and contextually relevant text responses based on input prompts. We feel that if the input prompt solicits a solution to the problem, GPT would generate a text that could be a practical idea to solve the problem. This characteristic makes GPT a potential asset during the conceptual phase of product design. By engaging in a dialog with GPT, designers can articulate their design challenges, ask questions, and receive prompt and personalized responses similar to brainstorming with multiple people, but instead with an expert omniscient. Furthermore, its capacity to process and synthesize information from various domains can help cross-pollinate ideas, thereby fostering innovation. The iterative interaction with a GPT can help novice designers overcome ideation bottlenecks by presenting a flow of ideas that can be refined and expanded upon, ensuring a dynamic and fluid creative process.
For instance, a GPT model might link a problem in ergonomic furniture design with insights from biomechanics and psychology, fields that may not typically be associated but can provide a deeper understanding of user interaction and comfort. By establishing such connections, designers are empowered to use knowledge more effectively, applying it in contextually appropriate ways to address the problem. The GPT’s role in this process is to act as a cognitive enhancer, expanding the designer’s ability to think laterally and draw upon a wider array of interdisciplinary insights, crucial for innovation and developing holistic design solutions.
Potential benefits of using GPT for ideation
The importance of making connections between knowledge in the design solution space cannot be overstated (Goncher et al., Reference Goncher, Johri, Kothaneth and Lohani2009). Design is inherently an integrative process, requiring the synthesis of various types of knowledge to create solutions that are not only innovative but also practical and feasible (Gero, Reference Gero2022). A GPT model can facilitate this synthesis by identifying patterns and relationships within the data they have learned that might not be immediately apparent to human designers. This ability to make unexpected connections would lead to breakthrough ideas and creative leaps in the design process. GPT models are adept at connecting disparate knowledge to specific problems, a key function during the ideation phase of product design. By drawing from a comprehensive database of information, GPT can bridge the gap between abstract concepts and concrete design challenges. When a designer inputs user needs or a design brief into a GPT-powered tool, the CAI model would be able to analyze the text, identify key themes and requirements, and then scan its vast repository of learned data to generate relevant ideas, analogies, and concepts. This ability of GPT to understand and generate diverse linguistic structures is expected to serve as an ideation partner that can offer novel perspectives and solutions that might not be immediately obvious to human designers. The following are the three important benefits of using a CAI such as GPT for ideation:
-
• Inspiration and knowledge expansion: It acts as a virtual collaborator, inspiring designers by presenting alternative perspectives and facilitating cross-pollination of ideas.
-
• Rapid iteration and feedback: Designers can quickly iterate their ideas by asking for feedback from GPT.
-
• Contextual guidance: GPT can guide designers by asking relevant questions, challenging assumptions, and offering suggestions.
Design of CAI-based ideation interface
An active ideation interface was designed and developed using a conversational AI system known as a generative pretrained transformer (GPT) (we used GPT-4 in our implementation) embedded over an interactive moodboard, as shown in Figure 2. GPT, as a design chatbot, provides the backbone for natural language interaction, allowing it to respond and generate textual statements based on user input (Zhu and Luo, Reference Zhu and Luo2023). The moodboard provides a means for rapidly putting down those ideas. Thus, the interface would provide the designers with a conversational and intuitive platform where GPT drives idea generation.
Interface of the conversational AI-based active ideation tool.
Fine-tuning GPT as an expert designer
The GPT model was fine-tuned to serve as an expert designer for active ideation, which involves initializing the model with a vast and diverse corpus of text data from various relevant online sources, viz., product design-related documents, books, and articles. This exposure to a wide range of design concepts, methodologies, paradigms, and approaches allows the model to develop a broad understanding of the design field. This process can also enable the model to learn the domain-specific language, creative design ideation, innovative problem-solving skills, and various design-thinking strategies applied by expert designers. By doing so, the fine-tuning phase essentially imparts the language model with the essence of design expertise that would allow it to generate insightful, context-specific, and creative ideas in the ideation process (Any mention of GPT from here on refers to this custom fine-tuned GPT model.).
Multisession ideation through contextual understanding
A contextual buffer memory was created and used in context continuation tasks, such as design ideation sessions. This enables sustained engagement in a context-aware conversation for a longer duration. This method of learning from an earlier conversation to keep the context of a current conversation is termed contextual understanding. The contextual buffer memory is designed to store the earlier conversations in a standard JSON file. In the case of an ideation session, if a designer prefers to carry out ideation in multiple sessions/sittings, the contextual buffer memory helps GPT to preserve the context of previous sessions and proceed from where it stopped. Thus, the insights or ideas from previous sessions are not lost, and the creative thinking flow remains consistent.
Nudging GPT for out-of-the-box ideas
The GPT allows setting a parameter called “temperature” to adjust the randomness of its responses. This randomness, interpreted as an index of novelty, can be used to adjust the creativity of the generated ideas. A higher temperature setting leads to more diverse and imaginative responses, encouraging the exploration of unconventional ideas and stimulating creative thinking. On the other hand, a lower temperature value produces more deterministic and focused responses, aligning closely with known patterns and preferences.
Evaluating effectiveness of CAI-based ideation
The potential of CAI-based ideation envisaged earlier needs to be empirically validated. Toward this end, the following two research questions are explored through practical design sessions in which graduate design students participated. This study uses the GPT model to create a design chatbot and a moodboard as an ideation interface using Unity and C# programming as discussed in Section “Design of CAI-based ideation interface”. The design chatbot engages with the designer in a naturalized conversation using text or voice interaction modality to generate ideas as responses for the input problem statements.
RQ1. Is there a benefit to using our conversational AI-embedded ideation over conventional ideation techniques?
Diverse conventional ideation methods are available to help designers provide relevant information and inspiration to generate ideas relevant to the problem at hand. However, due to their rule-ridden passive nature, these traditional methods impose the burden of generation on the designers. On the contrary, CAI-driven ideation takes an active role where the computer takes up the generation, and the designer’s role is in evaluating and selecting potential ideas.
Methodology for RQ1: We propose to use the interface discussed in Section “Design of CAI-based ideation interface” as an ideation tool. Ideas for solving a given problem are sought from the designer, first through the conventional ideation techniques namely synectics, brainstorming, Osborn’s checklist, random words, analogous thinking, and SCAMPER, and then using the CAI-based ideation tool in a time-restricted format. At the end of the experiment, all the ideas are assessed for their novelty and fluency. We then check if there are any significant differences in these parameters in the two modalities of ideation.
RQ2. Does conversational AI-based ideation help novice designers overcome ideation bottlenecks?
Novice designers transitioning from academia to the professional realm often suffer from design fixation due to common exposure to available solutions for mundane problems and mental blocks for significant apparent contradictions in difficult real-world problems. These bottlenecks pose a barrier for the designer, confining them to a small solution space. Such prolonged barriers can lead to cognitive fatigue when the designers withdraw from their ideation endeavors. Therefore, there is a need to support novice designers in overcoming their mental blocks by providing the ability to generate diverse ideas from multiple solution spaces.
Methodology for RQ2: The temperature parameter available in GPT to control the randomness of the responses in terms of their connection to the problem at hand. The designer adjusts this parameter while using the GPT design chatbot and acts on the response by framing elementary ideas in the moodboard provided adjacently. The designer sets a higher temperature setting to get potential responses from unconventional knowledge domains, pending further clarification and elaboration to form the implicit connection. On the contrary, using a lower temperature setting, the response generated itself is likely to be an idea, thereby explicitly showing a connection to the problem without the need for much clarification. The sets of ideas jotted down by the designer are then assessed for their variety.
The aforementioned research questions (RQ1 and RQ2) are answered by formulating three hypotheses (H1, H2, and H3), which are validated by conducting an empirical study as detailed in the following section (Empirical study on ideation).
Empirical study on ideation
To validate the envisaged benefits of the proposed CAI-based ideation interface for novice designers, 30 postgraduate product design students were recruited (Figure 3). The participants were randomly divided into six groups, each assigned a specific design problem statement. Before embarking on the design task, all participants received comprehensive training in utilizing various conventional ideation methods, and a practice session was conducted to ensure their familiarity with the techniques. Ethical considerations were upheld, and proper consent was obtained from all participants for using the study’s outcomes in further research endeavors. The study comprised two distinct parts, Parts A and B, described next.
Participants performing the study.
Comparative ideation study
The comparative ideation study is conducted to assess the effectiveness of idea generation when comparing human designers with conversational AI (CAI). The following sections provide an in-depth look at the study’s methodology.
Performance metrics
The assessment of idea generation effectiveness encompasses various commonly employed metrics, including novelty, variety, quality, and quantity (Shah et al., Reference Shah, Smith and Vargas-Hernandez2003). Generating a higher quantity of ideas can lead to the emergence of higher quality design concepts (Linsey et al., Reference Linsey, Clauss, Kurtoglu, Murphy, Wood and Markman2011). As reported in Shah et al. (Reference Shah, Smith and Vargas-Hernandez2003), novelty pertains to the degree of uniqueness of an idea that could serve as a potential solution to a problem, while variety encompasses the extent of exploration of ideas within the solution space. Novelty (
$ \eta $
), therefore, signifies the distinctiveness of an idea from existing ones, while variety (
$ \upsilon $
) encapsulates the richness of diversity among ideas. Fluency (
$ \Gamma $
) denotes the quantity of ideas generated within a specified period. Meaningfulness (
$ \mu $
) of an idea is about its relevance toward solving the given problem.
Hypothesis formulation
H1: CAI produces meaningful ideas.
During ideation, the main task is proposing methods (ideas) that potentially solve a given problem. Ideas can be communicated through diverse modalities. In the present context, ideas are presented textually. It is known that CAI is capable of generating meaningful textual content. This hypothesis asserts that, with suitable prompts, the GPT-generated statements would constitute meaningful ideas when presented to a designer.
H2: CAI promotes prolific novel ideas.
In a collaborative ideation scenario, the designers continuously engage in a conversation focused on a given problem. The output of such engagement is to generate multiple solutions using their collective knowledge. The level of involvement among designers during this scenario is characterized by the uninterrupted exchange of ideas. The effectiveness of such a collaboration can be assessed by counting the number of quality ideas generated. This hypothesis asserts that the set of ideas generated through CAI would have higher fluency and novelty than those in an unaided exercise using traditional techniques.
H3: CAI promotes diverse ideas.
Past experiences, events, and presented stimuli help the designer formulate creative ideas to solve a problem. The diversity of the ideas in a set refers to how different the ideas are from each other. High diversity is generally preferred as there is a higher potential for an out-of-the-box idea (Sankar and Sen, Reference Sankar and Sen2023). This hypothesis asserts that the diversity of the set of ideas generated through CAI will be higher than the unaided exercise, thereby establishing its potential to alleviate ideation bottlenecks.
Study design
A protocol was designed to conduct the study in two phases for each participating designer.
Part A: Ideas were generated by the designers for an unseen design task using traditional ideation techniques such as Brainstorming, Random Words, SCAMPER, Synectics, Osborn’s checklist, and Analogous Thinking. The allocated time for this exercise is 20 min. Here, there is no usage of a computer or any other digital gadget. The designers had to use their own knowledge resources. Each designer reports a list of ideas for a given problem. The details of the process and proceedings of their exploration are not recorded.
Part B: Participants were instructed to use the CAI-based ideation tool (as discussed in Section “Ideation through CAI”) for idea generation. A training and practice session was held for each participant to familiarize him or her with the interface. The allocated time for this exercise is 20 min. During this time, each designer provides a prompt (a statement seeking ideas for a given problem), and CAI generates responses accordingly. A typical response is presented as a list of ideas in the format shown in Table 3. If any of the ideas are found interesting, they are recorded in an auxiliary text file. The designers employed multisession ideation as discussed in Section “Multi-session ideation through contextual understanding” without any guidance or restriction for the prompts, selections, or temperature settings. Each designer reports the final list of selected ideas. The prompts used and the actual global set of ideas generated by CAI are not recorded. Designers were also not asked about their selection criteria for the ideas.
Set of idea statements generated during Parts A and B of the study
Thus, in both Parts A and B, a designer is given the statement of a problem, and he or she reports the statements of a set of ideas in a format of their choice. Each group comprising five designers was randomly allocated one of the following design problem statements (PS). The designers worked in groups only for the brainstorming session.
-
• PS1: Product for segregation as a means for effective waste management
-
• PS2: Product for footwear disinfection and cleaning for improved hygiene and safety
-
• PS3: Product for enhancing household dish cleaning efficiency and sustainability
-
• PS4: Product for enhancing comfort and efficiency for prolonged standing in queues
-
• PS5: Product for bird-feeding for fostering mental well-being of elderly individuals at home
-
• PS6: Product for convenient umbrella drying and storage on travel
Each of the earlier problems was elaborated through a presentation that covered the background, user needs, challenges, and requirements so that the designers could readily start on the ideation stage of designing.
Study setting
The ideation exercise was conducted wherein each group was assigned one conventional ideation technique during Part A of the study to generate ideas for the randomly assigned problems. Each designer generated ideas individually using the ideation technique assigned to their group. In the end, each group was instructed to create an unordered list of idea statements, contributing as many ideas as they saw fit without any constraint on quantity. The internal dynamics of the groups, such as the influence of dominant personalities or the collaborative synergy, were not monitored during this exercise. The goal was to capture the raw output of ideas generated through traditional and CAI-enabled ideation. The exercise was organized in a way that allowed for a natural flow of creativity and innovation without posing any restrictions to the designer. Part B centered around the refinement and curation of the ideas generated by GPT as a group. While CAI was used in generating ideas, recognizing and selecting potential ideas remained with the designers, underscoring the collaborative nature of the ideation process between CAI and humans. The outcome of this exercise was a curated set of ideas jotted down as textual statements, as shown in Table 3.
Assessment method for novelty and variety
In Part B of the exercise, participating designers shortlisted a few ideas from the large number produced by CAI. This implicitly established that CAI-generated ideas are meaningful for novice designers. However, they did not have to check for how good those ideas were, and they were left to be assessed by expert designers. The purpose of the assessment is to use the two sets of ideas to compare the outcome of the CAI-aided ideation (Part B) and the conventional ideation (Part A) by using prevalent performance metrics, as mentioned in Section “|Performance metrics”. Qualitative (Christensen and Ball, Reference Christensen and Ball2016; Puccio and Cabra, Reference Puccio, Cabra and Mumford2012; Fiorineschi and Rotini, Reference Fiorineschi and Rotini2023; Shah et al., Reference Shah, Smith and Vargas-Hernandez2003.) and quantitative assessment (Bao et al., Reference Bao, Faas and Yang2018; Luo et al., Reference Luo, Sarica and Wood2021) methods are available in the literature. It may be noted that quantitative methods are typically used to assess concepts that have significant functional details. Early-stage ideas, as is the case in this work, do not have the necessary implementational details. Therefore, using those methods to assess early-stage ideas is infeasible and inappropriate. Among the qualitative techniques, the consensual assessment technique (CAT) (Bao et al., Reference Bao, Faas and Yang2018) is a commonly used method that employs experts to evaluate ideas based on predefined criteria, such as novelty, variety, etc., on a Likert rating scale. The experts use intuition derived from their knowledge and experience to assess the viability and feasibility of the ideas. Therefore, we adopted CAT for the assessment.
A Google form was created using the ideas submitted by each group after conventional and CAI-based ideation sessions. To evaluate these ideas from both Parts A and B of the study, they were presented for assessment to 80 design researchers from renowned institutions in India, such as IIT Delhi, IIT Guwahati, IIT Bombay, and IISc Bangalore. The section below presents a comprehensive analysis of their responses (The set of data generated in the study can be accessed by clicking here google drive.). Although CAT advocates assessments by both end-users and experts, the distinction is critical when one considers either a small number of experts or a large number of end-users for the assessments. In our case, since the problems are from ordinary day-to-day experiences and we employed a large number of experts, the basic requirements of CAT are adequately addressed.
Results and outcome
The study undertaken in this research, as discussed in Section “Empirical study on ideation”, was meticulously designed to validate hypotheses H1, H2, and H3, each positing a different aspect of the effectiveness and utility of conversational AI (CAI) in the ideation process. This section details the outcomes of the study discussed earlier, presenting a comprehensive analysis of the findings and their implications.
Validation of hypothesis 1
An expert-driven validation process was used to compare the responses generated by the CAI-based ideation tool with ideas generated by human designers to assess the meaningfulness of ideas generated by CAI. A questionnaire presented six pairs of ideas wherein one idea in each pair was picked from Part A of the activity and the other from Part B, in random order. A panel of 20 expert product designers gave their opinion on which statement they perceived to be a more meaningful idea in each pair. The outcome is illustrated in Figures 4 and 5. It can be observed in Figure 4 that 68% of the experts found the ideas produced by the GPT more meaningful. Moreover, Figure 5 shows that the votes for the GPT-generated statements consistently exceeded those for designer generated ideas. Thus, Hypothesis H1: CAI produces meaningful ideas is validated.
Pie chart depicting the average voting for the meaningfulness of the ideas generated by designers and CAI.
Spider chart depicting the average voting for the meaningfulness of the ideas generated by designers and CAI.
Validation of Hypothesis 2
As indicated in Figure 6, the conventional versus CAI-aided ideation produced an average of 4.8 and 15 ideas in 20 min, respectively. Thus, fluency in Part B is nearly three times that of Part A. This notable increase in idea generation indicates a higher idea flow facilitated by GPT. Therefore, it supports the assertion that CAI can promote prolific ideation.
Bar plot of Fluency (
$ \Gamma $
) – number of ideas generated during Parts A and B.
The novelty ratings of individual ideas were evaluated on a 1–5 Likert scale by 80 experts using an online questionnaire. The findings are depicted in Figure 7. It can be noted that the average rating for Part A is 2.5, while for Part B is 3.86. Therefore, the ratings provided by experts significantly favored CAI-enabled ideation over traditional techniques in terms of novelty. In addition to this, in the box plot analysis shown in Figures 8 and 9, the ratings fall between 3.5 and 4.5 for GPT and 1.8 and 3.2 for traditional, indicating a general consensus among experts that ideas generated by GPT have better novelty. Thus, Hypothesis H2: CAI promotes prolific novel ideas, is validated.
Bar plot of novelty (
$ \eta $
) – uniqueness of ideas in Parts A and B.
Box and Whisker plot for novelty in Part A.
Box and Whisker plot for novelty in Part B.
Validation of Hypothesis 3
Variety or diversity of ideas is a key metric that reflects the effectiveness of an ideation tool for designers. The variety ratings of a set of 12 ideas randomly picked from a larger set were evaluated on a 1–5 Likert scale by 80 experts using an online questionnaire. The protocol ensured that the sampling adequately covered the complete set of about 80 ideas. The findings are depicted in Figure 10. The results show that the average variety rating for Part A is 2.9, while for Part B is 4.2. Therefore, the ratings favored CAI-enabled ideation over traditional methods. This considerable increase in variety demonstrates that GPT generates more diverse ideas. Thus, Hypothesis H3: CAI promotes diverse ideas is validated.
Bar plot of variety (
$ \upsilon $
) – diversity among ideas in Parts A and B.
Inferences and discussion
Every idea can be associated with some cognitive resource (wisdom – experience, knowledge – expertise, information – learning, data – observation, etc., as prescribed by the DIKW pyramid from information science) of the designer (Saulais, Reference Saulais2023). Table 4 illustrates this association for some ideas from Table 3. This implicit connection between knowledge and idea demands that the designers can connect their cognitive resources to the problem at hand while proposing novel ideas. However, voluntary recalling of relevant knowledge is a complex yet unreliable cognitive process. On the other hand, a CAI system is inherently efficient in associating a query with the resources it has been trained with. Given its training using the vast knowledge base, GPT has quick access to in-depth knowledge than an individual designer in any particular domain of relevance. Thus, CAI-enabled ideas scored higher in novelty consideration.
Association of idea with knowledge
Human designers often reach a saturation point where they cannot generate new ideas. This is referred to here as a mental block (one of the ideation bottlenecks). A broader knowledge base from multiple domains is important in helping the designers break free from these ideation bottlenecks (Mostert, Reference Mostert2007). GPT is trained in knowledge across many domains; its inherent stochastic nature ensures that different solutions are generated when queried repeatedly. A designer’s ability to acquire and access knowledge is limited to a select few areas of expertise. Therefore, GPT, unlike humans, does not exhibit saturation and generates more diverse ideas.
Figure 5 showed an interesting outcome: the average number of votes rated for the meaningfulness of an idea by experts for each idea statement was higher for CAI than for the designer across all problem statements. A few expert designers were personally interviewed to understand the rationale behind their choices. They acknowledged that the level of detail contained in a CAI-generated idea was richer, providing them with more insights about the ideas. This depth of detail can be seen as an advantage in providing a rich context for each solution, but there is also a potential risk of cognitive overload. Designers could potentially become overwhelmed by the surplus of information. This would detract the idea, distract the designers from their objective, and diverge them from the original problem.
The above points established that CAI-aided ideation is more beneficial than traditional methods (RQ1 is Answered). By integrating CAI into their workflow, designers shift their primary role from generating ideas to curating them. Thus, it relieves the designers of ideation bottlenecks (RQ2 is answered). This collaborative approach between machine creativity and human selectivity is not reported elsewhere in the literature. However, one limitation of the study is the relatively controlled environment in which it was conducted, using simpler design problems that most people face in their day-to-day lives. This may not fully capture the complexities of real-world challenges. Future studies are required to assess the applicability of our system to more complex tasks.
Structuring the interaction style with CAI
The study earlier has shown the capabilities of CAI in producing a rapid, detailed, novel, and diverse collection of ideas. However, the efficacy of CAI is intricately linked to the specificity of the queries posed to it. Novice designers often struggle to formulate a clear problem statement from the user needs, generate novel ideas, and create feasible concepts (Chang and Kuwata, Reference Chang and Kuwata2020). Therefore, they may not be able to exploit the CAI-based system fully. Structured prompts and responses have been identified as an effective strategy in managing the flow of information, ensuring that responses from LLMs are both relevant and succinct (Lynch et al., Reference Lynch, Jensen, Zamponi, O’Brien, Frydenlund and Gore2023). Therefore, we propose a structured input as described next which encapsulates the essential requirements of design, making it uniformly effective for any designer using our CAI system. A prompt is automatically synthesized from this input for soliciting ideas from CAI based on some essential input given by the designer.
The ideas presented by CAI contain a significant volume of detail in the text. This can potentially induce a cognitive overload for the designers during the selection phase, as they must sift through the extensive information to identify and select potential solutions. The problem gets amplified in practice due to the large number of ideas generated by the CAI. This is a problem of abundance due to CAI, which is typically not experienced in conventional methods. Therefore, we propose a structured format for the responses generated as output by the CAI, which would enable quick information retrieval by the designers. Thus, a novel structure for articulating problem statements from unstructured user needs, generating structured ideas from structured input prompts and synthesizing structured concepts from structured ideas within the CAI framework is given next.
Structuring the problem statement
Defining and understanding a problem’s nature is important for producing novel solutions in product design (Dorst, Reference Dorst, Edmonds and Cross2003). The use of solution neutral problem statement (SNPS) is a standard practice for this purpose. However, designers often find it difficult to state the problem in this format (Chang and Kuwata, Reference Chang and Kuwata2020). Hence, we are proposing a structure that will be used by the CAI to systematically formulate the problem statement by pinpointing the challenges that make the problem difficult, given the user needs as input by the designer. Ideation involves propositions by which these challenges could be overcome. The proposed structure for a problem statement is encapsulated in the “AI3C: Activity-Item-Contradiction-Constraint-Criteria” model (an example of the problem statement structure is shown in Figure 11).
Illustrative example of a problem statement structure.
The proposed problem structure is based on the interaction of two critical elements:
-
1. Element A (Activity): This represents the action or series of actions applied within the problem space.
-
2. Element B (Item): This is the target or subject upon which the activity is performed, leading to a desired state change.
Under ideal conditions, where knowledge and resources are available, the interaction between Elements A and B should not pose any challenge, and state change should be readily achievable. However, in practical scenarios, the interplay of the contradiction, constraint, and criteria complicates this interaction, giving rise to a problem, thereby making the core issue explicit.
Contradiction: It arises when there is a direct conflict between the desired state change and the known relationship between Elements A and B.
Constraints: These are the bounding conditions within which the problem must be solved. These may include technical constraints, such as the maximum weight a material can support; economic constraints, such as budget limits; or regulatory constraints, such as safety and environmental regulations.
Criteria: These represent the benchmarks for evaluating the success of a solution. They are the qualitative and quantitative goals the design must achieve to succeed. These could include performance criteria like speed or efficiency, usability criteria such as user-friendliness, or environmental impact criteria such as carbon footprint.
Structuring the input prompt style for idea generation
The response generated by CAI engines depends on the exact phrase used as input for a given query. Therefore, it is important to formulate the right style for the phrases to invoke more useful responses in the design context. Ideation is a common cognitive activity during the conceptual design phase, which generates propositions by which challenges could be overcome for the problem. The cognitive process of problem-solving involves the stages of searching, learning, synthesis, analysis, and inference (Goel and Pirolli, Reference Goel and Pirolli1992; Dorst, Reference Dorst2003; Wang and Chiew, Reference Wang and Chiew2010). During recursive ideation, the corresponding stages are referred to as exploration (searching), inspiration (learning), generation (synthesis), elaboration (analysis), and evaluation (inference). We map the different types of prompts available in the domain of prompt engineering (Chen et al., Reference Chen, Zhang, Langrené and Zhu2023; Lo, Reference Lo2023) in CAI to be given as input that facilitates each of these ideation stages. A prompt is an assertive sentence used to solicit a response from CAI. Each type of prompt is characterized by a context and a query. A context is defined by the nature of the fields contained in it. The essential fields for each context and the corresponding structure of the prompt are presented in Table 5 with illustrative examples for each stage of ideation.
Stages in ideation with the corresponding prompts, essential fields in each prompt with an example
The key to harnessing the ideation potential of CAI lies in the structured formulation of input prompts and output responses. By carefully crafting prompts, designers can guide CAI in generating structured responses pertinent to the problem. In this way, LLMs are not bottlenecked by the designer’s limited knowledge or the stochastic nature of human cognitive recall. Providing a structure for inputs guides the CAI in understanding the context and constraints of the problem space, ensuring that the generated ideas are relevant and focused. Without such a structure, the CAI produces responses that, while potentially creative and diverse, lack specificity to the task at hand and overwhelm the user with an excess of detailed procedural content, as seen from the results of the study (Table 3). Structured prompts help steer the AI toward the generation of “what to do” ideas rather than “how to do it” processes, thus aiding in the conceptual phase of solution development.
Exploration stage: role prompts
During the exploration stage, the designer looks for existing solutions externally in patents, academic research, markets, and so forth, and internally in their experience and memory that wholly or partially address the problem at hand. The efficacy of the exploration depends on the expertise of the person and the richness of the resource. The richness in the present context will be determined by the specific CAI system adopted. Similarly, in CAI, a role prompt is designed to position CAI as a domain expert, biasing it to source solutions and knowledge from specific fields. For example, if prompted to play the role of a designer, its response to subsequent queries will entail suggestions grounded in design principles, creative problem-solving techniques, industry-specific knowledge, etc.
Inspiration stage: shot prompts
With an understanding of the gap in the existing solutions, the designer advances to the inspiration stage, seeking stimuli from various domains, such as nature, scientific disciplines, and the external environment. Similarly, in CAI, shot prompt is employed to direct CAI to provide concentrated facts (in short phrases) from diverse domains. A shot prompt is crafted to extend beyond the immediate problem domain, encouraging the CAI to draw parallels and identify analogous situations or solutions in seemingly unrelated fields. It stimulates lateral thinking by cross-pollinating ideas from different domains, thereby fostering novel solutions.
Generation stage: open-ended prompts
In the generation stage, the designer leverages their creatively stimulated mind to conceive and formulate new ideas for the given problem. Similarly, in CAI, open-ended prompt is intentionally vague and broad, allowing for a wide range of creative responses. Open-ended prompts encourage CAIs to respond divergently, proposing novel ideas without constraints. This stage is crucial for brainstorming and expanding the horizon of potential solutions, as the CAI generates creative outputs that can be further refined.
Elaboration stage: leading prompts
Following the generation of a basic idea, the elaboration stage involves a deeper contemplation and refinement of an idea to better align it with the potential solution. Similarly, CAI, leading prompt help the designer to guide CAI with specific examples or scenarios, prompting it to expand further and detail the idea. These prompts are targeted, asking the CAI to elaborate on particular aspects of the idea, enriching the idea and adding depth to the proposed solution.
Evaluation stage: option prompts
The final stage, evaluation, occurs when the designer has a set of promising ideas and seeks to either select the most viable ones or amalgamate them to forge new concepts. Similarly, in CAI, option prompt is used by the designer in presenting CAI with a set of shortlisted ideas and instructing it to assess them to evaluate the ideas. These evaluative and comparative prompts enable the CAI to provide critical feedback or combine elements from different ideas to create superior solutions.
A problem statement was taken as an example to illustrate the corresponding prompts and their essential fields. The problem statement is to create an ecofriendly portable water purification device for hikers. This device should be lightweight, durable, and capable of removing contaminants from various water sources encountered in the wilderness. Table 4 displays an example of each prompt in the defined structure that designers can use for respective stages of ideation to converse with CAI.
Structuring the output response style for generated ideas and concepts
CAI systems have the propensity to deliver verbose content, leading to information overload (Huang et al., Reference Huang, Yu, Ma, Zhong, Feng and HaotianWang2023; Ye et al., Reference Ye, Liu, Zhang, Hua and Jia2023; Zhang et al., Reference Zhang, Li, Cui, Cai, Liu, Fu and Huang2023; Xu et al., Reference Xu, Jain and Kankanhalli2024). This impedes the designer’s ability to rapidly assimilate and act upon the information provided. Thus, parallel to establishing a standardized format for input prompts, as seen in the earlier section, a structured format is deemed necessary for the responses given by CAI. This would ensure that the output from CAI is uniform, concise, coherent, consistent, and tailored to the designer’s specific needs.
Structuring the response for idea
Historically, the articulation of ideas has predominantly taken the form of natural language sentences, which inherently consist of a subject and a predicate. These sentences serve as the medium through which designers express the connections they draw between a given problem and their reservoir of knowledge. The composition of these sentences often reveals the underlying structure of the ideation process, elucidating the “what” component of the problem–solution space, that is, what action (verb) applied to what object (noun) could potentially constitute a viable approach to addressing the challenge at hand.
This linguistic representation of ideas is a deeply rooted practice within the design domain, reflecting the natural tendency of designers to think and communicate in structured language patterns (Cheng and Do, Reference Cheng and Do2011). The prevalence of this practice provides a foundation upon which we establish a formalized structure for the presentation of ideas. The proposed structure for an idea is encapsulated in the “AOC: Action-Object-Context” model (an Example of the idea structure for a problem is shown in Figure 12).
Illustrative example of an idea structure.
Illustrative example of a concept structure.
Action is a verb that represents the transformative step or approach proposed to tackle the problem. It is the dynamic aspect of the idea, indicating how the designer envisions altering the current undesirable state.
Object is a noun that specifies the item or entity the action targets or involves. It is the focal point of change, the recipient of the action’s effects.
Context provides the setting or environment in which the idea is situated, offering additional dimensions and considerations that may impact the idea’s implementation and efficacy.
Structuring the response for concept
The transition from ideation to the concrete development of a concept in the design process requires a meticulous and structured approach to ensure that the nascent ideas are transformed into viable solutions. In the design domain, a concept is a proposal that outlines the practicality and the technicalities of “how” an idea can be realized. It is a blueprint that follows the idea and must be grounded in scientific principles to ensure feasibility. The proposed structure for a concept is encapsulated in the “PFIC: Principles-Features-Implementation-Characteristics” model (an example of the concept structure for a problem is shown in Figure 13).
Principles refer to the scientific laws, theories, or techniques that underpin the concept and guarantee its feasibility. These principles are the bedrock upon which the concept stands, providing the rationale and validation for why and how the concept can work in real-world scenarios.
Features detail the various components or attributes of the concept. These are the tangible elements that differentiate one concept from another, providing a clear picture of the concept’s design and functionality.
Implementation outlines the method or process by which the concept will be realized. It bridges the gap between theory and practice, ensuring that the concept can be operationalized and that the transition from article to prototype is feasible.
Characteristics define the qualities or behaviors of the concept, often described by adjectives or interjections. These descriptors will define the concept’s performance, usability, and overall impact. Characteristics provide insight into the concept’s interaction with its environment and end-users.
Redesign of CAI-based ideation interface
The design chatbot of the CAI-based ideation interface discussed in Section “Design of CAI-based ideation interface” was modified to include the structured interaction discussed earlier by providing two separate drop-down menu options for selecting the input prompt type (ideation stage) and output response type (problems, ideas, concepts) as shown in Figure 14a–c. By choosing the ideation stage, a pop-up window requires the designers to fill in the essential fields. The input prompt is automatically created based on a predefined structure where the essential items are obtained from data filled in by the designer in the fields. This prompt autopopulates with the predefined structure of the prompt in the prompt input field. It is the decision of the designer to choose the right stage depending on their requirement. Each prompt type initiates a new CAI conversation whose context is preset based on the user input. The choice of output selection provokes the CAI to provide the corresponding response in the structure defined within the CAI system. A pilot study was undertaken to understand the differences in ideas generated during traditional and unstructured and structured CAI-based ideation. The details of which is given in the following section.
Interaction style selection in the modified design chatbot of the structured CAI interface.
Analysis of traditional and unstructured and structured CAI-based ideation
A pilot study was conducted by the authors to compare the nature of the ideas produced through the traditional brainstorming method and unstructured and structured CAI-based ideation. The problem (PS2), which concerns people’s difficulty in cleaning footwear quickly and easily without causing damage to it, was taken as the case study. A group of five novice designers (senior postgraduate product design students) was chosen for this study. They were apprised of the problem statement and the user needs. These participants were different from those involved in the empirical study discussed in the previous section. These participants engaged in generating as many ideas as possible for the given problem using brainstorming, unstructured CAI, and then using structured CAI in this particular order. The participants were given a practice run to familiarize themselves with the interfaces before they started using them. The ideas generated through these methods were collected, a sample of which is shown in Figure 15. The following inferences were made by the authors on observing the ideas generated from these methods.
Outcome of traditional ideation versus CAI-enabled ideation (unstructured and structured).
Inferences and discussion
Utilizing the traditional brainstorming technique, a group of novice designers (senior postgraduate product design students) engaged in a session to generate solutions for the stated problem. It can be observed from Figure 15 that the ideas produced, while diverse, largely reflected solutions that were analogous to those used for cleaning other items, such as household or automotive cleaning tools. This outcome may be attributed to the “Lack of Experience” bottleneck, where participants reverted to familiar concepts rather than innovating new ones specifically tailored to footwear. In our view, “Design Fixation” also played a role, as evident from the repetitive use of brushes/bristles in the ideas generated. This limited the exploration beyond the initial set of ideas.
The unstructured CAI-based ideation process yielded an array of ideas with detailed text. The CAI produced more diverse solutions in terms of the associated principles. The text mostly contained significant procedural details from which working principles are required to be discovered, viz. “rotating brushes or a sponge” need to be identified in the sentence of 40 words (Figure 15b). This cognitive endeavor is likely to overwhelm the designers, especially when the number of ideas is large, hampering the judicious selection process.
In the structured response, the essential segments of information are segregated and labeled in the presentation. There is no long paragraph or sentence. This reduces the cognitive burden compared to the unstructured format during idea selection, viz. “use ultrasonic waves to dislodge dirt” can be identified even without going through the rest of the text (Figure 15c). Thus, we believe understanding the comparison and selection of ideas from a set would produce less mental workload. Therefore, the presented ideas are clearer to comprehend and easier to grasp.
A notable limitation of unstructured CAI-based ideation is the dependency on the designer’s ability to formulate effective input prompts. This can be particularly challenging for novice designers, whose limited experience might affect the quality and relevance of the generated ideas due to improper inputs. While the authors proposed structured prompts and responses to mitigate this, the effectiveness of such structures in diverse real-world scenarios remains to be validated. Furthermore, the potential cognitive overload from managing and selecting from a large volume of CAI-generated ideas poses a practical challenge, which the authors acknowledge and are currently working toward an automated method.
Summary
The results from the structured CAI-based ideation show the importance of guiding the CAI to focus on generating novel ideas through a structured format. The “Designer as a Curator” model becomes particularly relevant here, where the designer’s role evolves to evaluate and refine the ideas proposed by the CAI, harnessing the computational system’s capabilities while applying human intuition and expertise. The results reveal that while traditional brainstorming can yield a breadth of ideas, these may be limited by cognitive bottlenecks such as experience and fixation. Unstructured CAI-based ideation can overcome these bottlenecks by providing a vast array of detailed solutions. Structured CAI-based ideation could enable the generation of targeted, novel ideas that result in precise solutions that are easy to assimilate, compare, and shortlist by the designers for a given problem due to their segregated nature.
Limitations and caveats
Subjective assessment of idea variety and novelty: The evaluation of the diversity and novelty of ideas generated by the CAI tool was based on subjective assessments from expert reviewers. While these experts have extensive experience in their respective fields, personal biases and perspectives can influence their evaluations. We are working toward developing objective metrics to mitigate these biases.
Dual role of design researchers as experts and end-users: In this study, design researchers acted both as expert evaluators and as end-users due to the simplicity of the problems chosen. While their professional expertise provided valuable insights into the quality and feasibility of the generated ideas, their dual role may introduce bias. Their familiarity with the design process and potential personal preferences could influence their assessments, potentially affecting the objectivity of the evaluation. Future studies should consider involving a broader range of independent industrial experts and end-users to provide a more diverse and unbiased perspective on idea evaluation.
Scope and complexity of design problems: The study focused on relatively simple, everyday design problems. While this approach allowed us to effectively demonstrate the CAI tool’s capabilities, it may not fully capture the complexities encountered in more challenging, real-world design scenarios. Further research is needed to test the tool’s effectiveness in addressing complex design challenges.
Tool interface and user interaction: This study did not extensively explore optimizing the tool’s user interface and user interaction process. User experience, including ease of input and filtering out ideas, may be considered to enhance the overall usability of the tool in a professional studio.
Generalizability of findings: The findings from this study may not be directly applicable to all contexts or industries. The effectiveness of the CAI tool could vary based on the specific requirements and constraints of different fields. Further research is necessary to explore its applicability across a broader range of applications.
Future directions
The CAI system presented earlier is capable of generating many novel and diverse ideas quickly. This abundance makes the shortlisting process more challenging and resource intensive, potentially leading to decision fatigue with possible inconsistencies. The authors are actively involved in developing an automated system (Sankar and Sen, Reference Sankar and Sen2024) to assist designers in efficiently selecting the most promising ideas from the generated pool.
The structured format for input queries and output responses provides convenience to the designer and easy assimilation of information by the CAI system to generate appropriate and relevant responses. This article proposes one way to structure the prompts and responses, showcasing a few examples based on the natural way of communication familiar to the designers. The authors are currently exploring the diverse methods of prompt structuring. The structured response format for ideas also has the potential for employing standard embedding techniques to develop automated idea evaluation schemes. This would obviate the resource-intensive assessment by experts. The authors are actively working to this end (Sankar and Sen, Reference Sankar and Sen2024).
While this study focuses on the generation of ideas using a custom fine-tuned GPT, we recognize the importance of understanding how the responses of different models trained on different sets of data may vary in creative generation, coherence, and reasoning ability. The comparative performance of the different large language models (LLMs) as an effective idea generation CAI system is yet to be fully explored. This future exploration will involve establishing criteria for comparison and assessing the strengths and weaknesses of each model in facilitating ideation.
Conclusion
This article presented a conversational AI-enabled active ideation paradigm to enhance the ideation process, particularly for novice designers. Through empirical study and expert assessments in terms of fluency, novelty, and variety, it is shown that the proposed scheme alleviates the cognitive bottlenecks experienced by designers during conventional ideation and facilitates a dynamic, interactive, and prolific ideation environment. This “machine generates designer curates” paradigm makes the ideation process cognitively less burdening, leading to promising outcomes and making it accessible for less experienced designers. To mitigate the difficulty of formulating appropriate input prompts and deciphering the nuances of the CAI-generated ideas, a novel structured input and output formatting is presented in the later part of the work. Its effectiveness is demonstrated through a representative design problem. Thus, the work established the potential of effective, prolific, and meaningful ideation through a state-of-the-art LLM platform. The authors are currently developing automated evaluation strategies to analyze the idea landscape meaningfully.
Open access
