1. Introduction
As an educational discourse, design has often set itself up as a practice uniquely apart from others. This has manifested from academic design researchers’ early arguments for design as a separate educational discipline with its own unique practices and ways of understanding (Cross Reference Cross1982; Lawson Reference Lawson2004), to its extension into industry via practices such as design thinking (Beverland, Gemser & Karpen Reference Beverland, Gemser and Karpen2017). Since the advent of the digital revolution in the 1980s, design has also increasingly faced acute technological challenges (Drucker & Mcvarish Reference Drucker and Mcvarish2013), from digital publishing, to interactivity, to the increased ease of use of, and automation within, design software. This has resulted in cultural, societal, and organisational challenges such as the increased democratisation of design, which has increasingly allowed non-professionals to carry out tasks that were previously the preserve of dedicated design professionals, and contributes to continually evolving workplace challenges, for example, the move away from design agency services and towards in-house (Geraedts, Verlinden & Stellingwerff Reference Geraedts, Verlinden, Stellingwerff, Horváth, Albers, Behrendt and Rusák2012; Silk & Stiglin Reference Silk and Stiglin2016).
Against this background, the furious development pace and accessibility of advanced artificial intelligence (AI) tools are now beginning to challenge all areas of design (Verganti, Vendraminelli & Iansiti Reference Verganti, Vendraminelli and Iansiti2020). Within academia, educationalists and pedagogical policymakers are increasingly under pressure to adequately respond to challenges from AI (Ouyang & Jiao Reference Ouyang and Jiao2021). ChatGPT has emerged as a primary disrupter, prompting debates about the ethics and extent of its use by students in assignments. Academics and universities continue to adapt to these challenges – with the topic so new and seemingly regularly mutating, many debates currently manifest as unreviewed opinions. For example, some have gone as far as describing ChatGPT as a “plague” (Weissman Reference Weissman2023) to be legislated against. Within academia, others have started to accept that its pervasiveness is such, that drastic mitigation is no longer realistic (Rudolph, Tan & Tan Reference Rudolph, Tan and Tan2023).
With the public launch of ChatGPT in November 2022 (OpenAI 2022), many educational institutions started to look for ways of adapting to this new technological development. Lo (Reference Lo2023) conducted a rapid review by reviewing 50 articles published on or before 28 February 2023, to understand ChatGPT’s capabilities, how it might be used in education, and to identify potential issues since its release. Outcomes from these articles indicated that ChatGPT held promise for improving teaching and learning. However, its knowledge and performance in different subject areas fell short in many areas. For example, it was noted that ChatGPT was sometimes prone to generating inaccurate or deceptive information. Moreover, the risk of student plagiarism was highlighted (and remains) an ongoing concern. For Kwan Lo, this suggested an imperative to take measures to tackle such potential challenges and to optimise the educational application of ChatGPT (Lo Reference Lo2023). Taking a broader view, UNESCO (2019) has suggested that AI can assist educators by creating a professional environment that enables them to devote more effort to students’ challenges. With educators often consumed by repetitive administrative tasks, such as assignment creation and or responding to repetitive questions, the implementation of a dual-educator model (involving human teachers and virtual assistants), could address such routine responsibilities, allowing educators to concentrate on providing more personalised student guidance. Kasneci et al. (Reference Kasneci, Sessler, Küchemann, Bannert, Dementieva, Fischer, Gasser, Groh, Günnemann, Hüllermeier, Krusche, Kutyniok, Michaeli, Nerdel, Pfeffer, Poquet, Sailer, Schmidt, Seidel, Stadler, Weller, Kuhn and Kasneci2023) argue, using large language models in education shows promise for enhancing the student learning experience and assisting teachers. However, it is important to be cautious and critically assess their limitations and potential biases to fully leverage their benefits in education (Kasneci et al. Reference Kasneci, Sessler, Küchemann, Bannert, Dementieva, Fischer, Gasser, Groh, Günnemann, Hüllermeier, Krusche, Kutyniok, Michaeli, Nerdel, Pfeffer, Poquet, Sailer, Schmidt, Seidel, Stadler, Weller, Kuhn and Kasneci2023). Tlili et al. (Reference Tlili, Shehata, Adarkwah, Bozkurt, Hickey, Huang and Agyemang2023) discuss the importance of creating additional guidelines and policies to promote the use of ChatGPT in educational settings, such as schools and universities. It emphasises the need for further research to understand the potential consequences of excessive reliance on chatbots in education. The study acknowledges that, like all disruptive technologies, ChatGPT has advantages, challenges, and flaws that necessitate thoughtful discussion and analysis about its implementation in educational institutions (Tlili et al. Reference Tlili, Shehata, Adarkwah, Bozkurt, Hickey, Huang and Agyemang2023).
As we discuss, existing discourse has largely theorised on potential challenges of teaching using AI, as well as reflected on ethical considerations affecting grading student work. We also discuss how we utilised a self-study methodology (Hamilton et al. Reference Hamilton, Smith and Worthington2008) to engage with generative challenges and opportunities of AI technology in relation to the development of the content of courses for higher education design students, for which there has been sparse published academic research. One outcome of both design-specific and more general challenges from AI for design educators, is that creating assignment briefs for design students has become increasingly complex and fluid, even when targeted at specific disciplines of design. With cross-disciplinary design classes, alongside increasing societal and industry changes to design, the challenge becomes even more complex. Our study pays particular attention to the development of design courses in collaboration with ChatGPT as a “virtual colleague” in the educational design process.
2. Aims, context and significance
From the earliest days of digital design in the 1980s, the impact of (what was then known as) desktop publishing (DTP) software and hardware was revolutionary in its impact on the design industry (Romano & Mitrano Reference Romano and Mitrano2019). What had once taken teams of skilled professionals, including designers, pre-press operators, and printers, could now be achieved often by one person in a fraction of the time and cost. Not only was this revolutionary at a broader industry, cultural and societal level, but it even called into question the design process and design practice itself and whether a reliance of such technologies was negatively affecting design.
For example, as digital design began to mature, design applications began to accept third-party plugins – add-on tools that extended the functionality of a base software application. One of these plugins, for Photoshop, was called Kai’s Power Tools (KPT), which enabled the automatic creation of a series of wonderful abstract effects. One of the most popular effects was the “page curl” filter. As the name implies, it allowed the easy creation of an up-curled page corner to virtual pages. Similarly, a plugin for the 1990s most popular design layout tool QuarkXpress was called the “Starburst” tool. Once again, the name describes the tool’s function, enabling designers to instantly create start-shaped boxes, that could be filled with images, text, gradients, or colours. For a period in the 1990s, both effects became ubiquitous, especially in printed advertising design, to the degree that they eventually became a cliche for hackneyed design techniques and metaphorical symbols of the threat of technology driving design – acute examples of design being negatively affected as a result of being led by software tools, rather than the other way around. One could almost describe it as a form of moral panic, indicative of a fear of new technology.
In that sense, debates around the topic of automated design generation (of which the page curl and starburst tool are early examples) have existed from long before contemporary anxieties about AI’s capacity for the digital disruption of design practices. While one could still debate the degree to which DTP and other digital technologies have impacted design (and they clearly have), it is also evident that designers are still producing creative design outcomes despite (indeed, because of) such technologies. Nevertheless, clearly designers had to adapt to these digital technologies. Similarly, if designers and design educators are to address challenges from AI technologies by accepting it as a tool for design, it is likely to require the capacity for designers to effectively manage and use AI technology as a tool to inform design, rather than as a master that drives it. It is with this in mind that we approached this research, and our engagement with ChatGPT also intersects with narratives around current and future challenges and potential threats to design practices.
2.1. AI in educational discourse
AI technologies have introduced innovative methods for enhancing teaching and learning in higher education (Ouyang, Zheng & Jiao Reference Ouyang, Zheng and Jiao2022), and these potentially offer both opportunities and challenges (Qureshi Reference Qureshi2023). Nevertheless, ChatGPT and similar technologies have only recently emerged in public consciousness and levels of educational literacy with AI remain under investigation. For example, Laupichler et al. (Reference Laupichler, Aster, Schirch and Raupach2022) suggest that while there is emerging interdisciplinary research in the area, levels of scholarly knowledge about AI literacy are still in their infancy. The integration of AI text generators like ChatGPT into academic endeavours, spanning assessment, research, and teaching, is still largely unfamiliar to the academic community, despite the numerous possibilities it offers (Eke Reference Eke2023). Luckin et al. (Reference Luckin, Cukurova, Kent and Du Boulay2022, p. 4) further argue that the majority of educators are “oblivious” to forthcoming impacts of AI on their professions and that cross-disciplinary approaches need to be adopted to empower educators for these changes.
Educational discourse around AI has often revolved around ethical concerns about, for example, plagiarism (Grassini Reference Grassini2023), or for the potential for AI tools programmes to produce harmful outputs (Stokel-Walker & Noorden Reference Stokel-Walker and Noorden2023) such as falsifying information, leading to calls for developing broad practical guidelines for AI’s use in education and academia (Dis et al. Reference Dis, Bollen, Rooij, Zuidema and Bockting2023), as well as technical training for educators and ethical and integrity education for students. Others, such as Pavlik (Reference Pavlik2023), have embraced the potential educational benefits of AI by generatively collaborating with ChatGPT, treating it as an academic co-author. More cautiously, while expounding the tool’s undoubted technical abilities, Grant Cooper cautions that ChatGPT’s current lack of evidential grounding risks “positioning itself as the ultimate epistemic authority, where a single truth is assumed” (Cooper Reference Cooper2023, p. 449).
2.2. Intent
The authors of this article are both current lecturers and course coordinators in design, who are actively engaged in course curriculum planning, development, as well as teaching a variety of design courses to a multidisciplinary student cohort. Acknowledging the realities, threats, and potential opportunities of AI in higher education course development, the authors employ the use of ChatGPT metaphorically as a virtual colleague, with which to collaborate and create a set of higher educational design materials.
In using ChatGPT as a generative designing education tool, we aim to evaluate its functionality and usefulness within the curriculum design process. In doing so, we aim to reflect on the challenges that this process holds and if, to what degree, and in what ways ChatGPT might add value. How well ChatGPT responds to the requirements of developing and formatting a design course, or elements of that course, and what its limits are. Whether the effort involved in collaborating with ChatGPT outweigh its usefulness and the degree with which its output can be trusted and how much and in what form human intervention is required. Our outcomes and recommendations can help to inform how ChatGPT might help course coordinators and program directors to accommodate students from different disciplinary design backgrounds, or who are on different educational pathways, and who might have different levels of design experience. Other outcomes may provide learnings that address and inform potential challenges such as balancing ease of understand with the depth of a topic, aligning with industry goals, making the language content understandable, as well as pitching the course content at the appropriate educational level.
3. Methodology
The aim of this exploratory research is to document our experiences with ChatGPT during course design to reflect on its implications. The approach that we used is a self-study methodology (Hamilton et al. Reference Hamilton, Smith and Worthington2008). Informed by analogous studies into ChatGPT, such as (Cooper Reference Cooper2023), we acknowledge the subjectivity of our approach and that this subjectivity contains limitations resulting, for example, from our personal identities and professional affinities that have influenced the prompts that we gave ChatGPT. Nevertheless, in the context of an exploratory research project these same subjectivities, especially when contextualising outcomes resulting from both researchers’ prompts, allow for a qualitative reflection that might not emerge from more quantitative approaches. Indeed, acknowledging the novel peculiarity of engaging in an exploratory interaction with a non-human respondent, our position as both interviewers, interpreters, and narrators may be contextualised as aligned with qualitative creative practice approaches such as those from ethnography and coterminous disciplines (Erickson Reference Erickson1985; Sandelowski Reference Sandelowski1991; Denzin Reference Denzin2001).
4. Methods
Embracing the reality of readily available AI technology, we decided to conduct a series of conversations with ChatGPT. We chose ChatGPT, as its capacity to comprehend natural language queries and produce human-like responses has made it a popular tool for swiftly obtaining answers to a diverse range of questions (Adiguzel, Kaya & Cansu Reference Adiguzel, Kaya and Cansu2023). Indeed, the program describes itself as a “language model […] designed to understand and generate human-like text based on the input they receive” (ChatGPT 2023). ChatGPT has rapidly become a valuable tool for both students and professionals. The most recent release, ChatGPT-4, was launched in 2023, with claims of increased power and the ability to handle more complex tasks (Adiguzel et al. Reference Adiguzel, Kaya and Cansu2023).
Although not collaborating with ChatGPT overtly as a co-author in the way that Pavlik did (2023), we did treat it as a form of intelligent yet non-sentient academic partner – essentially a virtual colleague that might plausibly have the ability to support, inform, or add value to the process of creating post-graduate course materials. Thus, during the article, we discuss how we engaged in a series of conversations with ChatGPT around the development of different types of educational content, using fictitious higher education design courses as a testing ground. In doing so, we probed ChatGPT’s ability to provide helpful responses, as well as reflecting on how well it was able to critically adapt and respond to our requirements, as well as challenging and evaluating opportunities and limitations. Then, using a reflective approach, we analysed the responses alongside our requirements and the literature to see what outcomes and learnings emerged and to see how such an exercise might influence debates around AI within design education.
While AI in education has emerged as the central research theme within the field of computers and education, the interdisciplinary character of this domain poses a distinctive challenge for scholars coming from diverse design backgrounds (Hwang et al. Reference Hwang, Xie, Wah and Gasevic2020). Thus the relative experience and focus of the authors are relevant and engaged with during the following case studies. Dr Yaron Meron’s teaching of design is influenced by his professional practice, which was originally located in graphic design and branding, and later expanded into service design, design thinking, and strategic design. Dr Yasemin Tekmen Araci has a background in industrial design with many years of experience in higher education, primarily focused on instructing design courses. Her expertise lies in creative design thinking, design innovation, visual communication, human-centred design, and design research methods.
We adopted a semi-structured exploratory conversational approach with ChatGTP. This conversational method was employed so as to engage with, as Rospigliosi (Reference Rospigliosi2023) reflects, ChatGPT’s own approach of engaging in conversations by receiving questions from users and providing responses. These interactions revolved around questions and subsequent inquiries, actively engaging in dynamic conversations, expanding on answers and addressing user challenges (Rospigliosi Reference Rospigliosi2023).
In experimenting with using ChatGPT as a collaborative tool, we also acknowledged the disciplinary distinctions of each author’s design background. Thus, while agreeing a set of parameters of the research, each author engaged with our course design task with ChatGPT independently – essentially functioning as a semi-structured co-design investigation. This also allowed us to focus on a broader set of enquiries, allowing us to reflect on ChatGPT’s responses under different scenarios. For example, Yasemin chose to engage with the creation of an entire semester course, while Yaron focussed on the development of a specific student assignment. We acknowledge that this approach challenges a like-for-like analysis of outcomes. However, acknowledging this limitation is also an identification that such an analysis is unrealistic when ChatGPT will give differing and non-repeatable responses, even to identical questions. Thus, our choice of the method remains subjective, yet is also an intentional acknowledgement of the complexity of researching this topic.
The tasks that we assigned for ourselves – and by extension for ChatGPT as our virtual colleague – were to design content for post-graduate students of a fictional post-graduate design masters unit. While fictional, each course follows a realistic broad outline of actual masters-level course assignments from units that we have taught and developed at Australian higher education universities. The guidelines that we set ourselves were drawn from the following broad parameters: Students would be required to respond to a design challenge that involved them engaging, as designers, with an organisation, brand or digital product that is targeted at a retired demographic, with an approximate age group of over 70. As part of their responses, students would need to engage with demographic-specific aspects of this age group, which would require independent research and justification for their methods and digital design responses.
While following the same initial aim and broad outline, each author comes from a slightly different professional and academic design background and so has adapted specific approaches to fit our respective purpose and disciplinary design perspective. Where applicable this will be discussed in the following respective case studies. Moreover, the guidelines that we set for our experiments were broad, with the aim of allowing for our collaboration with ChatGPT the flexibility to narrow down and develop the course outcome with a wide degree scope. Doing so, also left the door open for us to push the boundaries of ChatGPT’s ability to “do the work” that would normally be done by a course coordinator or programme director.
Unless otherwise stated, our discussions are based on engagement with ChatGPT version 4. Where quotations from our interactions with ChatGPT have been used, we have cleaned up our spelling mistakes and minor typing errors for ease of comprehension. Although transcripts of the vast entirety of our interactions with ChatGPT are not included, the “designed” outcome for each of our courses is included in the Supplementary Material.
4.1. Case study 1: collaborating with ChatGPT to create a design student assignment
Having experimented with ChatGPT previously and mindful of version 4’s improved ability to conduct extended conversations (OpenAI 2023), Yaron decided to create the student assignment in stages. Firstly, ChatGPT was asked to develop the design task that students would be engaging with. This task would involve students responding to a simulated industry design brief. Responding to design briefs is not only a necessary industry skill within almost all design disciplines (Phillips Reference Phillips2014), but is also a useful pedagogical device in its own right (Sadowska & Laffy Reference Sadowska and Laffy2017). Indeed, in both authors’ experience as design educators, design briefs comprise a significant majority of design course assignments. Nevertheless, research into the pedagogy (and use in industry) of design briefs is limited (Phillips Reference Phillips2014), and so it would be intriguing to see how ChatGPT might rise to the task, and so ChatGPT was asked to:
Create a design/creative brief that relates to a digital outcome for people over the age of 67, which helps them learn and navigate digital technology that has, is and will be emerging as part of the 4th industrial revolution? What needs to be considered for such a project, in relation to the outcome itself and, importantly, the demographic being targeted? (ChatGPT 2023).
The request has defined the format of the assignment (a design brief), as well as also assigning some broad parameters and design constraints for the proposed assignment task. However, the format, potential design artefacts or solutions, as well as the outcomes of the design brief have been left up to ChatGPT. ChatGPT replied with an approximately 450 word response, compartmentalised under topic headings. These headings comprised of a project title, a background paragraph, a two-point project objective, a confirmation of the target audience, user needs and considerations, deliverables, a timeline and a four-point success metrics list. Acknowledging the subject of the design brief, ChatGPT titled the project “SilverSurfer: Navigating Digital Horizons,” and called for “an engaging, intuitive, and easy-to-use platform” to “bridge the digital divide” of our demographic. Learning considerations such as an “intuitive user interface” and “personalised learning paths” were also listed by ChatGPT, but were not elaborated on. Similarly, requirements such as “privacy and security” and “relevance of content” were listed, alongside generic examples such as “email,” “internet browsing” and “online banking,” while deliverables included a “fully-functional, accessible digital platform (website/app),” manuals, and a marketing strategy (ChatGPT 2023).
This initial formatted response was an impressive and professionally constructed boilerplate which might plausibly be capable of being used as a guiding template to combine with real data for creating a “real-world” briefing document of some sort. Nevertheless, as the above headings indicate and other researchers have noted (Chen et al. Reference Chen, Chen, Hassani, Yang, Amyot, Lessard, Mussbacher, Sabetzadeh and Varro2023), the content of ChatGPT’s initial responses were extremely generic and would therefore need to be populated with content to be considered as a usable teaching aid. Hoping to build on this start, some targeted follow-up requests were made to try and coax ChatGPT into creating a design briefing project that was more appropriate and usable for the course assignment. The first of which involved incorporating designer–stakeholder engagement into the assignment brief.
4.1.1. Stakeholder engagement
With the aim of encouraging ChatGPT to help generate content that would inform the challenge of stakeholder engagement in the design briefing process, ChatGPT was asked: “Who might you imagine would request such a brief? What kind of stakeholder?” ChatGPT responded with a list comprising of Government Agencies, Non-Profit Organizations, Education Institutions, Healthcare Providers, Technology Companies and Social Enterprises. Each listed stakeholder was accompanied by a short paragraph justifying why they were listed, with a broad summary stating; “The stakeholder would likely have an interest in education, social inclusion, digital literacy, or serving the needs of older adults” (ChatGPT 2023).
The generation of the list was helpful, as it can inform the scope of such an assignment brief for students. Moreover, knowing which stakeholders a design project is for and how to engage and interrogate design briefs (a “return brief”) is a critical part of design student pedagogy. A “return brief” is not a formal term in many design disciplines, but the term is sometimes used to describe the process whereby a design team ask clarification and scoping questions of clients or other stakeholders. This sometimes involves “returning” the brief to stakeholders by asking questions, for example, to acknowledge constraints and limitations of the project. As design educators, we try to teach this as part of some of our courses, because it encourages good practice for designers if they can learn to navigate these challenges of the design briefing process – a process that can become problematic if it is not properly managed (Meron Reference Meron2021). Mindful of this, ChatGPT was next asked; “What questions and clarifications might a design agency ask of such a stakeholder as part of their return brief?”
In response to the query, ChatGPT once again produced a professionally formatted response. Using plausible-sounding headings, with accompanying bullet points, it proposed a series of questions for clarification of the design brief. These included verifying technological requirements; suggesting questions about demographic and geographic data; queries about design accessibility guidelines; timescale, budget and resource requirements; success and key performance indicators; as well as support and maintenance requirement questions. Once again, ChatGPT’s responses were grammatically and structurally impressively formatted. However, as with its previous responses, these were largely generic. For example, under the heading of “Understanding the Project” Chat GPT suggested asking “Can you clarify the key goals you expect this project to achieve?” In isolation and without reference to any data, this is a perfectly acceptable response. Nevertheless, this lack of focus and specificity indicates the requirement for human editing and positioning to integrate this automated content into a real-world teaching environment. However, the planned student assignment could allow a degree of fluidity, with students free to choose what to focus on and so it was decided that ChatGPT’s current output would be enough to support an initial integration into the design of the student assignment. Thus, ChatGPT was asked to start populating this process with content:
How might the above exercise be set/formatted for a post-graduate/masters degree level design assignment? The students come from a variety of design disciplines, including graphic, interface (UI & UX), product and fashion design, as well as some from other non-design professions who are hoping to use design methods as part of their professional practices (Meron via ChatGPT 2023).
ChatGPT did an efficient job of collating the information from the design brief into a design masters degree course assignment format, providing content comprising of an Objective; a brief Background paragraph; a bullet-pointed Assignment Details list, a Deliverables list, some broad Grading Criteria, and other similar categories. As with its previous responses, the formatting essentially comprised a generic template which would require further human teacher intervention to input assignment-specific details. These are aspects that future iterations of ChatGTP or similar tools – especially ones that have improved memory and comprehension of previous conversations – might plausibly be useful for assignment development. At its current stage of development, the role that ChatGPT appears to best simulate is that of a virtual assistant that is capable of providing consistent formatting and structure for human data input. In that sense, it is somewhat reminiscent of existing automation features in design software packages such as InDesign, where recurring items such as page headers and formatting elements can be automatically and consistently recreated based on an initial set of design criteria. These are hugely time-saving features that support designers when creating large documents (for example) and allow designers to focus on the more creative aspects of their role. Such principles have of course been widely expanded in vernacular non-professional design tools, such as Canva (2023), where template-driven automation of design layouts can go far beyond that of traditional professional design software. Albeit, this is intentional. Professional designers want to retain control of their process and output, and so automation in professional design software tends to err towards empowering that process and automating more tedious and less creative aspects (Nolan Reference Nolan2018). Conversely, non-professional online design apps are targeted towards making the design process more accessible for non-designers and so tend to draw on more comprehensive template-driven solutions (Meron Reference Meron2022). This is not an absolute comparison and may be underselling ChatGTP’s abilities, scope and dataset. Moreover, it is problematic to make direct comparisons between such differently focussed tools – and complicated by broader discussions around definitions of AI. But the principle of contextualising such adjacent design programs as accessible software-driven “helper” applications remains intriguing and might benefit from further exploration elsewhere.
4.1.2. Content emphasis
While evaluating this stage of ChatGTP’s output, an emphasis towards technological elements of the design process, over creative and procedural aspects, became apparent. This was particularly important when being mindful about the cross-disciplinary makeup of most courses that the authors teach. To address this, ChatGTP was asked to revisit its outcomes:
That is good, but rather slanted towards a technological development approach. There also seems to be a rather quick a jump towards an artefactual outcome, without the reflective and stakeholder engagement aspect of the design process. Moreover, how might you integrate creative skills that designers working in design disciplines such as advertising, marketing & communications, graphics, branding and fashion might use? Also, non designers are still expected to use design methods (Meron via ChatGPT 2023).
ChatGTP’s response was to expand its existing categories. For example, “Research” became “Research & Empathy,” with an expanded explanatory paragraph. It also added categories, including “Stakeholder Engagement” and “Define,” as well as refining the explanatory text that accompanied each category to acknowledge different design disciplines (ChatGPT 2023). Chat GPT had by now essentially created two distinct project outlines; one comprising broadly of a research and conceptual stage, with the second focussing more on development and design outcomes. Intending to combine these into a more logically coherent design brief, it was suggested to ChatGTP that; “I think that is more on the ball as a broad general brief. Perhaps that might be used first and your previous idea used as a follow-up. What do you think?” “Absolutely, that’s a great approach!,” replied ChatGTP, (notably appearing to acknowledge the conversational style). It continued;
The more generalized brief I provided second encourages students to explore the problem space more broadly, engage deeply with stakeholders, and think creatively about a range of potential solutions […] Following this, the more specific brief I provided first could be used as a subsequent assignment […] This two-part assignment structure allows students to practice a wide range of design skills and methodologies, from broad problem exploration and stakeholder engagement to focused, user-centered digital design (ChatGPT 2023).
Chat GPT was queried for clarification: “If that was done, it would almost be as if the first part is a conceptual design brief and research assignment and the second would be development process that draws on the outcomes of the first. Do you agree?” ChatGPT responded:
Yes, absolutely. The first part of the assignment serves as a comprehensive exploration phase […] The second part of the assignment then builds directly on this foundation. […] In this way, the two parts of the assignment together provide a comprehensive design experience, covering everything from initial problem exploration to the detailed design of a specific solution (ChatGPT 2023).
It was intriguing how these last few human–AI exchanges took on a linguistically collegial format, with ChatGPT exhibiting a virtual exuberance. While obviously a machine-generated faux collegiality, it was intriguing to briefly pursue to what extent the programme was trying to “please” its questioner, and so ChatGPT was asked if its responses were merely acquiescing to human prompts. ChatGPT assured that it was just providing “[…] information and perspectives based on the data I was trained on,” with its goal being to “[…] assist and facilitate productive discussions by offering informative, respectful, and balanced responses” and that:
[…] your suggestion of using the two briefs as a two-part assignment, with the first focusing on conceptual design and research, and the second on development, is in line with accepted practices in design education. Hence, I affirmed that it’s a sound approach (ChatGPT 2023).
Following up on this response, ChatGPT was pressed on whether, “If I had suggested the other way around, might you have highlighted problems with that?” Chat GTP assured that it would have “[…] highlighted potential issues with that approach” (ChatGPT 2023).
Finally, mindful that many design course assignments for students are group projects, ChatGPT was asked to reframe the brief as such. Again, it produced a well-formatted output, repurposing and rephrasing content towards a group assignment and even offering some informative group-specific tasks, such as a “Group Collaboration Plan” and a “Peer Evaluation.” However, ChatGPT had notably omitted some of the content from the previous “approved” assignment format and the final assignment brief content appeared sparse, as well as lacking in guidance for students as to specific tasks and actions required. ChatGPT was asked to clarify the assignment one last time and it finally created a well-formatted and logical assignment proposal, albeit only summarising some of the previous content from the individual assignments. This is perhaps indicative of a current limitation of ChatGPT’s chat history function and limited ability to contextualise its responses (Hariri Reference Hariri2023).
4.1.3. Case study reflection
ChatGPT presents as competent at what it is overtly intended to achieve – that is, to be a language model. As it describes itself; “I’m a type of AI known as a language model, which means I generate text based on the input I receive” (ChatGPT 2023). For the tasks it was asked to respond to, it took human prompts and created plausible-looking, logically laid out, and remarkably well-formatted assignment outlines.
As a course designer, an apparent key limitation of ChatGPT as a tool for creating “real world” assignments for higher education students, is that all of the generated outputs were extremely generic, provided little practical guidance or direction for students, and the formatted output would still need to be edited and refocussed quite extensively by an experienced human design educator. ChatGPT’s outputs essentially functioned as templates, albeit very well-formatted ones. Even as templates, their usefulness was sometimes hard to quantify. For example, while they initially presented as clearly laid out standalone documents, it was often difficult or undesirable to then retrofit the content of the proposed design assignment into the ChatGPT-generated format. Indeed, to make full use of the ChatGPT outputs would require amending the aims and goals of the assignment, thus handing over human agency to ChatGPT to drive the content of the assignment. This is reminiscent of debates similar to those from the early days of DTP, when concerns were raised about DTP software driving the design process, rather than the other way around. Although, it is fair to say that such themes are similarly applicable to template-driven automated design tools such as Canva.
From a design educator’s perspective, ChatGPT’s generated output was most helpful as a “creative instigator” and brainstorming tool which could serve as a suggested repository of subject categories, or to spark ideas of topics for inclusion into a course assignment. However, such automated content generation would then likely require reorganisation and redesign by a human design educator.
The requirement for the agency of a trained design educator is further demonstrated in a follow-up exchange with ChatGPT regarding its apparent emphasis towards technical issues, when it was asked: “Going back to our discussion earlier about the technical focus of your initial responses to my requests in relation to a design course, as opposed to creative aspects of design, what might the reasons for that be?” ChatGPT’s response was partially to state that it was drawing on its data; “[…]my responses are generated based on a combination of the input I receive and the vast amount of text data I was trained on […]” (ChatGPT 2023). ChatGPT also confirmed that because of the technical nature of the product it was asked to create a brief about, it had responded in kind;
When we discuss design in the context of digital technology, it is quite common to emphasize the technical aspects, such as user interface, functionality, and programming, given their critical role in the realization of any digital product or service. Therefore, in our conversation about designing a digital solution for elderly users, my responses might have leaned toward the technical side, highlighting aspects such as prototyping, UX/UI design, and implementation strategy (ChatGPT 2023).
This is suggestive of a technological focussed dataset that is being drawn from which, as discussed, is potentially problematic if ChatGPT is to be used for interdisciplinary design-based assignment generation. Thus, it was queried;
Just because a project is of a digital nature, doesn’t mean that a technological response, solution or focus is the first thing that designers would think of. Given that this was specifically a design project, aimed at a multidisciplinary design cohort, I was surprised at the omission of creative and conceptual to your initial responses (Meron via ChatGPT 2023).
Through several rounds of further query, ChatGPT continued to defer to its data pool and training and emphasised that its technologically-driven response was due to the subject matter of the design brief chosen for the assignment for students. However, it eventually acknowledged that;
You’re correct that designers from various disciplines, including those not directly related to digital or tech fields, would approach a design problem from their unique perspectives. They would employ design thinking processes and tools, such as empathy mapping, user journey mapping, brainstorming, sketching, and prototyping, before delving into the technical specifics of a solution (ChatGPT 2023).
As with earlier responses, it is tempting to muse that part of ChatGPT’s communication persona sometimes appears to be doing its best to appease its human questioners. This is further implied by this response;
I appreciate your patience and insight in highlighting this. In the context of a design project, particularly one involving a multidisciplinary cohort, the focus should indeed be more balanced, incorporating both the creative and conceptual aspects of design alongside the technological. […] This is a valuable point of learning and a useful reminder that the approach to a design problem should not necessarily be primarily technology-driven, but rather should start with a human-centered, creative, and empathetic understanding of the user’s needs and experiences, regardless of the ultimate medium or technology platform in which the solution will be implemented. Again, I appreciate your feedback, and will strive to better balance responses in such contexts in the future (ChatGPT 2023).
On the surface, ChatGPT appears competent at its stated purpose as a language model. It is easy to use and its limitations in practical usage, for the design assignment tasks that it was set, appear to be more focussed on its data set and training. However, such limitations may also be reflective of the difficulty of aligning the intangible nature of some of the more conceptual and creative aspects of many design practices (Heller Reference Heller2019), for example, intuition (Taffe Reference Taffe2017), as well as in design’s inherent interdisciplinarity (Harland Reference Harland, Rodgers and Yee2015), with the more empirical principles of computer science based AI. Such limitations may also be partially attributable to the relative nascency of academic rigour in some of those design disciplines, such as graphic design (Corazzo et al. Reference Corazzo, Harland, Honnor and Rigley2019), which in turn may cast limitations on the data that ChatGPT is able to draw on.
As a design educator drawing on the experience of this exercise, it is possible to envisage using ChatGPT to save time planning and formatting assignments for design students and to generate ideas. Nevertheless, in its current form, it remains a tool that requires knowledgeable and experienced design educator guidance for both the content and format of the prompts it is fed, as well as in the work required to repurpose its outputs for “real world” design education use.
4.2. Case study 2: collaborating with ChatGPT to create a postgraduate design studio course
Initially, Yasemin engaged with ChatGPT to aid in the creation of a unique design studio course aimed at postgraduate level. A course outline within the educational system serves as a roadmap for a course. It encompasses critical details such as the course description, learning outcomes, the entire range of topics to be addressed, and assigned tasks. These core parts of a course outline provide a comprehensive layout for educators and students to follow, guiding the delivery of content while also setting clear expectations for students regarding what they will learn and what will be required of them throughout the course. Therefore Yasemin wanted to create a whole course design with a design brief, learning objectives, assignments and weekly content.
4.2.1. Design brief
The course would centre around the discipline of communication/interaction design and entail students developing projects for an audience of individuals aged 70 and above. To commence the process, Yasemin sought assistance in crafting a comprehensive design brief. This was her first prompt to ChatGPT: “I will be teaching a graduate design studio in the fields of communication and interaction design. Students will be expected to work on developing a design project for people over 70 years of age. Can you provide me three different design brief ideas?” (Araci via ChatGPT 2023).
ChatGPT generated three design brief options, all of which were relevant and reasonable, although lacking in novelty and originality. For example, the first option ChatGPT created was:
Digital well-being for seniors: Create a digital solution that promotes the well-being of seniors aged 70 and above. This could include anything from a mobile app that encourages physical activity, to a social media platform that connects seniors with one another, to a game that helps improve cognitive function. The solution should be accessible, easy to use, and tailored to the unique needs of seniors (ChatGPT 2023).
Despite this, considering the speed at which these briefs were generated, ChatGPT exhibited remarkable time efficiency. It might have taken Yasemin to create these design briefs a couple of hours, but it took ChatGPT a few seconds. Then she asked, “Can you provide me some already existing platforms/software/apps used for similar functions?” (Araci via ChatGPT 2023). ChatGPT provided eight different apps with short explanations for each: SilverSneakers, Lumosity, AARP Now, Stitch, Elder 411, Aging in Place Technology Watch, CareZone, and MyFitnessPal (ChatGPT 2023).
Yasemin was familiar with just a few of the suggestions, so she took the time to look into all of them. She discovered that they are indeed real apps and platforms that fit with the design project. Once again, ChatGPT has been a great help in putting together resources for the project she is working on. Using ChatGPT for research has made the job easier and saved her a lot of time that she would have spent finding these examples. Among the first three design briefs created by ChatGPT, one particularly stood out as promising more opportunities:
Smart home solutions for aging in place: Develop a suite of smart home solutions that make it easier for seniors to age in place. This could include an app that lets seniors control their home’s temperature and lighting from their phone, or a sensor system that alerts caregivers if a senior falls or wanders too far from home. The solutions should be intuitive and easy to use, with a focus on accessibility and safety (ChatGPT 2023).
Yasemin requested further elaboration for this brief, ChatGPT expanded upon the initial suggestion, providing a more comprehensive design brief with additional details. Notably, ChatGPT structured the brief under headings such as objectives, deliverables, key features, target audience, and constraints, demonstrating an impressive ability to organise the information without her explicit request.
4.2.2. Interdisciplinary collaboration
Given her broad teaching experience in multidisciplinary design studios, interacting with students from diverse backgrounds, Yasemin was interested to see how ChatGPT would handle the task of interdisciplinary collaboration within a design project. So, she asked ChatGPT:
How can an interdisciplinary group of graduate design students, having different backgrounds respond to this brief as a team? What professional capabilities can the students bring to the design solution, let’s say in a team of students from computer science, interaction design, design management, and marketing? How can they complement each other in the team towards working on this design brief? (Araci via ChatGPT 2023).
Yasemin was hoping to get an answer about how each student might contribute to the solution of the design brief in the creative idea generation process. However, ChatGPT’s response did not provide a detailed insight into the specific contributions of each team member based on their respective backgrounds. What ChatGPT suggests is simply a workload allocation in a team, like “interaction design students can focus on designing user interfaces” or “design management students can provide project management and leadership skills” (ChatGPT 2023). Yasemin insisted, asking ChatGPT about getting some ideas about how the creative process might evolve in an interdisciplinary student group:
This is a great workload allocation once you know what you will be doing. What about the ideation process? In such an open ended design brief, students are expected to come up with creative ideas. How do you think this idea generation process will evolve? (Araci via ChatGPT 2023).
However again, ChatGPT’s response did not offer a nuanced explanation beyond the conventional concept of workload distribution. This suggests that ChatGPT cannot provide a comprehensive understanding of how individuals with distinct backgrounds could contribute unique perspectives during the design ideation phase.
When requested to combine two design briefs, ChatGPT successfully amalgamated the concepts into a single design brief, once again providing comprehensive details including objectives, deliverables, and key features without any explicit prompting. Following this, ChatGPT was asked to generate relevant personas, “fictional characters used to represent typical users, customers or other stakeholders” (Tomitsch et al. Reference Tomitsch, Wrigley, Borthwick, Ahmadpour, Frawley, Kocaballi, Núñez-Pascheco, Straker, Loke and Straker2018, p. 100) for the target audience and later asked for the creation of “extreme characters,” a method that is used in the early phases of the design process by “using the unique needs of extraordinary people as design inspiration” (Tomitsch et al. Reference Tomitsch, Wrigley, Borthwick, Ahmadpour, Frawley, Kocaballi, Núñez-Pascheco, Straker, Loke and Straker2018, p. 62). ChatGPT excelled in crafting personas and extreme characters, one example is as follows:
Persona 3: Male Name: Samuel Johnson Age: 95 Background: Samuel is an extraordinary individual who defies traditional expectations for his age group. He is a retired World War II veteran and continues to lead an active and adventurous lifestyle. Samuel has always been tech-savvy and enjoys exploring the latest advancements in technology. Despite his advanced age, he remains physically fit and mentally sharp (ChatGPT 2023).
Although ultimately deciding to incorporate these persona examples directly into the design brief, Yasemin opted to leave their utilisation up to the students involved in the project.
4.2.3. Learning outcomes
ChatGPT named the design project brief as “Digital well-being and smart home solutions for seniors” (ChatGPT 2023). The next step was to create learning outcomes for this studio unit. While ChatGPT successfully generated seven relevant learning outcomes, it relied on generic verbs such as “understand,” “apply,” “integrate,” or “prototype.” These verbs, though commonly used, lack specificity and may benefit from more precise and action-oriented language. Furthermore, the initial suggestions for the learning outcomes were lengthy. In response, Yasemin requested ChatGPT to simplify them into concise sentences. For example, the initial suggestion for one of the learning outcomes was:
Understanding the unique needs and challenges of seniors: Students will gain knowledge and empathy towards the physical, cognitive, and social needs of seniors as they age. They will develop an understanding of the factors that impact well-being in this specific demographic (ChatGPT 2023).
The simplified version by ChatGPT was like this: “Understand the unique needs of seniors and their well-being” (ChatGPT 2023). These are all the learning outcomes ChatGPT came up with:
-
• Understand the unique needs of seniors and their well-being.
-
• Apply human-centred design principles to develop tailored solutions.
-
• Integrate digital technology with physical environments for seniors.
-
• Consider ethical considerations and user privacy in designing for seniors.
-
• Create prototypes and conduct usability testing for user-centered design.
-
• Develop effective communication and presentation skills for diverse audiences.
-
• Collaborative interdisciplinary teamwork (ChatGPT 2023).
4.2.4. Assignments
The next step was to create assignments for this studio unit depending on the principles of “constructive alignment” which is described by Biggs (Reference Biggs1996, Reference Biggs2001) and Biggs & Tang (Reference Biggs and Tang2015), as a framework for designing effective teaching and assessment practices. It involves three key components:
-
• Desired/Intended learning outcomes: The first step is to clearly define what students should be able to accomplish or understand by the end of a learning experience.
-
• Teaching and learning methods: Once the desired outcomes are established, educators should determine the most effective strategies and activities to engage students and guide their behaviour towards achieving those outcomes.
-
• Assessment tasks: Assessment plays a crucial role in constructive alignment. It aims to evaluate the quality of learning and to provide students with a clear understanding of what is expected of them. Assessments should be designed to measure how well students have achieved the intended outcomes and should align with the criteria set for those outcomes.
Overall, constructive alignment emphasises the importance of aligning learning outcomes, teaching and learning methods, and assessment practices to actively engage students in meaningful activities that directly contribute to their intended learning outcomes. Therefore, Yasemin asked ChatGPT about its familiarity with the theory of Constructive Alignment in curriculum design, it provided an answer that convinced her of its understanding and knowledge of the theory:
Yes, I am familiar with the theory of Constructive Alignment in curriculum design developed by John Biggs. Constructive Alignment is an instructional design approach that emphasizes the alignment of intended learning outcomes, teaching and learning activities, and assessment methods. It aims to ensure that the teaching and learning experiences are aligned with the desired learning outcomes, promoting deep understanding and meaningful learning for students. This approach helps create a coherent and purposeful learning experience that supports students in achieving the intended learning outcomes (ChatGPT 2023).
With confidence in ChatGPT’s understanding of constructive alignment, Yasemin proceeded to seek its assistance in designing assessment methods for the previously established learning outcomes. She asked: “What type of assessments can I design for this graduate design studio unit, to create a coherent and purposeful learning experience that supports students in achieving the intended learning outcomes?” (Araci via ChatGPT 2023). ChatGPT responded to this question by initially providing an extensive number of assessment suggestions, an assignment aligned with each learning outcome; in total six different assessment tasks. Even though they were all relevant, they were too many for a semester long unit.
Being aware of the limited time in a design unit for a semester, she intervened by guiding ChatGPT and specifically requested only four assessment tasks along with their corresponding deliverables: a report, an end product/prototype, a presentation, and a reflection. ChatGPT came up with these:
-
• “Research-based Essay on Seniors’ Needs and Well-being: A written report
-
• Design Project for Tailored Solutions: An end product/prototype
-
• Integrating Digital Technology with Physical Environments: A presentation
-
• Ethical Analysis: A written reflection paper” (ChatGPT 2023).
When prompted to align the assessment tasks with a graduate studio context, taking into account higher quality expectations compared to undergraduate-level assignments, ChatGPT provided a response.
In a graduate studio, students are typically expected to demonstrate a higher level of critical thinking, independent research, and advanced problem-solving skills. To align with the graduate level, the assessment tasks can be modified to incorporate more advanced components, such as deeper analysis, more sophisticated design solutions, and a higher level of reflection and evaluation. Additionally, the expectations for research and theoretical grounding may be higher in a graduate studio (ChatGPT 2023).
Satisfied with ChatGPT’s understanding about the level of requirements in a graduate-level unit, Yasemin asked to expand the deliverables of each assignment. Finally, she asked ChatGPT to wrap up everything by entering this prompt:
Ok, let’s wrap up all of these. Please consider all our correspondence from the beginning and finalise the unit outline for me, under these headings: Unit description, Learning outcomes, Design brief, Weekly Schedule, and Assignments. No need to make changes at this process. Just stick to what we have discussed so far and agreed. If you are not sure about any point, please ask me (Araci via ChatGPT 2023).
During the conclusion phase of their chat with ChatGPT, an inconsistency was observed between their earlier deliberations and ChatGPT’s output. Despite having established the incorporation of four assignments within the unit, ChatGPT presented only three of them during this wrap-up process. It is plausible that she could have expressly communicated to ChatGPT her intention to retain all four assignments. As a result, Yasemin found it necessary to manually transfer the relevant sections from ChatGPT’s initial drafts. In addition, she requested ChatGPT to rephrase the unit description in a generic manner, independent from the specific targets in the design brief, allowing for potential changes on a yearly basis, and this is how it responded:
This graduate design studio focuses on developing innovative digital solutions that enhance well-being and improve user experiences. Students will explore human-centered design principles, integrate technology with physical environments, consider ethical considerations, create prototypes, conduct usability testing, collaborate in interdisciplinary teams, and refine communication skills. Through practical projects and research-based activities, students will develop comprehensive design solutions that address user needs and promote positive user experiences (ChatGPT).
Additionally, Yasemin inquired about an appropriate name for this design studio, and she ultimately adopted the name suggested by ChatGPT: “Innovation Studio: Designing for Well-being and User Experiences” (2023).
4.2.5. Weekly content
Finally, it was time to create the weekly course content, so Yasemin asked ChatGPT’s assistance in crafting a 12-week course content plan. However, ChatGPT created overarching aims of the unit rather than specific weekly content, for example: “Collaborative teamwork and effective communication in design projects” (ChatGPT 2023). In response, Yasemin decided to create the weekly content herself by incorporating a few suggestions from ChatGPT, because by this stage she was tired of feeding prompts to ChatGPT to get quality responses about a weekly content.
As a final reflection on this process, the entire assignment/course development process took 5 days to create spending a few hours each day. Yasemin found that using ChatGPT as part of a course design process to be very time efficient. ChatGPT version 4 also mostly remembers what has been previously discussed, making it easier to return to a conversation after a break. This also “allows it to maintain context and carry on more natural and coherent conversations with users over time” (Farrokhnia et al. Reference Farrokhnia, Banihashem, Noroozi and Wals2023, p. 3). However, as seen in the final wrap-up process, when asked to bring everything together that has been discussed, it does not remember all the things discussed. Yasemin also found ChatGPT to be helpful for brainstorming and editing, but it is not a reliable “secretary replacement,” often overlooking important, previously discussed, details.
In their study conducted in 2023, Holmes et al. assess the application of AI in educational practices and propose that while AI will not substitute educators, however, it will lead to the evolution and transformation of their roles. The authors argue that teachers will utilise their time more efficiently and effectively by focusing on areas where their expertise can be better utilised, harnessed, and enhanced. AI will make educators’ job “easier and more effective” (Holmes, Bialik & Fadel Reference Holmes, Bialik and Fadel2023, p. 632).
Thus, as a course designer (and occurring with Yaron’s experience above), Yasemin does not think ChatGPT is likely to render educators or education designers obsolete. ChatGPT is a convenient tool for educators requiring unit outline generation, however, without the input of an experienced educator or education designer, it does not currently appear to have the capacity to create a learning unit on its own. Indeed, while AI can produce coherent and relatively accurate content and is efficient in retrieving data compared to humans (Zhai Reference Zhai2022), large language models “cannot replace the creativity, critical thinking, and problem-solving skills that are developed through human instruction” (Kasneci et al. Reference Kasneci, Sessler, Küchemann, Bannert, Dementieva, Fischer, Gasser, Groh, Günnemann, Hüllermeier, Krusche, Kutyniok, Michaeli, Nerdel, Pfeffer, Poquet, Sailer, Schmidt, Seidel, Stadler, Weller, Kuhn and Kasneci2023, p. 6). Yasemin’s observations align with Kasneci et al.’s (Reference Kasneci, Sessler, Küchemann, Bannert, Dementieva, Fischer, Gasser, Groh, Günnemann, Hüllermeier, Krusche, Kutyniok, Michaeli, Nerdel, Pfeffer, Poquet, Sailer, Schmidt, Seidel, Stadler, Weller, Kuhn and Kasneci2023) and others (Mhlanga Reference Mhlanga2023) suggestive use of language models as complementary tools within education, to improve teaching methods and foster critical thinking in students, rather than as a replacement for human instructors.
5. Discussion
In our investigation, we reflected that ChatGPT appears able to rapidly construct a comprehensive unit structure template, offering an ordered foundation upon which educators can build, based on efficiently conducting initial research and offering broadly relevant examples. Educators may then choose to personalise the content according to their preferences, or continue to input additional prompts to enhance the relevance and specificity of ChatGPT’s outputs. ChatGPT appears to excel at delivering acceptable results when faced with existing design briefs, merging ideas, or creating multiple brief options – for example if changes are required periodically. We also found ChatGPT useful for brainstorming and editing. ChatGPT is also relatively easy to begin to use, quick to respond, and potentially improves time management by “decreasing teaching workload” and “making key processes and tasks more efficient” (Farrokhnia et al. Reference Farrokhnia, Banihashem, Noroozi and Wals2023). We tentatively concur that it “[…] can help both teachers and students to improve teaching and learning experiences” (Eke Reference Eke2023, p. 2).
However, it does not replace the role of a human administrator, who can note and refer to previously discussed details for continuity and decision-making purposes. As Trust, Whalen & Mouza (Reference Trust, Whalen and Mouza2023) argue, the potential of ChatGPT is to provide support with teaching such as writing course syllabuses, lesson plans, course objectives, and learning activities.
Our experience further suggests that satisfactory outcomes in creating design course materials using ChatGPT requires considerable effort and calculated prompting by professional, design-educated, and experienced human course developers. Indeed, both authors had to actively guide (and sometimes assertively challenge with increasingly detailed prompts) ChatGPT, so as to achieve acceptable and accurate outcomes. Thus, Wiley’s (Reference Wiley2023) emphasis that instructional designers play a pivotal role as the primary prompt engineers in educational systems remains valid, with the quality of educational content heavily depending on incorporating their expertise into system prompts.
Both authors agreed that ChatGPT’s outputs were clearly structured and formatted. Albeit, we also noted how these outcomes often manifested as non-committal and generic, functioning essentially as boilerplate templates. This illustrated the powerful language model strengths of the tool, while highlighting its limitations of specialist knowledge and, perhaps most acutely, of context and diversity. For example, both authors discussed their struggle with and recognition of apparent flaws in ChatGPT’s ability to differentiate between different design disciplines and especially the need for contextual considerations when developing design course materials. This supports what Farrokhnia et al. (Reference Farrokhnia, Banihashem, Noroozi and Wals2023) argue is a weakness of ChatGPT: its apparent difficulty in evaluating the quality of responses and lack of higher-order thinking. Indeed, despite AI having been studied in relation to design since the 1980s (Hay, Cash & McKilligan Reference Hay, Cash and McKilligan2020), ChatGPT does not appear to add any education-focused design-specific value over and above what it contributes to any other educational discourse or professional practice.
Similarly, attempts to engage with specific aspects of design, such as notions of creativity, produced unsatisfactory outcomes for us, regardless of the amount of human prompting. This was found to be especially problematic for multidisciplinary design purposes and, while perhaps partially attributable to an incomplete dataset to draw from within some design discourses, is also suggestive of current limitations of the technology itself.
5.1. Limitations
Our engagement with ChatGPT draws from reflections based on an extensive amount of real-world design industry and teaching experience, and our interrogations were designed to draw on that. However, while our approach was intentionally chosen, we acknowledge its limitations. This has been a subjective exercise and, even between the two authors of this article, our relative experiences with ChatGPT and design backgrounds will have influenced our aims, choices and decisions. We have discussed how the authors are designers not computer scientists, and so our experiences, motivations and choices are from a position of relative technical limitation as regards the engineering development of AI models. Thus, our methodological approach, engagement, responses and outcomes are reflective of this. As we acknowledged earlier, each author carrying out independent and differently focussed engagements with ChatGPT challenges a direct comparison and does not provide a linear presentation of outcomes. However, having identified that ChatGPT is not a reliable subject with repeatable or reproducible responses, and being mindful of the interdisciplinary objectives of the authors, we opted for the semi-structured exploratory conversational approach. This is a limitation, but one that was chosen while balancing complex possibilities. Moreover, considering the nascency of the topic, such experiential approaches are informative for future research.
6. Future research
Wider and more in-depth research approaches towards course development using generative AI tools like ChatGPT would be of considerable value to the design and indeed wider educational community. The format of such investigations will require much thought and careful design, and it may be challenging to develop an empirical approach with the flexibility for engaging in the required interactive variables of educator-to-machine interaction. Such studies would no doubt provide valuable outcomes and be welcomed by design educators.
Even broader and more systematic subjective inquiries would add value to the still emerging and ongoing challenges of using AI in design course development. Although, the multiple challenges of running live trials of AI-enhanced design courses versus solely human-generated ones are vast.
It could also be of value to compare the needs of design courses in education with those of other design and educational requirements. This might aid in creating better approaches for design educators who use ChatGPT in their teaching, whether by choice or necessity.
7. Conclusion
We found ChatGPT to be an impressive tool for the purpose it was designed – a language model – and any potential limitations discussed are intended to reflect only our responses to this specific research. Both authors independently concurred that Chat GPT was a useful tool for brainstorming, as well as structuring and editing language, potentially saving time and providing outputs that may function as templates for the development of masters-level design course materials.
ChatGPT’s ability to intelligently amalgamate and organise design learning content, especially when carefully prompted or given direction by a human collaborator, was impactful and could provide time-saving clarity in the organising and structuring of such textual content. For example, Yasemin discovered that it was capable of generating persona-based tasks, as well as categories and headings referencing learning outcomes. However, both authors found that such categories and headings were often generic (regardless of the degree of prompting) and required human editing and restructuring before being useful for our course development purposes.
There are also broader questions as to whether such artificially generated homogeneous educational design outcomes would be beneficial educationally, for a syllabus aimed at designers. Moreover, whether such formulaically structured outputs ought to be used even as templates by design educators, with the potential risk of limiting the creative and analytical challenges and scope of future design professionals and thinkers, is debateable. These are challenges of desirability, ethics and responsibility that educational design academics may need to consider, in addition to questions of efficacy and capability, when designing future research into AI and the generation of design education syllabuses. With the acknowledged subjectivity of this research, alongside the complexity of the topic itself, further research may help to inform those educator considerations.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dsj.2023.28.
Open access
