1. Introduction
Artificial intelligence (AI) tools, including large language models (LLMs), are becoming increasingly relevant in engineering design. The ability of LLMs such as ChatGPT to generate, interpret and refine text in a conversational manner, by being trained on vast amounts of data representing a broad range of general knowledge, makes them applicable for engineering design activities (Reference Khanolkar, Vrolijk and OlechowskiKhanolkar et al., 2023). Because such activities are often supported computationally, LLMs are suitable to being explored in engineering contexts where Computer-Aided Design (CAD) is used. CAD supports the creation, modification, analysis and optimisation of design artefacts, and while CAD was originally developed for individual use, or standalone CAD, over the years, however, it has gone through a transformation to collaborative CAD, which enables multiple designers to interact synchronously regardless of their location. This synchronous interaction implies that at least two CAD users collaborate as a team, working jointly to achieve a shared goal; their success depends on both taskwork and teamwork (Reference Bedwell, Wildman, DiazGranados, Salazar, Kramer and SalasBedwell et al., 2012). Taskwork relates to the execution of the CAD task itself, while teamwork concerns how team members interact.
Central to teamwork is communication. Previous efforts to model team performance, such as the Input–Mediator–Outcome (IMO) framework, position communication as a core mediating process that predicts team performance (Reference PassmorePassmore, 2017). Communication allows the exchange of, mainly, task-related information that “forms and reforms” team’s attitudes, behaviours and cognitions (Reference PassmorePassmore, 2017). It is often studied in design research because it reflects what team members articulate and how they think during the task (Reference GoldschmidtGoldschmidt, 2016), by analysing speech occurrences or verbal engagement (Reference Martinec, Horvat, Škec and ŠtorgaMartinec et al., 2018), and the content of the verbal communication (Reference Martinec, Škec, Šklebar and ŠtorgaMartinec et al., 2019a). These analyses are particularly relevant for synchronous collaborative CAD activities, where designers primarily rely on verbal communication to manage the team, the CAD model and the design task. Therefore, the introduction of LLM tools such as ChatGPT creates a new factor that may influence both taskwork and teamwork. While existing studies have begun to examine the influence of ChatGPT on taskwork in synchronous collaborative CAD activities (Reference Šklebar, Martinec, Škec and ŠtorgaŠklebar et al., 2025b), no research has investigated its influence on teamwork, including verbal communication. The use of such tools may reshape verbal communication by changing how team members exchange information and coordinate their actions. Yet empirical evidence on how verbal engagement and the content of verbal communication differ when teams with ChatGPT support compared to teams without that support is still missing. Therefore, the present study examines whether using LLM tool such as ChatGPT affects verbal engagement and the content of verbal communication between team members involved in synchronous collaborative CAD activity while solving an embodiment design task. The research focuses on two research questions:
-
• Does the usage of LLM tool such as ChatGPT affect the overall verbal engagement during synchronous collaborative CAD activity?
-
• Does the usage of LLM tool such as ChatGPT affect the content of verbal communication during synchronous collaborative CAD activity?
The paper is structured as follows. Section 2 provides an overview of research including work on communication in collaborative CAD activities, approaches for analysing verbal communication in design, and findings on how AI support influences communication in teams. Section 3 presents the research methodology, including the experimental study and the verbal protocol analysis, whereas Section 4 reports the results on overall verbal engagement and the content of verbal communication, respectively. Section 5 discusses these findings in relation to synchronous collaborative CAD activities and human–AI collaboration. Section 6 concludes the paper by outlining the main implications for the influence of LLMs on synchronous collaborative CAD activities and the future research.
2. Related work
As communication is central to design teams, this section reviews research on communication in collaborative CAD activities (Section 2.1), approaches for analysing communication in design teams (Section 2.3), and AI-supported team communication (Section 2.3).
2.1. Communication analysis in collaborative CAD activities
In examining teamwork in collaborative CAD activities, communication has been the main focus (Reference Šklebar, Martinec, Škec and ŠtorgaŠklebar et al., 2025a). Studies analysing communication frequency and its temporal pattern have shown that high-performing teams often display verbal communication spikes at the beginning and end of the activity, whereas low-performing teams communicate more uniformly throughout the task execution (Reference Stone, Salmon, Eves, Killian, Wright, Oldroyd, Gorrell and RicheyStone et al., 2017). Other studies analysing verbal engagement found that pairs working in shared working mode (only one user has the control in CAD) communicate almost twice as much as those working in parallel mode (all of the users have control) (Reference Phadnis, Arshad, Wallace and OlechowskiPhadnis et al., 2021). Additional work revealed that communication in pairs introduces additional overhead compared to individual CAD users (Reference Phadnis, Arshad, Wallace and OlechowskiPhadnis et al., 2021). Expertise differences have also been investigated, with findings showing that novices communicate less than experts (Reference Vella, Olechowski and PhadnisVella et al., 2022), while expert–novice pairs exhibit higher verbal engagement than only novice pairs, because novices seeking advice and experts providing guidance (Reference Phadnis, Leonardo, Wallace and OlechowskiPhadnis et al., 2019). Furthermore, the other study shows that teams using standalone CAD report greater communication difficulties than those using collaborative CAD, assessed through self-reporting (Reference Greenholts, Verner and PolishukGreenholts et al., 2024). However, despite these contributions, prior research has not examined verbal engagement nor the content of verbal communication in the context of examining the influence of support of LLM tools such as ChatGPT on synchronous collaborative CAD activities.
2.2. Communication analysis in engineering design
Analysing verbal communication in design teams is often used as a proxy of the overall contribution of designers working in teams (DiMicco, 2004), their thinking processes (Reference GoldschmidtGoldschmidt, 2016), or their cognitive abilities (Reference Dorst and CrossDorst & Cross, 2001). It is suitable because design teams are concurrent and conversational, meaning that team members articulate their thinking acts during task execution (Reference Hay, Duffy, McTeague, Pidgeon, Vuletic and GrealyHay et al., 2017). To gain deeper insight into what is expressed, design research commonly uses verbal protocol analysis, where utterances are segmented and coded. Various coding schemes have been developed or utilised for this purpose. Reference Stempfle and Badke-SchaubStempfle and Badke-Schaub (2002) introduced a framework for capturing designers’ thinking through content and process categories, further refined into subcategories (design steps) of goal clarification, solution generation, analysis, evaluation, decision and control. Other studies used schemes based on the Function–Behaviour–Structure (FBS) model, originally developed as a theoretical model of design process, which classifies verbal communication according to whether it refers to the function, behaviour or structure of the design artefact (Reference Kan, Gero and TangKan et al., 2010). Since design is not a linear process, but one in which the design problem and possible solutions develop in parallel through an iteration of analysis, synthesis and evaluation (ASE) (Reference VisserVisser, 2009), many studies adopt the co-evolution perspective. According to this view, new problem entities arise through the exploration of possible solutions, and vice versa, leading to a conceptual separation between the problem and solution space (Reference Dorst and CrossDorst & Cross, 2001). Building on this, Martinec et al. (Reference Martinec, Škec, Horvat and ŠtorgaMartinec et al., 2019b) introduced a scheme that distinguishes ASE in both problem and solution spaces. The scheme was developed to combine, and thus allow mapping to, both the design steps introduced by Reference Stempfle and Badke-SchaubStempfle and Badke-Schaub (2002) and the FBS coding scheme describing the design artefact, while reflecting the elementary design operations of ASE, and the co-evolution of the problem and solution spaces. In addition to that, the scheme is domain-independent and allows both abstract and fine-grained segmentation of verbal engagement content.
As this study involves an embodiment design task within a synchronous collaborative CAD activity, the ASE coding scheme in problem and solution spaces was selected.
2.3. Influence of AI support on communication in teams
Research examining how AI support influences communication between team members or verbal engagement, is still limited, yet existing studies show several valuable findings. A common finding is that AI support influences team processes such as coordination and communication (Reference Schmutz, Outland, Kerstan, Georganta and UlfertSchmutz et al., 2024). Teams supported by AI tend to exchange less information, plan their activity to a lesser extent, and engage in limited nonverbal coordination, which in turn affects how team members communicate (Reference Zercher and JussupowZercher & Jussupow, 2023). This becomes visible through lower levels of both pushing (sharing information) and pulling (requesting when needed) information. Studies show that teams supported by AI show reduced levels of both, which are directly linked to lower team performance (Reference Demir, McNeese and CookeDemir et al., 2017). Findings also indicate that team performance is higher when team members communicate more, highlighting the importance of communication engagement in AI-supported teams (Reference Bogg, Birrell, Bromfield and ParkesBogg et al., 2021). Furthermore, more intertwined interaction between team members and AI tool contributes to underperformance compared to teams without AI support (Reference Demir, Likens, Cooke, Amazeen and McNeeseDemir et al., 2019). Recent work specifically addressing ChatGPT support shows that teams tend to shift their interaction toward the tool itself, become cognitively rigid by relying too heavily on the tool, which decreases willingness to contribute to the taskwork, thus lowering engagement and reducing direct communication between team members (Reference Cheng and ZhangCheng & Zhang, 2024). In contrast, other work shows that teams without AI support communicate more, reporting that verbal engagement is doubled compared to those supported by AI (Reference O’Neill, McNeese, Barron and SchelbleO’Neill et al., 2022). These findings together suggest that the quality of verbal communication may be more meaningful indicator than any quantitative one (Reference Zhang, Duan, Flathmann, McNeese, Freeman and WilliamsZhang et al., 2023).
Although existing research provides initial indications that AI support influences how team members communicate, the existing literature still appears to leave a research gap. No existing study examines how AI support, including LLM tools such as ChatGPT, influences verbal engagement or verbal communication content in engineering design task. In particular, work on synchronous collaborative CAD activities has yet to investigate the influence on communication in AI-supported teams.
3. Research methodology
This study followed a verbal protocol analysis with three stages: defining the experimental sample, task and setup (Sections 3.1. and 3.2.), coding verbal communication using a coding scheme (Section 3.3.), and analysing the data through descriptive statistics, assumption testing and inferential comparisons between conditions (Section 3.4.).
3.1. Experimental sample and task
The study involved 44 mechanical engineering students (11 females and 33 males) from the University of Zagreb, spanning undergraduate and graduate levels. Participation required completion of both basic (first year) and advanced (third year) CAD courses at the university, ensuring the students possessed both CAD knowledge, and the knowledge about design-for-manufacturability principles. The students were randomly divided into 22 pairs, and each pair was assigned to one of two experimental conditions: with or without ChatGPT support. The present analysis was part of a larger experimental study consisting of three consecutive collaborative CAD tasks, designed to require both CAD knowledge and domain-specific knowledge. This paper examined only the third task, as it required more domain-specific knowledge than the previous tasks, where the usage of ChatGPT was deemed more suitable for collaborative CAD activities, as indicated by previous research (Reference Šklebar, Martinec, Škec and ŠtorgaŠklebar et al., 2025b). In the task, participants redesigned a CAD model of a crankshaft based on design-for-manufacturability principles related to forging technology. Participants in the experimental condition were also required to consult ChatGPT to make every decision, the same as they would consult each other. They needed to consider the shape and dimensions of the part as it would be forged before machining, as shown in Figure 1 (left). Each pair had 20 minutes to complete the task, although they were not required to use the full time. Participants began with a pre-modelled crankshaft with a visible feature tree, which allowed them to modify the CAD model during the task execution.
3.2. Experimental setup
The experimental setup for both conditions consisted of one room with two working places facing each other, as shown in Figure 1 (right). Each working place was equipped with an office chair and table, a high-performance computer, two monitor screens, a keyboard and a mouse. In both conditions, participants utilised Onshape, a cloud-based CAD software accessed via web browser. Onshape enables synchronous collaboration on a single CAD model, allowing participants to simultaneously interact with the same CAD model, follow each other’s work and share views. Each monitor screen had a different layout. The left displayed Onshape document with a CAD model of a crankshaft that needed modification.
Embodiment task of the forged crankshaft (left) and the experimental setup (right)
The right screen presented another Onshape document with detailed task explanations (imported as a PDF) and the CAD model of the crankshaft in case participants needed it while modifying the model on the left screen. Both team members with the ChatGPT support had an additional web browser window on the right monitor screen with the ChatGPT application (Figure 2). To facilitate synchronous collaboration during the execution of CAD tasks, the content of the right monitor of one team member was shared on the right monitor of the other team member’s screen, using screen mirroring. This enabled team members to follow each other’s work in real-time, including synchronous use and viewing of ChatGPT.
Screen layout in the ChatGPT-supported experimental condition
Participants utilising ChatGPT received support from the ChatGPT-4 model, from the customised GPT Onshape Usage and Collaboration, developed specifically for the experiment. To prevent the information from being reused from other tasks, the ChatGPT’s training option was disabled, and its memory was cleared after each task. Additionally, all screen content was recorded for the entire duration of the experiment using OBS Studio. Video and audio recordings of the tasks were captured using a conference camera, and small video cameras that were placed on each monitor screen.
3.3. Data collection and coding
The analysis started with importing all recordings into ELAN software, which enabled time-aligned segmentation and coding of verbal engagement content. The coding scheme was ASE in problem and solution spaces (Reference Martinec, Škec, Horvat and ŠtorgaMartinec et al., 2019b), consisting of eight codes. Three codes captured communication in the Problem space: Problem analysis or examining constraints or manufacturability issues (for example, “Since the material is compressed, it needs to deform uniformly.”), Problem synthesis or formulating or reformulating the problem (“So the issue is that it can’t be removed from the mould”), and Problem evaluation or assessment the importance or clarity of the identified issue (“That’s the main problem”). Three codes described communication in the Solution space: Solution analysis or interpreting geometric constraints or model parameters (“Where is the neutral plane?”), Solution synthesis or proposing or generating CAD modifications (“Try applying a draft on this surface”), and Solution evaluation or judging whether a modification achieved the intended result (“This works better than before”). Two additional codes were used: Process, for communication related to planning, coordination or interaction with CAD or ChatGPT, and Other, for non-task-related communication. Segmentation followed the “one segment, one code” principle, assigning one code to each verbal engagement unit. A verbal engagement unit was defined as the smallest segment of speech that could be assigned a single code from the coding scheme. Although inter-rater reliability is generally recommended in verbal protocol studies, in this initial analysis the coding was conducted by a single researcher, as previous coding of the ASE scheme in design contexts have reported high agreement levels (Reference Martinec, Škec, Horvat and ŠtorgaMartinec et al., 2019b). The resulting ELAN files contained all time-stamped codes for each team and were later used for quantitative analysis.
3.4. Data processing and analysis
The coded ELAN files were exported as time-aligned text files and processed using Python. The analysis was conducted at the team level, and for each team, the duration of every coded segment was calculated and expressed as a percentage of the total task duration. This allowed quantification of (a) the total share of verbal engagement within the task duration, (b) the distribution of verbal communication content across the categories of Problem, Solution, Process, Other, and (c) the Analysis, Synthesis and Evaluation, and (d) the distribution of verbal communication content across the subcategories of Problem Analysis, Problem Synthesis, Problem Evaluation, Solution Analysis, Solution Synthesis and Solution Evaluation. All coded data were aggregated at the team level and statistically analysed to compare the two experimental conditions. Descriptive statistics were calculated, followed by assumption testing using the Shapiro–Wilk test for normality and the Levene test for homogeneity of variances. Depending on the results, either parametric t-tests or non-parametric version of Mann–Whitney U tests were applied. To correct for multiple comparisons, Bonferroni corrections were applied within each family of related tests (both the ASE and Problem/Solution and subcategories), and the adjusted p-values are reported in the results. Effect sizes were reported as Cohen’s d for parametric tests and r for non-parametric tests. All statistical analyses were conducted with a significance level of p < 0.05 (indicated by * in tables and visualisations).
4. Results
On average, teams without ChatGPT support exhibited a higher share of verbal engagement (M=52.29%, SD=11.54) compared to ChatGPT-supported teams (M=41.18%, SD=10.33). The assumption of normality was met for both groups (p=0.700; p=0.810), and variances were homogeneous (p=0.757). Furthermore, t-test showed that this difference was statistically significant (p=0.0274), with a large effect size (d=–1.015). These results are shown in Figure 3 and Table 1.
Descriptive statistics and test results for overall verbal engagement
Overall verbal engagement for teams with and without ChatGPT support
4.1. Distribution across verbal engagement content
On average, teams without ChatGPT support allocated most of their verbal engagement to the Solution (M=73.56%, SD=10.39) and the Problem (M=16.21%, SD=10.61), while smaller portions were devoted to Process (M=6.28%, SD = 3.25) and Other (M=3.94%, SD=2.79). ChatGPT-supported teams showed a similar dominance of Solution (M=76.59%, SD=7.20), but less engagement in the Problem (M=2.57%, SD=4.58) and a greater proportion in the Process (M=17.78%, SD=5.39). For the Problem, normality was violated in the ChatGPT condition (p<0.001), and variances were unequal (p=0.004). A Mann–Whitney test confirmed that the difference was significant with the large effect size (p=0.003, r=0.721). For Process, both normality and homogeneity (p=0.813; p=0.985) assumptions were met (p = 0.132), and the t-test revealed a significant difference with the large effect size (p=0.001, d=2.582). Conversely, differences in the Solution (p=1.000, r=0.119) and Other (p=1.000, d=–0.335) categories were not significant (Figure 4, Table 2).
Descriptive statistics and test results for problem, solution, process and other
Furthermore, teams without ChatGPT support devoted on average nearly half of their verbal engagement to Analysis (M=45.00%, SD = 11.44), followed by Synthesis (M=32.89%, SD=10.28) and Evaluation (M=11.88%, SD=7.07). ChatGPT-supported teams, on the other hand, dedicated more communication to Synthesis (M=47.97%, SD=7.25), less to Analysis (M=20.24%, SD=4.80), and similar amounts to Evaluation (M=10.95%, SD = 4.11). Normality and homogeneity assumptions were satisfied in all cases, except for Analysis, where homogeneity was violated (p=0.044). Consequently, a Mann–Whitney U test revealed a significant difference in Analysis (p=0.0002, r=0.68), and the t-test showed a significant difference for Synthesis (p=0.003, d=1.695), both with the large effect size. Conversely, no significant difference was found for Evaluation, with the small effect size (p=1.000, d=–0.161).
Distribution of verbal communication content across problem and solution (left) and analysis, synthesis and evaluation (right) for team with and without ChatGPT support
Descriptive statistics and test results for analysis, synthesis and evaluation
Descriptive statistics and test results for problem and solution subcategories
Table 4 Long description
The table presents descriptive statistics and test results for problem and solution subcategories with and without ChatGPT support. It has 8 rows and 7 columns. The columns are labeled: Subcategory, Experimental Condition, Mean, SD, Normality (p), Homogeneity (p), p-value, and Effect size. The rows are grouped by subcategories: Problem Analysis, Problem Synthesis, Problem Evaluation, Solution Analysis, Solution Synthesis, and Solution Evaluation. Each subcategory has two experimental conditions: With ChatGPT Support and Without ChatGPT Support. Row 1: Problem Analysis, With ChatGPT Support, Mean: 2.22, SD: 3.89, Normality (p): 0.000, Homogeneity (p): 0.002, p-value: 0.009*, Effect size: r=0.679. Row 2: Problem Analysis, Without ChatGPT Support, Mean: 14.19, SD: 10.23, Normality (p): 0.089. Row 3: Problem Synthesis, With ChatGPT Support, Mean: 0.18, SD: 0.52, Normality (p): 0.000, Homogeneity (p): 0.004, p-value: 0.189, Effect size: r=0.427. Row 4: Problem Synthesis, Without ChatGPT Support, Mean: 1.72, SD: 1.78, Normality (p): 0.120. Row 5: Problem Evaluation, With ChatGPT Support, Mean: 0.16, SD: 0.43, Normality (p): 0.000, Homogeneity (p): 0.485, p-value: 1.000, Effect size: r=0.154. Row 6: Problem Evaluation, Without ChatGPT Support, Mean: 0.30, SD: 0.48, Normality (p): 0.000. Row 7: Solution Analysis, With ChatGPT Support, Mean: 18.01, SD: 5.30, Normality (p): 0.874, Homogeneity (p): 0.578, p-value: 0.001*, Effect size: d=2.036. Row 8: Solution Analysis, Without ChatGPT Support, Mean: 30.82, SD: 7.14, Normality (p): 0.808. Row 9: Solution Synthesis, With ChatGPT Support, Mean: 47.79, SD: 7.44, Normality (p): 0.303, Homogeneity (p): 0.500, p-value: 0.001*, Effect size: d=1.943. Row 10: Solution Synthesis, Without ChatGPT Support, Mean: 31.17, SD: 9.55, Normality (p): 0.624. Row 11: Solution Evaluation, With ChatGPT Support, Mean: 10.79, SD: 4.13, Normality (p): 0.567, Homogeneity (p): 0.361, p-value: 1.000, Effect size: r=0.021. Row 12: Solution Evaluation, Without ChatGPT Support, Mean: 11.58, SD: 6.92, Normality (p): 0.044.
Lastly, the detailed coding of six subcategories within Problem and Solution showed several significant differences (Figure 5, Table 4). Teams without ChatGPT support exhibited higher shares in Problem Analysis (M=14.19%, SD=10.23) and Problem Synthesis (M=1.72%, SD=1.78) compared to ChatGPT-supported teams (M=2.22%, SD=3.89; M=0.18%, SD = 0.52). For these two subcategories, assumptions of normality and homogeneity of variances were violated (p=<0.001–0.120; p=0.002–0.004), and Mann–Whitney U tests showed significant difference for Problem Analysis with medium-to-large effect size (p=0.009, r=0.427), but did not show significance for Problem Synthesis with large effect size (p=0.189, r=0.679). Furthermore, ChatGPT-supported teams exhibited lower shares of Solution Analysis (M=18.01%, SD=5.30) but higher shares of Solution Synthesis (M=47.79%, SD=7.44) compared to teams without ChatGPT (M=30.82%, SD=7.14; M=31.17%, SD=9.55). For these subcategories, assumptions were met (Shapiro–Wilk p = 0.303–0.874; Levene p = 0.500–0.578), and t-tests confirmed significant differences (p=0.001; p=0.001) with large effect sizes (d=–2.036; d=1.943). No significant differences were observed for Problem Evaluation (p =1.000, r = 0.154) or Solution Evaluation (p=1.000, r=0.021), with the small effect size for both subcategories.
Distribution of verbal communication content across subcategories
5. Discussion
This study set out to explore whether the support from an LLM tool such as ChatGPT alters how much teams communicate and what they communicate about by affecting an overall share of verbal engagement, and the verbal communication content. ChatGPT-supported teams engaged in significantly less verbal communication, which may mirror the findings that AI-supported teams tend to exchange less information (Reference Zercher and JussupowZercher & Jussupow, 2023). This reduced engagement may also be aligned with studies suggesting that AI tools can shift interaction away from communication between team members (Reference Cheng and ZhangCheng & Zhang, 2024) and that teams without AI support are generally communicate more (Reference O’Neill, McNeese, Barron and SchelbleO’Neill et al., 2022). In the present context, the reduction in verbal engagement suggests that part of the interaction may have been displaced from communication between team members to interaction of team members with the AI tool. That may be evident in more verbal engagement about the process, such as coordinating how and when to use the tool, which aligns with findings that interaction, here with a conversational AI tool, may introduce overhead (Reference Phadnis, Arshad, Wallace and OlechowskiPhadnis et al., 2021). Also, team members in ChatGPT-supported teams may have relied more on the tool to obtain information, which reduced the amount of verbal communication with their teammate. As a result, they may be oriented toward reactive execution of the task, instead of proactively exploring for possible solutions and framing the problem in parallel. In addition, ChatGPT-supported teams devoted significantly less time speaking out-loud about the problem. At the same time, both types of teams devoted a similar share of verbal communication to the solution space, indicating that ChatGPT did not eliminate solution-focused communication but rather changed how teams arrived at it. These findings can also be interpreted through the co-evolution perspective of design, where progress typically emerges through iteration between the problem and solution spaces. Teams without ChatGPT support showed this expected alternation, spending more time analysing the problem and interpreting solution constraints (Reference Dorst and CrossDorst & Cross, 2001). In contrast, ChatGPT-supported teams invested less verbal communication in the problem space and reduced their share of solution analysis, while strongly increasing solution synthesis. This may suggest a shortened co-evolution cycle, implying that teams moved directly toward proposing and implementing solution, obtained from ChatGPT. Within the ASE categories, the lower verbal engagement in analysing and higher engagement in synthesising reflect a communication pattern oriented toward quicker solution generation, with less explicit articulation of constraints or verification (Reference Makatura, Foshey, Wang, HähnLein, Ma, Deng, Tjandrasuwita, Spielberg, Owens, Chen, Zhao, Zhu, Norton, Gu, Jacob, Li, Schulz and MatusikMakatura et al., 2023).
A more complete interpretation emerges when these communication engagement patterns are considered alongside findings from our previous analysis of the same task (Reference Šklebar, Martinec, Škec and ŠtorgaŠklebar et al., 2025b), where no statistically significant difference in the quality of resulted CAD models was found. This suggests that, for this specific embodiment design task, differences in communication did not translate into clear differences in outcome. Teams may have compensated for reduced verbal communication through relying on ChatGPT as an external knowledge resource. However, this access to external knowledge did not necessarily result in better solutions. Teams may not have formulated effective prompts, lacked the domain knowledge needed to interpret or validate ChatGPT’s suggestions, or may have been constrained by limited time. This implies that external knowledge alone is insufficient if teams can’t adequately prompt, assess or apply it. These point to questions about how teams utilise external knowledge during synchronous collaborative CAD activities, and addressing them represents a direction for future research. Also, to fully understand the influence of LLM support in synchronous collaborative CAD activities, future work should focus on other teamwork and taskwork variables. Only such integrated analysis can reveal whether ChatGPT supports synchronous collaborative CAD activities. From an industrial perspective where decisions often need to be explained, the observed may matter because teams verbalise less of the rationale behind design changes in CAD. Lastly, these findings should be interpreted in light of several limitations, such as small sample comprised of students, a specific embodiment design task, and that all coding were done by a single researcher.
6. Conclusion
This study examined how support from ChatGPT influences verbal engagement and the content of verbal communication during synchronous collaborative CAD activity while solving an embodiment design task. By analysing verbal engagement and verbal communication content through a verbal protocol analysis, the study shows that ChatGPT support resulted in teams who communicated less overall, devoted less verbal communication to problem- and analysis-related communication, and shifted toward a more action-oriented communication focused more on synthesis of a solution. The increase in process-related communication further indicates that interacting with ChatGPT introduces an additional layer of coordination that shapes how teams work together. ChatGPT is therefore likely to support synchronous collaborative CAD activities when its suggestions provide teams with possible solutions, but may hinder it when teams lack, for example, domain or prompt engineering knowledge, needed to critically interpret and integrate its outcome. Future work should therefore integrate verbal engagement analysis with the other measures of teamwork, as well as taskwork outcomes to determine how such shifts in verbal engagement relate to the quality of the outcome and to better understand when ChatGPT supports, or potentially hinders, synchronous collaborative CAD activities.
Acknowledgement
This work is funded by Ministry of Science, Education and Sports of Republic of Croatia, and Croatian Science Foundation project IP-2022-10-7775: Data-driven Methods and Tools for Design Innovation (DATA-MATION).
Open access
