1. Introduction
Consumers/users seem to be increasingly involved in societal consumer issues (Sikhwal & Childs Reference Sikhwal, Childs, Hankammer, Nielsen, Piller, Schuh and Wang2018) especially when they are integrated into the design process (Bendapudi & Leone Reference Bendapudi and Leone2003). Consumers/users also appear to want individualized products (Koren et al. Reference Koren, Shpitalni, Gu and Hu2015; Sikhwal & Childs Reference Sikhwal, Childs and Chakrabarti2019). To meet this need for individualization, companies are using collaborative design approaches that integrate the user into the design process (Pallot Reference Pallot2011). Beyond user-centered design (ISO 924-210: 2019) or the co-design, Do It Together approach proposes to make designer and non-designer (i.e., end-user) collaborate toward a common, unique and individual goal (Hirscher, Niinimäki & Joyner Armstrong Reference Hirscher, Niinimäki and Joyner Armstrong2018). According to Hirscher et al. (Reference Hirscher, Niinimäki and Joyner Armstrong2018) and Dupont et al. (Reference Dupont, Kasmi, Pearce and Ortt2023), Do It Together is an alternative design strategy to organize the design process toward a more local consumption mode and closer to the end-users who become active consumers through their own idea. According to Koren et al. (Reference Koren, Shpitalni, Gu and Hu2015), this collaboration could result in the creation of openness platforms that users and companies can adapt, designing individual products that fit the user’s need. In this social manufacturing approach, the non-designer becomes a value creator (Hirscher et al. Reference Hirscher, Niinimäki and Joyner Armstrong2018) and should be empowered to perform this function. This means that the non-designer who is an end-user must use and manage design tools, known to designers, without being trained to use them. These tools can be in several forms. For instance, designers are accustomed to using sketching which is an effective method of expressing ideas and a communication medium, essential for the success of the ideation phase (Dorta Reference Dorta, Lee and Choi2004; Goldschmidt Reference Goldschmidt, Chakrabarti and Blessing2014). Sketching is a sub-discipline of drawing and can be defined as a simplified version of this one. In our study, we use the term “sketching” but the underlying cognitive processes studied in this article could also apply to other forms of drawing.
Usually, in the drawing field, the use of a 2D medium is typically used to present three-dimensional (3D) elements. The 2D sketching makes it difficult to design and interact in 3D space and prevents the user from mastering spatial proportions (Dorta, Kinayoglu & Hoffmann Reference Dorta, Kinayoglu and Hoffmann2016). One might therefore think that the use of a 3D medium (such as virtual reality (VR)) would be beneficial compared to the use of a 2D medium (such as paper/pencil). However, in the study of Fleury et al. (Reference Fleury, Dupont, Chaniaud, Tamazart, Poussard, Gorisse and Richir2022), the authors show that users will systematically use paper/pencil sketching before making their VR sketches. Participants had the option of skipping this step of paper/pencil sketching. In addition, Kim et al. (Reference Kim, Oertel, Dobricki, Olsen, Coppi, Cattaneo and Dillenbourg2020) show on a VR software developed to support gardener apprentices in designing gardens that the design quality (in terms of proportion and composition aspects) was improved when it was carried out after the traditional paper/pencil sketching. If users need an intermediate step (i.e., paper/pencil sketching), it is probably related to the fact that they are not comfortable with the 3D tool used or cannot project themselves directly into a 3D environment to communicate. It would therefore be interesting to better understand the transfer of skills between 2D and 3D sketching.
Especially in a Do-It-Together design process, digitalized products must be as user-friendly as possible because the complexity of the interface is not only a limit to the use of the product but also to the creative thinking of the user. For example, this is the case with digital tools that force the user to make design choices too early in the process (Dorta et al. Reference Dorta, Kinayoglu and Hoffmann2016). Sketching allows for designers to quickly define problems, explore ideas, mobilize knowledge (Brun, Masson & Weil Reference Brun, Masson and Weil2016) and develop new solutions using paper, digital media, or both and foster feedback loops (Goldschmidt Reference Goldschmidt, Chakrabarti and Blessing2014) not insignificant in a collaborative project. It is important that sketches are produced in large quantities, quickly, easily discarded or modified and usable for others (Buxton Reference Buxton2007).
The traditional paper and pencil sketching is now being digitized and supports more effective distributed international collaboration (Pallot Reference Pallot2011). The digitalized tool of traditional sketching can take the appearance of VR sketching, which has received great interest in recent years. VR sketching allows collaborative drawing and has other advantages widely referenced in the scientific literature (e.g., Schnabel Reference Schnabel, Wang and Tsai2011; Milovanovic et al. Reference Milovanovic, Moreau, Siret and Miguet2017; Feeman, Wright & Salmon Reference Feeman, Wright and Salmon2018; Yang & Lee Reference Yang and Lee2020). The VR activities create a sense of presence that leads to better communicate the overall intentions of the designer (Schnabel Reference Schnabel, Wang and Tsai2011). Yang & Lee (Reference Yang and Lee2020) have shown that VR sketching boosts the creativity of designers in the ideation phase, because these tools allow the extension of the design solution space, improve the transformation of ideas and encourage a holistic approach to design for concept generation. VR sketching also allows sketches to be viewed from other perspectives because users can move around the three-dimensional sketch (Milovanovic et al. Reference Milovanovic, Moreau, Siret and Miguet2017). VR sketching seems superior to traditional sketching for creative sketching because it leads to a satisfying level of efficiency, effectiveness, ease of use and enjoyment (Van Goethem et al. Reference Van Goethem, Watts, Dethoor, Van Boxem, van Zegveld, Verlinden, Verwulgen, Ahram and Falcão2020). Moreover, with the advent of the Do-It-Together, it is crucial to have easy-to-use communication tools between designers and non-designers in order to exploit their ideas. According to Pallot et al. (Reference Pallot, Fleury, Poussard and Richir2023), the main drawback to the Do-It-Together process is the lack of appropriate customer skills to adequately contribute to design and manufacturing activities. Indeed, the ability to create visual images using freehand interactions remains a fundamental skill for designers (Booth et al. Reference Booth, Taborda, Ramani and Reid2016) and also for non-designers that is critically important to the proper understanding of ideas (Dorta Reference Dorta, Lee and Choi2004). In addition, ideas communicated with high-quality sketches are much more likely to be perceived as creative compared with the same ideas shown with low-quality sketches (Kudrowitz et al. Reference Kudrowitz, Te and Wallace2012). In fact, VR sketching provides exciting alternatives for creating and expressing new design ideas and communicating visually. However, VR sketching is questionable, would non-designers be able to use it? Improper use of VR sketching could harm the whole process of Do-It-Together.
Some studies have already highlighted collaborative case studies (e.g., de Klerk et al. Reference de Klerk, Duarte, Medeiros, Duarte, Jorge and Lopes2019; Safin Reference Safin, Werner and Koering2020; Fleury et al. Reference Fleury, Dupont, Chaniaud, Tamazart, Poussard, Gorisse and Richir2022) involving lay people in a design process via a VR sketching tool. However, to our knowledge, few studies seem to have focused on the skills needed to use VR sketching compared to traditional sketching, especially with non-designers. This suggests that we do not know if VR sketching is suitable for non-designers which is a crucial tool for collaboration between designers and non-designers in a Do It Together platform.
2. Skills needed for traditional sketching
Drawing/sketching is an active, creative and self-directed process leading to a “different way of seeing” (Edwards Reference Edwards1997). Sketching is an integral part of the engineering curriculum for conceptual understanding, communication and design (Merzdorf et al. Reference Merzdorf, Weaver, Jaison, Hammond, Linsey and Douglas2021). Designers need to have the ability to produce quality sketches that can be used by others (Goldschmidt Reference Goldschmidt, Chakrabarti and Blessing2014; Booth et al. Reference Booth, Taborda, Ramani and Reid2016). It means having drawing skills in two dimensions and three dimensions (e.g., for the sketches of blueprint, engineering drawing and CAD). That is why, we were interested in the skills needed for traditional sketching, then VR sketching.
A significant number of studies suggest that traditional drawing experts have acquired many skills such as better attention span, manipulation skills and better spatial skills (e.g., Orde Reference Orde1997; Alias, Gray & Black Reference Alias, Gray and Black2002; McManus et al. Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010; Perdreau & Cavanagh Reference Perdreau and Cavanagh2015; Contreras et al. Reference Contreras, Escrig, Prieto and Elosúa2018; Benear et al. Reference Benear, Sunday, Davidson, Palmeri and Gauthier2019; Chamberlain et al. Reference Chamberlain, Kozbelt, Drake and Wagemans2021; Park, Wiliams & Chamberlain Reference Park, Wiliams and Chamberlain2021). Drawing/sketching involves a wide range of major cognitive functions such as:
Spatial skills (e.g., mental rotation, spatial visualization, spatial orientation, spatial memorization, spatial exploration) to analyze, understand and visualize space in two and three dimensions (Linn & Petersen Reference Linn and Petersen1985; Tartre Reference Tartre, Fennema and Leder1990; Samsudin, Rafi & Hanif Reference Samsudin, Rafi and Hanif2011). This article focuses on visuospatial abilities. Alias et al. (Reference Alias, Gray and Black2002) were interested in the link between the visuospatial abilities of engineering and architecture students and their sketches. The authors found that visuospatial abilities are directly linked to the tendency to use sketching/drawing and indirectly to students’ view of the professional role of sketching/drawing.
Memory with many subsystems (sensory memory, short-term memory, long-term memory – for example, Atkinson & Shiffrin Reference Atkinson, Shiffrin, Spence and Spence1968). For instance, McManus et al. (Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010) and Perdreau & Cavanagh (Reference Perdreau and Cavanagh2015) showed that traditional drawing experts have a better visual memory than traditional drawing novices. This would be related to the ability to copy angles and simple proportions (McManus et al. Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010). Spatial abilities and spatial memory are interdependent skills.
Sketching skills take a long time to develop (Booth et al. Reference Booth, Taborda, Ramani and Reid2016). For example, perspective drawing is not intuitive. According to Booth et al. (Reference Booth, Taborda, Ramani and Reid2016), mastery of sketching skills requires training and practice. In a drawing task, when participants reproduce a simple or complex shape, they have to plan the sequence of elements to be drawn and they have to consider the spatial relationships between them (La Femina et al. Reference La Femina, Senese, Grossi and Venuti2009). From this point of view, drawing can be considered to be a particular type of construction task. This involves developing a strategy and learning how to draw (La Femina et al. Reference La Femina, Senese, Grossi and Venuti2009).
The use of strategy was evidenced by a visual exploration of specific drawings by trained draftsmen. Park et al. (Reference Park, Wiliams and Chamberlain2021) were interested in the eye movements of artists and non-artists while making drawings representative of photographic stimuli. The authors showed that it was possible to discriminate between the two groups based on their global and local ocular saccades when looking at the target stimulus during drawing. The results showed that these differences in eye movements are not specifically related to figurative drawing ability and may be a feature of artistic ability more generally. This would imply that participants with drawing experience would have greater artistic ability than those without drawing experience. Thus, expertise is generally acquired as a result of deliberate practice (from time spent drawing, to using drawing techniques) and a flexible approach to learning strategies (Chamberlain et al. Reference Chamberlain, McManus, Brunswick, Rankin and Riley2015). Conversely, McManus et al. (Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010) show that drawing skills were not related to socio-demographic characteristics. However, the authors showed that the subjective assessment of drawing skills was representative of the actual skills of the drawers. Students who perceived themselves as good drawers drew better than those who perceived themselves as poor drawers.
To sum up, traditional sketching requires several skills mainly including spatial abilities and training.
3. Skills needed for VR sketching
In the previous lines, we mentioned some key advantages of VR sketching, which has also some disadvantages. According to Wiese et al. (Reference Wiese, Israel, Meyer and Bongartz2010) and Arora et al. (Reference Arora, Kazi, Anderson, Grossman, Singh and Fitzmaurice2017), VR sketching is less accurate than traditional sketching. It seems more difficult to draw a 3D sketch than a 2D sketch. This is due to the absence of a physical surface (Arora et al. Reference Arora, Kazi, Anderson, Grossman, Singh and Fitzmaurice2017) and the “depth perception errors” (Cave & Kosslyn Reference Cave and Kosslyn1993; Tramper & Gielen Reference Tramper and Gielen2011; Arora et al. Reference Arora, Kazi, Anderson, Grossman, Singh and Fitzmaurice2017), that is, a lack of spatial representation. Compared to traditional sketching which is in 2D, VR sketching which is in 3D requires higher demands on the user’s perception, motor and visuospatial abilities (Barrera Machuca, Stuerzlinger & Asente Reference Barrera Machuca, Stuerzlinger and Asente2019). In addition, VR sketching requires movements and spatial inspection (Barrera Machuca et al. Reference Barrera Machuca, Stuerzlinger and Asente2019; Yang & Lee Reference Yang and Lee2020). Yang & Lee (Reference Yang and Lee2020) highlight that spatial inspection is a behavioral factor for successful VR sketching. Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019) explain that users need to use different views to plan their next hand movement. All of these arguments seem to converge toward the importance of taking into account the spatial environment and visuospatial abilities (e.g., La Femina et al. Reference La Femina, Senese, Grossi and Venuti2009; Branoff & Dobelis Reference Branoff and Dobelis2013; Barrera Machuca et al. Reference Barrera Machuca, Stuerzlinger and Asente2019; Obeid & Demirkan Reference Obeid and Demirkan2020). Nevertheless, visuospatial abilities could be improved with training (Dünser et al. Reference Dünser, Steinbügl, Kaufmann and Glück2006; Samsudin et al. Reference Samsudin, Rafi and Hanif2011). Wiese et al. (Reference Wiese, Israel, Meyer and Bongartz2010) showed that the quality of VR sketches improves over time. More specifically, prolonged use of virtual learning environment could improve mental rotation skills (Farzeeha et al. Reference Farzeeha, Omar, Mokhtar, Ali, Suhairom, Abd Halim, Shukor and Abdullah2017). In contrast, Bolier et al. (Reference Bolier, Hürst, van Bommel, Bosman and Bosman2018) found that the quality of VR drawings made by children improves over time but not their visuospatial abilities.
4. Research question and hypotheses
Thus, currently, there does not seem to be a consensus on the impact of visuospatial abilities and training on the use of VR sketching. Only a few studies have investigated the use of VR sketching especially among non-designers. The aim of this study is to better understand the skills needed for traditional sketching and VR sketching for non-designers. To do this, we formulated five hypotheses:
-
• First, according to La Femina et al. (Reference La Femina, Senese, Grossi and Venuti2009) McManus et al. (Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010) and Chamberlain et al. (Reference Chamberlain, McManus, Brunswick, Rankin and Riley2015), training seems to be an essential element for the quality of traditional and VR sketching. We hypothesize that users with high traditional drawing skills (e.g., who received a traditional drawing training) will produce higher quality traditional and VR Sketches than those with low drawing skills.
-
• Second, with the same approach, according to Wiese et al. (Reference Wiese, Israel, Meyer and Bongartz2010) and Bolier et al. (Reference Bolier, Hürst, van Bommel, Bosman and Bosman2018), we hypothesize that VR experts (e.g., who are used to VR sketching) will produce higher quality traditional and VR sketches than VR novices.
-
• Third, according to Alias et al. (Reference Alias, Gray and Black2002), La Femina et al. (Reference La Femina, Senese, Grossi and Venuti2009) and Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019), we hypothesize that the higher the visuospatial abilities (e.g., mental rotation) of the users, the higher the quality of traditional and VR sketching.
-
• Fourth, according to Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019) and Yang & Lee (Reference Yang and Lee2020), movement and spatial inspection are crucial to successful VR sketching. Hence, we hypothesize that the users who move (i.e., users move their body or/and their head in the 3D environment) the most are those who will have a better quality of VR sketching.
-
• Fifth, the perceived usability of the software used in this study (Time2Sketch) can have an impact on the quality of the VR sketches. If the software does not have a good perceived usability, then users will have VR sketches of lower quality.
5. Methods
5.1. Participants
Thirty-one participants, 15 females and 16 males aged 18–62 years (means = 34.03 ± 12.75) participated in this study. All the participants were native French speakers and signed an informed consent form. The data collected about participants was anonymous. This study was in line with the ethical recommendations of the Declaration of Helsinki. Participants did not receive financial compensation. The VR experts were recruited from a center of research or a VR compagnies specialized in VR and the VR novices were recruited through a call for studies. The experts work in VR and the novices do not work in this field.
5.2. Materials and measurements
5.2.1. Time2Sketch software
Time2Sketch is an immersive sketching software used in the experiment to allow the user to draw freehand lines in VR. Users can change color, brush size, erase the lines, undo the last action, resize the sketch, teleport in the environment and position a symmetry axis. The VR headsets used was Oculus Quest.
5.2.2. Measure of position and movement
The headset records the following participants’ movement while using the equipment: position in the scene on an x, y and z-axis which allows to deduce the horizontal and vertical displacement of the user (in meters) and headset rotation (yaw, pitch and roll).
5.2.3. Questionnaires
Four questionnaires were distributed to participants: socio-demographic, traditional drawing skills, VR skills, mental rotation test (MRT) and usability questionnaire.
Socio-demographic: This questionnaire included personal details: age and gender.
Traditional drawing skills: To assess the drawing skills, we asked the participants the following questions: did they have any training in traditional drawing (hobby or professional); how often did they use drawing (never, few, sometimes, often, very often); to evaluate themselves subjectively on their drawing skills, that is, their comfort in using drawing (not at all comfortable, a little comfortable, moderately comfortable, quite comfortable, completely comfortable). Clusters were created according to the answers to the three questions. For this, training is rated from 0 to 2 points (1 point for hobby training and 2 points for professional training); frequency is rated from 0 to 4 points; fluency is rated from 0 to 4 points. Then, participants were divided into groups of level (novice, apprentice, advanced, expert) using k-means.
VR drawing skills: VR expert versus VR novice: To assess the VR skills, we asked the participants the following questions: did they have any training in VR (yes: VR expert, no: VR novice); have you ever used VR sketching (yes, no). All the VR experts have already used VR sketching and all the VR novices have never used VR sketching.
Mental rotation test A (MRTA): The MRTA (by Peters et al. Reference Peters, Laeng, Latham, Jackson, Zaiyouna and Richardson1995) is a redrawn version of Vandenberg & Kuse’s (Reference Vandenberg and Kuse1978). The test has 24 items organized in 4 pages. Each item is composed of five figures: a reference model on the left side and four figures located on the right side of the reference model among which the participants have to indicate the ones that are similar to the reference model. There are always two correct answers per item. The time is divided into 2 × 3 minutes with a 4-minute break in between. One point is given per item if the participant finds the two correct figures. No points are given if the participant finds 1 or 0 figures. The sum of these points will give the MRT score ranging from 0 to 24.
System Usability Scale (SUS): This 10-item survey aimed at recording subjective assessments of usability (Brooke Reference Brooke1996; Lewis & Sauro Reference Lewis and Sauro2009) is a “quick and dirty” tool with five response options from strongly agree to strongly disagree. We used the French-validated version (Gronier & Baudet Reference Gronier and Baudet2021). The SUS score ranges from 0 to 100. The closer the score is to 100, the better the perceived usability.
5.3. Procedure
The average duration of this experiment was around 60 minutes and was structured in four steps: (1) participants were asked to complete a series of questionnaires (socio-demographic, traditional drawing skills, VR skills, MRT); (2) then the participants had to perform task 1 (traditional sketching), that is, to reproduce a writing-desk using a pencil and paper in 7 minutes. They were asked to reproduce as closely as possible the photo presented on a computer. They could look at the photo in front of them as many times as they wanted; (3) participants were immersed in a neutral virtual environment (hangar) and were trained to use Time2Sketch. There was no time limit for them to learn the software. Once they were familiar with the software, they had to perform the sketching tasks in VR. Then, the photos of the pieces of furniture to be reconstructed in 3D appeared in the immersive environment. The two VR drawing tasks were presented randomly and consecutively to the participants, who could look at the image for as long as they liked without having to remove the headset. Participants had 10 min to make the basic task (the shelf – task 2) and 20 min for the complex task (the buffet – task 3). The instructions imposed were always the same: “reproduce the furniture as faithfully as possible, taking into account the volumes” including the life-size of the furniture. These three pieces of furniture (Figure 1) have been selected according to their complexity. The writing desk (task 1) has been selected because it requires an effort on the perspectives. The shelf (task 2) is a simple task with a simple geometric shape, while the buffet is a complex task requiring to take into account the opening angle of the cabinets and drawers and many details. This choice of items of furniture (with details, door opening, etc.) makes it possible to reflect all the range of drawing skills of the participants in line with the evaluation criteria. (4) Finally, the participants were asked to answer the SUS questionnaire.
Figure 1. Presentation of the three drawing copy tasks. In task 1, the writing desk is copied with a pencil/paper. The basic task 2 (the shelf) and the complex task 3 (the buffet) are copied using Time2sketch software in a VR environment.
5.4. Measuring quality of the sketches
Two expert in VR drawing judges evaluated independently each sketch (VR sketches are presented under four faces: top, profile, front and ¾ frontal – see Figure 2) with a set of criteria using a grid detailing each point of each criterion (Supplementary Material 1): respect for volume/perceptive; respect for proportion; quality of the lines; fidelity with the original picture. Each criterion is scored from 1 to 5. The sum of the points gives a score between 4 and 25. The higher the score the better the quality of the sketch.
Figure 2. Example of sketches performed by two participants (24 and 25) according to the three tasks: task 1 (traditional sketching), tasks 2 and 3 (VR sketching).
5.5. Data analysis
Results were analyzed using SPSS version 22 (IBM Corporation, 2013), JASP Team (2022) version 0.16.4 and RStudio. Each score task was systematically compared to user characteristics and usability components. Bivariate correlations and ANOVAs were performed when the sample met the homoscedasticity criteria. K-mean was used to perform traditional drawing-level clusters.
5.6. Inter-judge reliability
Two expert judges analyzed the sketches using a grid with four variables (volume, proportion, quality of the line and fidelity). We used Intra-Class Correlation (ICC2,k) two-way random to verify inter-judge reliability for the quality of the sketches (Shrout & Fleiss Reference Shrout and Fleiss1979). The mean ICC2,k measurement for task 1 (the writing desk) was .946 with a 95% confidence interval of .884 to .947 (F(30,26.8) = 19.8, p < .001). The mean ICC measurement for task 2 (the shelf) was .935 with a 95% confidence interval of .745 to .976 (F(30,7.32) = 22.7, p < .001). The mean ICC measurement for the task 3 (the buffet) was .894 with a 95% confidence interval of .664 to .958 (F(30,9.57) = 12.8, p < .001). All criteria above 0.9 are considered excellent, above 0.7 good and above 0.5 moderate (Koo & Li Reference Koo and Li2016). The reliability has been evaluated by an “average k ratings” that is why we used the averages of the data of judges 1 and 2 for the results.
6. Results
Figure 2 shows some examples of sketches created by the participants. The details of the quality score of the sketches given by the two judges are presented in Supplementary Material 2. Table 1 shows the details of the user characteristics according to the quality score of the sketches for the three tasks.
Table 1. Descriptive analyses of user characteristics, sociodemographic, drawing skills, VR skills, mental rotation skills and SUS score according to the quality of the sketches
6.1. Traditional drawing skills
Traditional drawing training impacts significantly the quality of the traditional sketches (task 1: F(2,30) = 6.187, p = .006, η 2 = 0.31) and the quality of the VR sketches (task 2: F(2,30) = 9.239, p < .001, η 2 = 0.4); task 3: F(2,30) = 7.39, p = .003, η 2 = 0.345). Participants who received training in traditional drawing produce much higher quality traditional and VR sketches than those who have not received training. Similarly, the subjective evaluation of these traditional drawing skills (comfort in drawing) is not significantly linked to the quality of traditional sketches (task 1: F(3,30) = 2.328, p = .095) and but is significantly linked to the quality of VR sketches (task 2: F(3,30) = 4.86, p = .008, η 2 = 0.35, task 3: F(3,30) = 6.93, p = .001, η 2 = 0.44). Participants who perceived themselves as good traditional drawers drew better than those who perceived themselves as poor traditional drawers. The frequency of use of the traditional drawing does not impact significantly the quality of the traditional sketches (task 1: F(2,30) = 0.72, p = .49) and the quality of the VR sketches (task 2: F(2,30) = 0.485, p = .621; task 3: F(2,30) = 0.048, p = .95).
Based on the three variables (training, comfort and frequency), four clusters were performed with k-means R 2 = 0.743 (novice: mean = .36, SD = 0.67; apprentice: Mean = 1.9, SD = 0.74; advanced: mean = .4.18, SD = 0.98, expert: mean = 7, SD = 0). Clustering of traditional drawing skills impact significantly the quality of the traditional sketches (task 1: F(3,27) = 3.54, p = .028, η 2 = 0.28) and the quality of the VR sketches (task 2: F(3,27) = 4.61, p = .01, η 2 = 0.34); task 3: F(3,27) = 4.66, p = .009, η 2 = 0.34).
6.2. VR drawing skills
The VR experts performed significantly better quality of VR sketches than VR novices (task 2: F(1,30) = 6.61, p = .016, η 2 = 0.19; task 3: F(1,30) = 6.82, p = .014, η 2 = .19) but there is no significant difference for the traditional sketching (task 1: F(1,30) = 0.175, p = .68).
6.3. Mental rotation skills
There was no significant correlation between the mental rotation score and the quality of the traditional sketches (task 1: r = 0.34, p = .063). Conversely, the mental rotation score is significantly correlated (positively and strongly) with the quality of the VR sketches (task 2: r = .617, p < .001; task 3: r = .52, p = .006). The higher the participants’ mental rotation score, the higher the quality of the VR sketches.
In addition, there was no significant correlation between the mental rotation score and the VR skills (F(1,29) = 1.255, p = .272).
6.4. Movements in VR sketching
There was no significant correlation between the position in the scene and headset rotation and the quality of VR sketches for task 2 (horizontal movement: r = 0.11, p = .55; vertical movement: r = 0.05, p = .79; yaw: r = −0.18, p = .34; pitch: r = 0.087, p = .64; roll: r = 0.2, p = .28). There is a significant correlation between movements in the scene and the quality of VR sketches for task 3 (horizontal movement: r = 0.47, p = .008; vertical movement: r = 0.48, p = .006) but the correlation is not significant with the headset rotation (yaw: r = 0.1, p = .6; pitch: r = 0.26, p = .15; roll: r = 0.29, p = .16). Participants who moved more in the scene had better quality complex VR sketches. Conversely, head movements that created parallax did not improve the quality of the sketches.
There was no significant difference between the movements and headset rotation in the scene and the VR skills (task 2: horizontal movement: F(1,29) = 0.015, p = .9; vertical movement: F(1,29) = 0.03, p = .86; yaw: F(1,29) = 0.152, p = .7; pitch: F(1,29) = 0.52, p = .48; roll: F(1,29) = 0.11, p = .75; task 3: horizontal movement: F(1,29) = 0.9, p = .35; vertical movement: F(1,29) =0.79, p = .38; yaw: F(1,29) = 0.3, p = .59; pitch: F(1,29) = 0.61, p = .44; roll: F(1,29) = 0.07, p = .79) and traditional drawing skills (task 2: horizontal movement: F(1,29) =0.72, p = .49; vertical movement: F(1,29) = 1.1, p = .35; yaw: F(1,29) = 0.23, p = .8; pitch: F(1,29) = 0.3, p = .74; roll: F(1,29) = 0.42, p = .66; task 3: horizontal movement: F(1,29) = 0.47, p = .63; vertical movement: F(1,29) =0.37, p = .69; yaw: F(1,29) = 0.044, p = .96; pitch: F(1,29) = 0.48, p = .62; roll: F(1,29) = 0.29, p = .75).
There was no significant correlation between the movements and headset rotation in the scene and the mental rotation skills (task 2: horizontal movement: r = −0.01, p = .95; vertical movement: r = −0.08, p = .97; yaw: r = −0.22, p = .25; pitch: r = −0.08, p = .68; roll: r = −0.09, p = .61; task 3: horizontal movement: r = −0.04, p = .61; vertical movement: r = −0.04, p = .84; yaw: r = −0.15, p = .44; pitch: r = 0.02, p = .9; roll: r = 0.11, p = .54).
6.5. System Usability Scale
Participants gave an average SUS score of 75.57 (SD = 12.68, range = 50–97.5). There is no significant correlation between the SUS score and the quality of VR sketches (task 2: r = 0.14, p = .45; task 3: r = 0.15, p = .42), and no link is observed between the SUS score and the VR skills (F(1,30) = 0.001, p = .98) and the traditional drawing skills (traditional drawing training: F(2,30) = 0.02, p = .98; frequency of using traditional drawing: F(2,30) = 0.3, p = .74; comfort in traditional drawing: F(3,30) = 0.77, p = .52) made with the software Time2Sketch.
7. Discussion
This study’s objective was to better understand the skills needed for traditional sketching and VR sketching (with Time2sketch) for non-designers. To do this, we collected drawing skills, VR drawing skills using a questionnaire, mental rotation using the MRT, and two independent judges rated the quality of the traditional and VR sketches of the participants. We made four hypotheses in which traditional drawing skills, VR drawing skills, mental rotation and movement would have an impact on the quality of traditional and VR sketches.
Our first hypothesis was that participants with high traditional drawing skills would create a better traditional and VR sketches than those with low traditional drawing skills. Results validate our first hypothesis. Three variables were measured to assess the traditional drawing skills: traditional drawing training, frequency of use of the traditional drawing and comfort in traditional drawing. Our results show that participants who received training (hobbies and professional) and who rate themselves positively produce significantly better traditional and VR sketches. These results are in line with those of La Femina et al. (Reference La Femina, Senese, Grossi and Venuti2009), McManus et al. (Reference McManus, Chamberlain, Loo, Rankin, Riley and Brunswick2010) and Chamberlain et al. (Reference Chamberlain, McManus, Brunswick, Rankin and Riley2015). However, the frequency of use does not impact the quality of traditional sketches. Nevertheless, the data used are not very diversified. For instance, there are only six participants who have received training, none of whom consider themselves to draw often or very often. To limit this bias, we clustered this skill into four groups (novice, apprentice, advanced and expert) based on these three variables (training, comfort and frequency). The results show that there is a significant difference between the groups and the quality of traditional and VR sketches. More precisely, the better the assessed level of traditional drawing skill (e.g., expert or advanced) the better the quality of the traditional and VR sketches and vice versa.
Our second hypothesis was that VR experts will have a better quality of traditional and VR sketches than VR novices. Results validate partially our hypothesis. VR experts performed better quality of VR sketches than VR novices, but this is not the case for traditional sketching. These results are in line with those of Wiese et al. (Reference Wiese, Israel, Meyer and Bongartz2010) and Bolier et al. (Reference Bolier, Hürst, van Bommel, Bosman and Bosman2018). Because of the emergent nature of VR technology, there is not yet a cohort of designers trained in VR drawing that we could have used in this study. We were therefore more interested in the expertise in VR. We can wonder about what is a VR expert and when does one becomes an expert? To the best of our knowledge, no research is currently being done in this area. We considered that experience and training were the main elements to consider that a person was an expert in the field. Conversely, novices rarely interacted in a virtual environment. We could have provided more nuances to these two profiles by questioning the expertise in VR. For this, it would be interesting to carry out studies on the learning of drawing techniques in VR according to the training.
According to the first two hypotheses, the traditional and VR drawing skills are a crucial element in the traditional and VR sketches quality, regardless of the complexity of the task (no difference observed on the effect size between tasks 2 and 3). However, traditional sketching techniques seem transferable to VR sketching but not vice versa. One of the requirements for using VR sketching for novices would be strong traditional drawing skills, that is, have previous training in traditional drawing or feel comfortable with traditional drawing.
Our third hypothesis was that the higher the visuospatial abilities of the participants, the higher the quality of the traditional and VR sketches. We focused on mental rotation to assess the visuospatial abilities using the MRT (Vandenberg & Kuse Reference Vandenberg and Kuse1978). Results partially validate our hypothesis. A high mental rotation score is related to high quality of VR sketches but not linked to traditional sketches quality. These results are in line with Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019) but not with Alias et al. (Reference Alias, Gray and Black2002). Alias et al. (Reference Alias, Gray and Black2002) showed an impact of visuospatial abilities measured with the SVATI – Spatial Visualization Ability Instrument (Embretson Reference Embretson1997) which is highly correlated with the Vandenberg MRT (Alias Reference Alias2000) on the quality of the traditional sketches. Conversely, Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019) found that the user’s visuospatial abilities (using the vz-2 paper folding test – Ekstrom, French & Harmon Reference Ekstrom, French and Harmon1976 – and the perspective taking/spatial orientation test – Kozhevnikov & Hegarty Reference Kozhevnikov and Hegarty2001) affects the shape of the sketches, but not the line precision in the VR sketching. In line with Cohen’s (Reference Cohen2013) guidelines, we observed a difference in correlation strength between task 2 (strong correlation) and task 3 (moderate correlation) which suggests that mental rotation skills would not be related to accuracy since task 3 requires more accuracy because of the number of details. If the complexity of the VR sketching task increases, the training and the traditional drawing strategies (i.e., traditional drawing skills) would compensate for the mental rotation skills. On the one hand, visuospatial abilities could help the draftsman, but a training is required to achieve a good quality of traditional sketches. On the other hand, mental rotation skills seem to be an essential requirement for the basics of VR sketching. However, accuracy is one of the important challenges that VR sketching has to face (Wiese et al. Reference Wiese, Israel, Meyer and Bongartz2010; Arora et al. Reference Arora, Kazi, Anderson, Grossman, Singh and Fitzmaurice2017; Barrera Machuca et al. Reference Barrera Machuca, Stuerzlinger and Asente2019). The lack of accuracy is detrimental to the creation process because the sketch may not correspond to the user’s intention (Barrera Machuca et al. Reference Barrera Machuca, Stuerzlinger and Asente2019).
Our fourth hypothesis was that participants who move the most will have a better quality of VR sketches. After analysis of the movements recorded with the headset, we can partially validate our hypothesis. The movements impact the quality of the VR sketches only for task 3 and only for position in the scene and not the headset rotation. Thus, as with the previous hypothesis, the level of complexity of the task seems to impact user behavior and requires more spatial inspection and movement. If task 2 does not require movement to improve the quality of the VR sketches, it is probably related to the size of the task which required little backward movement and spatial cues. Indeed, task 2 is a square tube whereas task 3 includes different levels based on different sizes of rectangles. In addition, details such as open drawers or open doors added the consideration of angle, which is not required in task 2. These results are in line with Barrera Machuca et al. (Reference Barrera Machuca, Stuerzlinger and Asente2019) and Yang & Lee (Reference Yang and Lee2020). In addition, we observed that mental rotation skills, traditional and VR drawing skills are not related to spatial inspection. The visual knowledge we have is derived from a two-dimensional (2D) image of an object in a scene on the retina (Frith & Law Reference Frith and Law1995). Movements and spatial inspection provide information on the depth as well as the previous knowledge helps to prospect on the size and the shape of the object (Frith & Law Reference Frith and Law1995), that is why, the movement is an important variable to take into account, especially in VR sketching. It seems that the rotation of head is not sufficient to allow the perception of this depth. Nevertheless, it is not known if encouraging users to move would increase the quality of the VR sketching.
Our fourth hypothesis was that participants who perceive the usability of the software as good will have better VR sketches quality than those who perceive the usability of the software as bad. The results invalided the hypothesis. It is important to highlight that regardless of traditional and VR drawing skills and visuospatial abilities, users uniformly rated the perceived usability of the Time2Sketch software. The SUS score (measuring perceived usability) does not have a significant impact on the quality of VR sketches. This suggests that they did not report being bothered by the usability of the software. The mean SUS score was 75.57 (SD = 12.68), which is “satisfactory” (Lewis & Sauro Reference Lewis and Sauro2009). It has been known for a long time that usability is a major factor of technology acceptance (Davis Reference Davis1989). However, according to our results, perceived usability of theVR tool is not a determinant of sketching performance. If the software does not have an impact on the perceived usability, it seems that the problem comes mainly from the 3-dimensions. Users are able to use all the features of the application easily but it is their graphic skills that prevent them from having better quality sketches. Figure 3 shows a summary of the results. These data are related to a study done on the same software (Fleury et al. Reference Fleury, Dupont, Chaniaud, Tamazart, Poussard, Gorisse and Richir2022).
Figure 3. Schematic representation summary of the results (task 3) *p ≤ .05; **p ≤ 0.01.
We propose six perspectives to this study. First, the translation of a 2-dimensional picture into 3-dimensions necessarily implies visuospatial abilities. Participants have different strategies for viewing in 2D versus 3D (e.g., Popelka & Brychtova Reference Popelka and Brychtova2013). Movement can be a way to compensate for a weakness in visuospatial abilities. It would be interesting to force users to move when they use VR sketching. In addition, it would be interesting to observe the type of movements made in order to better understand the strategies used. Second, with the same approach, to better understand the role of visuospatial abilities, it would have been interesting to ask users to reproduce a task already presented in 3D to better understand the impact of visuospatial abilities. Third, it would be interesting to compare the traditional and VR sketches with a same task to see if the quality is maintained between both techniques. Fourth, the next study may examine the impact of VR drawing skills on creativity. For instance, Chan & Zhao (Reference Chan and Zhao2010) showed in a study with primary, secondary and university students, a correlation between drawing skills and artistic creativity. Yang & Lee (Reference Yang and Lee2020) showed that VR sketching allows to be more creative than traditional sketching, but the authors do not take into account the drawing skills and the visuospatial abilities. To our knowledge, no study has shown the impact of VR drawing skills on creativity in the ideation process. Fifth, our results are based only on Time2sktech software. In order to avoid the difficulties experienced by novices, it would be interesting to analyze if they use more easily sketching software adapted to their visual–spatial difficulty, such as software to create 3D blocks or sculptures or with a Hybrid Virtual environment – for example, the Hyve-3D used in the case of architectural co-design – using handheld tablets to manipulate a cursor on a plan (Dorta et al. Reference Dorta, Kinayoglu and Hoffmann2016) or more recently a pen and table interact (e.g., VRsketchIn – Drey et al. Reference Drey, Gugenheimer, Karlbauer, Milo and Rukzio2020). Nevertheless, the advantages of Time2Sketch compared to other devices (e.g., Hyve-3D, VRsketchIn) are its speed execution, its immediate scaling and its flexibility, especially its ambiguity and imprecision allowing multiple interpretation readings which supporting creative leaps (Ullman, Wood & Craig Reference Ullman, Wood and Craig1990) which is an essential tool for good sketches (Buxton Reference Buxton2007). Moreover, Time2Sketch is a mobile and affordable software since it only requires a VR headset and can be used everywhere. Sixth, last but not least, this study is the beginning of many others that will gradually be integrated into the Do-It-Yourself process in order to test and validate the new process. The ultimate aim will be to enable novice users to create a made-to-measure piece of furniture with the help of an experienced designer.
There are four limits of our study. First, we measured the visuospatial abilities with MRT. However, there are other dimensions of visuospatial abilities (Linn & Petersen Reference Linn and Petersen1985) such as mental transformation (Tartre Reference Tartre, Fennema and Leder1990) and visuospatial working memory (Logie Reference Logie2014). Nevertheless, mental rotation skills are associated with other visuospatial abilities (e.g., Alias Reference Alias2000; Ault & John Reference Ault and John2010). For instance, Muffato, Meneghetti & De Beni (Reference Muffato, Meneghetti and De Beni2020) observed a strong correlation between the sMRT (Short Mental Rotations Test), the sOPT (Short Object Perspective Taking Test) and the VSWM (visuospatial working memory). In addition, the MRT is one of the most cited and preferred tests in studies of industrial design education (Kelly Reference Kelly2012). Second, we chose a furniture use case which makes the task less pure than if we had presented volumes without realistic equivalent. In addition, users could sketch on their prior knowledge to help them consider the shape of the furniture (Frith & Law Reference Frith and Law1995). In the same way, we limited the time for the creation of the task (7 min for task 1, 10 min for task 2 and 20 min for task 3). Some participants were not finished by this time. To respect the real process, end-users will not have a time limit but in an experimental setting, it was necessary to put one. Third, Time2Sketch software will be useful in the case of creative tasks or communication with different users or designers in the early phase of the design process. We wanted to know if novice users in VR would be able to use such software and would be able to communicate on it. We do not have data on the ability of users to communicate though their sketches, this could be the subject of a new study. Then, when creating a sketch in a design process, users use their mental image to create a new and unique production using their mental image which is not the same task as duplicating a stimulus. We therefore have no data on the ability of users to sketch their own mental image of furniture but we would not have been able to evaluate the graphic quality of the mental image because the sketches would not have been comparable between them. We consider that if participants are able to reproduce an image, then they will be able to transfer their mental image. Fourth, we based our data on VR sketching on only on software (Time2sketch). Nevertheless, according to the SUS score, Time2sketch did not seem to disturb the participants. It would be interesting to generalize the results to other VR sketching software.
To conclude, traditional sketching is easier to use than VR sketching – with T2S software – which requires more drawing skills and visuospatial abilities of the novice users. Regardless of the usability of the software, we identified three requirements for the use of VR sketching: first, it is required to have previous experience in traditional drawing or VR. Second, it is required to have high visuospatial abilities. Third, it is strongly advised that users move around in the virtual environment during a VR sketching task to become aware of the depth of the drawing in progress. We recommend that novice users to use mainly traditional sketches in order to express their needs and avoid misunderstandings with the future designer or to use another sketching software in order to suggest a software more adapted to their needs (e.g., using 3D blocks or Hyve-3D).
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/dsj.2023.27.
Financial Support
This study was partly funded by the European Commission through the INEDIT E.U. innovation project (grant agreement no. 869952).
Open access
