1. Introduction
The constantly accelerating consumption and depletion of natural resources have made sustainability a priority now, more than ever. This emphasis on sustainability has been called for across disciplines, ranging from psychology (Climate Change Intensifies n.d.) to engineering (Mihelcic et al. Reference Mihelcic, Naughton, Verbyla, Zhang, Schweitzer, Oakley, Wells and Whiteford2017). Engineering design has the potential to play an important role in addressing the need for sustainability through the design, development and manufacturing of sustainable solutions. Motivated by this need for integrating sustainability in engineering design, several researchers have proposed initiatives that range from design educational interventions (Prabhu et al. Reference Prabhu, Starkey and Alsager Alzayed2021c) to design tools and methods (Ross et al. Reference Ross, Ferrero and DuPont2022). Furthermore, several researchers have proposed design tools, such as the eco-design guidelines (Maccioni et al. Reference Maccioni, Borgianni and Pigosso2021) and sustainable design heuristics (Blevis Reference Blevis2007), to help designers generate sustainable solutions.
Although it is important to generate sustainable engineering design solutions, it is also important to ensure that these solutions are selected for further development in the design process. In the concept selection stage of the design process, designers typically evaluate their solutions based on several criteria to identify the most effective solutions. Researchers have demonstrated that these criteria used to make concept selection decisions could range from creativity (Toh & Miller Reference Toh and Miller2016a) to perceived risk (Zheng & Miller Reference Zheng and Miller2017) to manufacturability (Prabhu et al. Reference Prabhu, Leguarda, Miller, Simpson and Meisel2021a), among several others – making concept selection a complex, multi-criteria decision-making process. Additionally, these multi-criteria decisions are often made by designers relying on their own evaluations of their solutions, individually or in groups (Toh & Miller Reference Toh and Miller2016b) or by employing external tools (Gosnell & Miller Reference Gosnell and Miller2016; Toh & Miller Reference Toh and Miller2016a; Booth et al. Reference Booth, Alperovich, Chawla, Ma, Reid and Ramani2017; Bracken et al. Reference Bracken, Pomorski, Armstrong, Prabhu, Simpson, Jablokow, Cleary and Meisel2020; Prabhu et al. Reference Prabhu, Simpson, Miller and Meisel2021e). Given the presence of human decision-making in concept selection, these decisions could be influenced by cognitive biases carried by the designer (Toh et al. Reference Toh, Strohmetz and Miller2016; Zheng & Miller Reference Zheng and Miller2019, Reference Zheng and Miller2021). One such cognitive barrier is recall inhibition due to prior knowledge: researchers have observed that designers’ prior experience and familiarity with a partial set of information could inhibit their ability to retrieve and use information in different stages of the design process (Prabhu et al. Reference Prabhu, Simpson, Miller and Meisel2021g).
In the context of sustainable design, designers might vary in their familiarity and prior experience with the various issues related to sustainability. These variances could result in designers demonstrating different levels of psychological distance (Spence et al. Reference Spence, Poortinga and Pidgeon2012) from sustainability-related issues. According to the construal level theory, psychological distance is defined as the degree of separation of certain external concepts from one’s immediate reality (Trope & Liberman Reference Trope and Liberman2010). Additionally, psychologically distant concepts are often thought of in more abstract and less concrete terms compared to psychologically near concepts (Trope & Liberman Reference Trope and Liberman2010). Researchers posit that issues related to environmental sustainability may be psychologically distant, especially among individuals in developed parts of the world. For instance, Spence et al. (Reference Spence, Poortinga and Pidgeon2012) find that individuals surveyed in the UK report that the effects of climate change were more likely to affect people in developing countries with little, albeit some, local effects. These results resonate with our prior findings wherein we see that student designers perceive the same sustainable design problem to be more temporally urgent when framed in socio-spatially distant settings (Prabhu et al. Reference Prabhu, Herzog, Fodale, Alsager Alzayed and Starkey2025). Extending this idea, Chu (Reference Chu2022) and Chu & Yang (Reference Chu and Yang2019) demonstrate that framing sustainability-related issues in a near-construal context could help encourage support for climate change mitigation. Therefore, psychological distance could influence their ability to effectively act toward solving these issues. These differences in ability, coupled with cognitive biases, could emerge in designers’ accuracy when evaluating the sustainability of their own solutions. Furthermore, these effects could be particularly important when designers are designing solutions for contexts that might be psychologically distant from themselves (e.g., designers in developed countries designing sustainable solutions for users in developing countries).
Prior research suggests that individuals’ empathy, i.e., the reactions of individuals to the observed experiences of others (Davis Reference Davis1983), could help them bridge their psychological distance, especially in the context of sustainability and climate change (Chu & Yang Reference Chu and Yang2019). Moreover, individuals’ beliefs, attitudes and intentions toward sustainability and climate change have been shown to predict their engagement in effective pro-environmental behavior (Steg & Vlek Reference Steg and Vlek2009). However, limited research has explored how these individual differences affect designers’ consideration of sustainability in the engineering design process, especially in the concept evaluation stage. Specifically, limited research has explored whether novice designers can accurately evaluate the sustainability of their solutions and whether this accuracy is influenced by individual differences such as trait empathy, beliefs, attitudes, and intentions toward sustainability. Additionally, given the complex nature of design evaluation, novice designers could demonstrate concept evaluation behavior that could vary from expert designers (Boudier et al. Reference Boudier, Sukhov, Netz, Le Masson and Weil2023). For instance, Atman et al. (Reference Atman, Chimka, Bursic and Nachtmann1999, Reference Atman, Cardella, Turns and Adams2005) find that experienced designers tend to spend more time making design decisions and gather more information in making these decisions, compared to novice designers. Dixon & Bucknor (Reference Dixon and Bucknor2019) make a similar observation and find that novices were more likely to use design heuristics that were local to the domain/problem at hand, whereas experts tend to use more external heuristics and bring new and potentially remote concepts to the design context. This behavior may also manifest in the form of self-reliance among expert designers, who, due to their domain knowledge and expertise, use design strategies that are best suited to the design context (Ahmed et al. Reference Ahmed, Wallace and Blessing2003). On the other hand, novice designers may employ a more “trial and error” approach (Ahmed et al. Reference Ahmed, Wallace and Blessing2003) and may therefore benefit from external stimuli to support their design decisions and actions (Reimlinger et al. Reference Reimlinger, Lohmeyer, Moryson and Meboldt2019; Budinoff et al. Reference Budinoff, McMains and Shonkwiler2024). Taken together, designers’ prior knowledge and experience may present themselves through their knowledge of and exposure to the different elements of sustainable design, which they leverage in making concept evaluation decisions. However, little research has explored novice designers’ behaviors in concept selection decision-making and the role of individual differences in moderating these decisions. Our aim in this research is to investigate this research gap through an experimental study. Such an investigation is particularly important as it can inform the development and effective integration of sustainability-focused concept evaluation tools. Additionally, this direction of investigation could inform the development of sustainable design educational interventions that also emphasize the role of individual differences in sustainable design behavior.
Before we introduce the specifics of our study, we discuss prior research that informed our research in Section 2, with the research questions (RQs) and corresponding hypotheses presented in Section 3. Our experimental methods are discussed in Section 4, the results of the experiment are discussed in Section 5 and the implications of these results are discussed in Section 6. Finally, we conclude with a discussion of limitations and directions for future work in Section 7.
2. Related work
To set up the foundation for the current study, prior research in the areas of (1) concept evaluation in engineering design and (2) the influence of individual differences on concept evaluation is reviewed, as discussed next.
2.1. Concept evaluation in engineering design
The concept selection stage is considered the “gatekeeping stage” of the engineering design process since it often impacts the success and final cost of a product (Mattson & Messac Reference Mattson and Messac2005; Pahl et al. Reference Pahl, Wallace and Blessing2007; Hambali et al. Reference Hambali, Sapuan, Ismail and Nukman2009). This stage of the design process is preceded by the concept generation stage, where designers ideate potential solutions for the problem at hand (Yang Reference Yang2009). During concept evaluation and selection, the generated concepts are evaluated and filtered according to the problem requirements and design objectives (King & Sivaloganathan Reference King and Sivaloganathan1999; Okudan & Tauhid Reference Okudan and Tauhid2008).
However, designers’ expertise in the problem domain could impact their design evaluations. For example, creativity researchers assert that expert ratings are the most appropriate measure of the creativity of an idea (Amabile Reference Amabile1996; Baer et al. Reference Baer, Kaufman and Gentile2004), whereas others argue that trained novices or quasi-experts can be used as a proxy for expert raters (Gosnell & Miller Reference Gosnell and Miller2016; Miller et al. Reference Miller, Hunter, Starkey, Ramachandran, Ahmed and Fuge2021). This hypothesis is the foundation for the Consensual Assessment Technique (CAT) – often considered the “gold standard” for creativity assessment. Amabile (Reference Amabile1996), p. 73) suggests that for the CAT to be effective, “judges should be closely familiar with works in the domain, at least at the level of those being produced by the subjects.” Moreover, researchers argue that nonexpert evaluations of creativity are not always consistent with expert raters when using the CAT (Kaufman et al. Reference Kaufman, Baer, Cole and Sexton∗2008). This research suggests that novice and expert raters might differ in their mental models when making concept evaluations, and these differences could influence the outcomes of their concept selection decisions. Furthermore, researchers in engineering design suggest that while experts and novices tend to have a high agreement in their evaluations of the novelty of their solutions, this agreement might not be present in the evaluations of quality when using the CAT (Miller et al. Reference Miller, Hunter, Starkey, Ramachandran, Ahmed and Fuge2021). In contrast, researchers observe that expert and novice raters could show high agreement in their evaluations of the quality, novelty, and overall creativity of ideas when using other creativity evaluation methods such as the tool for assessing semantic creativity (TASC) (Gosnell & Miller Reference Gosnell and Miller2016. These findings suggest that (domain) expertise may help designers be more effective in concept evaluations. Additionally, the accuracy of novice designers’ evaluations could not only vary based on the problem domain but also by the evaluation tools being employed.
Prior research in cognitive psychology also suggests that pattern recognition allows experts in a specific domain to develop automatic processing of perceived information (Shiffrin & Schneider Reference Shiffrin and Schneidern.d.; Baddeley Reference Baddeley2001). However, experts have also been observed to overlook important information that novice raters can retain due to their lack of expertise or “newness” in the domain (Licuanan et al. Reference Licuanan, Dailey and Mumford2007). Therefore, in sustainable design tasks, designers’ familiarity with issues related to sustainability could impact the accuracy of the evaluations of their solutions. A greater familiarity and expertise with sustainable design may result in designers overemphasizing sustainability over other important characteristics, such as novelty and quality. On the other hand, a lack of exposure to sustainable design may result in inexperienced designers generating novel solutions that may not be environmentally sustainable. Therefore, designers must balance environmental sustainability with other important design characteristics such as novelty and quality to ensure that their designs are creative and environmentally sustainable (Greeley et al. Reference Greeley, Mastroly, Alsager Alzayed, Starkey, Ritter and Prabhu2025). These tradeoffs may also depend on the problem context and its requirements (Nickel et al. Reference Nickel, Hurst and Duimering2024). This need for balance is analogous to the findings by Onarheim (Reference Onarheim2012), who suggests that designers must find an optimal level of constrainedness in their design problems to achieve creative design outputs. Moreover, designers’ proximity to a subset of these issues, coupled with their psychological distance (Maiella et al. Reference Maiella, La Malva, Marchetti, Pomarico, Di Crosta, Palumbo, Cetara, Di Domenico and Verrocchio2020) – social and spatial (Spence et al. Reference Spence, Poortinga and Pidgeon2012) – could further impact designers’ ability to accurately evaluate their solutions, and these factors remain largely unexplored.
To overcome these limitations in designers’ ability to evaluate their solutions, several researchers have proposed design tools and methods for concept evaluation and selection. These tools range from domain-agnostic tools (e.g., the concept screening sheet (Toh & Miller Reference Toh and Miller2016a) and TASC (Gosnell & Miller Reference Gosnell and Miller2016) to domain-specific tools (e.g., DfAM (Booth et al. Reference Booth, Alperovich, Chawla, Ma, Reid and Ramani2017; Bracken et al. Reference Bracken, Pomorski, Armstrong, Prabhu, Simpson, Jablokow, Cleary and Meisel2020; Prabhu et al. Reference Prabhu, Miller, Simpson and Meisel2020). The domain-specific concept evaluation tools help designers employ domain knowledge more effectively in making concept evaluation and selection decisions, thereby supporting accurate concept evaluations. Similar to other domains, researchers have proposed concept evaluation tools for integrating sustainability into concept evaluation decisions. For example, Ruiz-Pastor et al. (Reference Ruiz-Pastor, Chulvi, Mulet and Royo2022) propose a consolidated metric for evaluating the circularity of products in addition to their novelty. Their proposed metric balances the creativity of solutions against sustainability-oriented considerations such as raw material usage and product end of life. In a similar effort, Maccioni et al. (Reference Maccioni, Borgianni and Pigosso2021) propose a set of 10 eco-design guidelines that can help designers generate and select solutions that are both creative and sustainable. A similar set of measures for concept evaluation is proposed by Blevis (Reference Blevis2007) – this set of metrics can be employed to evaluate the sustainability of both physical and digital products, an important consideration given the accelerating digitalization of products and services.
Despite the introduction of these objective concept evaluation tools, limited research has investigated whether novice designers can accurately evaluate the sustainability of their solutions using these tools. Such an investigation is important, as expertise and domain knowledge could play an important role in accurate concept evaluations. Additionally, a majority of research on the role of expertise in concept evaluation focuses on the evaluation of the creativity or quality of the solutions. Little research has explored the influence of prior experience on the accuracy of sustainability evaluations of engineering solutions, and our aim in this research is to explore this research gap. Before doing so, prior work on the influence of individual differences on concept evaluation is reviewed as discussed next.
2.2. Individual differences in engineering design and concept evaluation
Concept evaluation plays a critical role in the design process, given its role in determining the outcomes of engineering design; however, prior research suggests that designers’ evaluations of solutions might be influenced by numerous inherent biases and individual differences. For example, Toh & Miller (Reference Toh and Miller2016a) observe that teams with a higher tolerance for ambiguity also tend to select novel concepts. Similarly, personality traits have been found to relate to designers’ concept evaluation behavior. Specifically, designers with higher levels of agreeableness and conscientiousness also showed a tendency to select more novel ideas (Toh & Miller Reference Toh and Miller2016b).
Individual differences could be particularly important in the concept evaluation and selection stages since designers often evaluate their own solutions, either individually or in teams (Toh & Miller Reference Toh and Miller2016b), and either using their domain knowledge or with the help of knowledge-based tools (Amabile Reference Amabile1996). These self-evaluations could be prone to cognitive biases such as ownership bias (Toh et al. Reference Toh, Strohmetz and Miller2016; Zheng & Miller Reference Zheng and Miller2019, Reference Zheng and Miller2021), where individuals attribute a higher value to their own ideas compared to others’ ideas (Onarheim & Christensen Reference Onarheim and Christensen2012). Ownership bias could, in turn, lead designers to overestimate the merits of their own ideas. Similarly, researchers observe the presence of self-serving bias in concept evaluation (Gigliotti & Buchtel Reference Gigliotti and Buchtel1990), which results in individuals creating a more favorable image of themselves by providing a high score to their own ideas (Davis & Stephan Reference Davis and Stephan1980; Heine & Markus Reference Heine and Markus1999; Mezulis et al. Reference Mezulis, Abramson, Hyde and Hankin2004). Prior research in engineering design suggests that these biases are often prevalent in both the concept evaluation and selection stages (Onarheim & Christensen Reference Onarheim and Christensen2012; Nikander et al. Reference Nikander, Liikkanen and Laakso2014; Neroni & Crilly Reference Neroni and Crilly2019), but may also manifest in other stages of the design process, such as problem framing (Seshadri & Reid Reference Seshadri and Reid2015; Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2022).
Of the various individual differences explored in the context of concept selection, designers’ empathy has particularly received recent attention (Tang X, Reference Tang2018b). Prior research suggests that certain empathic tendencies positively influence designers’ concept generation and evaluation. For example, empathic concern – the tendency to feel compassionate and concerned for others (Davis Reference Davis1980 – was observed to positively influence the fluency of idea generation (Alsager Alzayed et al. Reference Alsager Alzayed, Miller and McComb2021b). Similarly, perspective-taking (i.e., the tendency to adopt the perspectives of others and see things from their point of view (Davis Reference Davis1980) and personal distress (i.e., feelings of fear and discomfort at witnessing the negative experiences of others (Davis Reference Davis1980) were found to positively impact designers’ tendencies to select elegant solutions (Alsager Alzayed et al. Reference Alsager Alzayed, Miller, Menold, Huff and McComb2020; Alsager Alzayed et al. Reference Alsager Alzayed, Miller and McComb2021b). However, some studies also find contrasting results, wherein empathy had a negative influence on concept evaluation outcomes. For example, Chung & Joo (Reference Chung and Joo2017) report that designers evaluated concepts less favorably when they empathized with the target user, compared to a control group that did not receive an empathic instruction task. This has been explained by research in cognitive psychology suggesting that individuals with high levels of personal distress and empathic concern make fewer prosocial decisions (Schaefer et al. Reference Schaefer, Kühnel, Rumpel and Gärtner2021). Researchers also make the argument for the presence of “dark sides” to empathy, known as empathic vampirism, where an individual’s attempt to be empathic could result in “over-identifying” with the end-user (Breithaupt Reference Breithaupt2018). Consequently, while empathy could help designers more effectively account for users’ needs in the design process, higher levels of empathy could also have negative consequences.
In addition to empathy, intrinsic factors toward a design problem (e.g., motivation, prior knowledge and affinity) have been found to relate to the novelty of solutions generated in eco-design tasks (Ruiz-Pastor et al. Reference Ruiz-Pastor, Chulvi, Mulet and Royo2021). Moreover, individuals’ beliefs, attitudes and intentions toward sustainability have been shown to predict pro-environmental behavior (Sawitri et al. Reference Sawitri, Hadiyanto and Hadi2015). In this context, “beliefs” captures individuals’ inherent beliefs about the need for and their ability to take sustainable actions, while “attitudes” captures individuals’ tendency to presently take sustainable actions, and finally, the component of “intentions” captures individuals’ tendencies to take sustainable actions in the future (Runhaar et al. Reference Runhaar, Wagenaar, Wesselink and Runhaar2019). In engineering design, prior research suggests that student designers’ intentions toward sustainable actions positively correlated with their identification of environment-focused requirements (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2021d). However, limited research has investigated the effects of these intrinsic factors on novice designers’ concept evaluation.
Taken together, empathy development could positively influence designers’ abilities to effectively select concepts, especially in foreign contexts (e.g., sustainability). On the other hand, an overemphasis on empathy could have detrimental results. This “spectrum” of empathy development could influence the accuracy of designers’ concept evaluations. However, little research has explored the influence of trait empathy on the accuracy of designers’ self-evaluations, particularly in sustainable design. Furthermore, designers’ intrinsic factors, such as beliefs, attitudes and intentions toward sustainability, could also influence the accuracy of concept evaluations, and limited research has explored this relationship. Such an understanding can help create more effective methods for sustainability-focused concept evaluation in engineering design. Additionally, this research can inform the development of educational interventions on sustainable design. Consequently, our aim in this research is to investigate the moderating effect of designers’ trait empathy and their beliefs, attitudes and intentions toward sustainability on the accuracy of their self-evaluations. Toward this aim, we seek answers to the RQs discussed next.
3. Research questions
Our aim in this study (see Figure 1) is to investigate the accuracy of novice designers’ self-evaluations of the sustainability of their solutions and the moderating role of their trait empathy and attitudes toward sustainability. Toward this aim, we seek answers to the following research questions (RQs).
-
- RQ1: How do student designers’ self-evaluated sustainability of solutions relate to expert evaluations?
-
- RQ2: Do components of student designers’ trait empathy moderate the accuracy of the self-evaluations of their solutions, and if so, how?
-
- RQ3: Do student designers’ beliefs, attitudes, and intentions toward sustainability moderate the accuracy of the self-evaluations of their solutions, and if so, how?
Overview of our research aim in this paper.
First, we hypothesize that participants’ accuracy in self-evaluating their solutions will differ across the 10 sustainable design heuristics. This hypothesis is based on prior research suggesting that individuals demonstrate varying levels of proximity to sustainability-related issues, which, in turn, could influence their behaviors toward these issues (Maiella et al. Reference Maiella, La Malva, Marchetti, Pomarico, Di Crosta, Palumbo, Cetara, Di Domenico and Verrocchio2020). Additionally, we hypothesize that trait empathy will moderate the accuracy of participants’ self-evaluations, and participants with higher levels of trait empathy will demonstrate greater accuracy. This hypothesis is based on prior research suggesting that designers with higher levels of trait empathy have a better understanding of the design problem (Walther et al. Reference Walther, Miller and Kellam2012) and end-users’ needs (Strobel et al. Reference Strobel, Hess, Pan and Wachter Morris2013; Alsager Alzayed et al. Reference Alsager Alzayed, McComb, Menold, Huff and Miller2021a; Prabhu et al. Reference Prabhu, Starkey and Alsager Alzayed2021c) and, therefore, can provide more accurate evaluations. Finally, we hypothesize that participants’ beliefs, attitudes and intentions toward sustainability will moderate the accuracy of their self-evaluations. Prior research suggests that these intrinsic factors are a significant predictor of their tendencies to take pro-environmental action (Kollmuss & Agyeman Reference Kollmuss and Agyeman2002; Runhaar et al. Reference Runhaar, Wagenaar, Wesselink and Runhaar2019; Ruiz-Pastor et al. Reference Ruiz-Pastor, Chulvi, Mulet and Royo2021). Therefore, participants with positive attitudes might be likely to actively take sustainability into account when evaluating their solutions, leading to more accurate sustainability-focused evaluations. It should be noted that we used moderation analyses because we were interested in examining whether the relationship between the self-evaluation and expert evaluation (i.e., the accuracy of self-evaluations) changed based on the moderator variable – designers’ individual differences (i.e., trait empathy and attitudes toward sustainability) (Dawson Reference Dawson2014).
4. Experimental methods
To answer these RQs, we conducted an experiment in the form of a workshop. The experiment was reviewed and approved by the Institutional Review Board before it was conducted. The details of the experiment are discussed next.
4.1. Participants
The participants were recruited from a first-year course on engineering design at a large public university in the northeastern United States. The experiment was conducted in both the fall (Nf) and spring (Ns) offerings of the course. Of the 85 consenting participants (Nf = 40 and Ns = 45), 76 were in their first year of study (Nf = 34 and Ns = 42), five students were in the second year (Nf = 2 and Ns = 3) and one student in the third and fourth year each (Nf = 1 and 1 each). Additionally, 59 participants self-identified as male (Nf = 25 and Ns = 34), and 24 participants self-identified as female (Nf = 13 and Ns = 11). Some participants did not report their gender and/or year of study.
4.2. Procedure
The experiment consisted of four main components: (1) a sustainable design lecture, (2) a design activity, (3) the self-evaluation of the final design from the design activity and (4) an individual differences survey. It should be noted that one group (Nf = 22 and Ns = 22) received the sustainable design lecture before the design activity, whereas a second group (Nf = 18 and Ns = 23) received the lecture after the design activity. The overall experimental procedure is presented in Figure 2, and each component is discussed in detail next.
Overview of our experimental procedure.
The first component of the experiment was a lecture on sustainable design. In this lecture, participants were introduced to the concepts of lifecycle assessment, the “cradle to grave” journey of products and sustainable design heuristics. Specifically, students were introduced to the 10 heuristics proposed by Blevis (Reference Blevis2007): (1) disposal, (2) salvage, (3) recycle, (4) remanufacture for reuse, (5) reuse as is, (6) longevity, (7) sharing for maximal use, (8) achieving heirloom status, (9) finding wholesome alternatives and (10) active repair of misuse. The use of these heuristics was introduced for both digital and physical products. Finally, participants were introduced to the United Nations’ seventeen Sustainable Development Goals (SDGs)Footnote 1 with a special emphasis on SDG #6 (i.e., clean water and sanitation) given its relevance to the design activity. One half of the participants were given the sustainable design lecture before the design activity, whereas the second half received the lecture after the design activity. This manipulation was part of a larger study aimed at studying the impact of the timing of sustainable design interventions (Prabhu et al., Reference Prabhu, Simpson, Miller and Meisel2021b; Prabhu et al., Reference Prabhu, Alsager Alzayed and Starkey2021f). Our primary aim in this research is to test the accuracy of the self-evaluations of the solutions generated by participants, and since all participants received the lecture before evaluating their solutions, the effects of this manipulation are not discussed in this paper. Furthermore, the results concerning the effects of the order of the workshop on student designers’ self-evaluations are presented in (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2023). In that paper, we demonstrate that while the order of the workshop may influence the designers’ self-evaluations on some sustainable design heuristics, these effects are sparse and weak. Therefore, we did not control for the order of the workshop in our analyses in this paper.
The second component of the experiment was a design activity. The design activity was conducted in four steps. First, the participants were introduced to the design problem and its context. The design problem and persona developed by Prabhu et al. (Reference Prabhu, Alsager Alzayed and Starkey2021d) were provided to the students, and any questions were answered. In this design task – contextualized in SDG #6 – participants are asked to generate solutions to provide access to clean water and sanitation to a family in Sub-Saharan Africa. Participants are also provided with some background information, such as access to nearby natural water sources and a persona, Eli and his family, with the target users’ occupation as part of the prompt. The complete prompt is available in (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2021d) and in Appendix A. Next, the participants were asked to identify problem requirements and prioritize them using the Analytical Hierarchy Process (AHP) Chart. An example of an AHP produced by a participant is presented in Figure 3. Using these problem requirements, they were then asked to ideate solutions. Participants were then asked to compare the best five solutions using a concept screening matrix and to evaluate the ideas based on their identified problem requirements. Based on this comparison, they were asked to select one final design to solve the problem.
Example of an AHP generated by a participant to prioritize customer needs.
After selecting their final design, the participants were asked to self-evaluate the sustainability of their final design using the ten sustainable design heuristics proposed by Blevis (Reference Blevis2007). Upon completing the self-evaluations of their final design, they were asked to complete a survey collecting their trait empathy and their attitudes toward sustainability. The details of the metrics and survey instruments used are discussed next.
4.3. Metrics
Three metrics were employed in this study: (1) an individual differences survey comprising trait empathy and beliefs, attitudes, and intentions toward sustainability, (2) participants’ self-evaluation of the sustainability of their solutions and (3) external evaluations of the sustainability of participants’ solutions. Each metric is discussed next.
4.3.1. Individual differences survey
At the end of the workshop, participants’ trait empathy and their attitudes toward sustainability were collected using an individual differences survey. The details of each survey scale are discussed next.
-
1. Interpersonal Reactivity Index (IRI): Surma-aho & Hölttä-Otto (Reference Surma-aho and Hölttä-Otto2022) propose a framework for conceptualizing and operationalizing empathy in design research. Grounded in this framework, we aimed to measure designers’ empathic orientation, specifically, their ability to understand others’ needs and evaluate solutions based on these needs. Therefore, we employed the Interpersonal Reactivity Index (IRI) (Davis Reference Davis1983) to measure designers’ trait empathy, which operationalizes empathy as designers’ empathic tendencies. Moreover, the IRI – frequently used in engineering design research (Hess et al. Reference Hess, Fila and Purzer2016; Surma-aho et al. Reference Surma-aho, Björklund and Holtta-Otto2018) – captures both the cognitive and affective components of empathy (Duan & Hill Reference Duan and Hill1996), both of which help designers understand users’ needs (Hess et al. Reference Hess, Fila and Purzer2016). The IRI is a 28-item scale comprising four subscales: (1) perspective taking, (2) fantasy, (3) empathic concern and (4) personal distress. Specifically, perspective-taking captures the tendency to adopt the perspectives of others and see things from their point of view (Davis Reference Davis1980, fantasy captures individuals’ tendency to transpose themselves into the lives of fictitious characters, empathic concern captures the tendency to feel compassionate and concerned for others (Davis Reference Davis1980 and personal distress captures feelings of fear and discomfort at witnessing the negative experiences of others (Davis Reference Davis1980. We asked participants to respond to the 28 survey items on a 5-point Likert scale where 1 = “Strongly Disagree” and 5 = “Strongly Agree.” The reliability of the responses was established through Cronbach’s α (Cronbach Reference Cronbach1951) > 0.7 for each subscale.
-
2. Beliefs, attitudes and intentions toward sustainability: “Beliefs” refer to one’s inherent beliefs about the need for and their ability to take certain actions (Runhaar et al. Reference Runhaar, Wagenaar, Wesselink and Runhaar2019), and one’s beliefs have been shown to influence their tendency to actively perform said actions (Hornsey et al. Reference Hornsey, Harris, Bain and Fielding2016). Similarly, “attitudes” represent one’s tendency to engage in certain actions (Eagly & Chaiken Reference Eagly and Chaiken1993), and “intentions” represent one’s self-directives to engage in certain actions in the future (Gollwitzer & Sheeran Reference Gollwitzer and Sheeran2006). Both attitudes and intentions have been shown to correlate with one’s behavior (Rhodes & Dickau Reference Rhodes and Dickau2012; Sheeran & Webb Reference Sheeran and Webb2016). The 25-item survey proposed by K. H. D. Tang (Reference Tang2018a) was used to measure participants’ attitudes toward sustainable action. The survey consists of three components: (1) beliefs (6 items), (2) attitudes (13 items) and (3) intentions (6 items). The component of “beliefs” captures individuals’ inherent beliefs about the need for and their ability to take sustainable actions through items such as “I feel morally obliged to do something about environmental problems.” The component of ‘attitudes’ captures individuals’ tendency to presently take sustainable actions and consists of items such as “I make an effort to use energy and resources efficiently.” Finally, the component of ‘intentions’ captures individuals’ tendencies to take sustainable actions in the future and comprises items such as “I intend to change/continue to change my lifestyle for better sustainability.” The survey asked participants to respond to the items on a 5-point Likert scale (1 = “Strongly Disagree” to 5 = “Strongly Agree”). The reliability of the survey responses was established through an observed Cronbach’s α > 0.7 within each component.
4.3.2. Self- and expert-evaluation of design sustainability
Upon completing the design activity, participants were asked to evaluate the sustainability of their final design with the 10 sustainable design heuristics proposed by Blevis (Reference Blevis2007). They were asked to score their final design on a scale of one to five, where 1 = “Very Poor” and 5 = “Very Good.” Additionally, the participants’ final designs were evaluated by external evaluators, and these scores were used as the “expert evaluation.” First, 20 randomly selected designs were evaluated by two evaluators: (1) an associate teaching professor of engineering design with numerous years of prior experience in sustainability and humanitarian engineering and (2) an assistant professor of mechanical engineering. This independent evaluation of a subset of designs was used to train the second evaluator with the mental model of the first evaluator, who had more experience with sustainability-focused design. This approach to training quasi-experts has been demonstrated to be effective in prior research (e.g., see work by Kaufman & Baer (Reference Kaufman and Baer2012). The two raters used the same 10 sustainable design heuristics as used by the participants for evaluating the solutions and rated the solutions on the same 5-point scale. Upon observing sufficient inter-rater reliability (Koo & Li Reference Koo and Li2016) (intraclass correlation coefficient > 0.7), one of the two evaluators (i.e., the assistant professor of mechanical engineering and corresponding author of the paper) evaluated the remaining solutions. The scores from the one evaluator who evaluated all solutions (vs the mean of the two evaluators) were used for the analyses to maintain the scores on the same 5-point scale.
5. Data analysis and results
The data collected were analyzed using statistical methods. We computed linear regression models for the first RQ and linear regression-based moderation analyses (Hayes Reference Hayes2022) for the second and third RQs. Of the total 85 consenting participants, data from 67 participants were used in the analyses, either due to missing survey responses or unreadable design sketches. These missing data (primarily at random (Scheffer Reference Scheffer2002) could be attributed to two reasons. First, the experiment comprised several stages, which could have resulted in participant attrition between the various stages (Marcellus Reference Marcellus2004). Second, the experiment was conducted virtually, given the ongoing COVID-19 pandemic and the corresponding restrictions imposed. This loss in sample size is a potential limitation of the study and presents an opportunity for further exploration into the role of participant retention and attrition in human–subjects design research (Siddiqi et al. Reference Siddiqi, Sikorskii, Given and Given2008). The statistical tests used to answer the various RQs and the corresponding results are presented in the remainder of this section.
Following existing conventions, we used a threshold of p < 0.05 to indicate statistical significance. Additionally, given the exploratory nature of our study, we used the effect size benchmarks proposed by Funder & Ozer (Reference Funder and Ozer2019) as a starting point for interpreting the correlation coefficients. Specifically, they suggest that correlations in the range of r = 0.2 indicate medium-sized effects and are of importance in the short run. Similarly, they suggest that correlations in the range of r = 0.3 indicate large effect sizes that may be of importance in the short and long run. However, we acknowledge that the effects observed in our study are exploratory, and further replications with larger and more diverse samples can further reinforce the strength of these effects. Moreover, these effects may be a characteristic of our data, and replication efforts will establish the generalizability of these effects. Before performing the statistical analyses, we examined the normality of the self-and expert evaluations using the omnibus test by D’Agostino & Pearson (Reference D’Agostino and Pearson1973). From the results, we see that the self-evaluations for 7 out of 10 sustainable design heuristics were normally distributed. Similarly, expert evaluations for 5 out of 10 heuristics were normally distributed. Given these patterns and the robustness of linear models to violations of the normality assumptions in relatively large samples (Schmidt & Finan Reference Schmidt and Finan2018), we chose to use linear regression models in our analyses.
5.1. RQ1: How do student designers’ self-evaluated sustainability of solutions relate to expert evaluations?
We hypothesized that the accuracy of participants’ self-evaluations would vary across the 10 sustainable design heuristics. To test this hypothesis, we computed a series of linear regressions with the participants’ self-evaluations on the 10 sustainable design heuristics as the dependent variable and the expert evaluations on the corresponding heuristic as the independent variable. From the results, summarized in Table 1, we see that expert evaluations of the solutions significantly correlated with the participants’ self-evaluations of their solutions on the dimensions of (1) longevity and (2) finding wholesome alternatives. No significant relationships were observed in the other eight dimensions of sustainable design. Additionally, we see that the participants’ self-evaluations tended to be lower than the expert evaluations (standardized correlation coefficients = 0.30). This finding partially supports our hypothesis, and the implications of these findings are discussed in Section 6.
Model statistics and parameter estimates correlating self-evaluations to expert evaluations.
Note: Results of statistical significance are highlighted with **, indicating p < 0.05.
5.2. RQ2: Do components of student designers’ trait empathy moderate the accuracy of the self-evaluations of their solutions, and if so, how?
We hypothesized that participants’ trait empathy would moderate the accuracy of their self-evaluations and that participants with higher levels of trait empathy would demonstrate greater accuracy. As a reminder, a moderation analysis is used to examine whether the relationship between two variables – in our case, the self- and expert evaluations – depends on a moderator variable – in this case, the four subcomponents of trait empathy (Dawson Reference Dawson2014). To test this hypothesis, we conducted a series of regression-based moderation analyses using the Process Macro (version 4.0) (Hayes Reference Hayes2022). The participants’ self-evaluations on each sustainable design heuristic were used as the dependent (Y) variable, and the expert evaluations on the corresponding heuristic were used as the independent (X) variable. Participants’ scores for each component of trait empathy (Perspective Taking, Fantasy, Empathic Concern, and Personal Distress) were used as the moderator (W) variable. Additionally, given the small sample size, bootstrapping was performed with 5000 bootstrap samples.
From the results, summarized in Table 2, we see that some components of participants’ trait empathy moderate the accuracy of their self-evaluations compared to expert evaluations. This moderation effect was observed in participants’ evaluation of their solutions on the following heuristics: (1) sharing for maximal use and (2) finding wholesome alternatives. For these models with significant interaction effects between the independent and moderator variables, a post-hoc analysis was performed to compare differences in effect sizes at low, medium and high levels of the moderator variable (see Table 3).
Model statistics and parameter estimates testing the moderating effects of trait empathy.
Note: β 1 = parameter estimate for expert evaluation, β 2 = parameter estimate for component of trait empathy, β int = parameter estimate for the interaction term.
Bold and highlight indicate significant interaction effect, with further moderation effect analyses presented in Table 3.
Results for post-hoc testing of moderation effects for models with significant interaction effects.
Note: Bold indicates statistically significant effects at p < 0.05.
The post-hoc analyses revealed that participants with low scores on perspective-taking demonstrated a greater accuracy of self-evaluation on the sustainable design heuristic of sharing for maximal use (effect size = 0.31). Similarly, participants with low and medium scores on fantasy demonstrated a greater accuracy of self-evaluation on the sustainable design heuristic of finding wholesome alternatives (effect sizes = 0.48 and 0.24, respectively). Taken together, these results partially support our hypothesis by demonstrating that the accuracy of designers’ self-evaluations may depend on the components of their trait empathy, and the implications of these results are discussed in Section 6.
5.3. RQ3: Do student designers’ beliefs, attitudes and intentions toward sustainability moderate the accuracy of the self-evaluations of their solutions, and if so, how?
We hypothesized that participants’ beliefs, attitudes and intentions toward sustainability would moderate the accuracy of their self-evaluations. As a reminder, a moderation analysis is used to examine whether the relationship between two variables – in our case, the self- and expert evaluations – depends on a moderator variable – in this case, the designers’ beliefs, attitudes and intentions toward sustainability (Dawson Reference Dawson2014). To test this hypothesis, we conducted a series of regression-based moderation analyses using the Process Macro (version 4.0) (Hayes Reference Hayes2022). The participants’ self-evaluations on each sustainable design heuristic were used as the dependent (Y) variable, and the expert evaluations on the corresponding heuristic were used as the independent (X) variable. Participants’ scores for their beliefs, attitudes and intentions toward sustainability were used as the moderator (W) variable. Additionally, given the small sample size, bootstrapping was performed with 5000 bootstrap samples.
From the results, summarized in Table 4, we see that components of participants’ beliefs, attitudes and intentions toward sustainability moderated the accuracy of their self-evaluations on certain sustainable design heuristics. Specifically, we see significant moderation effects on the accuracy of evaluations with the sustainable design heuristics of (1) remanufacture for reuse, (2) longevity and (3) active repair of misuse. For these models with significant interaction effects between the independent and moderator variables, a post-hoc analysis was performed to compare differences in effect sizes at low, medium and high levels of the moderator variable (see Table 5).
Model statistics and parameter estimates testing the moderating effects of attitudes toward sustainability.
Note: β 1 = parameter estimate for expert evaluation, β 2 = parameter estimate for component of attitudes toward sustainability, β int = parameter estimate for the interaction term.
Bold and highlight indicate significant interaction effects, with further moderation effect analyses presented in Table 5.
Results for post-hoc testing of moderation effects for models with significant interaction effects.
Note: Bold indicates statistically significant effects at p < 0.05.
The post-hoc analyses revealed that participants with highly positive attitudes toward sustainability reported greater accuracy in their evaluation of the heuristic “active repair of misuse” (effect size = 0.47). A contrasting result is observed for the sustainable design heuristic of “longevity.” For this heuristic, participants with lower levels of beliefs and intentions show greater accuracy in their self-evaluations (effects = 0.62 and = 0.57, respectively) compared to those with moderate and low levels of beliefs and intentions. Taken together, these results partially support our hypothesis that participants’ beliefs, attitudes and intentions toward sustainability would moderate the accuracy of their self-evaluations. That is, the relationship between designers’ self-evaluations of their designs and expert evaluations varies based on designers’ beliefs, attitudes and intentions toward sustainability. The implications of these results are discussed in Section 6.
6. Discussions and implications
Our aim in this research is to investigate the accuracy of novice designers’ self-evaluations of the sustainability of their solutions when compared against expert evaluations. We also aim to investigate the moderating effects of designers’ trait empathy and their beliefs, attitudes and intentions toward sustainability on the accuracy of their evaluations. Toward this aim, we conducted an experiment with novice designers, and from the results, we see that components of trait empathy and participants’ beliefs, attitudes and intentions toward sustainability, moderated the accuracy of their self-evaluations. The implications of these findings are discussed next.
6.1. Participants’ self-evaluations positively correlated with expert evaluations on some sustainable design heuristics.
The first key finding is that participants’ self-evaluations positively correlated with expert evaluations on the dimensions of (1) longevity and (2) finding wholesome alternatives. A positive correlation suggests that participants’ self-evaluations for these four dimensions were consistent with the expert evaluations. That is, solutions that were scored higher by the expert were also scored higher by the participants and vice versa. This finding suggests that participants might be able to evaluate solutions with some level of consistency but only on some sustainable design heuristics. This finding could be attributed to participants’ greater familiarity with and understanding of a smaller subset of design heuristics compared to others. Prior research suggests that greater familiarity with a partial subset of information could inhibit the learning and recall of the remaining set of related information (Prabhu et al. Reference Prabhu, Simpson, Miller and Meisel2021g). Therefore, designers must be encouraged to think about sustainable design more holistically, particularly emphasizing concepts that might be relatively unknown to them. This inference is particularly reinforced since all participants were introduced to all 10 heuristics in the sustainable design lecture before they evaluated their solutions.
Second, the standardized regression coefficients for these significant relationships were 0.30 each. That is, although consistent, participants’ self-evaluations were lower than half of the expert’s scores. This result suggests that participants might be underconfident in the sustainability of their own solutions. This lack of confidence could, in turn, lead to inaccurate outcomes when evaluating and selecting solutions in the design process. Specifically, designers might demonstrate a tendency to discard solutions of high potential due to their lower levels of perceived sustainability – an observation also made in other contexts such as additive manufacturing (Prabhu et al. Reference Prabhu, Leguarda, Miller, Simpson and Meisel2021a). This finding is particularly important as it suggests that novice designers might not always be able to make accurate decisions when designing for sustainability. This finding calls for further research into design tools and methods that support novice designers to make effective design decisions in the context of sustainable design.
6.2. Components of trait empathy moderate the accuracy of self-evaluations on certain sustainable design heuristics
The second key finding from the results is that components of trait empathy moderate the accuracy of participants’ self-evaluations of the sustainability of their solutions. Specifically, significant moderation effects of trait empathy were observed in the accuracy of the evaluations for the sustainable design heuristics of (1) sharing for maximal use and (2) finding wholesome alternatives. Given that both of these sustainable design heuristics are strongly tied to the context of the design problem, this finding highlights the role of trait empathy in designers’ ability to accurately relate to new and potentially unfamiliar design contexts. For example, the design prompt asked participants to think about providing access to an individual and their family, all of whom are situated in a water-scarce village. For a solution to effectively enable the sharing of resources, it must account for several social norms, both within the family and beyond (e.g., within the village). The participants in our study might have limited prior experience, direct or indirect, working with these social norms. Therefore, they might have to overcome a social and spatial psychological distance (Brügger Reference Brügger2020) to accurately determine whether their proposed solution can be effectively shared for maximal use. The need to overcome psychological distance may be particularly emphasized in the context of environmental sustainability, as these issues are often perceived to be at a more abstract construal level, particularly by individuals in developed parts of the world (Lorenzoni et al. Reference Lorenzoni, Leiserowitz, De Franca Doria, Poortinga and Pidgeon2006; Spence et al. Reference Spence, Poortinga and Pidgeon2012), especially from a temporal lens (Prabhu et al. Reference Prabhu, Herzog, Fodale, Alsager Alzayed and Starkey2025). This ability to overcome psychological distance might be influenced by their ability to take the perspective of the user, a component of trait empathy. A similar argument could be made about the sustainable design heuristic of finding wholesome alternatives: to identify appropriate wholesome alternatives (e.g., energy sources), it is important to understand the geography of the region where the target user is situated. As suggested by prior research, designers’ trait empathy could influence their ability to overcome psychological distance, both social and spatial (Chu & Yang Reference Chu and Yang2019).
A further investigation into the moderation effects revealed two main findings. First, we see that participants with low scores on perspective-taking show a greater accuracy with the self-evaluations of their solutions’ ability to enable sharing for maximal use. Individuals’ perspective-taking is a measure of their ability to think about situations from others’ perspectives (Davis Reference Davis1980. Prior research suggests that in the context of human-centered design, higher levels of perspective-taking could help designers better understand the users’ needs (Hess et al. Reference Hess, Strobel and Brightman2017; Alsager Alzayed et al. Reference Alsager Alzayed, McComb, Menold, Huff and Miller2021a). While this ability to understand the needs of the primary user could help better meet the users’ needs, this could lead to designers fixating on the primary user and potentially ignoring the broader impacts of their solutions. For example, in the current study, the participants could have fixated on evaluating whether their solution provides access to clean water to the primary users – i.e., Eli and his family – without considering the needs of others in the village. This focus on the primary user could have resulted in participants with high levels of perspective-taking making inaccurate evaluations about their solutions’ ability to facilitate sharing for maximal use. This finding suggests that although perspective-taking could help novice designers better evaluate their solutions’ ability to meet the needs of their primary users, it could result in an inaccurate evaluation of the broader impacts of their solutions.
Second, we see that participants with low to medium levels of fantasy showed more accurate self-evaluations on the sustainable design heuristic of finding wholesome alternatives. The component of fantasy captures one’s ability to position oneself in the perspective of fictional characters and situations (Davis Reference Davis1980. Therefore, designers with higher scores on fantasy could be more removed from reality. As a result, they might make assumptions about their solutions that might not be grounded in reality, in turn, leading to inaccurate evaluations. For example, in the context of this experiment, novice designers with higher scores on fantasy could make assumptions about the types of wholesome alternatives that might be available in the region (e.g., solar energy), which might not necessarily be valid. These assumptions could, in turn, manifest as inaccuracies in the self-evaluations of their solutions, especially for their use of locally available, wholesome alternatives. One approach to productively leverage designers’ fantasy could be through the framing of the design problem. In a follow-up study, we find that certain dimensions of environmental sustainability may be positively correlated with design usefulness, and this relationship may be related to the framing of the design problem (Greeley et al. Reference Greeley, Mastroly, Alsager Alzayed, Starkey, Ritter and Prabhu2025). Specifically, we find that designs that more effectively addressed problem requirements (e.g., providing clean access to water for a village) also leveraged related sustainable design techniques (e.g., maximizing utility and accessibility). Therefore, by appropriately framing the information provided in a design problem, designers and educators may be able to more productively leverage designers’ empathy by either minimizing the assumptions made about user needs or by providing opportunities to verify the accuracy of these assumptions. Such framing of design problems may also require designers and educators to balance designers’ empathic response and their psychological distance from the problem context (Prabhu et al. Reference Prabhu, Herzog, Fodale, Alsager Alzayed and Starkey2025).
These results support prior work that shows that not all components of empathy might lead to prosocial behavior. For example, Schaefer et al. (Reference Schaefer, Kühnel, Rumpel and Gärtner2021) observe that individuals with high levels of personal distress and empathic concern make fewer prosocial decisions, further suggesting the presence of “dark sides” to empathy (Breithaupt Reference Breithaupt2018). In engineering design, empathy could have negative impacts on both designers’ concept generation and selection processes. For example, previous research reported that high team empathy negatively impacted teams’ propensity for selecting elegant and useful ideas (Alsager Alzayed et al. Reference Alsager Alzayed, Miller, Menold, Huff and McComb2020). Similarly, engagement in an empathic instruction task negatively influenced designers’ concept evaluation scores (Chung & Joo Reference Chung and Joo2017). In the context of sustainable design, especially when designing for socio-spatially distant users, higher levels of fantasy and perspective-taking could lead to inaccurate evaluations of the sustainability of solutions, especially on dimensions that might depend on the problem context and setting.
6.3. Beliefs, attitudes, and intentions toward sustainability moderate the accuracy of self-evaluations on some sustainable design heuristics
The final finding from the results is that participants’ beliefs, attitudes and intentions toward environmental sustainability moderate the accuracy of their self-evaluation on the dimensions of (1) longevity and (2) active repair of misuse. Two trends emerge from the results. First, we see that participants with low levels of beliefs and intentions toward sustainability showed greater accuracy in their self-evaluations for the heuristic of longevity. This finding suggests that participants with high levels of beliefs and intentions are more likely to be conservative in their self-evaluations of the longevity of their solutions. Our prior findings suggest that students who received an educational lecture in sustainable design before completing a design task report a greater increase in their beliefs and intentions toward sustainable action (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2023). Therefore, there may be a positive relationship between one’s beliefs/intentions toward sustainability and their knowledge about sustainable design. This greater awareness and familiarity with sustainable design could make designers more critical of their designs and provide lower self-evaluations. On the other hand, this tendency to possibly underestimate their own solutions could, in turn, result in designers discarding solutions of potentially high sustainability due to the perceived lower levels of sustainability. This finding also suggests that designers with high levels of beliefs and intentions might have higher expectations of themselves, and therefore are stricter with their self-evaluations. However, it is also important to note that our findings do not necessarily demonstrate the source of participants’ higher (or lower) beliefs and intentions, and whether these higher levels are associated with the new knowledge gained. Therefore, future research must uncover the factors that shape student designers’ beliefs, attitudes and intentions toward environmental sustainability.
A contrasting result is observed in the case of participants’ self-evaluations for the heuristic of “active repair for misuse.” For this heuristic, we see that participants with high levels of attitudes toward sustainability demonstrate more accurate self-evaluations. Prior research suggests that individuals with higher levels of pro-environmental attitudes and intentions are more likely to act pro-environmentally (Kollmuss & Agyeman Reference Kollmuss and Agyeman2002). Therefore, participants with more positive attitudes toward sustainability may be more likely to explore environmental sustainability in their personal and professional lives, leading to a better awareness of these concepts. This greater awareness of environmental sustainability could, in turn, result in more accurate self-evaluations, especially when considering uncommon and context-dependent sustainable design concepts such as “active repair of misuse.”
Taken together, these findings suggest that designers’ intrinsic perceptions of environmental sustainability (i.e., their beliefs, attitudes and intentions) could influence their ability to make effective design decisions. Moreover, these effects could manifest differently for different sustainable design concepts, based on novice designers’ knowledge of these concepts. Therefore, design education must work toward bringing about a positive effect on novice designers’ beliefs, attitudes and intentions toward sustainability. Our prior findings show that the sequence of the workshop used in our study neither influenced the students’ problem-framing (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2022) nor their concept evaluation (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2023). This lack of effects may be attributed to the inability of such an intervention to bridge psychological distance, particularly since concepts related to environmental sustainability may be at a more distant construal level, especially from a temporal lens (Prabhu et al. Reference Prabhu, Herzog, Fodale, Alsager Alzayed and Starkey2025). Therefore, researchers must explore the effectiveness of empathy-invoking educational approaches such as reflection (Brown & Prabhu Reference Brown and Prabhu2025) and perspective-taking (Liberman & Trope Reference Liberman and Trope2014; Raviselvam et al. Reference Raviselvam, Hölttä-Otto and Wood2016) to bridge students’ psychological distance toward environmental sustainability. Moreover, educators must also develop a sense of novice designers’ awareness of the various sustainable design concepts and make targeted efforts to emphasize concepts with which designers might have lesser familiarity. Such an integrated effort could result in novice designers learning to effectively take sustainability into account when making design decisions while being accurate in their decisions.
7. Concluding remarks
Our aim in this research was to investigate the accuracy of novice designers’ self-evaluations of the sustainability of their solutions and the moderating effects of trait empathy and beliefs, attitudes and intentions toward sustainability on this accuracy. From the results of our experiment, we see that participants demonstrated some degree of accuracy, but only with the sustainable design heuristics of longevity and finding wholesome alternatives. Additionally, we see that components of trait empathy moderate the accuracy of participants’ self-evaluations, with participants reporting lower levels of fantasy and empathic concern demonstrating more accurate self-evaluations on certain sustainable design heuristics (i.e., sharing for maximal use and finding wholesome alternatives). Moreover, participants with lower levels of beliefs and intentions toward sustainability also reported more accurate self-evaluations on some sustainable design heuristics (i.e., longevity and sharing for maximal use). Taken together, these findings suggest that efforts toward developing empathy and positive beliefs, attitudes, and intentions toward sustainability among designers might also result in inaccurate self-evaluations of their solutions. Therefore, educators must work toward both positively influencing student designers’ empathic reactions and intrinsic factors toward sustainability, while painting a realistic picture of the impact of their actions using external design tools and sustainable design methods. While our goal as educators is to drive future designers to design sustainable solutions, we must also be aware of how our efforts impact them as designers and decision-makers. Prior work on creativity suggests that designers who can effectively generate ideas might not always make effective concept selection decisions (Toh & Miller Reference Toh and Miller2016a), requiring educators to work around competing efforts in sustainability education. Future work must investigate how training in sustainable design impacts the relationship between designers’ interpersonal and internal traits and their concept evaluation behaviors.
8. Limitations and directions for future work
The findings from our study provide valuable insights into the influence of individual differences on novice designers’ self-evaluation of their solutions; however, our study has some limitations that present directions for future work. First, the participants in our study were mostly in their first year of engineering education. Prior research suggests that ideation patterns and design decision-making are influenced by educational level (Atman et al. Reference Atman, Chimka, Bursic and Nachtmann1999, Reference Atman, Cardella, Turns and Adams2005; Alsager Alzayed et al. Reference Alsager Alzayed, McComb, Hunter and Miller2019; Prabhu et al. Reference Prabhu, Simpson, Miller and Meisel2021b) and domain expertise (Dixon & Bucknor Reference Dixon and Bucknor2019). Therefore, future work must extend these findings to designers with greater levels of experience (e.g., upper-division and graduate students, and industry practitioners). Second, in this study, we focused on designers’ trait empathy and their beliefs, attitudes, and intentions toward sustainability. Future research should extend these findings to other individual differences, such as risk-taking (Zheng & Miller Reference Zheng and Miller2017) and personality traits (Toh & Miller Reference Toh and Miller2016b). Such an investigation could support a comprehensive understanding of the role of individual differences in concept evaluation. Third, following feedback received during the review and editorial process, we used the de facto standard threshold of p < 0.05 to indicate results of statistical significance. Some relationships observed in our results had relatively large effect sizes but had p-values marginally higher than this threshold (e.g., an effect size = 0.51 with p = 0.06 seen in Table 5). Researchers in metascience have posited that this threshold for p-values may be arbitrary and sometimes obscure effects of practical significance (Hubbard & Lindsay Reference Hubbard and Lindsay2008; Mitchell et al. Reference Mitchell, Yu and Whiteside2010; Greenland et al. Reference Greenland, Senn, Rothman, Carlin, Poole, Goodman and Altman2016; Amrhein et al. Reference Amrhein, Korner-Nievergelt and Roth2017). Therefore, future research must replicate our findings with a larger, more diverse sample to test their generalizability and uncover any relationships of practical significance that may have been missed in the interpretation of our results. Finally, participants only evaluated their final design for sustainability, therefore limiting our analysis to one evaluation per participant. Future work should extend these findings to understand the relationship between concept evaluation and concept selection. Furthermore, the effects of individual differences may manifest at different stages of the design process beyond concept evaluation and selection. For example, in a follow-up study, we found that novice designers with positive attitudes toward sustainable action also generated more problem requirements that focused on environmental sustainability (Prabhu et al. Reference Prabhu, Alsager Alzayed and Starkey2022). Therefore, future research must explore the effects of cognitive biases related to individual differences on designers’ emphasis on environmental sustainability in different stages of the design process.
Acknowledgments
An earlier, preliminary version of this work was published in the proceedings of the 2022 Design Automation Conference at the ASME International Design Engineering Technical Conferences (Alzayed et al. Reference Alzayed, Starkey, Ritter and Prabhu2022).
Appendix A: Design prompt
In Sub-Saharan Africa, nearly 46 people die per 100,000 people due to diseases caused by the lack of safe water, sanitation and hygiene (WASH) services. This is nearly four times the global average of 12 deaths per 100,000 people due to poor access to WASH services. You are tasked with designing a solution to help improve access to clean water and sanitation to Eli and others in his village.
Eli is a 40-year-old man who lives in the Sub-Saharan African region. He lives with his wife and two teenage children. He is a farmer by profession – a low-income profession – and has received some middle-school level education. Eli lives in a small remote village with some access to electricity but no access to other technological resources (e.g., internet and cellular service). The electricity is primarily used to operate water pumps that source water from either (1) a nearby polluted river or (2) contaminated and ill-maintained wells in and around the village. Since these are the only two sources of water for Eli and his family, they are highly prone to water-borne diseases.
With this persona as a starting point, you are tasked with designing a solution to help improve access to clean water and sanitation to Eli and others in his village.
Open access
