1. Introduction
Humans are an integral part of engineering design, ranging from designers who make various design decisions to users who engage with the products, with various stakeholders in between. Individuals exhibit various dimensions of social identity (e.g., race, gender and socioeconomic status), and these dimensions are influenced by personal experiences, self and external perceptions, and social structures among other factors (Ellemers, Spears & Doosje Reference Ellemers, Spears and Doosje2002). Moreover, the dimensions of social identity are often dynamic – that is, they interact with each other and with the situation’s context to influence behavior (Cikara, Martinez & Lewis Reference Cikara, Martinez and Lewis2022). It is crucial to consider social identity in engineering design because it could influence various facets of the design process, ranging from collaboration in teams, design evaluation preferences and product use (Kilker Reference Kilker1999) (Table 1).
Table 1. Glossary of key terms used in our paper and their corresponding definitions in the context of our study
Users’ social identity could play an important role in human-centered design, and in some contexts, the implications of not accounting for users’ social identity could be severe. A recent example was witnessed during the COVID-19 pandemic. Pulse oximeters are commonly used as a noninvasive method to measure blood oxygen saturation, and low blood oxygen is used as a preliminary diagnosis for COVID-19 (Luks & Swenson Reference Luks and Swenson2020). Skin pigmentation has been found to impact the accuracy of infrared pulse oximeters, particularly in low oxygen saturation (Bickler, Feiner & Severinghaus Reference Bickler, Feiner and Severinghaus2005; Feiner, Severinghaus & Bickler Reference Feiner, Severinghaus and Bickler2007). Despite prior work demonstrating this inaccuracy, these implications were not incorporated into the design of pulse oximeters leading to fatal misdiagnoses among Black patients (Tobin & Jubran Reference Tobin and Jubran2022). This example illustrates that designers must critically and carefully reflect on the impact of their users’ social identity when designing products.
Alongside users’ social identity, designers’ social identities could also influence the outcomes of the design process. For example, male-identifying designers have been observed to exhibit higher levels of ownership bias compared to female-identifying designers (Toh, Strohmetz & Miller Reference Toh, Strohmetz and Miller2016), a behavior that could have problematic consequences in gender-biased fields such as engineering (Cohen & Deterding Reference Cohen and Deterding2009). Similarly, in team-based design settings, (Cole et al. Reference Cole, Jablokow, Mohammed and Miller2023) found that women team members report a greater sense of psychological safety with other women-identifying team members compared to men on the team. Therefore, designers with different social identities may have different experiences in design settings while carrying their own biases and behavioral tendencies.
In the context of human-centered design, these two factors – that is, the designer’s social identity and the user’s social identity – could also interact to influence design outcomes. Cikara et al. Reference Cikara, Botvinick and Fiske2011 found that individuals show differences in pleasure-related neural activity when observing other individuals from similar vs different social groups. These differences also manifest in their behavior, especially as it relates to the likelihood of causing harm to other individuals. Similarly, (Li & Holtta-Otto Reference Li and Holtta-Otto2022) found that differences in cultural background between the designer and the user impact the designers’ ability to accurately understand the user. Designers must actively and intentionally prioritize overcoming these biases to ensure that their design decisions are inclusive and equitable. This need to consider social identity is particularly crucial given the increasing use of artificial intelligence-based tools in design (Zhu & Luo Reference Zhu and Luo2024) because these tools could carry systemic biases (Buolamwini & Gebru Reference Buolamwini and Gebru2018; Roselli, Matthews & Talagala Reference Roselli, Matthews and Talagala2019; Sham et al. Reference Sham, Aktas, Rizhinashvili, Kuklianov, Alisinanoglu, Ofodile, Ozcinar and Anbarjafari2023).
Taken together, social identity – both designers’ and the users’ – has important effects on the outcomes of the design process. Without reflecting on their social identity and position in society, designers may be blindsided by the various identity-oriented implicit biases they carry, and these biases could have effects throughout the design process. If not accounted for, designers’ biases could manifest as early as the task clarification stage, which sets the foundation for the rest of the design process (Pahl et al. Reference Pahl, Wallace and Blessing2007). In this stage, designers use stakeholder needs to first, define and record a set of design requirements. These requirements are then refined and extended as new information is obtained. The outcome of the task clarification stage is the generation of a list of requirements organized by their importance that guides the subsequent stages of the design process (e.g., concept generation).
Prior research has presented several approaches for integrating social identity considerations into engineering design; however, few of these methods encourage reflective, self-aware consideration of social identity. Such an approach is particularly important as designers must be mindful of their positionality – that is, the interactions between their own social identity and their decisions and actions as designers (Kohl & McCutcheon Reference Kohl and McCutcheon2015). One approach to encourage such reflexive practice is through reflective interventions. Reflective interventions are structured opportunities for individuals to examine their past experiences and actions through specific lenses – in our case, the effects of different dimensions of social identity. Such an examination, in turn, could build a sense of self-awareness and help identify opportunities for future actions that are informed by the reflection (Csavina, Nethken & Carberry Reference Csavina, Nethken and Carberry2016; Thomas et al. Reference Thomas, Orand, Shroyer, Turns and Atman2016; Long, Rajabzadeh & MacKenzie Reference Long, Rajabzadeh and MacKenzie2018). However, limited research has studied the effects of social identity-based reflection in engineering design, particularly in the task clarification stage. Our aim in this paper was to investigate this research gap through an exploratory experimental study. Before introducing the details of our study, we reviewed prior research that informed our study in Section 2 and concluded with our research questions. Next, in Section 3, we described our experimental methods including the metrics and coding scheme. Our data analysis approach along with the corresponding results are discussed in Section 4, and the implications of these results are presented in Section 5. We close with limitations and directions for future research in Section 6 and offer concluding remarks in Section 7. A glossary of the key terms used in our paper and the corresponding definitions are presented in Table 1.
2. Related work
We aimed to study the impact of social identity-based reflection on novice designers’ task clarification behavior in human-centered design. As a first step, we reviewed prior work in these areas as discussed next.
2.1. Impact of social identity in engineering design
(Kilker Reference Kilker1999) posits that designers’ social identity could have important effects on their design performance. In persona-centered engineering, identity is becoming increasingly more intertwined with design solutions, both for users and designers alike. Designers need to have a deep understanding of who they are, not only as individuals but also through the lens of their cultural background. By actively considering their social identity, designers can bring a more inclusive and nuanced perspective to their work, leading to designs that better meet the needs of diverse user groups (Özçam Reference Özçam2022). Designers can explore and better understand their social identity through verbal and visual representations, enabling them to become more in touch with their unique identity (Eastman Reference Eastman2001). Engineers can also explore their personal identity through written reflection, enabling them to understand how their identity influences their design approach. Strategies such as these empower students to ask new and thought-provoking questions, leading to fresh insights and innovative solutions in their design work (Özçam Reference Özçam2022). In addition, when engineers have a strong sense of belonging within the engineering community, they are more likely to thrive and contribute effectively to the field. They are also more likely to continue to partake in said field of choice (Godbole et al. Reference Godbole, Miller, Bothwell, Montfort and Davis2018). By embracing and acknowledging their personal social identity, designers and engineers can enhance their connection to the larger engineering community and the more minute details of a persona. They can also foster a sense of belonging, ultimately leading to personal growth and meaningful contributions in their professional lives.
Given that STEM disciplines, in particular engineering, have been historically dominated by men, research on the impact of gender on engineering design outcomes has received considerable attention. For example, male-identifying designers have been observed to favor their designs during concept selection, whereas female-identifying designers showed a preference toward their group members’ ideas (Toh et al. Reference Toh, Strohmetz and Miller2016). In another recent study, female-identifying team members tended to feel more psychologically safe when surrounded by other female-identifying people than when in all other male-identifying design teams (Cole et al. Reference Cole, Jablokow, Mohammed and Miller2023). Researchers have also found that student designers’ gender interacts with their perceptions of different problem contexts (Okudan & Mohammed Reference Okudan and Mohammed2006); specifically, design problems have been observed to have gendered undertones, which could influence designers’ engagement with these problems.
Alongside gender, one’s racial and cultural identity could also impact design decisions and outcomes. For example, (Peng, Menold & Miller Reference Peng, Menold and Miller2021) compared ideation outcomes between men and women recruited from an American and Moroccan sample. They found that cultural background was related to the quantity of ideas generated; while the American sample generated more ideas, no differences were observed in the quality of the ideas generated. Moreover, they found that both samples showed a greater preference for their own ideas, suggesting that ownership bias in concept selection transcends cultural background. Similarly, (Ji Young Cho, Lee & Woo Reference Ji Young, Lee and Woo2018) presented a study on the effects of expertise and cultural background on the evaluation of and preference toward creative ideas. They found that the participants recruited in the US scored ideas to be more creative compared to those recruited in South Korea. Another similar result was observed by Ham, Guerin & Scott (Reference Ham, Guerin and Scott2004); they found that Chinese participants preferred simpler but more coherent spatial design concepts compared to Americans.
The effects of designers’ social identity may be particularly pronounced in the context of human-centered design, wherein designers design products for users who may be similar or different from themselves. As an example, (Li & Holtta-Otto Reference Li and Holtta-Otto2022) presented a study on designers’ ability to empathize with users with similar and dissimilar cultural backgrounds and genders. They find that designers more accurately understand their users’ needs and recognize variations in users’ emotional tone when they listen to interviews of users with similar cultural backgrounds and gender. Similarly, (Stürmer et al. Reference Stürmer, Snyder, Kropp and Siem2006) found that one’s tendency to help another individual is higher when they both belong to the same cultural group. They also found that the strength of these effects is moderated by one’s perceived level of similarity with the other individual. These findings have also been extended to racial identity; (Azevedo et al. Reference Azevedo, Macaluso, Avenanti, Santangelo, Cazzato and Aglioti2013) found that participants show greater activation in brain regions corresponding to emotional experience when they observe individuals belonging to the same race experiencing a painful situation, when compared to individuals belonging to a different race. That is, one may perceive situations to be more painful when experienced by others belonging to the same race, therefore suggesting the presence of implicit biases when processing empathy (Eres & Molenberghs Reference Eres and Molenberghs2013). Therefore, designers must make an active effort to develop empathy toward their users and their needs, especially with users with social identities different from themselves.
Despite the importance of incorporating social context concerning the user into design decisions, researchers find that novice designers often struggle to do so effectively (Alsager Alzayed et al. Reference Alsager Alzayed, McComb, Menold, Huff and Miller2021; Burleson et al. Reference Burleson, Herrera, Toyama and Sienko2023). To overcome this barrier, researchers have tested numerous methods to support designers in taking the users’ perspective to better understand their needs. For example, Lee & Li (Reference Lee and Li2023) demonstrated that immersive media could be used to reduce sociospatial psychological distance by exposing individuals to new and removed perspectives. Similarly, Pahl and Bauer (Reference Pahl and Bauer2013) found that explicitly asking participants to take the perspective of a given user persona could encourage prosocial and proenvironmental action when compared against objective, fact-based decision-making.
Leveraging this utility of perspective-taking and empathy development in human-centered design, researchers have proposed design tools that help designers understand and capture the users’ perspective in design decisions. For example, empathy maps help designers represent user needs by taking into account their thoughts and feelings, and this method has been adapted to various contexts such as designing for users with autism (Melo et al. Reference Melo, Rivero, Santos and Barreto2020). Another such tool is extreme user simulation wherein designers simulate the experience of users with disabilities such as visual impairment to understand their experiences and better design to meet their needs (Raviselvam, Hölttä-Otto & Wood Reference Raviselvam, Hölttä-Otto and Wood2016). Of these various tools, user personas are commonly used in human-centered design given their effectiveness and ease of implementation. User personas are fictional representations of the archetype user that include their needs alongside other characteristics such as their social identity, interests, and often, a visual (Grudin & Pruitt Reference Grudin and Pruitt2002; Chang, Lim & Stolterman Reference Chang, Lim and Stolterman2008; Ferreira et al. Reference Ferreira, Silva, Oliveira and Conte2015).
Taken together, these studies suggest that designers’ social identity impacts their design behavior and their preferences. Moreover, designers with different social identities may carry implicit and cognitive biases, which, if unchecked, could have negative effects on design outcomes. These implicit biases may be particularly pronounced when designing for out-group users, that is, users belonging to social identities that are different from the designer. Although personas are an effective tool for empathy development, they alone may not be sufficient to fully overcome cognitive biases among designers, especially when designing for users who have social identities and experiences that significantly differ from their own (Li & Holtta-Otto Reference Li and Holtta-Otto2022). One approach to address these biases is to make designers aware of the biases they carry through reflective practice. However, limited research has studied the effects of social identity-based reflection in engineering design, and our aim in this research is to explore this gap. Before doing so, we review prior work on the use and effectiveness of reflective interventions, next.
2.2. Effectiveness of reflection-based interventions
Reflection has been identified as an important skill among successful practitioners (Schön Reference Schön2017), and this importance has also been extended to engineering design practice and education (Allen et al. Reference Allen, Mistree, Newstetter and Turns1997; Turns et al. Reference Turns, Sattler, Yasuhara, Borgford-Parnell and Atman2014; Sepp et al. Reference Sepp, Orand, Turns, Thomas, Sattler and Atman2015). Researchers also find that students report reflection to be one of the most effective methods of learning engineering design (Krause et al. Reference Krause, Yasuhara, Huneke and Atman2013). Adams et al. attempted to map reflective practice on engineering design processes (Adams, Turns & Atman Reference Adams, Turns and Atman2003). They argued that the iterative nature of engineering design presents a unique opportunity to embed reflection in action. Since effective designers often retrace their steps to improve their solutions, these iterations could act as opportunities for designers to reflect on their decisions in the previous iteration and find areas to improve upon in the following iteration.
(Carberry et al. Reference Carberry, Harding, Cunningham, Csavina, Ausman and Lau2018) presented a study on engineers’ use of reflection in their personal and professional lives. They found that improvement was one of the most common uses of reflection. All three groups in the study (i.e., students, faculty, and practitioners) used reflection to look back on their past actions and identify opportunities to improve as people, an observation also made by Csavina, Nethken & Carberry (Reference Csavina, Nethken and Carberry2016). Alongside reflecting on learning and the design process, reflection-based interventions have also been used to introduce practices such as mindfulness in engineering design practice and education (Nolte, Huff & McComb Reference Nolte, Huff and McComb2022; Nolte et al. Reference Nolte, Soria Zurita, Starkey and McComb2023). These reflective mindfulness interventions have been shown to increase the quality of ideas generated by student designers, with no effects on creativity or stress experienced by them.
Reflective practice could be integrated into engineering education and practice in numerous ways (Turns, Mejia & Atman Reference Turns, Mejia and Atman2020). (Thomas et al. Reference Thomas, Orand, Shroyer, Turns and Atman2016) posit that of the various definitions and implementations, two factors are critical to the integration of reflection into engineering education. First, reflective practice must encourage students to examine their past experiences and actions through various lenses – in our case, through the lens of social identity. Second, students must use the findings from this examination to inform their future actions. The authors argue that such an open interpretation and implementation allows educators and students to acknowledge that reflection is “messy, personal, and complicated” (p. 1). Moreover, employing such an open definition allows educators to implement reflective practice through various lenses and adapt it to suit the design context. This recommendation also aligns with the findings by Csavina, Nethken & Carberry (Reference Csavina, Nethken and Carberry2016); they observe that engineering students most frequently use reflective practice to look back at their past actions through a given lens, followed by identifying opportunities for future action.
Taken together, reflection is an effective method to encourage designers to critically examine their (past) actions and design decisions, and improve upon them in subsequent iterations. Social identity-based reflection could help designers be more mindful of the effects of social identity – both their own and their users – as they progress through the design process (Sengers et al. Reference Sengers, Boehner, David and Kaye2005). Such interventions could also encourage the reflexive practice of design, wherein designers are mindful of their positionality and account for the potential effects of their own social identity on their design decisions. However, limited research has investigated the utility of social identity-based reflection in human-centered design, especially when designers are designing for users dissimilar from themselves. These effects are particularly little understood in the task clarification stage, during which designers generate and prioritize functional requirements based on stakeholder needs. Our aim in this paper is to explore this research gap and answer the research questions presented next.
2.3. Research questions
Based on these identified research gaps, we aimed to explore the effects of a social reflection-based reflection exercise on novice designers’ task clarification behavior. Toward this aim, we seek to answer the following research questions (RQs):
-
• RQ1: How does a social identity-based reflection impact task clarification behavior? Do these effects vary based on the user persona?
-
• RQ2: How does a social identity-based reflection impact self-reported empathic response? Does this relationship vary based on the user persona?
Building upon these RQs aimed at understanding the effects of the reflective intervention, we also seek to examine the content and quality of participants’ reflections. Toward this aim, we seek to answer the following RQs:
-
• RQ3: What topics do participants discuss in their reflection responses? Does the content of their reflection responses vary based on their social identity?
-
• RQ4: What is the quality of participants’ reflection responses? Does the quality of participants’ responses vary based on their social identity?
3. Experimental methods
To answer these RQs, we conducted an experiment comprising a reflection exercise followed by design task clarification. The experiment was approved by Lafayette College’s Institutional Review Board and informed consent was obtained from the participants before the experiment was conducted. The details of the experiment are discussed next.
3.1. Participants
Participants (N = 22) were recruited from a capstone design course in mechanical engineering at a small liberal arts college in the northeastern USA. Participants were fourth-year undergraduate students in engineering, with most participants studying mechanical engineering. Twenty participants identified as men and two participants identified as women when asked for their gender. Additionally, 15 participants self-identified as White, four as Hispanic, two as Asian and one as Black, when asked for their racial identity. Sixteen participants identified as White, and one each as Chinese American, African American, Hispanic and Irish American, with two preferring not to answer when asked for their ethnicity. Although our sample size is within the recommended range for one-tailed experimental and causal-comparative studies, it is smaller than that for two-tailed causal-comparative studies (Cash et al. Reference Cash, Isaksson, Maier and Summers2022). Moreover, our sample is relatively homogenous in racial, ethnic, and gender identities – an artifact of the student population at Lafayette College. A homogenous sample provides high internal validity but also calls for future work with a diverse sample (Reynolds, Simintiras & Diamantopoulos Reference Reynolds, Simintiras and Diamantopoulos2003).
3.2. Procedure
First, participants were asked to complete a baseline survey. In this survey, we collected demographic information such as gender and ethnicity (see Section 3.1). The demographic questions were developed based on the guidelines proposed by Hughes et al. (Reference Hughes, Camden, Yangchen, Smith, Domenech Rodríguez, Rouse, McDonald and Lopez2022). Upon completing the survey, approximately half of the participants (N = 10) were asked to complete a reflection intervention exercise on social identity. Participants were asked to reflect on the following open-ended cues, to encourage reflection on different dimensions of their social identity:
-
1. What is your relationship with your gender identity and how do you think it impacts you as an engineer?
-
2. What are your thoughts about your racial, cultural and ethnic backgrounds, and their impact on you as an engineer?
-
3. How do you feel that your socioeconomic status has affected your life and how do you think this factor will shape you as an engineer moving forward?
-
4. What are some unique challenges you have faced because of aspects of your social identity, and their effects on you as an engineer?
These cues were formulated based on prior work and prompted participants to look back on their past experiences and reflect upon how these experiences could impact their future actions (Thomas et al. Reference Thomas, Orand, Shroyer, Turns and Atman2016; Csavina, Carberry & Nethken Reference Csavina, Carberry and Nethken2017).
After completing the reflection exercise – or the baseline survey for the control group – both groups were introduced to the persona and the design problem. All participants were given the following design problem focused on public transport:
Design a sustainable public transit system for a medium-sized city with a population of 500,000. The system should significantly reduce the carbon footprint of transportation in the city and improve air quality. It should be cost-effective and easy to use for commuters.
Participants were then given background information on the design problem using one of the two randomly assigned personas. As discussed in Section 2.1, persona-based design, first proposed by Cooper (Reference Cooper, Arend, Eberleh and Pitschke1999), enables designers to bridge the psychological distance between them and their users (Prabhu et al. Reference Prabhu, Herzog, Fodale, Alsager Alzayed and Starkey2024) and prioritize and focus on user needs in the design process (Miaskiewicz & Kozar Reference Miaskiewicz and Kozar2011). Therefore, we used personas to provide an efficient and effective representation of potential users and their needs and help participants incorporate the social context of the design problem into their design decision-making. Participants were randomly given one of the two personas:
-
1. Sarah is a 37-year-old woman who is a mother of two and works as a nurse at the local hospital. She is a first-generation immigrant from Mexico. Sarah is considered low income and does not have a strong grasp of English. Despite facing financial challenges, Sarah worked hard to earn a nursing degree and is dedicated to providing quality healthcare to her patients. In her free time, Sarah is active in her community and works with local organizations to promote social justice and environmental sustainability. She is environmentally conscious and is always looking for ways to reduce her carbon footprint. She currently drives her own car to work, but the cost of gas and maintenance is a financial burden for her family. Sarah is also concerned about the air quality in the city, as her children have asthma and are sensitive to air pollution.
-
2. Jack is a 37-year-old parent to two children. Jack is nonbinary and a college student who relies on public transportation to get to and from campus. Jack made the decision to attend a local community college after a workplace accident left them with a lower body mobility issue and thus unable to continue in their blue-collar profession. They do not have a car and find the current bus system to be unreliable and time-consuming. Jack is on a tight budget, so they are looking for a cost-effective and convenient transportation option. In addition, they are Hindu and value living in harmony with the natural world and are actively involved in local organizations which promote environmental sustainability and protection. Jack does not currently own their own car and is concerned about the air quality in the city due to their parents’ asthma and sensitivity to air pollution.
While the first persona emphasizes the aspects of first-generation status and low income, the second persona emphasizes the aspects of gender identity and disability. Both these personas are significantly different from the demographics of the studied sample and were formulated to test whether varying certain aspects of the user’s persona affected designers’ task clarification behavior. However, varying more than one dimension of the persona could create confounding effects, calling for a systematic investigation into these effects.
Upon introducing the design problem and the persona, participants were given 25 minutes to generate functional requirements for the given design problem. They were given the freedom to define functional requirements as they saw best fit the design task. Participants were also asked to complete an empathy survey after completing the task clarification activity, created along the guidelines proposed by Hess, Sanders & Fila (Reference Hess, Sanders and Fila2022).
3.3. Metrics and coding scheme
The experimental data were analyzed using a mixed-methods approach. The metrics used in our analyses are discussed next.
3.3.1. Coding scheme used to examine functional requirements
The functional requirements generated by the participants were qualitatively coded using a deductive content analysis approach (Elo & Kyngäs Reference Elo and Kyngäs2008). In this approach, the collected data is explored to identify recurring themes. This approach contrasts theory-driven inductive content analysis. First, one of the two authors reviewed the complete dataset to identify the most common themes and created an initial codebook. Then both authors independently reviewed 25% of the dataset using the initial codebook. Any disagreements were resolved through discussions, resulting in the four main themes being identified in Table 2: (1) sustainability, (2) cost, (3) social identity and (4) usability. Upon reaching sufficient agreement, one of the two authors coded the remaining data. After coding the dataset, the frequency of each node was calculated for each participant and these frequencies were used as the data to answer RQ1. We also calculated the total number of requirements generated by each participant. It should be noted that the coding was performed at the phrase level instead of the functional requirement level. Therefore, some complex requirements were split and coded under multiple nodes.
Table 2. The coding scheme used for coding the functional requirements
3.3.2. Scale for assessing perceived empathic response
After completing the problem-framing task, participants were given a 12-item survey that measured their perceived empathic response. The survey comprised a subset of the survey proposed by Hess et al. (Reference Hess, Sanders and Fila2022). Specifically, we used 12 items under the “need finding’ category, including items such as “I imagined the user’s everyday activities in their real-life context.” Participants were asked to respond to each item on a six-point scale with 1 = strongly disagree and 6 = strongly agree. The 12 items were then consolidated into the following four constructs: (1) imagine-self cognitive empathy, (2) imagine-other cognitive empathy, (3) self-oriented affective empathy and (4) other-oriented affective empathy (Hess et al. Reference Hess, Sanders and Fila2022). The average scores for the four subconstructs were used as the data to answer RQ2.
3.3.3. Coding scheme used to examine reflection responses
The reflection intervention responses provided by the participants in the treatment group (N = 10) were qualitatively coded using a deductive content analysis approach (Elo & Kyngäs Reference Elo and Kyngäs2008). The same iterative procedure discussed in Section 3.3.1 was used to build the coding scheme and implement the content analysis, with any disagreements being resolved through discussion. This iterative process resulted in four main themes being identified as seen in Table 3: (1) experience-based, (2) perspective and awareness, (3) design and action and (4) no effect. After coding the dataset, the frequency of each node was calculated for each participant and these frequencies were used as the data to answer RQ3.
Table 3. The coding scheme used for coding the reflection responses
3.3.4. Scale used to evaluate the quality of participants’ reflections
The quality of participants’ reflections was then evaluated at the reflection using the four-point scale proposed by Kember et al. (Reference Kember, McKay, Sinclair and Wong2008). According to this scale, the lower level of reflection is labeled “non-reflection.” At this level, participants did not attempt to reach an understanding or attempt to reflect. The second level is labeled as “understanding” in which participants present surface-level reflections without any critical analysis. The third level is labeled as “reflection” in which participants consider the question in relation to their personal life. Finally, the highest level of reflection is labeled as “critical reflection.” At this level, participants deeply reflected and offered some form of critical analysis. Participants’ responses to each of the four reflection cues were evaluated using this four-point scale at the question level and these scores were used to examine RQ4. Examples of reflection responses corresponding to each level are presented in Table 4.
Table 4. The four-point scale used to evaluate participants’ reflection responses and corresponding examples
4. Data analysis and results
We analyzed the data collected using mixed methods – i.e., a combination of qualitative and quantitative research methods. The details of our analyses and the corresponding results are discussed in this section. It should be noted that data from all 22 participants was used in our analyses with the distribution of participants across the four groups presented in Table 5.
Table 5. Distribution of participants across the four groups
4.1. RQ1: How does a social identity-based reflection impact task clarification behavior? Do these effects vary based on the user persona?
To answer our first RQ, we first tested whether our data was normally distributed, using the frequency of each node from Table 2 as the dependent variable. Using Shapiro–Wilk tests, we find that the data was not normally distributed (p < 0.05) (Shapiro & Wilk Reference Shapiro and Wilk1965). Despite this violation, we first checked for two-way interactions between the independent variables (i.e., persona and reflection treatment) using a series of two-way ANOVAs. From the results, we see no significant interaction between the two independent variables (p > 0.05). Therefore, we performed separate Wilcoxon signed rank tests with the persona and treatment as the independent variables (Blair & Higgins Reference Blair and Higgins1985). From the results (see Figure 1), we see that the reflection exercise had a significant effect on the frequency of three of the four nodes. Specifically, participants from the treatment group generated more functional requirements related to sustainability and social identity (p < 0.05). In contrast, participants who received the treatment generated fewer usability-focused requirements. Overall, participants who received the treatment also generated more requirements (see Figure 3). No significant differences were observed for the number of requirements related to cost. Next, we analyzed the effect of the personas provided (i.e., Sarah and Jack) on the functional requirements generated. As visualized in Figure 2, we observed no significant differences in the frequency of requirements coded under the four nodes (i.e., cost, usability, social identity, and sustainability). Additionally, we see no statistically significant difference in the total number of functional requirements generated by the participants between the two personas (see Figure 3). The implications of these results are discussed in detail in Section 5.
Figure 1. Comparing frequencies of the various nodes based on the treatment group.
Figure 2. Comparing the frequency of the various nodes based on the persona provided.
Figure 3. Comparing the total number of requirements generated based on treatment group and persona.
4.2. RQ2: How does a social identity-based reflection impact self-reported empathic response? Does this relationship vary based on the user persona?
To answer the second RQ, we first tested the normality of our data using Shapiro–Wilk tests (Shapiro & Wilk Reference Shapiro and Wilk1965). Participants’ scores on the four components of the perceived empathic response scale (see Section 3.3.2) were used as the dependent variables. We observed that the data were not normally distributed (p < 0.05). Despite this violation, we conducted a series of two-way ANOVAs to test for interaction effects between our independent variables. We used the four components of the perceived empathic responses scale as the dependent variables, and treatment group and persona as the independent variables. From the results, we observed no significant interaction effects between the treatment group and persona (p > 0.05). Therefore, we conducted a series of Wilcoxon signed rank tests with the four components of perceived empathic response as the dependent variable, and either the persona or the treatment group as the independent variable (Blair & Higgins Reference Blair and Higgins1985). From the results (see Figures 4 and 5), we observe no significant effect of either persona or treatment group on the self-reported empathic response scores.
Figure 4. Comparing participants’ self-perceived empathic response based on treatment group.
Figure 5. Comparing self-perceived empathic response based on persona.
This lack of significant differences could be attributed to participants overestimating their ability to empathize with the persona, despite the observed differences in their problem-framing behavior performance in RQ1. To test this inference, we conducted a series of single-tailed, one-sample t-tests wherein we compared participants’ scores to the scale center of 3.5. From the results, we see that overall, participants reported high levels of empathy responses compared to the scale center (p < 0.05). The implications of these results are discussed in Section 5.
4.3. RQ3: What topics do participants discuss in their reflection responses? Does the content of their reflection responses vary based on their social identity?
To answer our third RQ, we first qualitatively analyzed participants’ reflection responses at a phrase level using content analysis. Participant responses were assessed using a deductive content analysis approach using the coding scheme presented in Section 3.3.3 (Elo & Kyngäs Reference Elo and Kyngäs2008). We conducted a one-way ANOVA to test whether some nodes were discussed more frequently than others, and this result was not statistically significant (p = 0.40).
Next, we separated the participants into two groups based on their self-reported social identity: majority and minority. The ‘majority’ group comprised six participants, all of whom identified as white men whereas the ‘minority’ group comprised four participants, two of whom identified as racially or ethnically Hispanic, one as female, and one who identified as mixed-race (Asian and White). This distinction was determined based on prior identity-based distribution data in STEM (National Center for Science and Engineering Statistics 2023). A two-way ANOVA was performed with frequency as the dependent variable, social identity group (i.e., majority or minority) as the first independent variable, and the node as the second independent variable. The results of this ANOVA are presented in Table 6; we observe a significant interaction between the node and group variables. We also noticed that the social identity group had a significant impact on the frequency. Therefore, we separated our results by node and compared the frequencies for each node between the two social identity groups. The results of this separation can be seen in Figure 6. We decided to use a larger threshold for significance (p < 0.1) due to the small sample size, making it difficult to ascertain significance owing to a lack of statistical power, therefore, prompting the use of follow-up qualitative analyses.
Table 6. Results of the two-way ANOVA testing the effects of node and social identity group on the frequency of references
Figure 6. Comparing frequencies of the various nodes based on the social identity group.
We first conducted a one-way ANOVA comparing the frequency of responses for the “experience-based” node between the majority and minority groups. This node included aspects such as personal experiences, positive experiences and negative experiences. The results indicated a significant effect of the social identity group on the frequency of this node (p < 0.06, F = 3.77, SSE = 12.15). Participants in the minority group discussed matters relating to this theme on an average of 4.25 times while the members of the majority group reflected on their personal experiences an average of 1.75 times. An example of a positive experience is LYON11 (majority group) stating in their reflection that “I think my socio-economic status has allowed me to succeed as an engineer.” On the other hand, experiences such as “a feeling of isolation and imposter syndrome,” felt by participant NATY05 (minority group), are examples of negative experiences. From this result, we infer that participants from minoritized identities may be more mindful of the effects of their social identity and be more adept at reflecting on their prior experiences in this context.
We conducted a similar analysis for our second node, that is, “perspective and awareness.” This node includes themes such as reflecting on things that have given the designer a new viewpoint on things, or, more commonly, an acknowledgement of privilege. We see a significant difference in the frequencies of the second node between the majority and minority groups (p < 0.06, F = 3.77, SSE = 22.8). Participants in the majority group reflected on the themes related to this node an average of two times throughout the reflection, with most of those being an acknowledgment of privilege. On the other hand, participants in the minority group reflected on this topic an average of 4.66 times with participant SAYN02 stating “it also shapes the way I budget costs for materials I need for projects I may work on in the future….” This reflection shows a different priority than those typically seen by the majority group wherein minority group members tend to evoke an aspect of their social identity to describe a unique perspective, in this case, budgeting, into a design team.
Our third node was labeled as “design and action” and consisted of items that represented themes ranging from the development of engineering morals to the context in which design decisions are made. In this node, we see no significant differences in frequencies between the two groups (p = 0.37, F = 0.89, SSE = 2.02). In this instance, both groups reflected nearly the same amount on how different aspects of their social identity impacted their design outcomes. An interesting observation was that some participants discussed the difficulty of designing for someone different from themselves, such as majority group member ANRK07 saying “In terms of problem framing…I could only come up with problems that reflect me.” Interestingly, the only participants who did reflect on how their social identity impacts their design outcome were members of the minority group, with LEON09 reflecting that “culturally I have been exposed to a lot of different cultures and my designs reflect this.”
The final node was labeled “no effect” and participants’ phrases coded under this node did not show reflection on their social identity. An example of a non-reflection is participant NENG11 discussing “this has no impact on me.” We see a significant difference in the frequencies of this node between the majority and minority groups (p < 0.09, F = 3.49, SSE = 9.6). This node shows that majority group members said more often that social identity has no effect on decision/engineering experiences compared to minority groups. Looking deeper, if we were to remove the one minority participant who identified as “White, Asian,” or ‘mixed’, the differences are statistically significant (p < 0.001, SSE = 8.4). This result leads us to believe that more work should be done to understand how a designer’s mixed background social identity, particularly between minority and majority groups, impacts them as a designer. The implications of these results regarding differences in the content of participants’ reflections are discussed further in Section 5.
4.4. RQ4: What is the quality of participants’ reflection responses? Does the quality of participants’ responses vary based on their social identity?
To answer our fourth RQ, the quality of participants’ reflections was assessed using a four-point scale (see Section 3.3.4.), and each written response was scored according to the criteria seen in Table 4. The data are visualized in a series of graphs (see Figure 7). From a high-level perspective, we can see that very few participants scored a four, with most participants scoring a two. Therefore, we examined differences in reflection quality scores for each reflection cue based on the participants’ social identity group, as discussed next.
Figure 7. The overall distribution of reflection quality scores based on the reflection questions.
For the first reflection cue, relating to gender identity, we see that an overwhelming majority of the participants scored either a one or a two, meaning that they engaged in little to no reflection, we can see this in Figure 8. An example of this would be participant LERE03’s response: “I find this has almost no impact on [me] as an engineer.” All of these participants were in the majority (i.e., White men) group. However, the sole female participant scored higher, receiving a three, indicating that they had reflected on a deeper level. A quote for this participant was: “Therefore, being a woman has had some positive impact that I’ve seen. I’ve also been fortunate enough to be at a college with a higher percentage of women.” Interestingly enough, the only participant to receive a four on the reflection, representing a critical level of reflection, was the only participant who criticized the idea of gender having any impact on engineering. In response to the first question, participant LYON11 stated “I do not think [my gender identity] has impacted me more than if I was a different gender. With that being said, women have been welcomed into the STEM field, as well as other non-binary genders, but the one thing that could be fixed is the wage gap between women and men…it does not matter what gender you are.” Despite this criticism, they still showed signs of deep reflection including personal experiences and critical analysis.
Figure 8. Reflection quality scores based on social identity groups for each of the four reflection cues.
The second reflection cue asked participants to reflect on the effects of their racial, cultural, and ethnic backgrounds on their performance as engineers. From the results in Figure 8, we can observe three things. First, the two participants who self-identified as Hispanic both scored higher on reflection quality than the rest of the majority group. An example of higher level reflection from these participants includes a response from LEON09 (minority group member) saying “I think culturally I have been exposed to a lot of different cultures and my designs reflect this.” In addition, the sole female participant showed a critical level of reflection, discussing themes such as “as an engineer, this has impacted me in my design solutions to include as many diverse stakeholders as possible.” In contrast, participants from the majority group, such as NENG11 said “I think that my background has not impacted me as an engineer.” This result suggests that participants in the minority group are more easily able to reflect deeply on how their unique racial, cultural, or ethnic backgrounds affect them as engineers while members of the majority group do not acknowledge them.
The third reflection cue asked participants to think about their socioeconomic status and how it has impacted them as engineers. We can see in Figure 8 that there was very little difference between the scores of the two social identity groups, with almost all the participants either acknowledging that they had an advantage because of their socioeconomic status (which received a score of two) or sharing personal experiences about how their status has impacted them (which received a score of three). For example, participant LERE03 (majority group and scored a two) mentioned “I think it [socio-economic status] has given me an edge” while participants who received a three shared a personal story, such as SAYN02 in the minority group reflecting about how hard it was to afford the price of books and how that has impacted their studies. However, the two participants who scored a one are both in the majority group. These results suggest that there was a small difference between minority and majority groups in regards to the quality of reflection when considering socioeconomic status; however, the highest scores in this category were all members of the minority group.
The fourth and final reflection cue was open-ended and prompted the participants to reflect broadly on any unique effects of their social identity on their experience as engineers. Once again, the participants’ scores were evenly distributed from a three and below, with the only female-identifying participant scoring a four. Members of the minority group scored an average of three whereas members of the majority group scored an average of 1.66. Participants reflected on a variety of factors, to varying degrees of efficacy. Participants such as LERE03 said “I have dyslexia so often I take longer to do stuff,” which received a three. Others, such as IAWA04 (of the majority group) stated that they are “not really in a position to face…challenges.” These results suggest that participants, particularly those in the majority group, struggled to reflect in a meaningful manner when provided with an open-ended reflection cue.
Another interesting finding is that the average reflection quality scores across the four reflection cues were higher among participants belonging to the minority social identity group and this difference was statistically significant (F = 5.48, p = 0.05, SSE = 2.60). Specifically, participants belonging to the majority group, on average, scored 1.70 on reflection quality whereas those belonging to the minority group scored 2.75 on average. This result suggests that members of the minority group tend to have higher quality reflection performance than members of the majority group. The implications of these results are discussed further, next.
5. Discussions and implications for design practice
Our aim in this paper was to study the impact of a reflection exercise on social identity on designers’ task clarification behavior in human-centered design. Toward this aim, we conducted an experimental study with novice student designers, and five key findings were observed from the results:
-
1. Participants who completed the reflection generated more functional requirements, especially those related to the user’s social identity (RQ1).
-
2. The persona provided to the participants did not impact their task clarification behavior (RQ1).
-
3. No significant differences were observed in the self-reported empathic response scores between groups; however, all participants reported high levels of empathic response (RQ2).
-
4. Higher frequencies of responses related to personal experiences and design/action were observed among members of the minority group (RQ3).
-
5. Participants in the minority group provided higher-quality reflections compared to those in the majority group (RQ4).
The first finding is that participants who completed the reflection exercise generated more functional requirements compared to the control group. This effect was particularly pronounced with the frequency of design requirements related to social identity. This result corroborates prior work wherein reflecting on past experiences has been shown to improve future performance (Csavina et al. Reference Csavina, Carberry and Nethken2017). Therefore, social identity-based reflection could motivate designers to consider the implications of their users’ social identity in human-centered design. These considerations could, in turn, motivate designers to frame problems that better account for the users’ social identity, therefore setting a foundation for the design of inclusive products. This finding also supports previous research suggesting that reflection is a vital part of understanding one’s own identity (Sengers et al. Reference Sengers, Boehner, David and Kaye2005).
Although participants who completed the reflection exercise generated more functional requirements related to social identity, they generated fewer functional requirements that focused on usability. This result could be attributed to priming (Schacter & Buckner Reference Schacter and Buckner1998); participants who reflected on their social identity before task clarification could have been primed to emphasize social identity, and consequently, give a lesser emphasis on other aspects of the design prompt (e.g., usability). This finding also calls for a further investigation into designers’ cognitive processes when making trade-offs between user-related factors and those related to the overall problem.
The second finding is that the persona provided to the participants did not influence their task clarification behavior. This finding could be attributed to the psychological distance between our sample and both personas. For example, most of the participants in our sample were men, and both our personas were formulated to be nonmale identifying. Therefore, the participants in our study could have been similarly psychologically distant from both personas, resulting in the lack of differences. This result, combined with our first key finding, suggests that a social identity-based reflection could help bridge the psychological distance between the designer and the user, especially when they belong to different social identities. However, this inference calls for a systematic investigation into the effects of differences in different dimensions of social identity (e.g., gender and race) on task clarification behavior and the effects of reflection.
The third finding is that although the treatment and persona did not impact participants’ self-reported empathic responses, all participants reported high levels of empathic response compared to the scale median. This result contradicts the differences observed in the results of the first RQ. Taken together, these results suggest that novice designers may not be able to accurately report their empathic response in human-centered design. The lack of accuracy could be particularly pronounced when designing for users whose social identities are different from theirs, an observation also made in prior research (Li & Holtta-Otto Reference Li and Holtta-Otto2022). Furthermore, the high empathic response scores provided by the participants suggest that novice designers tend to overestimate their ability to empathize with the user. An overestimation of their empathic response could fail to account for one’s implicit biases, which could, in turn, lead to an inaccurate understanding of the users’ needs. Therefore, future research must formulate interventions that impact both designers’ task clarification behavior and their self-evaluation of it.
The fourth finding is that we observed a higher frequency of reflection occurrences related to personal experience and design/action among members of the minority group compared to the majority group. This finding could be attributed to the inherent reflection that comes with being a member of a minority group. Previous research has found that students can lack the motivation to deeply engage in a reflection exercise, which can be observed in the majority group (Wilson Reference Wilson2013). This difference can be seen more clearly as members of the two groups tend to have vastly different experiences during design projects (Smith & Trede Reference Smith and Trede2013). This finding suggests that designers with different social identities may require differently formulated prompts when implementing reflective interventions in design practice and education. These prompts must be carefully designed to encourage designers, especially those who occupy majority identities, to recognize the effects of social identity on their decisions as engineers and designers. The formulation of these prompts may be particularly challenging as designers who occupy majority identities may have prior experiences that starkly contrast those with minority identities, making it difficult to anchor their experiences within the reflective context.
Finally, we see that members of the minority group tended to provide higher-quality reflections than members of the majority group. It is interesting to note, that even the one participant who self-identified as a White woman provided higher quality reflections when concerning other aspects of social identity, such as racial, cultural and ethnic backgrounds, compared to members of the majority group. This result further reinforces our previous inference calling for the need to carefully formulate the prompts used in reflective interventions as these prompts could not only impact the content but also the quality of reflections. Furthermore, these prompts should be formulated such that designers – especially those who occupy majority identities – are empowered to provide meaningful and high-quality reflections. Such efforts will enable designers to be aware of the effects of social identity on their decisions as engineers and designers, and enable the design of inclusive engineering solutions that account for users of diverse social identities and needs.
Although these results lend evidence in support of the effectiveness of reflective interventions, it should be noted that the participants in our study were sampled from a population of fourth-year undergraduate engineering students. Although more experienced compared to first-year students, the participants in our sample may not have the same experience and expertise as that of practicing engineers. Prior research suggests that individuals develop metacognitive skills as they gain expertise in their domain and these skills also tend to be domain-agnostic (Veenman & Elshout Reference Veenman and Elshout1999). These correlations between metacognition have been demonstrated in various domains ranging from sports (MacIntyre et al. Reference MacIntyre, Igou, Campbell, Moran and Matthews2014) to engineering design (Ball et al. Reference Ball, ST, Evans, Dennis and Ormerod1997; Dixon & Johnson Reference Dixon and Johnson2011). Therefore, the participants in our study may have low baseline levels of metacognitive skills, leading to the low quality of reflections observed among participants from the majority groups. Experienced designers may show different results, and this inference presents an opportunity for future exploration.
Taken together, our findings highlight the effectiveness of reflective practice toward inclusive design, and design educators can consider the following aspects when formulating such interventions. If educators aim to encourage student designers to generate more functional requirements, especially those that focus on the users’ social identity, a rudimentary reflection exercise may be effective in doing so. Such an intervention could prime designers to focus on the effects of social identity when generating and refining functional requirements. Moreover, a reflective intervention such as this one may be effective when designing for users across dimensions of social identity. However, more care may be required if educators aim to encourage deeper, higher-quality reflection. Specifically, a rudimentary reflective exercise may not be sufficient to encourage deeper, high-quality reflection among students belonging to social identities that are dominant in engineering (e.g., men). Students belonging to marginalized groups may have prior experiences with biases, providing impetus for deeper reflection. On the other hand, students with majority identities may not have similar experiences and, therefore, may require the formulation of structured reflection cues that enable deeper reflection. Such interventions must provide structured opportunities for students with majority identities to contextualize their past experiences into the lens under consideration and identify future action toward improvement. Such structured reflective interventions may be particularly important for student and novice designers who may not have yet developed the metacognitive skills that come with experience.
6. Limitations and directions for future work
Our findings provide important insights into the use of social identity-based reflection in human-centered design; however, some limitations exist, presenting opportunities for future work. First, the personas used in our study varied in multiple dimensions (e.g., gender and culture), which could have resulted in confounding effects. Therefore, participants could have been psychologically distant from the users along more than one dimension (Liberman & Trope Reference Liberman and Trope2014). Future research must systematically explore the effects of this distance along the different dimensions of social identity. Second, we had a small and relatively homogenous sample. A homogeneous sample provides a high degree of internal validity to our findings (Reynolds et al. Reference Reynolds, Simintiras and Diamantopoulos2003) and internal validity is often considered a prerequisite for establishing external validity and generalizability (Slack & Draugalis Reference Slack and Draugalis2001; Ferguson Reference Ferguson2004). Moreover, the granular data collected using qualitative methods provides an in-depth understanding and representation of participant behavior, compared to surface-level measurements provided by commonly used quantitative methods. However, future research needs to replicate and extend these results to a larger and more diverse sample. Third, when considering participants’ empathic responses, we used self-reported data. Self-reported scores are prone to over- and underestimation (Asher Reference Asher1974), especially among novices and in the presence of performance expectations (Brock et al. Reference Brock, Edelman, Edwards and Schuck1965). While self-reported scores provide a preliminary assessment of empathic response, future work must compare these perceptions to behavioral data. Moreover, we studied task clarification behavior using the frequencies of the various nodes, without considering the quality and accuracy of the functional requirements. Prior research suggests that empathy is often incomplete and inaccurate (Li & Holtta-Otto Reference Li and Holtta-Otto2022), calling for an analysis of the quality of the requirements generated.
7. Concluding remarks
Our aim in this study was to investigate the use of a social identity-based reflection exercise on novice designers’ task clarification behavior. Toward this aim, we conducted an experiment with fourth-year engineering students comprising a reflective intervention and a design task clarification activity. From the results of our experiment, we see that participants who received the reflection exercise generated more functional requirements overall, especially those focused on their user’s social identity. On the other hand, we see no significant effect of either the treatment or the persona on participants’ self-reported empathic response. Moreover, participants reported significantly higher empathic responses, therefore, suggesting that novice designers may not be able to accurately evaluate their empathic response in task clarification. Upon a deeper, qualitative analysis of the reflection responses, we see that participants who occupy minoritized identities (e.g., women in STEM) provided detailed and higher-quality reflection responses. These findings suggest that reflecting on their social identity before engaging in a design task could help designers make inclusive design decisions in the task clarification stage. Such inclusive design decisions could set the foundation for the design of products that can be equitably used by users across dimensions of social identity. Moreover, our results suggest that the designers’ social identity may impact their engagement with reflective interventions and the effects of these interventions on their design decisions. These results call for a further investigation into the formulation of reflective interventions in design practice and education, especially those that are carefully tailored for designers with different social identities.
Acknowledgments
Part of this research was funded through the Clare Boothe Luce Ally Scholarship offered by the Engineering Division at Lafayette College. Preliminary versions of this paper were presented at the Design Computing and Cognition ‘24 Conference (Brown & Prabhu Reference Brown, Prabhu and Gero2024) and the 2025 International Conference on Engineering Design (Brown & Prabhu Reference Brown and Prabhu2025).
Open access
