Design is a complex, ill-structured problem-solving process characterized by ambiguously defined requirements (Jonassen Reference Jonassen2000), and incomplete knowledge and information about the decision variables involved or the constraints affecting the viability of potential solutions (Goel & Pirolli Reference Goel and Pirolli1992). The process is situated in socio-technical contexts involving diverse internal and external stakeholders with conflicting goals and priorities (Shai & Reich Reference Shai and Reich2004; Clegg et al. Reference Clegg, Robinson, Davis, Bolton, Pieniazek and McKay2017), interfacing technical systems and various other constraints that limit design decisions. As designers formulate their problems and develop solutions, they often encounter trade-off situations (Moriarty Reference Moriarty1994), where performance improvements relative to one goal come at the expense of performance on another (Byggeth & Hochschorner Reference Byggeth and Hochschorner2006).
The engineering design literature primarily views trade-offs through prescriptive, quantitative approaches, where decisions are informed by various analyses of well-defined goals and constraints. However, these approaches may ignore the situated and temporal nature of design activity (Clancey Reference Clancey1997; Kimbell Reference Kimbell2012) and assume the designer understands how the context will influence the design entirely in advance of performing the analysis. Designers’ ongoing interactions with their context may be both driving the trade-offs they encounter and providing the information and flexibility required to resolve them. On one hand, conflicting stakeholder goals, rigid requirements and nonnegotiable technical constraints limit solution options, potentially forcing designers to accept trade-offs that compromise design performance. On the other hand, knowledge and information from others can increase the degrees of freedom available to navigate trade-offs, potentially expanding the design space or reframing the problem in ways that help designers bypass trade-offs altogether (Nickel, Duimering & Hurst Reference Nickel, Duimering and Hurst2022).
Little is known about the effects of context on design trade-offs, or how engineering designers interact with context to manage trade-off situations. Addressing this gap, the present study responds to calls in the literature for more research of design in the real-world context of design practice (Maffin Reference Maffin1998; Cash, Hicks & Culley Reference Cash, Hicks and Culley2013; Gericke, Meißner & Paetzold Reference Gericke, Meißner and Paetzold2013; Hodges et al. Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppert2017). Using a qualitative interview methodology, this research explores the experiences of practicing designers interacting with their design contexts, to examine how contextual factors helped or hindered their ability to resolve design trade-offs.
1 Background
Trade-offs are one of the most basic characteristics of engineering design because of the variety of different objectives, value systems, needs and preferences that influence design decisions (Fischer Reference Fischer2018). The most critical decisions are likely to involve complex trade-offs subject to many uncertainties that cross disciplinary and functional boundaries in the organization (National Research Council 2001). It has been argued that reasoning in trade-off situations requires breadth in multidisciplinary knowledge (Quintana-Cifuentes & Purzer Reference Quintana-Cifuentes and Purzer2022). Experienced designers are skilled at identifying the “pros and cons” of design situations (Crismond & Adams Reference Crismond and Adams2012) and recognize trade-off decisions as an important aspect of their practice (Goldstein, Adams & Sóenay Reference Goldstein, Adams and Sóenay2018).
In design and new product development, approaches to decision-making in trade-off situations have typically been prescriptive. For example, quantitative approaches such as tornado charts and trade-off rules can help product development teams understand and manage the interactions between different internal and external factors to inform critical business and design decisions (Ulrich, Eppinger & Yang Reference Ulrich, Eppinger and Yang2020, pp. 384–387). Similarly, decision matrices and selection charts – Pugh’s method (Reference Pugh and Hubka1981) being a prime example – can aid designers in systematically assessing the extent to which different solution concepts meet (often conflicting) requirements and constraints to identify promising solution concepts and eliminate unsuitable ones (Pahl et al. Reference Pahl, Beitz, Feldhusen, Grote, Wallace and Blessing2007, pp. 106–109). Once the nature of the solution is more clearly understood, quantitative modeling of requirements, constraints and solution performance enables more precise mathematical approaches such as optimization, where decisions are framed in terms of maximizing an overall preference function reflecting various performance goals with respect to a set of design constraints (Otto & Antonsson Reference Otto and Antonsson1991). The objective is to identify an optimal solution or a set of Pareto optimal solutions (Rafiq Reference Rafiq2000).
A limitation of these approaches for resolving trade-offs is that they can only operate on parameters that have been explicitly included in the model. Yet, rather than accepting trade-offs as a given and seeking optimal solutions under a fixed set of constraints, designers may resolve trade-offs by manipulating the structure of their design spaces, either by altering the constraints bounding their design space or by changing the parameters used to define it (Stacey & Eckert Reference Stacey and Eckert2010; Nickel et al. Reference Nickel, Duimering and Hurst2022). Importantly, prescriptive trade-off analyses often focus on measurable quantities and depend on the validity of underlying assumptions and available data (Ulrich et al. Reference Ulrich, Eppinger and Yang2020, Reference Ulrich, Eppinger and Yangp. 388). Yet, designers work at the nexus of complex socio-technical networks of other actors and interfacing technical systems (Sosa & Gero Reference Sosa and Gero2005) that influence the design process (Bucciarelli Reference Bucciarelli2002; Hodges et al. Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppert2017, Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppertp. 68). Designers must coordinate interdependent inputs of knowledge, information, goals and constraints from the socio-technical networks in which they are embedded, and must often make decisions under incomplete information, relying on their experience and skill to identify those parameters (Gero & Kannengiesser Reference Gero, Kannengiesser, Van Leeuwen and Timmermans2006). Requirements ambiguity and task interdependence create challenging coordination demands as decisions made in one part of the network affect decisions and problem-solving activities in other parts (Duimering et al. Reference Duimering, Ran, Derbentseva and Poile2006).
Trade-offs result in part from how the designer has formulated (Cross Reference Cross, Newstetter, Eastman and McCracken2001) and framed the design problem (Schön Reference Schön1983; Kelly & Gero Reference Kelly and Gero2021). Design goals and constraints may change as the designer interacts with their context and their understanding progresses (Dorst & Cross Reference Dorst and Cross2001). Some trade-offs may be apparent at the start of a design project, such as those due to conflicting goals of different stakeholders (Jonassen, Strobel & Lee Reference Jonassen, Strobel and Lee2006). For complex or novel designs, however, the designer may not be aware of, or fully comprehend, all the factors influencing performance at the outset, or when they make design decisions. Many relevant variables and constraints may be latent initially and trade-offs only emerge as the design process unfolds, as designers make decisions about their design problem and those decisions interact with aspects of the design context. As these decisions compound, previously unknown or uncertain aspects of the context may become highly relevant to future decisions, and the combination of these contextual influences and the designers’ decisions may lead to new trade-offs between design goals that the designer must contend with. Some trade-offs may even emerge later after the finished design is implemented in its operational environment (Goel & Pirolli Reference Goel and Pirolli1992), and discrepancies between the designer’s prior assumptions and the objective situation are discovered (Schön Reference Schön1988). Finally, interactions with contextual factors may also provide the necessary information, insight, or inspiration for successfully managing trade-offs. Resolving trade-offs often requires significant creativity in problem formulation, problem framing and solution development and interactions with context stimulate and support creativity (Glaveanu et al. Reference Glaveanu, Lubart, Bonnardel, Botella, De Biaisi, Desainte-Catherine, Georgsdottir, Guillou, Kurtag, Mouchiroud, Storme, Wojtczuk, Zenasni and Kalbfleisch2013; Abraham Reference Abraham2022).
In summary, the outcome of a design project is influenced by the situation in which the design process occurs, and by the designer’s interpretation of that context (Coyne & Gero Reference Coyne and Gero1985; Moriarty Reference Moriarty1994; Simonsen et al. Reference Simonsen, Svabo, Strandvad, Samson, Hertzum, Hansen, Simonsen, Svabo, Strandvad, Samson, Hertzum and Hansen2014). A designer’s interaction with, and reflection upon (Schön Reference Schön1983), contextual factors are key to the design process (Gero & Kannengiesser Reference Gero and Kannengiesser2004, Reference Gero, Kannengiesser, Van Leeuwen and Timmermans2006). Yet, design research has typically abstracted away the complexities of the design context (Dorst Reference Dorst2008), prompting calls in the literature to study design phenomena in the context of real-world design practice (Hubka & Ernst Eder Reference Hubka and Ernst Eder1987; Bucciarelli Reference Bucciarelli1988; Maffin Reference Maffin1998; Cash et al. Reference Cash, Hicks and Culley2013; Gericke et al. Reference Gericke, Meißner and Paetzold2013; Hodges et al. Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppert2017; Abraham Reference Abraham2022). This exploratory research aims to investigate engineering trade-off management in context, specifically the role that contextual influences play in the emergence and evolution of design trade-offs, and how designers interact with their context as they attempt to navigate and resolve trade-offs.
2 Methodology
We conducted an exploratory qualitative interview study to investigate designers’ experience managing design trade-offs and the influences of their context on the design process. The study received ethics approval from the institution’s review board.
2.1 Participant recruitment
Nine designers were interviewed for this study, recruited from the engineering alumni of a major university using a combination of search and referral strategies. They were trained to either the undergraduate or graduate level and had from 3 to 11 years of engineering design experience in various organizational settings, including two student design teams, two technology start-ups, two design consultancies and three large firms with several thousand employees. They were working on design projects in the robotics, automotive, building systems, software and healthcare industries. Table 1 provides a breakdown of the interviewees’ backgrounds and the specific design problems they discussed in the interviews. The recruitment criteria were that participants were currently engaged with some form of engineering design and had recently confronted a trade-off, described as a conflict between two or more design goals on a project. Prior to their interview, participants were sent an email asking them to prepare by recalling this situation, and how the trade-off was resolved. The nine interviewees discussed 11 distinct trade-off situations (P4 and P5 each discussed two), across various engineering disciplines and design domains, ranging from vehicle component design in a university’s student design team to software design in a large metropolitan public transit system.
Table 1. Summary of participant details and design problem domains investigated.
2.2 Interview protocol
Semi-structured interviews investigated how aspects of the socio-technical context of specific design projects influenced design trade-offs, and either helped or hindered designers’ ability to resolve trade-offs. We used an adaptation of the echo interview method originally proposed by Bavelas (Reference Bavelas1942) to identify contextual influences affecting behavior from the perspective of the individuals involved (Cunningham Reference Cunningham2001). Previous research has used the echo method to study interactions in socio-technical networks of interdependent tasks and roles in requirements engineering, manufacturing and other domains (Barthol & Bridge Reference Barthol and Bridge1968; Duimering et al. Reference Duimering, Purdy, Safayeni and Road1998; Safayeni et al. Reference Safayeni, Duimering, Zheng, Derbentseva, Poile and Ran2008).
Interviews were conducted using video conferencing software and lasted between 60 and 90 minutes each. Interviews were recorded and automatically transcribed, and transcripts were later corrected manually for errors or inaudible portions. Each interview was conducted by a minimum of two of the coauthors, with seven of the nine interviews conducted by all three coauthors.
In the first portion of the interview, participants were asked to describe a recently encountered trade-off situation, and how they managed the situation, in their own words. Trade-offs were defined as “a situation where any gains in one area or aspect of the design are associated with sacrifices in other area(s) or aspect(s).” The researchers took notes on any elements of the social or technical context that influenced the design trade-off as the interviewees described the situation.
To further specify the socio-technical context, interviewees were asked to identify any individuals or groups, as well as various technical, structural or other nonhuman aspects of the situation that they felt had influenced their decisions and approach to resolving the design trade-off. Together with the influences already mentioned during the description of the trade-off situation, these constituted the socio-technical network relevant to the design trade-off situation.
To apply the echo method, the remainder of the interview was spent attending to each influence in detail, by asking the participant to provide specific and concrete examples of how that influence made managing the trade-off easier, and how it made it more difficult. Specifically, for influences representing individuals or groups, interviewees were asked for examples of these others’ behaviors that were either helpful or unhelpful with respect to resolving their trade-offs. For influences representing nonhuman contextual elements, examples of attributes or characteristics that were helpful or unhelpful for managing the trade-offs were requested.
In all, for each trade-off case, interviewees described detailed interactions with 6–10 individuals, groups, and other contextual elements that constituted the socio-technical network influencing trade-off resolution. Our dataset consisted of an extensive array of 325 specific excerpts describing socio-technical behaviors and influences on trade-off resolution. These excerpts constituted the unit of analysis in this study and were the subject of coding as described next.
2.3 Thematic analysis
A thematic analysis (TA) following Braun and Clarke’s (Reference Braun and Clarke2006) guidelines was used to analyze the transcripts, combined with Kurasaki’s (Reference Kurasaki2000) procedure to generate a codebook from the dataset. Coding and theme development were conducted by the three coauthors as follows. Initial coding of excerpts and identification of themes were performed by the first author using the Dedoose software package (DeDoose 2021). The themes and underlying excerpts were iteratively reviewed and refined by the three co-authors until the themes met the criteria of internal homogeneity (coherence within a theme) and external heterogeneity (clear distinctions between themes) (Patton Reference Patton2015).
The transcripts were initially coded based on whether interviewees described influences as helping or hindering their ability to manage and resolve trade-offs. These codes were applied both to their explicit responses to interview questions about helpful and unhelpful influences, and to other influences mentioned in their initial descriptions of the trade-off situations.
The next rounds of coding were inductive. The helpful or unhelpful excerpts were reviewed and annotated to identify common topics or patterns of behavior mentioned by different interviewees. For example, annotations included representative quotes or brief descriptions of typical behaviors. These annotations were then iteratively combined and revised into a set of generic topical codes. Although the resulting codes were generic, the coding process was interpretive and context-dependent, based on a detailed understanding of each design situation described by interviewees. At the end of this coding stage, 93 unique codes were created and applied within the dataset across the 325 specific excerpts.
Finally, nine broader themes were developed using an iterative affinity diagramming process (Haskins Lisle, Merenda & Gabbard Reference Haskins Lisle, Merenda and Gabbard2020) to identify groups of codes related to similar contextual influences. Themes were named to reflect the common ideas connecting the underlying codes. As the themes were topical in nature, and an excerpt may discuss more than one topic, a single excerpt could be related to multiple themes. Figure 1 shows an example of how raw transcript excerpts were translated into topical codes, and from codes into themes. All 93 codes and their assignments to themes are listed in the Appendix. In the list, we also note instances when a code is assigned to more than one theme.
Figure 1. Process of coding and theme generation with examples.
Data complexity and the context-dependent interpretive coding process precluded the use of inter-rater reliability checks by independent raters to validate results (Ryan & Bernard Reference Ryan and Bernard2003; Patton Reference Patton2015, pp. 965–966). The iterative coding procedure described above was designed to support the reliability, coherence and consistency of the findings, but the researchers recognize the potential effects of their positionality on the interpretive coding process in qualitative research (Walther, Sochacka & Kellam Reference Walther, Sochacka and Kellam2013). The team consists of researchers with diverse academic and professional experience in design, mechanical and electrical engineering, management and organizational behavior, socio-technical analysis, decision-making, cognition and other domains. This background inevitably shaped their perspectives, influencing aspects such as the methods chosen, research design, data collection, analysis and interpretation of findings. With respect to the study goals, the team’s combined engineering and social science expertise helped to establish rapport with the interviewees and to understand both the technical issues involved and the broader social and organizational influences affecting each trade-off. This in turn facilitated asking probing interview questions for the participants to clarify in depth how specific contextual influences supported or hindered their ability to resolve trade-offs.
3 Results
Designers identified many influences that affected design decision-making in trade-off situations. These included the intended users of the design, other designers and subject matter experts, organizational structures and norms, resources and interfacing technologies. Nine topical themes were identified, reflecting designers’ perceptions of how different aspects of context helped or hindered their attempts to resolve design trade-offs. Definitions for each theme are given in Table 2. The values in Table 3 show how many unique codes (out of the 93, listed in full in the Appendix) were present in each interview, broken down by theme. Table 3 also reports the number of excerpts that were sourced from each interview, in brackets. These counts indicate that the emergent themes were well-distributed across the various trade-off situations, with each theme represented by coded excerpts from a minimum of five of the nine interviewees. However, we do not make any further claims based on these counts or their distributions, due to the qualitative nature of this study.
Table 2. Definitions of topical themes identified within the data
Table 3. Distribution of unique code applications (and excerpts) for each theme, by participant
What follows is a detailed description of each theme, with representative quotations from the designers.
3.1 Degree of complexity
Complexity in design trade-offs refers to the number of different stakeholder goals, requirements, variables and constraints that interact or conflict with one another and affect design decisions. In general, influences that increased complexity were perceived as having a negative effect on trade-off resolution. Interviewees noted that having more design goals and constraints increased the time needed to reach a solution and made it more difficult to attend to all relevant information, to account for all potential consequences of design decisions, or to predict whether a decision would help or hinder trade-off resolution.
In contrast, influences that limited design decision options made it easier to reach a resolution, so long as one of those options led to a viable solution. For example, a medical equipment designer explained how industry regulations helped reduce complexity by limiting the options available:
“The [constraint on finishes] actually […] does […] make things easier just because you have less choice. You pretty much have, like, 3 different finishes that are approved in that regard. In more consumer spaces you have so many options on finishes that it can get a bit overwhelming, whereas in the medical industry it’s 1 of these 3, and they’re all for different things. So that actually did make it easier, in a weird sort of way.”
To reduce complexity, designers and other stakeholders sometimes focused on a specific portion of the design rather than dealing with the full complexity of the project. But ignoring other influences on the problem might introduce new trade-offs and reduce the degrees of freedom needed to resolve them. One interviewee described how a team responsible for need identification and project ideation in their organization worked “in a silo … not thinking about other parts of the system and how to go forward with it.” By taking decisions in isolation, they fixed certain design parameters without considering the effects on other interdependent aspects of the design. Although this approach reduced local complexity for them, the factors they ignored still influenced and constrained the success of the overall design project. The interviewee was now tasked with identifying and resolving trade-offs resulting from these isolated decisions, without having the flexibility to alter the goals or requirements involved in these trade-offs.
3.2 Limitations of designer agency
Designers have limited agency to make design decisions. Other stakeholders or contextual constraints limit both the decisions that a designer can make and the available options for those decisions. The excerpts assigned to this theme described how factors outside the designer’s direct control influenced the options available to them to resolve trade-offs.
Influences that restricted designers’ flexibility and made trade-offs more difficult to manage were typically described as unhelpful. For example, designers found it unhelpful when project resources and limits, like budget, manpower and deadlines, prevented them from approaching a design conflict with the flexibility they felt they needed. Project resource constraints can sometimes be negotiated and relaxed with enough effort and rationale. However, the designers also struggled with contextual influences that were completely nonnegotiable, driven by regulations, local socioeconomic conditions or other contextual factors. For example, a medical equipment designer discussed the difficulties of satisfying design goals while complying with safety regulations in the medical industry: “Just designing [in] a space with so many rules and so many limitations is more difficult than if this was a consumer product.”
Limited designer agency was described as helpful for managing trade-offs when externally imposed constraints reduced the complexity or uncertainty of design decisions. By taking certain decisions out of the designer’s hands, clearly defined limits can reduce both the number of design variables under consideration and the cognitive load of attending to different variables. For example, for one designer, strict industry regulations reduced uncertainty about the safety of their design: “… They have very well-defined requirements for what is a pinch and what is not a pinch. So it did save us the sort of wondering, ‘Is this design okay? … Do you think people will think it’s okay?’ … It’s very obvious if it’s okay or not.” Such helpful influences were quite rare, however, and the designers mostly discussed limits on their decision-making agency as an unhelpful influence, which reduced flexibility and the options available to resolve trade-offs.
3.3 Operationalizing goals
Requirements operationalize the needs and goals that the design project aims to fulfill. For the designer to develop solutions that meet these needs and goals, they must be translated into criteria that can be evaluated in terms of either subjective judgements or objective performance measures. Choices about how goals are operationalized can result in trade-offs between conflicting requirements, even when the goals themselves may not be directly antagonistic (Nickel et al. Reference Nickel, Duimering and Hurst2022).
The designers noted that it was particularly unhelpful when stakeholders did not communicate their needs and goals clearly enough for the designer to operationalize them accurately. In some cases, the process of translating needs and goals into requirements was performed by another department, adding a degree of organizational separation between the designer and the client or end user of the design. When requirements were unclear, in conflict with one another, or simply could not be met, this separation added further challenges to resolving trade-offs, since designers were unable to interact directly with clients to determine their real design goals. An interviewee expressed their frustration as follows: “I’d say that [the] process [of gathering and operationalizing requirements in a separate department] in itself was probably detrimental because there are some of these constraints that, if it had been the engineering team gathering them from the start, probably would have been questioned earlier.”
Direct communication with stakeholders helped designers to better understand their underlying needs and goals, which ensured requirements were operationalized appropriately and made it easier for designers to understand and resolve trade-offs. Operationalizing goals differently sometimes allowed designers to circumvent apparent constraints on their problem. For example, a consulting designer described how a trade-off involving the cost of certain robot components was resolved after the CEO re-operationalized goals in terms of overall cost for the company:
“It helped clarify it a little bit when it was framed by the CEO as ‘okay, for every week that we delay this design, for every week that these robots are not out in the field, it’s going to cost us about 25000 dollars’. So, it made it a lot easier to kind of standardize things into a single variable. Which is not necessarily the cost of this robot. It’s the cost to the company.”
Interestingly, another designer observed that because their original requirements and constraints were poorly defined and operationalized, it encouraged them to explore their problem in more depth: “…It forced us to ask some more questions. We actually probably found out more than we would have if they just had a really well written set of constraints.”
The influences classified in this theme reflect the importance of understanding the true aims of project stakeholders when attempting to resolve trade-offs between the operationalized requirements. The designers interviewed all worked to requirements that were derived from stakeholders’ underlying goals and needs, but the operationalization process did not always result in requirements that completely aligned with those goals and needs. By operationalizing the goals and needs differently in these cases, designers could sometimes mitigate or avoid trade-offs that prevented them from reaching viable solutions under the original requirements.
3.4 Different perspectives on design
In the process of operationalizing project goals into requirements, the designers dealt with different stakeholders with different perspectives and interpretations of the design problem, and different design priorities. This subjectivity led to conflicting requirements and goals, but also to unexpected insights and changes to the problem or solution.
The designers discussed the difficulties of reconciling conflicting interpretations of the problem, and ensuring that different stakeholders’ needs, goals and priorities for the design were fairly represented and given appropriate weight. For example, the designer of a speech accessibility device explained that they had been “cautioned against taking all the advice from speech language pathologists… because … the speech language pathologist and the end users, the people with disabilities, have conflicting goals in terms of what they’re looking for in a product.” To understand and reconcile the different priorities involved in the project, the designers spoke about the value of empathizing with other stakeholders’ goals and priorities.
The designers noted that effective communication helped to harmonize the goals and perspectives of different stakeholders, and to obtain buy-in when design decisions conflicted with stakeholder perceptions. As one designer explained, “…before you go down an expensive avenue, you have to convince a lot of people and show them that it should work, it will definitely work.”
Designers also described how different stakeholder perspectives sometimes helped to resolve trade-offs. For example, they discussed the value of brainstorming and other group ideation methods, where sharing new perspectives on the design problem helped them better understand the true constraints on the situation. For one designer, it was very helpful to include the intended operator of the device they were designing in a brainstorming session: “…She vetoed a lot of ideas right away that, to us seemed very mechanically sound, but to her, they were just not usable.”
Design requirements are driven by different stakeholders with potentially divergent views, needs and goals that lead to challenging trade-offs. Influences that made designers aware of these differing and conflicting priorities made it easier to account for them, and to resolve their trade-offs.
3.5 Consequences of previous design decisions
The designers mentioned they often encountered the consequences of previous design decisions that were made earlier in the design project, or in the design of an interfacing component or system. They found it more difficult to identify solutions that resolved trade-offs when the available design options were limited by previous decisions. For example, an electronic device designer explained that a prior decision to pursue a specific product certification limited their choice of components to those that met this certification:
“[The certification] had an unhelpful influence in that it limited the selection that we had to pick, as building blocks… So if we had a feature that we needed, and there were 100 parts on the market that did it, maybe 5–10% of those have this rating and so now you can only select from that.”
Designers found it helpful when prior decisions retained design flexibility to avoid trade-offs in later decisions. For example, the designer of an assistive device for people with speech impairments appreciated an earlier decision to design their application for tablet devices, rather than smartphones, because the larger screen size avoided trade-offs with respect to the amount of information that could be presented:
“[The choice of] optimizing for [tablet allowed us to] have both core and fringe [vocabulary], because those are a lot of different words to show on one screen.”
As design decisions accumulated over time, each potentially added constraints that could reduce the degrees of freedom available for later decisions. Such dependencies could even extend beyond a designer’s current design project, since some of the limits discussed by interviewees were due to decisions made on earlier projects that affected the current design.
3.6 Ability to revise previous design decisions
Designers often discovered trade-offs when they were already part-way through the design process. In some cases, the conflict only became apparent after the designers had a solution that was fully verified and validated against the original requirements. In these situations, the designers tried to revise previous decisions to circumvent a trade-off or relax constraints, by reconsidering the rationale or underlying goals that drove those decisions. They found it particularly helpful when they had the freedom to revisit and alter previous decisions, without being limited by organizational procedures and norms. For example, a consulting designer appreciated having the freedom to operate “…with kind of carte blanche authorization …to keep or change any of that that I saw fit in my judgment. We knew that [the robot] had to drive on a sidewalk and it had to fit food in it. Everything else [was] very loosely defined.”
The designers found it unhelpful when efforts to revise earlier decisions were thwarted by other project stakeholders, who had their own vested interests in not allowing decisions to be altered. For example, one designer described how an earlier decision to contract with a particular software vendor limited their ability to satisfy a new design requirement. Attempts to revisit the contract were thwarted by the vendor who was more interested in protecting their contract than resolving the trade-off:
“…We met with [the contractors] once a week for like a month there. Most of the conversation was them trying to sell us that their system could do what [project] was trying to do. Which is a fine idea, I’m happy to explore (that), maybe their system could do [the task] better. But it was a lot of that, because … they have a financial interest in place, so they wanted to make sure that they’re not going to get cut out.”
Interestingly, a consulting designer noted that they too were personally reluctant to revisit their own previous design decisions when a change in stakeholder requirements led to new constraints being introduced late in the design process. They described how they were “kind of questioning ourselves on why we were getting so angry and that kind of led down the rabbit hole. Okay, we’re just really attached to this design and it’s not that the new constraints are stupid, it’s just that our design doesn’t meet it anymore and that’s okay.” This designer also noted that norms related to their organization’s sequential design process discouraged revisiting earlier decisions: “So, that reluctance to go back a step and start sketching again is kind of built into our environment … and our company culture.”
Revisiting previous design decisions potentially allowed designers to navigate around trade-offs and identify better solutions, but revisiting a prior decision was typically itself a trade-off that sacrificed project time and resources. Whether the time and resources were worth it depended on subjective assessments of the potential performance gains relative to the goals in the original trade-off.
3.7 Domain expertise
Given the complexity of design problems and different stakeholder perspectives involved, the designers sought information to help them understand the relevant constraints, interactions between constraints, and their effects on design performance in the intended operating environment. The designers depended both on their own expertise and familiarity with the design domain, as well as the experience and advice of others.
Interviewees frequently mentioned that it was helpful to consult with other domain experts when the demands of the project did not align with their own areas of expertise, as illustrated by a manufacturing designer:
“[If] we were confused between two sensors, which one would work better, which one would last longer, which one is more financially sustainable, which one is more repetitive, which one has higher accuracy, which one is more precise. So [for] all of these things we brought in someone – a few people – who are able to answer these questions.”
Influences described as unhelpful included the designers’ own lack of domain expertise, and the inexperience of other stakeholders. For example, an interviewee described how another stakeholder could not provide the information needed for a decision: “They were not confident whether the method would work in an ideal manufacturing situation. They were unsure, because they [lacked] professional experience, or experience in how a design can be done into an actual product.” In this instance, both the designer and the other person lacked sufficient domain knowledge to provide precise information about design constraints that was needed to determine the viability of a solution concept.
In some cases, differing expertise led to disagreements about how constraints affected the viability of proposed design solutions. In one example, an interviewee believed that a certain constraint existed, but another stakeholder proposed a solution that fell outside of that limit. In this instance, the interviewee regarded what they believed to be an infeasible solution as an unhelpful distraction. The opposite situation also occurred, where others thought constraints existed which the designer did not believe to be present. For example, one designer described solving a difficult problem after previous attempts by other designers had already failed. On one hand, consulting the previous designers was helpful because their prior failed designs provided information necessary to identify the real limits of a solution approach and make decisions accordingly: “[The previous designers] knew what went wrong, so those inputs were valuable from time to time, and we always reviewed what we were doing with them, just to make sure that we are not making the same mistakes.” On the other hand, it was viewed as unhelpful when the previous designers expressed negativity after an initial failed attempt, and suggested a solution was impossible: “The negativity that comes with a failure always impacted [us]. Because now you have someone who is telling you this is impossible to do, but you are trying to make it work. It was an emotional setback, not financial or anything else, but it does affect you.”
Domain expertise is related to the earlier theme of different stakeholder perspectives. The key difference is that although stakeholders sought to influence design decisions based on their different goals and values, designers had to integrate these diverse perspectives into real designs based on what was actually possible, given the physical, regulatory, or other constraints affecting their decisions. Domain expertise was viewed as a source of objective information about the true nature of relevant design constraints, which helped to resolve conflicts between different stakeholder perspectives and to clarify the extent to which their different goals could be satisfied.
3.8 Prototyping, simulation and testing
Prototyping, testing and other forms of simulation were commonly used by the designers to clarify how constraints affected designs and to reduce uncertainty about the consequences of decisions. Prototyping allowed the designers to determine the objective performance of a solution faster and with fewer unintended consequences than by observing the design’s performance after the project was completed, thereby allowing the designer to modify the solution based on that information:
“We generally have like a mentality of you prototype early and you prototype often, and that is definitely a useful mentality for us. It finds issues very quickly.”
High fidelity prototypes were not always needed. As one designer explained, “We had … scissors and cardboard cut-outs and we … just made very bad, quick prototypes of different ideas to quickly iterate through things that would and wouldn’t work.”
Such methods allowed the designers to fill gaps in their own understanding that were not filled by consulting domain experts, and to explore novel solution approaches to resolve the trade-offs they encountered. One interviewee noted the value of using low fidelity prototypes to help communicate solution concepts to stakeholders, which helped predict the viability and performance of potential designs, saving valuable time in the ideation phase:
“We would go to [the device’s eventual operator] fairly often to get feedback. So really her main contribution was we would give her very basic prototypes [and ask] ‘how would you use this? Would that be acceptable to you?’. She was very good about giving incredibly articulate feedback. She would never say, like, I don’t like things. She would give you exactly why she didn’t like it.”
Designers also identified some limitations and negative aspects of prototypes and simulations. For example, the designer of an engine component suggested that testing and simulation was insufficient to predict real performance, and doubted the ability of a simulation to fully represent the real operating conditions or capture all the factors affecting performance:
“We were unable to test it in real world conditions. We were able to simulate it on a 3D space or a computer software, but it would have been much better if it was available on an automobile itself, so that we can test drive it and have a look at how it works, or if it doesn’t work. Because … unless you have it as a product, it’s going to be much difficult to understand how it is going to work.”
A consulting designer spoke about emotional barriers to starting over with low-fidelity and imperfect prototypes, after a change in requirements forced them to abandon a nearly finished design: “I think it’s difficult when you go from an almost finished design to something made out of cardboard and very rough sketches and CAD. It’s difficult to see it as better, because it looks so much worse.”
In general, though, the designers valued the processes of simulation, prototyping and testing as aids for resolving trade-off situations by helping them to understand better how different factors and constraints interacted in a design, allowing them to gauge more closely the real performance of a solution concept and to assess the potential viability of different design alternatives.
3.9 Timeliness of information availability
When making design decisions, the designers could only utilize the information available to them at the time of the decision. The designers discussed how the timeliness of information availability influenced the difficulty of resolving trade-off situations.
Some designers were frustrated by requirements that were introduced late in the project, creating new trade-offs between the new requirements and the original goals. They also did not appreciate discovering late in a project that constraints which had prevented trade-off resolution were actually more flexible than previously assumed. For example, one interviewee spent significant resources investigating ways to meet a standard company requirement, only to be granted an exemption later:
“If I knew what we were going to change our scope to, I would have ignored everything before [system name redacted] and just said, okay, let’s start [there]. Do not go into all these weeds and understand what’s happening…”
Long time-lags between making decisions and learning the effects of those decisions on solution performance also made it difficult to resolve trade-offs, as the design project had progressed and revisiting an earlier decision would require revisiting subsequent decisions as well.
“With hardware it’s interesting because there’s a long lead time between making engineering decisions and seeing how those pan out.”
On the other hand, it was helpful when designers knew about limits and conflicts early in the process as it allowed them to intentionally design to avoid those limits: “The action that the manufacturer … took was to notify me early enough, before the commencement of the whole manufacturing process. So he notified me quickly enough and I was able to make adjustments and issue another revised design.”
Overall, the designers preferred as much information as possible so they could predict the consequences of their decisions, whereas being forced to make “blind” decisions with incomplete information made it more challenging to resolve trade-offs. As previously discussed, they often consulted experts and used prototypes to obtain this information, as early awareness potentially allowed them to manage or avoid trade-offs before becoming constrained by too many other decisions.
4 Discussion
There have long been calls to investigate design phenomena in relation to context (Hubka & Ernst Eder Reference Hubka and Ernst Eder1987; Maffin Reference Maffin1998; Cash et al. Reference Cash, Hicks and Culley2013; Gericke et al. Reference Gericke, Meißner and Paetzold2013; Hodges et al. Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppert2017; Rodriguez-Calero et al. Reference Rodriguez-Calero, Coulentianos, Daly, Burridge and Sienko2020; Abraham Reference Abraham2022), but the role of context in resolving trade-offs has not been well understood. Improved understanding of how designers interact with context to manage trade-offs is also of heightened importance as designers face increasing challenges balancing performance, economic, sustainability and other conflicting goals related to their designs (Hahn et al. Reference Hahn, Figge, Pinkse and Preuss2010; Gibson Reference Gibson2013; Grêt-Regamey et al. Reference Grêt-Regamey, Brunner, Altwegg, Christen and Bebi2013; Fogli et al. Reference Fogli, Piccinno, Carmien and Fischer2020).
Our study identified nine themes reflecting designers’ perceptions of the effects of context on design and their ability to manage and resolve trade-offs. Our interview method encouraged designers to identify and discuss specific contextual influences that they perceived to help or hinder trade-off resolution. The trade-offs they described varied in problem scope and were situated in a wide range of industries and organizations. Each theme was represented across a majority of the interviews, but further work is needed to evaluate their generalizability and relative importance in different design situations. Each of the nine themes raises potential questions for future research, to better understand specific ways they affect design behavior and decision-making, and their implications for improved strategies to manage and resolve design trade-offs in practice.
Collectively, our findings provide a rich account of how designers learn about design requirements and constraints and how these variables interact to create trade-offs. Working through trade-offs entails managing the degrees of freedom available to the designer, for example by seeking better information to understand precisely the degrees of freedom available, seeking increased degrees of freedom to relax or even bypass trade-offs, or avoiding decisions that might limit future degrees of freedom. The next two sections highlight two key threads that emerge from our findings: 1) how designers learn about their design situation and the contextual influences affecting trade-offs, and 2) the path dependencies that arise and compound within trade-off situations as a result of the interplay between decisions the designers make and contextual factors. We conclude with a discussion of the goals and challenges of conducting rigorous qualitative research in design and consider questions of assessing sample size sufficiency in qualitative research.
4.1 Learning while managing trade-offs
Design is an ill-defined problem-solving activity, with goals, solution requirements and other constraints characterized by varying degrees of ambiguity and uncertainty (Goel & Pirolli Reference Goel and Pirolli1992; Jonassen Reference Jonassen2000). Designers must often make decisions with incomplete knowledge and information (Bucciarelli Reference Bucciarelli1994, pp. 123–124), and without fully understanding the effects of their decisions on other design variables. They may be aware of some trade-offs between conflicting constraints at the beginning of a project, but only discover others later, after a solution has begun to take shape. Our results suggest the designers devoted considerable effort to learning about their design problem (Lawson & Dorst Reference Lawson and Dorst2009, pp. 32–34), seeking information and prototyping to understand better the variables and constraints involved, and the interactions between them (Cash & Kreye Reference Cash and Kreye2018). Although they had limited agency to modify certain constraints affecting their designs, better understanding of the constraints helped them to clarify the degrees of freedom available in trade-off situations, and sometimes led to new ways of operationalizing requirements that bypassed trade-offs altogether. Further work is needed to understand the role and effectiveness of different learning strategies for different design problems and domains.
Domain knowledge and expertise helped the designers predict the influence of contextual factors on the design in advance and identify which decisions would impose significant constraints on other design variables. This helped them to make design decisions that maintained the degrees of freedom needed to avoid future trade-offs. The designers valued such expertise and experience highly, and frequently consulted with other domain experts to fill gaps in their own understanding. These consultations were surrogates for testing the actual performance, allowing the designers to make decisions that were more informed earlier and without expending further project resources. Findings here are in line with Aurisicchio, Bracewell and Wallace (Reference Aurisicchio, Bracewell and Wallace2010), who highlight how interactions with colleagues provide designers with valuable “process information,” such as analyses of solutions and their evaluation against requirements.
The designers also consulted with different project stakeholders, who approached the design from different frames (Schön Reference Schön1983; Kelly & Gero Reference Kelly and Gero2021) and had different knowledge about the problem at hand. On one hand, different stakeholder perspectives were a major source of trade-offs, and frustrated the designers as they sought to balance conflicting goals and requirements. On the other hand, learning more about stakeholders’ perspectives and situational constraints helped the designers identify different ways of operationalizing requirements that sometimes increased their degrees of freedom and avoided trade-offs.
Interviewees also valued the use of prototypes and simulations to learn about the effects of contextual factors on the performance of their solutions. Such models allowed them to explore the performance implications of different options quickly and at low risk, rather than discovering these implications in the intended operating environment after finalizing the design. Models reflect the influences of a design’s functional environment on performance with varying degrees of fidelity (Tiong et al. Reference Tiong, Seow, Camburn, Teo, Silva, Wood, Jensen and Yang2019). As a result, their value depends on how well the designer understands the contextual influences on their design and can properly model those influences, such that simulation results accurately represent real design performance (Rodriguez-Calero et al. Reference Rodriguez-Calero, Coulentianos, Daly, Burridge and Sienko2020). If a designer is not aware of an influence or models it incorrectly, the simulation results may be biased and misrepresent true performance. Further research is needed to understand how designers balance this contradictory need to use prototypes to discover the influences of context on solution performance while also needing to understand the solutions’ functional context to set up valid prototypes and tests (cf. Camburn et al. Reference Camburn, Viswanathan, Linsey, Anderson, Jensen, Crawford, Otto and Wood2017, p. 25; Maier, Eckert & John Clarkson Reference Maier, Eckert and John Clarkson2017; Petrakis, Wodehouse & Hird Reference Petrakis, Wodehouse and Hird2021). The designers in our study used an iterative approach (Camburn et al. Reference Camburn, Viswanathan, Linsey, Anderson, Jensen, Crawford, Otto and Wood2017) to learn through design, beginning with low-fidelity prototypes and increasing fidelity as they gained confidence in their knowledge and ability to model the relevant contextual influences accurately.
4.2 Path dependency of decisions in trade-off situations
Our findings also highlight the importance of path dependencies in design trade-off situations, and further work is needed to better understand how the sequential distribution of decisions over time limits the ability of designers to resolve trade-offs. Design problems are temporally situated, such that designers are constrained by past decisions over which they may have no direct control and try to avoid making choices that will limit the options available in future decisions.
Designers cannot always predict the consequences of their own decisions, due to ambiguously defined requirements, design complexity and unknown interdependencies between decision variables. Despite efforts to learn about their design problem, the interviewed designers were often forced to make early decisions without sufficient knowledge and information to predict all the consequences for later decisions. As design problems increased in complexity, it was difficult for them to attend to all aspects of the design simultaneously. Decisions under these conditions could have unexpected consequences later in the design process, introducing new constraints that create new trade-offs to be resolved, while reducing the degrees of freedom available to resolve those trade-offs. As decisions and constraints accumulated, the designers found it increasingly challenging to create solutions that satisfied all the requirements of the project.
When new trade-offs were identified, the designers responded by attempting to revisit previous decisions to mitigate or avoid conflicts between different constraints. Strategies included reformulating the problem to operationalize goals differently, negotiating requirements, or restructuring their entire solution approach. Modifying the design space by revisiting previous decisions is a form of iteration, which has long been recognized as fundamental to the design process of expert designers (Dorst & Cross Reference Dorst and Cross2001). The designers in our study valued the freedom to iterate and revisit previous decisions, as this potentially allowed them to resolve trade-offs in ways that avoided sacrifices to the conflicting performance goals involved.
Since design is an inherently explorative process (Hay et al. Reference Hay, Duffy, McTeague, Pidgeon, Vuletic and Grealy2017; Hodges et al. Reference Hodges, Ruecker, Scaletsky, Rivera, Faller and Geppert2017), the sequence of design decisions does not necessarily align with the impact of those decisions on a trade-off being resolved. Decisions made earlier in the process may be more relevant to the trade-off than more recent decisions and revisiting earlier decisions may result in more successful and creative designs. However, design decisions are often a result of negotiation between organizational parties with different interests (Bucciarelli Reference Bucciarelli1988), and revisiting early decisions is often difficult and resource intensive. For the designers in our study, revisiting earlier decisions to resolve trade-offs was stymied by both internal and contextual factors. In some cases, the designer’s own emotional attachment to their design made them reluctant to abandon completed work and reconsider prior decisions. In other cases, organizational restrictions on project resources like cost and timeline, or decision-making and project management norms (e.g., a linear project structure) restricted designer agency, encouraged forward project inertia and discouraged backtracking.
4.3 Qualitative research in design
A potential limitation of the study is that the results are based on only nine interviews, raising the question of whether either a larger sample or study replication is needed to validate the themes identified. This concern leads us to a more in-depth discussion of the aims and challenges of qualitative research, including sample size considerations.
4.3.1 Goals and challenges
Concerns about the validity of results are consistent with assumptions of scientific evidence typical of quantitative research methods, but arguably do not reflect the goals and challenges of rigorous qualitative research. Strauss and Corbin (Reference Strauss and Corbin2008) go so far as to discourage the use of the term “validity” applied to qualitative research, precisely because of the inappropriate quantitative implications (pp. 301–302). The primary goals are not to validate a set of assertions or to obtain conclusive evidence to support claims that become more robust with a larger sample size or study replication.
We would argue that the primary goal of qualitative research is theory development, aimed at identifying relevant new themes and factors, or clarifying relationships between variables that were not previously recognized or sufficiently investigated in prior research. Qualitative research is inherently interpretive and exploratory in nature, aiming to reach a sufficient “conceptual depth” in understanding (Nelson Reference Nelson2017) that may, at best, lead to new research questions or tentative hypotheses to be investigated in future research. As such, even a sample size of one might be sufficient in certain cases to provide relevant insights that contribute to theory development (cf. March, Sproull & Tamuz Reference March, Sproull and Tamuz1991).
Conducting rigorous qualitative research is extremely time-consuming and challenging. In this study, it entailed hundreds of hours of iterative coding, delving deeply into findings, and interpreting evidence from multiple points of view. Although codes and categories should accurately reflect the available evidence (in this case, the descriptive information provided by interviewees), the inductive nature of coding and the large volume of information to be reduced imply that different interpretations and abstractions are possible depending on the goals of the study, and different themes and conclusions can result from the same set of data. In this study, the goal was not just a set of codes and categories reflecting the perceived information content in our data, but also theoretical convergence and coherence, where we attempted to identify themes reflecting the most prominent contextual variables relevant to trade-off resolution.
Such a coding process is certainly subject to potential errors of interpretation (including due to the coders’ positionality, as discussed in Section 2.3), and a different or larger sample of interviews might have led to somewhat different thematic categories. As such, the categories identified do not provide an exhaustive account of all the contextual factors affecting trade-off resolution or a theoretically complete explanation of the processes involved. However, this does not weaken the methodological rigor of the analysis or refute the theoretical relevance of the themes identified. They reflect potentially important contextual variables affecting the resolution of design trade-offs, which have practical relevance for design decision-making and design organizations and should be investigated further in future research.
4.3.2 Sample size sufficiency
The question of how to define and assess sample size sufficiency in qualitative research is worth considering. The idea of theoretical saturation proposed by Glaser and Strauss (Reference Glaser and Strauss1967) is probably the most common way of discussing sample size sufficiency in qualitative research, but there is much disagreement about both the meaning and relevance of saturation in the literature (O’Reilly & Parker Reference O’Reilly and Parker2012; Hennink, Kaiser & Marconi Reference Hennink, Kaiser and Marconi2017; Nelson Reference Nelson2017; Saunders et al. Reference Saunders, Sim, Kingstone, Baker, Waterfield, Bartlam, Burroughs and Jinks2018; Vasileiou et al. Reference Vasileiou, Barnett, Thorpe and Young2018; Braun & Clarke Reference Braun and Clarke2021). A recent review (Hennink & Kaiser Reference Hennink and Kaiser2022) found that qualitative studies in which authors attempted to determine saturation empirically are very rare (only 26 of almost 5000 screened articles), with no standard ways used to define saturation. Sixteen of the 26 were interview studies, and most “reached saturation between 9 and 17 interviews, with a mean of 12–13 interviews, despite using different approaches to assess saturation” (p. 6). Although the sample data in this study (nine interviews, and 11 cases of design trade-off) is within this range, we do not claim to have reached saturation.
In fact, we do not believe saturation criteria are appropriate in this study, given the nature of the themes and the process by which they were identified. Seeking saturation is most meaningful when qualitative data are coded into categories that represent different subtypes of a higher-order class or concept (e.g., Francis et al. Reference Francis, Johnston, Robertson, Glidewell, Entwistle, Eccles and Grimshaw2010). In such cases, a researcher might claim that saturation is achieved once the set of categories has stabilized and collecting additional data does not result in identifying new categories. However, the thematic categories identified in the present study are not subtypes of a single higher-order concept. They reflect different aspects of the design context, perceived by designers to have some influence on their ability to resolve design trade-offs. As such, they reflect different, potentially independent, or orthogonal, variables, each of which could be the focus of targeted qualitative, or quantitative, studies in future research. For example, future qualitative work focused on our theme “degree of complexity” might seek to identify different sources of complexity that affect trade-offs, different ways complexity affects the difficulty of resolving trade-offs, different strategies used by designers to deal with complexity, etc., and saturation criteria could potentially be used to determine a sample size that achieves stability of the resulting categories. Future quantitative studies could investigate the generalizability of the themes, independence of the underlying variables, or their potential effects on outcome variables reflecting different aspects of design performance. Using different terminology, Braun and Clarke (Reference Braun and Clarke2021) discuss the relative value of saturation for different approaches to thematic analysis (TA). They suggest saturation may be relevant for what they call “coding reliability” or “codebook” TA, where themes reflect general constructs that subsume lower-level codes, but they argue there is “incompatibility between data saturation and an organic reflexive TA approach.” In “reflexive TA,” themes are developed through the meaningful interpretation of codes, which are viewed as “… conceptual tools in the developing analysis and should not be reified into ontologically real things” (p. 207; emphasis in original).
Saturation criteria also tend to assume an additive coding process, where saturation is achieved when no additional codes or categories are identified in the data. In our case, coding was highly iterative, not strictly additive. Codes and categories were iteratively revised (i.e., merged, divided, added and deleted) by the three coauthors, until we converged on the resulting set of nine thematic categories. Braun and Clarke (Reference Braun and Clarke2021) similarly observe that “[i]n reflexive TA, codes are never finally fixed. They can evolve, expand, contract, be renamed, split apart into several codes, collapsed together …, and even be abandoned” (p. 207), and although researchers use many different kinds of codes, such as “semantic (surface, obvious, explicit meaning) or latent (implicit, underlying meaning),” few “discuss … what this might mean, conceptually and practically, in terms of data saturation” (p. 208). Although it may be possible to define saturation criteria for such iterative, recursive coding and different kinds of codes (e.g., such that data collection stops when there are no further revisions to the structure of the categorization scheme), this is not how saturation is typically discussed in the literature.
Further research is needed to understand better the role, value and limitations of saturation criteria for different approaches to qualitative research. For example, the implications of defining saturation criteria at the level of codes, (thematic) categories or some combination of the two, are not well understood. More fundamentally, little is known about how the number and variety of codes and categories identified through qualitative analysis in the first place, are influenced by such factors as the perceptions, goals, prior knowledge and cognitive information processing limits of the researchers performing the analysis. Both theoretical and experimental work are needed to clarify the many conceptual issues involved. Computer simulation modeling may also be appropriate, whereby different assumptions about the preceding factors and the interactions between them could be systematically varied to explore their effects on sample sufficiency under different saturation criteria. Such studies could help clarify the conceptual differences between different qualitative methodologies, make explicit their underlying assumptions about how conclusions are derived from, and relate to, the qualitative data collected, and assess their relative pros and cons for investigating different research questions in different empirical settings.
It is beyond the scope of this article to propose specific ways of addressing the limitations of saturation criteria in qualitative research. Instead, our approach in the present study has been to provide as much transparency as possible about the research process, including detailed description of interview methods, data coding, theme development, and the prior experience of the researchers. Hopefully, this provides others with the information needed to evaluate the quality of the results and their relevance for understanding how designers perceive and interact with design contexts to navigate and resolve trade-offs.
5 Implications and future research directions
This study has investigated design trade-offs in context. Using a novel interview methodology based on socio-technical networks, this exploratory qualitative study has identified a range of contextual influences perceived by the interviewed designers to have affected trade-off resolution. Notably, in the cases described in the dataset, only one of the designers attempted to apply any of the quantitative trade-off resolution methods in the design optimization literature as outlined in Section 1. Given the prevalence of these quantitative approaches in the design literature describing trade-off situations, these findings support the importance of studies on trade-offs in their contextual environment of real-world design practice, where designers may not be defaulting to or applying rigorous quantitative methods to optimize trade-offs.
Qualitative field research is especially valuable and appropriate for investigating research topics like design, where behaviors and attitudes resist straightforward quantification and can be best understood within their natural setting. The results of this study contribute to a theoretical understanding of the effects of context on design, and each of the themes identified raises potential questions for future research. The work also provides a foundation for developing strategies to support designers in the management of trade-offs, to reduce the barriers that inhibit trade-off resolution, and to inform design methods and practices to mitigate or avoid trade-offs.
Our results highlight the importance of learning about the design context to resolve trade-offs. Designers need to communicate effectively with multiple stakeholders and domain experts, and utilize prototyping, simulation and testing to obtain accurate information about constraints and the degrees of freedom available for design decisions. The path dependency of decisions implies that designers need early access to such contextual information to potentially predict and avoid trade-offs, but also the flexibility to revisit decisions and renegotiate constraints when new trade-offs are discovered late in the design process. While revisiting prior decisions may help a designer resolve or navigate around a trade-off, such actions (e.g., the required information gathering) may incur their own costs in time and resources. Future research could explore how designers assess the relative costs and benefits of revisiting prior decisions to manage trade-offs.
These findings suggest that the organization of design projects (Bucciarelli Reference Bucciarelli1994) is likely to influence the management of trade-offs and future research is needed to understand how different organization structures support or hinder trade-off resolution. For example, structures that support communication and coordination between key stakeholders early in the design process should help designers clarify and understand constraints and interdependencies between decision variables, and to identify decision sequences that reduce the likelihood of trade-offs. Organizational strategies that enable direct connections between designers and key stakeholders should also support the iterative decision-making and negotiations needed to address trade-offs discovered late in the design process. Future research could also explore decision sequencing strategies that delay fixing design variables that are likely to constrain other variables for as long as possible (e.g., for decisions that are not on the project critical path) and potentially help designers retain flexibility until information is available to characterize the interdependencies sufficiently to avoid trade-offs.
Despite such potential strategies, our findings suggest that complexity, ambiguity and uncertainty sometimes lead to early decisions being made that introduce trade-offs and limit decisions later in the design process. Further work is needed to understand the organizational and personal barriers that prevent designers from revisiting such prior decisions. Future research could explore the effects of different kinds of constraints on designs, contrasting how designers manage constraints that are impossible or very difficult to change (e.g., due to laws of physics, or the need to interface with some existing system) with ones that are potentially more flexible (e.g., due to organizational policies, or personal reluctance to revisit decisions). Better understanding of such barriers and constraint responses can potentially lead to strategies that help to maintain flexibility in design projects and aid in resolving trade-offs, either by keeping future trade-offs under-constrained or facilitating easier iteration throughout the design process.
Finally, our discussion has emphasized the negative effects of complexity, ambiguity and uncertainty on trade-offs, but complex, ambiguous contexts may also give designers the degrees of freedom needed to restructure their design spaces sufficiently to resolve trade-offs. Early decisions made under conditions of ambiguity may unintentionally and serendipitously provide the necessary flexibility to resolve the trade-offs, but this may not be readily apparent to the designers themselves, since they would not encounter any difficulties due to the required flexibility already being available to them. Such serendipitously positive effects would not be easily detectable with our current methods of investigation and warrant further study.
Acknowledgements
We are grateful to the interviewees who generously gave their time to participate in this study. We would also like to thank Oscar Nespoli, Scott Rose and Brendan Wylie-Toal for their assistance with recruiting.
Financial support
This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
1. Appendix
Below we list all nine themes (T) and respective assignment of all 93 codes (C). Asterisk (*) indicates a code that falls under more than one theme (other theme in square brackets).
T1. Degree of complexity
-
C39* – Prototyping helps designer conceptualize complicated solutions more easily [T8].
-
C42 – Having limited number of choices aids analysis and decision making.
-
C49 – Difficult to focus on multiple complex projects simultaneously.
-
C55 – Modular design limits options and permutations of feature/capability sets.
-
C66 – System complexity and number of interacting parameters.
-
C83 – Not able to keep current up-to-date requirements and limits in mind while designing.
T2. Limitations of designer agency
-
C24 – Removing organizationally imposed limits on project resources.
-
C46* – The challenge of convincing stakeholders and decision makers to increase project resources [T4].
-
C57 – Regulations preventing access to specific levels of performance in components.
-
C64 – Environmental context of the design negatively influencing parameter behavior.
-
C67 – Exploring alternative solutions “too long.”
-
C68 – Environmental context of the design imposing a limit or barrier not present in other contexts.
-
C69 – Restricting project resources (time, money, manpower, etc.).
-
C76* – Organizational structure impeding speed and clarity of information transfer about goals and viability assessments [T3].
-
C80 – Reluctance to relax project deadline.
-
C87* – Time and effort required to prototype using a particular method [T8].
-
C89 – Overanalyzing the conflict wastes time.
-
C91 – Organizationally imposed limits on project resources.
-
C92* – Unable to test for real performance in accurate environmental contexts [T8].
T3. Operationalizing goals
-
C13* – Talking with stakeholders to validate operationalization of project goals [T4].
-
C19* – Operationalizing a goal differently on one parameter to improve performance on another [T5].
-
C30* – Reformulating a requirement based on better understanding of the goal or need that drove it [T7].
-
C35 – Vague requirements initially led to deeper exploration of what the real goals and needs were.
-
C76* – Organizational structure impeding speed and clarity of information transfer about goals and viability assessments [T2].
-
C77 – Vague requirements.
-
C79 – Requirements do not align well with the goals and the underlying behavior necessary to meet those goals.
-
C81* – Poorly articulated rationale for why goal is not being attained [T4].
-
C84 – Requirement gathering and formulation organizationally separated from design.
-
C85 – Formal formatting of requirements discouraging questioning the rationale and operationalization fit of those requirements.
T4. Different perspectives on design
-
C4 – Coordinating decisions across all or most of the stakeholders.
-
C13* – Talking with stakeholders to validate operationalization of project goals [T3].
-
C14 – Making stakeholder biases and priorities explicit.
-
C25 – Ideation with multiple designers.
-
C26* – New perspective on a solution prompting discovery of a boundary through critique and criticism of concepts [T7].
-
C27 – Empathy for other stakeholder’s priorities.
-
C28 – Prioritizing goals by number of stakeholders aligned (majority rules).
-
C31 – Communicating strategically to prevent bias driven barriers to information transfer.
-
C34* – Prototypes help convince stakeholders and decision makers [T8].
-
C40 – Value of stepping back/out of the immediate design situation to see the problem differently.
-
C46* – The challenge of convincing stakeholders and decision makers to increase project resources [T2].
-
C52 – Conflicting goal priorities between different stakeholders and influences.
-
C60* – Unreasonably high goals or targets from the designer’s perspective [T7].
-
C61 – Differing expectation of performance targets between stakeholders.
-
C81* – Poorly articulated rationale for why goal is not being attained [T3].
-
C82 – Poorly framing communication of changes to other stakeholders and decision makers.
T5. Consequences of previous design decisions
-
C7 – Adding complementary parameters/features that relax constraints on other parameters.
-
C19* – Operationalizing a goal differently on one parameter to improve performance on another [T3].
-
C21* – Discovering complementary parameter relationships from previous design decisions [T7].
-
C23* – Implementing a feature that facilitates the discovery of limits on other parameters [T7].
-
C47 – Confronting the repercussions of sacrificing or deprioritizing a goal or parameter.
-
C48* – Stakeholder and organizationally driven rigidity on requirement causing limit on other requirement [T6].
-
C54 – Other entwined antagonistic parameter relationships (knock on trade-offs).
-
C56 – Adding a goal or requirement that causes other related parameters to be limited.
-
C58 – Previous design decisions “locking out” options in future design decisions.
T6. Ability to revise previous design decisions
-
C10 – Accommodating changes to other parts of the design that they have “ownership” of.
-
C16 – Questioning rationale behind design decisions.
-
C32 – Releasing emotional attachment to previous failed solutions.
-
C38 – Willingness to revisit previous design decisions.
-
C43 – Organizational structure that allows backtracking and iteration.
-
C48* – Stakeholder and organizationally driven rigidity on requirement causing limit on other requirement [T5].
-
C71 – Influencer bias toward their own, already attained, goals preventing changes that may improve overall design performance.
-
C73 – Attachment to features of previous, failed, solution attempts.
-
C74 – Attempting to maximize salvaged content from previous, failed, solution attempts.
-
C75 – Fixation on own or others ideas.
-
C78* – Imposing new requirements after design decisions have been made [T9].
-
C86 – Organizational tendency to forward project inertia.
T7. Domain expertise
-
C3 – Stakeholder with better knowledge of parameter behavior.
-
C5 – Consistent preestablished limits that are common across a problem class or domain.
-
C6 – Good understanding of where the limits are make it easier to “design around” them.
-
C9 – Networking designer to experts with a better understanding of parameter behavior.
-
C11 – Understanding how parameters in the design/system interact with each other.
-
C15 – Awareness of available options in modular design.
-
C18 – Familiarity with aspects of the design solution makes it easier to design related aspects.
-
C20 – Discovery of unexpected or unknown options in modular design.
-
C21* – Discovering complementary parameter relationships from previous design decisions [T5].
-
C22 – Non-design domain expertise helps understand the behavior of parameters from those domains (subject matter experts).
-
C23* – Implementing a feature that facilitates the discovery of limits on other parameters [T5].
-
C26* – New perspective on a solution prompting discovery of a boundary through critique and criticism of concepts [T4].
-
C29* – Predicting future goals or requirements of designs that may interact with the current concept [T9].
-
C30* – Reformulating a requirement based on better understanding of the goal or need that drove it [T3].
-
C36 – Well articulated rationale for predicted behavior/failure.
-
C45 – Suggesting solutions the interview knows or strongly believes to be not viable.
-
C50 – “Push back” against imposed constraints that are seen as unachievable.
-
C53 – Preference for known, “typical” solutions.
-
C60* – Unreasonably high goals or targets from the designer’s perspective [T4].
-
C62 – Negativity from others predicting challenges, barriers, or failure.
-
C63 – Designing in a “silo,” without information on the behaviors and relationships with interfacing systems.
-
C70 – Poorly articulated information about other elements or components of the design/system.
-
C90 – Lack of experience and understanding.
-
C93 – Generalized solution properties within a domain.
T8. Prototyping, simulation and testing
-
C8 – Failed designs provide better understanding of parameter behavior and where the real limits are.
-
C12 – Using benchmark solutions to understand performance and parameter behavior.
-
C17 – Prototypes help understand underlying real performance.
-
C33 – Prototyping allows low cost, low commitment exploration of “atypical” solutions.
-
C34* – Prototypes help convince stakeholders and decision makers [T4].
-
C37 – Ability to quickly predict likely real behavior and failures from low fidelity prototypes.
-
C39* – Prototyping helps designer conceptualize complicated solutions more easily [T1].
-
C44* – Testing and simulation to quickly identify uncertain/unknown barriers [T9].
-
C51 – Negative feelings of impossibility after a failed design.
-
C59 – Anxiety from using unoptimized designs or prototypes to explore the behavior of the design space.
-
C87* – Time and effort required to prototype using a particular method [T2].
-
C88 – Comparing concepts at different levels of aesthetic refinement introduces bias.
-
C92* – Unable to test for real performance in accurate environmental contexts [T2].
T9. Timeliness of information availability
-
C1 – Early awareness of barriers and limitations.
-
C2 – Early awareness of parameter behavior.
-
C29* – Predicting future goals or requirements of designs that may interact with the current concept [T7].
-
C41 – Considering limits earlier in the design process prevents unexpected failure.
-
C44* – Testing and simulation to quickly identify uncertain/unknown barriers [T8].
-
C65 – Aversion to thinking beyond the scope of the current project.
-
C72 – Emotionally frustrated reaction to new constraints.
-
C78* – Imposing new requirements after design decisions have been made [T6].
Open access
