1. Introduction
“As a team, how did you decide which ideas to pursue? How did you build confidence that this idea is the right one to invest in?” asked a NASA leader in an early-stage design team meeting. The team that was pitching their concepts had spent months in discovery, triangulating between quantitative and qualitative insights to identify new investment opportunities. They were given a short time to pitch. Then, program leaders made a go/no-go decision.
This leader’s questions are common and expected in many professional design settings. They are fair yet challenging to answer. How can design teams effectively bring others into their reasoning to build understanding and confidence, especially when the process of defining an innovation is often complex and non-linear? An incomplete narration of the process can leave leaders, stakeholders, and even team members wanting more. While great designers do have unique abilities to balance the art and science of their work, organizations are becoming more data-driven, cross-functional, and horizontal, and stakeholders expect transparency at a wide variety of organizations, including NASA.
There is a longstanding need for designers to reveal the detailed rationale behind the directions followed within a design journey. In response, we offer the Design Logic Visualization (DLV)—developed by a team within the NASA Convergent Aeronautics Solutions (CAS) project—to document and clarify the rationale behind key steps in a design process. The DLV was created during a period of uncertainty when traditional documentation methods and existing design process visualizations failed to capture and translate what Jon Kolko describes as the abductive reasoning or “logic of possibilities” (Reference KolkoKolko, 2010).
Convergent research, or the deep integration across disciplines, has been championed as a “blueprint for innovation” (Sharp et al. 2011). This kind of robust transdisciplinary collaboration, such as what is implemented at NASA CAS, is also frequently met with language and communication challenges. By transforming the design journey into a clear, graphical narrative, the DLV serves as a boundary object that bridges disciplinary divides and enhances communication among team members and decision makers (Reference Ashton, Sungu, Davis, Agarwalla, Burke, Benavides, Espat, Harper, Knight, Labruto, Shea, Verba and WilsonAshton et al., 2024).
The present paper presents a detailed case study of the DLV’s development and application. While created within a single organization and applied in this paper to a single case study team, the DLV tool is positioned within the broader context of convergent research and is intended to be applicable to a wide variety of teams working to address convergent or sociotechnical challenges. We first situate our work within the broader context of process visualizations and boundary objects. We then describe how the DLV was iteratively refined, providing both a record of the process and a means for stimulating reflective dialogue and decision making within the team. Section 4 then details our autoethnographic methodology, and sections 5 and 6 present the findings and implications for the DLV design tool.
2. Background
Design is a practice and methodological framework that leverages stakeholder research and iterative development to translate research insights into actionable solutions . Originally rooted in product design, elements of user-centered design processes, such as Human Centerer Design (HCD) have increasingly found applications in complex, technical and cross-disciplinary domains (Reference Lubis and ShahriLubis & Shahri, 2022; NSF Convergence Accelerator, n.d.).
The design process relies heavily on divergent and convergent thinking patterns in which teams alternate between broad exploration followed by focused refinement (Reference KolkoDesign Council, n.d.; Kolko, 2010). As described above, such cyclical, non-linear approaches can present challenges in documenting progress and the rationale behind new directions, especially for decision-makers and stakeholders who are not actively engaged in the daily process or are less familiar with design’s reflexive nature. As a result, there is a need to succinctly explain the justification, or design logic, of a team’s decision-making. In this context, design logic is defined as the structured reasoning by which each phase of the design process builds and iterates upon the preceding phases to reveal outcomes (Reference Easterday, Lewis and GerberEasterday et al., 2018; Reference JiangJiang, 2023).
2.1. Visualizing design activity and discourse over time
A variety of visualizations have been developed to examine and communicate sequences of design activity and discourse over time. Process diagrams are especially common since the popularity of IDEO and “design thinking” entered the zeitgeist. In these visualizations, specific rationale remains largely implicit, however, and the detailed logic and data are generalized or lost at such high levels. Design process visualizations range in form. The VALiD Project explored student designers editing video footage of their design activities, revealing how narrative construction plays a role in the perception of design events (Reference Lloyd, Valkenburg and McDonnellLloyd et al., 2003). Atman et al. (Reference Atman, Adams, Cardella, Turns, Mosborg and Saleem2007; Reference Atman2019) developed “design signatures” to visualize the order and amount of time that designers spend in different design phases (e.g., problem definition, idea generation, concept evaluation). In examining differences between the design signatures of novice versus more experienced designers, they found that experienced designers spent significantly more time conducting problem definition (Reference Atman, Adams, Cardella, Turns, Mosborg and SaleemAtman et al., 2007). Their design signature work has grown and is now available as an app-based tool (UW Center for Engineering Learning & Teaching, n.d.).
Seow et al. (Reference Seow, Tiong, Teo, Silva, Wood, Jensen and Yang2018) proposed a different kind of design signature visualization, mapping design teams journeys across four quadrants of activities broadly categorized as discovery, define, develop, and deliver. More similar to Atman et al., Ge et al. (Reference Ge, Leifer and Shui2021) studied design team discourse and created design signatures, but instead of design phases, focused on designers’ expression of emotion. Ge et al. plotted signatures of designers expressing intense emotion over time, as assessed by vocal pitch, and found that this was correlated with design re-framing, as characterized by the emergence of new words not previously spoken (Reference Ge, Leifer and ShuiGe et al., 2021). Also focusing on sequences of design discourse, Sonalkar et al. (Reference Sonalkar, Mabogunje and Leifer2013; Reference Sonalkar, Mabogunje, Hoster, Roth, Plattner, Meinel and Leifer2016) developed Interaction Dynamics Notation (IDN), a visual notation tool that makes visible patterns in interpersonal interactions among design teams. In comparing IDN visuals with outcomes, they found that certain interactions, such as “yes-and,” “move,” and “question,” were associated with more unique ideas generated during design team discussions (Reference Jablokow, Sonalkar, Edelman, Mabogunje and LeiferJablokow et al., 2019).
These design discourse visualizations have advanced the communication of design, but they do not fully capture the sequence of insights that lead design teams to frame the rationale behind the twists and turns of their deign journeys. Design reasoning typically involves divergent and convergent phases with down-selection between phases based on stakeholder feedback, rubrics, and other criteria. While this reasoning process is reflexive in nature (Reference SchönSchön, 1983) and therefore inherently difficult to map, it can be mapped across a design team to reveal key decision-making moments. Doing so, as we illustrate here, may not only allow design reasoning to be better studied by researchers but also communicated to stakeholders and across design team members, especially across disciplinary and other group boundaries, helping to further improve cross-functional collaboration and decision-making.
2.2. Boundary objects in inter/trans-disciplinary teams
Boundary objects—such as sketches, prototypes, and visualizations—have long helped facilitates shared understanding across disciplinary and knowledge-group boundaries (Reference Star and GriesemerStar & Griesemer, 1989), including in engineering and design (Reference BechkyBechky, 2003; Carlile, Reference Carlile2002; Henderson, Reference Henderson1991). Boundary objects help to translate information across group boundaries, allowing members of different groups to draw different meanings from the same artifacts. This interpretive flexibility allows many different groups to find meaning in a boundary object and use it to clarify consequential group differences and dependencies (Nicolini et al., 2012). Especially when sequenced alongside other collaborative objects, this can lead to more regular and systematic co-discovery of potential failure modes, design risks, design opportunities, and workflow bottlenecks (Reference Brubaker, Sheppard, Hinds and YangBrubaker et al., 2023). When might visualizations of design reasoning or logics act as a boundary object in a convergent or transdisciplinary team, and how might this influence design decision making and outcomes?
2.3. Research questions
Drawing upon autoethnographic reflections, we address two primary research questions:
1) How do Design Logic Visualizations (DLVs) act as boundary objects in convergent/transdisciplinary design teams?
2) How does creating and using DLVs influence design decision making outcomes?
3. Visualizing the design logic
Drawing on principles from process maps, flowcharts, and network graphs—and building on the concept of design signatures—DLV offer a concise narrative of the key rationale or logic of a design journey. In this section, we outline the origin of the DLV tool and illustrate its application with a detailed example.
3.1. Origin of the design logic
The first DLV was created by a NASA CAS early-stage design sprint team during a period of uncertainty when CAS leaders struggled to understand how a set of concepts had evolved. In a pivotal moment, the team’s design facilitator asked, “Do you want to see the logic?” to which a decision maker replied, “Yes, I want to see the logic.” This exchange spurred the first iteration of the visualization (Figure 1).
Figure 1. The first iteration of the DLV
Seeing the potential of DLVs, the decision maker asked for them to be developed for all the concepts to be presented in an upcoming down-selection meeting with upper-level management. Creating these DLVs offered a foundational storytelling tool for the team during their presentation to leadership. Ultimately, the DLVs helped the team and management to consolidate and make informed decisions around which concepts to move forward.
Figure 2 shows the iterated DLV “Unlocking the UAS Supply Chain” with five major phases of activity shown at the top: Research and Insights, Concepts, Iteration: External Potential, Iteration: CAS Potential, and writing the final Opportunity Concept Reports (OCRs). These phases progressed over time, starting with Research and Insights in January 2024 and culminating in OCR writing in August 2024. This more evolved DLV captured key decision points and served as a dynamic roadmap that informed both the writing of the Opportunity Concept Reports and the pitching of the ideas to upper-level management. They were built collaboratively with design facilitators who collected and organized the bulk of the data and then team members commented and refined key points.
Figure 2. An example DLV, flowing left to right, for the “ Unlocking the UAS Supply Chain” concept
Within the team—made up of a range of subject matter experts in various career stages (full list available in Table 1)—the communication benefit of creating the DLV was immediately relevant. Not only did the cross-functional team have more succinct and clear conversations with decision-makers, but they also gained greater clarity internally regarding their design journey to-date. In effect, the iterative nature of the DLV allowed it to function as a boundary object, bridging disciplinary divides and directly influencing the decision-making process. As time went on, it became increasingly exciting to observe how each discipline translated the ever-evolving visualization from their own perspective.
Table 1. NASA CAS 3C team members and disciplinary expertise (author-participants in this study)
Upon establishing the origin story and potential for DLV as a boundary object, we now delve deeper using mathematical and graph theory lenses. By applying these analytical perspectives, we demonstrate how additional disciplinary expertise can translate and unlock tools for reasoning and analysis, further enriching a team’s understanding of their design logics and making space for emergent methodologies.
3.2. The DLV as a graph, a mathematical lens
In discussions on the design process that utilized the DLV, team members naturally related the visualization to objects that appear in their field of expertise. A mathematician and data scientist both related the DLV to an object at the intersection of their respective fields known as a graph. This section details how the team members applied their subject matter context to the DLV and how, by doing so, new types of analysis on the design process and decision-making may be unlocked.
In the field of mathematics, a graph is a structure that consists of a set of information and some relation between elements in the set. This structure can be represented visually by nodes, which are the abstraction of the objects in the set, and edges that connect nodes by the defined relation between them. The study of these objects in mathematics is called graph theory and the applications of graph theory to analyze data many fields including chemistry, sociology, and computer science (Balaban, Reference Balaban1985; Dobrjanskyj & Freudenstein, Reference Dobrjanskyj and Freudenstein1967; Smith, Reference Smith2013).
The interpretation of the DLV as a graph opens the possibility for future work on building the structure of the visualization itself and relating it to fields which also utilize graphs as an abstraction. One example of this is the study of artificial neural networks, wherein the network is abstracted to a graph with edges assigned a numerical value called a weight (Reference SpearsSpears, 2017). These weights, along with another measurement called a bias, are found through “training” and signify the influence of that input on the next stage of the decision process. This type of training could lead to a larger reflection on what types of information and biases a team or organization inject into the design process. The analysis itself is outside the scope of this work, but the convergence of expertise highlights the functionality of the DLV as a boundary object.
4. Methods
4.1. Reflective autoethnography and collaborative inquiry
This paper employs a reflection-based autoethnographic (Reference Secules, McCall, Mejia, Beebe, Masters, L. Sánchez-Peña and SvyantekSecules et al., 2021) and collaborative inquiry approach (Reference Walther, Sochacka, Benson, Bumbaco, Kellam, Pawley and PhillipsWalther et al., 2017) to help us make shared meaning of ourselves as both researchers and research participants. Autoethnography and collaborative inquiry reposition research “subjects” as active researchers, and vice versa, and work to build knowledge from the collective and systematic exploration of researchers’ own experiences (Reference Walther, Sochacka, Benson, Bumbaco, Kellam, Pawley and PhillipsWalther et al., 2017, p. 399)—thereby “creating new knowledge drawn systematically from the life experiences of persons most centrally involved in the context of the inquiry” (Reference Yorks and KaslYorks & Kasl, 2002, p. 4). Each of us, as author-participants, contributed to this NASA CAS 3C design team, helped conceive or prototype the team’s DLVs, and used the visualizations to further our team’s design concepts. Drawing inspiration from Secules et al. (Reference Secules, McCall, Mejia, Beebe, Masters, L. Sánchez-Peña and Svyantek2021), we engaged in reflective journaling around a set of common prompts to explore how our unique disciplinary lenses influenced what we saw in a given visualization, what we saw as strengths or gaps in the development of a design concept, and how we proceeded to fill gaps and down-select our top concepts. We used our autoethnographic reflections as primary data and looked across them to examine how the DLVs served as boundary objects in this professional engineering design setting. The present study highlights the role of reflexivity in informing design research outcomes and emphasizes the importance of positional understanding within the research process.
4.2. Research site: NASA Convergent Aeronautics Solutions 3C team
Convergent Aeronautics Solutions (CAS) is a cross-center project within the Transformative Aeronautics Concepts Program and Aeronautics Research Mission Directorate at NASA. It aims to convene transdisciplinary teams to formulate and address “wicked problems” that could transform the future of aeronautics for the benefit all humanity. The CAS Connecting rural Communities and Capabilities team (3C) began in 2022 (Reference Rieken, Brubaker, Best, Bond, Burleson and SimpsonRieken et al., 2023), took a pause, and was further advanced with a newly assembled team in 2024. While many people have contributed to 3C, the team considered in this study is ten people, including six NASA engineers, mathematicians, and scientists, two interns, and two design-innovation strategy contractors who served as design process facilitators. The team represented a wide set of disciplinary expertise, ranging from mathematics to space biology to non-destructive evaluation (Table 1).
4.3. Autoethnographic reflection prompts
The following reflection prompts were given to each member of the 3C team:
1) “Before looking at the Design Logic Visualization, quickly jot down your recollection of the design process for our team’s Opportunity Concept Report: Unlocking the UAS Supply Chain. Note any critical moments or deciding factors.”
2) “Before seeing the Design Logic Visualization, how well did you understand and recall our team’s design process?”
3) “What do you notice in the Design Logic Visualization, and how has it changed your understanding of our process and team communication?”
4) “Does this visualization remind you of other tools or concepts in your field? If so, how?”
The first two questions focus on understandings and recollections of our team’s design process, and the second two questions focus on the DLV as a boundary object (what each of us saw in it and gained from it). For questions 1 and 2, our team displayed varied levels of detail in our recollections of our design process. Many described a relatively vague “broad brush strokes” understanding, a process that was “more organic than the well-structured graphic.” Some team members provided more detail around specific points in the process or in the utilization of particular tools, such as large language models. Note that some team members joined partway through the design process. Multiple team members mentioned that the specific criteria for down-selecting concepts was something that they did not recall. This supports the origin story of the DLV, which was a need to clarify the rationale, key insights, and inflection points that led to each concept, both for the team and management. We now turn to our Findings to review the responses to questions 3 and 4—how the design logic was used as a boundary object and what different team members saw in or gained from it.
5. Findings
In this section the questions and autoethnographic responses that focused on the DLV are discussed. The purpose of these questions was to understand how each team member connected with the visual as well as how it influenced their perspective of the design process. Themes emerged from each question and are highlighted in the respective subsections.
5.1. Reflections of the DLV as a boundary object
To set the stage for understanding how different team members viewed the DLV, everyone was asked the following question: “Does this visualization remind you of other tools or concepts in your field? If so, how?” In their responses, all the team members identified similarities between the DLV and tools or visual representations used in their respective areas. These connections prompted a few team members to respond by introducing potential improvements on the DLV:
“Could the design logic use arrow width to denote the amount of insights or level of stakeholder validation/excitement behind each node and connection? I found myself speculating and mentally overlaying such possibilities on the design logic map.” -Eric
“I wondered how we might utilize the tools that have been built in graph theory to gain a deeper understanding of the design process?” -Lauren
“It reminds me a bit of a neural network, with all of the layers and interconnections… the OCR could be similar to the output layer in a neural network….” -Annie
These builds and analytical studies are a topic of future work and further discussed in Section 6.3.
5.2. Reflections on the impact of the DLV
To understand how the DLV served the team during the design process, the team members were asked to reflect on the following question: “What do you notice in the Design Logic Visualization, and how has it changed your understanding of the process and team communication?” In response, many of us commented on how the DLV helped us to both the big picture and details of our team’s critical design directions and the justifications or precursors that led to them:
“I see the design logic as a map of critical decisions and rationales (why did the team decide to pause this line of inquiry? From where/who did that data derive?), which is invaluable, not only for this project at hand, but also for the formation of NASA’s institutional knowledge—to serve as a record of insights, and as a body of evidence from which to draw meta insights across all centers.” -Kathleen
“Seeing the big picture of a process is very helpful to me personally before ’going through it’ to contextualize and help me understand the ’why’ behind what I’m doing. It could just be my personality that I need to understand the ’why’ first.” -Angela
“It aided our discussions by making it easier to understand/explain/justify suggestions for improvements and allowed me to easily envision changes suggested by others. I can also see how this can help influence process management decisions to alter or modify instructions for creating Opportunity Concept Reports.// When a figure was drafted that captured this process it helped me enormously and allowed us as a team all to communicate about the process better, because we could easily visualize the flow and locations of main decision points.” -Jon
“Mostly I notice how much the work shows the way in design. Often we are looking for input that either validates or invalidates research and ideas. It’s so important to have stakeholder feedback to do that effectively. Different from other forms research that can be validated using direct data, innovation requires triangulating the abductive reasoning// to give enough evidence to move forward. It’s exciting to see the important steps that feed into what is a deeply creative process.” -Ariella
These responses show how the DLV aided the team’s communication and understanding of the design process by clarifying the decision-making process at each step. This was especially the case for those team members that were new to the design process. As noted in section 3.1, the DLV was used as a tool to communicate both within the team and to decision-makers that evolved in the final stages of down-selection. This evolution into both a justification tool for decision-makers and an internal communication tool for the team was seen as important part of the final down-selection process.
The inclusion of certain information in later iterations of the DLV led some team members to reflect on a process or organizational emphasis:
“It helps me see the logic, and I also have to think that it alters my memories.” -David
“The [DLV] really clarified specific details of the Team’s design process… Until viewing the design logic visualization, I had forgotten how much the theme of “Unlocking the UAV Supply Chain” was really the center of our final [OCRs] at roundtable. I also didn’t realize that the process was as organized and seemingly linear as the DLV demonstrates… This graphic implies that there were defined timelines during which smaller subgroups within the team would go out and gather data and when the entire team would communicate and share information.” -Meghan
“The visualization is one method of getting to Opportunity Concept Report like things but there are other methods.” -Michael
“The [DLV’s] emphasis of iteration in its top-level processes helps remind me that the work unfolded in a nonlinear, iterative manner. This was both a deliberate and naturally emerging component of the work. Before being refreshed on the design process, it was easy to conceptualize it in a linear manner just for mnemonic and communication convenience. After being refreshed on the design process, it’s easier to understand it as an unfolding, organic progression.” -Erik
These responses highlight the reflexive nature of the DLV construction which might cause the presence or absence of material that was utilized in real-time. Team members also pointed out that the layout of the information as a linear progression of ideas, while clarifying, does not match with their perception of the process as more “organic” given that some of the top-level categories happened concurrently.
6. Discussion
6.1. Implications for design and communication in technical spaces
Transdisciplinary design teams that converge multiple domains are challenged to communicate and build shared understanding of their design rationale or design logics across broad sets of participants, stakeholders, and decision makers. There is also an expectation of transparency of the rationale behind design choices that are made along complex, non-linear design processes. By explicitly showcasing the design rationale as a collaborative object, design teams can better articulate the process steps and decision-making justifications by which they navigate their design journeys. The autoethnographic reflections illustrated the critical role that temporal visuals of design rationale can play as boundary objects in facilitating communication across disciplines, stakeholders, and decision-makers. This may be especially useful for transdisciplinary teams working on convergent problems across sectors and domains, such as industry efforts like Alphabet X’s Loon project spanning aviation and energy or Uber’s efforts to bridge transportation and healthcare, among the many other transdisciplinary teams across industry, academia, and government examining convergent, multi-sector challenges.
6.2. Limitations
Potential limitations emerge when deciding what is included and excluded in a DLV. The act of synthesizing complex processes into a clean, simplified visual can inadvertently lead to the omission of critical nuance or contextual details. This simplification may result in stakeholders forming incomplete or altered perceptions of the design process, as David Fuller noted in his reflection, “it helps me see the logic, and I also have to think that it alters my memories.” If the visualization fails to effectively communicate the depth or complexity of a process, key decisions, and rationale, its efficacy is reduced or rendered ineffective. There are inherent trade-offs between clarity and fidelity in visual storytelling, which engineers have been found to use strategically (Reference Barley, Leonardi and BaileyBarley et al., 2012), and questions could be raised around how best to balance the need for intelligibility with the need for comprehensive representation.
6.3. Future research directions
The DLV tool suggests multiple avenues for future research, as discussed in Sections 3.2 and 5.1. One avenue is to refine the DLV by including more details on the rationale for each design decision or step. Another is to apply DLVs to a larger set of design efforts from various sectors and industries and to examine patterns in the ways that design journeys unfold and design decisions are made. Further, future research could transform the DLV into neural network models. Such an approach could enable researchers to analyse and ‘train’ those networks, leveraging these systems to assist in decision-making processes while providing insights into the weights and biases that influence a particular team’s design methodology. Through modelling and reflecting on these computational representations of design processes, teams could uncover patterns, assumptions, and areas for improvement that may otherwise remain implicit. Applying this technique could serve as a novel method for bridging technical and design disciplines, fostering greater communication and thus innovation in convergent contexts.
Acknowledgments
This work was funded by NASA’s Aeronautics Research Mission Directorate, Transformative Aeronautics Concepts Program, Convergent Aeronautics Solutions (CAS) Project. The authors extend our gratitude to the many community stakeholders in St. Mary Parish, LA, and Refugio, TX as well as the government, academic, and industry subject matter experts who contributed their expertise in discussions with our team. We would like to thank the CAS Project Managers for their valuable technical feedback and input over the course of this research. We would also like to thank the three anonymous peer reviewers, whose comments and feedback strengthened the final manuscript.
Open access
