1. Introduction
It is frequently suggested that designing products for a broad range of stakeholders and environmental settings requires a deep understanding of context (Reference Jones, Jones and KijimaJones, 2018). Incorporating context is recommended so that products can be tailored to the situations in which they must function (Reference Aranda-Jan, Jagtap and MoultrieAranda-Jan et al., 2016). Prior research has revealed the extent to which contextual information is collected and utilized in engineering design, highlighting the critical role of integrating contextual factors to achieve successful product outcomes (Reference Burleson, Herrera, Toyama and SienkoBurleson et al., 2022, Reference Burleson, Wojciechowski, Toyama and Sienko2024). Literature has documented instances of product failure (i.e., a product failing to achieve its intended objectives) attributed to designers neglecting to incorporate pertinent contextual information (Reference Arshad-Ayaz, Naseem and MohamadArshad-Ayaz et al., 2020; Reference ToyamaToyama, 2015). A study of failed Engineers Without Borders projects further emphasized this point, identifying that engineers “failing to have adequate contextual knowledge” was the top reason for project failures (Reference Wood and MattsonWood & Mattson, 2016). Therefore, understanding context and its influence on product development is crucial to ensure contextual suitability, meet user needs, and achieve successful outcomes.
Scholars suggest that engineering designers should consider context throughout all stages of a design process (Reference Burleson, Toyama and SienkoBurleson et al., 2023). An engineering design process consists of many stages and is often categorized into two high-level phases: Front-end design and Back-end design. While various design process models in the literature are used to explain these phases, we use the Double Diamond for its simplicity and generality (Figure 1). The first diamond represents front-end design, where a problem is identified, scoped, and defined (Reference AlmqvistAlmqvist, 2017). First, the discover phase focuses on researching and engaging with stakeholder to gain a deep understanding of the problem rather than relying on assumptions. Next, during the define phase, insights from discovery are analyzed to clearly articulate a core design challenge. The second diamond represents the back-end design, where a solution is realized. The develop phase encourages exploring multiple solutions, co-designing with stakeholders, and seeking diverse perspectives, while the deliver phase involves testing, refining, and implementing the most effective solutions.
Front-end and back-end design stages, derived from The Double Diamond
Established methodologies, such as human-centered design (HCD), contextual inquiry, and design ethnography, provide designers with tools for incorporating context during front-end design phases. However, fewer methodologies focus on supporting designers in integrating contextual factors during the back-end stages, where decisions about product refinement, implementation, stakeholder alignment, and emerging real-world constraints often arise. This is particularly important because stakeholder needs and contextual conditions may continue to evolve, and prototypes developed during these stages allow stakeholders to provide more precise and contextually relevant feedback (Reference Burleson, Wojciechowski, Toyama and SienkoBurleson et al., 2024). Indeed, prototypes enhance communication and facilitate more detailed discussions between designers and stakeholders (Reference Lauff, Kotys-Schwartz and RentschlerLauff et al., 2018; Reference Rodriguez-Calero, Daly, Burleson and SienkoRodriguez-Calero et al., 2023). Our work seeks to bridge this gap by developing and testing a protocol called Contextual Product Testing (CPT). This protocol builds on existing theoretical frameworks (Reference Aranda-Jan, Jagtap and MoultrieAranda-Jan et al., 2016; Reference Burleson, Herrera, Toyama and SienkoBurleson et al., 2022) to provide designers with a tool for investigating the potential influence of contextual factors—such as cultural, environmental, and technological considerations—on their designs through in situ stakeholder engagement.
2. Background
In engineering design, context refers to the broad characteristics of the particular setting or environment in which a solution is intended to function and be used (Reference Burleson, Wojciechowski, Toyama and SienkoBurleson et al., 2024). Herbert Simon described how an engineer must design the inner environment of an artifact to be “at the service of the goals in the context of the outer environment” (Reference SimonSimon, 1969). Indeed, scholars agree that context (i.e., the “outer” environment) encompasses many factors, including broader societal, cultural, environmental, and ethical considerations (Reference Atman, Borgford-Parnell, Deibel, Kang, Ng, Kilgore and TurnsAtman et al., 2009). Beyond technological feasibility and local implementation, context includes critical aspects of a specific setting that impact user needs, safety, societal impact, and sustainability, encouraging a designer to consider how a solution fits within larger systems and environments.
Context can be subdivided into multiple contextual factors, which are specific elements of a technology’s broad use setting that influence its implementation, use, and overall outcomes (Reference Aranda-Jan, Jagtap and MoultrieAranda-Jan et al., 2016). To systematize these influences, Aranda-Jan et al. (Reference Aranda-Jan, Jagtap and Moultrie2016) proposed a framework organizing contextual factors into nine categories: Institutional, industrial, technological, infrastructure, environmental, economic, political, public health, and socio-cultural. In this study, we build on the recent work by Burleson et al. (Reference Burleson, Wojciechowski, Toyama and Sienko2024) which expanded these categories to include 32 secondary categories. While these frameworks provide a foundation for categorizing contextual factors, they do not offer sufficient guidance on effectively gathering, documenting, and integrating these influences during a design process.
Several methodologies effectively incorporate contextual factors in front-end design by emphasizing user engagement and stakeholder collaboration. For example, HCD emphasizes a deep study of user needs, including their contexts during the process of identifying and scoping design problems (Reference Rodriguez, Burleson, Linnes and SienkoRodriguez et al., 2023). Another well-known method is design ethnography, which encourages designers to use qualitative methods, such as interviews and observations, to gather contextual information more efficiently (Reference Salvador, Bell and AndersonSalvador et al., 1999). Furthermore, contextual inquiry and contextual design offer tools for stakeholder analysis and environmental understanding (Reference Augstein, Neumayr, Pimminger, Ebner, Altmann and KurschlAugstein et al., 2018; Reference Holtzblatt and BeyerHoltzblatt & Beyer, 2014). However, these methodologies are often underutilized in later design stages and do not equip designers with specific strategies for a comprehensive contextual analysis.
Few established methods aim to support designers in incorporating contextual factors during back-end design stages. While usability testing is considered a back-end methodology that addresses user interaction, its primary focus lies in evaluating functionality and user satisfaction, often overlooking broader contextual factors (Reference Baxter and SommervilleBaxter & Sommerville, 2011). User clinics, another back-end method, involve observing and engaging with users in real or simulated environments within their intended context (Reference Presse, Steinhoff and ErnerPresse et al., 2008), yet they lack a structured framework for integrating contextual insights into design decisions. Approaches for navigating evolving requirements, such as agile methodologies, emphasize flexibility, iterative development, and continuous stakeholder involvement (Reference Schön, Thomaschewski and EscalonaSchön et al., 2017). However, agile approaches do not explicitly prioritize contextual factors, which often emerge more clearly in later stages of the design process when prototypes and concepts provide tangible references for real-world applications, user interaction, and environmental constraints.
3. Methodology
This work is part of a larger effort to develop a verified methodology to support designers in identifying and integrating contextual factors during the back-end stages of their design process. Specifically, this study aimed to develop and pilot an initial protocol to assist engineering designers in systematically addressing contextual factors during back-end design stages. Our study was guided by the following research question:
How can designers systematically identify and integrate contextual factors in later design stages to enable more effective and contextually aligned product outcomes?
First, building on prior research that identified 32 categorizations of contextual factors that can be relevant to designer (Reference Burleson, Wojciechowski, Toyama and SienkoBurleson et al., 2024), we developed a protocol that guides designers with the essential methods and strategies to effectively integrate these factors into design iterations. Inspired by user-clinic models, our Contextual Product Testing (CPT) protocol employs in situ testing with stakeholders to gather detailed feedback on contextual factors, usability, and adoption barriers or enablers within their environment.
To test and refine the CPT protocol, we conducted a case study with a product that was in a back-end prototyping phase: An interactive toy chest designed to encourage children to clean up after playtime. Through ten iterative rounds of testing and adaptation with potential end users, we adjusted the protocol to more effectively capture contextual factors and user-centered insights, enhancing its value for designers aiming to align products closely with stakeholder needs and contextual nuances.
3.1. Theoretical foundation and protocol design
We grounded the development of the protocol in three theoretical foundations: (1) Diffusion of Innovation, (2) Prototypes as boundary objects for communication, (3) and classifications of context. First, Rogers’ Diffusion of Innovation theory, provides a systematic framework for analyzing how contextual factors influence the adoption of designed artifacts. This theory was selected for its ability to explain how perceptions of risks and barriers shape societal adoption of new technologies (Reference Rogers, Singhal and QuinlanRogers et al., 2008). Next, we build on recent literature that identifies prototypes as boundary objects, providing a means for more effective communication and facilitation of perspectives between designers and stakeholders (Reference Lauff, Kotys-Schwartz and RentschlerLauff et al., 2018). Lastly, the protocol uses frameworks developed by Aranda Jan et al., (Reference Aranda-Jan, Jagtap and Moultrie2016) and refined by Burleson et al., (Reference Burleson, Wojciechowski, Toyama and Sienko2024) that categorize 32 contextual factors into nine groups to ensure a comprehensive understanding of the context. We developed the CPT process to follow these steps (Figure 2):
1. Preparation: In the initial stage, designers identify key stakeholders and analyze their contexts. This is followed by prioritizing the most relevant contextual factors to effectively guide the design process and gather valuable feedback.
2. Data Collection: This involves conducting observations and semi-structured interviews. These activities are conducted in the use environment, allowing designers to observe stakeholders interacting with prototypes in real-world settings, a practice supported by recent studies, such as Rodriguez-Calero et al. (Reference Rodriguez-Calero, Daly, Burleson and Sienko2023), for its ability to yield deeper contextual insights. A structured template (see results for final iterations of the template) guides the designer through this step to ensure both consistency and depth in data gathering.
3. Contextual Factors Analysis: The collected data is analyzed to uncover patterns, and relationships across observations and interviews from different contextual factors.
4. Barriers and Enablers Identification: The designer then identifies contextual factors that result in barriers (undesirable or inaccessible features or experiences) and enablers (features or experiences that promote adoption and use) for the stakeholder(s). This process of analyzing data into barriers and enablers is inspired by prior work to identify strategies for identifying design iterations based on the theory of Diffusion of Innovations (Reference Burleson, Herrera, Toyama and SienkoBurleson et al., 2019).
5. Design Recommendations and Sustainable Design: Finally, these findings are translated into actionable design recommendations and sustainable design concepts, ensuring the proposed solutions are user-centered, context-appropriate, and supportive of long-term adoption.
Contextual Product Testing (CPT) protocol process
3.2. Case Study: Interactive toy chest
The interactive toy chest was designed by one of the members of the research team. Shown in Figure 3, the critical components of this product include a screen device attached to a toy chest with an inbuilt software system for interactive animation, which works in tandem to provide instructions and rewards to the child. Our study aimed to identify the contextual suitability of the toy chest with the ultimate goal of piloting and iterating the CPT protocol. This study was approved by our university’s Institutional Review Board (#24-0209).
Interactive Toy Chest (left) and Interface Controls (right)
3.2.1. Participants
Recruitment involved snowball and convenience sampling through the distribution of emails and flyers in our local community, leading to ten families with 15 children (aged 3 to 7) participating. Families varied in size, with most having one child, and participant demographics reflected diversity in gender, age, and income. Parents met criteria such as living within 20 miles of the research team, having a secondary education or higher, employment, and providing an indoor play area with Wi-Fi. The sample included participants who identified as White, Hispanic, and Asian.
3.2.2. Data collection and protocol iterations
We conducted ten home-based sessions consisting of 90-minute observations and interviews, focusing on children’s natural play environments where they typically engage with toys. We obtained informed consent from parents and assent from children; each family was compensated with a $50 gift card and a small toy for participating. The study used a setup where a research assistant, hidden from view, controlled the animated character shown on the interactive toy chest via Zoom, allowing children to interact naturally. Each session included the use of the CPT protocol by a second member of the team, which consisted of a 20-minute observation period followed by a 40-minute interview. Observations and interviews were audio recorded. The remaining time was allocated for introductions, setting up the toy chest, and wrapping up the session.
The protocol underwent iterative modifications to improve its ability to capture contextual information effectively. Initially, it relied on a rigid and lengthy checklist, which proved impractical during observations. To address this, the method evolved into a flexible, blank-section format for spontaneous notetaking, which later consolidated into a streamlined, color-coded template for easier classification of contextual factors. For this iteration of the protocol, we focused on five of the nine contextual factor categories: technological, socio-cultural, environmental, economic, and infrastructural. This decision was driven by the characteristics of our stakeholder group, as they were not directly involved with or impacted by the remaining categories, and they were unlikely to provide meaningful feedback on those factors. Midpoint adjustments introduced a dedicated “Usability Factors” section to separate product-specific insights from other contextual data, addressing usability issues like toy chest size and material alignment.
3.2.3. Data analysis
The data gathered included transcribed interviews, field notes, and observation documents. Using a qualitative content analysis approach (Reference Hsieh and ShannonHsieh & Shannon, 2005), a single coder categorized data into five contextual factors and their potential to influence product adoption. We used a qualitative analysis software, Dovetail, to facilitate this process. As the study progressed, the CPT data analysis protocol was iteratively refined to improve understanding of contextual factors’ influence and end-user feedback. We organized participants’ comments within each category, making it easier to recognize patterns, leading to findings categorized as barriers and enablers for product adoption based on the Diffusion of Innovation framework. Based on these insights, we developed design recommendations to enhance the contextual suitability of the toy chest.
4. Results
Our results are structured first to present the case study findings, which detail the barriers and enablers identified across contextual factor categories, along with corresponding design recommendations. Following this, we introduce the final iteration of the CPT protocol, refined through iterative testing to effectively support designers in integrating contextual factors.
4.1. Case study findings and design recommendations
This initial analysis identified a total of 23 barriers and 15 enablers across five examined contextual factor categories (technological, socio-cultural, environmental, economic, and infrastructural). While the full analysis of barriers can be found in the appendix, the following subsections highlight key challenges and strengths for integration into user contexts.
4.1.1. Technological factors
Major technological barriers included parental concerns about overreliance on technology, younger children’s unfamiliarity with screens, and connectivity issues such as animation lag and delayed responsiveness, which disrupted engagement. On the positive side, enablers included parental openness to educational screen time, reliable internet access, and children’s emotional connection to the character, Toby. Based on these findings, design recommendations include implementing screen-time limits using a sand timer, developing an intuitive interface with minimal buttons, enabling offline functionality, and enhancing Toby’s educational features.
4.1.2. Socio-cultural factors
In the socio-cultural category, some important barriers were the limited educational features beyond cleaning, cultural preferences for non-noisy, non-fast-paced animations, a need for multilingual options (Spanish being the top request after English), and a lack of diverse symbols and visual cues. Enablers included compatibility with home decor, Toby’s interactive style promoting curiosity, and its role as a fun, indirect clean-up assistant valued by parents. Design recommendations focus on promoting physical toy play to develop motor skills and creativity, avoiding loud sounds and vibrant colors, incorporating multiple language options, and integrating cultural traditions linked to language and seasonal events.
4.1.3. Environmental factors
Environmental barriers included the chest’s lack of weather-resistant features, such as waterproof materials and wheels for transport, and concerns about pet damage. Some parents also felt it was not sturdy enough for children to sit on, and opinions on size varied (some preferred a larger chest for outdoor toys, while others preferred a more compact design for living rooms). Despite these challenges, we found that most families planned to use the product indoors, and parents appreciated its aesthetic appeal. Design recommendations include using waterproof, pet-friendly materials, ensuring reinforced stability, and offering size-varied options.
4.1.4. Economic factors
Findings on economic factors highlighted price sensitivity, as eight parents expressed unwillingness to spend more than $50–$100, making the current $250 price a significant barrier. Many families preferred second-hand toys or traditional educational tools over digital options, and those with older children felt it would not have enough use. Enablers included interest in a subscription model for ongoing updates, customizable features, and positive word-of-mouth influence. Design recommendations are tiered pricing with a basic and premium model and a subscription option for long-term value.
4.1.5. Infrastructural factors
Lastly, Infrastructural barriers included issues with moving the toy chest through baby gates, indicating a need for collapsible or modular designs. Positively, families had no electrical compatibility issues and found its size ideal for car transport. Design recommendations include retaining standard power requirements and enhancing portability with a compact, adaptable design.
4.2. CPT protocol
The CPT protocol offered a structured approach to identifying and integrating contextual factors during back-end design, following six key steps (Figure 2). First, we prepared for testing by identifying relevant contextual factors. Next, we collected qualitative data through in situ testing, using user observations and semi-structured interviews. Findings were then categorized into barriers and enablers. Barriers were translated into actionable design recommendations, while enablers were refined into sustainable design concepts that should be preserved for success. As such, the proposed CPT protocol includes an observational template (Figure 4) and an interview template that will vary based on the studied product and the stakeholders involved.
In our case study, the interview was structured to explain the study’s purpose, which is to identify key contextual factors that may influence the design, followed by a section exploring parents’ initial reactions. The protocol included 2-3 questions for each relevant contextual factor (technological, socio-cultural, environmental, economic, and infrastructural). The interview concluded with a reflection on similar experiences with other relevant technologies and products, in our case toys, and a section for participants to ask questions or provide any additional feedback.
A critical step in the CPT protocol was analyzing the data gathered. We clustered data from multiple engagement sessions into contextual factor categories. Each contextual factor was then evaluated to determine whether it enabled a positive experience within a given context or posed a potential barrier to use and adoption. Enablers were identified as aspects of the design to be preserved, while barriers were analyzed to generate specific recommendations for design modifications and iterations. While our current protocol did not include specific guidelines for analyzing trade-offs that may arise when identifying recommendations from enablers and barriers, future work aims to explore these considerations.
CPT protocol for observations
5. Discussion
Our pilot implementation of the CPT protocol enabled us to uncover critical contextual factors that can facilitate the adoption of the interactive toy chest across diverse use settings. Additionally, it provided a tool for identifying significant barriers that must be addressed through detailed design decisions and broader considerations for effective product implementation.
Identifying and integrating contextual factors during later design stages is a critical practice in engineering. It enables the recognition of barriers and enablers that could influence the product’s adoption and should be accounted for in design decisions. However, a review of the literature reveals no clear consensus regarding how designers should evaluate or incorporate context into their processes or proposed solutions (Reference Lissenden, Maley and MehtaLissenden et al., 2015). The CPT protocol supports these challenges by enhancing alignment with user needs and relevant contextual factors during back-end design stages. By encouraging designers to systematically identify and integrate relevant contextual factors into their design process, the CPT Protocol could help mitigate adoption barriers, recognize the design strengths through enablers, and ensure products resonate with diverse end-user perspectives and environments.
The CPT protocol builds on existing design theory and methodologies by addressing the gap in contextual integration during back-end stages. Future research is needed to deepen our understanding of how contextual factors influence design outcomes and to explore how integrating these factors during back-end design stages can lead to more effective and successful design practices. Additionally, future work should investigate the use of the CPT protocol as an educational tool to support the development of skills in identifying and integrating contextual factors in engineering design. This includes equipping engineers with strategies for stakeholder engagement and the ability to collect and analyze qualitative data to improve design outcomes.
This study has several limitations; we did not assess the relationship between using the CPT protocol and improving the success of a design process. We also conducted our case study with only one technology in one specific context. These factors may limit the generalizability of findings to broader cultural and geographic contexts. Lastly, participant backgrounds, geographic location, and the exclusion of certain contextual factors may have influenced the results and likely affect the protocol’s transferability to other settings. Future research could address these limitations by involving a more diverse range of stakeholders from various locations and cultural backgrounds, conducting observations in different environments and settings, and testing the protocol with different product types to better assess its broader impact.
6. Conclusion
The CPT protocol represents a step forward in overcoming the challenges of identifying and integrating contextual factors during back-end design stages. By aligning products more closely with their intended contexts, this method aims to enhance the understanding of user needs, mitigate adoption barriers, build upon enablers that constitute sustainable design concepts, and emphasize the relevance of contextual factors in driving impactful design decisions. This research establishes a strong foundation for refining methods that leverage contextual factors to inform design decisions such that realized solutions are suitable for their intended use settings.
Acknowledgments
We express our gratitude to the families that volunteered to participate in this study and to our research assistant, Lowell Glovsky, for his dedicated assistance with data collection. We also extend our thanks to Prof. James Harper for his guidance on development of the interactive toy chest prototype.
Barriers and Proposed Design Recommendations Across Contextual Factors.
Open access
