2.1 Pathway 1: the ‘technology-driven’ pathway

Pathway 1 ( $\mathbf{P}_{\mathbf{1}}$ ) as shown in Figure 1 is an idealized depiction of how new technologies are first built and then deployed in the domain. However, the resultant consequence of these properties is that vector $\mathbf{R}_{\mathbf{1}}$ in Figure 2 is inherently unattainable. Any degree of technological enhancement will inevitably shift the characteristics of the work domain (e.g. change in work goals or introduce undesired work demands) to some degree as represented by vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ . Vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ can shift only to the right to signify that the existing domain becomes novel to some extent but does not necessarily imply that the shift is entirely intentional or desired. This shift highlights the inherent coupling that exists between new technologies and the inability of the work domain to fully utilize/accommodate them.

Figure 2. Pathway ( $\mathbf{P}_{\mathbf{1}}$ ): the ‘technology-driven’ approach to addressing the envisioned world problem. The dashed vectors represent the idealized design intentions. Solid vectors represent the realistic progress made via the technology-driven approach. Dashed arrows indicate idealized vectors and solid arrows indicate actual vectors.

New technologies may provide innovative capabilities that appear to align with work domain goals, but in practice may fail to provide meaningful support to domain operators. Some examples of this phenomenon include the under utilization and resistance of electronic health records by health professionals (Declerck & Aimé Reference Declerck and Aimé2014), the rejection of electronic flight strips in air traffic control (Mackay Reference Mackay1999), and the limited adoption of electronic checklists by astronauts (Simonds & Chen Reference Simonds and Chen1991). As depicted in Figure 2, once a new technology is fielded in the work domain, a resulting shift in domain structure of intended aim and consequence occurs as represented by vector $\mathbf{R}_{\mathbf{2}}^{\prime }$ to compensate for the presence of the new technology.

Challenges exist in aligning the resultant state $\mathbf{B}^{\prime }$ with the desired end state $\mathbf{B}$ across the horizontal axis. Often times the extent to which the work domain can adapt with the new technology is a function of a number of compounding factors such as: the way the technology was implemented, the training of users, operator willingness to adopt the technology, and the ‘work-arounds’ developed to overcome technological deficiencies. A more alarming characteristic is that the desired end state was only partially defined or not defined at all at the onset of the technological development. Britcher (Reference Britcher1999) provides an example of the consequences of when the end state is not clearly defined and a $5 billion, 14 year automated air traffic control system development effort fails to deploy. We argue this result is due to the lack of adequate envisioned work domain definition and a narrow focus on only technology-driven solutions.

As a consequence of trying to resolve actual and ideal end states, the work domain settles for what the technological capabilities afford. Human operators develop ‘work-arounds’ as coping mechanisms to overcome technological deficiencies or use the technology in unintended ways (Vicente et al. Reference Vicente, Roth and Mumaw2001; Flanagan et al. Reference Flanagan, Saleem, Millitello, Russ and Doebbeling2013). The difference between the desired and actual modifications in work domain states (e.g. the difference between $\mathbf{R}_{\mathbf{2}}-\mathbf{R}_{\mathbf{2}}^{\prime }$ ) represents the compensation that human operators must dedicate to utilize (or not) the new technological capabilities. As a result, the difference between vector $\mathbf{R}$ and $\mathbf{R}^{\prime }$ is a visual representation of the moving target problem that plagues the envisioning process (Woods & Dekker Reference Woods and Dekker2000). The magnitude of unexplained variance introduced under this notional schematic highlights the resultant iterative nature of a ‘technology first’ development perspective.

In summary, pathway $\mathbf{P}_{\mathbf{1}}$ involves a familiar process of first developing new technological capability based on the premise of solving today’s problems by applying technology while ignoring broader system perspectives. The work domain incorporates the new technology and is enhanced to some extent. The unintended consequences are realized and operators develop work-arounds to adopt the technology or the technology is outright rejected. Subsequent iterative development efforts are invoked and the cycle continues ad infinitum to reach the desired end state or until budgets restrict development efforts. The application of EMRs in the health care work domain highlights some of the challenges that can accompany pathway $\mathbf{P}_{\mathbf{1}}$ such as increased time required for data entry, mismatches between user interfaces and clinical workflow, interferences with physician–patient face-to-face conversation, hampered information exchange, information overload, and deterioration of clinical documentation (Declerck & Aimé Reference Declerck and Aimé2014). However, a movement towards an alternative approach within the health care domain has recently begun; one that involves first capturing the necessary work domain demands to then inform technological development (Declerck & Aimé Reference Declerck and Aimé2014; Hettinger, Roth & Bisantz Reference Hettinger, Roth and Bisantz2017). Additionally, our own case study exemplar adopts the perspectives of pathway $\mathbf{P}_{\mathbf{2}}$ as described in the subsequent section.

2.2 Pathway 2: the ‘work-driven’ pathway

Rather than emphasizing technological capabilities at the onset of addressing the envisioned world problem, we contend that first envisioning the work domain in a future context is a more useful first step. Under this perspective, we view the work domain as ‘the system being controlled, independent of any particular worker, automation, event, task, goal, or interface’ (Vicente Reference Vicente1999, p. 10). An important aspect of this perspective is that tasks (e.g. actions to be performed by agents within the domain to accomplish a goal) are heavily influenced by hypothesized technological systems and physical artefacts being utilized. The artefacts themselves influence the specific tasks to be performed. Therefore, the problems and constraints of the future domain should be articulated. Formative modelling efforts such as CWA provide a viable avenue to systematically examine the work domain via constraint definition that can be more readily extended into a future context (Rasmussen et al. Reference Rasmussen, Pejtersen and Goodstein1994; Vicente Reference Vicente1999; Miller & Vicente Reference Miller and Vicente2001; Salmon et al. Reference Salmon, Jenkins, Stanton and Walker2010). To the best of our knowledge, the CWA framework has yet to be fully integrated with the envisioned world problem as we have posited.

We contend that the constraints that shape the existing work domain will more often than not also be present in a future context and therefore provide a valuable starting point for envisioning. More specifically, the CWA insights gained from the first three levels of the work domain abstraction hierarchy (AH) models, combined with decision ladders provide a critical linkage between the existing and hypothesized future domains (Miller, McGuire & Feigh Reference Miller, McGuire and Feigh2017). Nevertheless, work domain examination at the onset of the envisioning process is necessary, but not sufficient. The extension of current work domain understanding should be clearly linked to the future context. This perspective leads us to explore how the necessary work domain insights might be derived to place operators in a realistic future work context.

Figure 3. Pathway $\mathbf{P}_{\mathbf{2}}$ : the ‘work-driven’ approach to addressing the envisioned world problem.

Under this ‘work-driven’ perspective, as shown in Figure 3, the existing work domain attributes (e.g. constraints, problems, expectations and technologies) are extended to a novel envisioned setting by defining the problem(s) and constraint(s) that are likely to exist. An important component of this process is to convey the likely shifts in work domain structure and distribution of work functions within the future context. Examination of the domain from a formative perspective allows for the identification of problems and constraints on the work domain. Once identified, these issues that link and/or change between the current and future domains can be analysed more thoroughly.

The physical constraints of the domain are often times more tangible to articulate, define and justify, given their physical nature. For example, future human spaceflight operations will contend with a communication delay constraint between crew and Earth-based support personnel (Love & Reagan Reference Love and Reagan2013; Rader et al. Reference Rader, Reagan, Janoiko and Johnson2013) (e.g. one-way light time communication between Mars and Earth varies from 4 to 20 minutes depending on relative planetary positions). Examining the application and consequences of such constraints is a valuable part of the envisioning process. Furthermore, establishing the presence of new or altered constraints enables the discussion of subsequent implications and work demands those constraints will impose within the future work domain.

A choice exists regarding what technological capabilities may have a place in the future domain. An important assumption made along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ in Figure 3 is that all existing technological artefacts are first considered, where applicable, within the future context. Asking questions such as: what would that resultant system look like if the similar artefacts are utilized? Would these artefacts provide the desired capabilities to fulfil the expected work functions? provide an initial step towards progressing along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ . These incremental shifts must be made explicit so that the existing domain can be more clearly mapped to an envisioned setting.

These translational efforts help establish a baseline definition of the future work domain and offer two benefits: (1) the apparent deficiencies in existing technologies are more readily visible when examined from a future context, and (2) potential desirable technological enhancements can be more readily identified and described to reach the desired end state. Note here the separation of identifying deficiencies and potential solutions. All too often technological solutions are proposed without fully understanding the intended work to be supported. Vector $\mathbf{R}_{\mathbf{4}}^{\prime }$ is shown in Figure 3 in the grey region as a range of potential directions which is in contrast with vector $\mathbf{R}_{\mathbf{1}}^{\prime }$ found in the ‘technology-driven’ approach.

The examination of existing technologies within a future context, rather than an existing context, enables a more detailed examination of what capabilities may be desired to support the work domain. Some existing domain technologies may not need enhancement to remain useful in a future context. Technologies should be hypothesized from a vantage point that more accurately encapsulates the context of intended use. The inclusion of new technologies from a future context where a better understanding of work demands can be represented provides a more narrow solution space of hypothesized design solutions thereby increasing the likelihood of effective design solutions.

2.3 Summary

In an idealized sense, technology-driven pathway $\mathbf{P}_{\mathbf{1}}$ represents a familiar and commonly adopted approach to the envisioned world problem: a new technology is developed and installed in a work domain and a new desired state is reached by operators compensating for the deficiencies or burdens the new technologies impose on the work being performed. Contemporary challenges of pathway $\mathbf{P}_{\mathbf{1}}$ include the digital revolution of EMRs in the medical field (Buntin et al. Reference Buntin, Burke, Hoaglin and Blumenthal2011; Ano 2015b ), next generation automated systems in air traffic control (Sarter & Amalberti Reference Sarter and Amalberti2000; Durso & Manning Reference Durso and Manning2008; Ano 2015a ), military command and control (Jenkins, Walker & Rafferty Reference Jenkins, Walker and Rafferty2012), and rail transport (Bearman et al. Reference Bearman, Naweed, Dorrian, Rose and Dawson2013). However, as discussed in the previous section, there exist systematic limitations to the affordances provided by pathway $\mathbf{P}_{\mathbf{1}}$ and challenges that must be addressed.

But there is another way to approach the envisioned world problem that first de-emphasizes technological development and prioritizes work domain definition. Work-driven pathway $\mathbf{P}_{\mathbf{2}}$ emphasizes the transition of the current work domain into a hypothesized future work domain ( $\mathbf{R}_{\mathbf{3}}$ ) to first consider the envisioned work to be expected in that domain as a means to then determine what technological capabilities might be necessary ( $\mathbf{R}_{\mathbf{4}}$ ) in that setting. The translation from State A to B can, and should, be performed by first emphasizing the definition of the future work domain where the current technologies, actors, environment, and problems are considered and translated to a future context. Under this concept, the focus is not on what new technologies could afford but rather how might the future domain resemble (or not) the existing domain.

For example, if NASA aims to extend a crew of 4 or 6 people from low-Earth orbit into deep space where they are effectively isolated from Earth-based support personnel, an understanding of the shifts in the current domain of human spaceflight operations (e.g. redistribution of work functions, responsibility and authority) must be acquired. A key aspect of vector $\mathbf{R}_{\mathbf{3}}$ is that the future context is considered with heritage technological capabilities. In other words, the already existent technological artefacts remain constant while the work domain characteristics (e.g. roles, responsibilities and work domain functions) are allowed to shift to meet hypothesized new work conditions, expectations and environment. We contend that this distinction offers a more realistic and meaningful way to reach the desired target (State B) as represented by vector $\mathbf{R}_{\mathbf{4}}$ . Some recent examples that implement this approach along pathway $\mathbf{P}_{\mathbf{2}}$ include the development efforts in the domains of human spaceflight (Miller et al. Reference Miller, McGuire and Feigh2017) (to be used as a specific case study example in this paper) and health care informatics (Pennathur et al. Reference Pennathur, Cao, Sui, Lin, Bisantz, Fairbanks, Guarrera, Brown, Perry and Wears2010; McGeorge et al. Reference McGeorge, Hegde, Berg, Guarrera-Schick, LaVergne, Casucci and Hettinger2015; Hettinger et al. Reference Hettinger, Roth and Bisantz2017). In an era where technological capability is ever increasing, systems engineers and designers of technology must understand the realistic work demands before hypothesizing technological solutions.

Both pathways shown in Figure 1 constitute a spectrum of various approaches to addressing the envisioned world problem. However, the envisioned world problem has yet to be fully explored along these pathways, and furthermore there exists a lack of theoretical or practical guidance as to what methods and considerations could be leveraged to define, develop, and ultimately advance envisioned world problems along these various pathways. We argue that work-driven pathway $\mathbf{P}_{\mathbf{2}}$ offers a unique and promising avenue that could overcome some of the traditional challenges faced along the technology-driven pathway $\mathbf{P}_{\mathbf{1}}$ . No longer can technological advancement be thought about in isolation from realistic work domain expectations (Carroll Reference Carroll1991; Stary & Peschl Reference Stary and Peschl1998; Lintern Reference Lintern2012; Dekker, Hancock & Wilkin Reference Dekker, Hancock and Wilkin2013). Additionally, cognitive systems engineering (CSE) practitioners will play an increasingly important role in the envisioning process, acting as the arbiter of design insight early in the design process that links both work domain and technological attributes to yield effective designs solutions (Feigh et al. Reference Feigh, Miller, Bhattacharyya, Ma., Krening and Razin2018).

We propose that it is imperative to correctly identify and articulate the necessary functions within a domain as a necessary starting point for appropriately envisioning a future domain. It is from this functional understanding of the domain that the envisioning process can take place. As the requirements articulate, functions (that are already identified and desired) must be allocated on the basis of appropriateness and capacity with deliberate design reasoning. The application of technological capabilities should be thought of as a design hypothesis about how best to complete and overcome work domain demands. Therefore, speculating on the utility of new technological capabilities should not necessarily be the place to dedicate time, attention, and resources. The envisioning process at the onset requires an intimate understanding of the functions and constraints to be accommodated within the envisioned work domain, and should be where attention and efforts are devoted.

To this point, we have elaborated and discussed the envisioned world problem from two different pathways. We have argued that a work-driven pathway is a more desirable and useful approach for the envisioning process. In the following section, we will discuss how to follow the work-driven approach of $\mathbf{P}_{\mathbf{2}}$ by acquiring and articulating the constraints in the form of CSE requirements that shape domain behaviours.

3.1 Natural history

The ‘natural history’ stage of observation refers to the examination of traits exhibited by the existing (or historical) work domain. At this stage, the focus should be on ‘discovering or identifying the processes that drive performance and adaptation (Woods Reference Woods2003, p. 43)’ within the work domain. Additionally, establishing how and for what purposes should the domain be studied is important to consider at the onset of investigation. The work domain under this context includes not only psychological and technical components/limitations that already exist, but also the social and relationships within the domain (see (Bisantz & Burns Reference Bisantz and Burns2009, Ch. 8) for more in-depth discussions of work domain considerations.)

Fortunately, many methods already exist to examine an existing work domain; spanning from ethnographic investigations to applied CSE methods that we will not discuss in detail in this paper. Cognitive systems engineering methods collectively strive to ‘identify the basic requirements for how to support work that must be met if new technology will be useful to practitioners in context (Woods & Hollnagel Reference Woods and Hollnagel2006, p. 178).’ Cognitive systems engineering methods in particular provide a host of appropriately defined vantage points to conduct the examination of a current/historical domain that emphasize the characterization of work and the interplay between people, technologies and organizational factors. For example, contextual inquiry, provides a set of representations to help articulate aspects of work that range from the physical artefacts to the cultural arrangement of the work environment. Cognitive work analysis elicits through a set of modelling tools the underlying constraints that shape work domain behaviour (Rasmussen et al. Reference Rasmussen, Pejtersen and Goodstein1994; Vicente Reference Vicente1999). For a comprehensive review of CSE methods and their applications, see Bisantz & Roth (Reference Bisantz and Roth2008), Bisantz & Burns (Reference Bisantz and Burns2009), Read, Salmon & Lenne (Reference Read, Salmon and Lenne2012), Jiancaro, Jamieson & Mihailidis (Reference Jiancaro, Jamieson and Mihailidis2013). We acknowledge other methods outside of the CSE literature are likely applicable to the envisioning process but we constrain our discussions to the applicability of CSE methods, and more specifically CWA, in this paper.

While CSE methods provide an abundance of modelling tools that aim to capture the work as-practised, limited guidance is currently provided for progressing work domain insight into actionable envisioning efforts. More specifically, limited methodological support exists to help traverse what we identify here as the gulf of conception. In other words, how do we bring work as-practised understanding into a future context? To answer this question, we propose a greater emphasis on requirements definition using CWA modelling to overcome the challenge defined here as the gulf of conception, or translation transition along vector $\mathbf{R}_{\mathbf{3}}$ in our envisioning efforts as shown in Figure 1.

3.2 Gulf of conception

The challenge of the gulf of conception is to adequately translate existing/historical work domain understanding into a future setting. CWA methods to-date still leave practitioners with the responsibility of ‘crafting their own approach to design’ (Read et al. Reference Read, Salmon and Lenne2015, p. 171). Therefore, inherent work domain demands should be grounded in past/current domain experience. This premise is slightly different yet complimentary to existing approaches such as the future incidents methods (Smith et al. Reference Smith, Woods, McCoy, Billings, Sarter, Denning and Dekker1998) or ecological interface design (Vicente Reference Vicente2002). Rather than immediately explore what the envisioned work space might entail in particular hypothesized scenarios or propose design solutions in isolation, we contend the appropriate mechanism to translate existing domain characteristics to the future setting is the articulation of requirements similar to those traditionally used by systems engineers to specify things like power, weight, and performance of an engineered system. Here we propose work domain requirements that reflect the cognitive demands and information relationships inherent to the work domain as defined below:

1. (i) Cognitive Work Requirement (CWR): Specifies the cognitive demands, tasks, and decisions that arise in the domain and for which the operator requires support.

2. (ii) Information Relationship Requirement (IRR): Specifies the proper context for the required data, turning it into information that the decision maker requires.

These particular requirements were first developed by Elm et al. (Reference Elm, Potter, Gualtieri, Roth and Easter2003), Potter, Gualtieri & Elm (Reference Potter, Gualtieri and Elm2007) to achieve a similar goal of informing the systems design process. As a result, these requirements reflect the intrinsic demands associated to the work domain and are thus requirements that must be satisfied in any future hypothesized work domain. Note that traversing the gulf of conception does not involve proposing any specific design solutions, but rather focuses on the specification of requirements at some higher level of abstraction. With regards to the envisioning process, all envisioned aspects of both work domain characteristics (e.g. how it is arranged, the envisioned operators, allocation of responsibility and authority) should be considered hypotheses that strive to meet the previously stated requirements. In effect, these work domain requirements offer a standard by which each hypothesis can be prioritized, assessed, and ultimately validated.

At this point, we have explored opportunities and challenges that exist with studying the natural world and overcoming the gulf of conception. The following sections examine the challenges and perspectives relevant to the development of more controlled instances of the envisioned domain.

3.3 Staged world

The staged world provides an opportunity to both construct and situate observations within an envisioned context to examine the nature of practice and to help reveal what would (could) be useful (Woods & Roth Reference Woods and Roth1988). Some research efforts provide guidance in constructing the staged world including scenario-based design (Carroll Reference Carroll2000), the future incidents technique (Smith et al. Reference Smith, Woods, McCoy, Billings, Sarter, Denning and Dekker1998), and synthetic task environment design (Flach et al. Reference Flach, Schwartz, Courtice, Behymer, Shebilske, Miller and Patterson2012). The important aspect here is that the future context should be imagined in as many ways and from as many, relevant stakeholder perspectives as possible.

This stage should be considered fluid so that it can accommodate a variety of perspectives from which the envisioning process can be scoped within the larger community of stakeholder perspectives. Additionally, this stage helps discern aspects that are similar or dissimilar from the existing work domain. Finally, the staged world enables the exploration of the variability that arises from situated operations (Turvey, Shaw & Mace Reference Turvey, Shaw, Mace and Requin1978; Vicente Reference Vicente2000). This opportunity not only allows subjects to become familiar with the envisioned context, but also enables the familiarization of the implications of new/future demands.

In summary, the staged world provides an opportunity to gain operational experience within the future context while positioning their work within the larger body of development that may be taking place amongst the team of stakeholders. This observational opportunity allows contrasts to be made between the existing domain and what the future might entail and helps identify areas for more targeted research objectives to be examined in a more controlled setting, known as the spartan laboratory.

3.4 Spartan laboratory

The spartan laboratory encapsulates the more traditional experimental environments that abound within the academic literature. Commonly known as micro-worlds, spartan labs provide researchers a platform with extensive control to explore theoretical model development and technology evaluation. Historically, spartan labs have suffered from a lack of work context by operating under a simplified environment which makes results difficult to generalize to the natural world (Brehmer & Dörner Reference Brehmer and Dörner1993; Omodei & Wearing Reference Omodei and Wearing1995; Brehmer & Elg Reference Brehmer, Elg, Sterman, Repenning, Langer, Rowe and Yanni2005; Gonzalez, Vanyukov & Martin Reference Gonzalez, Vanyukov and Martin2005), or limited advancement beyond paper-based conceptual designs (Naikar et al. Reference Naikar, Pearce, Drumm and Sanderson2003).

If the laboratory setting is viewed as part of the envisioning process, where the relevant constraints and problems found in the natural world are extended via the staged world to the spartan settings, then a strategically scoped and relevant environment can be generated. The spartan laboratory under this context still utilizes artefacts as a tool of discovery, but within a contextually relevant setting that enables a more detailed data collection and synthesis effort. In a spartan lab setting, the opportunity to explore more targeted evaluations of new technological capabilities can be made, without sacrificing the important contextual demands of the work domain. Furthermore, the construction of the spartan laboratory itself is an excellent exercise in defining the unavoidable assumptions (e.g. specific work practices, goals, constraints) that face the envisioned domain.

3.5 Gulf of extent

Consider for a moment that the gulf of conception is traversed and the development of the staged and spartan environments are under way. The challenge now shifts to being able to effectively iterate between the staged and spartan environments. Some useful simulation descriptions exist that describe aspects of the domain that are important to consider (Harvey et al. Reference Harvey, Buondonno, Kopardekar, Magyarits and Racine2003; Pennathur et al. Reference Pennathur, Cao, Sui, Lin, Bisantz, Fairbanks, Guarrera, Brown, Perry and Wears2010; Williams, Medicine & Administration Reference Williams, Christopher, Drechsler, Pruchnicki, Rogers and Silverman2014); these resources however remain embedded within their own specific domains and do not address the envisioned world problem directly. The envisioned context in some ways must maintain traits that resemble the existing domain to promote subject-matter expert (SME) adoption, but in some ways there will be significant departures.

Coping with this extensibility challenge is defined here as the Gulf of Extent where work domain aspects must be prioritized and adopted by the community at large. Individuals embarking on the envisioning process will quickly realize that the ability to prioritize work domain aspects (e.g. specific problems or demands) will be a necessity. Conveying these priorities and in some cases convincing the community at large that these perspectives are important will be a sizeable task. Questions such as: what are the important problems to simulate; what are the scenarios (with appropriate demands) worth simulating; how do we capture objectifiable insight from those experiences? must be answered.

The central tenant here is that the staged and spartan worlds provide spaces for design hypothesis generation and exploration to help support the envisioning process by trial/error and subsequent refinement. Assumption management becomes important to articulate and control what is and is not represented in these work settings. Unfortunately, mechanisms for controlling the domain assumptions are not well defined in the literature. Furthermore, the development and implementation of appropriate measures by which to quantify simulation fidelity and assessing work performance is lacking. Some literature does exist to assist with measures development (Patterson & Miller Reference Patterson and Miller2010); however, the ability to apply performance measures that span both spartan and staged worlds is an open area for investigation.

3.6 Summary

By unpacking the envisioning process, system behaviours can be observed and documented that capture the realities of the observed stages as opposed to only measuring ‘successful’ system performance (Brown et al. Reference Brown, Reeves and Sherwood2011). Furthermore, by contemplating the envisioned domain at various stages of definition, a more clear articulation of ‘what is being argued against in order to be able to articulate what is in favour of [in terms of technological capabilities] (Rooksby Reference Rooksby2013, p. 19:11)’ can be made. We argue these collective perspectives provide a tenable approach to traverse along vector $\mathbf{R}_{\mathbf{3}}^{\prime }$ as shown in Figure 4. However, to this point, limited theoretical discussion exists to more fully elaborate on the gulfs that presently exist.

To this point, we have emphasized a lens through which the envisioned world problem can be approached. The perceived benefits of technologies must be weighted against the demands and needs of the work domain in a more explicit manner. It is through this process of acquiring work domain insight and perspective that how work domains operate can be understood (Hutchins Reference Hutchins1995; Woods Reference Woods2003). In doing so, technological capabilities can then be better hypothesized to meet those work demands. By systematically approaching an envisioned world domain state, via the various aforementioned stages of observation and simulation, we contend that a more tenable process for work domain and technology development can be reached. It is important to note that up to this point in the paper we have offered one theoretical approach to tackling the envisioning process, i.e. the $\mathbf{P}_{\mathbf{2}}$ pathway. This paper goes one step further to provide a concrete example of how we applied this approach and to link it to other CWA literature. The remainder of this paper explores the application of the $\mathbf{P}_{\mathbf{2}}$ pathway approach to the envisioned world problem in practice.

4.1 Addressing the envisioned world problem in practice

Figure 5 shows the various data collection, model development, and in situ observation efforts performed at each stage of the EVA envisioning process via Pathway 2 for this study. These efforts spanned four years of research and offer one collective perspective that we found successful to design and test a DSS for use in a future EVA setting. The various milestones indicated in Figure 5 were guided by questions that pertained to how the current work domain could inform the design of new technological capabilities to be used in future EVA operations. The list below describes the motivating objectives used to guide this research along with how each objective relates to the stages of observations. Additionally, published manuscripts associated with each stage of research are provided for the interested reader to elaborate on the details that supported our envisioning process. The following sections depict the advances and challenges experienced during this envisioning process and highlight various aspects of the process that we believe others will find useful.

Figure 5. The research activities involved in our envisioning process that follows Pathway 2.

Natural History:

Identify the system constraints on EVA operations (Miller, McGuire & Feigh Reference Miller, McGuire and Feigh2015a ,Reference Miller, McGuire and Feigh b ; Miller et al. Reference Miller, Claybrook, Greenlund and Feigh2016a ; Greenlund, Miller & Feigh Reference Greenlund, Miller and Feigh2017; Miller et al. Reference Miller, Claybrook, Greenlund, Marquez and Feigh2017a ).

Natural History/Gulf of Conception:

Develop the requirements for a DSS for EVA operation within an envisioned context (Miller et al. Reference Miller, McGuire and Feigh2017) (Miller Reference Miller2017, Appendix A2).

Staged World:

Identify the characteristics of the DSS design that are likely to fulfil the DSS requirements in an envisioned context (Chappell et al. Reference Chappell, Beaton, Miller, Halcon, Michael and Abercromby2016; Miller et al. Reference Miller, Lim, Brady, Cardman, Bell, Brent Garry, Reid, Chappell and Abercromby2016b , Reference Miller, Coan, Abercromby and Feigh2017b ; Beaton et al. Reference Beaton, Chappell, Abercromby, Miller, Nawotniak, Hughes, Brady and Lim2017; Miller, Pittman & Feigh Reference Miller, Pittman and Feigh2017) (Miller Reference Miller2017, Ch. 4).

Spartan Lab:

Assess how well the prototyped DSS performs in envisioned EVA operations (Miller Reference Miller2017, Chs 5 and 6) (Miller et al. Reference Miller, Pittman and Feigh2017).

The remainder of this section is a revisit of recently published work that applies CWA for the development of work requirements (Miller et al. Reference Miller, McGuire and Feigh2017). We briefly discuss these efforts from an envisioned world perspective.

4.2 Traversing the gulf of conception

One of the critical components of the envisioning process itself is to articulate existing knowledge about a work domain in a way that is relatable to a future context. To support this effort, we distilled the volume and variety of observations made during the natural history stage of observation by constructing cognitive work and information relationship requirements based on the SME SoKs derived directly from the decision ladders. In doing so, we compiled a comprehensive set of requirements that we then prioritized to explore in the envisioned future context.

Throughout our progression of pathway $\mathbf{P}_{\mathbf{2}}$ , we aimed to integrate our CSE insight into the larger systems engineering process which expects requirements to guide system development; see Elm et al. (Reference Elm, Gualtieri, McKenna, Tittle, Peffer, Szymczak and Grossman2008) for a review. Previous examples of the requirements derivation process utilized AHs as a tenable way to derive these types of requirements (Burns et al. Reference Burns, Bryant and Chalmers2005). However, we found that using the decision ladders proved more useful in terms of minimizing overhead in data management on our part and facilitating fruitful discussion with SMEs. Additionally, AHs can be difficult to explain to SMEs, whereas the Decision Ladder model is a more straightforward model to utilize in an interview setting. Each SoK was assigned at least one cognitive work requirement (CWR) and information relationship requirement (IRR) pair that recast the SoK questions into a more standardized requirement format. Figure 6 below shows an excerpt of this process for the Set of Observations stage of the DL.

Figure 6. Requirements example for the set of observations stage of the DL to depict the SoK, CWR, and IRR statements for timeline management as the overall DL goal.

We assessed the requirements based on the standard requirements format expected by the traditional engineering community as stated by Turk (Reference Turk2006). Requirements are expected to be necessary, correct, unambiguous, traceable, prioritisable, results oriented, verifiable, and feasible. Our initial assessments of our derived early design requirements meet these requirements with only verifiable and feasible attributes left as outstanding characteristics to be more concretely examined at later stages of the design process. The envisioning process requires the identification of what is necessary and correct to examine early in the design process. The WDA and ConTA help distil this necessary information in a traceable and prioritisable way and the instantiation of requirements provided a useful mechanism for extending our domain understanding into a future context.

Three main insights were gained from the natural history examination: (1) the identification of two key work domain functions (life support and timeline management) to be examined in a future context; (2) we distilled the high level requirements associated with performing those functions as they exist in the current domain to help define what constraints and considerations need to be accounted for in the future; (3) the intravehicular operator was identified as the likely user for an advanced DSS system. The IV operator will likely take at least some of the current work expectations found in the EVA flight controller community when we shift the operational context to a future, time-delayed communication setting. By narrowing a broad natural history investigation to a few key intrinsic domain attributes, we now had the necessary prerequisite understanding to develop, explore and study future EVA operational settings in the staged and spartan laboratory settings.

4.3 Situating design solutions within envisioned settings along the $\mathbf{P}_{\mathbf{2}}$ pathway

At this stage of the envisioning process along Pathway 2, we explored what the envisioned EVA work domain might look like using both the large-scale simulated staged and controlled spartan laboratory settings. By leveraging both environments, we could compare and contrast envisioned domain components with heritage domain characteristics, and begin to explore the impact of new technologies in an envisioned setting. We engaged in a range of research activities that capitalized on already active development of future EVA operations.

The following sections describe our envisioned staged and spartan world research activities and discuss what we found important while we were engaged in such activities. For comparison, Figure 7 shows a summary depiction of our activities made across all three stages of our envisioned world problem. We studied and built an information flow model of the existing domain which we then used to compare our staged and spartan models. With respect to our DSS development efforts, we highlight in Figure 7 the intravehicular operator who is our key user of our envisioned system. It was from this user perspective that we joined the research teams of various on-going NASA analogue research programs that aligned with the staged world phase of the envisioning process. These efforts provided the opportunity for us to (1) build familiarity with complementary research activities, (2) contrast these observations with insight gained from the natural history examination of the existing domain, (3) enabled an understanding of where current research interests are within the community at large.

Figure 7. Domain understanding elicitation among the various stages of observation with the intended goal of enabling the Intravehicular (IV) operator under different degrees of simulation fidelity.

4.3.1 Staged worlds – becoming embedded in the future context

Multiple concurrent efforts were performed by NASA to better understand future human spaceflight operations in the form of simulated mission analogues. We were able to leverage these efforts instead of creating them from scratch. Figure 8 shows the Earth-based analogue research programs we participated in as part of our envisioning process.

Figure 8. Staged world participation within three NASA analogue research programs: PLRP: Miller et al. (Reference Miller, Lim, Brady, Cardman, Bell, Brent Garry, Reid, Chappell and Abercromby2016b ), NEEMO: Chappell et al. (Reference Chappell, Beaton, Miller, Halcon, Michael and Abercromby2016), BASALT: Beaton et al. (Reference Beaton, Chappell, Abercromby, Miller, Nawotniak, Hughes, Brady and Lim2017), Deans et al. (Reference Deans, Marquez, Cohen, Miller, Deliz, Hillenius and Hoffman2017).

We found it important to treat these staged worlds as an opportunity to observe and learn what the community at large was interested in understanding. Therefore, we integrated our research efforts for developing an IV-specific DSS within the scope of the larger research objectives pursued by these analogue programs. Below are a few important characteristics and insights that shaped our envisioning process within the staged worlds:

1. (i) We found it important to actively seek out experiences that had relevance to the staged/spartan stages of the envisioning process. The opportunities available to us predominately resided in the staged world phase of the envisioning process which we leveraged for our subsequent spartan laboratory development efforts.

2. (ii) The staged world allowed us to assess what aspects of the requirements derived from the natural world were actively being pursued in the envisioned context. We internally assessed the requirements in relation to what was observed. Table 1 shows a comparison of our resultant spartan laboratory setting to these larger-scale staged world experiences.

3. (iii) The staged world enabled us to observe what physical and software artefacts currently exist. This is important because up to this point in the design process, we purposefully limited generating design hypotheses so that we could gain familiarity with the future context. In our specific example, the construction of IV support systems had limited formal investigation prior to our own research efforts. Therefore the staged world helped us confirm that our spartan lab and DSS design efforts were in fact novel and we were not duplicating previous works.

4. (iv) The staged world enabled the identification of what current measures and metrics of EVA execution were deemed valuable. The natural history observations revealed the meticulous nature of EVA timeline execution and life support system scrutiny, whereas, the staged world was interested in quantifying EVA performance and assessing the timeline designs. These comparisons were useful to identify what was worth considering for our own spartan laboratory setting.

Table 1. Staged world and spartan lab support system comparisons (adapted from Miller et al., Reference Miller, McGuire and Feigh2017b)

As it related to our EVA work domain, we recognized from our WDA modelling that EVA operations are fundamentally influenced by the mission objectives. Consequently, the specific research objectives being pursued in these staged worlds had profound impacts on what could be observed in the staged work domain. For instance, the staged worlds shown in Figure 8 emphasized incorporating realistic scientific field investigations within the constraints of EVA operations. But from our natural history investigations, we recognized that the pursuit of scientific objectives has not been incorporated in the EVA domain since the Apollo program. Therefore, we aimed to incorporate available data from the Apollo program since that appeared to be a more representative domain example of future operations.

Finally, we needed to be strategic in contributing to the staged world development to avoid over-committing our time and resources. In our case, since we were interested specifically in timeline and life support system management from the IV operator perspective, we were actively engaged in all timeline development efforts and provided the staged worlds a simulated life support system software derived from present-day management displays. Furthermore, since we were interested in supporting EVA execution, we made sure to embed ourselves in the execution either within simulation subject to acquire a first-person perspective or as a silent observer for a third person point of view.

4.3.2 Spartan laboratory – targeted construction of the future context

A spartan laboratory simulation environment was built to test both a baseline and an advanced support system to examine the utility of each within relevant hypothesized operations (Miller & Feigh Reference Miller and Feigh2019). These support system designs are shown in Figure 9 with how each design element was linked to relevant operational data. Both designs provide the same underlying functional support to meet a prioritized set of requirements derived from the natural world, but the way in which that functionality is supported differs. Most notably, aspects of life support system and timeline management were performed using both static paper-based tools within the baseline design to represent present-day work domain practices. The advanced DSS supported the same calculations but required interfacing with a novel software tool. Most importantly, the design features found in each of these work stations could be traced to specific CWR and IRR requirements derived from the decision ladder models. (See Ch. 4 of Miller (Reference Miller2017) and Miller & Feigh (Reference Miller and Feigh2019) for more detail.)

Figure 9. Baseline and advanced DSS design configurations and the various information relationships expected to be performed by the IV operator. The arrows show how the information was originally scattered across many locations and artefacts; it is consolidated in the second design. Equations represent higher level operational data that is necessary to support EVA operations. ETA: estimated time of arrival, PET: phased-elapsed time, LSS: life support system.

Below are a few key aspects that we gleaned from our spartan laboratory design and testing campaign:

1. (i) An important question to be addressed is: what is important enough to simulate/emulate in the envisioned context? Building a contextually relevant environment is a time and resource intensive endeavour and we found that starting with the present-day work artefacts from the existing domain provided a useful starting point to build the simulations from scratch.

2. (ii) Another important component of our spartan laboratory was appropriately simulating the natural variability exhibited during operations. Crew timeline execution can vary both in terms of time and tasks performed. Therefore, to better understand the magnitude of this variability, we performed a detailed examination of Apollo lunar surface EVA operations as a surrogate to Mars surface operations. In doing so, we quantified the variability exhibited both to shape our spartan scenarios (see Miller et al. (Reference Miller, Claybrook, Greenlund and Feigh2016a , Reference Miller, Claybrook, Greenlund, Marquez and Feigh2017a ) for our operational assessment of the Apollo program).

3. (iii) The construction of the simulation elements, both hardware, paper products, and software is a non-trivial effort. We replicated many of the existing artefacts first in addition to designing an envisioned software tool that could support the same sort of work – timeline tracking and life support system monitoring. To conduct the simulation itself required the involvement of one researcher, two surrogate astronauts and the participant. See Figure 7 for a depiction of our spartan laboratory.

4. (iv) We also had to define the measures and metric definitions and incorporation of those data collection efforts into the scenario development. These data types needed to be domain specific to adequately capture performance. Complementary to this effort is the challenge of setting appropriate measures of control to implement during simulations to maintain simulation repeatability while still allowing for envisioned natural variability throughout the scenarios. For example, our simulation included both scripted and unscripted communication periods to ensure certain actions and information was exchanged as precise periods throughout the simulations.

Given the lack of formal work domain definition of possible future Mars surface operations, we assumed that utilizing novice EVA personnel provided the opportunity to examine the limits of crew performance within our spartan lab. As a result, we provided a one-to-one ratio of training and testing time to ensure the participants were adequately trained to perform the job functions specific to our requirements derived from the natural world.

4.4 Case study limitations

It is important to consider a number of limitations of the present study. The first is that this paper includes a single example in a single work domain application. While much of the presented work is largely domain agnostic, these insights were shaped by our experiences and interactions with our specific domain. Additional work is required to fully validate this envisioning process using other complex sociotechnical work domains, such as efforts demonstrated by Burns, Bisantz & Roth (Reference Burns, Bisantz and Roth2004).

Second, this integrative approach is not a complete theoretical description of the entire envisioning process. The collective envisioning process described in this paper attempted to integrate three theoretical components by incorporating aspects of CWA with adaptations of Wood’s observational stages of the envisioning process (Woods & Dekker Reference Woods and Dekker2000; Woods Reference Woods2003) along with Norman’s approach to bridging gaps in the design process (Norman Reference Norman1986). In particular, the latter staged and spartan stages and modulating characteristics between them have ample room for improvement. Contributions from fields such as Naturalistic Decision Making (Hoffman & Klein Reference Hoffman and Klein2017), and Macrocognition (Patterson & Miller Reference Patterson and Miller2010) are likely to add more resolution to the envisioning process and is an area of future work.

Finally, this study was limited in that we were only able to explore a small subset of the design space and out of necessity used a substantial number of assumptions, especially during the staged and spartan phases, to scope our desired research outcomes. The challenge still remains in how to manage the variety of assumptions required for work domain construction and how to scale these assumptions to include more complex DSS capability and more sophisticated and representative simulation scenarios. This research demonstrated how to incorporate a small subset of derived DSS requirements across the envisioning process as a useful starting point but the design hypotheses and associated assessment criteria are still at the research practitioner’s discretion.