1.1 Objectives for design prototyping

There are many potential objectives to design prototyping. This section explores common objectives of prototyping. To acquire this list, first the Authors’ reviewed the full set of articles to filter those which discuss the objectives of prototyping as a process. This resulted in a list of forty articles. Next the individual objectives discussed in each article were listed in a separate document. These objectives were grouped into emergent categories based on iterative binning. The binning was repeated by two raters and differences resolved through discussion to form a final integrated set of bins. Figure 2, presents the binning results. More commonly cited objectives are indicated through a larger chart area in the diagram (direct proportion to number of times listed). The remainder of this section will later define each objective. Remaining sections will highlight various techniques to achieve each objective.

Figure 2. Frequently cited prototyping objectives, relative prevalence (indicated by area). Design refinement was the most commonly cited objective.

Refinement

Refinement is the process of gradually improving a design. Several of the benefits of prototyping relate to design concept refinement. Prototyping is used to validate requirements, reveal critical design concerns (Gordon & Bieman Reference Gordon and Bieman1995), reduce errors (Viswanathan Reference Viswanathan2012), to identify performance-enhancing design changes (Viswanathan & Linsey Reference Viswanathan and Linsey2011, Reference Viswanathan and Linsey2012a ), design feature optimization via the sequential testing and manipulation of parameters (Anderl, Mecke & Klug Reference Anderl, Mecke and Klug2007), and design refinement through simulated use (Otto & Wood Reference Otto and Wood2001) via individual or multiple testing (Otto & Wood Reference Otto and Wood2001) efforts.

Communication

Communication is the process of sharing information about the design, and its potential use within the design team and to users. Research in design supports prototypes as essential to explore and enhance design usability (Gordon & Bieman Reference Gordon and Bieman1995; Barbieri et al. Reference Barbieri, Angilica, Bruno and Muzzupappa2013). Prototypes enable observation of real-user interaction with each other and with the design (Kurvinen, Koskinen & Battarbee Reference Kurvinen, Koskinen and Battarbee2008). Prototypes are also valuable for communicating concepts within the design team (Reed Doke Reference Reed Doke1990; Buchenau & Suri Reference Buchenau and Suri2000). This objective encompasses usability testing to explore human factors in design.

Exploration

Exploration is the process of seeking out new design concepts. Two high-level processes in design with which prototyping can be associated, are divergence and convergence. Divergence implies gathering information and generating a range of new concepts, while convergence implies down-selecting to a smaller set of refined concepts (Lennings et al. Reference Lennings, Broek, Horváth, Sleijffers and Smit2000). Industry studies show that practicing designers frequently use physical prototypes to aid in the concept generation process (exploration) (Hess & Summers Reference Hess and Summers2013).

Active learning

Active learning is the process of gaining new knowledge about the design space or relevant phenomena. In this context, active learning applies not only in the educational sense, but in terms of advancing designer’s mental or analytical models of phenomenal interactions. Hands-on prototyping activities support gaining tacit knowledge regarding complex phenomena (Telenko et al. Reference Telenko, Wood, Otto, Elara, Foong, Pey, Tan, Camburn, Moreno and Frey2016). Physical models demonstrate flexibility across a range of disciplines and generally support design education (Green & Smrcek Reference Green and Smrcek2006). Prototypes are critical in the psychological experience of the designer, for re-evaluating failure as an opportunity to learn, and enhancing a sense of progress (Gerber & Carroll Reference Gerber and Carroll2012a ). Physical model construction may also help identify differences between a concept and real behavior (Lemons et al. Reference Lemons, Carberry, Swan, Jarvin and Rogers2010).

1.2 Guidelines for incorporating prototyping in the design process

Numerous studies have explored how and when to integrate prototyping into the design process. This section reviews key findings with regard to prototyping and the design process. This is to provide overall insights in planning. Later sections detail methods for prototyping. Table 1 highlights a few points that will be examined.

Testing

One key feature that separates a prototype from a design concept is testing (see Figure 3). However, every test requires some cost or effort (time) to produce (Dahan & Mendelson Reference Dahan and Mendelson2001). A key objective of design methods for prototyping is to acquire sufficient information to move forward in product development with minimal expenditure of time and cost. Therefore, every prototype test should answer a specific question (Otto & Wood Reference Otto and Wood2001).

Figure 3. A sketch of a tiled surface, or tectonic structure to produce a re-entry capsule shell. The sketch on the left is a concept of the shell tiling, it is not a prototype as it is not tied to a test. The prototype on the right allows for testing, as it includes simulated stress loading, in a finite element model (FEM).

Timing

Both empirical, and industry studies highlight that early prototyping is critical for innovation (Rothenberg Reference Rothenberg1990; Drezner Reference Drezner1992; Yang Reference Yang2005; Jang & Schunn Reference Jang and Schunn2012), specifically during the first 30% of a design project (by time) (Elsen et al. Reference Elsen, Häggman, Honda and Yang2012), it is especially critical to test the challenging systems (Otto & Wood Reference Otto and Wood2001). In fact, late prototyping is correlated with unsuccessful efforts (Jang & Schunn Reference Jang and Schunn2012). The precise selection of when to engage prototyping should be considered strategically (Otto & Wood Reference Otto and Wood2001), this decision may depend on complexity (Schrage Reference Schrage1993). It is important to note that committing a larger amount of time to prototyping does not correlate with greater success (Yang Reference Yang2005). Positioning efforts early within the overall timeline is critical.

Ideation

Prototypes are critical for inquisitive exploration of concepts, as they enable organic learning and discovery (Gill, Sanders & Shim Reference Gill, Sanders and Shim2011; Hess & Summers Reference Hess and Summers2013). Frequent early prototyping leads to new design ideas (Kershaw, Hölttä-Otto & Lee Reference Kershaw, Hölttä-Otto and Lee2011). Traditional development processes can be inverted to use prototyping for ideation, for instance to mold a clay object and scan it (see Figure 4). This process has been employed at Toyota (Schrage Reference Schrage1993). Ideas developed with prototypes tend to be more functional (Viswanathan & Linsey Reference Viswanathan and Linsey2012a ,Reference Viswanathan and Linsey b ).

Figure 4. Demonstrating the replication of a handmade object using a 3D scanner to capture the design as a point cloud, which is converted to a sterolithography (STL) file and then reproduced with a fused deposition modeling (FDM) printer.

Fixation

In the presence of an exemplar solution, prototyping to ideate over pure sketching leads to less fixated solutions (Youmans Reference Youmans2011; Schubert et al. Reference Schubert, Jacobitz, Morse and Ngo2012). It is important to note that a slow fabrication process may induce fixation, but a rapid one will not, as compared to sketching only (Viswanathan & Linsey Reference Viswanathan and Linsey2013). Prototypes may be more often used to communicate within domain analogies, and sketches for between domain analogies (Christensen & Schunn Reference Christensen and Schunn2007).

Feedback

Designers may hesitate to show prototypes to executives or managers, but be more comfortable showing their peers (Schrage Reference Schrage1993). In some prototyping cultures, stakeholders expect high-fidelity prototypes, and may reject a project when presented with a low-fidelity prototype (Schrage Reference Schrage1993). Feedback should be carefully timed and planned so as to not adversely affect perception and reward cycles (Davis Reference Davis1990; Schrage Reference Schrage1993). Also, it is important to note that receiving feedback on a design may increase fixation to that particular design concept (Kershaw et al. Reference Kershaw, Hölttä-Otto and Lee2011). This can be mitigated by presenting multiple designs (Dow et al. Reference Dow, Glassco, Kass, Schwarz and Klemmer2009).

Usability

It is essential to capture the voice of the customer (end user) during testing (Carulli, Bordegoni & Cugini Reference Carulli, Bordegoni and Cugini2013). One aspect of prototyping fidelity is the interactivity level (Nam, Park & Verlinden Reference Nam, Park and Verlinden2009). The design team can assess to what degree a prototype will capture potential user interactions. This in turn will delineate the scope of testing (Nam et al. Reference Nam, Park and Verlinden2009).

Fidelity

An experiment demonstrated that higher fidelity representations lead to more accurate interpretations by third parties reviewing the design (Hannah, Joshi & Summers Reference Hannah, Joshi and Summers2012). This is leads to a trade-off against the observation that more rapid prototyping leads to less fixation. Therefore, the appropriate level of fidelity must be selected depending on the objective of a particular effort.

1.3 A framework for design prototyping strategies

There are a number of reasons to initiate a prototyping effort, ‘Why to prototype’ (e.g. exploration). Simultaneously, there are many various strategic techniques to implement prototyping, ‘How to prototype’ (e.g. iteration). A prototyping strategy is the planned combination of various techniques to achieve a given set of objectives for a prototyping effort. Techniques guide the embodiment of a prototype. Table 2 highlights a mapping from common techniques, ‘How’, to the outcomes that they directly support, ‘Why’ (Camburn et al. Reference Camburn, Dunlap, Kuhr, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013a ,Reference Camburn, Dunlap, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood b , Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014; Dunlap et al. Reference Dunlap, Hammon, Camburn, Crawford, Jensen, Green, Otto and Wood2014). The corollary process objectives of reducing cost, and reducing time are included here. They are critical in defining a strategy (Thomke Reference Thomke1998; Dahan & Mendelson Reference Dahan and Mendelson2001; Thomke & Bell Reference Thomke and Bell2001). This chart functions as a reference index to prepare a prototyping effort.

Sections 2 and 3 explore the major techniques and specific techniques in more detail. A division is made between techniques. First, there are a set that lead directly to improving design performance and user experience. These are discussed in Section 2. The second category is the set of techniques used to reduce the effort and cost required to develop a prototype. The second set enables earlier, and more frequent prototyping. These are discussed in Section 3. This critical observation is fundamental to support prototyping activities. The mapping between techniques and objectives is taken directly from the critical summaries provided below.

Figure 5, further explains the relationship between the major techniques. It is a map of the prototyping strategy space. This map provides a structured view of the way prototyping can be implemented, ‘How’. The prototyping strategy for a particular product, service, or system can consist of the evaluation of multiple concepts. Each of those concepts may be tested or developed with multiple iterations. The embodiment of each iteration of each concept can be described with four characteristics: the scale, the system level, the requirements fidelity, and the media. These approaches are independent of problem type or design domain. It applies to development of electrical, mechanical, systems, architectural, software, or service designs. This model is inferred from extensive and critical review of the literature as well as the Authors’ industrial design experience. It advances preliminary efforts in previous work (Camburn et al. Reference Camburn, Jensen, Crawford, Otto and Wood2015). Definitions and critical evaluation of the techniques that support navigating this space are detailed below. It should be noted that Iteration and Parallel testing can be used in combination with any of the other techniques. This would enable the related outcome from Table 2. For example, Iterative-Virtual Prototypes would support refinement at lower cost.

Figure 5. A map of the prototyping strategy space. Where $I_{i}C_{n}$ is the $i$ th iteration of the $n$ th concept. Note that each block represents a single prototype or build. The expanded grey area depicts that each build has several factors of construction. The number of iterations of each concept may vary. For systems, this process can be applied at the system, subsystem, and component level.

2.1 Iterative prototyping

Iteration is the sequential testing and refinement of a prototype (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012) (see Figure 6). It is as a key technique. Iteration allows the gradual achievement of requirements (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). It is particularly useful to meet challenging requirements (Moe, Jensen & Wood Reference Moe, Jensen and Wood2004). Iteration is critical to obtaining insights into difficult issues, managing high uncertainty, identifying errors, and simplifying parts (Zemke Reference Zemke2012). A design team can choose how often to iterate based on the cost of each test relative to the value of potential performance increase (Thomke & Bell Reference Thomke and Bell2001).

Figure 6. An example of iterative design. (Left) Initial design; (centre) a series of three iterative refinements; (right) the final design for a large-scale 3D printer extrusion head. The reliability of the print process gradually increased with each iteration (test prints shown below each design). The final design required nearly 40 iterations to achieve reliable printing. Courtesy of Gilmour Space Technologies.

Empirical studies show that teams iterating on a design significantly outperform teams without iteration, based on the evaluation of self-efficacy and design requirement satisfaction (Dow, Heddleston & Klemmer Reference Dow, Heddleston and Klemmer2009; Camburn et al. Reference Camburn, Dunlap, Kuhr, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013a ,Reference Camburn, Dunlap, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood b , Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014; Dunlap et al. Reference Dunlap, Hammon, Camburn, Crawford, Jensen, Green, Otto and Wood2014). It has also been shown that knowledge of this effect can induce the use of iteration, compared to a control (Dunlap et al. Reference Dunlap, Hammon, Camburn, Crawford, Jensen, Green, Otto and Wood2014). Empirical evidence indicates a geometric progression of performance increase over a series of iterations (Camburn et al. Reference Camburn, Arlitt, Perez, Anderson, Choo, Lim, Gilmour and Wood2017); this is consistent with Moore’s law. This increase can continue over many iterations (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2015). The time to produce each iteration subsequently decreases. In particular the second iteration takes very little time compared to the first, all other factors being equal (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2015). It is important to note that selection of fabrication methods (and implicitly cost per test) has a significant impact on how many iterations are pursued on average (Thomke Reference Thomke1998). With iteration, it is important to consider the performance potential. For example, more iteration is encouraged when the potential for performance increase is higher, and the cost of each test relatively low.

Evolutionary prototyping

Evolutionary prototyping is a variant of iteration in which the design is iteratively refined without replacement (see Figure 7). The idea is that the design gradually transitions into a final workable product, service, or system (Reed Doke Reference Reed Doke1990). Evolutionary iteration is in contrast to throwaway iteration in which each embodiment is used only for one round of testing and then discarded. Throwaway iteration may be used when the cost is very low or radical changes are required.

Figure 7. The SUTD-MIT International Design Centre has gradually evolved over time into the concept shown above: (left) café space; (centre) open ideation area; (right) prototyping and materials store. The space has been in continuous use and undergoes frequent design reviews to apply revisions, along a path towards a converged final design. The space opened as a bare concrete studio, and is now a vibrant innovation space.

2.2 Parallel prototyping

In parallel prototyping, multiple design concepts are embodied and compared concurrently (see Figure 8). Parallel prototyping can help provide critical feedback for concept selection (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). A theoretical model suggests that the number of parallel concepts can be determined in a similar manner to that for iteration. The number of parallel designs can be proportional to the ratio of potential profit to the cost of each design (Dahan & Mendelson Reference Dahan and Mendelson2001). Alternately, multiple designs in parallel are encouraged when cost (budget) is flexible and exploration has more potential value (Moe et al. Reference Moe, Jensen and Wood2004). The performance increase associated with parallel prototyping is less predictable, but the increase may be large (Camburn et al. Reference Camburn, Arlitt, Perez, Anderson, Choo, Lim, Gilmour and Wood2017).

Figure 8. An example of parallel prototyping. (Left) Eight alternative textile prototypes for the casing of a collapsible carrier; (right) prototype of the selected materials in the final concept (right).

Controlled experiments show that groups producing prototypes in parallel produce designs that significantly outperform groups producing a single design (Dow et al. Reference Dow, Glassco, Kass, Schwarz and Klemmer2009, Reference Dow, Glassco, Kass, Schwarz, Schwartz and Klemmer2010, Reference Dow, Fortuna, Schwartz, Altringer, Schwartz and Klemmer2012; Neeley et al. Reference Neeley, Lim, Zhu and Yang2013; Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014). This result holds true whether the teams are also iterating (Dow et al. Reference Dow, Glassco, Kass, Schwarz and Klemmer2009, Reference Dow, Glassco, Kass, Schwarz, Schwartz and Klemmer2010), or only executing a single test (Neeley et al. Reference Neeley, Lim, Zhu and Yang2013). Pursuit of parallel prototypes may lead to higher concept diversity (Dow et al. Reference Dow, Glassco, Kass, Schwarz, Schwartz and Klemmer2010). The use of parallel prototyping can be induced in design teams (Camburn et al. Reference Camburn, Dunlap, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013b ). However, parallel prototyping may increase the design team members’ sense of time constraint (Neeley et al. Reference Neeley, Lim, Zhu and Yang2013), this seems self-evident as distinct concepts require effort to produce. Presenting multiple prototypes for feedback allows comparative evaluation of features (Dow et al. Reference Dow, Fortuna, Schwartz, Altringer, Schwartz and Klemmer2012).

Competitive prototyping

Competitive prototyping is a form of prototyping where separate teams independently develop concepts in parallel to solve a common problem. Employing separate teams allows for high design divergence, even within tight requirements (see Figure 9) (Borowski et al. Reference Borowski2012). The concept of competitive prototyping was popularized by the US department of defense to decrease expenditure (Borowski et al. Reference Borowski2012). Competitive prototyping also increases awareness of latent design requirements and aids in quantifying feasibility (Davis Reference Davis1990). The embodiment of competitive designs is critical for large-scale projects due to high risk factors (Borowski et al. Reference Borowski2012). Although it may increase cost to explore competing prototypes, the overall design costs likely will be reduced (Crouch Reference Crouch1972). To maximize the outcome of this approach it is critical to integrate findings from each team (Thomke Reference Thomke2003).

Figure 9. An example of competitively produced prototypes, from the 30.007 course at SUTD. (from left to right) A flying-rolling robot; rolling robot with enhanced traction wheel; rolling robot with active camouflage; rolling robot using exo-skeletal modular pendulums; integrated design for full project (semi-final design). These designs are part of a student design hackathon, which in turn is part of a larger university wide competitive project to develop advanced ISR robotics.

3.1 Requirements (relaxed or full)

Relaxed requirement prototyping is applied when a prototype is developed to address a subset or reduced version of the design requirements (see Figure 10). This approach leverages the trade-off between model accuracy and cost (Otto & Wood Reference Otto and Wood2001). A less accurate model may at times be sufficient because of the benefit of reduced costs. They are essential for early stage risk reduction and refining design requirements (Rothenberg Reference Rothenberg1990; Drezner Reference Drezner1992; Drezner & Huang Reference Drezner and Huang2009). Early stage low-fidelity prototyping can facilitate ‘moving the ball forward’ (Kelley Reference Kelley2001). Relaxation of requirements must be implemented carefully to ensure that useful information is obtained. To enhance accuracy, the prototype should be tested in a realistic environment (Drezner & Huang Reference Drezner and Huang2009). It has been suggested that threshold minimum requirements can be addressed in early development stages (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). Relaxed requirement prototyping can act as a facilitator to enable parallel prototyping and iteration (Thomke & Bell Reference Thomke and Bell2001) at a more rapid pace (Drezner & Huang Reference Drezner and Huang2009). Some techniques in this category (storytelling, vignettes, cartoons, and amateur videos) can be more holistic and touch on several points at once (Blomkvist & Holmlid Reference Blomkvist and Holmlid2010).

Figure 10. Example relaxed requirement prototype, (left) very low-fidelity and relaxed requirement prototype; (right) medium fidelity prototype. The design is for an evaluation tool that is used by multiple organizations. The prototype on the left allows for early, rapid prototyping of readability with the design team. While the higher fidelity, functional offline interface on the right allowed for testing with end users. Both are reduced fidelity prototypes, at different degrees.

Experimental studies have explored prototype requirement specification. Prototype fidelity predetermines potential for certain insights from a prototype (Hannah et al. Reference Hannah, Joshi and Summers2012). Requirement relaxation can be induced in design teams (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2015). Relaxed requirement prototypes cost less and require less time to build (Camburn et al. Reference Camburn, Dunlap, Kuhr, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013a ,Reference Camburn, Dunlap, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood b ; Dunlap et al. Reference Dunlap, Hammon, Camburn, Crawford, Jensen, Green, Otto and Wood2014). The strategic relaxation of requirements on early prototypes does not seem to have any adverse effects on final performance (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014). Low-fidelity prototyping fosters a sense of forward progress, learning from failure, and concept expression without fixation on detail (Gerber & Carroll Reference Gerber and Carroll2012b ).

Figure 11. (Left) Paper mockups of the structure of an atomic force microscope casing; (center) intermediate functional prototype; (right) sample scanned image from a later prototype. In this particular design context, mockup prototyping permitted high-level assessment of many potential frame designs in parallel (with limited resources). More than thirty mockups, similar to the design in the left frame, were fabricated with paper and clay in less than 45 minutes.

Product mockups

A mockup is an extreme case of relaxed requirement prototyping that demonstrates a feature(s) of a system or product concept in an abstract, high-level way (see Figure 11). Although mockups may only approximate a system’s or product’s actual physical behavior, they are very useful and may be more practical in some stages of development (Otto & Wood Reference Otto and Wood2001). Paper mockups could represent a system with multiple distinct constructions, e.g. software and hardware are each represented with distinct mockups (Yamazaki Reference Yamazaki2009). Paper mockups are effective for obtaining usability feedback (Sefelin, Tscheligi & Giller Reference Sefelin, Tscheligi and Giller2003). Mockups may be particularly useful when capabilities do not permit fabrication of a functional form, or if modifications are desired from non-experts (Sefelin et al. Reference Sefelin, Tscheligi and Giller2003). Mockups enable storytelling and help make ideas concrete.

System mockups

A system mockup is a specific form of relaxed requirement prototyping. The concept is to emulate the characteristics of complex systems (see Figure 12). ‘Systems’ are composed as a superset of simpler, individual mockups. This is a single-build technique and is not to be confused with a system prototyping effort that would consist of many strategic subsystem prototypes (see subsystem isolation). The method allows for identification of basic needs, or areas of concern. Systems modeling is often difficult and complex (Drezner Reference Drezner1992). Usage of a system mockup permits rapid assessment of basic system characteristics and may facilitate team discussion (Yamazaki Reference Yamazaki2009).

Figure 12. (Left) System mockup, low fidelity of traffic simulation at a large-scale integrated facility, useful to identify possible traffic congestion modes; (right) higher fidelity traffic simulation, useful to quantify traffic congestion in each mode (Volkhin 2016).

Service mockups

Low-fidelity prototyping can also be applied to enhance services (Blomkvist & Holmlid Reference Blomkvist and Holmlid2010), as well as products and systems (see Figure 13). In service prototyping, various agents take the role of stakeholders in a service chain. Designers act out usage scenarios (Buchenau & Suri Reference Buchenau and Suri2000). Scenario methods can be enhanced with preconceived action prompts to guide activities, and with requirements for quantitative observation such as note taking at timed intervals (Buchenau & Suri Reference Buchenau and Suri2000), or video recording. Mockups of products may also support testing when functional equipment is not available. It is possible to collect detailed user feedback and take measurements of performance using this approach; especially useful in the development of product-service systems (Green et al. Reference Green, Dutson, Wood, Stone and McAdams2002).

Figure 13. Service mockup, of medical facility. The simulation enabled gathering quantitative data on the performance of two different service flow designs. (Left top) wire frame of current layout; (left bottom) wireframe of proposed layout; (center) snapshot of workflow simulation; (right) analogous workplace final design (actual prototype withheld under NDA).

Cooperative prototyping

In cooperative prototyping, a group of users and designers are actively engaged with each other during prototype development (Grønbæk Reference Grønbæk1990) (example shown in Figure 14). This tangibility makes the user’s feedback concrete (Bødker & Grønbæk Reference Bødker and Grønbæk1991). In cooperative prototyping, the physical model is a catalyst for discussion about actual usage. In the ideal case, the users construct the prototype rather than the designer (Trigg, Bødker & Grønbæk Reference Trigg, Bødker and Grønbæk1991). An experimental study shows that providing multiple designs for group feedback in a cooperative setting produced a better outcome, compared to providing only a single design (Dow et al. Reference Dow, Fortuna, Schwartz, Altringer, Schwartz and Klemmer2012). Cooperative prototyping can be employed in concert with other mockup approaches. It is possible to engage end users as ongoing members of the design team during prototyping.

Figure 14. An example of cooperative prototyping. In this case, a designer is discussing requirements with potential users who have prototyped and presented a simple mockup of a product. For final design, see Figure 11.

Storyboarding

Storyboarding is one of the lowest fidelity, and therefore lowest cost, approaches which can still be considered a prototype (Lim et al. Reference Lim, Stolterman and Tenenberg2008). Time dependent behavior of a prototype and a user are graphically depicted (see Figure 15). Typically, the interactions are cognitively derived by the design team, though simulations or mockups may be constructed and then documented. Storyboarding can permit focus on the core concept rather than details in testing (Wong Reference Wong1992). It is limited to the designer’s domain knowledge in terms of the accuracy of the representation. Storyboarding may be supported or enhanced with other approaches such as calculations, simulations, or even physical testing of key steps.

Figure 15. Storyboard mockup of an energy harvesting product from the 3.007 course at SUTD. It depicts the relationship between the user’s movement, time, and energy captured.

Zygotic narratives

Recent studies have identified that scenarios can be explored through narratives to achieve novel predictive modeling results. An applied narrative for the purpose of exploring a functional design is known as Zygotic, or science fiction prototypes (Grimshaw & Burgess Reference Grimshaw and Burgess2014). The designer constructs a narrative to introduce the design into various potential future scenarios (Grimshaw & Burgess Reference Grimshaw and Burgess2014). This method enables an immersive experience of the design that might be otherwise impossible (Roberts & Middleton Reference Roberts and Middleton2014). It is a cognition-based simulation approach (Grimshaw & Burgess Reference Grimshaw and Burgess2014), of the lowest fidelity. The advantages of this technique are low cost and a highlight of potential future needs.

3.2 System (isolated or integrated)

In isolated subsystem testing, a concept or prototype is segmented, and a single subsystem (or set of subsystems) is explored in isolation (example shown in Figure 16). Constructing subsystems in isolation can permit development of a tailored prototyping strategy for that subsystem (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). A study of military design efforts found that isolated subsystem testing (for large and complex systems) can provide useful data for about 60% of the cost of an integrated prototyping effort (Drezner & Huang Reference Drezner and Huang2009). Other factors of the prototyping strategy, e.g. requirements, can be unique to each subsystem (Yan, Williams & Li Reference Yan, Williams and Li2002). Each of the subsystem prototypes may be a mixture of physical, virtual (Yan et al. Reference Yan, Williams and Li2002), or scaled modeling (Faithfull, Ball & Jones Reference Faithfull, Ball and Jones2001).

Figure 16. (Left) Isolated subsystem prototype of an electric vehicle drive train; (center) integrated functional design of the same vehicle; (right) final model of the market product, rendering. Courtesy of Gilmour Space Technologies.

Systems can be modeled using multiple subsystem prototypes where forces, signals, material, and energy transfer across interfaces are emulated. Such models can be highly accurate (Yan et al. Reference Yan, Williams and Li2002). Subsystem isolation enables rapid in-depth exploration of the design in detail (Horváth & Bois Reference Horváth and Bois2012). A strategic method can encourage use of subsystem isolation (Camburn et al. Reference Camburn, Dunlap, Kuhr, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013a ). Isolating a subsystem significantly reduces the cost of prototyping without adversely affecting performance (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014). However, it may be more difficult for isolated subsystems to address unanticipated needs (Rogers et al. Reference Rogers, Church, Patel and Menon2013).

3.3 Scale (reduced or final)

A scaled prototype mimics behavior(s) of a larger (or smaller) design through similitude (example shown in Figure 17). This scaling may be geometric (i.e. size) or relating to complexity (e.g. number of users). The discovery of similitude is attributed to William Froude, in the 1870s (Kempf Reference Kempf1940). Froude discovered that by identifying critical factors of a phenomenon (e.g. the way gravity drives wave formation), one can develop the models to extrapolate real conditions from scaled models (Kempf Reference Kempf1940). Scaling may enable prototyping in cases where a full-scale model is not feasible (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). Therefore, scaling can also be used to save resources (Otto & Wood Reference Otto and Wood2001). Virtual modeling enables scaling through mathematical algorithms and instant display reconfiguration. This enables large designs with high detail complexity (Mitchell Reference Mitchell2004).

Figure 17. (Left) Scaled prototype, 5cm height, of the sculptural representation of a Penrose rectangle design; (right) 75cm final installation of the same design, on display at the ArtScience Museum, Singapore.

Scaling may be applied to other phenomena. Empirical similitude is an advanced technique based on forming the vector product of scaling laws across multiple factors (e.g. scaling material strength, and geometry separately) (Cho et al. Reference Cho, Dutson, Wood and Crawford2005; Dutson & Wood Reference Dutson and Wood2005). Use of similitude scaling can be induced with a strategic method, and scaled models cost significantly less than full prototypes without adverse performance effects (Camburn et al. Reference Camburn, Dunlap, Kuhr, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood2013a ,Reference Camburn, Dunlap, Viswanathan, Linsey, Jensen, Crawford, Otto and Wood b , Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014).

3.4 Media (virtual or physical)

A virtual prototype is one which is developed (and tested) on a computational platform (example shown in Figure 18). A review of the prototyping literature identifies that virtual models can permit complex analysis and facilitate production (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). Virtual prototypes are limited to modeling phenomena that are directly programmed in the model, and may therefore be partial models of a system (Otto & Wood Reference Otto and Wood2001). Nonetheless, simulation offers affordances in the generation of complex designs (Mitchell Reference Mitchell2004). Virtual models allow for coupling design, testing, and manufacture (Sghaier & Soriano Reference Sghaier and Soriano2008). It also enables rapid international communication of design details (Cugini, Bordegoni & Mana Reference Cugini, Bordegoni and Mana2008). Virtual prototypes allow for testing at a more rapid pace and reduced cost compared to physical prototypes (Christie et al. Reference Christie, Jensen, Buckley, Ziegler and Crawford2012). This enables rapid iteration, and parametric perturbation studies.

Figure 18. (Left) Stress loading in a finite element model (FEM) of a reusable launch vehicle (RLV) simulator; (right) Final, complete functional prototype of an RLV simulator. Courtesy of Gilmour Space Technologies.

Virtual prototypes have drastically lower costs than physical prototypes with roughly equal performance (Camburn et al. Reference Camburn, Dunlap, Gurjar, Hamon, Green, Jensen, Crawford, Otto and Wood2014). A computer prototype was reported to require less effort, and to perform equally well with a paper prototype for usability testing (Dahan & Srinivasan Reference Dahan and Srinivasan2000; Sefelin et al. Reference Sefelin, Tscheligi and Giller2003). Virtual prototypes were produced more rapidly and lead to higher performance over physical prototyping for a simple design task (Hamon et al. Reference Hamon, Green, Dunlap, Camburn, Crawford and Jensen2014). Practicing designers report predominant use of virtual models (CAD) for documentation, while sketches are often used in ideation (Römer et al. Reference Römer, Pache, Weißhahn, Lindemann and Hacker2001). Virtual models require less ‘unnecessary work’ (Sefelin et al. Reference Sefelin, Tscheligi and Giller2003), such as lead time to acquire parts. Several variants to virtual prototyping are discussed below. Some of these techniques may consist of complete research fields in other contexts; herein key observations are taken from the subset of literature in those fields intersecting with research on design prototyping. The categorizations are developed through critical review of the literature.

Distributed design

Virtual prototyping offers the opportunity for large, geographically diverse teams to collaborate on development. Online platforms are emerging that support the sharing of designs (e.g. Thingiverse.com), other tools enable concurrent design of the model itself (Onshape.com) (see Figure 19) (Juracz et al. Reference Juracz, Lattmann, Levendovszky, Hemingway, Gaggioli, Netterville, Pap, Smyth and Howard2013). This ability to collaboratively prototype online has become increasingly accessible to the public. GitHub, a popular site for individuals to collaborate on software development, is a prolific example of this (GitHub 2016). Empirical studies have demonstrated the effectiveness of remote design and prototyping, using 3D printing for fabrication (Ertekin & Chiou Reference Ertekin and Chiou2012). While this technique has been relatively unexplored in the literature, there are many potential applications to support customized prototypes and prototyping for complex problems.

Figure 19. Leveraging distributed design. (Left) Engineers from Singapore and Australia co-created the CAD for this zero-gravity wall climb simulator using a web-based platform that allows for multiple users to access one file simultaneously using the Onshape software; (right) final functional design of the wall climb. Courtesy of Gilmour Space Technologies.

Parametric optimization

There are powerful tools to apply design optimization with virtual prototypes. Open-source tools, such as Modelica (Association Reference Association2012), provide text-based, object-oriented approaches to constructing dynamic simulations of physical systems. Semantic operations, such as those provided by Grasshopper or other enterprise tools can permit the use of high-level logic to design and arrange complex components, e.g. placing of machine equipment in a factory (Kádár, Terkaj & Sacco Reference Kádár, Terkaj and Sacco2013). Online resources of common components allow the fast construction of simulations without a corresponding physical model (Biscani, Izzo & Yam Reference Biscani, Izzo and Yam2010; Wetter et al. Reference Wetter, Zuo, Nouidui and Pang2014). While parametric optimization is relatively unexplored as a prototyping technique it is a powerful tool for design and to enhance prototyping outcomes (see Figure 20).

Figure 20. Launch of hybrid rocket using 3D printed, optimized fuel grains. Courtesy of Gilmour Space Technologies.

Automated design and design synthesis

Virtual modeling can be used as a tool to automate some aspects of design. In contrast to parametric optimization (parametric feature refinement), automated design includes the generation of new design concepts. Certain classes of problems that lend themselves to functional synthesis can be decomposed to grammatical formalisms, allowing for algorithmic generation and evaluation of virtual models of design solutions (Shea & Cagan Reference Shea and Cagan1999; Chakrabarti et al. Reference Chakrabarti, Shea, Stone, Cagan, Campbell, Hernandez and Wood2011). Computational algorithms can also support multi-component heuristics (Oxman Reference Oxman2012). For example, a genetic architectural design algorithm might test both aesthetic and structural features of a generated model (Oxman Reference Oxman2012). Automated design prototype testing can even include simulations of users interacting with the design (Silva, Campos & Harrison Reference Silva, Campos and Harrison2014). It is possible to generate solutions using multi-disciplinary evaluations, such as architecture and fluid mechanics (see Figure 21).

Figure 21. (Left) Solution space of synthesized designs using a hybrid computational fluid dynamics (CFD) and architectural design approach (Kaijima et al. Reference Kaijima, Bouffanais, Willcox and Naidu2013); (centre) production of the final prototype; (right) testing of the final design. The structure can passively reduce temperature at a bus stop by accelerating ambient winds, Courtesy of the SUTD-MIT International Design Centre.

Mixed prototyping

Prototypes can be developed by integrating various physical and virtual elements in one model (see Figure 22). There are several reported forms of this mixed approach which are discussed further below. The prototype is a mixture of virtual and physical elements, and the user can be real or virtual (Bordegoni et al. Reference Bordegoni, Cugini, Caruso and Polistina2009). They permit critical usability assessment testing (Grønbæk Reference Grønbæk1990). Simulations can be an important tool to evaluate system dynamics in conjunction with a physical model (Moten et al. Reference Moten, Pipeleers, Swevers and Desmet2014). Mixed prototypes enable prototyping in complex systems where it may be difficult to model the entire system with a single approach. They also enable the integration of various levels of fidelity. Mixed prototypes typically emerge at later stages of prototyping once subsystem prototypes are integrated. Mixed prototypes enable the evaluation of system behaviors. The design team may critically discuss which elements should be physical or virtual (Filippi & Barattin Reference Filippi and Barattin2014).

Figure 22. Application of mixed prototyping for a multi-phase 3D printer. (Left) Software based control software simulation – flow diagram; (center-left) scaled, empirical similitude, physical test of the slurry injection valve; (center-right) full system CAD model; (right) final integrated design prototype. Courtesy of Gilmour Space Technologies.

Augmented or virtual reality

Augmented reality facilitates capturing the voice of the customer and user experience in a uniquely quantitative fashion (Carulli et al. Reference Carulli, Bordegoni and Cugini2013) (see Figure 23). An augmented reality prototype might for example integrate eye tracking for detailed use assessment (Bordegoni et al. Reference Bordegoni, Cugini, Caruso and Polistina2009). Cost and time are also saved (Ferrise, Bordegoni & Cugini Reference Ferrise, Bordegoni and Cugini2012). Augmented reality facilitates the potential to engage more senses in the user-interaction environment at an earlier stage (sight, sound, touch, etc.) (Barbieri et al. Reference Barbieri, Angilica, Bruno and Muzzupappa2013). A unique benefit is the ability to independently manipulate each sensory feedback (Ferrise et al. Reference Ferrise, Bordegoni and Cugini2012). Integration of 3D scanning can expand the capabilities of virtual prototyping by providing a starting point from a database of existing objects which can be redesigned in CAD (Junk & Matt Reference Junk and Matt2015). A controlled study found an augmented reality prototype to provide equivalent usability data, from participants, as a physical prototype (Bruno et al. Reference Bruno, Cosco, Angilica and Muzzupappa2010). Testing may integrate haptic feedback (Abidi et al. Reference Abidi, Ahmad, Darmoul and Al-Ahmari2015). Technologies that enable this technique are: augmented reality, embedded screens, motion tracking, QR code tracking, and projection mapping (Buchenau & Suri Reference Buchenau and Suri2000; Evans et al. Reference Evans, Wallace, Cheshire and Sener2005; Kanai & Verlinden Reference Kanai and Verlinden2009; Verlinden, Horváth & Nam Reference Verlinden, Horváth and Nam2009; Barbieri et al. Reference Barbieri, Angilica, Bruno and Muzzupappa2013). These technologies may also enable live modification of a virtual (Ng, Ong & Nee Reference Ng, Ong and Nee2015) or physical model (Anderl et al. Reference Anderl, Mecke and Klug2007).

Figure 23. Using augmented reality to simulate various design layouts for a living room. Courtesy SUTD-MIT International Design Centre.

Hardware in the loop simulations

Hardware in the loop simulations allow for simulation of field testing with a mixed prototype (see Figure 24). Information is transmitted between each subsystem in order to simulate field use (Yan et al. Reference Yan, Williams and Li2002). Physical components are typically used for subsystems with high predictive error, such models may be scaled (Faithfull et al. Reference Faithfull, Ball and Jones2001). Sensing and controls are typically simulated and re-injected into the physical system to achieve the desired emulation. User interaction is encouraged where applicable (Haug et al. Reference Haug, Choi, Kuhl and Wargo1995). Image and motion capture systems can be integrated to obtain highly detailed information of usage (Murakami & Fujii Reference Murakami and Fujii2009). Integrating isolated physical subsystems with virtual prototypes of the full system allows engineers a fast method of evaluating system performance.

Figure 24. (Left) Hardware in the loop schematic for testing to the performance of a large UAV; (right) the plane is fixed to a rigid stand and through control injections, exercises the same activities as would occur during flight. Courtesy of Gilmour Space Technologies.

Haptic prototypes

Haptic prototypes provide tactile feedback to the user based on data extracted from dynamic analysis of a virtual product or system model (Moussette & Banks Reference Moussette and Banks2011) (see Figure 25). An empirical study reported that adding haptic feedback to a mixed prototype was preferable to users over a static prototype (Evans et al. Reference Evans, Wallace, Cheshire and Sener2005). Progress has been made on enabling traditional fabrication processes such as sculpting through a combination of virtual reality, motion capture, and haptic feedback (Leu et al. Reference Leu, Maiteh, Blackmore and Fu2001). One advantage of prototyping with haptic interfaces is in designing for visually impaired users (Panëels et al. Reference Panëels, Ritsos, Rodgers and Roberts2013). Forces from a virtual model can be supplied as the designer interacts with it via a haptic interface, e.g. to simulate textures (Evans et al. Reference Evans, Wallace, Cheshire and Sener2005).

Figure 25. Haptic interface, in this case, a multi-functional display is simulated for a reusable launch vehicle with various interactive display mechanisms and controller feedback points. Courtesy Gilmour Space Technologies.

5.1 Summary of key observations

Design science is the study of designs and the behavior of humans as they conduct design. Often the objective of such research is to understand how and why we design so as to be able to improve that process (Papalambros Reference Papalambros2015). Prototyping is ubiquitous in the development of innovative products, services, and systems. There are several reasons why a design team might develop a prototype, and an extensive space of techniques for their development. Prototyping is perhaps most well known for design refinement. It may be complex to navigate the space of possible objectives, and techniques when developing a series of prototypes throughout a development effort. This review provides a novel framework for navigating the prototyping space as well as a critical review of each technique that is supported by the literature. Numerous industrial examples are provided from the authors’ design efforts to support the depiction of each technique.

In general, research provides compelling evidence that it is critical to prototype multiple iterations, and in the early phases of design. Research has supported the use of prototypes for ideation as a means to increase the percentage of functional ideas, versus pure sketching. The time required to produce a prototype is inversely related to fixation. Therefore, a rapid process can reduce fixation when employing prototypes to develop new concepts (Figure 28).

Figure 28. (Left) Usage of low-fidelity prototypes for design development of next generation refuge shelters via a series of hackathons in which many low fidelity concepts were constructed; (right) design concept informed by these low-fidelity models – a top scoring submission in IKEA’s ‘What Can Design Do’ challenge, 2016.

The ultimate goal of prototyping is to enhance performance and user experience of the final design. Iterative prototyping is strongly correlated with performance increase with regards to meeting difficult requirements. Parallel prototyping is also correlated with increased performance in terms of discovering new ideas. However, both of these techniques may create additional strain on a design team.

To alleviate the strain evoked through the above methods to enhance performance, various techniques can enable earlier and lower cost prototyping in resource constrained projects. Prototypes with reduced requirements or mockups are a powerful tool to rapidly explore concepts and enable communication; however, they may misrepresent physical principles and must be evaluated with caution. Isolated subsystem testing can also reduce costs and permit tailored requirements. Scaled prototypes can enable testing in cases when a full-scale system might be infeasible; however, well-planned testing is required to achieve an accurate model. Finally, virtual prototypes (simulations) can supply detailed test data. They may also drastically reduce prototyping costs. However, significant technical expertise may be required to develop an accurate virtual model.

The careful implementation of a mixture of these techniques can be strategically employed to rapidly develop innovative designs.

5.2 Opportunities in design prototyping research

There are several topics to be explored in more rigorous scientific detail relating to design prototyping. Key hypotheses can be formed and tested that relate to these topics. Regarding the development of prototypes: the value of information, contextual factors, and systems prototyping are relatively less explored. A clearer understanding of quantified information gained from a prototype is needed. The relationship between representation and decision making is critical. For example, what is the effect of fidelity on consumer emotional preference, or engineering accuracy? Various contextual effects are relatively less explored including, time scale, problem type, and design team experience level. Finally, the prototyping of complex systems, social interactions, organizations, and the evaluation of interfaces in prototyping has been relatively underexplored.

As far as the research is concerned from a holistic perspective, the focus is on customers and functionality in use. There is little exploration of other concerns which emerge in later stages of prototyping as a product, service, or system nears deployment. There is limited discussion about using prototypes to design the development (e.g. manufacturing for products, or training for services) process and integration with other systems.  There is very little (no?) discussion about designing the aftermarket support:  designing the field service kits, the field service procedures, the repair manuals with prototype versions, or sustainable disposal.  Designing the user training, the user instruction manuals with prototype versions.  Marketing and other product sales aspects, e.g. brochures or talking points, are rarely incorporated in prototypes. In short, most designs are part of a product-service system, and there is relatively little research on product-service system prototyping and the transition to market. These aspects are necessary for the development of successful designs.

The advertising industry, by contrast has demonstrated enormous success with more tightly integrated prototyping and evaluation through to release. This may be due to the media of delivery; however, with the advent of digital manufacturing and open-source technology, later stage revisions may also be more achievable in product, service, and system development.

While studies have evaluated the effects of sketching ability, few studies have explored a designer’s ability to construct models, fabrication proficiency, and how that may affect prototyping outcome. This could be a key element in enabling designers to prototype more effectively.

There are opportunities for seminal work in integrating design science with the open-source movement, and to explore the full capabilities of additive manufacturing as a prototyping tool. In parallel with this, it appears that there is an enormous untapped potential to explore distributed task decomposition. Directed crowd source studies have explored how designers may work with distributed groups for assessment and survey. However, there are few studies in directed crowd-based development efforts in which the crowd develops new designs or prototypes.

5.3 Emerging trends and visions of the future

Significant studies have been performed in the emerging field of the scientific research of prototyping. However, much research and development lies on the horizon. In many ways, only the tip of the proverbial iceberg has been explored. The digital age, additive manufacturing, and the emergence of data-driven design will see new approaches and a transformation in what we know today as design prototyping. These emergent trends have been relatively less explored in the research.

A trend is emerging towards instant prototyping with an emphasis on simulation. For example, vetted models can now be used instead of hardware for regulatory certification. Augmented reality and virtual reality enable immersive and convincing simulations. Emerging paradigms such as the open-source movement, and machine learning reduce simulation development cost. New technology that supports an internet of things (IOT) enables frictionless documentation and more data intensive prototyping. This enables testing with more video, deep sensing, and continuous wireless monitoring of real-time quantified field use. This opens opportunities for more detailed usage analysis.

Human centric prototypes are becoming essential to develop successful products, services, and systems. At the same time, prototypes can be more functional. Additive manufacturing (3D printing), open-source software repositories, and reconfigurable electronic hardware enable a drastic cost reduction in single unit manufacturing. These technologies may even enable new levels of complexity, through leveraging advanced capabilities (e.g. direct texture printing through multi-material 3D printing).

Prototype testing can also be distributed as the fabrication platforms become largely autonomous and standardized. A prototyping facility with access to a simple computer on board module (e.g. Raspberry Pi) and an FDM printing capability could readily produce the core elements needed for structure and control of countless products. Hosts of example schematics are available online as guides. Enhanced processing, out-of-the-box AI solutions, visual interface tools, plug-and-play modules, and virtual supply chains enable production of capable, emotive, realistic, aesthetic, and intuitive prototypes. The barrier of entry to prototyping and product develpment is lowering dramatically.