1. Introduction
The Criticality-Based Planning of Prototype Sequences method provides a structured approach to planning and prioritising prototyping activities within product development. Its central premise is that the sequencing of prototypes can be optimised by assessing and prioritising a system’s functional areas according to their criticality. By systematically identifying and addressing the most critical sub-functions first, the method facilitates an efficient allocation of development resources, promotes early risk mitigation, and accelerates the maturation of product solutions. It conceptualises prototyping as a continuous, iterative process that spans the entire development cycle rather than a discrete phase.
The method was first illustrated using the example of the redesign of additively manufactured machine elements (Reference Zorn, Glaser and GerickeZorn et al., 2024) and was applied in a feasibility study on the development of orthoses for head deformities in infants. (Reference Zorn, Krügener and GerickeZorn et al., 2025).The study showed that the method adds value for designers by focusing prototyping activities on the most important areas, thereby offering the potential to quickly achieve a high level of product maturity and minimise prototyping effort.
An important factor for transferring the method into practice is the precise description and convincing evaluation of all elements involved. Here, decomposition serves as the basis for method evaluation, since all different elements need to be evaluated separately, as well as the method as an integrated whole. (Reference Gericke, Eckert and StaceyGericke et al., 2022). There are various approaches in the literature for the detailed description and abstraction of the essential elements of methods. The approaches of Reference LindemannLindemann (2009) for example, aim to facilitate the organisation of methods for various purposes, such as method selection or method comparison (Reference BirkhoferBirkhofer, 2008). Other authors, such as Reference Zier, Bohn and BirkhoferZier et al. (2012), break down methods into elementary methods, i.e. basic information units and basic operations, to facilitate their communication. Based on these approaches, Reference Gericke, Eckert and StaceyGericke et al. (2022) propose a systematic method for the decomposition and description of methods, which is used to describe the method of criticality-based planning of prototyping sequences in this paper.
While expert use has shown that the criticality assessment works in principle, it remains unclear whether the verbal descriptions of the three evaluation parameters—novelty, technical difficulty and importance—are sufficient for reliable use across different project types and evaluators.
This leads to the following research questions:
RQ1: Are the scale definitions for evaluating novelty, technical difficulty and importance applicable in different development contexts?
RQ2: Do method users understand the scale definitions as intended?
2. Criticality-based planning of prototyping sequences
2.1. Core idea
The Criticality-Based Planning of Prototype Sequences method provides a systematic and rationalised approach to structuring and prioritising prototyping activities within product development. Its core idea is to support the planning of prototype sequences at the system level by prioritising functional areas based on their criticality—that is, their relative novelty, technical difficulty, and importance to the overall product (see section 2.5).
The overall process from generating a function model to the prioritisation regarding criticality and the initial planning of the prototyping sequence can be seen in the following Figure 1.
Simplified visualisation of the general process for the criticality-based planning of prototyping sequence- using the example of a Cranial remolding orthosis for infants
Figure 1 Long description
Panel A: Generate a function model. This panel shows a flowchart with various interconnected components representing different functions and their interactions. Key components include testing frame, locally applied pressure, socially applied pressure, and orthotic function. Panel B: Prioritisation regarding criticality. This panel features a matrix that rates the criticality of different functions based on their importance in the product. The matrix includes categories such as applicability locally, necessity, and lack of orthosis. Panel C: Derive the prototyping sequence. This panel displays a decision flowchart with different stages labeled as F1, F2, F3, and F4, indicating the sequence of prototyping activities. Panel D: Testing the sequence. This panel shows images of different prototypes being tested, labeled as P1.1, P1.2, P1.3, and P1.4.
Through this prioritisation, the method enables designers to address high-risk and uncertain sub-functions early, thereby minimising development risks and optimising the use of time and resources throughout the iterative prototyping process.
The method integrates two classical paradigms of design methodology: the problem-oriented and the process-oriented approaches (Reference Bender and GerickeBender & Gericke, 2020; Reference FrickeFricke, 1993). In contrast to purely sequential or isolated prototyping strategies, it combines their respective advantages into a hybrid procedure. Early prototyping activities follow a problem-oriented logic, focusing on verifying the most critical sub-functions individually. Once these sub-functions reach a satisfactory maturity level, the approach transitions toward a process-oriented perspective in which less critical or well-understood sub-functions are merged and validated collectively within more comprehensive prototypes. This combined strategy supports both focused learning and system-level integration, thereby providing flexibility and robustness in managing design uncertainty.
2.2. Procedure
The method proceeds through the following sequence of interlinked steps (Reference Zorn, Glaser and GerickeZorn et al., 2024) wich are also shown schematically and in a simplified form in Figure 1.
-
1. System analysis and function modelling
The product is decomposed into its primary and secondary functions using a function structure. System boundaries and interactions are defined to understand dependencies between sub-functions.
-
2. Criticality assessment
Each sub-function is evaluated regarding novelty, technical difficulty, and importance. Based on the individual scores, an overall criticality rating is derived (low/medium/high).
-
3. Prioritisation of sub-functions
Functions are ranked based on their criticality ratings. Functions of high criticality are considered first, ensuring that uncertainties and risks are addressed at an early stage.
-
4. Selection of planning strategy
Depending on the distribution of criticalities and the nature of interactions, a suitable strategy is selected:
Problem-oriented approach for focused validation of high-criticality sub-functions, or
Process-oriented approach for integrated, system-level development.
In most applications, a combined strategy is used to exploit the advantages of both.
-
5. Planning of prototype sequences
Prototypes are defined to verify specific functions or combinations thereof. Early prototypes typically focus on isolated critical functions, while later prototypes integrate multiple functions to assess interactions and system-level performance.
-
6. Implementation and iteration
Virtual and physical prototypes are created, tested, and iteratively refined according to the planned sequence. Termination criteria or gates may be defined to transition between prototyping stages.
-
7. Integration and validation
Mature sub-functions are combined into comprehensive prototypes for system-level testing, validation, and performance evaluation. Knowledge gained throughout the process is documented to inform future planning cycles.
2.3. Representation of design information
The method employs multiple formal representations to capture and organise design information. The functional structure of the product serves as the foundation, decomposing the overall system into its constitutive sub-functions (step 1). A criticality matrix (see section) is then constructed to evaluate each sub-function according to three parameters—novelty, technical difficulty, and importance—each rated on a three-level scale (low, medium, high) (step 2). The aggregate of these evaluations (step 3) classifies the sub-functions into criticality levels (low, medium, or high). Subsequently, prototyping cascades and proto-maps are used to visualise the planned prototype sequence, indicating for each prototype its objectives, represented functions, embodiment type (virtual or physical), and degree of comprehensiveness (step 4 and step 5). These representations jointly serve as a structured foundation for decision-making and documentation throughout the prototyping process (step 6).
2.4. Intended use
The method is intended to guide the planning of prototyping sequences for products and systems that exhibit functional complexity, multiple interdependencies, or novel technological elements. It is particularly suited for complex or moderately complex mechanical systems and for contexts in which additive manufacturing or other innovative production methods are employed, as exemplified by its application to the development of a hydraulically clamping toothed belt pulley. The method supports both exploratory and validation-oriented prototyping activities across the entire product development process.
Its purpose is to enable a structured and transparent planning of prototype sequences that ensures high-risk and high-uncertainty areas are addressed early, while promoting efficient integration and verification of lower-risk functions in later stages. The method thus covers the problem of unstructured or redundant prototyping efforts, late discovery of functional incompatibilities, and inefficient allocation of development resources.
The expected benefits include:
-
• Targeted reduction of prototyping time and cost through prioritisation of critical issues;
-
• Improved risk management by early verification of high-uncertainty sub-functions;
-
• Structured progression from focused to holistic prototype validation;
-
• Enhanced traceability and documentation of design rationale and knowledge gain; and
-
• Increased flexibility through iterative and adaptive sequencing.
The method assumes that the designer or development team possesses sufficient understanding of the system architecture and its interdependencies to conduct the functional decomposition and criticality evaluation. Tool support is advantageous, particularly for complex systems with numerous interacting sub-functions.
2.5. Criticality assessment
As previously discussed, the objective of this study is to evaluate criticality assessment (step 2). Specifically, it will examine the applicability of the assessment scales both for different individuals conducting the assessment and in terms of content, concerning different projects. For this reason, criticality assessment will be presented in greater detail in the following section.
2.5.1. The criticality level
The overall assessment of functional criticality follows the approach proposed by (Reference Schork, Güttinger and KirchnerSchork et al., 2020), which builds upon a prioritisation framework for validation activities from (Reference Albers, Klingler and WagnerAlbers et al., 2014). In this approach, the criticality of each sub-function within a system is determined using three evaluation parameters: novelty, technical difficulty, and importance in the product. Each parameter is rated on an ordinal scale from low (1) to high (3), representing the degree of uncertainty, complexity, or relevance associated with the respective function (see Figure 2a).
Criticality matrix based on Reference Schork, Güttinger and KirchnerSchork et al. (2020)
The combination of these three individual ratings yields the overall criticality level of a sub-function. A low criticality (green) is assigned when no more than two of the three ratings are of medium importance. If at least one of the ratings is high, the sub-function attains medium criticality (yellow). When two or more parameters are rated as high, the sub-function is classified as highly critical (red).
2.5.2. Description and definition of the scale
As shown in Figure 2, the three evaluation criteria can be summarised broadly. However, a detailed description is required for the application of the evaluation. The starting point for the study is the following summarised description (see Table 1) provided by Reference Zorn, Glaser and GerickeZorn et al., 2024.
Definition of the assessment scales
3. Study design
3.1. Study overview and objectives
The evaluation of the criticality assessment for the prototyping planning method was conducted through two consecutive Master’s theses, each constituting an independent application study. The studies were deliberately staggered to enable iterative refinement of the assessment procedure, particularly concerning the rating scale for functional criticality.
At the outset of each study, the respective mechanical engineering student received the methodological description of the criticality-based planning approach as published in the literature (Reference Zorn, Glaser and GerickeZorn et al., 2024), alongside a prior application study that demonstrated its feasibility (Reference Zorn, Krügener and GerickeZorn et al., 2025). Both students had been trained in the use of selected methods according to Pahl and Beitz (Reference Bender and GerickeBender & Gericke, 2020) through their previous participation in the ‘Design Methods’ module of the Master’s programme in Mechanical Engineering. Additionally, they were granted full access to the primary sources in which the three-dimensional criticality scale—novelty, technical difficulty, and importance in the product—had originally been introduced (Reference Albers, Klingler and WagnerAlbers et al., 2014; Reference Schork, Güttinger and KirchnerSchork et al., 2020). Over a five-month project period, each student was instructed to apply the method to a design task provided by an industrial partner and to develop and implement the corresponding sequence of prototypes.
Study 1 focused on unassisted interpretation of the scale descriptions. After completion of the first study, the author analysed the results with particular attention to the interpretation and practical use of the criticality scale. The insights gained informed a refinement of the written instructions for the second study, in which more explicit guidance regarding the operationalisation of the rating dimensions was provided. The second study followed the same overall design and used the same set of methodological materials, thereby enabling a controlled comparison of the method’s application under incrementally improved instructional conditions.
This staged design allowed the research to observe both the unassisted interpretation of the scale (Study 1) and the effect of clarified guidance (Study 2), providing a robust basis for evaluating the comprehensibility and practical applicability of the criticality assessment within the wider method.
3.2. Data collection
The application of the method was documented through retrospective reflections in the written theses, forming the main source of qualitative data on the usability and interpretability of the rating scale. In addition, expert interviews with each student were conducted by the author, focusing on: (a) interpretation of scale definitions, (b) difficulties in assigning ratings, (c) perceived ambiguity or overlap between criteria.
There were at least 3 interviews. An initial semi-structured interview to capture the students’ understanding of the method and scale definition. This understanding is also expressed in the thesis.
One to two interviews during the project period to capture emerging insights, difficulties and contextual influences on the assessment process. And one final reflective interview focusing on scale usability, ambiguities and perceived overlaps.
3.3. Data analysis
All interview statements and written reflections were analysed using qualitative thematic coding. Statements were compared against the intended methodological definitions. Deviations were coded as different understandings when interpretations conflicted with the conceptual intent of the scale dimensions. These coded observations form the basis of the comparative Tables presented in sections 4 and 5.
3.4. Application examples
The criticality-based assessment of sub-functions was applied and evaluated in two independent industrial development projects conducted in collaboration with CORTRONIK GmbH and Nagel Maschinen- und Werkzeugfabrik GmbH & Co. KG. Both projects represent distinctly different prototyping contexts: (1) the creation of a prototype for a production process in a medical device environment, and (2) the redesign of an assembly in a high-precision machining system. These application examples were chosen to represent a broad spectrum of development projects. The aim was to make the scale applicable to a wide variety of use cases.
3.4.1. Example 1 - production process for medical devices
The first application concerns the development of a prototype for the automation of a manufacturing process for medical implants. The biggest challenge was not to manufacture a conventional product, but to ensure the consistency of the manufacturing process for the implants and the additional materials required for production. The architecture of the production line consisted of an electropolishing system, an expansion system and a microblasting system.
The criticality assessment was therefore essential to determine which elements of the test architecture required early prototyping. The project’s agile setting added a further challenge: design knowledge evolved rapidly, and assessments had to be revisited across multiple sprints as new manufacturing insights emerged. The example demonstrates how the method supports risk reduction in manufacturing process-oriented prototyping, where prototypes serve primarily to gain knowledge about the viability, repeatability and boundary conditions of a production process rather than a final product.
3.4.2. Example 2 - fixture assembly in a high-precision machining system
The second application involved the redesign of a precision fixture assembly used in a horizontal honing machine—an environment governed by tight tolerances, rigid geometric constraints and high sensitivity to function deviations. Unlike the first case, this project focused on an assembly that is fully integrated into a machine tool, where micrometre-level positioning accuracy, repeatable clamping behaviour, and robust mechanical interface are mandatory for process quality.
The primary challenge was the system’s dense network of physical and kinematic interactions: small changes to individual sub-functions (e.g., clamping force transmission, alignment features, actuation paths) had a disproportionate impact on overall performance. To manage this complexity, the importance dimension of the criticality assessment was quantified via a Design Structure Matrix, ensuring that tightly coupled or accuracy-critical functions were identified early and addressed through focused prototypes.
This case illustrates the strength of the method in high-precision engineering contexts, where prototyping must verify functional interactions under strict constraints and where dependency-driven prioritisation is essential to avoid costly late-stage design errors.
4. Application evaluation - Example 1
4.1. Results
The first step was to examine the understanding of the evaluation criteria and the explanation for the application of the rating scale. Table 2 is followed by a presentation of the ratings for all three factors, as formulated by the user in case 1.
Comparison 1: Understanding and guidance for applying the rating
The first study applies the three-dimensional criticality scale – novelty, technical difficulty, and importance in the product – in a predominantly qualitative and discursive manner. The scale is used as a communicative tool to support prioritisation within an agile development environment and was verbalised as follows:
Novelty
-
• 1 – Low: Relevant experience with the function is available; knowledge from previous or analogous projects can be transferred directly.
-
• 2 – Medium: Partial experience exists, although adaptations are required. The function is somewhat familiar but not fully understood.
-
• 3 – High: No comparable experience is available. The function is new or only poorly understood, and potential interactions with other sub-functions remain unclear.
Technical Difficulty
-
• 1 – Low: The function can be realised with minimal effort. Technical relationships are transparent, and no significant constraints or risks are expected.
-
• 2 – Medium: Realisation requires moderate engineering effort, with individual technical risks or cross-component dependencies.
-
• 3 – High: Implementation is expected to be complex, involving significant interactions, demanding technologies or high development effort.
Importance in the Product
-
• 1 – Low: The function has a limited influence on overall system performance. A partial failure would not significantly impair the main product function.
-
• 2 – Medium: The function contributes to overall performance but is not central to system viability. Its failure would cause noticeable but not critical degradation.
-
• 3 – High: The function is essential for the product’s performance. Its failure or insufficient fulfilment would strongly impair the system or cause a functional breakdown.
The first application of the scale emphasises interpretative judgement and situational team discussion rather than formalised rating rules.
In the accompanying interviews, it was observed that users had considerable difficulty classifying the scale descriptions. This was particularly evident in the case of the evaluation criterion ‘novelty’. Here, for example, there was considerable uncertainty as to whom ‘existing knowledge’ referred to and how this should be interpreted.
4.2. Discussion
It is noticeable that the scale is rather heuristic and descriptive, geared towards understanding and discussion. It is evaluative and is intended to assess ‘how familiar’ a function is, ‘how difficult’ it is to implement and ‘how strongly’ it influences the overall system. There are no instructions for implementation – narrative explanations, but no explicit evaluation guidelines or scale aids. These are, therefore, only qualitative descriptions that serve as guidance but need to be specified for consistent application in teams. There was also considerable uncertainty as to whom, for example, the existence of knowledge refers to and where the boundaries for the different point ranges lie.
4.3. Refinement
Based on these findings, the scale definitions were refined. The revised guidance explicitly related “existing experience” to the evaluator’s available resources (individual, team, company). This adjustment aimed to reduce ambiguity and improve applicability.
The degree of novelty serves as an example here: 1 point is awarded if the following applies: ‘Experience with this function already exists – either from previous projects or from comparable applications. Knowledge is available and transferable.’ The interpretation of the scale now ensures that a connection is made between ‘existing knowledge’ and the resources. So if the person implementing the project has access to experts within the company, then it refers to ‘company knowledge’; if they are working on the project alone, then it refers to their direct experience.
5. Application evaluation II
5.1. Results
For the second study, the understanding of the evaluation criteria and the explanation of the application of the evaluation scale were also examined and compared with the initial situation in Table 3. The evaluations for all three factors, as formulated by the user in case 2, are then presented afterwards.
Comparison 2: Understanding and guidance for applying the rating
The second study applies the same three-dimensional scale in a more formalised and operational manner. Particular emphasis is placed on traceability and reproducibility, especially through the use of a Design Structure Matrix (DSM) to quantify functional interactions. The rating scale is adjusted to the maximum number of interactions that occur by dividing the maximum value by the three rating ranges. The second interpretation is therefore more analytical and rule-based, enabling consistent application across reviewers and facilitating quantitative prioritisation in a high-precision engineering context.
The criticality assessment is an inventory taken at a given moment in time, based on the facts available at that time. It was observed that in the second case, the criticality assessment was understood as a snapshot at a specific point in time, based on the facts available at that time.
5.2. Discussion
There are clear differences in precision and methodological understanding between applications evaluation I and II. In application evaluation II, the scale is understood as a rule-based assessment tool and no longer as a heuristic discussion aid. The person conducting the evaluation formulates explicit, operationalisable rules (‘if … then …’) and follows Zorn et al. and Schork et al. (explicitly: DSM + 3×3 matrix). They give genuine instructions for implementation (table structure, DSM totals, colour coding = green/yellow/red, definition of threshold values).In summary, it can be said that it is very analytical and methodical, and the evaluation thus forms the basis for an algorithmically comprehensible sequence.
Looking at the individual evaluation criteria, we see that novelty is cited with roughly the same importance. However, it is added that functions with a high degree of novelty are particularly relevant for prototypes. He deliberately shows that novelty is the primary trigger for prototyping. (A clear scale description is provided, which makes it clear that novelty is understood as a risk indicator.
Furthermore, it is noticeable here that he refers to the available resources in his evaluation. - ‘in the company’ is understood as the project implementer and the resources available within the company.
Technical difficulty is clearly understood as the technical implementation effort and is linked to design constraints (installation space, specific tolerances, etc.). Importance in the product is understood as quantitatively measurable system relevance, the threshold values of which can be clearly determined via functional interactions. The formulated instructions can be used directly as assessment guidelines.
6. Refined scale
The refinement addresses the specific ambiguities and interpretation issues identified during application. The refined definitions, therefore, constitute procedural guidance intended to enable future systematic application. The refined scale is shown below.
Novelty: Reflects the extent to which existing knowledge or prior experience can be transferred to the function under consideration. The knowledge and experience must be related to the assessor and the resources available to them. It is understood as a risk indicator.
-
• 1 – Low: Established solutions exist in the company, albeit in a modified form; previous projects or documented knowledge can be directly applied.
-
• 2 – Medium: The function has unresolved aspects and has not yet been implemented within the company. Similar applications already exist outside the company. Related solutions are known but require adaptation; some uncertainty remains.
-
• 3 – High: The function is completely new and has not been implemented outside the company either. Research has not revealed any similar applications.
-
• The function is new or unfamiliar, with no applicable experience or references available.
Technical Difficulty: Technical difficulty is understood as the technical implementation effort and is linked to design constraints (e.g. installation space or specific tolerances).
-
• 1 – Low: The function can be realised using standard technologies without special engineering requirements. The function can be implemented with little design effort.
-
• 2 – Medium: Implementation requires increased attention due to moderate constraints. (e.g., limited installation space, specific tolerances).e.g. High difficulty because of tight tolerances or complex installation space situation. The function can be achieved with moderate complexity.
-
• 3 – High: The function is associated with high technical demands, such as tight tolerances, complex kinematics, intricate assembly processes or the need for specialist knowledge. There are numerous possibilities for fulfilling the function. It is unclear which partial solution will be chosen. The design effort is therefore estimated to be very high. Creativity is required.
Importance in the Product: The importance in the product is understood as the amount of interaction (
$$SumI$$
) of the functions with each other. Which can be represented with a DSM. The threshold value for evaluation levels 1-3 is formed by dividing the maximum number of interactions (
$$maxI$$
) by the number of evaluation levels (3).
-
• 1 – Low: Few interactions with other sub-functions (
); limited systemic relevance. -
• 2 – Medium: Several interactions (
); moderate influence on system behaviour. -
• 3 – High: Many interactions (
$$2 \cdot {{{maxI}}\over{3}} \lt SumI$$
) or a function classified as safety-critical; failure would substantially affect system performance.
To apply the refined criticality scale, the following steps are recommended: (1) Define the assessment perspective (individual, team or organisation) and the available knowledge resources. (2) Identify sub-functions based on a functional decomposition. (3) Assess novelty and technical difficulty using the refined qualitative definitions. (4) Quantify importance using interaction analysis (e.g. DSM) where applicable. (5) Document assumptions and revisit the assessment as knowledge evolves.
The refined scale was not re-applied quantitatively to the case studies, as the objective of this paper is to evaluate and improve the interpretability and operational clarity of the scale definitions themselves.
7. Conclusion and outlook
The study has shown two main findings regarding the research question: First, that the scale can be applied across different development contexts when adapted with operational guidance (RQ1)
Second, the original descriptions are insufficient for systematic application; operationalised definitions significantly improve reproducibility (RQ2).
With the described adjustment to the definition of the scale for the evaluation criteria, it can function very well as a rule-based assessment tool and allow for a targeted approach. The definition of explicit criteria works very well for evaluating interaction as “importance in the product”. It is difficult to formalise explicit criteria for novelty. As it is understood as a risk indicator, only an estimate can be made here. Constructive boundary conditions were included for orientation in order to estimate the technical difficulty. On the one hand, this facilitates the interpretation of the scale, but on the other hand, it once again underlines the engineering perspective on the planning of the prototyping sequence.
The operational notes on understanding the position of the evaluator (viewed as an individual, as a team or company with the associated available resources and experience) have also contributed to the improved applicability of the evaluation.
Future work should focus on developing context-specific criteria, weighting factors for the evaluation parameters and the integration of additional tool support.
Open access
