1. Introduction
Numerous studies show that a significant number of new products fail in the market with failure rates ranging from 25% according to Cooper Reference Cooper(2001) to 95% according to Fuchs and Golenhofen Reference Fuchs and Golenhofen(2019). Retrospective analyses of market failures, such as the Amazon Fire Phone and the Microsoft Zune, highlight key issues. The Amazon Fire Phone failed due to its lack of differentiation and offering features that did not provide lasting value in a saturated market (Reference LuckersonLuckerson, 2014). Similarly, the Microsoft Zune struggled against the iPod due to late market entry, limited ecosystem, ineffective marketing, and inferior design and user experience (Reference FussFuss, 2019). These products did not become innovations because they did not meet valid needs or have additional benefits. Therefore, it is important to validate the underlying needs and potential benefits of a new product. This publication presents a dual perspective on the validation of needs and benefits in the early phases of development projects.
2. State of the art
Innovation is defined in various ways in the literature. Hauschildt et al. Reference Hauschildt, Kock, Salomo and Schultz(2016) categorize these definitions into seven distinct categories. Innovation can be seen as qualitative (1), first-time (2), and perceived (3) novelty, as well as the combination of need and means (4) to meet that need. Additionally, market diffusion is crucial, (5). Other categories include the type of innovation (6) and innovation as a process (7). Building on these perspectives, Albers, Heimicke, et al. Reference Albers, Heimicke, Walter, Basedow, Reiß, Heitger, Ott and Bursac(2018) define innovation as the successful realization of a novelty, creative idea, or invention in the market, providing extended benefits to customers, users, and providers.
They highlight three specific elements of innovation: product profile, invention, and market diffusion (see Figure 1). The invention addresses the technical solution for a specific need, while market diffusion refers to the product’s presence and introduction in the market. Additionally, the concept of the product profile is introduced, which outlines the need and benefit situation. The product profile (see Figure 1) is a model representing a bundle of benefits, and explicitly defining the solution space for designing a new product, system, or service. The product profile is thereby a solution-open description of the product. A bundle of benefits refers to the complete set of products and services created to be sold to a customer, to provide them with extended benefits. One purpose of the product profile is to increase the accessibility of the need and benefit situation for validation. (Reference Albers, Heimicke, Walter, Basedow, Reiß, Heitger, Ott and BursacAlbers, Heimicke, et al., 2018) In the early phase of product development, it is crucial to minimize uncertainties through careful validation of product profiles in order to make informed decisions and reduce development risks (Reference Albers, Behrendt, Klingler, Reiß and BursacAlbers et al., 2017). A defined and validated product profile can help ensure that the developed product meets the actual needs and expectations of the target group.
Figure 1. Understanding of innovation (left) and the product profile template for modelling the need situation (right) (Reference Albers, Heimicke, Walter, Basedow, Reiß, Heitger, Ott and BursacAlbers, Heimicke, et al., 2018)
Based on literature, Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024) present nine challenges that prevail in the continuous validation of product profiles (see Figure 2). By linking these challenges with the characteristics of the early phase, Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024) discuss these challenges in the early product development phases and define four fields of action that support the development of design support regarding the early and continuous validation of product profiles. Enabling Prototyping emphasizes the use of prototypes to validate product profiles throughout the engineering process. Early validation supports decision-making and project success. Due to limited resources, suitable prototypes must be developed based on specific needs. High uncertainty and low documentation levels hinder require support for developers to improve prototype planning and development. Integrating Stakeholders focuses on involving stakeholders in the validation process. This includes defining stakeholders as part of the validation system or developing suitable prototypes for them. Complex information processes may complicate stakeholder integration, necessitating support for selecting, defining, and integrating stakeholders to ensure effective involvement. Managing Variety aims to define what, when, and how to validate product variants. Poorly structured processes and unclear responsibilities hinder variability management. Support is needed to address uncertainties in information, responsibilities, and processes, focusing on selecting and prioritizing validation activities. Enabling Interpretation of Validation Results supports developers in taking next steps after validation by proposing actions before and after validation activities. Unstructured processes and high uncertainty make it difficult to draw conclusions from validation results.
Figure 2. Identified challenges regarding the continuous validation of product profiles (Reference Schwarz, Thümmel, Kuebler, Doostkam and AlbersSchwarz et al., 2024)
3. Research profile: goal, questions and approach
Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024) present four consolidated fields of action related to the early and continuous validation of product profiles as modelling of demand and benefit situations. In these fields of action, possible design supports are described for an approach to validate product profiles. However, these action fields are only briefly described, and for the development of an approach, the evaluation and more detailed description of the action fields through insights from literature and industry perspective are missing. Therefore, the goal of this publication is to evaluate and describe the presented action fields for the validation of product profiles in the early phases of development projects through literature and industry in more detail, to enable the development of design support. Validation of product profiles describes if the derived product profile (see Figure 1) is representing the desired need on the market. In this work, term “early phases of development projects” corresponds to the development of products with long time-to-market periods and “early validation” is defined by conducting validation activities early on. To operationalize the research goal, three research questions were derived:
1. To what extent do further literature evaluate the presented challenges?
2. To what extent do industry interviews evaluate the presented challenges?
3. How can the insights of literature and industry be used to described the fields of action better?
To address the research questions, the following approach were selected. The literature review followed a systematic and structured approach to analyse the current state of research on methods, processes, tools, best practices, and challenges in validating product profiles during early development phases (keywords for systematic review marked in italics). Specific search strings were employed in the scientific databases Web of Science and Scopus, resulting in the identification of 28 relevant publications, which were subsequently analysed using a qualitative content analysis according to Mayring Reference Mayring, Mey and Mruck(2019). The findings in the publications were categorised by the challenges and fields of action, and the identified challenges were evaluated, answering the first research question. The expert interviews were conducted to gain practical insights and expand upon the findings from the literature review. Semi-structured interviews were carried out with professionals from industry, guided by an interview protocol containing open-ended questions on the relevant topics. The interviews were transcribed, qualitatively analysed according to Mayring Reference Mayring, Mey and Mruck(2019), and compared with the literature-based findings to further address the evaluation of the challenges and therefore the fields of actions regarding the second research question. Subsequently, based on the results of the literature review and the expert interviews, the fields of action were described in greater detail answering the third research question.
4. Evaluation of the challenges through literature and interviews
In the following chapters, the identified challenges from Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024) will be evaluated by literature and interviews.
4.1. Results of the conducted literature review
The identified literature was systematically reviewed to identify the most relevant studies. Initially, the search results were screened based on their titles and abstracts to exclude irrelevant sources. The remaining studies were then further evaluated based on their conclusions. In the final phase, the full texts of the studies were read and analysed in detail to ultimately determine their relevance and quality. The exclusion is based on no or lack of focus on validation of product profiles, use and benefits. Based on the search string 2.152 results were found, 1.428 in Scopus and 724 in Web of Science. After checking for duplicates, the remaining 1,890 publications were systematically filtered by analysing title and abstract, reducing the number to 510. Further selection based on summaries resulted in a reduction to 109 results. Finally, the remaining studies were examined, leading to the selection of 28 scholarly papers for the final analysis (see Table 1).
Table 1. Resulted publications from the conducted literature review numbered from S1 to S28
These 28 relevant publications served as the basis for the systematic identification of methods, processes, and tools, as well as best practices and challenges in the validation of product profiles. From these sources, key findings for the early phase of development projects were derived.
4.2. Results of the conducted interviews
As part of this publication, eight semi-structured interviews were conducted with eight experts from various departments of pre-development or product planning in the automotive industry (see Table 2). The selection of interview partners was executed based on their expertise and position within the early phases of product development in the automotive industry, as well as their experience and expertise in the early validation of needs and benefits of new vehicles. This was to ensure that there is expertise in early validation and early product development projects.
Table 2. Characterization of the participants regarding their experience in early development and validation
The interview was structured into three parts. At the beginning, participants were asked about their background to characterize them in terms of their expertise. In the second part, the topic was introduced to the participants by explaining the necessary background on innovation and product profile, as well as the overarching objectives of the research project. In the third and final part, questions were asked based on three areas: general challenges, best practices, and currently used approaches in the field of product profile validation. The answers were then evaluated as explained earlier and the findings were used in the evaluation of challenges.
4.3. Evaluation of the challenges through literature and interviews
This section focuses on evaluating the challenges identified by Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024) for the later evaluation of the fields of action. The evaluation is based on literature review and expert interviews.
Figure 3 provides an overview of the overall confirmation of challenges based on both the literature and the expert interviews. In the following chapters, a more detailed evaluation of each challenge is presented by presenting mentions (citation) in the identified publication or statements (P#) in the conducted interviews.
Figure 3. Confirmation of the challenges based on literature and interviews
4.3.1. C1 - Humans as part of the validation system
The literature supports the notion that subjective judgements undermine the objectivity of the validation process when humans are a part of it (Reference Bordegoni, Cugini, Ferrise and GraziosiBordegoni et al., 2014; Reference WangWang, 2012). Additionally, subjective interpretations of user experiences (Reference Albers, Hirschter, Fahl, Reinemann, Spadinger, Hünemeyer and HeimickeAlbers, Hirschter, et al., 2019) and subjective relationship (Reference Chaudhuri and BhattacharyyaChaudhuri & Bhattacharyya, 2005) exacerbate validation challenges, necessitating more support to mitigate these biases. The interviews also confirmed this challenge. It is often difficult to obtain feedback as costumers or users may be reluctant to share their true opinions (P1) or have diverse cultural backgrounds (P3; P5). In addition, the differing perspectives between developers and stakeholders further complicate the validation, because internal and external reality are often not identical (P7).
4.3.2. C2 - Existence versus necessity of prototype
The uncertainty associated with the availability of prototypes, especially in early development stages, is further confirmed by the literature. Danquah et al. Reference Danquah, Riedmaier, Rühm, Kalt and Lienkamp(2020) highlight the central role of simulation models as substitutes for physical prototypes, which often depend on incomplete data (Reference Reitmeier, Paetzold, Culley, Hicks, McAloone, Howard and DongReitmeier & Paetzold, 2011). In the interviews, it is stated that the development of prototypes plays a crucial role in the innovation process but presents challenges related to timing, cost, and resource allocation (P1). In addition, prototypes are at times created too early, making it challenging to ensure they meet future market needs or are even validating uncertain needs regarding their future relevance (P3). Furthermore, developing a single prototype consumes significant resources, limiting flexibility and the ability to explore multiple concepts and may be inefficient or not necessary during multi-concept-phases (P7; P8). The literature and interviews indicate that it is a challenge to develop prototypes for the right situation and need, and to assess when to use other validation methods for specific validation objectives and refrain from using prototypes.
4.3.3. C3 - Developing the adequate prototype
The literature reveals that prototypes often fail to adequately reflect realistic product attributes, complicating the interpretation of test results (Reference Schork, Güttinger and KirchnerSchork et al., 2020). Furthermore, determining realistic product attributes is inherently complex (Reference Chaudhuri and BhattacharyyaChaudhuri & Bhattacharyya, 2005). The adaptability of prototypes is also an issue, as they may not be flexible enough to meet the evolving needs of the development process (Reference Pinner, Jost, Schmid and AlbersPinner et al., 2015). The interviews support these challenges, as prototypes, especially physical prototypes, are not very flexible in responding to changing needs or situations (P8). Literature and interviews show that the process of developing the appropriate prototype involves balancing flexibility for different situation and functionality regarding purpose.
4.3.4. C4 - Uncertain maturity level of prototypes
This challenge is further confirmed by the work of Albers, Haug, et al. Reference Albers, Haug, Fahl, Hirschter, Reinemann and Rapp(2018) and Murukina et al. Reference Murukina, Tipner, Prilutskaya and Kalinina(2020), who note that fluctuating market conditions and technological uncertainties make it difficult to assess prototype readiness, hindering early fault detection or early feedback. In industry, uncertainty about the maturity of prototypes is a key challenge because long development times and changing market conditions make early decisions difficult to revise (P1; P7). Also, incorrect expectations about market demands or technological advancements often lead to the premature or delayed development of prototypes (P2). Using different approaches like virtual models often has the problem of non-existent validation of virtual models due to the lack of appropriate methods (P8). The uncertain maturity level of prototypes describes the issue that in the early phases of a development project, the use of prototypes cannot be clearly determined.
4.3.5. C5 - Variety of validation methods
The literature highlights the challenges in choosing the appropriate method, especially with the integration of emerging technologies like Virtual Reality (VR) and Augmented Reality (AR) (Reference Bachvarov, Maleshkov and ChotrovBachvarov et al., 2014; Reference Eckertz, Anacker and DumitrescuEckertz et al., 2022; Reference Moessner, Haegle, Eiler and KloepferMoessner et al., 2024). Despite their potential to enhance validation, VR and AR introduce new complexities related to their integration and application. Moreover, the subjective nature of some methods further complicates the process (Reference WangWang, 2012). The variety of validation methods presents several challenges for organisations, concerning both the selection of the most appropriate method and the adaptability to new technologies. The wide range of validation methods offers flexibility but makes it difficult to select the most suitable one for different markets and target groups (P1; P3). Early decisions about methods can limit future flexibility (P4; P6) and adding new technologies could increase complexity by requiring methods to adapt (P7). This challenge describes the issue about selecting the most suitable validation method and its characteristics.
4.3.6. C6 - Prioritization of validation activities
Literature findings support the challenge of prioritization of validation activities, with the effective prioritization of test cases and resource optimization being crucial for efficient validation (Reference Köber, Behrendt and AlbersKöber et al., 2021). This challenge is further compounded by the need for flexibility in adapting validation strategies to dynamic market conditions and changing costumers and users as well as provider needs (Reference Murukina, Tipner, Prilutskaya and KalininaMurukina et al., 2020). Choosing the right timing and approach for validation activities is challenging. Long development cycles hinder timely responses to market changes, requiring constant adjustment of priorities (P1; P5). Also, early decisions often restrict flexibility later on, making it difficult to change already started or planned validation activities (P7; P8). Prioritization of validation activities is a problem in early validation of product profiles because the decision on what and how to validate depends on many factors such as relevance or effort.
4.3.7. C7 - Difficult interpretation of validation results
Interpreting validation results, particularly those from simulations and prototypes, is often challenging due to the absence of clear accessible data on market failures (Reference Borgianni, Cascini, Pucillo and RotiniBorgianni et al., 2013). This challenge is compounded by uncertainties in model outcomes (Reference Danquah, Riedmaier, Rühm, Kalt and LienkampDanquah et al., 2020), which hinder the ability to draw reliable conclusions. Furthermore, the interpretation of customer requirements is often complex and unclear (Reference Wilmsen, Dühr, Heimicke and AlbersWilmsen et al., 2019), adding another layer of difficulty in interpreting results. The interpretation of validation results is fraught with uncertainties due to limited data, changing market conditions, and the unpredictable nature of long-term customer acceptance (P3; P6; P7). The interpretation in the validation of product profiles is challenging because there is no standardized way of analysing the results of the validation to feedback it into the development process.
4.3.8. C8 - Difficult validation due to the variety of variants and versions
The diversity of product variants and validation methods creates different challenges in the early validation of product profiles. The complexity of the models and uncertainties in modelling user behaviour make it difficult to validate multiple product variants (Reference Pinner, Jost, Schmid and AlbersPinner et al., 2015; Reference Reitmeier, Paetzold, Culley, Hicks, McAloone, Howard and DongReitmeier & Paetzold, 2011). This challenge is compounded by the need to consider both the technical specifications and variable user behaviours, increasing the likelihood of errors and adding to the cost of corrective actions. Validation is complicated by the variety of product variants, as differing requirements, regional market needs, and conflicting objectives must be considered. The growing number of product variants complicates validation, as not all requirements can be tested for every version (P2). Conflicting objectives between variants necessitate compromises (P4) and adapting to regional market needs further complicates validation (P5; P7). Therefore, in validation product profiles in early project phases the interviews and literature support the challenge that the variety in concepts, products or systems often results in conflicted validation activities.
4.3.9. C9 - Low access to stakeholders
The challenge of low access to stakeholders is corroborated by the work of Livotov Reference Livotov(2016) and Munch et al. Reference Munch, Trieflinger and Heisler(2020), who stress that the success of validation heavily relies on active stakeholder participation. Limited access not only leads to delays but also diminishes the quality of feedback, further complicating the development process. Limited access to stakeholders impedes the collection of feedback and collaboration. Especially in international markets, this poses a major challenge, making it difficult to gather feedback from external customers (P1; P3; P4). Product complexity and cultural differences also hinder effective communication (P7; P8). The low access to stakeholder was evaluated by the interviews and literature and is defined as the challenge to overcome differences in culture, quality deficits in feedback and geographical or corporate distance.
5. Evaluation and description of pre-defined fields of action
In the previous chapter, the challenges were validated through literature and interviews. These challenges can be linked to the action fields according to Schwarz et al. Reference Schwarz, Thümmel, Kuebler, Doostkam and Albers(2024), allowing them to be evaluated. This integration of theory and practice emphasises the importance of a structured approach during early stages of validation.
By the predefined links between challenges and fields of action, the field of actions are evaluated (see Figure 4). The 28 selected studies provide valuable insights for refining the product profile validation and enhancing risk management during the early stages of development projects. The eight expert interviews confirm and expand upon this statement. These insights presented in the challenges emphasise the need for approaches to improve the validation of product profiles in early stages of development projects. Therefore, the next step is to describe these fields of action in greater detail to provide deeper insights into possible implementation and impact of a design support for validating product profiles in early phases of development projects. In the following chapters, insights from the literature and expert interviews were assigned to the existing four fields of action based on the challenges. The references P# for participants from the interviews (see Table 2) and S# for the source from the literature (see Table 1) are used for the individual descriptions and their origin.
Figure 4. Evaluation of the fields of action based on links to challenges by (Reference Schwarz, Thümmel, Kuebler, Doostkam and AlbersSchwarz et al., 2024)
5.1. Field of action I: integrating stakeholder
The field of action regarding the integration of stakeholder is defined on the one hand by the human factor of the stakeholders and on the other hand by the limited ability of the developers to integrate the stakeholders due to accessibility or confidentiality. Regarding the first point, it is important to define efficient and essential communication between developers and stakeholders, as regular alignment reduces misunderstandings and promotes the quality and speed of the validation process. (P1; P4; P6) Furthermore, the clear definition of roles and responsibilities enables efficient collaboration, ensuring that all parties are aware of their tasks and can make informed decisions. (P1; P4; P6) To address the second point, the crucial early involvement of relevant stakeholders, such as customers, users, suppliers and internal or external departments is to be considered (P1; P3; P5; P7; P8; S7; S18; S19; S20; S21; S24). This ensures that product profiles are adapted to market needs at an early stage, enhancing success rates and shortening development timelines. That means the developers get early insights regarding their system in development. To achieve this, use of insights, a systematic collection and analysis of stakeholder feedback guarantees that customer needs are continuously considered, thereby improving market acceptance and the product’s relevance. (P1; P8; S2; S3; S4) Therefore, the overall goal in this field of actions is to enable the situation- and need-adequate integration of relevant stakeholders. There should be a possibility to assess if it is necessary or crucial to integrate stakeholders and thus define what exactly the stakeholder should feedback. Depending on the situation adequacy, the integration could be limited or enhanced by confidentiality, accessibility, or effort of the project.
5.2. Field of action II: enabling interpretation
Enabling the interpretation of the results from validation, the second field of action, is hindered by three challenges. First, different validation methods have different results for interpretation and therefore different interpretation could exist. Furthermore, the uncertain level of maturity, especially in prototypes, during these stages could lead to incorrect interpretation and decisions. Second, often there is no standardised way of interpretation of the question, if a product profile is valid. A standardised analysis methods ensure consistency and reliability, allowing product adjustments based on accurate and reproducible data (P1; P8; S2; S3; S4). Also, a not only standardised analysis method is necessary, but also a continuous process that integrates real-time insights into the development process, enabling timely adjustments and improving product profile validity (P1; P2; P3; P4; P6; S9; S11; S12; S27; S28). This could be enhanced by integrating creative and agile methods allowing quick responses to market changes and fostering inventions with high innovation potential (S28). Lastly, improving the efficiency and responsiveness of the validation process of product profiles through available information for decision making. (P1; P8). In summary, the overarching goal of this field of action is to develop an approach, method, or process that enables the interpretation of validation results regarding the validity of the product profile. It should be possible to integrate new insights into the validation not only initially but continuously throughout the process.
5.3. Field of action III: managing variety
Managing Variety in the Validation of Product Profiles refers to both the variety of variants in the product profiles and the variety of validation methods. Firstly, managing different or synergistic validation goals or needs is crucial, as these can result from various customers, users, and providers. A clear prioritisation system for variants ensures that the most validation-intensive variants are addressed first, optimising resources and reducing risks (P1; P 2; P 4; P 5; P 6; P 8; S24; S26; S28). Additionally, efficient resource allocation is essential to validate all relevant variants on time, considering market and technical requirements (P1; P2; P4; P5; P6; P8; S24; S26; S28). Secondly, selecting the appropriate validation method concerning the validation goal and need is important to enable early prioritisation of validation activities based on the situation and requirements. Flexible validation methods allow for tailored approaches to different variants, simplifying the validation process. This means determining which method to use for which variant (P1; P8; S2; S3; S4). Lastly, clearly defined and prioritised validation objectives for each variant ensure targeted and efficient testing, improving accuracy and resource use (P1; P4; P6). In summary, the objective of this action field is the prioritisation of validation activities in the product profile concerning variance and method. The variance lies in the different generated product profiles and the number of different benefits that need to be validated. Regarding the method, support is needed to determine which method should be used for which situation.
5.4. Field of ction IV: enabling prototyping
Prototyping during the validation of product profiles aims to integrate stakeholders early in the decision-making process. It is important to consider whether an immature prototype is appropriate or if other approaches are more suitable. Therefore, developers need support in deciding whether, which, and how a prototype should be built, because the early availability of prototypes is crucial. Digital prototypes enable early validation and quick feedback, reducing development time and costs (P1; P3; P5; P7; P8; S1; S8; S10; S11; S13; S15; S16; S17; S21; S23). Cost efficiency in prototype development is achieved by minimizing the need for physical prototypes in early stages through hybrid or virtual prototypes, which lowers production costs (P1; P3; P5; P7; P8; S1; S8; S10; S11; S13; S15; S16; S17; S21; S23). The integration of digital and physical prototypes enhances the validation process by combining the speed of digital models with the accuracy of physical testing (P1; P3; P5; P7; P8; S1; S8; S10; S11; S13; S15; S16; S17; S21; S23). Finally, prototype adaptability ensures that prototypes can be quickly adjusted based on real-time feedback, allowing the product to evolve without delays (P1; P2; P3; P4; P6; S9; S11; S12; S27; S28). In this action field, the overarching goal is to enabling prototyping in product profile validation by combining the aspect of solution-openness with the necessity of situation- and need-based prototypes.
6. Summary and outlook
In summary, this publication evaluated and further elaborated on the nine challenges and associated action fields. Initially, the challenges were evaluated through 28 publications from a literature review and eight interviews with experts in the early validation of product profiles. Subsequently, the action fields were evaluated and described in depth by assigning the challenges to them. This detailed description of the action fields allows them to be used to develop and evaluate design support, thereby achieving the goal of this publication. Further research can focus on the development of design support to enable the validation of product profiles in early development phases.
Open access
