2.1 Design creativity and problems

Creativity is the production of novel and useful ideas in any domain, and innovation is the successful implementation of creative ideas within an organization (Amabile Reference Amabile1996). Kao (Reference Kao1996) defines creativity as the process through which ideas are generated, developed and transformed into value. Creativity in engineering design occurs through a process by which a person uses their ability to generate ideas, solutions or products that are novel and useful (appropriate, social value) (Sarkar & Chakrabarti Reference Sarkar and Chakrabarti2007). The definitions provided by Stein (Reference Stein1953), Amabile (Reference Amabile1996) and Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2007), (Reference Sarkar and Chakrabarti2015), after their thorough analysis, highlight novelty and usefulness as the two prime indicators of creativity.

Creativity plays a crucial role in design problem formulation and solving, which is characterized by ill-structured and non-routine problems (Casakin & Kreitler Reference Casakin and Kreitler2005). The ability to restructure problems and develop new problem descriptions is essential for creative design solutions in both architecture and engineering (Akin Reference Akin2008). Some problems, when properly structured, allow more space for exploration and thus foster creativity, whereas the narrower problems offer limited scope for substantial improvement (Goel & Pirolli Reference Goel and Pirolli1992). An attempt has been made in Singh & Chakrabarti (Reference Singh and Chakrabarti2024) to assess the structure of the design problem with the intention that it may lead to an understanding of the creativity of design problems.

While creativity in design encompasses both problem-exploring and problem-solving aspects, traditional focus has largely been on problem-solving, with very limited attention given to problem-exploring (Nidamarthi et al. Reference Nidamarthi, Chakrabarti and Bligh1997; Obieke, Milisavljevic-Syed & Han Reference Obieke, Milisavljevic-Syed and Han2020).

To address this imbalance, emergent technologies such as data mining, natural language processing and machine learning can support human efforts in identifying creative design problems, potentially leading to breakthroughs in global problem exploration and inspiring more innovative solutions (Obieke et al. Reference Obieke, Milisavljevic-Syed and Han2020, Reference Obieke, Milisavljevic-Syed, Silva and Han2023). In architectural design studios, key factors influencing creativity include pedagogy, cognitive approach, interaction, information representation and creativity measurement (Casakin & Wodehouse Reference Casakin and Wodehouse2021). Computational tools can enhance creativity in design, particularly for form finding, complexity optimization and ideation variety (Hanna Reference Hanna2018). In urban design, generative methods utilizing artificial intelligence (AI) have emerged to efficiently explore complex solution spaces and generate design options that satisfy conflicting objectives (Jiang et al. Reference Jiang, Ma, Webster, Chiaradia, Zhou, Zhao and Zhang2023). Although these approaches overcome the limitations of traditional manual design processes, in engineering, AI is not capable of combining elements to create something novel, which is the space the human still necessarily needs to fill (Lockhart Reference Lockhart2025). Human creativity involves unique neurobiological machinery and experiential components that AI cannot replicate (Aru Reference Aru2025). Hendler (Reference Hendler2006) argues that AI excels at well-defined problems but struggles to combine known elements in novel ways, emphasizing the need to explore human cognitive capabilities that remain beyond AI’s reach. These studies collectively highlight the importance of creativity in various design disciplines and the potential for computational tools, along with human efforts, to enhance creative problem formulation and solving.

2.2 Creativity assessment methods

Past research has focused on developing systematic approaches to assess creativity in product design, addressing both the design process and outcomes. Feldhusen & Goh (Reference Feldhusen and Goh1995) assessed creativity comprehensively, considering multiple measures of cognitive processes, motivations and environmental factors. Moss (Reference Moss1966) defined creativity as a combination of usefulness and unusualness. Unusualness is rated on a 0–3 scale based on how different a solution is from existing ones. Usefulness is also scored 0–3 by comparing it with a benchmark solution that meets all requirements. Shah, Smith & Vargas-Hernandez (Reference Shah, Smith and Vargas-Hernandez2003) developed metrics to evaluate the effectiveness of idea generation methods based on the quantity, quality, novelty and variety of ideas generated, proposing two experimental approaches to evaluate the effectiveness of idea generation methods: a direct method that examines the idea as a whole and an indirect method that decomposes the idea into its components and studies their individual and interactive effects. Ranjan, Siddharth & Chakrabarti (Reference Ranjan, Siddharth and Chakrabarti2018) proposed a method using novelty and requirement satisfaction as key indicators of creativity throughout the design process. Yuan & Lee (Reference Yuan and Lee2013, Reference Yuan and Lee2014) developed a computational approach that quantifies the design process and correlates it with creativity ratings, establishing a formula for creativity assessment. Their method identified six critical factors in the design process that strongly correlate with creative outcomes. Yin et al. (Reference Yin, Han, Huang, Zuo and Childs2021) compared four creativity assessment methods, finding that the Consensual Assessment Technique (CAT) and Creative Product Semantic Scale (CPSS) produced consistent rankings between experts and non-experts. These studies aim to improve the validity and reliability of creativity assessment in product design, offering potential applications in both academic and industrial settings. Chakrabarti & Khadilkar (Reference Chakrabarti and Khadilkar2003) measured product novelty by comparing it with a reference product. They used vertical criteria (like need and technology) and horizontal criteria (main and supplementary functions) with differences weighted and summed to calculate novelty.

Lozano (Reference Lozano2009) described creativity as innovation degree (novelty) and requirement accomplishment (usefulness). Novelty is rated as non-innovative, moderate or radical, and usefulness uses weights based on requirement types. These weights are often unknown during design, and usefulness scores may lack detail on which requirements are met. Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2007), (Reference Sarkar and Chakrabarti2011) combined assessment scores of novelty and usefulness to compute scores for creativity. Novelty assessment is carried out using SAPPhIRE models to compare products at different abstraction levels, as shown in Figure 1. Usefulness is assessed as a product of the following factors: (L) level of importance (LoI), (R) rate of the popularity of use, (F) frequency of usage and (D) duration of use or duration of benefit per usage. The LoI is assigned on a scale of 1–5 according to the purpose of the product being designed. The other factors, R, F and D, apply only to those products that are already out in the market. Sternberg & Lubart (Reference Sternberg, Lubart and Sternberg1999) define “usefulness” in terms of “appropriateness.” Similarly, Mumford & Gardner (Reference Mumford, Gardner and Sternberg1994) define “utility” in terms of “usefulness, appropriateness and social value.” From these definitions, we understand that “useful” things are those that carry “social value.” Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2011) further argued that the usefulness of a product is indicated by its use (i.e., a product is not useful if it is not used) and proposed the above four parameters for the assessment of usefulness of a product.

Figure 1.

Proposed novelty assessment method by Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2007).

Srinivasan & Chakrabarti (Reference Srinivasan and Chakrabarti2010b) used a similar novelty assessment, comparing new ideas to those in the existing idea space. The distinction is that the SAPPhIRE model’s various abstraction levels assign scores of n_a = 7, n_s = 6, n_i = 5, n_ph = 4, n_e = 3, n_r = 2 and n_p = 1; if there is no difference at any of those abstraction levels, n = 0.

2.3 SAPPhIRE model of causality

The SAPPhIRE model of causality was developed by Chakrabarti et al. (Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005), as shown in Figure 2, to explain the causality of natural and engineered systems and is often cited for novelty assessment. The model gets its name from the highlighted letters of its constructs: Action, State change, Physical Phenomena, Physical Effect, Input, oRgan and Parts. They together cover the system’s physical components, interfaces, interactions, structural context and scientific laws governing them and provide a rich, comprehensive description of the functioning of biological and technical systems. To improve the comprehension of the constructs and, consequently, the application of the model, Srinivasan & Chakrabarti (Reference Srinivasan and Chakrabarti2009) improved the clarity of the definitions of these constructs. Later, using appropriate rules and reasoning, Bhattacharya et al. (Reference Bhattacharya, Majumder, Bhatt, Keshwani, BSC, Venkataraman and Chakrabarti2024) improved the process for automated extraction of SAPPhIRE abstraction levels from a natural language document. Singh & Chakrabarti (Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025) interpreted the three types of SAPPhIREs – normal state, failure state and recovered state proposed by Siddharth, Chakrabarti & Ranganath (Reference Siddharth, Chakrabarti and Ranganath2020) as solution, problem and intended solution SAPPhIREs, respectively, and used problem SAPPhIREs in their preliminary framework for assessment of problem novelty.

Figure 2.

SAPPhIRE model of causality (Chakrabarti et al. Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005).

The comparison between abstraction levels for reference and current products in terms of their novelty was illustrated by Srinivasan & Chakrabarti (Reference Srinivasan and Chakrabarti2010b) and Ranjan et al. (Reference Ranjan, Siddharth and Chakrabarti2018). As a method for usefulness, Ranjan et al. (Reference Ranjan, Siddharth and Chakrabarti2018) evaluated the degree of requirement satisfaction (DRS) against solutions using the weighted objective tree method. Maher & Fisher (Reference Maher and Fisher2012) assessed novelty, value and surprise by comparing products in the same domain, where products are represented as attribute vectors and novelty is represented as the Euclidean distance between them. Grace et al. (Reference Grace, Maher, Fisher and Brady2015) identified novelty and value as key features of creative design, wherein novelty is measured by the distance from a design to others using clustering and normalization; and value is estimated by the device’s utility or popularity, such as web search results. The distance between word vectors as a method for comparing SAPPhIREs for idea novelty was proposed by Siddharth, Madhusudanan & Chakrabarti (Reference Siddharth, Madhusudanan and Chakrabarti2019).

2.4 Observations from the literature

Design problem identification and formulation have a strong influence on the creativity of solutions it produces; even a slight change in formulation can lead to very different design directions for the solution. By examining how the problem is stated, we can anticipate whether it is likely to spark novel and useful ideas (Wang et al. Reference Wang, Nguyen and Zeng2015). The novelty of design problems is defined as the extent to which they are new and original, serving as a direct translation of the definition originally applied to design solutions. For instance, the creation of Airbnb, which is unquestionably novel, was a novel solution to the novel problem of affordable and distinctive lodging options, particularly for tourists looking for experiences outside of traditional hotels (Guttentag Reference Guttentag2013). In contrast to novelty, the definition of usefulness cannot be directly translated when applied to design problems, as opposed to design solutions. The usefulness of solving a design problem indicates only the potential usefulness to society if the design problem is solved well. In contrast, the usefulness of the solution represents the realized usefulness when the problem is worth solving and is solved well. We argue that if the problem is not solved well, the solution will not be a useful one. However, if the problem lacks potential usefulness, no matter how well it is solved, the solution will not be useful. If we choose a problem arbitrarily, potential usefulness is left to the mercy of chance, regardless of how well all its alternative solutions solve it. Hence, the importance of usefulness of a design problem is the potential usefulness (scope of usefulness) created by the problem. Choosing an arbitrary solution at the end of the problem-solving process does not necessarily ensure that a problem’s potential is realized.

Creativity of design solutions is defined as the product of novelty and usefulness (Sarkar & Chakrabarti Reference Sarkar and Chakrabarti2011), and this formulation may also be extended to define the creative potential of design problems. The difference between problem creativity and solution creativity is that in problem creativity, among possible, alternative problems that are explored, some provide greater creative potential: are more novel than others (i.e., more different than the problems that have existed up until now) and are potentially more useful to solve than others. For any of the above problem alternatives, each with a different creative potential, there can be multiple possible solutions, some of which would be better at achieving the creative potential offered by the problem than others. Problem creativity is about creating the potential for being creative, while solution creativity is about realizing the potential. If the potential to start with does not exist or is limited during problem identification, it limits the scope of what can be realized during problem-solving. Assessing the creative potential of design problems adds an additional layer to compound creativity, as the current focus remains limited to the creativity of design solutions alone.

The methods and metrics have been proposed for the design outcomes, such as products or solutions, but not for the creativity of design problems, which is a bigger research gap, as found out. There are very few qualitative approaches in the literature that have examined this topic. A simple method for accomplishing this has been proposed by Roser & Dartnall (Reference Roser and Dartnall2007). It incorporates the use of simple assessment indicators, rating the level of problem novelty and complexity. Since these qualitative measurements can only provide subjective judgments based upon problem understanding, a quantifiable metric is required to overcome this bias. Singh & Chakrabarti (Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025) made one of the initial attempts to quantitatively assess the novelty of design problems with a limited and simpler problem set and analysis. Current literature also lacks in the assessment of the (potential) usefulness and creative potential of design problems. The study proposed by Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2011) incorporates the quantitative assessment of the usefulness and creativity of design solutions. However, the parameters proposed in this study could also be used as indicators for whether a design problem, if solved, is likely to be useful or not.

4.1 Adaptation of the method for assessment of novelty of a design problem

An evaluation of the existing methods for assessing novelty (Srinivasan & Chakrabarti Reference Srinivasan and Chakrabarti2010b; Ranjan et al. Reference Ranjan, Siddharth and Chakrabarti2018) reveals the following: For comparing solutions, these methods use a scoring system that gives higher values to ideas or solutions that differ at higher abstraction levels than those at lower abstraction levels in the SAPPhIRE model of causality. However, various aspects of a problem may exist at different levels of abstraction. Therefore, a problem cannot necessarily be assessed at a single level of abstraction; each aspect of the problem may have to be addressed at its relevant level of abstraction. Assessment for design problems with a single cause (represented using single-instance SAPPhIREs) is demonstrated in our previous study (Singh & Chakrabarti Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025). For complex problems with multiple root causes, the problems need to be represented using a multi-instance problem SAPPhIRE model and subsequently assessed. Based on this understanding, the previously developed method by Singh & Chakrabarti (Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025) is adapted for a broader set of problems and for a detailed evaluation of novelty of design problems. Method steps are explained below.

To address the first research sub-question (how to assess the novelty of a design problem), data on similar design problems are collected from primary stakeholders, i.e., designers (of 2–3 years’ experience in the product design domain) and customers (using such products for 3 years or more), for their design solutions. The research methodology (Figure 3) involves the following steps:

1. 1. Existing design problems in the same context from patents or online searches are collected, and their classification as “past problems” is carried out.

2. 2. Current (i.e., new) design problems are collected using a questionnaire and survey, as faced by stakeholders (designers and customers) in that context, and classified as “current problems.”

3. 3. Each of the past and current problems so collected is described using the SAPPhIRE model of causality, henceforth called problem SAPPhIRE (Siddharth et al. Reference Siddharth, Chakrabarti and Ranganath2020). The completeness of SAPPhIRE models is taken care of using the method given by Bhattacharya et al. (Reference Bhattacharya, Majumder, Bhatt, Keshwani, BSC, Venkataraman and Chakrabarti2024). Design problems with a single root cause are represented using single-instance problem SAPPhIREs, while those with multiple root causes are represented using multi-instance problem SAPPhIREs. A multi-instance problem SAPPhIRE is constructed by linking multiple single instances, where the input of each next instance is derived from the action level of the previous instance.

4. 4. The natural language description of each past and current problem SAPPhIRE in these two distinct databases is compared with one another for similarity at the action level of SAPPhIRE (the comparison is facilitated using Bidirectional Encoder Representations from Transformers (BERT) semantic similarity with a pre-trained dataset). For multi-instance SAPPhIREs, comparison is carried out after concatenating the sentences at each abstraction level of SAPPhIRE.

5. 5. For those problem SAPPhIREs for which the action level of SAPPhIRE matches, sentence similarity and word-to-word similarity are calculated between each pair of problem SAPPhIREs by converting their words into word vectors and measuring the cosine distance between them at each remaining level of SAPPhIRE abstraction (Siddharth et al. Reference Siddharth, Madhusudanan and Chakrabarti2019). Semantic similarity is first assessed between the word vectors and then extended to the sentences. The pre-training of data is performed using the model “sentence-transformers/all-MiniLM-L6-v2.”

6. 6. Similarity score for each level of SAPPhIRE abstraction is then calculated, where dissimilarity (novelty) scores for each pairwise comparison are determined using the following equation: novelty = (1 – similarity) (Siddharth et al. Reference Siddharth, Madhusudanan and Chakrabarti2019).

7. 7. Overall novelty score of the given, current problem is now calculated, in comparison with the previous problems under consideration, by averaging these pairwise novelty scores.

8. 8. The novelty score is interpreted qualitatively using the following ranges: 0 to 0.3 as low novelty; 0.3 to 0.7 as medium novelty; and 0.7 to 1 as high novelty.

Figure 3.

Proposed methodology for the assessment of the novelty of design problems (adapted from Singh & Chakrabarti Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025).

Using the pre-trained dataset and a web-based tool for sentence comparison, some of the above processes (Steps 4 and 5) are automated. To further visualize the better applicability, a case study on one system is explained below.

4.1.1 Case study (electric kettle) demonstrating the application of the method

An electric kettle is used as the system for problem analysis in our preliminary case study (Singh & Chakrabarti Reference Singh, Chakrabarti, Chakrabarti, Singh, Onkar and Shahid2025). Different parts of the electric kettle with their functions are shown in Figure 4. Utilizing online patent libraries and patent searches, a comprehensive examination of patents pertaining to electric kettles from the last 100 years is carried out. Beyond the initial data collection, additional design problems are collected, encompassing both single-cause and multi-cause cases. After that, useful data in the form of past design problems are extracted, and using these, a database of past problems for this system is developed. In Table 1, a few of the problems collected are listed. We assume that the problems gathered from patents are “old” and problems gathered currently through a questionnaire are “new.”

Figure 4.

Electric kettle (Miller Reference Miller2019).

Table 1.

Some of the design problems collected from patents and web sources

Moving forward, we collected a set of current design problems, as mentioned by the stakeholders (i.e., designers and customers), using a survey questionnaire (see Appendix, Figure A1). These problems are stored in a database of current problems. Some of these problems are enlisted in Table 2.

Table 2.

Some of the design problems collected from the survey questionnaire

The problems written in Tables 1 and 2 are ill-structured problems which, during the application of methodology, are converted into the unified category of “problems to be solved in the design process,” but somehow, they are the problems that exist at the beginning of a design process. The past problems are analyzed and transformed into their problem SAPPhIREs using the problem SAPPhIRE model, as shown in Figure 5 (Siddharth et al. Reference Siddharth, Chakrabarti and Ranganath2020) and rules given by Bhattacharya et al. (Reference Bhattacharya, Majumder, Bhatt, Keshwani, BSC, Venkataraman and Chakrabarti2024). The current problems (at the time of analysis) are also analyzed and transformed into their problem SAPPhIREs. Both are stored in the problem SAPPhIRE database (see Appendix, Table A1).

Figure 5.

Problem SAPPhIRE model (Siddharth et al. Reference Siddharth, Chakrabarti and Ranganath2020).

The problems of “spilling of liquid,” “non-optimal heating,” and “scalding of human hand” are selected as the example problems for novelty assessment, as they belong to different categories, i.e., efficiency and safety, and also based on their high frequency of occurrences as reference past problems for this case (see Appendix, Table A1). Similarly, new problem SAPPhIREs are chosen as current problems from the database. The design problem is the opposite of the action achieved by the solution (Siddharth et al. Reference Siddharth, Chakrabarti and Ranganath2020). So, the action level, which is the highest level of abstraction, is used as a design problem in this case (Figure 5). There may be instances where the problem exists at different levels of abstraction that authors have not yet encountered. So, for comparison, their action-level description includes “spilling of liquid.” The database (see Appendix, Table A1) is managed using a Microsoft Excel tool created by the researcher to save time while creating problem SAPPhIREs.

A search for common, action-level SAPPhIREs is needed, since only similar problems can be compared for novelty assessment. While selecting SAPPhIREs for problems, there may be situations where a single instance is present, while others have multiple instances. A single instance indicates that the structure includes one set of SAPPhIRE elements, while a multi-instance structure contains multiple, connected sets of SAPPhIRE elements, typically due to the presence of several root causes. Direct comparison is used for single instances (Figure 6). The abstraction levels are first concatenated in the case of multiple instances, and then, the same comparison approach is used, as illustrated in Figure 7. Following a comparison of sentence semantics at the action level, the semantic similarity of sentences is compared at further abstraction levels, which include state change and phenomena. Some of the formulated and assessed design problems, drawn from the broader set of assessed problems (see Appendix A1, Table A2), are presented below:

1. 1 Spilling of liquid due to liquid expansion

2. 2 Spilling of liquid due to seal and gasket rupture

3. 3 Non-optimal heating due to a faulty switch and damaged wire

4. 4 Non-optimal heating due to a faulty thermostat

5. 5 Non-optimal heating due to reduced contact area

6. 6 Scalding of the human hand due to hot steam

Figure 6.

Comparison between two single-instance problem SAPPhIREs.

Figure 7.

Comparison between two multi-instance problem SAPPhIREs after concatenation of sentences.

4.1.2 Results

The comparison is carried out among each set of problems, using their current and past problem SAPPhIREs. Comparison scores corresponding to each abstraction levels are computed and integrated into an overall score for the entire problem. These are tabulated for some of the problems in Column 3 of Table 3. Based on the proposed range of quantitative values for qualitative interpretation (see Section 4.1), inferences are made about the degree of novelty for each problem (see Column 4 of Table 3). The level of novelty (high, medium and low) of the design problems related to an electric kettle, according to their scores, is shown in Table 3 (5 having the highest novelty and 3 having the lowest):

Table 3.

Novelty scores as computed from the proposed method for different design problems

4.1.3 Expert validation for correctness and effectiveness of the proposed method

In design literature, several researchers have emphasized expert judgment as one of the most reliable approaches for evaluating the creativity of design solutions (Amabile Reference Amabile1996; Sarkar & Chakrabarti Reference Sarkar and Chakrabarti2011). Accordingly, for the correctness of the proposed method, the inferences that are made for the degree of novelty of assessed design problems need to be compared with the intuitive notion of experts.

Three experts are chosen, and they are asked to assess the novelty of a representative set of design problems, selected to capture a wide variety while minimizing the resource-intensive (time and effort-intensive) nature of evaluation. Each expert had at least three years of experience in new product development and is aware of the SAPPhIRE model of causality. To ensure a diverse perspective, the experts are from different regions of the world, accounting for diversity in product understanding and usage.

As illustrated in Figure 8, the experts evaluated six design problems ((1) spilling of liquid due to liquid expansion; (2) non-optimal heating due to a faulty thermostat; (3) scalding of the human hand due to hot steam; (4) spilling of liquid due to seal and gasket rupture; (5) non-optimal heating due to reduced contact area; and (6) non-optimal heating due to a faulty switch and damaged wire) from a similar context by rating their novelty at five abstraction levels (which are expressed in more general terms to make it suitable for the experts), consolidated from the original seven for greater clarity (Siddharth et al. Reference Siddharth, Madhusudanan and Chakrabarti2019).

Figure 8.

Template for expert scoring based on their intuition.

The general terms (see Appendix, Table A2 for definitions) are purpose (Action), output (State Change), process (Physical Phenomena and Effect), input and components (Parts and oRgan). Novelty scores ranged from low (0.3) to medium (0.6) to high (0.9). Novelty scores given by all three experts, abstraction-level-wise, are shown in Figure 9, with the values distinguished by variation in color. Dark green shows highly novel, light green followed by yellow for medium novelty, and red shows low novelty. Judgment is based upon the experience and intuition of experts; they were not asked to follow any method.

Figure 9.

Novelty scores by three experts for all five abstraction levels across six different design problems.

Considering the degree of novelty of the whole problem (low – L, medium – M and high – H; explained in Section 4.1), in four out of the six cases, all three experts agree with the researcher’s scores and inferences as obtained from the proposed assessment of design problems (see Table 4). In five of the six cases, at least one expert out of the three agrees with the researcher’s scores and inferences. Another way is to count all the matches (Cohen Reference Cohen1960) and divide them by the total number of pairs compared (see Table 4). Out of 18 pairs compared, the number of matches is (3 + 0 + 1 + 3 + 2 + 3 = 12); so % match is 12/18 = 0.67. This indicates that there is over 67% agreement with the scores of all the experts and over 83% agreement with the scores of at least one expert. Abstraction-level-wise agreements are shown in Figures 10–12.

Table 4.

Comparison of the researcher’s novelty scores with those of experts for various design problems

Figure 10.

Comparison between the researcher’s and experts’ scores for the state change level.

Problem 5 has received high novelty rankings, and problems 1, 4 and 6 have received medium novelty rankings from all three experts, with which the researcher also agrees. Figure 11 compares the novelty scores given by researchers and experts for the state change level. It is noted that the state change level, due to its higher level of abstraction, provides a higher novelty score. A similar trend exists at the phenomenon and effect levels. This is because more problems are typically found at the part or organ level, and so the similarity, where the experts have given a lower novelty score except for the case where a completely different part is invented. Problems at the phenomenon level are uncommon, like state change. According to Srinivasan & Chakrabarti (Reference Srinivasan and Chakrabarti2010b), there is a strong correlation between novelty scores and levels of abstraction, which is partially supported by our research.

Figure 11.

Comparison between the researcher’s and experts’ scores for phenomena and effect level.

Figure 12.

Comparison between the researcher’s and experts’ scores for the part and organ level.

Figure 13.

Weighted objectives method (Ranjan et al., Reference Ranjan, Siddharth and Chakrabarti2018) illustrating the calculation of DRS.

Figure 14.

Proposed methodology for the assessment of usefulness of (solving) design problems.

Figure 15.

Weight distribution of the three variables for usefulness score using the analytic hierarchy process tool (https://www.spicelogic.com/Products/ahp-software-30).

Figure 16.

Problem tree for an electric kettle.

Figure 17.

Variation of creativity with novelty and usefulness.

Furthermore, a comparison of the proposed method with one of the methods available in the literature (Srinivasan & Chakrabarti Reference Srinivasan and Chakrabarti2009) is carried out to evaluate the effectiveness of the proposed method. The method proposed by Srinivasan & Chakrabarti (Reference Srinivasan and Chakrabarti2009) is a manual assessment method used for assessing the novelty of an idea. Novelty of an idea is calculated by looking at the highest level of abstraction of dissimilarity in the SAPPhIRE model of causality and assigning the scores based on the following ranges:

“If action level is dissimilar, assign a score of 7; if state change level is dissimilar, assign a score of 6; if input level is dissimilar, assign a score of 5; if phenomena level is dissimilar, assign a score of 4; if effect level is dissimilar, assign a score of 3; if organ level is dissimilar, assign a score of 2; if part level is dissimilar, assign a score of 1 and if nothing is dissimilar, assign a score of 0.” After the score is assigned, it will be averaged considering seven no. of elements, and a final averaged score is used for qualitative inferencing of the degree of novelty.

We have applied this procedure for manual assessment on the corresponding sets of problem SAPPhIREs, which we have used in our proposed method, and calculated the novelty scores, respectively (as shown in Column 3 of Table 5).

Table 5.

Comparison of effectiveness of the proposed and the manual method using experts’ average scores

The novelty scores from the proposed method (method 1; see Column 2 in Table 5) and the manual method (method 2; see Column 3 in Table 5) are compared to the average novelty scores of all three experts. The root-mean-square (RMS) formula is used to calculate the deviation between method 1 and method 2 scores and the expert scores. The RMS deviation between method 1 scores and experts’ average scores is 0.13, while the RMS deviation between method 2 scores and experts’ average scores is 0.30. The lower the RMS, the better the method is. This demonstrates that the proposed method is the least deviant from the experts’ intuitive notion and therefore more effective than the existing method in capturing the intuitive notion of the experts. We argue that subjective models are developed primarily due to the limited availability of experts; however, when validating such models, expert judgment carries greater weight than the models themselves (Sarkar & Chakrabarti Reference Sarkar and Chakrabarti2011).

4.2 Methodology for assessment of usefulness (of solving a design problem)

The weighted objectives method, which was adapted by Ranjan et al. (Reference Ranjan, Siddharth and Chakrabarti2018) for assessing “degree of requirement satisfaction (DRS)” as a proxy for assessing usefulness, and the parameter-based usefulness assessment method by Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2011) are two major, existing methods for assessing the usefulness of design solutions. We have employed the weighted objectives method adapted by Ranjan et al. (Reference Ranjan, Siddharth and Chakrabarti2018) as part of our earlier descriptive study (Blessing, Chakrabarti & Wallace Reference Blessing, Chakrabarti and Wallace1995). In this method, low-level requirements are initially assigned weightages based on prior knowledge of users and requirement-satisfaction scores. Then, the weightages and requirement-satisfaction scores are calculated using the formula provided in Figure 13 for the next high-level requirements and so on. Following the same approach, we have determined the usefulness scores of design problems utilizing a problem-requirement tree as illustrated in Figure 13.

However, the method proposed by Ranjan et al. (Reference Ranjan, Siddharth and Chakrabarti2018) involves a higher degree of subjectivity, which limits its applicability. This subjectivity arises primarily from the distribution of weightages based on the prior knowledge of users, as well as the dependency of high-level problems on low-level problems for the calculation of requirement-satisfaction scores.

To address the second research sub-question, that is, how to assess the usefulness of solving design problems, the method proposed by Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2011) is adapted instead because it is more comprehensive and objective, making it more suitable for assessing the (potential) usefulness of design problems, using the following three parameters:

1. 1. Prevalence among users – If a larger number of people face a problem, it is more useful to solve it as it benefits a greater number of users.

2. 2. Importance to society – There is greater usefulness accrued from solving those problems that more significantly impact the users’ lives than those that do not.

3. 3. Frequency of occurrence – If a problem arises more repeatedly or across more instances (than another), solving it is more useful (than solving the other).

We have performed a similar type of data collection as in our novelty assessment. The following are the steps in the methodology proposed (see Figure 14):

1. 1. Design problems within the same context are collected through case studies, system analysis and stakeholder interviews/questionnaires (see Appendix, Figure A1).

2. 2. Problems and requirements are organized into a hierarchical tree structure using the data collected to understand the importance of a problem (Ranjan et al., Reference Ranjan, Siddharth and Chakrabarti2018).

3. 3. Usefulness indicator analysis is conducted using three literature-derived and modified variables:

   • • Prevalence (representing the rate of popularity of the problem)

   • • LoI (representing how important a problem is for stakeholders)

   • • Frequency of the problem (how frequently a problem occurs for stakeholders)

4. 4. Prevalence is calculated using the formula “n/N,” where n = number of individuals experiencing the problem and N = total number of participants in the study.

5. 5. The LoI and frequency of the problem through binary scoring (low: 0 and high: 1) is assessed by taking qualitative data from multiple people, quantifying it and then averaging the maximum binary scoring for a single inference, e.g., if the maximum values correspond to the category “high,” they are assigned a value of 1, and the average of these values is then computed.

6. 6. The weighted average usefulness score (Eqn. 1) is calculated by assigning weights to each variable through the analytic hierarchy process (AHP), using an online tool to derive the weights (Figure 15), i.e., 0.27 for prevalence, 0.61 for LoI and 0.12 for frequency of a design problem.

(1) $$ \mathrm{U}\hskip0.35em \left(\mathrm{Usefulness}\ \mathrm{score}\right)=0.27\mathrm{P}+0.61\mathrm{L}+0.12\mathrm{F} $$

where P is prevalence, L is LoI and F is frequency of a design problem.

1. 7. Results based on the scores are interpreted: Higher values indicate more useful problems to address, while lower values suggest lower usefulness.

2. 8. Usefulness of various design problems is assessed both within their original context and across different contexts, as shown in Table 6.

Table 6.

Calculation of the usefulness score using values of all variables for different problems

4.2.2 Results

Twenty participants (customers and designers), who are the main stakeholders, have been given a questionnaire-based form (see Appendix, Table A3) with the following questions:

• • Are you aware of this system and its problems?

• • Are you encountering this problem?

• • Do you feel it is important to solve this problem?

• • How frequently do you encounter this problem?

To calculate the weighted average usefulness score, qualitative data pertaining to the necessary variables is recorded and transformed into a measurable metric, as shown in Table 7.

Table 7.

Usefulness score with values of all the variables for different problems

From the scores obtained, it is found that Problem 3 (scalding of the human hand due to hot steam) is the most useful to solve, while Problem 4 (spilling of liquid due to seal and gasket rupture) is the least useful. After application of the metric in every related system, this interpretation will be later generalized.

4.3 Assessment of the creative potential of design problems

In this section, we discuss the third research sub-question: How to assess the creativity (creative potential) of a design problem? Following the method proposed by Sarkar & Chakrabarti (Reference Sarkar and Chakrabarti2011), we propose that the assessment of the creative potential of design problems should be computed as a product of the novelty and usefulness scores of the design problems (Eqn. 2):

(2) $$ \mathrm{Creative}\ \mathrm{Potential}\;\left(\mathrm{Creativity}\right)=\mathrm{Novelty} \times \mathrm{Usefulness} $$

A low score indicates low creative potential, and a high score indicates high creative potential. The ranges proposed are [0, 0.33] – low creative potential, (0.33, 0.67) – medium creative potential and [0.67, 1] – high creative potential.

The steps proposed for assessing the creative potential of design problems are as follows:

• 1. Identify and collect the design problems associated with the system investigated

• 2. Convert each into a problem SAPPhIRE as described in Section 4.1

• 3. Compare the old and new problem SAPPhIRE sets with each other for their novelty scores

• 4. Create a problem tree map to identify the sets of problems for usefulness assessment

• 5. Assess them for their usefulness scores using the proposed method as described in Section 4.2

• 6. Compute the creative potential using the creative potential formula provided in Eqn. (2) (Figure 17)

4.3.1 Results

Computation based on the existing case study is given in Table 8. Novelty and usefulness scores are taken from Sections 4.1.1 and 4.2.1, respectively.

Table 8.

Calculation of creative potential scores

According to this calculation, Problem 2 has the highest creative potential (having both high novelty and high usefulness), followed by Problem 5 (although a novel problem, it is less useful to solve). The problem with the least creative potential is Problem 1 and 4, which have a medium novelty but a very low usefulness score.

Correlation between novelty and creative potential is found to be 0.6, between usefulness and creative potential is found to be 0.64 and between novelty and usefulness is found to be 0.02, which is well justified in the literature. This gives further credence to the correctness of the proposed method.

4.4 Discussion

The methods for the assessment of novelty, usefulness and creative potential of design problems are illustrated using a case study of an electric kettle. During the collection of design problems, categories such as safety, reliability and efficiency are considered within the context of the kettle, and these categories are common when identifying design problems across a wide range of engineering and product design domains. The method for evaluating novelty employs the SAPPhIRE model of causality, which is recognized for its general applicability in engineering design, while the variables used to evaluate usefulness have been widely used in the literature across diverse products. By following the proposed steps: collection of design problems, analysis using problem SAPPhIREs (provided the SAPPhIRE representation is feasible) and comparison using semantic similarity for the assessment of novelty, together with the use of three variables in the assessment of usefulness, and consequently the creative potential of design problems, the proposed methods can be useful to a broad range of engineering and product design contexts and are not limited to any specific product type.