1. Introduction
Designers within the manufacturing industry are required to innovate to address the increasing complexity of societal needs and environmental challenges (Ceschin and Gaziulusoy Reference Ceschin and Gaziulusoy2016; Costa Junior et al. Reference Costa Junior, Diehl and Snelders2019; Delaney et al. Reference Delaney, Liu, Zhu, Xu and Dai2022). Such open and complex challenges necessitate the evolution of design thinking through the reconsideration of what is being designed (the design object), how to design (the design process), and the intended outcomes of design (the intended values) (Dorst Reference Dorst2011). One example of such a possible change in design thinking is expected to occur in the case of manufacturers who are transforming their practices from designing, developing and selling need-fulfilling products towards selling need-fulfilling solutions that are realized through product-service systems (PSSs) (Brambila-Macias et al. Reference Brambila-Macias, Sakao and Kowalkowski2018; Kang et al. Reference Kang, Feinberg and Papalambros2019; Kim Reference Kim2020).
PSSs can be considered systems consisting of integrated combinations of tangible products and intangible human-based service activities that can jointly fulfill customer needs (Tukker and Tischner Reference Tukker and Tischner2006; Meier et al. Reference Meier, Roy and Seliger2010; Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022). Despite the growing interest in transforming pure product-based to PSS-based business models, the literature has reported that product manufacturers struggle to effectively design and provide such systemic integrated solutions (Cavalieri and Pezzotta Reference Cavalieri and Pezzotta2012; Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022). Compared to product designing, a broader range of knowledge (Akasaka et al. Reference Akasaka, Nemoto, Kimita and Shimomura2012), and a significant shift in design thinking is deemed necessary to effectively design PSSs (Morelli Reference Morelli2003; Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022). Fine-grained insights into design thinking, which can be represented by the design-specific cognitive processes of human designers, are crucial to improving the design process, developing effective design support and pedagogy, and thus enhancing the outcomes of designing (Dinar et al. Reference Dinar, Shah, Cagan, Leifer, Linsey, Smith and Hernandez2015; Gero and Milovanovic Reference Gero and Milovanovic2020).
Although product and PSS design problems are characteristically distinct (Kim Reference Kim2020; Maussang et al. Reference Maussang, Zwolinski and Brissaud2009), the respective design processes are assumed to be similar forms of problem-solving activities (Vasantha et al. Reference Vasantha, Roy and Corney2015) due to a lack of comparative studies of the two. More specifically, there is a lack of an empirically grounded understanding of how designers experienced in product designing (which has been dominant in the manufacturing industry) would conceptually address PSS design problems. As a result, it is currently not possible to ascertain whether product designers need dedicated support for conceptual PSS designing and, if so, how to effectively support them. Therefore, in this article, we investigate and compare the cognitive characteristics of conceptual product and PSS designing by experienced product designers from the manufacturing industry.
1.1. Related work
Multiple factors, such as characteristics of design problems, design environment, use of design tools, type and level of expertise of the designers, and other conditions of the design setting, have been shown to influence the patterns of design cognition (Visser Reference Visser2009; Liu et al. Reference Liu, Li, Xiong, Cao and Yuan2018; Ball and Christensen Reference Ball and Christensen2019). Several works have reported differences in domain-specific design strategies (Adelson and Soloway Reference Adelson and Soloway1985; Schraagen Reference Schraagen1993; Akin Reference Akin2001), fixation patterns (Purcell and Gero Reference Purcell and Gero1996; Crilly Reference Crilly2015), and other dimensions across different design domains, cohorts of designers amongst other factors. These have been reviewed by Visser (Reference Visser2009), Dinar et al. (Reference Dinar, Shah, Cagan, Leifer, Linsey, Smith and Hernandez2015) and Ball and Christensen (Reference Ball and Christensen2019). Despite the availability of a rich understanding of the variant nature of design cognition, there is a lack of understanding of the influence of the complexity of design problems on the patterns of design cognition. More specifically, the relationship between the increasing complexity of societal design problems and the ability of designers to understand and address them has been minimally explored in the literature (Costa Junior et al. Reference Costa Junior, Diehl and Snelders2019).
PSS design problems are expected to have relatively more sources of complexity than similar product design problems (Machchhar et al. Reference Machchhar, Bertoni, Wall and Larsson2024; Mourtzis et al. Reference Mourtzis, Fotia, Boli and Pittaro2018 and several prescriptive methods and tools have been developed to support PSS designing (Qu et al. Reference Qu, Yu, Chen, Chu and Tian2016; Cong et al. Reference Cong, Chen, Zheng, Li and Wang2020; Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022). Although there are several macro-scaled descriptive case studies of PSS design and development practices (e.g., see Boucher et al. Reference Boucher, Coba and Lamy2024; Clayton et al. Reference Clayton, Backhouse and Dani2012; Morelli Reference Morelli2003) and reviews (Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022; Meier et al. Reference Meier, Roy and Seliger2010), there are limited replicable investigations of how designers conceptually address PSS design problems on the micro-scale. There is only a handful of protocol studies that have investigated the conceptual PSS design process: A few exploratory studies have introduced protocol analysis-based approaches to analyze the conceptual PSS design process (Shimomura et al. Reference Shimomura, Nemoto and Kimita2015; Sakao et al. Reference Sakao, Gero and Mizuyama2020; Neramballi and Sakao Reference Neramballi and Sakao2021). A study by Bertoni (Reference Bertoni2013) revealed differences in the patterns of PSS designing under the influence of two different design tools.
Other exploratory studies of PSS designing have investigated the influence of designer characteristics (Won Lee et al. Reference Won Lee, Lee, Jo and Kim2013) and the effects of an external prompt (Neramballi et al. Reference Neramballi, Sakao and Gero2019a) on the PSS design process. These protocol studies of the conceptual PSS design process had relatively small cohort sizes, composed mainly of student designers or a mix of experts and nonexperts. On the other hand, there is a substantial set of replicable studies that have investigated the cognitive processes of conceptually designing products or other similar material-based artifacts, as reviewed by Hay et al. (Reference Hay, Duffy, McTeague, Pidgeon, Vuletic and Grealy2017). Since there is a lack of empirically grounded insights into the cognitive characteristics of conceptual PSS designing, it is currently not possible to ascertain whether and how the cognitive characteristics of conceptual product and PSS designing are different (Sakao et al. Reference Sakao, Gero and Mizuyama2020.
1.2. Interplays of characteristics of design problems and design cognition
The conception of the understanding of the design problem can be referred to as a problem space, and the conception of possible solutions can be referred to as a solution space (Maher et al. Reference Maher, Poon and Boulanger1996; Dorst and Cross Reference Dorst and Cross2001; Kelly and Gero Reference Kelly and Gero2021). Characteristics of the design object may emerge as sources of dynamic and static complexities in the design problem and solution spaces for the designers. Dynamic complexity can be described in terms of the amount of information required to predict the potential temporal changes in the states of the object being designed with respect to the formulated design objectives (adapted from ElMaraghy et al. Reference ElMaraghy, ElMaraghy, Tomiyama and Monostori2012). This type of complexity can be linked to the conception of the problem space as it may emerge during the formulation of the functions and simulation of the expected behavior of the design objects (e.g., simulations of approaches to or scenarios of failure to transmit power from a combustion engine to the rotating load within an automobile and corresponding service repair activities or scenarios of failure of such activities).
The number of elements and the degree of interrelatedness of the elements of the objects being designed contribute to the static complexity of the design problem. Static complexity is the expected amount of information required to describe the state of a designed object (adapted from ElMaraghy et al. Reference ElMaraghy, ElMaraghy, Tomiyama and Monostori2012). This type of complexity of a design activity can be linked to the conception of the solution space. For instance, such complexity may emerge during the synthesis of the solution structures (e.g., torque converter of an automobile and maintenance schedule of torque converter) and the analysis of the derived behavior of structures (e.g., torque transmission and costs of maintenance).
1.3. Differences between product and PSS designing
Characteristic differences in product and PSS design objects are argued to lead to differences in the characteristics of their respective design processes as well. For instance, during functional modeling in conceptual PSS designing, both aspects of tangible products and intangible services (i.e., human processes) should be considered simultaneously in the problem space, as opposed to only tangible product-related aspects such as specifications of mechanics, material, energy, and information during conceptual product designing (Eisenbart et al. Reference Eisenbart, Gericke and Blessing2013). Simulation and analysis of the behaviors of product and PSS design objects in the respective problem and solution spaces tend to vary due to the latter’s relatively more human-centered aspects. More specifically, deviations in the temporal behavior of product elements are governed by physical laws (i.e., physical deterioration over time) (ElMaraghy et al. Reference ElMaraghy, ElMaraghy, Tomiyama and Monostori2012). In contrast, human user and service activities are heterogeneous (Regan Reference Regan1963), i.e., they are characterized by human behavior, which is more varied and thus difficult to predict (Cziko Reference Cziko1989; Alonso-Rasgado et al. Reference Alonso-Rasgado, Thompson and Elfström2004; Moreno et al. Reference Moreno, Hernández, Yang, Otto, Hölttä-Otto, Linsey, Wood and Linden2014).
Due to these characteristic differences between product and PSS design objects and the corresponding design processes, the respective design problems are expected to have different sources of complexity. More specifically, apart from the homogenous behavior of products, the prediction of the heterogenous behavior of users and services and the interrelated behaviors of the different system elements over an extended period (e.g., the use and end-of-life phase) emerge as additional sources of dynamic complexity in the PSS design problem space (based on (Pezzotta et al. Reference Pezzotta, Cavalieri and Gaiardelli2012; Kreye et al. Reference Kreye, Roehrich and Lewis2015; Lee et al. Reference Lee, Han and Park2015). In addition to the static complexity emerging from the product design object, a prototypical PSS design solution space has added sources of static complexity, such as the service counterparts, along with the consideration of the potential inter-relations between the structures of the product and service objects (based on (Kreye et al. Reference Kreye, Roehrich and Lewis2015; Mourtzis et al. Reference Mourtzis, Fotia, Boli and Pittaro2018). These characteristic differences in the sources of complexity of the two types of design objects are summarized in Table 1.
Table 1. Sources of static and dynamic complexities of product and product-service systems as conceptual design objects
Note: These sources of complexity are not comprehensive and are simplified. For more comprehensive lists of sources of complexity, see (ElMaraghy et al. Reference ElMaraghy, ElMaraghy, Tomiyama and Monostori2012; Mourtzis et al. Reference Mourtzis, Fotia, Boli and Pittaro2018; Machchhar et al. Reference Machchhar, Bertoni, Wall and Larsson2024).
2. Aim and hypotheses of this research
This study aims to empirically characterize and compare the design cognition of experienced product designers from the manufacturing industry in the conceptual designing of a product and a PSS (hereafter referred to as conceptual product and PSS designing, respectively). Here, design cognition is characterized in terms of design issues, design processes, and design strategies. The aim is operationalized by the following research question:
What are the differences in the cognitive characteristics of conceptual product and PSS designing by experienced product designers from the manufacturing industry?
Due to the presence of relatively more sources of dynamic complexity in the PSS design problem space than in the product design problem space (Section 1.3), the following hypothesis is formulated (in line with the findings of (Sakao et al. Reference Sakao, Gero and Mizuyama2020)):
H1. Conceptual PSS designing has more design issues and design processes in the problem space than conceptual product designing.
Designers are expected to explore a wider range of systemic functions during conceptual PSS designing (based on (Maussang et al. Reference Maussang, Zwolinski and Brissaud2009)). PSS functional modeling in the problem space entails the consideration of a systemic-value creation view involving a higher diversity of stakeholder views, product and service elements, and their interactions, rather than only focusing on product functions in the latter (Eisenbart et al. Reference Eisenbart, Gericke and Blessing2013; Kim Reference Kim2020; Brissaud et al. Reference Brissaud, Sakao, Riel and Erkoyuncu2022). This leads to the following hypothesis:
H2. Conceptual PSS designing has more of the design issue function than conceptual product designing.
Sakao et al. (Reference Sakao, Gero and Mizuyama2020) suggest that the focus on behavior is expected to be more dominant in PSS designing than product designing. Owing to the presence of relatively more sources of dynamic complexity (Section 1.3, Table 1), a stronger focus on simulating the expected system behavior in the problem space that can address the needed functional performance is also expected during PSS designing than during product designing. This leads to the third hypothesis:
H3. Conceptual PSS designing has more of the design process of formulation of function and expected behavior than conceptual product designing.
Sakao et al. (Reference Sakao, Gero and Mizuyama2020) also predict that the process of analysis of derived behavior is expected to be dominant in PSS designing, which is linked to the presence of more sources of static complexity in the PSS design solution space than the product design solution space (Sections 1.3, Table 1). This leads to the formulation of the following hypothesis:
H4. Conceptual PSS designing has more of the design process of analysis of derived behavior than conceptual product designing.
Designers were reported to employ a depth-first search approach to handle relatively more complex design requirements while opting for a breadth-first approach to handle design requirements with middle or low-level complexities (Ball et al. Reference Ball, Onarheim and Christensen2010). Since the PSS design problem has relatively more sources of static and dynamic complexities than the product design problem (Section 1.3, Table 1), designers may employ a more dominant depth-first-search approach while designing PSSs than while designing products. This leads to the generation of the final hypothesis:
H5. Conceptual PSS designing exhibits more depth-first search than conceptual product designing.
3. Research design
3.1. Description of the quasi-experiments and participants
To test these hypotheses, a comparative quasi-experimental study of two sets of design sessions was carried out in a laboratory environment with a sample size (N) of 56 experienced practitioners from different product manufacturing companies. A criterion-based purposive sampling technique was used to select the voluntary participants, and only the factors that have been previously reported in the literature to have influenced design cognition parameters are chosen to be controlled in this study for defining the sample, in line with Cash et al. (Reference Cash, Isaksson, Maier and Summers2022). For instance, factors such as type and length of domain-expertise of designers have been found to influence design cognition parameters in previous works (Dinar et al. Reference Dinar, Shah, Cagan, Leifer, Linsey, Smith and Hernandez2015). Thus, the participants were selected based on the following two criteria: (1) the participants should be working as practitioners in product manufacturing companies, and (2) the participants should have an average work experience of 10 years working in domains associated with product development.
The participants were nonrandomly assigned into two groups. Specifically, Group A was a cohort of 36 experienced practitioners from various product manufacturing companies based in the United States from sectors such as the aerospace and automotive industries. The participants had an average work experience in product development-related domains of 13.5 years (standard deviation: 4.29 years). This cohort included 10 females and 26 males. Group B was a cohort of 20 experienced practitioners also from various product manufacturing companies based in Sweden from sectors such as telecom, automotive, construction equipment, and healthcare consumable industries. They had an average work experience of 11.25 years (standard deviation: 5.99 years) in product development-related domains. This cohort included 8 females and 12 males. Although the country of practice or origin of the designers could potentially pose as a confounding variable, it was not controlled in this study primarily for practical reasons. This has also not been found to influence design cognition parameters in previous literature. Based on their experience, both cohorts are more knowledgeable in product design than PSS design. The design sessions were carried out after obtaining the informed consent of the participants, and no incentives were offered in return for their participation.
Group A (NA = 18 pairs of experienced practitioners) was treated to Condition A, where the design task was a prototypical product design problem that may occur in product manufacturing companies operating in a goods-dominant environment (Vargo and Lusch Reference Vargo and Lusch2008). All the participants in each of the 18 pairs of this quasi-experiment played the role of product designer. Group B (NB = 10 pairs of experienced practitioners) was treated to Condition B, in which the design task was a prototypical design problem that may occur in a product manufacturing company undergoing a transformation from a product-based business model to a PSS-based one. For condition B, one participant in each pair was suggested to play the role of a product designer and the other that of a service designer. However, they were not confined to those specific roles as they were free to discuss both product and service design issues throughout the sessions. Brief descriptions of the design tasks for both groups are provided in Table 2. An outline of the design briefs can be found in Appendix A, and full details can be found in (Lammi and Becker Reference Lammi and Becker2013; Neramballi et al. Reference Neramballi, Sakao and Gero2019b).
Table 2. Prototypical product and PSS design problems adopted in the quasi-experiments
Differences in the design object (type of the artefact: e.g., different types of home appliances) specific to the same domain could potentially pose as confounding variables. However, such differences have not been reported to influence design cognition in previous literature. The following measures were taken to constrain the effects of other factors that were reported in the literature to have influenced design cognition (Ball and Christensen Reference Ball and Christensen2019). A controlled laboratory setting was chosen over the practitioners’ real-world settings (partly based on (Cash et al. Reference Cash, Hicks and Culley2013; Ball and Christensen Reference Ball and Christensen2018) (i) to mitigate potential extraneous variables emerging from the highly variable natural settings of the practitioners that may influence design cognition, (ii) due to different design problems and work contexts of the participating practitioners and (iii) since product and service design processes tend to be both spatially and temporally separated due to organizational structures in practice (Matschewsky et al. Reference Matschewsky, Kambanou and Sakao2018). Participants of both sets of design sessions were not provided with any form of prescriptions, guidance, tools, or methods that could have posed as extraneous variables.
3.2. Description of data collection and analysis
Both groups of participants were asked to verbalize their thought processes – that is, to “think aloud” (van Someren et al. Reference van Someren, Barnard and Sandberg1994) during the process of designing. The average length of the sessions in both Conditions A and B was 45 min. Think-aloud protocol data were collected from the two sets of quasi-experiments in both audio and video formats (Figure 1).
Figure 1. Video frames of some design sessions.
All participants from Group A spoke in English. Most of the participants from Group B spoke in Swedish, and a few in English. The verbalizations were transcribed manually. Protocol data collected in Swedish were translated into English. Subsequently, the transcribed data were segmented based on the ontological segmentation approach outlined by Kan and Gero (Reference Kan and Gero2017). This segmentation approach entails the division of transcribed protocol data into the smallest possible units, known as segments, so that only one code from an ontologically based coding scheme can be assigned to each segment. Two suitable frameworks (Function–Behavior–Structure [FBS] ontology and hierarchical levels of systems abstraction [HSA]) that are applicable across different domains and conditions of design were chosen to analyze the protocol data. The two frames of analysis, the respective coding schemes, and the coding procedure are described below.
3.2.1. Function-behavior-structure ontology
The FBS ontology (Gero Reference Gero1990; Gero and Kannengiesser Reference Gero and Kannengiesser2004) is adopted in this study to model the patterns of design cognition specific to product and PSS design problems. It has been widely used in the past to model the design cognition of designers across a variety of domain-specific design problems, such as architecture, engineering, software, and systems design (Gero and Milovanovic Reference Gero and Milovanovic2020). This framework represents the different issues of the design object under three main classes of ontological variables: (1) Function (F): describes the teleology of the design object; (2) Behavior: describes the attributes that are derived from the structure of (Behavior of structure, Bs) or the ones that are expected to be derived from the design object (Expected behavior, Be); and (3) Structure (S): describes the components and their relationships of the design object.
This cognitive activity of designing is represented as a series of eight transformation processes linking the design issues at the different stages of design, which are described in detail in Gero (Reference Gero1990). Figure 2 presents the relationships between the different design issues and the fundamental processes linking them.
Figure 2. Function–Behavior–Structure ontology, design issues and processes, adapted from (Gero Reference Gero1990; Kannengiesser and Gero Reference Kannengiesser and Gero2015).
Each of the variables defined in the FBS ontology maps onto a code, as shown in Table 3. The coders used this ontological scheme to both segment and code the transcribed protocol data (see Appendix B for example excerpts). The transitions between these design issues across the coded segments linked to the eight transformational design processes of the FBS ontology were modeled as syntactic design processes. This framework has also been utilized to perform meta-level semantic analysis of the design activity in terms of the relative distribution of cognitive effort on Problem (P) and Solution (S) spaces (Dorst and Cross Reference Dorst and Cross2001) by using a measurement technique known as the P-S index (Jiang et al. Reference Jiang, Gero and Yen2014). This technique can be used to determine the relative cognitive focus on design issues and processes associated with the problem and solutions spaces. Design issues and processes associated with the P-S spaces are described in Table 3.
Table 3. Coding scheme based on the FBS design issues and the categorization of the issues in problem and solution spaces based on (Gero et al. Reference Gero, Jiang and Williams2013)
To test the four hypotheses (H1–H4) concerning the differences in the patterns of product and PSS design cognition, the procedure for the application of the FBS ontology to the PSS domain described in Sakao et al. (Reference Sakao, Gero and Mizuyama2020) is followed in this present study.
3.2.2. Levels of systems hierarchy
A framework to represent the designer’s navigation through the design task in terms of the different hierarchical levels of systems abstraction has been proposed (Gero and Mc Neill Reference Gero and Mc Neill1998) and is used here (Table 4). This framework has been used to investigate differences in the distributions of design issues on the different levels of systems abstraction across cohorts (Ho Reference Ho2001; Song et al. Reference Song, Becker, Gero, Deberard, Lawanto and Reeve2016; Neramballi et al. Reference Neramballi, Sakao, Gero and Gero2022).
Table 4. Coding scheme based on the hierarchical levels of systems abstraction
If designers move from a higher level to a lower level across segments, the transition is modeled as problem decomposition, while if they move from a lower level to a higher level, the transition is modeled as problem recomposition (Song et al. Reference Song, Becker, Gero, Deberard, Lawanto and Reeve2016). The coding scheme, based on hierarchical levels of systems abstraction and the inter-level transitions in terms of problem decomposition and recomposition, is shown in Figure 3. The coding scheme derived from this framework is utilized to investigate the search strategies employed by the designers during the respective design sessions.
Figure 3. Visualization of the framework for hierarchical levels of systems abstraction, the respective codes, and the transition between the codes; adapted from (Neramballi et al. Reference Neramballi, Sakao, Gero and Gero2022).
The search strategy in which subproblems or subsystems (Level 2) are more frequently focused is referred to as the breadth-first strategy, while the one in which elements are independently solved to a considerable level of detail (Level 3) is referred to as the depth-first strategy (based on (Ball and Ormerod Reference Ball and Ormerod1995; Liikkanen and Perttula Reference Liikkanen and Perttula2009). This framework is also independent of the design domain, expertise of the designers, or other design conditions. Therefore, it is deemed applicable to the cross-domain analysis of product and PSS design cognition. This framework is used to test Hypothesis 5.
3.2.3. Coding and data analysis procedure
The transcribed data were segmented until each segment could be assigned only one code from the coding scheme derived from the FBS ontology (first layer of codes). If a segment could not be associated with any of the FBS codes, it was coded as “O” (other) and was not considered for further analysis. An average of 765 segments (SD = 196) and 972 segments (SD = 360) were obtained from each of the product and PSS design sessions, respectively. Subsequently, the protocol data, coded and segmented with the first layer of codes, was coded again with the coding scheme derived from the HSA framework (second layer of codes). This entire process of segmentation and coding of the protocol data was carried out by two independent coders and a third expert arbitrator (Gero and Mc Neill Reference Gero and Mc Neill1998).
The final arbitration of both layers of codes was done by the entire coding team to ensure high levels of consistency and final arbitrated code reliability. Since the final arbitrated code is the one used in the results and is not the result of a random process, Cohen’s Kappa was not utilized to measure the reliability of the coding against the final arbitrated code. Rather, the reliability of the final code was measured in terms of the percentage of agreement between the individual codes of the independent coders and that of the final arbitrated version.
The average coding-arbitration (CA) reliability for the protocol data analyzed with coding schemes for FBS and levels of systems hierarchy was 81.37% (SD = 3.96) and 78.80% (SD = 6.90) for the product design sessions and 72.74% (SD = 3.05) and 73.08 (SD = 4.52) for the PSS design sessions, respectively. Although there is no widespread consensus regarding an acceptable intercoder agreement (ICA) ratio, a range of 61%–80% or 70% and over is widely considered to be substantial and acceptable (Landis and Koch Reference Landis and Koch1977; Campbell et al. Reference Campbell, Quincy, Osserman and Pedersen2013). Moreover, in this study, the CA reliability was considered over ICA, and a threshold of 70% CA agreement was deemed sufficient by the authors as it was an outcome of extensive arbitration involving an expert, which would limit the occurrence of “agreement by chance.” The FBS syntactic design processes that represent fundamental transitions of FBS design issues were modeled using LINKODER, a publicly available software application (linkoder.com). The transition between HSA codes across segments was modeled by determining the total occurrences of each type of transition of the HSA codes across the segments (e.g., 1–1, 1–2 … 3–3). Excerpts of coded data are presented in Appendix B.
4. Results
4.1. Distribution of cognitive focus on FBS design issues
The average distributions of the different FBS design issues (see Section 3.4.1) in product and PSS design sessions are presented in Figure 4.
Figure 4. Mean percent occurrences of FBS design issues between Conditions A and B. Note: The error bars indicate the standard deviations of the mean values.
The results given in Table 5 indicate statistically significant differences in the occurrences of design issues of Requirement, Function, Structure, and Design descriptions, with significant effect sizes. The results also indicate a statistical difference in the P-S issue index, with PSS designing having a higher index value.
Table 5. Results of independent t-tests (p < 0.05) of percent occurrences of FBS design issues between Conditions A and B, respectively
* Indicates statistical significance with p < 0.05.
The mean percent of the cumulative occurrence of FBS design issues across 20 windows for 10 sessions each of Conditions A and B are visualized in Figures 5 and 6, respectively, to provide a basis for a qualitative evaluation of the temporal differences between these two conditions. To normalize the data within the 10 sessions of the two conditions, window lengths that correspond to one-twentieth of their complete sessions were used. The graphs qualitatively visualize the distribution of the cumulative occurrence of design issues over time. Compared to product designing (Figure 5), in the graph depicting PSS designing (Figure 6), it can be qualitatively interpreted that the distribution of all the design issues is relatively uniform over time, except for Requirements and Design descriptions. Further comparing the two conditions, it can also be qualitatively interpreted that the designers’ focus on Structure and Behavior of structure in the product design sessions remains stable and dominant over time, while in Condition B, the focus fluctuates comparatively more between the design issues of Function, Expected behavior, Behavior of structure and Structure over time. For instance, designers in Condition A were primarily focusing on structural and solution-space based issues such as: “…so how do you--is there a way to make them? Because I know one double-hung windows in my house…So we’ve got to come up with some sort of pulley…So the double-hung windows…a window crank-type thing?” While designers in Condition B were consistently fluctuating between problem- and solution-space based issues such as: “Because there, and we could have that as an argument too. As a matter of cost. Because then we can charge a slightly higher price against our competitors…then we save resources. Yes, we must have that as a prerequisite for it to work. But otherwise in terms of resources it should probably be smart, because it is the user phase is most resource-demanding.”
Figure 5. Dynamic model of the mean cumulative occurrence of FBS design issues across 20 windows for 10 sessions of Condition A.
Figure 6. Dynamic model of the mean cumulative occurrence of FBS design issues across 20 windows for 10 sessions of Condition B.
4.2. Distribution of cognitive focus on FBS design processes
Average distributions across the eight design processes for the full lengths of both product and PSS design sessions are presented in terms of percentages of occurrences in Figure 7. The results of the product design sessions indicate that the highest focus is on Reformulation 1, with an average percent occurrence of 36.5% (SD = 5.98). The results of the PSS design sessions indicate that the highest focus is on Evaluation, with an average percent occurrence of 20.7% (SD = 6.37). Similar to the results from the analysis of design issues, product designing is more solution-focused, with an average P-S process index of 0.18 (SD = 0.04), and PSS designing is more problem-focused, with an average P-S process index of 0.50 (SD = 0.14).
Figure 7. Mean values of occurrences of FBS design processes in percent for Conditions A and B. Note: The error bars indicate the standard deviations of the mean values.
The differences in the average distributions of the design processes between product and PSS designing were analyzed using independent t-tests (p < 0.05) (Table 6).
Table 6. Results of independent t-tests (p < 0.05) of comparison of percent occurrences of FBS design processes and the P-S process indexes between Conditions A and B
* Indicates statistical significance with p < 0.05.
The results indicate that all eight design processes are statistically different (p < 0.05), with large effect sizes. The results statistically confirm that while Reformulation 1, Analysis, Reformulation 2 and Synthesis are more focused on product designing, Formulation, Reformulation 3, Evaluation and Documentation are more focused on PSS designing. This points towards the dominance of a solution-focused approach during product designing and that of a problem-focused approach during PSS designing. This observation is further supported by the statistically significant difference in the P-S process index, with PSS designing having a significantly higher index value than product designing with a large effect size. For instance, in Condition A, the designers were primarily engaging in solution space-based design processes such as structural reformulations and analysis: “I know a 404 slider, or a single-hung window will. Or double-hung window will meet it… You almost need, you need the force on all four corners to go up. Because if not they bind pretty easy‥” On the other hand, in Condition B, the designers were more consistently engaging in problem space-based formulations of functions and expected behaviors and reformulations of functions from structures: “…so I think it’s in the material in the machine which can weigh less or be lighter to bring forth…that you should think less materially there, or natural. Or its power consumption. Or it’s all three…but my experience that usually just falls between the seats, to investigate or to keep order for coffee…one, two, three times and it might destroy the machine, so maintenance?”
4.3. Distribution of design issue occurrences in levels of systems abstraction
The average distributions of the design issues in the different levels of systems abstraction for both product designing and PSS designing, respectively, are presented as follows. The occurrence of segments (in percentages) associated with the three hierarchical levels of systems abstraction for both groups is illustrated in Figure 8.
Figure 8. Mean values of occurrences of hierarchical levels of systems abstraction in percent between Conditions A and B. Note: The error bars indicate the standard deviations of the averages.
The results indicate that both groups have the highest design issue occurrences in the element level (Level 3), with Group A having an average of 54.2% (SD = 7.86) and Group B having an average of 57.0% (SD = 4.06). The lowest design issue occurrences are at the systems level (Level 1), with Group A having an average of 15.6% (SD = 6.58) and Group B having an average of 19.0% (SD = 5.72) of the total share of their respective design issue occurrences in their respective design sessions.
The results of the independent t-test (p < 0.05) for the design issue occurrences in hierarchical levels of systems abstraction by Groups A and B are presented in Table 7. A statistically significant difference was found between the design issue occurrences in the subsystems level (Level 2) by the two groups, with p = 0.003 and an effect size of 1.111. No statistically significant differences were found in the distribution of designers’ design issue occurrences in Level 1 and Level 3.
Table 7. Results of independent t-tests (p < 0.05) of comparison of the distribution of design issue occurrences in different hierarchical levels of systems abstraction between Conditions A and B, respectively
* Indicates statistical significance with p < 0.05.
4.4. Markov transitions of design issue occurrences in intra-level transition, problem decomposition, and recomposition
Temporal distributions of the designers’ cognitive focus on the processes of problem decomposition and recomposition were modeled by calculating the average values of Markov transition probabilities between the different hierarchical levels of systems abstraction (inter- and intra-level transitions) for both product and PSS design sessions.
The probabilities for both inter- and intra-level transitions for the product design sessions are presented in Figure 9. The results for the product design sessions indicate that the probability of intra-level transitions of designers’ design issue occurrences in the hierarchical levels of systems abstraction is higher than the inter-level transitions.
Figure 9. Markov model of intra-level and inter-level transitions for Condition A. Note: The thickness of the arrows representing inter- (problem decomposition and recomposition) and intra-level transitions are relative to the respective mean values of Markov transitions.
The probabilities for both inter- and intra-level Markov transitions for the PSS design sessions are depicted in Figure 10.
Figure 10. Markov model of intra-level and inter-level transitions for Condition B. Note: The thickness of the arrows representing inter- (problem decomposition and recomposition) and intra-level transitions are relative to the respective mean values of Markov transitions.
The results for the PSS design sessions indicate that the probability of intra-level transitions of the hierarchical levels of systems abstraction is also higher than the inter-level transitions.
Markov transitions within and between the hierarchical levels of systems abstraction were also analyzed using independent t-tests (p < 0.05) for the full length and the equal fractions of the first and second halves of all the product and PSS design sessions (Table 8). The results of the full-length sessions indicate that there are statistically significant differences for problem decomposition from Level 1 to Level 2 (p = 0.003), for problem recomposition from Level 3 to Level 2 (p = 0.0002), with product designing having higher values, and for intra-level transitions within Level 3 (p = 0.005), with PSS designing having higher values.
Table 8. Results of independent t-tests (p < 0.05) of Markov transitions of intra-hierarchical systems-level transitions, problem decomposition and recomposition (inter-hierarchical systems-level transitions) between Conditions A and B, respectively.
* Indicates statistical significance with p < 0.05.
The results of the first half of the design sessions suggest no statistically significant differences in the system-level transitions other than for problem recomposition from Level 3 to Level 2 (p = 0.002), with product designing having the higher value. The results of the second half of the design sessions suggest statistically significant differences for problem recomposition from Level 3 to Level 2 (p = 0.0008), with product designing having the higher value and intra-level transitions within Level 3 (p = 0.006), with PSS designing having the higher value.
5. Discussion and Conclusion
5.1. Hypothesis testing: Differences between product and PSS design cognition
The findings of the independent t-tests revealed that there are statistically significant differences in the distribution of design issues, processes, problem-solution spaces, subsystems hierarchical levels of systems abstraction and design search strategies between the two groups that addressed the product and PSS design problems. The results of the hypothesis testing are presented in Table 9.
Table 9. Inferential findings from Condition A (Product designing) versus Condition B (PSS designing) related to the hypotheses
5.2. Implications of this study
The experienced product designers were found to focus more on the problem space, Function as a design issue and Formulation as a design process during PSS designing than product designing, potentially due to the presence of more sources of dynamic complexities in the PSS design problem (Hypotheses 1–3 are supported). Sources of such dynamic complexities could include the need to predict and handle user behavior (e.g., frequency of use of the coffee machine and paper cups) and service heterogeneity (e.g., frequency of failure of coffee machines and maintenance activities) emerging throughout the PSS lifecycle. In line with these findings, several works within PSS design literature have prescribed support for designers to simulate the expected system behaviors, such as product performance and customer or user behaviors while formulating the functionalities of integrated product and service structures (Qu et al. Reference Qu, Yu, Chen, Chu and Tian2016; Trevisan and Brissaud Reference Trevisan and Brissaud2016; Cong et al. Reference Cong, Chen, Zheng, Li and Wang2020). Contrary to Hypothesis 4, the distribution of the solution space-based design process Analysis was found to be more dominant in product designing than PSS designing, despite the PSS design problem having more sources of static complexities. A possible explanation for this unexpected finding could be that the designers may have perceived a higher level of uncertainty in the PSS design solution space than in the product design solution space, as they are relatively less experienced in PSS designing than in product designing.
More specifically, designers tend to rely on their past knowledge in terms of memories and knowledge of their previous design experiences while addressing a problem at hand (Lawson Reference Lawson2004; Cash et al. Reference Cash, Gonçalves and Dorst2023). For instance, Kannengiesser and Gero (Reference Kannengiesser and Gero2012) suggest that to derive and analyze the behavior of synthesized structures, designers may require knowledge about the effects of the interactions of the design objects with each other and with the environment (e.g., the natural environment and social elements such as users). In the absence of such pre-existing knowledge, the designers may perceive uncertainty and seek to resolve it through information action by seeking and gathering relevant data (based on (Christensen and Ball Reference Christensen and Ball2016; Cash and Kreye Reference Cash and Kreye2017; Cash et al. Reference Cash, Gonçalves and Dorst2023). Thus, it could be inferred that to reduce uncertainty in the solution space during PSS designing, product designers may need dedicated design support that could provide them with information concerning the lifecycle behaviors of the product and service structures. This inference is corroborated by the empirical findings of extant literature that qualitatively revealed that experienced product designers were able to more comprehensively predict and analyze the behaviors of integrated product-service elements and the overall PSS with dedicated support (Kimita et al. Reference Kimita, Sakao and Shimomura2017, Reference Kimita, Brambila-Macias, Tillman and Sakao2021). Furthermore, several analytical techniques that utilize lifecycle data have been proposed in the state-of-the-art to support such design processes, especially in the context of smart PSS (Qu et al. Reference Qu, Yu, Chen, Chu and Tian2016; Cong et al. Reference Cong, Chen, Zheng, Li and Wang2020).
Statistically significant differences were found between product and PSS designing in terms of the focus on the subsystems level of systems abstraction, with the former exhibiting a higher focus. Product designing was also found to be more likely to engage in problem decomposition and recomposition to the subsystems level than PSS designing (Table 8). These results point towards the use of a more dominant breadth-first search approach during product designing than PSS designing. On the other hand, PSS designing was found to be more likely to engage in a depth-first search approach (Level 3 to Level 3, higher within element level Markov transitions across the full session, first and second halves of the session) than product designing with statistically significant differences (Hypothesis 5 is supported).
These results reveal the applicability of the findings of Ball et al. (Reference Ball, Onarheim and Christensen2010) to the case of product and PSS designing: they showed that designers tend to apply a depth-first search approach while addressing design tasks or requirements with higher complexity. Thus, based on the findings of this current study, it could be inferred that designers used a more dominant depth-first search approach during PSS designing than during product designing since the prototypical PSS design problem has relatively more sources of complexity than the product design problem (Table 1, Section 1.3). Literature suggests that design processes can be improved by using a breadth-first search strategy over a depth-first search strategy (Ball et al. Reference Ball, St.B.T. Evans, Dennis and Ormerod1997; Ho Reference Ho2001). Further, the literature suggests that a lack of examination of the interrelated behaviors of the subsystems (characteristic of the breadth-first search strategy) could lead to a critical failure of the systems being designed (Tomiyama et al. Reference Tomiyama, D’Amelio, Urbanic and Eimaraghy2007). Thus, it could be implied that product designers may need support to effectively carry out a more breadth-first search during PSS designing.
Overall, the findings of this research corroborate those of previous literature (Adelson and Soloway Reference Adelson and Soloway1985; Schraagen Reference Schraagen1993) that have shown that designers with experience in a specific design domain tend to reason differently when confronted with a design problem specific to a relatively less familiar domain. While previous research has reported differences in the search strategies (problem decomposition and recomposition) among experts and novices (Ho Reference Ho2001; Song et al. Reference Song, Becker, Gero, Deberard, Lawanto and Reeve2016), the findings of this current study contribute to an improved understanding of differences in such search strategies used by experienced practitioners while addressing two different prototypical design problems with differing sources of complexity. Overall, these findings corroborate the notion that the characteristics of the design problem may influence design reasoning (Adelson and Soloway Reference Adelson and Soloway1985; Ullman et al. Reference Ullman, Dietterich and Stauffer1988; Schraagen Reference Schraagen1993) and design cognition (Visser Reference Visser2009). However, it is important to note that further research is needed to investigate the generalizability of these claims and that there may be other variables underlying these differences.
5.3. Limitations of this study
The findings of this study are limited as follows: First, although the cohort sizes of the two groups are sufficient to statistically test the hypotheses, the internal validity of the findings can be further enhanced with larger cohort sizes. Second, although the characteristics of the participants of the quasi-experiments have been constrained in terms of their level of experience working within different product manufacturing companies in domains associated with product development, their functions and type of work experiences within their respective organizations may vary. Further, other uncontrolled variables such as nonrandom assignment of the participants to the two conditions of the study (e.g., in terms of the country of practice or origin of the designers) and differences in the types of products given in the two design tasks may also have confounding effects on the results. There may be other differences in the conditions and among the participants of the study, which are difficult to control without randomized controlled trials. Such factors are widely acknowledged limitations of protocol and quasi-experimental studies in design research (Dinar et al. Reference Dinar, Shah, Cagan, Leifer, Linsey, Smith and Hernandez2015; Ball and Christensen Reference Ball and Christensen2019).
Although the treatment conditions for the two groups have been curated to represent prototypical product and PSS design problems, differences in the characteristics of the respective real-world design problems may pose as confounding variables, potentially restricting the external validity. Further, the study’s external validity is also limited by the choice of laboratory setting over that of an ecologically valid setting (Ball and Christensen Reference Ball and Christensen2018; Cash et al. Reference Cash, Hicks and Culley2013). However, these limitations are characteristic of most protocol studies and are considered here as a necessary trade-off to be able to perform statistical analysis.
5.4. Conclusion
In this work, we set out to ascertain whether and how product and PSS designing by experienced product designers are different from a design cognition perspective. Based on a theoretical conceptualization of the effects of the differences in the sources of complexity between prototypical products and PSS domain-specific design problems on design cognition, we hypothesized that conceptual PSS designing (1) is more problem focused, (2) focuses more on Function as a design issue, (3) focuses more on Formulation as a design process, (4) focuses more on Analysis as a design process and (5) utilizes a depth-first search strategy than product designing. The empirical findings of this study support Hypotheses 1, 2, 3, and 5, suggesting that the differences in the sources of the complexity of the design object specific to product and PSS design problems may contribute to differences in the distribution of cognitive effort on design issues, processes, the Problem-Solution spaces and search strategies. The findings, however, did not support the fourth hypothesis but indicated that designers focus more on Analysis of Derived Behavior of Structure during product designing than PSS designing. In conclusion, these findings contribute to an improved empirical understanding of the variant nature of design cognition while also providing an empirically grounded basis to develop effective design support catering specifically to PSS design cognition of experienced product designers.
5.5. Future research directions
A number of promising future works are envisioned based on the discussion above. First, increasing the cohort size of the design sessions analyzed is necessary for further knowledge generalization. Second, investigating the influence of various characteristics of PSS designers, such as a comparison between expert and novice PSS designers, will potentially yield results that may influence design education and practice. Third, examining the effects of PSS design methods and tools on PSS design processes would fill a gap in our knowledge of the efficacy of design support tools for PSS designers.
Acknowledgements
This research is supported in part by the Mistra REES (Resource Efficient and Effective Solutions) program, funded by Mistra (The Swedish Foundation for Strategic Environmental Research) (grant number DIA 2014/16). This research was supported in part by a US National Science Foundation grant to J. S. G., grant number 1762415.
Appendix A – Design briefs
This appendix presents the detailed design briefs used for both the product and PSS design sessions.
Product design brief – Condition A
Double-Hung (Sash) Window Opener.
Your design team has been approached by Warm Heart Estates, a local nursing home, to design a new product to assist its elderly residents.
The nursing home administrators have noticed that changes in humidity during the summer months cause the windows of the 65-year-old building to “stick,” thus requiring significant amounts of force to raise and lower the windowpanes. The force required to adjust the windows is often much too large for the nursing home tenants, making it very difficult for them to regulate their room temperature.
Your team has been tasked with designing a device that will assist the elderly tenants with raising and lowering the building’s windows. Since each window is not guaranteed to be located near an electrical socket, this device should not rely on electric power.
Your team has identified the following websites as potential sources of useful information:
“Double Hung Window Construction.”
http://www.oldhouseweb.com/how-to-advice/double-hung-window-construction.shtml
“Double Hung Windows – Everything You Need to Know” (1 min. 34 sec.):
http://www.youtube.com/watch?v=xW7OMHYI4kY
American Disabilities Act (ADA) information:
ADA Accessibility Guidelines for Buildings and Facilities (ADAAG):
http://www.access-board.gov/adaag/html/adaag.htm
This design brief has been used in previous studies of product design (Gero Reference Gero2010; Lammi and Becker Reference Lammi and Becker2013).
Product-Service System Design Brief – Condition B
Coffee machine and related services.
The design is carried out for the company that develops, manufactures, and delivers coffee machines (Figure 11) and related services. This hypothetical firm is named Jobbkaffe and is based in Sweden. Instructing the use, installing the machines, supplying consumables, and carrying out MRO (maintenance, repair, and overhaul) are all parts of the company’s service portfolio. A University, whose employees are mainly professors, PhD students, and administrative staff, is the client of Jobbkaffe. The employees and their guests want to get something warm to drink, typically early in the morning as well as during a morning break and an afternoon break. The design object addressed is one of their major offerings including both product and service.
Figure 11. Photo of the product.
Each group is demanded to derive a concept with the highest potential for the offering that improves resource efficiency. The deliverable is a concept for the offering containing products and services described on a blank paper. The concept should be derived from the group discussion, including choices and reasons for the developed concept. The concept needs to have sufficient information before the detailed designFootnote 1 begins. The improvement can be all on the products, the services, and the payment model, but it could be on one or two of them.
Reference
The product model: http://www.Jobmeal.se/sv/automater/kaffeautomater/p/psl-50btc
This design brief has been used in previous studies of PSS design (Neramballi et al. Reference Neramballi, Sakao and Gero2019a; Neramballi et al. Reference Neramballi, Sakao and Gero2019b; Neramballi et al. Reference Neramballi, Sakao, Gero and Gero2022).
Appendix B – Excerpts of coded protocols
This appendix presents some excerpts of segmented and arbitrated protocol data retrieved from one product design session, Table B.1, and one PSS design session each, Table B.2.
Table B.1 Excerpt of segmented and arbitrated protocol data using the coding schemes for Function–Behavior–Structure (FBS) ontology and Hierarchical levels of Systems Abstraction (HSA) retrieved from a product design session
Table B.2 Excerpt of segmented and arbitrated protocol data using the coding schemes for Function-Behavior-Structure (FBS) ontology and Hierarchical levels of Systems Abstraction (HSA) retrieved from a PSS design session
Open access
