1 Introduction
During the last decades, both academia and industry have developed practices, methods, and tools for decision support centered on technical and engineering quantifiable aspects, such as product performance, robustness, and producibility. The use of model-based decision support systems is common in engineering design environments, especially in those embedding a high level of complexity, such as the aerospace, automobile, and naval industry (Wierzbicki, Makowski & Wessels Reference Wierzbicki, Makowski and Wessels2000). The exploration of the design space is often enabled by methods and tools based on well-established statistical and mathematical analysis, whose validity is independent of the context of the application (Dépincé, Guédas & Picard Reference Dépincé, Guédas and Picard2007; Tedford & Martins Reference Tedford and Martins2010). The application of multiple simulation techniques allows engineers to systematically narrow down the design space by eliminating undesirable solutions (Malak, Aughenbaugh & Paredis Reference Malak, Aughenbaugh and Paredis2009). Such an approach is commonly described in literature with the term set-based concurrent engineering (Sobek, Ward & Liker Reference Sobek, Ward and Liker1999).
Recently, researchers in the area of value-driven design (VDD) and sustainable product development (SPD) have recognized the need to include models for value and sustainability assessment in early design concept evaluation in order to expand the design space exploration to more than product feasibility and technical performances (Ross et al. Reference Ross, Hastings, Warmkessel and Diller2004; Steiner & Harmon Reference Steiner and Harmon2009; Bertoni, Hallstedt & Isaksson Reference Bertoni, Hallstedt and Isaksson2015b; Bertoni et al. Reference Bertoni, Bertoni, Panarotto, Johansson and Larsson2016; Hallstedt Reference Hallstedt2017).
The term ‘value’ is nowadays increasingly used to refer to a large and heterogeneous set of needs from multiple stakeholders (Bertoni, Bertoni & Isaksson Reference Bertoni, Bertoni and Isaksson2013; Matschewsky, Lindahl & Sakao Reference Matschewsky, Lindahl and Sakao2018). VDD has become an umbrella term that collects several methodologies using value models to balance performance, cost, schedule, and other measures important to the stakeholders to produce the best possible outcome. The spirit of VDD is to open the solution space for consideration by designers, systems engineers, program managers, and customers by promoting quick what-if analyses that use a value function as metric to judge the goodness of a design (Collopy & Hollingsworth Reference Collopy and Hollingsworth2011).
Integrating the breadth of sustainability into product development is labeled sustainable product development or sustainable design (Gagnon, Leduc & Savard Reference Gagnon, Leduc and Savard2012). The concept of SPD refers to a strategic sustainability perspective that is integrated and implemented into the early phases of the product innovation process, including life-cycle thinking (Hallstedt & Isaksson Reference Hallstedt and Isaksson2017). The term sustainability refers to the definition of socio-ecological sustainability using overarching sustainability principles at the basis of a backcasting perspective. These principles are the key parts in the Framework for Strategic Sustainable Development presented in (Broman & Robèrt Reference Broman and Robèrt2017). Sustainability implementation refers to the practical usage and application of tools, methods, processes, approaches, and practices that aim to improve an organization’s contribution to sustainable development and provide opportunities for enhancing product competitiveness (Chiu & Chu Reference Chiu and Chu2012; Choi, Nies & Ramani Reference Choi, Nies and Ramani2008).
In early design space exploration, it is a challenge to model the link between the mechanical performance of multiple design variants, the value generated for the stakeholders, and the derived sustainability implications. Commonly, engineers strive to work with requirements that are clear, concise, and unambiguous to translate the original design intent (Monceaux & Kossmann Reference Monceaux and Kossmann2012; Isaksson et al. Reference Isaksson, Kossmann, Bertoni, Eres, Monceaux, Bertoni, Wiseall and Zhang2013). However, it is difficult to explicitly formalize value and sustainability into transparent and quantifiable terms. Poor availability and high uncertainty of data are commonly recognized as critical issues when using VDD and SPD models. Value and sustainability implications are often tacitly and subjectively perceived by engineers and rarely populate any kind of computational model (Soban, Price & Hollingsworth Reference Soban, Price and Hollingsworth2012; Isaksson et al. Reference Isaksson, Bertoni, Hallstedt and Lavesson2015; Bertoni, Bertoni & Isaksson Reference Bertoni, Bertoni and Isaksson2018a).
Concurrently, the evolution of data science and information communication technologies has opened the possibility of collecting, analyzing, and using data in different decision-making contexts. Data science is applied in a variety of contexts, such as remote monitoring, failure prediction, preventive maintenance, and fleet management (Murthy, Atrens & Eccleston Reference Murthy, Atrens and Eccleston2002; Painter et al. Reference Painter, Erraguntla, Hogg and Beachkofski2006; Tango & Botta Reference Tango and Botta2013). In these fields, the application of machine learning (ML) (Mitchell Reference Mitchell1997) and data mining (Fayyad, Piatetsky-Shapiro & Smyth Reference Fayyad, Piatetsky-Shapiro and Smyth1996) allows engineers and analysts to create, manage, correlate, and forecast a large amount of data with a relatively low effort with respect to time and resources (e.g., Akhavian & Behzadan Reference Akhavian and Behzadan2013; Pouliezos & Stavrakakis Reference Pouliezos and Stavrakakis2013). In aerospace product development, ML has been used to approximate the results of expensive simulations by creating surrogate models. Research studies have proven the effectiveness of the use of ML to reduce the computational cost for design optimization, design space exploration, and sensitivity analysis. For instance, Huang et al. (Reference Huang, Wang, Chen and Tian2011) have built a model that approximates the computational mechanical analysis of engine components to reduce the computational cost. Jeong, Chiba & Obayashi (Reference Jeong, Chiba and Obayashi2005) have used data mining approaches for aerodynamic design space to identify the effect of design parameters on design objectives. ML has been also used to explore the design space and to identify the rationale of the improved performance of an optimal solution (Jeong et al. Reference Jeong, Chiba and Obayashi2005; Mack et al. Reference Mack, Goel, Shyy, Haftka, Yang, Ong and Jin2007). Similarly, experimental investigations have been performed using linear regression, support vector machine (SVM), and tree models (Dasari et al. Reference Dasari, Lavesson, Andersson and Persson2015). Besides these initial attempts, the application of ML to early design space exploration is still in its infancy. Limited research has been done in understanding how to derive design indications for new product development, and the challenge of how to support the assessment of value and sustainability remains largely unexplored.
The aim of the research presented in this paper is to enable a more complete and effective exploration of the design space by developing an approach integrating the assessment of value and sustainability in a decision support environment. The use of ML to deal with the heterogeneity of data of VDD and SPD models is proposed as an enabler of wider and faster evaluations of the design space. The approach was developed in collaboration with a first-tier aerospace subsystem manufacturer and applied in the case of the development of a component for a commercial aircraft engine.
The paper is structured as follows: the next section describes the research approach. Section 3 discusses the challenges of assessing value and sustainability in aerospace product development. Section 4 presents the design support requirements derived from the empirical study, the role of ML as a technology enabler, and describes in detail all the steps of the proposed approach. Section 5 presents the application of the approach in the case of the development of a turbine rear structure (TRS) for a commercial aircraft engine. Section 6 discusses the findings in relation to the actual theory and practice and Section 7 draws the final conclusions highlighting the next steps to be taken in research.
2 Research approach
The research presented is based on a participatory action research (PAR) (Whyte, Greenwood & Lazes Reference Whyte, Greenwood and Lazes1989) approach applied in the frame of the Design Research Methodology (DRM) (Blessing & Chakrabarti Reference Blessing and Chakrabarti2009). The aim of PAR is to solve practical problems that also have theoretical implications by directly involving researchers and practitioners in the research design. A PAR approach involves cycles of actions where researchers plan an action, act in relation to the plan, observe the effects, and reflect on the observations, ultimately leading to a new plan or solution. PAR has been criticized in literature for the limitation in researchers’ independence because of the biases introduced by the direct collaboration between researchers and practitioners and has also been criticized for lacking academic standards (McNiff Reference McNiff2014). This exposes the approach to the risk of lacking rigor and technical validity. To mitigate such a risk, the application of PAR has been steered by the DRM proposed by (Blessing & Chakrabarti Reference Blessing and Chakrabarti2009). The DRM consists of four nonlinear stages encompassing a research clarification stage, the definition of an AS-IS model of a specific design situation (Descriptive Study I), the design and development of the desired design support (prescriptive study), and the evaluation of the effectiveness of such support (Descriptive Study II). The difference between PAR and the DRM consists in the fact that PAR draws conclusions about a specific support in a specific situation, aiming for a continuous improvement until an optimized implementation is achieved, while the DRM stresses the importance of the validation of a design support in terms of generic statements of partial implementation. The two approaches can mutually nurture each other, with the PAR cycles that mostly concern the prescriptive study and its validation, and the DRM that stresses the rigor in the problem definition and in the generalization of the research statements (Blessing & Chakrabarti Reference Blessing and Chakrabarti2009).
During the research, the focus of the investigation was initially divided into three streams, the first related to VDD, the second related to SPD, and the third related to the use of ML in early aerospace development. The division of the topics of investigation eased the identification of relevant stakeholders at the partner company that could easily relate the topic of investigation to their own working responsibilities, those involved, respectively, systems engineering practitioners, sustainability experts, and data science experts. The coordination of the activities and the sharing of the intermediate results happened through bi-weekly research team meetings.
The descriptive study was based on the data collected from a five-year research program encompassing multiple project studies and involving several researchers in the VDD, SPD, and ML domain. Data were collected through the constant interaction between researchers and practitioners facilitated by focus groups, workshops, and semi-structured interviews, complemented by internal document reading and literature review. The semi-structured interviews were organized around a set of predetermined open-ended questions, with additional questions emerging from the dialog. The choice of this form of interviews, instead of using pre-defined specific questions, served the purpose of collecting information from individuals with different roles in the company giving the freedom to the participants to raise emerging issues and challenges, individually perceived as important but not identified in the interview protocol (see DiCicco-Bloom & Crabtree Reference DiCicco-Bloom and Crabtree2006). Some of the intermediate results from the descriptive study were formalized in a scientific publication in 2015 (Isaksson et al. Reference Isaksson, Bertoni, Hallstedt and Lavesson2015), proposing an overall framework for the development of model-based decision support for value and sustainability. The formalization of such a framework was functional to the definition of the high-level requirements of the approach proposed in this paper and served as a boundary object around which the researchers from VDD, SPD, and ML domains have coordinated the subsequent research work. A consistent part of the descriptive study took place in parallel with the prescriptive study and focused on collecting data about the case of the development of a hot-structure component for a commercial aircraft engine component. With such focus, interviews took place to narrow the investigation into aspects that would have otherwise been difficult to capture through informal conversations. Focus groups were organized to complement the findings of the interviews by capitalizing ‘on communication between the research participants in order to generate data’ (Kitzinger Reference Kitzinger1995, p. 299). The enrollment to the focus groups was managed by direct invitation to the participants issued by the research team, with the recommendation to extend the invitation to any other company employee who could have an interest in the topic. The results from the focus groups were collected either by summarizing the produced material or by taking pictures. Field notes were occasionally collected when taking part in project meetings at the company facilities in the role of either project participant or observer. In such cases, the researchers had the chance to capture the context and the setting in which the problem was discussed and to record behaviors and reactions. Internal company documents and publicly available information (including academic theses, presentations, and reports) describing the existing engineering challenges related to the development of hot-structure components were also analyzed for a triangulation purpose.
In the prescriptive study, the development of the proposed approach focused on the case of the design of a TRS. Different versions of the proposed approach were developed through a series of cycles in which different prototypes were defined and presented to practitioners to obtain feedback and evaluations. They served to incrementally improve the approach and converge toward the final solution. The feedback was collected both during bi-weekly project meetings between academic researchers and industrial practitioners and during ad hoc organized seminars involving the practitioners directly impacted by the possible introduction of the new approach. The data populating the models of the case study were partially obtained as a result of computer-aided engineering (CAE) simulations on real design cases and partially complemented with realistic, but artificial, data to avoid issues of industrial secrecy.
During Descriptive Study II, the computational capabilities of the approach were tested. A partial test of the applicability of the approach was performed in relation to the usability of the proposed approach by engineers, and the delivery of the desired performances, corresponding to the application evaluation stage of the DRM. The latter evaluation did not encompass the integration of ML. Both the evaluations did not happen sequentially, but they were part of the continuous improvement activities during the development of the support. Figure 1 provides a visual representation of the focus, deliverables, and methods used for data collection in the different stages of the research.
Figure 1. Visual representation of the research approach and data collection methods used in the frame of the Design Research Methodology.
3 Value and sustainability challenges in aerospace product development
Aerospace product development is a long process involving companies at different levels of the supply chain, and it is commonly steered by set-based concurrent engineering (Sobek et al. Reference Sobek, Ward and Liker1999). The process starts with a set of assumptions and requirements that mature and are adjusted as time goes by and decisions are made on specific product features. The long lead time creates a challenging situation for subsystem and component manufacturers. They need to start the development process long before requirements are signed. They need to deal with requirements uncertainty while being constantly pushed to increase design robustness, reduce weight and cost, and improve product performances. Normally, multidisciplinary design optimization is used early in the process to consider an open set of feasible design solutions, rather than focusing on specific solution points in the design space. This is to enable engineers to understand the implications of decisions made in early design stages, considering products featuring several decades of operations and production periods of about 20 years.
The VDD research field has its origin in aerospace product development and it is based on the idea of optimizing a system toward its best value, rather than toward the fulfillment of requirements, by proposing an innovative process to either replace or complement traditional design methods (Collopy & Hollingsworth Reference Collopy and Hollingsworth2011; Soban et al. Reference Soban, Price and Hollingsworth2012). VDD methods are based on the use of the so-called ‘value functions’ to drive the multidisciplinary optimization of a design. Authors have argued about the need for having a single value function or a combination of multiple value functions to use for design concept trade-off (Soban et al. Reference Soban, Price and Hollingsworth2012). The use of value functions has been often linked to the need to monetary quantify the profits, or losses, linked to a specific design solution (e.g., Castagne, Curran & Collopy Reference Castagne, Curran and Collopy2009; Cheung et al. Reference Cheung, Scanlan, Wong, Forrester, Eres, Collopy, Hollingsworth, Wiseall and Briceno2012). In these cases, the value function took the form of a net present value assessment or of surplus-value calculation (e.g., Price et al. Reference Price, Soban, Mullan, Butterfield and Murphy2012; Selva & Crawley Reference Selva and Crawley2013). In other cases, VDD models featured a qualitative nature, aiming to increase decision-makers’ awareness during multidisciplinary trade-off analysis, rather than providing optimization results (e.g., Ross et al. Reference Ross, Hastings, Warmkessel and Diller2004; McManus et al. Reference McManus, Richards, Ross and Hastings2007; Bertoni et al. Reference Bertoni, Bertoni and Isaksson2018a).
SPD is based on the idea that the product development team must know what sustainability means, how sustainability can be achieved, and how sustainability can be measured to reach more sustainable solutions (Arena et al. Reference Arena, Ciceri, Terzi, Bengo, Azzone and Garetti2009). Research in SPD has combined a forecasting approach with a backcasting approach. Backcasting means imagining success in the future and then looking back to the present to assess the current situation through the lens of this success definition, ultimately exploring ways to reach that success (Vergragt & Quist Reference Vergragt and Quist2011; Quist, Thissen & Vergragt Reference Quist, Thissen and Vergragt2011). This includes an understanding of how a design solution influences social and environmental sustainability from raw material acquisition to disposal phase (Joung et al. Reference Joung, Carrell, Sarkar and Feng2013; Hallstedt Reference Hallstedt2017) and how to define the most prioritized sustainability criteria from a backcasting perspective. This led to the formulation of the so-called sustainability design space (SDS) (Hallstedt Reference Hallstedt2017), including the definition of criteria and indicators to support SPD. A criterion is defined as a target of a prioritized aspect or as the level of the aspect that we strive for (e.g., ‘no raw material used’ and ‘no hazardous chemicals used’). An indicator is defined as a measurement (qualitative or quantitative) that can indicate the state or level of the related criterion (e.g., ‘material used in total and per unit of product’ and ‘kilograms of persistent bio-accumulative and toxic chemicals used’) in line with the definitions presented in Renn et al. (Reference Renn, Jäger, Deuschle and Weimer-Jehle2009). The SDS as described by Hallstedt (Reference Hallstedt2017) consists of three parts. The first part is the strategic sustainability criteria based on overarching sustainability principles at the basis of a backcasting perspective (Broman & Robèrt Reference Broman and Robèrt2017). This defines the ideal long-term sustainability targets for each product life-cycle phase. The second part consists of the tactical sustainability design guideline to support the development toward the related long-term strategic sustainability criteria, including current and short-term, industry-specific and company-specific requirements and expectations. The third part is a qualitative measurement scale, called sustainability compliance index (SCI), linked to each of the strategic criteria to assess to what degree a product concept performs in relation to a sustainable solution. All three parts of the SDS have been applied separately and used in different combinations with other support tools in case studies in aerospace product development, e.g., when integrating sustainability in the product innovation process through a technology readiness assessment method (Hallstedt & Pigosso Reference Hallstedt and Pigosso2017) or when assessing critical alloys in the early design stages (Hallstedt & Isaksson Reference Hallstedt and Isaksson2017). Material selection is one example of a design feature that needs to be decided early in the innovation process and can be linked to sustainability risk as it has a direct impact on upstream decisions (e.g., extraction activities in rural areas) and downstream decisions (e.g., the selection of manufacturing processes and end-of-life solutions) (Giudice, La Rosa & Risitano Reference Giudice, La Rosa and Risitano2005), and thereby has an impact on business success in the long run. Sustainability risk is defined as ‘threats and opportunities that are due to an organization’s contribution or counteraction to society’s transition toward strategic sustainable development’ (Schulte & Hallstedt Reference Schulte and Hallstedt2018, p. 11).
The high heterogeneity, poor maturity, and scarce availability of data are the common denominators when combining VDD and SPD models for design space exploration. Despite the examples of applications of VDD and SPD models in different aerospace development stages, little effort has been spent in integrating VDD and SPD models into design space exploration, combining them with the numerical results of the product structural simulations. Three challenges make the use of computational models for value and sustainability particularly cumbersome. First, the availability of first-hand data is limited because the data that populate VDD and SPD models reside not only within the product definition but also within the product life cycle and usage environment (Gautam & Singh Reference Gautam and Singh2008; Curran Reference Curran2010; Hallstedt, Thompson & Lindahl Reference Hallstedt, Thompson and Lindahl2013). Second, the relations between the design variables that have an impact on value and sustainability are difficult to identify (Bertoni et al. Reference Bertoni, Bertoni and Isaksson2018a; Watz & Hallstedt Reference Watz and Hallstedt2018). Third, value and sustainability models need to be coupled with effective approaches to communicate the results to engineers, who may not be accustomed to this type of information (Bertoni et al. Reference Bertoni, Bertoni and Isaksson2013). Such communication shall be done in a form as close as possible to the ‘natural thinking pattern’ of engineers, which is a key factor for the acceptance of a new formal method (López-Mesa & Bylund Reference López-Mesa and Bylund2011). The consequence of not addressing these challenges in current decision situations is the weakness in clarifying and understanding the value and sustainability implications compared to, for instance, the mechanical, thermal, or fluid dynamic performance of an engine. In other words, engineers have poor model support to answer questions like: ‘Which is the most valuable component to develop?’, ‘What would its sustainability profile look like?’, and ‘What is the sustainability impact of the product during its life cycle and how does this affect stakeholders in the value chain?’ Such questions would need to be answered before committing high resources on a development project; however, computational models to be used in the design space exploration are nowadays missing.
4 Value and sustainability assessment in design space exploration enabled by machine learning – the proposed approach
Based on the challenges described in Section 3, a list of ‘design support requirements’ to guide the development of the proposed approach was formulated. In accordance with the DRM, those requirements were intended as high-level indications for the definition of the ideal design support. Three aspects were identified as relevant to effectively develop and implement the approach. First, the proposed approach needs to be able to deal with data of different natures and levels of granularity. Second, the integration of the model in the current working practices needs to be as smooth as possible to overcome the resistance to changes of an established development context and to not prolong the computational time. Third, the reliability of the models and data used in the approach need to be evident.
Based on such reflections, a list of design support requirements was defined as follows.
(i) The support shall aggregate into a unique model the results of both VDD and SPD models despite their multidimensionality.
(ii) The support shall be able to deal with both qualitative and quantitative data.
(iii) The support shall be easily integrated into the current working practices at the partner company.
(iv) The support shall provide results that are easy to read and trade off with traditional structural analysis.
(v) The support shall present the results in terms of seconds for hundreds of potential design concepts.
(vi) The support shall be able to quantify the uncertainties and assumptions related to the use of VDD and SPD models.
(vii) The support shall not provide a false impression of accuracy in the results when accuracy is not present.
In order to deal with such requirements, the use of ML emerged as a technological enabler to support value and sustainability assessment, as further described in Section 4.1.
4.1 Machine learning as a technological enabler
ML comes into play as a possible technique to support the prediction of value and sustainability performances. ML enables the identification of hidden correlations on extensive sets of multidisciplinary and multidimensional data (encompassing both categorical and numerical variables), discovering correlations otherwise difficult to be found with traditional statistical analysis techniques. The use of ML algorithms is identified as an opportunity to lower the uncertainty of the decision-making by populating models with data-driven information rather than experience-driven assumptions. The incompleteness and low maturity of the data in early design space exploration introduce large uncertainties in VDD and SPD models. Engineers’ experience and intuition are often the main decision drivers since data to populate VDD and SPD are missing. Historical databases are the key source of data for such applications and, despite being based on data collected from previous products, they can provide important information for engineers to reduce the uncertainty of their assumptions. ML emerged in the study as an approach to analyze the data related to customer revenue, maintenance cost, and manufacturing costs and to explore the correlation with more qualitative value aspects. Similarly, sustainability models could be built by analyzing the actual sustainability performance, looking in retrospect to the correlation with the engineering configurations of the products already on the market.
The use of ML was further identified as useful in allowing the creation of predictive models approximating the results of those areas of the design space where limited data is available. This approximation analysis mimics the complex behavior of the underlying simulation analysis and provides a great opportunity for engineers to explore many design variations without the need to set up computationally expensive simulations. This is commonly referred to as the development of approximation models, also known as response surface models, surrogate models, and meta-models (Mack et al. Reference Mack, Goel, Shyy, Haftka, Yang, Ong and Jin2007). Statistical methods, such as Kriging and polynomial methods, were identified as relevant to construct surrogate models. Furthermore, the literature review showed that ML methods, such as support vector machines, tree-based models, artificial neural networks, and radial basis functions, have been successfully used to construct response models in aerospace product development (e.g., Queipo et al. Reference Queipo, Haftka, Shyy, Goel, Vaidyanathan and Tucker2005; Shan & Wang Reference Shan and Wang2010; Dasari et al. Reference Dasari, Lavesson, Andersson and Persson2015). In general, surrogate models deal with quantitative data; however, in the proposed approach, surrogate models need to deal with both quantitative and qualitative data, including value and sustainability data which are not numerically quantifiable. Hence, a suitable method needs to be selected to construct surrogate models. While linear regression or polynomial methods cannot be used for the purpose, a tree-based method is more suitable to handle both quantitative and qualitative data and is also capable of fitting nonlinear and high dimensional data.
4.2 Overview of the proposed approach
The proposed approach consists of an extension of the design space exploration process commonly in use during the early stages of product development. Figure 2 provides an overview of the logic of the approach showing the role of VDD and SPD models in relation to the CAE simulation results.
Figure 2. Overall logic on the model-based approach for value and sustainability enabled by machine learning.
The design space exploration begins with the definition of design parameter variations in the CAE environment (e.g., geometrical variations in between a range of values). Based on such parameters, the structural analysis, the modal analysis, and the thermal analysis of different design cases are normally run in the CAE environment. Here a design of experiments (DOE) analysis is performed on the CAE simulations to determine the relationships between the inputs and the outputs of the CAE simulations. At this point, the outputs of the CAE simulations are used to populate the value models and sustainability models. The assessment of value is divided into two parallel activities: the qualitative assessment and the quantitative assessment. Concurrently, the assessment of sustainability models is conditioned by the definition of strategic long-term criteria, which give a set of leading criteria from which sustainability indicators are identified with data intervals. Only after those steps are conducted and the indicators defined, more specific sustainability models can be performed in relation to a defined product. The outputs from the sustainability models and the value models are finally integrated with the original outputs from the CAE simulation. This is referred to as post-analysis data and it is used to generate surrogate models to expand the design space exploration while embedding value and sustainability considerations. The following sections describe in detail the logic and rationale of all the steps of the proposed approach.
4.3 Value modeling
As presented in Section 3, research in VDD has long discussed the identification of the most suitable criteria to be used in aerospace product development, also developing customized solutions to specific situations (e.g., Ross et al. Reference Ross, Hastings, Warmkessel and Diller2004; Steiner & Harmon Reference Steiner and Harmon2009; Bertoni, Eres & Isaksson Reference Bertoni, Eres and Isaksson2011b). Among those, a framework of reference for value model development, encompassing qualitative and quantitative criteria, has been proposed by Bertoni, Amnell & Isaksson (Reference Bertoni, Amnell and Isaksson2015a), studying the specific case of aerospace components development. This framework has been used as a starting point for the identification of the value criteria.
The value criteria are divided into two main families: those quantifiable numerically encompassing operational performances, production, and servicing, and those quantifiable qualitatively, using categorical variables, encompassing ‘ilities’ such as commonality, survivability, and scalability (see McManus et al. Reference McManus, Richards, Ross and Hastings2007). The context dependency of the methods used for the computation is the main difference between the two families of criteria; while the quantitative criteria can be computed using numerical functions that are context-independent, and thus generalizable (e.g., the cost of raw material does not depend on its final application), the qualitative criteria assessment is based on judgments that are dependent from the specific industrial context in which a new design is developed (e.g., the commonality in technology is dependent on the technology development of a specific company at a specific moment in time).
As shown in Figure 3, the quantitative criteria included in the approach are divided into three macro-categories: operational performances, production, and servicing. The first includes the assessment of fuel savings and reduction of 
$\text{CO}_{2}$ emissions granted by each single design case. The production criteria include the cost of raw material and the cost of manufacturing. The latter includes a trade-off analysis between the cost of welding, the cost of the casting, and the use of an additive manufacturing process. The maintainability of a product is instead the criterion linked to the servicing. Based on such criteria, three different models are identified as relevant for quantitative value modeling, which are a customer revenue model (including the modeling of operational savings), a maintenance cost model, and a manufacturing feasibility and cost model.
Figure 3. Value criteria identified in the study.
The customer revenue model reflects the monetary value created for the customer for each specific design case. In the aerospace business, fuel consumption is the main driver of operational cost and has a large impact on the revenues of the airlines. By consequence, the customer revenue model depends on the possible savings in fuel consumption granted by each specific design case, which is highly correlated with the reduction of the aircraft weight. To be able to create such a model, three aspects need to be considered: (1) the cascading impact on the overall weight of the aircraft of marginally reducing the weight of a component, (2) the models of the aircrafts in which the new component will be installed, and (3) the impact on the overall fuel consumption of an aircraft of marginally reducing its weight. Addressing the first point, the model needs to include a system of weight reduction multipliers to project the impact of a component change to the overall weight of the aircraft. This part of the model uses as input the value of the design parameters related to different components mass obtained from the simulations, and it is highly dependent on the type of component or subsystem under redesign. To include the impact of a marginal weight reduction of an aircraft on its overall fuel consumption, ML is applied to the publicly available datasets. To this concern, the dataset made available by the International Civil Aviation Organization (ICAO 2017) is identified as a relevant source of data to build multilinear regression analysis on fuel consumption data classified by the aircraft type and the aircraft flight range. The link between the component mass calculated in the simulation, the ad hoc defined weight multipliers, and the multilinear regression analysis on ICAO fuel consumption data allows to provide an estimation of the impact of a specific design case on the customer revenues. Eventually, the impact on customer revenue model can be projected to a life-cycle perspective. This is possible if the expected life of the component obtained from the simulations is considered critical to product life. The reason for this condition is that other components might have sensibly shorter expected life than the one under investigation, making the expected life of the latter not corresponding with the realistic length of the product life cycle.
The proposed approach for modeling the maintenance cost is in line with the one described by Seemann et al. (Reference Seemann, Langhans, Schilling and Gollnick2010), who proposed a surrogate function to model the life-cycle cost of jet engine maintenance based on a large set of historical data related to aircraft maintenance operations. Their findings found a correlation between the maintenance cost of life-limited parts, the take-off thrust, and the weight. This approach implies the extensive collection and classification of aircraft specifications, performance, and maintenance data to explore and quantify emerging correlations through data mining algorithms.
The assessment of supplier costs impacting the manufacturing feasibility and cost model is proposed by applying ML on a suppliers’ deliveries database, investigating the correlations between the design specifications (e.g., geometrical features, material types, and mass) and the historical dataset of supplier performances (e.g., cost, delays, and supplier criticality). Additionally, the model can be improved by exploring the correlation between geometrical properties (e.g., angles, thickness, length, and positioning of welding) of the components and a database of information about casting and additive manufacturing in terms of cost, time, and material scrap.
The approach deals with the modeling of qualitative value aspects by focusing on three main areas, namely, commonality, survivability, and scalability of solutions. Commonality evaluation consists of analyzing the design cases from a technology perspective (e.g., reuse of material type or reuse of qualified welding), from a product perspective (e.g., number of features shared with other components), and from a system architecture perspective (e.g., common features in relation to the system in which the product/component will be integrated). The survivability of a design is defined as the ability to avoid or withstand a hostile environment (e.g., the ability to fly into an ash cloud). The scalability of a design is instead defined as the ability to change the level of one or more system specification parameters while maintaining the product value.
4.4 Sustainability modeling
Sustainability modeling starts with a sustainability identification process, leading to the definition of a SDS, based on a combined forecasting and backcasting approaches (Hallstedt Reference Hallstedt2017). This SDS was used as a starting point for the identification of leading criteria and indicators with the purpose to inform sustainability models to be used in an automated decision support environment. In the proposed approach, the computation of sustainability models is conditioned to the identification of sustainability criteria. They need to be in line with the industry’s strategic long-term development criteria and set the basis for the quantification of sustainability consequences related to different design cases. The last step of the sustainability identification process is the definition of indicators for sustainability. An indicator makes it possible to compare and measure the relative differences between solutions and allows such indications to be included in simulation models. This means that to understand how a design solution influences social and environmental sustainability aspects, and vice versa, a key step is to identify which socio-ecological criteria and indicators are relevant for a product throughout its whole life cycle.
Table 1 lists the sustainability indicators identified in the study, representing the early life-cycle phases of a product. The indicators are defined as measurements or facts (qualitative or quantitative) that indicate the state or level of the leading criteria.
Table 1. Leading sustainability criteria and indicators for each leading criterion at the case company
*REACH: Registration, Evaluation, Authorization, and restriction of chemicals candidate list.
**IAEG: chemical list for the aerospace industry, 3000 substances include global requirements.
After the sustainability identification process, three sustainability models are proposed to clarify what sustainability data to be included, how they are weighted, and how they relate to other design variables in a model-based approach for value and sustainability. One proposed model is ‘sustainability criteria and product life-cycle data simulation – SCADS model’ that aims to connect identified sustainability criteria and relevant indicators to concept data such as geometric design, manufacturing process, and materials. The model aims to automatically calculate a ‘sustainability merit’ for each concept design that is generated and thereby give an indication of the sustainability profile for each concept. The ‘System analysis model of sustainability indicators and functional design requirements’ is meant to guide the weighting of the different sustainability indicators in the SCADS model. Finally, the ‘Correlation studies and system analysis studies between sustainability indicators and design variables’ step aims to give a better understanding of the relationships to support the development of algorithms in a model-based approach for value and sustainability assessment in design space exploration. When the sustainability indicators, its intervals, and relationships to other criteria are decided, correct models and algorithms can be developed. ML can thereby be used to predict sustainability profiles and find patterns of concept solutions.
4.5 Digital model integration and use of machine learning
Due to the multidisciplinary objectives of the design analysis, aerospace product simulations can take up to weeks depending on the complexity of the problem of interest. This is because the multidisciplinary analysis can include several tasks that may require a large number of simulations (hundreds or thousands). Since simulations are expensive, both in terms of time and resources, those are commonly run on a small set of design cases. To expand the analysis of the design space, ML is used to build surrogate models, also called meta-models or response surface models, that can mimic the complex behavior of the underlying simulation models. Through surrogate models, many more design concepts can be analyzed without the need to run more computationally expensive simulations. However, surrogate models are generated based on an original dataset of known input and output and, given the complexity of the simulation activities, such a dataset is often of relatively small size, making it cumbersome to generate accurate and robust models. Thus, the challenge is to generate surrogate models as accurate as possible despite the small size of the datasets.
For this reason, different algorithms to generate surrogate models have been studied, as extensively reported by Shan & Wang (Reference Shan and Wang2010) and Dasari, Cheddad & Andersson (Reference Dasari, Cheddad and Andersson2019). Black-box models, such as linear regression or support vector machines, have been compared to ‘decision tree’ methods, such as random forest (RF) and M5P, to conclude that tree models perform as similar as black-box models when building surrogate models. The relevance of this finding in an engineering design context lies in the fact that tree models can provide ‘if–then’ rules, enhancing the comprehensibility of the model behavior. In other words, tree models allow mapping the procedures linking input parameters to output parameters, thus helping to understand the design space better and make informed decisions about design parameters. Among the tree models, RF is an ensemble method, that is, a combination of multiple methods, and can handle nominal, categorical, and continuous data; thus, it is used for both classification and regression (Breiman Reference Breiman2001). RF contains several decision trees and each tree in the forest represents a model. Furthermore, it has been proven to be the one providing the higher accuracy with small samples (Dasari et al. Reference Dasari, Lavesson, Andersson and Persson2015, Reference Dasari, Cheddad and Andersson2019).
The proposed approach generates prediction models for design space exploration integrating the use of RF for the creation of surrogate models. The first step concerns the setup of a design study, identifying the key design variables to investigate and create computer-aided design (CAD) models. Step 2 concerns the generation of the surrogate model, starting with the definition of the DOE (thus of the systematic variation of input variables). This phase is followed by the numerical simulation of selected points in the design space to build a consistent dataset to train and validate the surrogate model. For this activity, different sampling strategies can be used, and Latin Hypercube sampling is one of the most common sampling strategies applied (Zhao & Xue Reference Zhao and Xue2010). The surrogate model is then built using the analysis results from simulations with RF. Each of the trees in RF is built using a deterministic algorithm by selecting a random set of variables and random samples from the training set. Two of the hyperparameters of RF are needed to build a forest: Ntree, i.e., the number of trees to grow in the forest, based on a bootstrap sample of observations, and Mtry, i.e., a number of features which are randomly selected for all split in the tree. The following steps allow the creation of the surrogate model.
(i) From the dataset D, a bootstrap sample
$D^{\prime }$ is drawn randomly with replacement for each tree construction.(ii) A tree T using the bootstrap sample is built, at each node, choosing the best split among a randomly selected subset of Mtry descriptors. The tree is constructed until no further splits are possible or reaching a given node size limit.
(iii) The second step is repeated until the user-defined number of trees is reached.
Both quantitative and qualitative variables are considered to build the forest. For RF model generation, two-thirds of all training samples are used to build a tree model, whereas one-third forms the out-of-bag samples to test for the accuracy of the tree.
The third step in the study consists of the improvement of the RF model performances, which is obtained by the tuning of the hyperparameters, which is by testing the possible combinations of Ntree and Mtry and selecting the one with the least root mean square error (RMSE) in the prediction. From this model, ‘if–then’ rules are extracted to understand the reasoning for the predictions. Furthermore, design parameter importance is extracted to analyze which parameters have high importance in the model.
The final step of the approach is to visualize the performance level of different variables based on specific trade-off analyses performed by engineers. Engineers need to concurrently visualize, and trade off, the results of hundreds of design cases for a long list of variables. The use of dynamic parallel diagrams emerged as a powerful tool to navigate through the design cases and generate visual feedback on trends and trade-off between the mechanical performance and value and sustainability scores. The choice of dynamic parallel diagrams is in line with the literature presenting them as an established practice for the visualization of mechanical performances obtained from CAE simulation (Kipouros & Isaksson Reference Kipouros and Isaksson2014). In this way, the results are visualized in a way that easily integrates in the current working practice.
Figure 4. The approach applying random forest to create a surrogate model of the design space (adapted from Dasari et al. Reference Dasari, Cheddad and Andersson2019).
Figure 4 summarizes the process of digital model integration using machine learning indicating the five steps of the process (on the left-hand side), detailing the steps to generate the surrogate model (a), visually showing the structure of the different decision trees (b) and providing an example of a possible ‘if–then’ rules extracted from the RF model (c).
5 Application of the approach for the case of a turbine rear structure
The proposed approach has been applied to the case of the development of an aircraft engine component, namely a TRS. The TRS attaches the rear part of the engine to the wing of the aircraft. The component provides the load path from the engine mounts to the core engine and access for service utilities. In addition, the TRS component contributes to directing the airflow as it passes through the engine. The multidisciplinary design problem with high temperatures and complex manufacturing solutions provide a complex design challenge. Figure 5 illustrates the cross-section of a jet engine with the TRS at the rear end.
Figure 5. The cross-section of a jet engine and the location of the TRS.
In this case, different geometric design parameters were varied with the purpose to explore the design space and understand the impact of design parameters on value and sustainability performance.
The first step of the approach consisted of verifying, through DOE, the engineering performance of a high number of design variants (called design cases) in the CAE environment. In total, 56 design cases were used to investigate the design space for the TRS. Each design case contained 21 design variables. A typical example of a key design question for such kind of design is represented by the shape of the outer case of the TRS. As shown in Figure 6, the outer case can be either circular or polygonal. The latter has advantages in the stiffness of the component but can bring disadvantages in aerodynamics and manufacturability. The variations in performances of the design cases in relation to the other design parameters could be investigated in the DOE. Those outputs served as inputs for both value and sustainability modeling. Customer revenue models, manufacturing feasibility and cost (in particular related to the welding process), and maintenance cost models were run as quantitative value assessment models.
Figure 6. A TRS concept with a polygonal outer case (on the left) and a TRS concept with a rounded outer case (on the right).
5.1 Generation of VDD model in the TRS case
In the TRS study, some of the outputs of early simulations (i.e., mass, volume, geometrical dimensions, and the length and thickness of the weld) were used as inputs for the computation of the quantitative criteria by developing specific value functions. The computation of qualitative criteria was based on the possibility to access specific databases to compute the value of categorical variables. Table 2 describes how the quantitative value criteria were computed in the TRS case and how qualitative value criteria could be computed based on data availability. The first column of Table 2 lists the criteria used, the second column lists the inputs from the simulations, the third column shows the external sources that have been collected to populate the models, and the fourth column describes the specific method used for the quantification of the criteria. The lack of availability of data recorded about commonality, scalability, and survivability caused the qualitative assessment models not to be fully implemented in the case study. Estimations were therefore based on assumptions rather than real data.
Table 2. List of quantitative and qualitative value criteria, data collected, and computational methods
The calculation of the customer revenue model depended on the possible savings in fuel consumption granted by each specific design case. In order to calculate such savings, an aircraft fuel performance model was created, considering the aircraft fuel consumption to be directly proportional to the weight of the aircraft. The model also included a system of weight reduction multipliers to project the impact of a change in weight of the TRS component on the overall weight of the aircraft, as described in Section 4.3. Two aircraft types were considered: the Airbus A380 and the Boeing 787 Dreamliner. The aircraft models were, respectively, introduced in 2007 and in 2011 and have been operating long enough to obtain reliable fuel consumption data made available by the ICAO (2017). Multilinear regression analysis was applied to fuel consumption data to derive aircraft fuel consumption models based on the aircraft flight range and type. Maintenance cost model analysis in the case study was limited to the maintenance cost linked to the life-limited parts and assumed the take-off thrust not to be impacted by the different TRS designs. The assessment of supplier costs suffered from the unavailability of a structured and large enough set of data; thus, such an activity was performed on a demonstrative database with the intent to demonstrate the approach rather than providing verified results for the case study.
The manufacturing model was based on data about machine performances for different technologies and operations available in literature. The model compared casting versus additive manufacturing technologies and investigated feasible welding technologies. Correlations between geometrical properties and a database of information about casting and additive manufacturing performances were explored using realistic data.
The qualitative models suffered from a general lack of data records in the case study. Data about performances during product test and unexpected behavior in operations were not directly available to the engineering dealing with design space exploration. This is because such data were normally collected at different organizational levels and are typically not requested and shared unless a clear request is done and a need for those is evident. Similarly, data about commonality and scalability would need to be collected, structured, and stored for a certain period of time before allowing the investigation of possible correlations in the data. From this perspective, the case study application of the approach did not reach an ideal implementation, rather it raised the awareness of the necessity to allow such kind of data to be collected in an accessible format to be used during design space exploration.
5.2 Generation of SPD models in the TRS case study
A sustainability assessment was performed on the design case by first identifying strategic long-term and leading sustainability criteria to define sustainability indicators and the related data intervals. An interval between an acceptable and a minimum level, including a target value, was defined for each leading criterion. These intervals were derived from dialogs with company experts and from information and data found in documents such as sustainability reporting guidelines, environmental sustainability index reports, and strategies and guidelines from the Advisory Council for Aviation Research and Innovation in Europe (ACARE 2011).
The indicators and related intervals are presented in Table 3. For the TRS case, one of the sustainability indicators, SCI, was derived from a material criticality assessment method based on values characterized by qualitative assessments translated into a quantitative indicator, i.e., an SCI score. This method was presented in Hallstedt & Isaksson (Reference Hallstedt and Isaksson2017) and is shown in Figure 7.
Table 3. Indicators with a suggested interval for each leading criterion for the case are presented. The intervals go from acceptable to a minimum level (worst level), including a target level
Figure 7. Illustration of the material criticality assessment method generating a sustainability compliance index score. (Hallstedt & Isaksson Reference Hallstedt and Isaksson2017).
5.3 Generation of surrogate models in the TRS case study
ML was applied in the case study both to generate descriptive models, thus to populate value as described in Section 4.1, and to create predictive digital models of performances.
Approaching the design of complex components like the TRS. Many design studies are incrementally built to investigate different design aspects and identify behavior and constraints. The case study focused, in particular, on investigating four design parameters (inputs) of hub configuration such as hub rear stiffener height, forward hub wall angle, hub knee point radial position, and bearing flange axil position. Based on those, 17 geometrical and thickness parameters were studied. In total, 21 design parameters were used in generating the CAD model analyzed through finite element analysis. Using a Latin Hypercube strategy, a total of 56 concepts were generated and simulated and were used as a dataset to build the surrogate models following the steps previously shown in Figure 4.
The dataset obtained from the simulation was combined with value and sustainability into one combined dataset to build a prediction model using RF. The RMSE for each predicted variable was calculated before and after applying hyperparameter tuning. Table 4 provides an extract from the analysis of the normalized RMSE of the prediction of some key design variables.
Table 4. Extract from the analysis of the normalized RMSE of the prediction of TRS mass and welding life from the simulation (adapted from Dasari et al. Reference Dasari, Cheddad and Andersson2019)
The accuracy of the prediction of variable ‘TRS mass’ had particular importance for value and sustainability assessment. This is because, as shown in Tables 2 and 3, the TRS mass was the main variable for the computation of six out of seven quantitative value criteria, while also being a sustainability indicator for ‘use and maintenance’.
The use of the RF model also allowed not only the creation of surrogate models for predictions but also the analysis of the importance of the design parameters with respect to the TRS performances. Figure 8 shows the difference in relative importance (on the 
$y$-axis) of the 21 different design parameters (on the 
$x$-axis). For issues related to industrial secrecy, the names of the design parameters have been substituted with an indicative Design Parameter ID.
Finally, the results were visualized by means of dynamic parallel coordinated. Figure 9 shows an example of such a visualization. In the visualization, a selection of variables is included, and the results are evaluated with respect to performance metrics such as mass, cost, and SCI score (named Sust_Compliance in Figure 9). Each line of Figure 9 describes a single design case and its performance in terms of mechanical properties, value, and sustainability. The first column indicates the nature of the data differentiating those obtained from real simulation results (bottom-left corner) and those generated by the surrogate model (top-left corner). The other columns indicate different design parameters and value and sustainability scores, such as, for instance, if the design case considered has a circular or polygonal outer case (third column). It has to be noted that Figure 9 has a demonstrative purpose and visualizes only a partial set of the design parameters possible to evaluate.
Figure 8. Visual representation of the relative importance of each design parameter with respect to the TRS performances (note that the name of the parameters has been omitted for industrial secrecy issues).
Figure 9. Illustration of the dynamic parallel diagram, which has a demonstrative purpose and visualizes only a partial set of the design parameters possible to evaluate.
6 Discussion
Early phases of product development are sometimes described as ‘fuzzy’ as they are characterized by incomplete information about both the product solution and the factual conditions wherein the product will be ultimately used. Literature often cites the design process paradox (Ullman Reference Ullman1992) as the founding rationale of many design decision support methods, underlying the need of making relevant design decisions in a moment in time where limited information about the future product is available. Both VDD and SPD literature describe how such a challenge escalates in complexity in the presence of value and sustainability considerations. The poor availability of data, combined with their heterogeneous and cross-disciplinary nature, causes value and sustainability considerations to be poorly integrated into the decision models. Value and sustainability evaluations often fall outside the technical horizon of the engineers, whose activities and design decisions are rather driven by what they best perceive as reliable and understandable (Charnley, Lemon & Evans Reference Charnley, Lemon and Evans2011; Bertoni et al. Reference Bertoni, Bertoni, Panarotto, Johansson and Larsson2016). Initial works on data science in product development (e.g., Tseng & Jiao Reference Tseng and Jiao1997; Kusiak Reference Kusiak2006; Geng, Chu & Zhang Reference Geng, Chu and Zhang2012) have presented some applications of data mining and ML to support early design decision-making. Although, there is a need for smoother integration of data science methods in traditional engineering working practices to obtain effective use of data.
The approach presented in this paper moves a step toward a seamless integration of ML into a CAE-based decision support environment. This is meant to have an impact on the design practice by providing value and sustainability assessment as part of the results of a design space exploration, rather than as external complementary models, with different levels of detail, used at different moments in time during the development process.
From the perspective of the development of VDD models, the proposed approach contributes to the theory presenting the capability to link the value models to parametric variations of the CAD models, thus allowing the automatic calculation of the value of hundreds of different product configurations in a restricted time frame. This allows the application of VDD models to subsystem parts and components, limiting the application to the feasible design space obtained from structural simulations and avoiding the need to test specific objective functions decided a priori (such as in Castagne et al. (Reference Castagne, Curran and Collopy2009)). At the same time, it allows an engineer to perform the analysis on his/her own, being the owner of the results (thus avoiding the need of the figure of a ‘value analyst’ as previously proposed by Bertoni, Bertoni & Johansson (Reference Bertoni, Bertoni and Johansson2011a) and Panarotto, Larsson & Larsson (Reference Panarotto, Larsson and Larsson2013)).
From the SPD perspective, this approach is a unique way to include sustainability aspects early in the design process. What makes it unique is the systematic process to identify the sustainability criteria to focus on and to include those in the assessment. The SDS method supports the identification of a set of sustainability indicators that covers all dimensions, i.e., social, ecological, and economic, of sustainability and thereby avoid a suboptimization. This approach is different from other eco-design assessments or life-cycle assessment as it is based on a strategic sustainability perspective using overarching sustainability principles from a backcasting perspective.
The approach presented was also developed with the intention to move toward an integration of the VDD and the SPD models, which have been historically developed in different research streams. VDD and SPD share the common goal to estimate the impact of a design decision from a life cycle and system perspective; however, the results of value and sustainability models have traditionally been considered separately. While recognizing similarities in the nature and logic of the formulation of value criteria and sustainability criteria and indicators, a deeper theoretical discussion about differences and similarities of VDD and SPD models is outside the scope of this paper.
The work does not claim to contribute to the advancement of the ML research field in terms of the development of new algorithms, but it proposes a novel application area for ML techniques to support integrating VDD and SPS in design space exploration.
The needs and challenges at the basis of the logic of the approach were identified as relevant to the design space explorations of different components in aerospace development. The generalization of the approach in other industrial contexts cannot be claimed since extensive research was not performed. A limitation to consider in terms of generalization is that the presented approach is particularly suitable for a product with a relatively static architecture, that is, for design concepts with a defined number of parts and a geometry that can be parametrically changed in the CAD environment. In the presence of design alternatives radically different in the number of parts and geometry, the need for different CAD models and the need to study new correlations between different variables escalate the complexity of the calculation. This would most likely negatively impact the usability of the approach as decision-making support.
The proposed approach was applied in the case of the development of a TRS for a commercial aircraft engine. As described in Section 5, the case study application slightly differed from the ideal proposed approach described in Section 4. Such discrepancy is in line with the DRM theory describing the emergence of possible misalignment between the ideal design support and the actual design support. The main reason for not achieving the ideal implementation of the approach was the limited availability of data that were accessible to the engineers. In some situations, data were stored in databases that were not readily accessible but were possible to integrate into the case study after a short screening (such as in the case of revenues per flights or maintenance or manufacturing cost). In other situations, data did not exist not because of the presence of technological barriers, rather because the need of collecting and storing such data was not perceived and the effective way to use those data was never planned (this is, for instance, the case for product commonality and scalability). Similarly, data concerning the environmental impact and sustainability implications are increasingly collected in response to the needs from governments and society, but they are at a level of granularity that did not allow them to be effectively integrated into the TRS case.
Concerning the communication of the results of the models, the visualization by means of dynamic parallel diagrams allowed the integration of the results in an environment which was familiar with the working methods of the engineers. However, such a visualization did not provide any indication in terms of effectiveness toward reducing the risk of providing a false sense of accuracy in the results. Concerning the reliability of the results, the measurement of the errors of the ML algorithm provided good indications on the accuracy of the surrogate models (Table 4), while the application of the approach did not integrate a method to quantify the assumptions and uncertainties ingrained in the value and sustainability assessment.
The validation of the proposed approach (i.e., the support evaluation activity according to the DRM) was performed making use of a simplified parametric model of a TRS tested through hundreds of design variations by researchers in the university laboratory. The objective of the support evaluation was to verify the process logic by testing the consistency of the mathematical results in terms of the order of magnitude. A simplified parametric model, only consisting of four components, was developed and DOE was run to simulate the mass, the volume, and the weld length and thickness of 160 design alternatives. Those results were used to populate the quantitative value models and obtain a final result for the 160 fictitious designs. This evaluation was performed to test the functionalities and the limitations of the approach prior to the application in the case study. As a result, the mathematical logic of the computational algorithm was improved. Later in the case study, the so-called ‘application evaluation’ took place by verifying the usability of the approach in relation to the desired performance. At this stage, industrial practitioners from the case company were invited to provide feedback on the utility and usability of the approach that was introduced through interactive presentations and demonstrative videos. This stage was enabled by the implementation of the proposed approach in the so-called model-driven decision arena (Bertoni, Wall & Bertoni Reference Bertoni, Wall and Bertoni2018b), which is an interactive visual environment designed to support group decision-making, by computing and visualizing the results of multidisciplinary analyses. The presentation of the results to the industrial practitioners happened in an iterative fashion through bi-weekly distance meetings and occasional physical meetings. The aim of such activities was to collect and implement constructive feedback both about the structure and reliability of the approach and about its ability to support decision-making. In terms of the verification of industrial benefits introduced by the approach, the long development process of aerospace products renders a situation in which it is difficult to practically verify the impact of the approach in terms of lead time reduction, customer satisfaction, or revenue generation, whose complete effects will need to be verified in future research.
7 Conclusion
The paper has presented a prescriptive approach toward exploiting the use of ML in combination with value and sustainability assessment to create a model-driven approach to support engineers in the design space exploration. The approach allows integrating value and sustainability assessment models in a unique modeling environment, enabling the automatic computation of VDD and SPD models for a high number of design cases, encompassing the estimation of both numerical and categorical variables by means of ML.
The approach presented supports engineers in performing multidisciplinary design space, allowing the identification of design solutions with poor value contribution or low sustainability performance early in the development process. This generates savings in time and resources, reducing the risk to further develop design concepts that would most likely show poor value for the stakeholders, or bad sustainability performances, later in the design process. From a theoretical perspective, the paper showed an example of the possible integration of the VDD and SPD models in a unique approach. The combined use of VDD models and SPD models is proposed as a complementary approach supporting CAE simulations in design space exploration and ML is presented as a technology enabler for model results integration. The use of ML allows reducing the time for simulating design variations using surrogate modeling. To this concern, a novelty of the approach is the creation of surrogate models including value and sustainability criteria, enabling a multidisciplinary analysis of hundreds of design alternatives guiding the selection of the more promising solutions.
The work is part of a larger research initiative aiming at the development of an integrated model-driven methodology for early design decision support. The application in the TRS case study was an intermediate step toward the final development of the methodology, and it was performed to test the applicability and effectiveness of the proposed model-based approach in a limited scenario. More research challenges still need to be addressed focusing on the following:
(i) Investigate the definition of more specific parameters to be used as proxies for the quantification of ‘ilities’ (e.g., survivability, commonality, and scalability), concepts otherwise perceived as too vague by engineers. This would make it easier to identify relevant datasets to explore in the search of correlations with design parameters.
(ii) Perform correlation studies between sustainability indicators and value criteria to give a better understanding and clarify what sustainability indicators to include in the models and how sustainability risks are related to value criteria, e.g., fuel cost saved,
$\text{CO}_{2}$ emissions saved, cost of raw material, commonality in product, and commonality in production.(iii) Develop the proposed SCADS model and perform system analysis of sustainability indicators and functional design requirements to guide the weighting of the different sustainability indicators in the SCADS model.
(iv) Perform correlation studies and system analysis studies between sustainability indicators and design variables. There is a need to have a better understanding of the relationships and the influences between sustainability indicators and design variables to support the development of algorithms in the model.
(v) Investigate the use of ML in the sustainability assessment models with the purpose to predict and estimate the sustainability profiles of numerous variants of solutions within the constraints defined by the intervals for each sustainability indicator. This could allow the identification of an optimal solution from a sustainability perspective.
Acknowledgments
The research leading to these results has received financial support by the Swedish Knowledge and Competence Development Foundation (Stiftelsen för kunskaps- och kompetensutveckling) through the Model-Driven Development and Decision Support research profile at Blekinge Institute of Technology.
Open access
