1. INTRODUCTION
The aviation sector is projected to be one of the fastest growing contributors to anthropogenic greenhouse gas emissions (Lee, Reference Lee, Fahey, Forster, Newton, Wit, Lim, Owen and Sausen2009). If left unconstrained, the emissions from aircraft in 2050 are projected to be quadruple the emissions from aircraft in 2006 (International Civil Aviation Organization, 2010a ). Many groups are using simulation-based tools to study technologies, designs, and policies that address this challenge. In conducting such studies, it is critical to quantify the effects of uncertainties and to account for their impacts in decision making. Yet, uncertainty quantification for such a complex multicomponent system, with multiple components ranging from individual aircraft technologies to fleet-wide operations, is a significant challenge. This paper demonstrates how a decomposition-based approach can manage the complexity to make uncertainty quantification tractable for a large-scale feed-forward problem in environmental impacts of aviation technology and operation. Our problem is large scale in several regards: the dimension of the uncertain parameter space is 12; there are 100 variables describing the coupling between the two components; and a single analysis (forward simulation) of the system takes approximately 3 min on a desktop computer. To manage the high-dimensional multicomponent interface, we apply a combination of dimension reduction to identify the important coupling variables and importance sampling to transform the information across the interface in an efficient and dependable manner.
As a measure to manage the climate impact of aviation, the Committee for Environmental Protection under the International Civil Aviation Organization and with support from the Federal Aviation Administration, adopted a 2% annual efficiency improvement goal for aviation through 2050 (International Civil Aviation Organization, 2010b ). To satisfy this fast-paced fleet-wide improvement requires significant enhancements to aviation technology, sustainable fuels with low CO2 emissions, and efficient operational procedures (Rutherford & Zeinali, Reference Rutherford and Zeinali2009). To meet these demanding requirements, Committee for Environmental Protection assembled a panel of independent experts with varying backgrounds to establish long-term technology goals for aviation fuel consumption (Cumpsty et al., Reference Cumpsty, Alonso, Eury, Maurice, Nas, Ralph and Sawyer2010). Within their study they investigated future aviation technology scenarios, which represented varying regulatory pressure to reduce fuel consumption. The future aircraft technology scenarios were then applied in analysis tools as “technology packages” to assess the technology improvement on aircraft fuel consumption. However, these technological enhancements are inherently uncertain, and thus their respective impacts on the environment are also uncertain.
Through a rigorous characterization and management of uncertainty, one can provide quantitative estimates of uncertainty necessary to calculate relevant statistics and event probabilities. However, to estimate relevant statistics and event probabilities requires an uncertainty quantification of the entire system. Uncertainty quantification is a broad field encompassing a number of different aspects (Smith, Reference Smith2013); this work focuses on uncertainty analysis and global sensitivity analysis. The objective of an uncertainty analysis, also known as forward propagation of uncertainties, is to quantify the distribution of the output of interest, given distributions on the uncertain input variables. The objective of a global sensitivity analysis is to apportion variability in the output quantity of interest according to contributions from the uncertain input variables and their interactions.
Quantifying uncertainty of an entire system may be cumbersome due to factors that result in inadequate integration of engineering disciplines, subsystems, and parts, which we refer to collectively here as components. Such factors include components managed by different groups, component design tools or groups housed in different locations, component analyses that run on different platforms, components with significant differences in analysis run times, lack of shared expertise among groups, and the sheer number of components comprising the system. These challenges are only heightened by the fact that globalization has spread the design and analysis of the complex systems throughout the world. Recognizing the challenge of quantifying uncertainty in multicomponent systems, we establish a divide-and-conquer approach, inspired by the decomposition-based approaches used in multidisciplinary analysis and optimization (Braun & Kroo, Reference Braun and Kroo1997; Sobieszczanski-Sobieski et al., Reference Sobieszczanski-Sobieski, Agte and Sandusky2000; Kim et al., Reference Kim, Michelena, Papalambros and Tao Jiang2003; Kroo, Reference Kroo2004).
Previous works have tackled these challenges through the use of surrogate modeling and/or a simplified representation of the uncertainty. Using surrogates in place of the higher fidelity components in the system provides computational gains and also accommodates the task of integrating components (Martin & Simpson, Reference Martin and Simpson2006). Using a simplified uncertainty representation (e.g., using mean and variance in place of full distributional information) bypasses the need to propagate the full nonparametric uncertainty from one component to another. Such simplifications are commonly used in uncertainty-based multidisciplinary design optimization methods as a way to avoid a system-level uncertainty analysis (see, e.g., Yao et al., Reference Yao, Chen, Luo, van Tooren and Guo2011, for a review of these methods and their engineering applications). Other methods have exploited the structure of the multicomponent system to manage the complexity of the system uncertainty analysis where the system contains a feedback loop (Arnst et al., Reference Arnst, Ghanem, Phipps and Red-Horse2012a , Reference Arnst, Ghanem, Phipps and John Red-Horse2012b ; Sankaraman & Mahadevan, Reference Sankararaman and Mahadevan2012; Arnst et al., Reference Arnst, Ghanem, Phipps and Red-Horse2014; Constantine et al., Reference Constantine, Phipps and Wildey2014; Chaudhuri & Wilcox, Reference Chaudhuri and Willcox2016). In addition, previous works have tackled decomposition-based global sensitivity analysis within the application of feed-forward systems. A top-down (i.e., assuming that all system variables are independent) sensitivity analysis strategy was developed to determine critical components in the system and used a simplified formulation to evaluate the main sensitivity indices (Yin & Chen, Reference Yin and Chen2008; Liu et al., Reference Liu, Yin, Arendt, Chen and Huang2010). Limitations in these existing approaches include the introduction of approximations of the components comprising the system, parametric approximations of the uncertainty, a need to repeatedly evaluate the system in its entirety, and a need to apply correlation between system variables to account for dependency structure. Instead, we approach the problem using a decomposition-based vision of the multicomponent uncertainty quantification task: performing uncertainty quantification on the respective components individually, and assembling the component-level uncertainty quantifications to quantify the system uncertainty.
In Section 2, we introduce the multicomponent aviation system of interest and describe its constituent components: the aircraft technology component and aviation environmental impacts component. In Section 3, we present the coupling between the aircraft technology component and the aviation environmental impacts component. We discuss the challenges that arise from this component-to-component coupling and our proposed solution for overcoming said challenges. Our decomposition-based uncertainty quantification approach is introduced in Section 4 along with the uncertainty quantification results. Our results are compared to the standard Monte Carlo simulation of the entire system. Finally, our conclusion and future work are provided in Section 5.
2. SYSTEMS MODELING OF AVIATION ENVIRONMENTAL IMPACTS
The multicomponent system of interest consists of a conceptual-level aircraft design component, the Transport Aircraft System Optimization (TASOpt; Drela, Reference Drela2010), and an aviation environmental impacts component, the Aviation Environment Design Tool (AEDT) version 2a (Roof et al., Reference Roof, Hansen, Fleming, Thrasher, Nguyen, Hall, Grandi, Kim, Usdrowski and Hollingsworth2007). This multicomponent system is depicted in its entirety in Figure 1.
Fig. 1. System-level uncertainty quantification of the toolset consists of quantifying how uncertainty in aircraft technologies and operations impacts the uncertainty in the output of interest, here the aircraft's payload fuel energy intensity.
2.1. TASOpt
The TASOpt component is an aircraft performance tool that allows users to evaluate and size future aircraft with potentially novel airframe, aerodynamic, engine, or operation variables using low-order physical models implementing fundamental structural, aerodynamic, and thermodynamic theory. TASOpt uses historical-based correlations only when necessary, in particular only for some of the secondary structure and aircraft equipment. The TASOpt component takes as input aircraft technology and operational variables and can either optimize an aircraft over a specified set of constraints or resize an aircraft to meet a desired mission requirement. The aircraft configuration selected for this study is the Boeing 737–800W shown in Figure 2. This aircraft operates in the short-to-medium range while seating approximately 180 passengers.
Fig. 2. Boeing 737–800W airframe configuration. (Schematic from Boeing.com)
Table 1 contains the 27 uncertain TASOpt random input variables selected for this study and their respective distributions. These input variables represent the technological and operational variables of an aircraft that are considered to be uncertain in the design phase. The uncertainty associated with the technology input variables captures our lack of knowledge due to material properties and measurement capabilities. The uncertainty associated with the operational input variables captures the designer lack of knowledge in the design phase of an aircraft. We start by defining a baseline aircraft. The baseline aircraft configuration used for this study was created using TASOpt's optimization capabilities to best represent the Boeing 737–800W aircraft configuration (defined in Figure 1’s “Boeing 737–800 Template Input File”). We then generate a realization of the random variables associated with each of the 27 input variables in Table 1. This defines a “sampled aircraft.” Next, TASOpt resizes this sampled aircraft in order to ensure that it is a feasible aircraft.
Table 1. TASOpt random input variables and their respective distributions
The next step of the process quantifies the sampled aircraft's performance by flying 99 mission profiles, which are generated using a Latin hypercube design of experiments. The Latin hypercube design of experiments populates the 12 mission input variables contained in Table 2. Of the 100 mission profiles (baseline plus 99 additional missions) simulated in TASOpt, the first 50 are flown under international standard atmosphere (ISA) conditions while the remaining 50 are flown under non-ISA conditions. For any mission variable (e.g., Range) we generate 99 realizations from the uniform distribution [i.e.,
${\rm \cal U}(a\comma \,b)$
] using the Latin hypercube sampling scheme and the parameters provided in Table 2. This process allows, for example, the Range to vary between 750 (nmi) and 3250 (nmi). Finally, the aircraft's configuration, operational procedure, and performance over multiple flight segments and atmospheric conditions are provided by the TASOpt component. This information is then transformed through a regression process in order to construct a similar representative aircraft in the AEDT database. We shall discuss this multicomponent coupling procedure further in Section 3.
Table 2. TASOpt mission input variables and their respective uniform distribution parameters
Note: The performance of each sampled aircraft configuration is evaluated using a Latin hypercube design of experiments.
aParameters that are also TASOpt random input variables; in those cases, the parameters here represent deviations from the input realization to TASOpt.
2.2. AEDT
The AEDT component is a suite of integrated aviation environmental impact tools. AEDT provides users with the ability to assess the interdependencies among aviation-produced fuel consumption, emissions, and noise. For cruise conditions the AEDT component implements the Eurocontrol's Base of Aircraft Data (Nuic, Reference Nuic2010), which uses an energy-balance thrust model and thrust specific fuel consumption modeled as a function of airspeed. The Base of Aircraft Data fuel consumption model has been shown to work well in cruise, with differences from airplane reported fuel consumption of about 5% (Nuic, Reference Nuic2010). For terminal conditions (e.g., departure/arrival flights until 10,000 feet above ground level), the AEDT component implements a set of energy-balance equations to support a higher level of fidelity in fuel consumption modeling.
The AEDT component characterizes an aircraft using 100 input variables as depicted in Figure 1 as the AEDT aircraft coefficients. A detailed description of the AEDT input variables are provided in the AEDT technical manual (Nuic, Reference Nuic2010; Koopermann & Ahearn, Reference Koopmann and Ahearn2012). These input variables characterize the aircraft's configuration and the operational procedure, and define the aircraft performance over multiple flight segments and atmospheric conditions. To initialize the AEDT component, we first generate a temporary Boeing 737–800W aircraft within the AEDT fleet database. Any TASOpt-generated aircraft then replaces the 100 aircraft input variables in the temporary Boeing 737–800W AEDT fleet database through the multicomponent coupling procedure described in the next section. The objective of the multicomponent coupling procedure is to ensure that an AEDT aircraft characterized through these 100 AEDT input variables is an adequate representation of the sampled TASOpt aircraft.
For each sampled aircraft, we must quantify its respective environmental impacts. To do so, we fly each sampled aircraft over a set of deterministic flight trajectories using the AEDT component. The flight trajectories for this study were selected from a 2006 representative day flight scenario database (Noel et al., Reference Noel, Allaire, Jacobson, Willcox and Cointin2009). Using the representative day, the flights associated with the Boeing 737–700 aircraft are included as possible flight trajectories, as the Boeing 737–800W was not comprehensively represented in the 2006 representative day flight scenario. Twenty flight trajectories were selected; they are presented in Table 3 and illustrated in Figure 3. For computational purposes, these flight trajectories are approximated by a great circle path from the departure airport to the arrival airport. Each flight trajectory was generated by the TASOpt component using the baseline Boeing 737–800W aircraft configuration.
Fig. 3. Illustrated here are the 20 representative flight trajectories flown by the Boeing 737–800W in the Transport Aircraft System Optimization–Aviation Environment Design Tool uncertainty quantification study.
Table 3. Representative flight trajectories flown by the Boeing 737–800W in the TASOpt-AEDT uncertainty quantification study
The output of interest, which we use to quantify an aircraft's environmental impacts, is the PFEI fuel consumption performance (i.e., fuel energy consumption per payload-range) and is defined as
$$\hbox{PFEI} = \displaystyle{{\mathop \sum \nolimits_i W_{{\rm fuel}\comma i}h_{{\rm fuel}}} \over {\mathop \sum \nolimits_i W_{{\rm pay}\comma i}R_{{\rm total}\comma i}}}\comma \; $$
where the summation is over number of missions, w fuel,i is the total fuel consumption of the ith mission, R total,i is the total range of the ith mission, W pay,i is the payload weight of the ith mission, and h fuel is the specific heating value of kerosene.
3. MULTICOMPONENT COUPLING AND DIMENSION REDUCTION
The objective of the multicomponent coupling procedure is to couple the tools and create a consistent representation of the system, that is, to accurately represent the sampled TASOpt aircraft within the AEDT component. In this section, we summarize the multicomponent coupling procedure and demonstrate how to transform the TASOpt outputs (i.e., aircraft configuration and performance) into AEDT inputs. We validate the multicomponent coupling by comparing the fuel consumption over similar scenarios (i.e., flight trajectories and operations) of a sampled TASOpt aircraft flown in TASOpt to that same aircraft imported into the AEDT component and flown in AEDT. Finally, we reduce the dimensions of the multicomponent interface coupling by identifying which of the AEDT inputs have a significant impact on the systems output of interest uncertainty.
3.1. Multicomponent coupling
The TASOpt component outputs the aircraft's configuration variables, which include the maximum takeoff weight, empty weight, maximum fuel weight, wing area, and maximum thrust. In addition, the TASOpt component outputs the aircraft's performance for each mission, where each mission is composed of 15 individual flight segments (i.e., 3 takeoff, 5 climb, 2 cruise, and 5 descent segments). The TASOpt generated aircraft performance data contains,
$$\left[ \matrix{ {\hbox{Range}} \cr {\hbox{Altitude}} \cr \hbox{True Airspeed} \cr \hbox{Mach Number} \cr \hbox{Lift Coefficient} \cr \hbox{Drag Coefficient} \cr \hbox{Aircraft Weight} \cr \hbox{Thrust} \cr \hbox{Fuel Burn Rate} \cr \hbox{Angle of Attack} \cr \hbox{Total Temperature at Engine Inlet} \cr \hbox{Total Pressure at Engine Inlet}} \right].$$
For demonstration purposes, we present an example of how to compute the AEDT thrust coefficients
$\hbox{T}\hbox{C}_{C_1}$
,
$\hbox{T}\hbox{C}_{C_2}$
, and
$\hbox{T}\hbox{C}_{C_3}$
in a flight's climbing segment and under ISA conditions. The AEDT aircraft thrust coefficients are related to the TASOpt outputs thrust, F, and altitude, h, by the following relation (Noel et al., Reference Noel, Allaire, Jacobson, Willcox and Cointin2009; Nuic, Reference Nuic2010):
$$F = \hbox{TC}_{C_1} \times \left( {1 - {h \over \hbox{TC}_{C_2}} + \hbox{TC}_{C_3} \times h^2} \right).$$
To perform this transformation, we use the TASOpt thrust and altitude performance data in the first 50 missions (i.e., under ISA conditions) and in the five climbing flight segments. With the TASOpt performance data collected, we perform a regression to obtain the AEDT coefficients using Eq. (2). A similar procedure to the one explained here is performed for the remaining AEDT input variables. This multicomponent coupling procedure is depicted in Figure 1 as the “TASOpt to AEDT Transformation.”
To validate our multicomponent coupling procedure, we compare the total fuel consumption and relative error over three flight trajectories. The results from our validation study are provided in Table 4. The results indicate that we obtain a fair comparison between the two components. Of the three trajectories, Flight B had the highest relative error; however, this discrepancy can be attributed to how AEDT computes the aircraft's takeoff weight with respect to the flight trajectories range (Koopmann & Ahearm, Reference Koopmann and Ahearn2012). In Flight B, the AEDT takeoff weight was significantly higher than the TASOpt takeoff weight, which resulted in the increased total fuel consumption. The deviation in total fuel consumption in Flight A and Flight C are significantly lower than Flight B, as the AEDT aircraft takeoff weight was approximately equal to the TASOpt aircraft takeoff weight.
Table 4. Fuel consumption results over the three flight trajectories
Note: The TASOpt row represents an aircraft generated by and flown in TASOpt. The AEDT row represents the TASOpt aircraft imported into the AEDT component through the multicomponent coupling procedure and then flown on the same flight trajectory as the TASOpt flight trajectory.
3.2. Dimension reduction
The challenge with performing a decomposition-based uncertainty quantification of the system illustrated in Figure 1 lies in the high-dimensional multicomponent interface. The reason this is challenging is because our decomposition-based uncertainty quantification of the multicomponent system, which we present in Section 4, requires that we deconstruct then reconstruct this multicomponent interface. To mitigate this challenge, we exploit the fact that many of the AEDT input variables have an insignificant impact on the uncertainty in the system output of interest. This permits us to reduce the dimensions of the multicomponent interface to only those AEDT input variables that have a significant influence on the system output of interest uncertainty. As a result, we perform the decomposition-based uncertainty quantification on a reduced multicomponent interface. In this section, we quantify the influence of the AEDT input variables on the system output of interest and use this information to determine which of the AEDT input variables should take part in the decomposition-based uncertainty quantification.
We quantify the influence of the AEDT input variables on the system output of interest using a variance-based global sensitivity analysis of AEDT (Saltelli, 2004). A variance-based global sensitivity analysis of AEDT apportions the output of interest variance among the AEDT input variables. Implicit in a variance-based global sensitivity analysis is the assumption that the variance characterizes the uncertainty of the output of interest (Borgonovo, Reference Borgonovo2007; Borgonovo et al., Reference Borgonovo, Castaings and Tarantola2011). However, in many applications, the variance only provides a restricted representation of the output of interest uncertainty. In this work, we will assume the variance adequately characterizes the output of interest uncertainty for our purposes; however, in general, the results of a variance-based global sensitivity analysis should be carefully interpreted. In addition, as the inputs to AEDT are obtained from the outputs of TASOpt, there is no guarantee that the AEDT inputs are independently distributed, which is a strong assumption in many methods for performing variance-based global sensitivity analysis (Saltelli, 2004). As a result, we implement a generalized analysis of variance (ANOVA) dimensional decomposition, which addresses dependent variables explicitly, that is, without invoking any isoprobabilistic transformations (Rahman, Reference Rahman2014).
A generalized ANOVA dimensional decomposition aims to represent the high-fidelity AEDT component using multivariate orthonormal polynomials as basis functions. The multivariate orthonormal polynomials are constructed with respect to the AEDT dependent input probability distribution. The multivariate orthonormal polynomials form the basis for the high-dimensional model representation subcomponent functions by which the coefficients are solved through a coupled system of equations satisfying the hierarchical orthogonal condition of the subcomponent functions. The AEDT component, which we represent here as f, admits a unique, finite, hierarchical expansion,
$$\eqalign{ f ({\bf t}) &= f_{\{ 0\}} + \sum\limits_{i = 1}^{d} {f_{\{ i\}} (t_i)} + \sum\limits_{i - = 1}^d {\sum\limits_{j \gt i}^d {f_{\{ i\comma j\}} (t_i\comma\,t_j)}} + \cdots + f_{\{ 1\comma2\comma \cdots\comma d\}} (\bf t)\comma \cr &= \sum\limits_{{\rm \cal A} \subseteq \{ 1\comma \ldots\comma d\}} {f_{\rm \cal A}({\bf t}_{\rm \cal A})} \comma }$$
where t = [t
1, t
2, . . . , t
d
]
T
are the inputs to f, f
{0} is a constant, f
{i} is a function of only t
i
, f
{i,j} is a function of only t
i
and t
j
, and so on (Rahman, Reference Rahman2014). Using the generalized ANOVA dimensional decomposition, the expectation of the AEDT output of interest (i.e., PFEI) is given by the first subcomponent function (i.e.,
${\rm \cal A}$
= {0}),
$$\mu = {\open E}_X[ f ({\bf t})] = f_{\{0\}} \comma $$
where X : Ω → ℝ
d
defined on the probability space
$({\rm \Omega}\comma {\cal F}\comma {\open P})$
is the AEDT random input variable. In addition, the variance of the AEDT output of interest is given by
$$\eqalign{& \sigma ^2 = {\rm \open E}_X\left[ {{\left( {f\left( {\bf t} \right) - {\rm \mu }} \right)}^2} \right] = \mathop \sum \limits_{0 \ne {\rm \cal A} \subseteq \left\{ {1, \ldots ,d} \right\}} {\rm \open E}_X [f_{\rm \cal A}^2 \left({{\bf t}_{\rm \cal A}} \right)] \cr & \quad \quad + \mathop \sum \limits_{\matrix{ {0 \ne {\rm \cal A},{\bar {\cal A}} \subseteq \left\{ {1, \ldots ,d} \right\}} \cr {{{\rm \cal A} {\nsubseteq} {\bar {\cal A}} {\nsubseteq} {\cal A}}} \cr }} {\rm \open E}_X\left[ {f_{\rm \cal A} \left({{\bf t}_{\rm \cal A}} \right)f_{{\bar {\rm \cal A}}}\left( {{\bf t}_{{\bar {\cal A}}}} \right)} \right],}$$
where d = 100. The second sum accounts for the covariance contributions from two distinct (i.e.,
${\cal A} \subseteq \{ 1\comma \, \ldots\comma \, d\}$
and
${\bar {\cal A}} \subseteq \lcub {1\comma \, \ldots\comma \, d} \rcub $
) nonconstant component functions that are not hierarchically orthogonal.
When the input variables involve dependent probability distributions, we require a triplet of global sensitivity indices (Rahman, Reference Rahman2014). The three
${\cal A}$
-variate global sensitivity indices of
$f_{\cal A}$
, where
${\cal A} \subseteq \{ 1\comma \ldots\comma d\}$
, are denoted by
$S_{{\cal A}\comma v}$
,
$S_{{\cal A}\comma c}$
, and
$S_{\cal A}\comma $
, and are defined by the ratios
$$S_{{\cal A}\comma v} = {{\open E}_X[\,f_{\cal A}^2 ({\bf t}_{\cal A})] \over {\rm \sigma ^2}}\comma $$
$$\hbox{S}_{{\rm \cal A}\comma \,{\rm c}} = {\rm \;} \displaystyle{{\mathop \sum \nolimits_{\matrix{ {0 \ne {\bar {\rm \cal A}} \subseteq \{ 1\comma \, \ldots\comma \, d\}} \cr {{{\rm \cal A} {\nsubseteq} {\bar {\cal A}} {\nsubseteq} {\cal A}\;}} \cr}} {\rm \open E}_X \left[ {f_{\rm \cal A}({\bf t}_{\rm \cal A})f_{{\bar {\rm \cal A}}}({\bf t}_{{\bar {\rm \cal A}}})} \right]} \over {{\rm \sigma} ^2}}\comma \,{\rm \;}$$
and
$$S_{\rm \cal A} = S_{{\rm \cal A},v} + S_{{\rm \cal A},c}$$
The first two indices,
$S_{{\cal A}\comma \,v}$
and
$S_{{\cal A}\comma \,c}$
, represent the normalized versions of the variance contribution from
$f_{\cal A}$
to σ2 and of the covariance contributions from
$f_{\cal A}$
and all
$f_{\;{\bar {\cal A}}}$
, such that
${\cal A} \, {\not\subseteq}\, \;{\bar {\hskip-3pt\cal A}} \not\subseteq {\cal A}$
, to σ2. They are termed the variance-driven global sensitivity index and the covariance-driven global sensitivity index, respectively, of
$f_{\cal A}$
. The third index,
$S_{\cal A}$
, referred to as the total global sensitivity index of
$f_{\cal A}$
, is the sum of variance and covariance contributions to σ2 that are associated with
$f_{\cal A}$
. Summing over all of the triplet global sensitivity indices adds up to
$$\mathop \sum \limits_{0 \ne {\cal A} \subseteq \{ 1\comma \ldots\comma d\}} S_{{\cal A}\comma v} + \mathop \sum \limits_{0 \ne {\cal A} \subseteq \{ 1\comma \ldots \comma d\}} S_{{\cal A}\comma c} = \mathop \sum \limits_{0 \ne {\cal A} \subseteq \{ 1\comma \ldots\comma d\}} S_{\cal A} = 1.\; $$
We use these sensitivity indices to identify the most influential AEDT input variables on the AEDT output of interest. We construct the generalized ANOVA dimensional decomposition of the AEDT component using first-order basis functions (i.e.,
${\cal A} = \lcub i \rcub $
) with at most first-degree polynomials as the subcomponent functions. The AEDT random input variable is distributed according to a multivariate Gaussian distribution where the mean and covariance terms are computed using a subset of TASOpt output realizations. The reason for using TASOpt output realizations is because we did not have adequate experience or prior knowledge of this system to accurately guess these parameters; instead, a few TASOpt realizations provide an informed estimate. With the subcomponent functions in hand, we compute the absolute maximum allowable variation (i.e.,
$\vert {S_{{\cal A}\comma \,v}} \vert + \vert {S_{{\cal A}\comma \,c}} \vert $
) due to each AEDT random input variable. Using this criterion, we rank the AEDT random input variables in decreasing order of influence on the AEDT output of interest, as illustrated in Figure 4. These results confirm that only a small subset, here just 15 AEDT random input variables, of the original d AEDT random input variables have a substantial influence on the AEDT output of interest variation.
Fig. 4. This plot presents the absolute variance- and covariance-driven sensitivity indices from each of the Aviation Environment Design Tool (AEDT) random input variables on the AEDT fuel consumption performance variance. This plot illustrates that the absolute variance- and covariance-driven sensitivity indices decay rapidly and that 15 AEDT random input variables capture almost all of the AEDT fuel consumption performance variance.
At this stage in the analysis, we could approximately identify which of the TASOpt inputs contribute the most to the variation of the important variables presented in Figure 4 by performing a global sensitivity analysis of TASOpt with respect to the 15 important variables. However, as our system is only composed of two components, there is no need to do this. We can complete the uncertainty quantification study as is, through the computationally efficient procedure described in the next section. If our system were composed of more than two components, then one could possibly justify performing global sensitivity analysis on more than one component to reduce the coupling interface dimensions across two or more components.
4. UNCERTAINTY QUANTIFICATION
Applying the results from Section 3, we reduce the multicomponent coupling interface and decompose the multicomponent system as illustrated in Figure 5. Consequently, instead of performing the uncertainty quantification of the multicomponent system presented in Figure 1, we perform a decomposition-based uncertainty quantification of the reduced multicomponent system presented in Figure 5.
Fig. 5. We partitioned the Aviation Environment Design Tool (AEDT) input variables into two sets: an influential set and a noninfluential set. The noninfluential set contains AEDT input variables that were deemed not to influence, by the AEDT component global sensitivity analysis, the AEDT output of interest. The influential set contains AEDT random input variables that were labeled as influential by the AEDT component global sensitivity analysis.
4.1. Uncertainty analysis
As illustrated on the left in Figure 6, we wish to perform the multicomponent system uncertainty analysis that propagates uncertainty in system inputs to uncertainty in system outputs. To tackle the complexity of a multicomponent uncertainty analysis, we propose to decompose the system uncertainty analysis into individual component-level uncertainty analyses that are then assembled into the desired multicomponent system uncertainty analysis (Amaral et al., Reference Amaral, Allaire and Willcox2014). The decomposed system uncertainty analysis is illustrated on the right in Figure 6. The decomposition-based uncertainty analysis approach comprises two main procedures, which are illustrated in Figure 7: local uncertainty analysis, performing a local Monte Carlo uncertainty analysis on each component using their respective proposal distributions; and global compatibility satisfaction, resolving the coupling among the components without any further evaluations of the components or of the system as a whole.
Fig. 6. The proposed method of multicomponent uncertainty analysis decomposes the problem into manageable components, similar to decomposition-based approaches used in multidisciplinary analysis and optimization, and synthesizes the system uncertainty analysis without needing to evaluate the system in its entirety.
Fig. 7. The process depicts the local uncertainty analysis and global compatibility satisfaction for our two component system. First, local uncertainty analysis is performed on each component. Second, global compatibility satisfaction uses importance sampling to update the proposal samples so as to approximate the target distribution. Here we use X and Y to represent the proposal and target random variables, respectively.
In the offline phase, demonstrated on the top in Figure 7, each local uncertainty analysis is performed concurrently for each component. The challenge created by decomposition is that the distribution functions of the inputs for each component are unknown when conducting the local uncertainty analysis. Therefore, we propose an initial distribution function for each component input, which we refer to as the proposal distribution function. Local uncertainty analysis uses the proposal distribution function to generate samples of the uncertain component inputs and propagate them through the component analysis to generate corresponding samples of component outputs.
In the online phase, demonstrated on the bottom in Figure 7, we learn the true distribution function of the inputs of each component. We refer to these true distribution functions as the target distribution functions. For those component inputs that correspond to system inputs, the target distribution functions represent the particular specified scenario under which we wish to perform the system uncertainty analysis. For those component inputs that correspond to coupling variables (i.e., they are outputs from upstream components), the target distribution functions are specified by the uncertainty analysis results of the corresponding upstream component(s).
Global compatibility satisfaction is ensured by starting with the most upstream components of the system and approximating their respective target distribution functions using importance sampling on the corresponding proposal distribution functions. The approach we implement for the importance sampling step entirely avoids the probability density function and works with the well-defined and determinable empirical distribution function associated with the random samples (Amaral et al., Reference Amaral, Allaire and Willcox2016). A key attribute of the approach is its scalability: it lends itself well to handling a large number of samples through a scalable optimization algorithm. The approach also scales to problems with high-dimensional distributions, an important property we use in this engineering example. The updated output importance weighted samples of the upstream component then define the target distribution function for the downstream component. The process of importance sampling is repeated for the downstream component, resulting in the downstream components importance weighted output samples. The downstream components importance weighted output samples characterize the multicomponent systems outputs of interest uncertainty under the target distribution and are used to quantify the desired statistics of interest.
The proposal distribution selected for the AEDT component is the same distribution used to construct the generalized ANOVA dimensional decomposition in Section 3 with the covariance term multiplied by a factor of three. The proposal covariance term was multiplied by a factor of three to ensure the unknown target distribution from the upstream TASOpt component is supported by the AEDT input proposal distribution. In the situation that the input proposal distribution does not support the forthcoming target distribution, we must resample the AEDT module using the latest information from the forthcoming target distribution. The uncertainty analysis results are presented in Figure 8. The results illustrate the system output of interest distribution function under three different scenarios. The first scenario is the outcome of running the AEDT component under the proposal distribution assumption. The second scenario is produced by performing an all-at-once uncertainty analysis of the system illustrated in Figure 1 using Monte Carlo simulation. The last scenario is the result of performing a decomposition-based uncertainty analysis of the system illustrated in Figure 5.
Fig. 8. The Aviation Environment Design Tool (AEDT) fuel consumption performance distribution is shown, on the left, using the AEDT output proposal distribution, all-at-once Monte Carlo uncertainty analysis, and the decomposition-based uncertainty analysis. On the right are the resulting probability density functions. These results suggest that our decomposition-based uncertainty analysis performed adequately, which implies the change of measure across the 15 AEDT random input variables was successful and that the correct 15 AEDT random input variables were selected by the AEDT component-level global sensitivity analysis.
These results demonstrate that our decomposition-based uncertainty analysis of the reduced system in Figure 5 accurately produces the result from the standard approach, the all-at-once Monte Carlo simulation of the full system in Figure 1. The discrepancy between the decomposition-based approach and the all-at-once Monte Carlo simulation can be attributed to performing the importance sampling on a subset of the coupling variables using only a finite number of samples. The results presented imply that we successfully identified the 15 influential AEDT random input variables and also that we accurately performed the importance sampling procedure across the resulting 15-dimensional interface between the TASOpt component and the AEDT component. By reducing the dimensions of the multicomponent interface, we illustrated that our decomposition-based uncertainty analysis approach can be extended to calculate the relevant statistics and failure probabilities of complex and high-dimensional systems.
4.2. Global sensitivity analysis
The purpose of a global sensitivity analysis is to identify how the variability in a system output quantity of interest is related to a system input and which of the system input sources dominate the response of the system output. Our motivation for a system-level global sensitivity analysis is research prioritization: which system input factor is the most deserving of further analysis or measurement? To address our objective, we apply the Sobol variance-based global sensitivity analysis method, which quantifies the amount of variance that each input factor contributes to the unconditional variance of the output (Saltelli et al., Reference Saltelli, Ratto, Andres, Campolongo, Cariboni, Gatelli, Saisana and Tarantola2008).
To perform the global sensitivity analysis of the multicomponent system, we apply the generalized ANOVA framework presented in Section 3 to this context, where f now represents the entire multicomponent system. Because the multicomponent system inputs are independently distributed, we can discard the covariance contribution in Eq. (9) from our
${\cal A}$
-variate global sensitivity indices. The resulting global sensitivity indices of the multicomponent system,
$$\; 1 = \mathop \sum \limits_{i = 1}^d S_{\{ i\}\comma \, v} + \mathop \sum \limits_{i \lt j}^d S_{\{ i\comma \,j\}\comma \, v} + \cdots + S_{\lcub {1\comma \,2\comma \ldots\,\comma d} \rcub \comma \,v}\comma \,$$
apportion the output variance among the system inputs. Global sensitivity indices with only a single subscript (e.g., S {i}) are called main effect indices. Inputs with large main effect indices are known as the “most influential factors,” or the inputs that, on average, once fixed, would result in the greatest reduction in variance. Global sensitivity indices with multiple subscripts (e.g., S {i,j}) are called interaction effect indices.
In this context, we cannot construct a generalized ANOVA dimensional decomposition of the multicomponent system because this would result in having to evaluate the entire multicomponent system. Instead, to evaluate the global sensitivity indices using a decomposition-based approach, we first use a restated but equivalent definition of the global sensitivity indices. For example, the main effect indices can be restated using the conditional and unconditional variances,
$$S_i = \displaystyle{{\hbox{va}\hbox{r}_{X_i}({\open E}_{X_i^c} [\,f_{\{ i\}} ])} \over {{\rm \sigma} ^2}}\comma \,\; $$
where X
i
is the ith component of the vector of random variables X, and
$X_i^c $
is the vector containing all components of X except X
i
(Saltelli et al., Reference Saltelli, Ratto, Andres, Campolongo, Cariboni, Gatelli, Saisana and Tarantola2008).
As Eq. (11) requires evaluating the expectation of f conditioned on random variables, we cannot yet directly apply a decomposition-based methodology in a straightforward way. Instead, we will approximate Eq. (11) by evaluating the expectation of f conditioned on the random variable existing in a finite range. By relaxing the conditional dependence in this way, we can evaluate the expectations contained within Eq. (11) using our decomposition-based uncertainty analysis algorithm, because expectation is just a statistic of interest. To construct these conditional sets of finite range, we partition the input space into a finite number of bins (Saltelli et al., Reference Saltelli, Ratto, Andres, Campolongo, Cariboni, Gatelli, Saisana and Tarantola2008). For each bin, we evaluate the expectation contained in Eq. (11) using our decomposition-based uncertainty analysis algorithm and modifying the system input target distribution to be contained in the bin of interest. After evaluating the expectations over each bin, we can evaluate the variance over those expectations to approximate Eq. (11). This procedure relies only on the offline step of our decomposition-based uncertainty analysis algorithm. Therefore we can repeat these steps for each system input without having to evaluate the components comprising the system.
The results of the system-level main sensitivity indices are presented in Figure 9. The results depict the system-level main sensitivity indices computed using the all-at-once Monte Carlo simulation approach of the system illustrated in Figure 1 and the decomposition-based approach of the system illustrated in Figure 5. These results confirm that our decomposition-based sensitivity analysis algorithm can accurately quantify the main sensitivity indices of the system in Figure 1. As previously mentioned, the decomposition-based sensitivity analysis algorithm hinges on the fact that we can evaluate the decomposition-based uncertainty analysis. The main sensitivity indices suggest that the output of interest variation is mostly dominated by a handful of aircraft technological and operational variables. That is, by improving our understanding of these TASOpt random input variables through research, we can reduce the variation of the overall system output of interest, which is beneficial for decision making and policymaking.
Fig. 9. The decomposition-based main sensitivity index and all-at-once system-level Monte Carlo main sensitivity index. These results suggest that our decomposition-based global sensitivity analysis performed adequately and that only a handful of technological and operational system input variables have a significant influence, on average, on the system output of interest. A description of the system inputs are provided in Table 1.
5. CONCLUSION
This paper presents uncertainty quantification of a realistic application problem involving a complex multicomponent system model of the environmental impacts of aviation technologies and operations. The multicomponent system comprises a conceptual-level aircraft design tool and an environmental impacts tool. The challenges of applying uncertainty quantification to this multicomponent system include long computational run times, a high-dimensional component-to-component interface, and a lack of software integration among the system components. These challenges are overcome through a combination of dimensionality reduction and decomposition. A component-level global sensitivity analysis identifies the most influential component-to-component interface variables and permits reduction of the dimensionality of the component-to-component interface. With this reduction in dimensions, a decomposition-based uncertainty quantification approach becomes feasible. This permits local uncertainty analyses to be conducted, and the results synthesized to compute system-level uncertainty estimates and system-level sensitivity indices. The results reveal the most important sources of uncertainty across the system, which informs policy and decision making as well as future tool development. Future work will extend these ideas to different and more complex architectures of feed-forward and feed-back multicomponent systems. This will require accounting for dependency among variables using only the samples but with no explicit description of their underlying probability density functions. An additional future direction includes performing decomposition-based uncertainty quantification with a goal-oriented objective. The aim in this case is to minimize the system complexity while approximating the quantity of interest to within a specified tolerance.
ACKNOWLEDGMENTS
The work presented in this manuscript was funded in part by the United Stated Federal Aviation Administration Office of Environmental and Energy Award Number 09-C-NE-MIT, Amendment Numbers 028, 033, and 038, and by the MIT-Singapore University of Technology and Design International Design Center.
Sergio Amaral is a Quantitative Researcher at Millennium Management LLC. He is a former Postdoctoral Associate of the Aerospace Computational Design Lab at Massachusetts Institute of Technology (MIT). He completed his doctoral studies in uncertainty quantification of multicomponent systems in 2015 under the supervision of Professor Karen E. Willcox. He attained BS and MS degrees in aerospace engineering at Pennsylvania State University. Dr. Amaral is currently pursuing uncertainty quantification and optimization with application to financial instruments.
Douglas Allaire is an Assistant Professor in the Mechanical Engineering Department at Texas A&M University. He attained SB, SM, and PhD degrees in aerospace engineering from the Department of Aeronautics and Astronautics at MIT, followed by a Postdoctoral Associate position and then a Research Scientist position in the Department of Aeronautics and Astronautics at MIT. He has held that position since January 2014. His current research focuses on the development of computational methods for the analysis, design, and operation of complex systems. Dr. Allaire is specifically interested in aspects of uncertainty quantification, multidisciplinary design optimization, and compositional methods for simulation-based design.
Elena De la Rosa Blanco is a Research Staff Member at IDA’s Science and Technology Policy Institute. She obtained her PhD from the Whittle Laboratory at the University of Cambridge in secondary flows in low pressure turbines and her BS and MS from Escuela Técnica Superior de Ingenieros Aeronáuticos in Madrid. She has worked in academia at the University of Cambridge, MIT, and Universidad Politécnica de Madrid; in industry at Aerodyne Research Incorporated and Industria de Turbopropulsores; and in government as a contractor for the FAA Office of Environment and Energy. She has experience in the areas of turbomachinery, aviation noise, aircraft instrumentation, airframe–engine integration, and aircraft design for the environment. Dr. De la Rosa Blanco is currently working on space policy and technical related issues.
Karen E. Willcox is a Professor of aeronautics and astronautics at MIT. She is also Co-Director of the MIT Center for Computational Engineering and formerly the Associate Head of the MIT Department of Aeronautics and Astronautics. Before joining the faculty at MIT, she worked at Boeing Phantom Works with the Blended-Wing-Body Aircraft Design Group. Dr. Willcox is currently Co-Director of the Department of Energy DiaMonD Multifaceted Mathematics Capability Center on Mathematics at the Interfaces of Data, Models, and Decisions, and she leads an Air Force MURI on optimal design of multiphysics systems. She is also active in education innovation, serving as Co-Chair of the MIT Online Education Policy Initiative, Co-Chair of the 2013–2014 Institute-Wide Task Force on the Future of MIT Education, and Lead of the Fly-by-Wire Project developing blended learning technology as part of the Department of Education's First in the World program.
