3.1. VDD of SoS

The literature on SE emphasizes that as systems grow in complexity, defining the system boundary, that is, the line distinguishing the system of interest from its context, becomes increasingly ambiguous. To maintain a consistent perspective during the design stages, Kossiakoff & Sweet (Reference Kossiakoff and Sweet2003) propose “development control” as a definition criterion, wherein the parameters outside the engineering team’s control yet impacting the system’s operational value are considered context. Thus, the operational scenario encompasses all the forms of the system of interest, that is, the system being developed along with all possible contexts (Machchhar et al. Reference Machchhar, Toller, Bertoni and Bertoni2022). This viewpoint is preserved throughout this article.

In SE projects, the term SoS has gained significant traction over the past two decades, where the system is often regarded as a part of a more extensive integrated system known as the SoS. An SoS integrates independent, geographically dispersed systems, even those with different technologies, to achieve a unified goal through coordinated interaction (Walden et al. Reference Walden, Roedler, Forsberg, Hamelin and Shortell2015). An SoS is a solution for achieving novel capabilities that its constituent systems cannot achieve in isolation (Silva & Braga Reference Silva and Braga2020). Rather than a rigorous universal definition of SoS, the SoS community identifies key characteristics that differentiate SoS from traditional large-scale systems. As per Maier (Reference Maier1998), these five characteristics set SoS apart from traditional systems: (1) operational independence, (2) managerial independence, (3) geographic distribution, (4) emergent behavior and (5) evolutionary development of the system or components. In such a context, operational and managerial independence highlight that each constitutive system of an SoS shall be capable of operating independently from all other constitutive systems. SoS engineering must consider the design of all constitutive systems and their interactions (Keating et al. Reference Keating, Rogers, Unal, Dryer, Sousa-Poza, Safford, Peterson and Rabadi2003). Geographic distribution suggests that the constitutive system can be positioned at different locations, yet connected to functions as a single interconnected system. Emergent behavior implies that the behavior of an SoS can be more complex than the sum of its individual parts, stressing the need to study interactions and interconnections between constitutive systems, down to the subsystems and components in the system hierarchy, is of prime significance. Finally, evolutionary development means that the SoS can evolve over time as new technologies, requirements or contexts emerge. This necessitates an adaptable architecture, making the design process a multistage decision-making loop involving adding new systems, replacing or removing existing ones and constantly optimizing the network of interdependencies (Fang Reference Fang2022).

Within the realm of SE, a widely adopted framework for informed decision-making during the design stages is VDD (Lee & Paredis Reference Lee and Paredis2014). This approach aims to enable the selection of the “best” overall design from a more holistic view rather than designs that merely fulfill the requirements (Collopy & Hollingsworth Reference Collopy and Hollingsworth2011; Cheung et al. Reference Cheung, Scanlan, Wong, Forrester, Eres, Collopy, Hollingsworth, Wiseall and Briceno2012). To enable the selection of the best designs, a clear definition of value is often necessary to quantify the relative goodness of one design compared to the others. Thus, while the concept of value is multifaceted, it is helpful to view it as a numerical score reflecting an individual’s preferences (Collopy & Hollingsworth Reference Collopy and Hollingsworth2011). VDD aims to achieve the highest score for this value metric instead of optimizing each design parameter and requirement in isolation (Cheung et al. Reference Cheung, Scanlan, Wong, Forrester, Eres, Collopy, Hollingsworth, Wiseall and Briceno2012). This also means that it is possible to compare more widespread design options and concepts (Bertoni et al. Reference Bertoni, Bertoni and Murat Hakki2019) and easier alignment between stakeholders and partners in complex systems (Monceaux & Kossmann Reference Monceaux and Kossmann2012). Thus, one of the essential tasks is to define “value” so that a single numerical score can represent it. As per the value management standard, value is seen as a measure of how well the inferred design meets the desired requirements or objectives in relation to its cost (EN12973 2020). Value functions can be derived to achieve this measure. A value function is, in that case, the overall score of the design based on the fulfillment of needs and expectations concerning expected costs. From this point of view, VDD can be seen as the aggregation of different performance metrics into a unified metric. The value function itself can adopt a qualitative form, such as “early value-oriented design exploration with knowledge maturity” (Bertoni, Bertoni & Isaksson Reference Bertoni, Bertoni and Isaksson2018), or a quantitative form, such as net present value (Price et al. Reference Price, Soban, Mullan, Butterfield and Murphy2012).

VDD can also be extended to SoS, although it has an increased challenge of identifying and modeling relevant attributes. A recent review (Fang Reference Fang2022) highlighted several challenges in developing SoS, particularly because of the interdependency and heterogeneity of constituent systems in an SoS. Each constituent system in an SoS may be built using different technologies, have diverse data formats and follow different operational rules. Thus, a unified model that effectively captures interdependencies is usually a trade-off between granularity and computational cost. At the same time, these interdependencies and collaboration between constituent systems often make an SoS more valuable than deploying the constituent systems independently (Silva & Braga Reference Silva and Braga2020). From an SoS modeling perspective, it is crucial to sufficiently model the SoS and its internal dependencies to achieve a reliable representation of value creation. The following section sheds some light on different modeling methods for SoS, particularly from an operational perspective.

3.2. Modeling and simulating approaches for SoS

Modeling and simulation are a better choice when real-world prototyping is expensive or practically impossible (Borshchev & Filippov Reference Borshchev and Filippov2004) and are considered a core part of the design process today. The aim of a simulation is to reflect reality as well as possible, allowing design teams to try different concepts and scenarios more rapidly without the need for physical intervention. Within SoS, Kinder et al. (Reference Kinder, Henshaw and Siemieniuch2014, p. 151) state that “the failure of many SoS endeavors can be attributed to the inappropriate application of systems engineering processes, including modelling approaches, within the SoS domain because of the mistaken belief that a SoS can always be regarded as a single large, or complex, system.” An SoS is evolutionary, with its structures, functions and purposes added, removed or modified as the experiences and community grow and evolve (Jamshidi Reference Jamshidi2008). This means that SoS requires simulation and modeling approaches capable of effectively capturing its unique characteristics.

Several different simulation approaches have been proposed for SoS. These can be broadly categorized into network-based, probability-based, simulation-based or other methods (Fang Reference Fang2022). Each method category has its specific application area, inherent advantages and disadvantages. Network-based methods use graph theory to show interconnections between a set of entities, and in this case, the hierarchical chain constituting the SoS. By representing an SoS as a network, one can analyze its overall behavior from a high-level view (Fang Reference Fang2022). For instance, Thacker, Pant & Hall (Reference Thacker, Pant and Hall2017) used a multilevel network for disruption analysis in large-scale infrastructure. Notably, such approaches focus on the connections between entities and their statistical properties, like how often they interact or how critical they are to information flow. Probability-based methods include methods like Bayesian networks, Markov chains and so forth. A Bayesian network is a probabilistic graphical model in the form of a directed acyclic graph that describes the interconnections for an SoS; for example, Han & DeLaurentis (Reference Han and DeLaurentis2013) employed Bayesian networks to measure the interdependencies and evaluate risks at an SoS level. A Markov chain is a special kind of network graph describing a sequence of possible events, where the nodes represent states of a system, and the edges show the probability of moving between those states. The key aspect is that the transition probability to the next state only depends on the current state, making them memoryless. Mane, DeLaurentis & Frazho (Reference Mane, DeLaurentis and Frazho2011) used the Markov chain model to better understand dependencies in a network of systems. Network graphs have also been explored in other problems, such as measuring and analyzing complexity, optimization of constrained SoS, and so forth (Harrison Reference Harrison2016). However, a higher level of abstraction is generally observed while utilizing these techniques and, hence, they are usually computationally less expensive.

Simulation-based methods are preferred to understand how a SoS behaves over time, predict its overall performance or explore potential operational scenarios. In this approach, computer simulations are utilized to model the behavior of SoS components and their interactions, enabling the evaluation of different design alternatives. Mainly, three different simulation techniques exist, including discrete-event simulation (DES), agent-based simulation (ABS) and system dynamics (SD) (Borshchev & Filippov Reference Borshchev and Filippov2004) or a combination of these techniques in a so-called hybrid simulation. SD is a technique where the system is modeled using stocks (accumulations) and flows (rates of change), further using feedback loops to represent change in one aspect of this system and analyze the system’s long-term behavior. Again, it has a high level of abstraction, making it unsuitable for understanding detailed-level interactions between various SoS components (Borshchev & Filippov Reference Borshchev and Filippov2004). DES is a top-down technique where the operational process is broken down into a sequence of events. These simulations require a good grasp of events and interactions, and hence, it is particularly suited for logistics and supply chain simulations. On the other hand, ABS is a bottom-up modeling technique where different agents are modeled, that is, systems, with predetermined rules and behavior and then placed in an environment/context (Siebers et al. Reference Siebers, Macal, Garnett, Buxton and Pidd2010). These agents are autonomous and possess operational and managerial independence, making them a good option for SoS engineering (Baldwin et al. Reference Baldwin, Sauser and Cloutier2015). In contrast to other modeling techniques, ABS is decentralized, meaning that the behavior is determined at an individual level, and the global behavior is thus a result of all agents’ behaviors within the simulation environment and their interaction with each other (Borshchev & Filippov Reference Borshchev and Filippov2004). This means that ABS is a good technique for discovering the unanticipated emergent behavior of a SoS (Baldwin et al. Reference Baldwin, Sauser and Cloutier2015) and modeling human-like behavior (Hester & Tolk Reference Hester and Tolk2010). The downfall with ABS is that it is more cumbersome to set up compared to other modeling techniques and struggles to manage high variabilities and emergent changes throughout the life cycle (Hester & Tolk Reference Hester and Tolk2010; Kinder et al. Reference Kinder, Henshaw and Siemieniuch2014).

Overall, ABS has high flexibility in modeling different abstraction levels, from high-level abstraction and macro level to detailed operational and micro levels (Borshchev & Filippov Reference Borshchev and Filippov2004). However, the granularity of the behaviors of each agent might differ. A simple probabilistic function or lookup table is enough for some, while others might require more advanced dynamic models. Shanthikumar & Sargent (Reference Shanthikumar and Sargent1983) present the analytic-simulation modeling approach, where analytical models are combined with simulation models to reflect reality better. This approach is good when the entire problem cannot be analytically modeled. In the context of SoS, where intricate interactions and hierarchical structures exist, purely analytical modeling is rarely feasible, even if the constituent systems might be analytically tractable. The question is then rather the degree and complexity that the systems should be modeled with respect to required granularity and computational effort.

Looking at ABS’s current efforts in managing SoS and complex systems, An et al. (Reference An, Grimm, Sullivan, Turner, Malleson, Heppenstall, Vincenot, Robinson, Ye, Liu, Lindkvist and Tang2021) push that ABS research and development must tighten the collaboration between disciplines to identify mechanisms and interdependencies affecting the SoS outcomes. Moreover, Liang et al. (Reference Liang, Luo, Hu and Li2022) identified eight research clusters in conjunction between ABS and decision-making, with the three largest being intelligent agents, model validation and collaborative decision-making. In general, ABS is recognized as a promising simulation approach for complex contexts, both in its design and emergent behaviors. Examples of this are an ABS for assessing coal mine safety in a dynamically changing context (Cheng, Guo & Lin Reference Cheng, Guo and Lin2021) and ABS for energy power distribution in networks with the insertion of renewable-based energy systems, for example, solar power (Fichera, Pluchino & Volpe Reference Fichera, Pluchino and Volpe2018). Both these contexts can be characterized as SoS, in which ABS has been selected as a suitable modeling approach. Looking specifically at agent model fidelities, Schön et al. (Reference Schön, Marcus, Amadori and Jouannet2021) studied the effect of combining high-fidelity models with ABS. It showed that adding modeling complexity and high-fidelity models affects the performance of the simulation model, but it also prolongs the time for results to converge, stressing the need for a strategic fidelity selection. All the abovementioned examples show that ABS can be a powerful and flexible approach for simulating SoS and capturing its unique characteristics. However, the question still remains of how this can be done effectively, especially with respect to fidelity.

5.1. Problem formulation

The first step in the process was to better frame the challenge in the electric-autonomy transition for mines and quarries. This was achieved through a collaborative effort between researchers and practitioners. Figures 4 and 5 depict the quarry process, specifically the quarry operations, from the extraction of blasted rock to the final stage of crushed material production. The process description is developed using the IDEF0 functional modeling method. Since the hauling of material (Step A3) is of primary interest, that step has been expanded.

Figure 4. IDEF0 diagram of quarry operations.

Figure 5. IDEF0 diagram of rock transportation.

A causal loop diagram was developed alongside the process model to understand the complex interactions within the simulation (refer to Figure 6). This diagram maps the internal and external dependencies between different elements of the included systems. It visually shows how changes in one element can affect others within the SoS. Positive (+) and negative (−) signs on the connections indicate whether an increase/decrease in one element has a beneficial or detrimental effect on the other. Ultimately, this diagram, together with the process flow in Figure 4, helps determine the appropriate level of fidelity needed for each simulated agent and whether an agent type should be considered.

Building on the work of Toller Melén & Watz (Reference Toller Melén, Watz, Camarinha-Matos, Boucher and Ortiz2023), the causal loop diagram was developed using a weighted value aggregation model. This model recognizes that different performance metrics contribute to overall value creation to varying degrees. Thus, weights were assigned to each metric in the aggregation, reflecting their relative importance. The value dimensions and their weights were determined collaboratively and then normalized for consistency. The thickness of the arrows in the causal loop diagram visually represents the weight assigned to each connection. The top section of Figure 6 identifies three key value aspects for a quarry (yellow bubbles): productivity, sustainability and utilization. Design objectives or performance metrics are then linked to each value dimension, explaining how these objectives contribute to achieving that value (green bubbles). Finally, different parts of the quarry operation are mapped to these design objectives, showing how they influence overall value delivery. The diagram reveals that an electric-autonomous quarry has multiple interdependencies, with context (e.g., road conditions and weather) and haulers having the most substantial influence. This makes sense as haulers are the case study’s focus on the hauling process in a mine.

Figure 6. Causal loop diagram and the weighted aggregation for value for the quarry scenario.

5.2. Value function definition

Soban, Price & Hollingsworth (Reference Soban, Price and Hollingsworth2012) emphasize that the value function is the most central aspect of a VDD approach and essential for achieving comparability between dispersed concepts. The case study utilizes the causal loop diagram, where performance metrics, value dimensions and their respective weighting have been derived. The value aggregation further allows for stepwise evaluation of the value creation and thus enables a design team to obtain both width and depth in the analysis. Equations (1) and (2) illustrate how the overall value is calculated:

(1) $$ V={\sum}_{i=1}^n{w}_i{D}_i $$

(2) $$ {D}_i={\sum}_{j=0}^m{P}_j $$

Here, $ V $ indicates the overall value, $ w $ denotes the weights used for aggregation, $ D $ implies a value dimension and $ P $ implies a performance metric. The weights assigned to each value dimension reflect their relative importance in determining overall value. This weighting is crucial because the value dimensions (i.e., cost, sustainability and utilization in Figure 6) use normalized units that fit into the value function. Normalization ensures that the value dimensions range between 0 and 1, while the weights reflect their relative importance to the overall value. Such weights may also be applied to the performance metrics. However, since all performance metrics used to calculate a specific value dimension share the same unit, no additional normalization and weighting are needed within those dimensions, which is also a preferable option as it allows for better transparency.

For this case study, the weighting numbers are selected for illustrative purposes. The weights for the value dimensions are cost (0.5), sustainability (0.25) and utilization (0.25). Table 1 details all performance metrics and the formulas used to derive them. It is noteworthy that several aspects have not been included in the performance metrics, for example, connectivity costs, emissions from manufacturing and so forth. Only those considered most critical for the selection of design options have been included to maintain the simplicity of the model in this early stage.

Table 1. Performance metrics and their formulas

In the performance metrics, $ {C}_{xxx} $ represents the cost, $ {E}_{xxx} $ represents the energy consumed, $ {S}_{xxx} $ represents the emissions, $ {U}_{xxx} $ represents the utilization and $ {Q}_{xxx} $ the queuing time at the respective factions is denoted by $ xxx $ in the suffix. $ {N}_{SoS} $ is the total number of haulers in the SoS, $ \Delta SoH $ is the difference in the battery’s state of health (SoH) after the operational period and $ {t}_{model} $ is the model time. All operational costs are extrapolated, and all fixed costs are divided to represent the total annual cost.

5.3. Simulation setup using modular scenario builder

The integrated simulation framework can now generate a baseline and repository for operational scenario generation based on the case study once the process flow and system interdependencies are established through the process description and causal loop diagram. The operational scenarios incorporate both $ \mathrm{SAs} $ and $ \mathrm{CAs} $ , as explained in Section 4. Adjacent $ \mathrm{EA} $ s are created to support the simulation execution. Furthermore, the integrated simulation framework provides access to a prebuilt library of elements used to build the virtual world. This allows for creating an environment that serves as the framework for agent interaction within the simulation.

Interactions between agents or the environment involve exchanging information, allowing the operational scenario to evolve as the simulation unfolds. The fidelity assigned to each agent is determined by its impact on the overall system and its internal complexity (estimated using the interdependencies identified in the causal loop diagram). Figure 7 depicts the integrated simulation framework with the selected fidelity levels developed for the case study.

Figure 7. Integrated simulation framework with the chosen fidelity levels.

The $ \mathrm{SAs} $ modeled in the simulation are as follows:

• • Hauler: Haulers (or load receivers) are commonly used vehicles at a quarry to transport material from one point to another. From an energy consumption standpoint, once the propulsion models are established for such a vehicle system, they can be analyzed with varying levels of accuracy depending on the computational resources available (Guzzella & Sciarretta Reference Guzzella and Sciarretta2007). The simplest is the average operating point method, which offers a quick but rough estimate by averaging all operating conditions. The quasi-static approach improves accuracy by dividing the operational cycle into segments with constant parameters, and recursive techniques like dynamic programming can then be used to calculate the global optimum. The most precise method is the dynamic approach, which utilizes differential equations to capture the system’s dynamic behavior. The quasi-static approach was chosen since the hauler is the main constituent of the SoS. A dynamic model may not be tractable as the simulation spans more extended periods with numerous variables. Dynamic programming calculates the optimal velocity profile for each movement based on a variable called “time penalty.” The time penalty acts as a tuning knob, adjusting the balance between minimizing energy consumption and minimizing travel time during the control optimization. This allows for control policies prioritizing energy efficiency, time efficiency or a balance of both.

• • Charger: Charger is a generic type assigned to stations that replenish energy in vehicles in the desired format, that is, the charging poles or fuel pumps. Thus, they could recharge or refuel stations based on the arrival of the vehicle. Since this case study examines electrical haulers, the recharging method is predominantly used. Different levels of fidelity can be used; for instance, a linear model assuming a constant rate, a mixed switch model where the charger switches from constant-current mode to constant voltage mode at given battery state of charge (SoC) thresholds (Wu et al. Reference Wu, Pang, Choy and Lam2018), or a more detailed nonlinear model considering temperature variations (Senol et al. Reference Senol, Bayram, Naderi and Galloway2023). This work chooses a linear model as the ΔSoC is often constrained in the linear region, and higher granularity is deemed unnecessary. The time required to recharge the battery is calculated based on two critical inputs: the power output and the battery specifications. For diesel machines, a simple fixed flow rate is used.

• • Control tower: The control tower is the backbone of the quarry process. It may be instantiated at global and local levels. Global control towers oversee the entire production line, ensuring smooth workflow and optimizing resource allocation across the whole operation. On the other hand, local control towers focus on specific stations, managing queues and minimizing wait times for haulers by coordinating the local processes. This multilayered control system performs several functions in the operational scenario, such as keeping track of the SoC window for all machines, assigning different time penalties and so forth, and dictates specific agent interactions based on the user-defined logic. For instance, a hauler may be commissioned to a wheel loader or a charger based on availability and SoC status. Parts of the logic are based on static rules such as FIFO queuing, while others use more dynamic logic, for example, charging policy.

The $ \mathrm{CA} $ s modeled in the simulation are:

• • Wheel loader: The quarry operation relies heavily on wheel loaders. These are powerful machines with a large bucket at the front end. Their primary function is loading haulers, but they can also be quite versatile, tackling tasks like feeding crushers or sorting stockpiles. In this case study, wheel loaders are beyond the development control. Hence, they are considered as context and modeled using $ \mathrm{CAs} $ in the modular scenario builder. Analogous to haulers, such a vehicle system can be analyzed at various fidelity levels. The average operating point approach is chosen to save computational resources. Probabilistic functions define the time required to perform the steps in short loading cycles (Frank, Kleinert & Filla Reference Frank, Kleinert and Filla2018), such as approaching the pile, loading the bucket, retracting, approaching the hauler, dumping the bucket and again retracting. The energy consumed is averaged based on the utility.

• • Crush feeder: Haulers unload their payload into a crush feeder. This feeder acts as a buffer, regulating the flow of blasted material to the crusher at a rate that matches the desired production output. However, the crush feeder has a limited capacity. This means a hauler might need to wait to unload if the feeder is currently full and cannot accommodate the entire payload. To prevent equipment malfunctions, the crusher itself cannot operate empty. This creates a balancing act between keeping the feeder from overflowing and ensuring the crusher has a steady material supply. Given the unchangeable nature of the feeder’s behavior in this study, a simple linear function is used within the simulation to model its performance.

In addition, the environment library is a comprehensive repository for all data related to the physical world where agents operate. This data encompasses route properties (distances, elevation, curvature), ambient temperature, weather conditions and so forth. Since no behavioral dynamics of the environment are needed in this case study, they are modeled as static objects for agent interaction.

The simulation can be initialized with the agent being modeled. Repast Symphony (https://repast.github.io/), an open-source and Java-based agent-based modeling toolkit, emerged as the ideal choice due to several advantages. First, its Java foundation ensured seamless integration with MATLAB®, the software used for high-fidelity dynamics simulation within the project. Second, Repast Symphony offered a high degree of flexibility by its ability to import any Java library to support the simulation model. Synchronously with the agent development from Figure 7, some additional agents were created and modeled to support the simulations. Referred to as $ \mathrm{EA} $ s, these agents are not part of the operational scenario but are essential for the simulation execution itself:

• • Data collector: Logging simulation data is crucial to effectively evaluate the value of the SoS in different operational scenarios. A dedicated data collector agent was built to capture vital metadata (e.g., configuration details), results data specific to the SoS (e.g., energy consumption and battery health metrics) and statistics data relevant to the operational scenario (e.g., material production, cycle counts and queue times). These data are further used to assess the SoS value of each scenario and inform future refinements through iterative analysis. The $ \mathrm{SAs} $ and $ \mathrm{CAs} $ might also be equipped with virtual data sensors, for example, GPS trackers or SoC monitoring, which transmit the sensed value to the data collector for logging and storage.

• • MATLAB Engine: As MATLAB® is used for more advanced dynamics modeling, a separate MATLAB Engine agent was created to be called whenever necessary.

Building trust in the simulation’s ability to replicate real-world quarry operations required a thorough verification process. This step involved leveraging extensive data from a real quarry provided by the partner company and their expertise. Both machine and site models were verified. Machine models were compared against the partner’s existing models and field data to confirm their behavior accurately reflects reality. The site was verified against the company partner’s operational site to ensure the ABS model evolves as expected. With verification done, the simulation model was ready to run the experiments and explore different operational scenarios.

5.4. Experiment setup

These experiments aimed to evaluate the performance of the SoS (comprising a fleet of haulers and supporting infrastructure) under various operational contexts. The following key design and contextual variables were chosen for the experiment:

• • SoS options: Experiments explored a range of fleet configurations by manipulating vehicle payload capacity (25, 35 and 45 tons), engine type (diesel or electric), control policy or driving style (eco-mode and speed-mode), number of vehicles (4, 5 and 6) and charging station capacity (1 or 2 poles). These options together resulted in 72 unique SoS combinations.

• • Context options: Three distinct virtual sites were chosen to represent typical quarry industry topologies, including hilly, curvy, and long-haul environments. Road friction (0.03, 0.04 and 0.05) and ambient temperature (−3, 12 and 27 °C) were incorporated as contextual variables to enhance complexity further. These options together resulted in 27 different contextual scenarios.

The combination of 72 SoS configurations and 27 contextual scenarios yielded a total of 1944 operational scenarios to be simulated. The selection of the SoS configuration is mainly for illustrative purposes. The framework has no theoretical limit in terms of configurations and scenarios, yet a practical limit is typically available in computing capacity and time. In addition, parameters that remained fixed throughout the simulation were introduced in the experiment, denoted in Table 2. Operating at total capacity would make the quarry operation susceptible to higher impacts from fluctuations and potentially lead to warranty issues due to excessive strain. To ensure smooth operation and avoid such complications, a 10% buffer was deemed necessary. Thus, a 90% utilization target was introduced as a buffer against unforeseen events like equipment failures, adding some resilience to the SoS.

Table 2. Design parameters and their values

Figure 8 shows the different routes the haulers adopted for Sites A (blue), B (yellow) and C (green), respectively. These routes were selected meticulously to cover different contextual possibilities that a hauler might encounter during its operational phase. All sites are fictive to avoid exposing confidential data. However, their characteristics are based on existing sites. More specifically, these aspects characterize the three sites:

• • Site A: This route exemplifies a long-haul, significantly consuming the energy reserves of the hauling machines. Otherwise, it contains a manageable elevation profile and relatively gentle curves.

• • Site B: In contrast to Site A, this route is relatively shorter. It has a smooth and low elevation profile but incorporates a few sharp turns that restrict speed.

• • Site C: Similar to Site B, this is also a short-haul route. However, it is a prime example of a highly degenerative route since the hauler must climb up the elevation in the loaded state. Moreover, there are numerous bends requiring frequent speed adjustments.

Figure 8. Hauler routes in Sites A, B and C, respectively.

The top view of the sites shows the curvature and paths in X and Y using a Cartesian coordinate system. The total distance for Site A is ~3.2 km, Site B is ~2.1 km and Site C is ~2.4 km.

Figure 9 depicts the elevation profiles for Sites A, B and C. The x-axis in the height profile represents normalized distance, and the y-axis represents height in meters. Normalization is necessary because the actual distances for each site differ.

Figure 9. Elevation profile of Sites A, B and C, respectively.

With these variables and parameters defined, the experiment setup was deemed complete. Figure 10 shows an example of an operational scenario built using the integrated simulation framework for Site A using the parameters for fleet number 4 in Appendix A. This fleet comprises four haulers (marked as $ {H}_1\dots {H}_4 $ ) and two charging stations (marked as $ {C}_1 $ and $ {C}_2 $ ) within the development boundary. In addition, the global control tower that dictates a suitable operational strategy has been marked as $ {T}_G $ . The local control towers are $ {T}_1\dots {T}_3 $ associated with avenues in the operational scenario that require external support in decision-making. These include the chargers $ {C}_1 $ and $ {C}_2 $ within the development boundary and the wheel loaders $ {W}_1 $ and crusher feeder $ {F}_1 $ in the context.

Figure 10. Operational scenario composed of Site A for fleet number 4 (referring to Appendix A).

A post-processing tool was developed to analyze simulation results. This tool calculates vital performance indicators comprising cost, sustainability and utilization, ultimately generating a composite value metric for each operational scenario. Looking at cost, the metadata and results data are used in conjunction with a cost table, inspired by partner company data, which depicts the acquisition and operational costs for all viable options. Examples include hauler purchase cost, electric price, service fees and so forth. The sustainability factors are addressed similarly through a lookup table and using the metadata and results data as input. Sustainability is limited to looking at the CO_2eq emissions emerging from the energy consumption, as Life-Cycle Assessment data were unavailable or is confidential. Finally, utilization is calculated using the results data table as a factor of queuing time in relation to total modeling time. For simplicity, the post-processing was added to the simulation model as an extra step before closing the simulation run. This allowed the results data to be calculated and stored together with the metadata for easier data management.

5.5. Results

With the operational scenarios and contexts established, it was time to execute the experiment. This article used RePast Simphony as an ABS, which can perform batch runs. Hence, a parameter sweep was created and executed to simulate all scenarios. The statistical data, results data and metadata were collected and aggregated. Finally, some post-processing was conducted to obtain value scores, which could then be imported, filtered and plotted using R language and RStudio.

Figures 11–13 visualize the operational scenarios tested for each demo site. The bubble charts display all possible fleet configurations (x-axis) and their value varying over the operational contexts (y-axis). A nonfeasible combination is highlighted through a red box (nonfeasible from a vehicle perspective, e.g., total battery depletion) or a black triangle (nonfeasible from a site perspective, e.g., crush feeder runs empty). The feasible options are marked with yellow bubbles, where the size and intensity of the color dictate the value creation. The value creation achieved by each scenario according to the predetermined value assessment strategy; see Section 5.1 for the causal loop diagram and value function. At the top, the average score of all operational contexts for a fleet configuration is displayed through a blue bubble (similar to the yellow bubble) to show the value creation of a specific fleet over the operational scenario. Each fleet configuration and context number refers to a particular combination of previously stated design parameters. The conversion table can be found in Appendix A.

Figure 11. Value of different fleet configurations in varying contexts within Site A.

Figure 12. Value of different fleet configurations in varying contexts within Site B.

Figure 13. Value of different fleet configurations in varying contexts within Site C.

Data analysis of fleet performance at Site A, a long route with a moderately hilly profile and gentle curves, reveals that the 25-ton haulers are insufficient. Their smaller battery capacity limits their feasibility, and even in some workable scenarios, value creation remains minimal. Conversely, both 35- and 45-ton haulers demonstrate significantly better performance. They offer greater capacity, leading to improved feasibility and value creation. Interestingly, road friction appears to be a more critical factor than ambient temperature at Site A. In addition, the number of available chargers seems to be a crucial factor for overall feasibility.

Unlike Site A, Site B has a significantly lower rate of infeasible sites. This is likely because Site B has a shorter route with a smooth and relatively flat profile. This advantage, however, translates to generally higher value creation across all scenarios at Site B. While no 25-ton configurations are feasible at Site B, the difference in feasibility between 35- and 45-ton haulers is smaller compared to Site A. In addition, implementing eco-driving practices and minimizing the number of deployed haulers can further optimize value creation on this route.

Site C presents a more significant challenge for haulers than Site B due to its highly uneven elevation profile with frequent curves, necessitating frequent speed adjustments. Notably, feasibility at Site C is dependent almost entirely on two chargers, with only a few exceptions. Furthermore, the length of the route causes some 35-ton hauler options to fail, which in some cases can be solved by using a more eco-driving profile, thus rendering a lower total energy consumption.

While bubble charts offer a quick snapshot of each site, they lack detail. Bar charts focused on the top six performers were created to gain deeper insights into specific fleet performance. These charts allow the development team to analyze value dimensions like cost, sustainability and utilization across various contexts. For example, Figure 14 presents a detailed comparison of these value dimensions for the top fleet configurations at Site A, including their average performance. Interestingly, despite electric-driven fleets having more unfeasible configurations, those that function effectively deliver the highest value. This finding suggests that electric vehicles have the potential to significantly benefit the quarry industry, even if achieving optimal configurations requires further development. Otherwise, all six top performers have similar scores for cost and sustainability while performing higher in utilization.

Figure 14. Bar chart highlighting the values of the value dimensions for the top six performing fleet configurations (Site A).

The line chart (see Figure 15) depicts the same fleet configurations but shows the average value across diverse operational contexts. Fleet configuration 54 stands out as the top performer across most contexts but fails in one specific scenario. This failure drags its average value down to fourth place. In essence, while it demonstrates strong potential, it struggles with robustness to changing conditions. In addition, it would be worthwhile to evaluate if the failed context is likely to occur in operation or not to assess the need to consider this in the design process. If minor tweaks could be made to those fleets, it might significantly affect the value.

Figure 15. Line chart highlighting the values varying over contexts for the top six performing fleet configurations (Site A).

Overall, the experiment and results portray a comprehensive view of how the operational context and design parameters affects the value creation in quarry operations. Based on this, a few conclusions can be made. First, the simulation is able to evaluate a broad range of SoS designs from both a feasibility and value perspective. This helps provide an understanding of where the SoS challenges lie in quarrying and what design parameters are essential for success. Second, it shows that the electromobility transition is beneficial for enhancing the value of hauling operations and that the autonomy can effectively adjust the operations toward higher value. Finally, the operational context can significantly affect the performance of a fleet, especially electric fleets, which should, therefore, be explored in the SoS design.

5.5.1. Impact of varying fidelity levels

The results portrayed above have used the chosen fidelity level as displayed in Figure 7. The fidelity chosen for a model is an essential consideration during design decision-making since it significantly affects performance (Schön et al. Reference Schön, Marcus, Amadori and Jouannet2021) while influencing crucial decisions like system architecture modularity and sequencing of integration tasks (Maier, Eckert & John Clarkson Reference Maier, Eckert and John Clarkson2017). Higher fidelity enables capturing parts of interest with greater detail and better capturing extrinsic influences. In the proposed integrated simulation framework, agents can be assigned different fidelities based on such requirements. To demonstrate the impact of varying fidelity levels, consider a situation where the development team needs a better understanding of how a certain contextual variation affects the value of the top-performing solutions. Specifically, the team aims to investigate the influence of wheel loader performance on the value metric and, consequently, the rationale for selecting a solution from among the top performers. Previously, an average operating point approach was used to characterize wheel loader performance. Thus, a higher fidelity level was chosen, where a quasi-static model enabled performance characteristics evaluation for the wheel loader.

A standard loading cycle typically comprises four phases: transport without load, loading, transport with load and unloading, where typical V-shaped trajectories allow the wheel loader to navigate between the hauler and pile positions (Frank et al. Reference Frank, Kleinert and Filla2018). As the machine is almost stationary in the unloading phase, it is excluded, while the remaining three phases have been visualized in Figure 16. A combination of dynamic programming and reinforcement learning was used to evaluate the control policy. The choice of algorithm was driven by the precision of the model and the complexity of the problem. Dynamic programming is well-suited for finite-horizon problems within known environments. Conversely, reinforcement learning is viable for problems characterized by larger state and action spaces, particularly in stochastic environments (Bertsekas Reference Bertsekas2019). Figure 16 shows exemplary results of a loading cycle, excluding unloading. It starts from the hauler’s position, executes a reverse maneuver along the optimal trajectory and reaches the turning point. Subsequently, it drives forward toward the pile. On reaching the pile, it interacts with the gravel pile and loads the attachment. Once the attachment is satisfactorily filled, it reverses along the same trajectory, reaches the turning point and drives forward until it reaches the hauler position. Finally, it unloads the payload into the hauler. Relevant states of the wheel loader during this time have been mapped in Figure 16, with data normalized to ensure confidentiality.

Figure 16. Simulation of an optimal control policy for the wheel loader.

The top performer fleet configuration (No. 62) at Site A (refer to Figure 15) was selected for comparison. The fleet configuration was simulated in the ABS model across all contextual options with the same setup. For comparative purposes, the fleet’s moved material and energy consumption were selected. Figure 17 shows the normalized moved material for a low- and mid-fidelity wheel loader model. The results entail that the low-fidelity models deviation from the mid-fidelity model varies over the contexts; it is highly accurate at 0.05 road frictional loss and 270 K ambient temperature, while it renders ~10% lower production at a road frictional loss of 0.03 and an ambient temperature of 285 K. In general, the contextual impact is less on the low-fidelity model compared to the mid-fidelity.

Figure 17. Normalized moved material for different fidelity models of the wheel loader.

A similar pattern occurs when looking at energy consumption, as shown in Figure 18. In general, the low-fidelity model results in lower energy consumption than the mid-fidelity model, which affects the final value of the fleet configuration.

Figure 18. Normalized energy consumption for different fidelity models of the wheel loader.

An interesting pattern occurs when this is aggregated to value, as is shown in Figure 19. In contrast to previous figures, here, the total value for the low-fidelity model is more stable than the mid-fidelity model, which sometimes performs better and sometimes worse than the low-fidelity model, indicating a sensitivity to contextual constraints. The total fleet value with a low-fidelity model performs approximately 0.9 lower in total value (0.6559 for low-fidelity and 0.6645 for mid-fidelity model) compared to the mid-fidelity wheel loader, indicating that the fidelity selection has a minor impact on the results for the supporting system wheel loader at larger compared to specific contextual setups. To put this in perspective, changing the driving mode from eco to speed has a 0.09 difference in value (eco-mode is at 0.6559 speed performs at 0.6473).

Figure 19. Value of the solution varying over contexts for the top performer using different fidelity models of the wheel loader.

In conclusion, the selection of fidelity will influence the performance and, ultimately, the value creation of a fleet. The contextual changes will be better captured with a higher fidelity model, but depending on the system, the significance of the fidelity selection will vary. Another important remark is the impact of the fidelity level on computational effort. The shift from a low to a mid-fidelity model on the wheel loader changed the simulation time from ~5 to 15 min, meaning that running the same experiment with mid-fidelity wheel loaders will increase the total simulation time by 324 h or 13.5 days. This can, of course, be reduced by parallel computing and increasing computational power. Hence, the selection of a fidelity model needs to be balanced depending on the system’s potential impact on value creation and modeling efficiency.

5.6. Case study validation

The simulation model and experiment results were presented to an expert group at the partner company as a final step. The intention of this was both to show the tool and framework itself as well as to obtain initial validation and feedback on its applicability and usefulness. For this purpose, the group was asked a set of open-ended questions. As the simulation model developed was only a proof-of-concept (PoC), there are limited possibilities for fully validating all its features and results. Instead, the focus was more on assessing its general applicability and how well the developed model can address the challenges targeted and experienced in the daily work at the partner company.

An initial remark was that it could be a good tool for better understanding how design decisions and operational contexts affects the value. However, further validation of system models is needed for the values to be trustworthy. The simulation otherwise received good remarks on its applicability and usefulness for the design process. It was also suggested that it could be a good decision support tool for early design stages and sales, since it was deemed fairly easy to import new sites and rapidly explore the design space. However, as previously stated, the experts were reluctant to believe the results without further refinement and verification.

Moreover, the value creation model developed as a part of the case study was seen as an excellent visual representation of the quarry context and a support for a better understanding of what drives and contributes to value. The participants stated that it could be a good approach for stakeholder alignment and adding transparency to the decision-making process. Finally, the PoC model was interesting for the partner company, but further development efforts must be considered commercially viable.

From a simulation and agent models standpoint, recommendations can be formulated for further development and efforts to increase validity. First, the granularity of the behavior models for operating machines should be increased to capture smaller fluctuations in quarry operations better. This is especially true for the wheel loader model. Second, the simulation model only looked at the “load and haul” – part of the operations. In the real world, many more tasks and activities are taking place. Including these and obtaining a more complete coverage of operations is essential for extending validity. Finally, the PSS perspective needs to be further incorporated into the models. Including aspects such as maintenance and failures was considered critical to allow the service perspective to be simulated and accounted for in the value exploration. In general, the simulation model needs to expand in contextual and temporal validity to best support the value exploration of SoS in a PSS context.