2.1. Empirical findings regarding factors affecting MTS performance

In the following, empirical findings in psychology regarding effective MTS functioning, which can be used to rate different MTS-related and organisational aspects in the BPMN++, are introduced.

Besides behaviours and processes such as leadership, coordination and communication, which have been mainly studied in the context of MTS functioning (Shuffler, Jimenez-Rodriguez & Kramer Reference Shuffler, Jimenez-Rodriguez and Kramer2015), emergent states have been neglected in MTS analysis. However, due to their impact on subsequent behaviours and processes, emergent states such as trust and shared mental models drive effective MTS functioning, both within and across MTS levels. Trust and commitment in MTS may be especially important among MTS leaders (Shuffler, Jimenez-Rodriguez & Kramer Reference Shuffler, Jimenez-Rodriguez and Kramer2015). Trust facilitates MTS collaboration (Vogel-Heuser et al. Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b), whereas distrust across teams harms system-level performance, as “teams will over-monitor one another to the detriment of MTS goals” (Shuffler, Jimenez-Rodriguez & Kramer Reference Shuffler, Jimenez-Rodriguez and Kramer2015, p. 29). Shuffler et al. further elaborate that MTS with prior interaction may have fewer difficulties in establishing trust. Moreover, MTS that are very large may face more challenges in establishing shared mental models compared to smaller MTS. Mental models are organised knowledge structures that help individuals to describe, explain and predict events in their environment (Mathieu et al. Reference Mathieu, Heffner, Goodwin, Salas and Cannon-Bowers2000). Shared mental models allow team members to draw on their own well-structured knowledge as a basis for selecting actions that are consistent and coordinated with those of their teammates (Mathieu et al. Reference Mathieu, Heffner, Goodwin, Salas and Cannon-Bowers2000, p. 274). Shared mental models can include models of task (e.g., task procedures and strategies), technology (e.g., equipment, operating procedures and system limitations), team interaction (e.g., roles, responsibilities and communication channels) or the team (e.g., teammates’ skills, attitudes, preferences and knowledge) (Mathieu et al. Reference Mathieu, Heffner, Goodwin, Salas and Cannon-Bowers2000). According to prior work (Vogel-Heuser et al. Reference Vogel-Heuser, Böhm, Brodbeck, Kugler, Pantförderer, Zou, Buchholz, Bauer and Lindemann2020a) common understanding among all teams within the MTS (i.e., shared mental model) is emerging about important issues of the MTS like shared mental model of processes in the MTS (e.g., how is work conducted in the MTS? mentioned 20 times by 9 interviewees) and a shared mental model of partners in the MTS (i.e., who in the MTS is doing what with which goal?).

In our work, we focussed on trust as effective emergent state and on shared mental models as cognitive emergent states because these two factors have been empirically supported to foster MTS functioning (Zaccaro et al. Reference Zaccaro, Dubrow, Torres and Campbell2020). Moreover, to be more precise on what is meant by shared mental models, we added a definition which includes both technical and psychological aspects.

Integrating psychological findings on MTS emergent states into a graphical notation of the systems engineering workflow will provide more elaborated descriptions of MTS and their characteristics, and enrich the quality and predictive power of technical descriptions by the consideration of the actors’ psychological states. Whenever empirical findings for selected aspects of the MTS evaluation are available, they should be incorporated instead of purely subjective ones gained by expert evaluation. Consequently, empirical findings need to be added as notational elements.

2.2. Empirical findings regarding factors affecting organisational structures

Empirical studies in organisational research, specifically in organisational sociology, have focussed on establishing a ‘microfoundation’ (Powell & Colyvas Reference Powell and Colyvas2008) of organisational and institutional theory. This foundation highlights that in organisational practice or ‘institutional work’ (Lawrence Reference Lawrence2006), a formal organisational structure is only one aspect that members of organisations use to coordinate both cooperation and competition within and between organisational boundaries. Most of the empirical research has been on the microfoundation of intra-organisational structures and inter-organisational relations in cases of higher education, in and between companies and institutional actors such as standardisation bodies, trade and inter-trade organisations or regulators. Empirical studies on interdisciplinary engineering practice and technical decision-making are still rare compared to decision-making in other application areas such as management in the frame of Industry 4.0 (Meyer, Schaupp & Seibt Reference Meyer, Schaupp and Seibt2019) or in the field of Empirical Software Engineering (Fernandez & Passoth Reference Fernandez and Passoth2019). Formalising existing studies in other application areas can help to extend their scope and to identify relevant gaps regarding decision-making in interdisciplinary engineering.

When competing companies collaborate, there often appear competitions in the meantime of collaboration, which facilitate and support innovation (Ritala, Kraus & Bouncken Reference Ritala, Kraus and Bouncken2016). However, this includes challenges such as tensions in cooperation between different actors, which need to be managed properly to reach desired results. Thereby, specifically stakeholders with cooperation experience can profit by gaining knowledge and capturing value from the cooperation (Bouncken & Friedrich Reference Bouncken and Friedrich2016). Research has shown that cooperation is also supported (or hindered) by shared memberships (or the lack thereof) between organisations in inter-organisational institutions (in this article: equal to inter-companies) or ‘meta-organisations’ (Garaudel Reference Garaudel2020), such as sectoral coordinating bodies, consortia or unions. While contracts or binding legal regulations always carry the risk of being broken or circumvented in practice, the empirical research on personal relationships between members of different organisations has shown that they play a decisive role in helping organisations trust each other by creating and carefully supporting existing interpersonal networks (Kroeger Reference Kroeger2012). Such mixtures of formal and informal coordination create helpful and effective forms of ‘partial organisation’ (Ahrne & Brunsson Reference Ahrne and Brunsson2011) where some mechanisms and practices of formal organisations are implemented, while others are missing. This can alter the flexibility, as informal operations increase with more personal relationships. If a formal agreement is in place in between two companies, stakeholders from both companies can act in a clearly described frame and have a better understanding of the accountability of each partner. This does not hinder companies to act against a formal agreement, but the status of the agreement set the stage for further handling resulting disputes. Integrating selected organisational mechanisms into BPMN++ will help mapping and understanding the role of such partial organisations in decision-making processes in engineering. Another important aspect for successful collaboration is the ability and experience of stakeholders to adapt their work style and communication to heterogeneous partners as well as to partners with diverse cultural backgrounds. This does not only include cultural diversity in the sense of actors and stakeholders from different countries or geographical regions (Winkler & Bouncken Reference Winkler and Bouncken2011), but also in the sense of work cultures, innovation cultures or organisational cultures (Calhoun & Sennett Reference Calhoun and Sennett2007).

2.3. Historical post calculation data to evaluate MTS and organisational aspects

Several approaches estimating project and change costs to support decision-making are introduced in the following. Cost Model (COCOMO) II (Boehm et al. Reference Boehm, Abts, Brown, Chulani, Clark, Horowitz, Madachy, Reifer and Steece2000) estimates costs using several approaches based on function points and cost drivers. Function point analysis (Dreger Reference Dreger1989) estimates the size of a system by adding up the number and weights of all transaction and data elements and compares these indicators to already finished projects and their costs (post calculation). Additionally, architecture-based change impact analysis exists for cross-disciplinary classical software systems, for example, Karlsruhe Architectural Maintainability Prediction (KAMP) as well as automated production systems, for example, KAMP4aPS (Heinrich et al. Reference Heinrich, Koch, Cha, Busch, Reussner and Vogel-Heuser2018). Schmied et al. (Reference Schmied, Gebhardt, Mörtl and Lindemann2017) describe an approach to analyse indirect costs of engineering changes. Such changes can lead to ripple effects, that is, they propagate through the whole chain of the workflow. All these approaches lack to include MTS or organisational aspects and mostly focus on technical aspects.

3.1. Cooperation diagram

The design of the Cooperation Diagram is based on the BPMN 2.0 choreography diagram [Object Management Group (OMG2) 2014]. This focusses on a sequence of concrete interactions between the involved partners and does not consider the internal behaviour of them. In our approach, these interactions correspond to the cooperation arrows in the BPMN^+I diagram. We propose to enrich the sequence of interactions by adding visual information in the Cooperation Diagram to describe team qualities and MTS performance further.

Basic structure of the cooperation diagram

In our proposed BPMN++ approach, we adapt the standard notation of the BPMN choreography diagram; instead of modelling the cooperation process and associated teams as a ‘coherent block’, we separate cooperation processes and teams and model them with individual elements. In addition, we introduce lanes representing companies analogous to the BPMN 2.0 collaboration diagram [Object Management Group (OMG2) 2014] or SysML activity diagram. Teams/actors belonging to one company are assigned to the corresponding lane (cp. Figure 6, left side with the lanes ‘company G’, ‘company B’, and ‘supplier U2’ enclosing associated teams involved in cooperation processes). A cooperation process (Figure 1 and Table 1) corresponds to the cooperation that results from two or more technical processes in the BPMN^+I diagram, which are performed in cooperation of different teams. In the cooperation diagram, these technical processes are combined to one cooperation process, which is named accordingly.

Figure 5. Modelling the cooperation of the two alternative solutions in BPMN with BPMN^+I extension (blue arrows): software change on-site ($ \alpha $, left), software change in design office ($ \beta $, right) (excerpt from Vogel-Heuser et al. Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b, Figure 5).

Figure 6. Software change on-site (workflow $ \alpha $): Cooperation Diagram (left) and Network of Actors (right).

Related cooperation processes and teams/actors are connected with a dotted line (Figure 1). The teams/actors are positioned below their related processes. Also, all teams are assigned to the pool lane that corresponds to their associated company.

3.2. Network of actors

A second subdiagram, the Network of Actors, illustrates the inter-organisational relations between the companies involved in the cooperation processes. Thus, it allows for a quick overview of possible effects of inter-organisational relationships on cooperation between teams/actors from different companies. In the Network of Actors, all teams/actors are listed and assigned to the location graphically in which they are operating.

3.3. Source of rating

To distinguish different sources (types) used as assessment basis for the rating of all blue elements (Table 3), a filled (black) or empty (white) triangle can be added. A filled triangle indicates the availability of historical data to support the rating of the respective evaluation aspect, whereas an empty triangle indicates underpinning by empirical data. In prior works, only a subjective rating by a group of experts was used (cp. Vogel-Heuser et al. (Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b)). For some of the MTS and organisational aspects, empirical studies (Sections 2.1 and 2.2) and/or historical postcalculation data (Section 2.3) are also available and can be used to ensure a more empirically validated rating. However, since this rating concerns the subjective evaluation of a psychological variable, judgement distortions can naturally never be completely excluded, but reduced by empirically validated and reliable judgement scales. As a tool for evaluation, such subjective ratings by different experts can even be used to map different assessments and, therefore, identify conflicts and potential controversies. Figure 2 depicts how the symbol is applied to the notation elements to indicate the source of the rating. In this context, available historical data are rated better than empirical data because they are more concrete, often quantitative and closer to the individual industrial use case. If both types of data exist, only the filled triangle for historical indexing data are used.

3.4. Enriched and adapted rating and procedure for the quantitative assessment of workflows

To make a quantitative comparison of the alternative workflows modelled in the BPMN++ (step , Figure 3), we calculate an overall score for each alternative based on the ratings (+/o/−) (step , Figure 3). For this purpose, the ratings are converted into points from a quantitative assessment procedure (step , Figure 3). In order to obtain a meaningful total score at the end, it must be taken into account that the alternative scenarios can have a different number of cooperation processes/teams and, thus, a different number of ratings. Therefore, we propose to calculate the score by dividing the summarised number of points by the respective total number of ratings (step , Figure 3). Additionally, this reduces the influence of different modelling styles on the resulting score (e.g., a project may be modelled by one person with x, by the other with y processes). We further discuss the normalisation of the rating points in Section 5.2.

Since different evaluation aspects have varying degrees of impact on the success of the project, we introduce weighting schemes of the factors, which allows for prioritisation of certain aspects. These weightings must be empirically determined or estimated. It should be noted that small scores are correlated with a better evaluation. The best possible rating is represented as one, while average and poor ratings receive higher values than one (cp. Table 4). Critical aspects receive higher scores than noncritical aspects for average and poor ratings. Vogel-Heuser et al. (Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b), Table 1) proposed four different sets of weights for the three level scales (+/o/−). In order to enable a clearer differentiation in the evaluation of alternatives, the weighting for the three levels is, as proposed by industrial experts, carried out in a nonlinear manner (DIN 323-1 for example like a Fibonacci sequence, DIN 323-1:1974:08 1974), for example 1/2/4 for less critical aspects up to 1/5/13 for most critical aspects (cp. Table 4). To verify the plausibility of the selected weightings, we conducted an analytic hierarchy process (AHP) approach (Saaty Reference Saaty1990). This approach enables multicriteria decision-making and is, for example, often applied to socioeconomic operation decisions when both qualitative and quantitative factors need to be considered (Subramanian & Ramanathan Reference Subramanian and Ramanathan2012). By pairwise comparison of the influence of the aspects [applying the defined factors for intensity of importance (Saaty Reference Saaty1990, Table 1)], we calculated Priority Vectors from the resulting matrix. Compared to the normalised weightings of the medium-rated aspects from our approach, we obtained similar values. However, with the AHP approach, the nonlinear spreading effect can only be partially resembled.

Table 4. Weighting of evaluation criteria

Additionally to the evaluation criteria in Table 4, we propose to include tool-coupling, data backbone and maximum allowed duration for the workflow. Data backbone and toolcoupling are essential technical means to support and ease the design workflow and should consequently be included explicitly.

For a precise assessment, the ratings for data backbone and tool-coupling are each divided into two subcriteria with the weighting 2/1/0 for the options yes/partially/no. Because the benefit can only be reached if both subcriteria are fulfilled, the ratings of the respective subcriteria are multiplied, resulting in a range of 0–4 points. Since they lighten the cooperation effort, the points of data backbone and tool-coupling are to be subtracted from the ITA points of the respective cooperation process.

For the quantitative assessment of data backbone we propose the subcriteria:

1. (i) Whether using the data backbone is beneficial for the task.

2. (ii) Whether actors provide sufficient data for meaningful use of the data backbone.

And for tool-coupling we propose the subcriteria:

1. (i) The benefit for the task using tool-coupling.

2. (ii) The experience in tool usage.

Additionally, the compatibility of the team structures also affects ITA: two points are subtracted in case of optimal compatibility, and two points are added in case of poor compatibility. For example, by applying the weights (Table 4) to a cooperation process with ITA with items of (MTS, IPP, AP) rated as (o, o, o), we obtain 13 points (ITA_pre). The cooperation process is beneficially supported by tool-coupling and conducted by teams that are experienced in tool-coupling resulting in four points to be subtracted. Additionally, the participating teams have a matching organisation structure (compatibility of team structure, resulting in two points to be subtracted). Consequently, six points are subtracted from ITA_pre resulting in seven points for the ITA of this cooperation process.

In the following, the calculation of the score for the kth workflow is described. For the calculation of a preliminary score S _pre,k for each alternative workflow (cp. Figure 3, step ), the cooperation aspects of all cooperations within the workflow are rated. Using the weighting table, the respective points are determined. According to the previous section, points are added or subtracted from the ITA points, depending on data backbone, tool-coupling and team structures. The sum of all determined points is taken and then divided by the number of rated cooperation aspects. For example, a cooperation process with three involved teams has 15 cooperation aspects (3 × 3 team aspects, three ITA and three organisational aspects). To obtain the preliminary score for this workflow (in this case only one cooperation process), the sum of points from the ratings is divided by 15. By normalising the sum with the number of rated cooperation aspects, the preliminary score S _pre,k is independent from the number of cooperation processes and involved teams.

In industrial settings the duration to design or realise a solution must be taken into account for example due to contractual penalties in the event of a time overrun. To consider the time needed to finish the necessary cooperation processes within one workflow (step , Figure 3), we calculate the total time of each workflow by summing up the required cooperation process times t_i,k (i represents the index of the cooperation process) for all n_k cooperation processes in a first step.

(1)$$ {t}_k=\sum \limits_{i=1}^{n_k}{t}_{i,k}. $$

In a second step, the calculated total times t_k of the alternative workflows are divided by t_max in order to obtain a dimensionless time ratio rt_k. Thereby, t_max represents the allowed duration to complete the project. As a consequence, the total time within all alternative workflows must be equal to or smaller than t_max. Hence, rt_k, the time ratio value of a workflow, represents the normalised time ratio compared to the maximum allowed time. That is, it will approach to zero for a workflow that only requires a small fraction of the time or increase towards one for a workflow that requires more time fraction.

(2)$$ {rt}_k=\frac{t_k}{t_{max}}. $$

To determine the highest possible impact of the time on the score of the alternative workflows, a dimensionless time criticality factor TC is introduced. The product of TC and rt_k represents the proportion of the preliminary score that is added to obtain a final score S _final,k that includes time consideration. Thereby, TC defines the maximum possible proportion of S _pre,k in case the time ratio rt_k equals one.

(3)$$ {S}_{final,k}={S}_{pre,k}\cdot \left(1+ TC\cdot {rt}_k\right)\hskip0.5em \mathrm{with}\hskip0.5em 0\hskip0.3em \le \hskip0.3em {rt}_k\hskip0.3em \le \hskip0.3em 1;0\hskip0.3em \le \hskip0.3em TC\hskip0.3em \le \hskip0.3em 0.5. $$

TC must be determined in a use case specific way since the time criticality depends on boundary conditions of the project. We propose to use values between 0 and 0.5 (inclusively) for TC. To include the possibility to consider the time but to avoid override of the evaluation of cooperation aspects, a maximum value of 0.5 for TC was found to be appropriate by performing exemplary calculations. In general, a high value for TC might be chosen for projects with a high time-criticality and a low value for projects that have high requirements regarding the quality of the results. If a value of 0.5 is chosen for TC, a maximum of half the value of S _pre,k is added to itself to obtain S _final,k. In this case, the time consideration has a considerable influence on the final score, and thus the decision which workflow is to be preferred. However, if only the quality of the result is of interest, that is, it does not matter what duration is expected for a workflow, TC is set to zero. Therefore, S _final,k equals S _pre,k, and thus only the rating of the cooperation aspects (and not the time) is included in the score.

3.5. Meta model of BPMN++

To avoid ambiguities of the proposed BPMN++, a formal definition of the notation is provided as a meta model (Figure 4) according to da Silva (Reference da Silva2015). The meta model is used to describe the concrete syntax in accordance to Sections 3.1–3.4 as UML class diagram [Object Management Group (OMG3) 2017].

Four abstract classes, which are Container, Node, Connector and Team/Actor represent all elements of the presented views. Using compositions, the concretely derived elements are assigned to each diagram, network of actors (Figure 4, right-hand side) and cooperation diagram (Figure 4, left-hand side), respectively. Containers couple nodes and teams/actors, connectors can connect teams/actors. The shaded fields show elements that are similar to elements defined in the BPMN Collaboration Diagram specification and the BPMN Choreography Diagram specification. It should be noted that only selected cardinalities are given in this meta model that are of particular interest for the description.

4.1. Introduction of industrial use case: software change during start-up

The use case scenario starts with the identification of a technical problem in the mechanical unit 1 (ANSI/ISA-88.01-2010 2010) during the commissioning of a production plant on a customers site.

According to scenario ‘VIb’, that is, a mechanical part change, the mechanical commissioning engineer (team ‘Machine Optimization’, company G) cannot solve the problem mechanically and, therefore, proposes as a workaround a software change to the start-up electrician for area 1/2 (team ‘Startup Area 1/2’, company B), who then presents the potential change to the site manager (team ‘Site Supervision’, company G). Unfortunately, the start-up electrician of company B realised that due to this software change, a new issue is created in the interface between the software of company B and company U2. The start-up electrician of company U2 (team ‘Startup Area 3’) would need to change the software by altering the acceleration procedure. Since company U2 is only indirectly affected, the start-up electrician would be forced to make changes that are not included in the initial contract of U2.

There are two possible solutions to address this scenario. On-site troubleshooting (Figure 5, workflow $ \alpha $, left-hand side) is the most time-efficient when time constraints require a quick solution. However, according to the ISO 9000 procedure (ISO 9000:2015 n.d.), changes should be made by the design team (Figure 5, workflow $ \beta $, right-hand side) to ensure appropriate bug and change management, that is, to ensure structured and consistent application of required engineering changes. In workflow $ \beta $, the commissioning mechanic (team ‘Machine Optimization’, company G) contacts his design office, and the designer (team ‘Detailed Mechanical Design’, company G) contacts the software design of company B and also contacts the software developer of company U2. The design departments work jointly on a solution and send it to the construction site as soon as it is ready. The start-up personnel updates and tests the software on the controllers concerned. In the case of an overseas start-up site, the time delay is potentially aggravated by a closed design office during the night or even weekend that may not be acceptable for the customer and consequently for the site manager. With our approach, we aim to support the site manager in his/her decision-making to select one of two solution workflows together with the on-site team. In the following, the two alternative workflows will be introduced using BPMN++.

4.2. Modelling the use case using the integrated graphical concept of BPMN++

The BPMN^+I diagram of the industrial use case forms the basis for modelling the submodels of the BPMN++ notation. Figure 5 depicts an excerpt from the entire BPMN^+I diagram (Vogel-Heuser et al. Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b, Figure 5), which illustrates the design process and the commissioning and start-up of the production plant.

The excerpt contains all relevant information for BPMN++ modelling of the two alternative workflows $ \alpha $ and $ \beta $ for software change. For better comprehensibility, locations are indicated at the top. In the following, the two alternative solutions will be modelled in BMPN++ in order to estimate the benefit of BPMN++ modelling when comparing the two variants.

When evaluating the cooperation reasonably, references to empirical and historical data are to be included to reduce subjective ratings as much as possible, but the necessity for expert ratings for some aspects will remain.

Software change in the design office (workflow β)

The software change workflow in the design office would consist of three cooperation processes since the solution finding is worked out only between the design offices of companies G and B (‘bug report’ and ‘solution finding’). In this scenario, a subsequent alignment of interfaces with the software design (involving company U2) is necessary.

The Network of Actors shows that team ‘Machine Optimization Unit 1’, which detects the error, partly consists of the same personnel from team ‘Detailed Mechanical Design’ (Figure 7, top right, dotted circle). This allows for a better solution finding process since personnel from team ‘Detailed Mechanical Design’ who know the plant and the bug from their own experience are involved. As the two teams were involved in the planning of the plant, they are able to find a sustainable solution to the problem. For this use case, we assume empirical evidence shows that in general the teams in the design office are better at finding solutions than the on-site teams. Accordingly, the team aspects are well evaluated and marked with underpinning by empirical data (Figure 7, I).

Figure 7. Software change in the design office (workflow $ \beta $): cooperation diagram (left), network of actors (right, top) and an overview of the calculation of the preliminary score (right, bottom).

The design offices of companies G and B have already worked together successfully on a number of occasions and an appropriate infrastructure for tool-coupling and data exchange has been established. Accordingly, the ITA of this cooperation are also well evaluated, and the filled triangles indicate their underpinning by historical project data (Figure 7, II). Since the ‘solution finding’ (Figure 5, B2) is enhanced by tool-coupling and a common data backbone facilitating data exchange, four points are subtracted from the ITA of this cooperation.

Here, the personal relationships and networks already established between G and B are indexed in the Network of Actors as an evaluation criterion backed up by historical project data (Figure 7, top right, III).

After the solution has been found, necessary interface adjustments (as a consequence of the solution workflow) between Company B and U2 are determined (Figure 5, B3). Neither tool-coupling nor a common data backbone is available here. Since the two companies have never cooperated before, openness and communication are rated worse for the ITA (Luciano, DeChurch & Mathieu Reference Luciano, DeChurch and Mathieu2018). In contrast to the software change on-site, the teams in the design offices are better suited for finding a solution for engineering changes, and thus receive a proper evaluation for their team aspects. The estimation of the times needed for the ‘bug report’ and the ‘solution finding’ is done in analogy to the first scenario. Based on the previous projects between G and B, historical data on the time required for ‘solution finding’ is available. The time for the third cooperation process, that is, ‘align interfaces’, must be estimated because there are no data available. All cooperation partners operate at varying locations. Thus, the organisational aspects are modelled between different location circles (Figure 7, top right). In this scenario, too, contracts between the companies and, in some cases, personal networks exist, which is reflected in a good overall assessment of the organisational aspects.

We obtain a preliminary score of S _pre,β = 2.44 for a change in the design office. In comparison to an on-site change, the value is lower, and thus, means a better rating. A detailed evaluation as well as the consideration of workflow duration to determine the final scores are conducted in Section 5.1.

5.1. Application of the integrated notation BPMN++ to evaluate the industrial use cases scenarios

In the following, the modelling (Figures 6 and 7) of the two alternative workflows in the introduced use case will be used to determine the extent to which the proposed integrated graphical BPMN++ modelling adds value in the assessment of cooperation processes.

In contrast to the BPMN^+I diagram, the cooperation diagram shows the number of effective cooperation processes. For both workflows, three cooperation arrows were modelled (Figure 5), but a different number of cooperation processes were found due to cooperation with more than two participants. This allows a more precise display, which leads to a better estimation of the complexity of cooperation processes.

Eight experts from three disciplines (system engineering, psychology in the field of team work, sociologist expert in company assessment and training as well as engineering in machine and plant automation) confirmed in a workshop that the Cooperation Diagram allows a quick overview of the involved teams/actors or companies in comparison to BPMN^+I. Furthermore, it can be used to check the modelled workflows. Typically, a team/actor from the previous cooperation process should be the initiator of the next cooperation process. If this is not the case, this can be an indication of poor planning or incorrect modelling.

Data or documents resulting from a cooperation process cannot be modelled in BPMN^+I in a meaningful way, as they always need to be assigned to a cooperation partner. In the Cooperation Diagram, documents can be modelled to enable a better understanding.

The modelling of the process with respect to team- and ITA (Figures 6 and 7, left-hand side), and organisational aspects (Figures 6 and 7, right-hand side) allows a quick and rough estimation of the feasibility, respectively obstacles when comparing the scenarios. For example, the Cooperation Diagram shows that the software change in the design office (i.e., workflow $ \beta $) requires 1 day longer in total because it needs one more cooperation process than the on-site change (i.e., workflow $ \alpha $). However, the team aspects are very well-rated for the change in the design office, and tool-coupling and a data backbone enhance the cooperation process during the solution finding process. In contrast, the on-site workflow ($ \alpha $) is faster, but the team aspects are rated poorly. On the other hand, the inter-team and organisational aspects of the on-site teams are excellent since all teams/actors are working at the same location and know each other.

In this case, a qualitative decision on which of both alternative workflows (i.e., change on-site ($ \alpha $) or in design office ($ \beta $)) is to be preferred is difficult. That is because both sides are balanced, with each having pro and contra points.

Therefore, we conduct a quantitative analysis according to the calculation procedure proposed in Section 3.4. For the assessment, we use the weights shown in Table 4. The calculation of the preliminary score is performed for the two alternative workflows based on the determined ratings depicted in Figures 6 and 7. For workflow $ \beta $, the calculation is thoroughly outlined in Figure 7 (right-hand side, bottom). First, we calculate the preliminary scores (without consideration of time) for both workflows. We obtain S _pre,α = 3.4 for an on-site change compared to S _pre,β = 2.44 for the design office. That means the change in the design office (workflow $ \beta $) is to be preferred (since smaller values indicate a better rating).

However, the consideration of time can change the result since the duration of both alternatives varies significantly. Applying Eq. (1) (Section 3.4), we obtain the total duration of the workflows with t_α = 4 hours while t_β = 3 days. To illustrate the influence of the time consideration, we assume a high-time criticality induced by the customer and potential back charges (contract guarantees). Therefore, the maximum allowed time is limited to 3 days (t _max = 72 hours), and the highest possible time criticality factor is chosen with TC = 0.5. To calculate the time ratios according to Eq. (2) (Section 3.4), the times of both workflows t_α and t_β are divided by the allowed time t _max = 72 hours and we obtain rt_α = 4/72 = 0.06 and rt_β = 72/72 = 1. Inserted in Eq. (3) (Section 3.4), we obtain final scores of S _final,α = 3.59 for on-site change and S _final,β = 3.66 for the change in the engineering office. In contrast to the result based on the preliminary score, the on-site change is to be preferred when considering the duration of the workflows.

The obtained scores correspond to the result of the previous work in the BPMN^+I approach (Vogel-Heuser et al. Reference Vogel-Heuser, Brodbeck, Kugler, Passoth, Maasen and Reif2020b). Since time was not considered in this approach, we compare the results using the preliminary score. For the same use case, a ratio of 140–120 (about 17%) on-site ($ \alpha $)/design office ($ \beta $) was determined in the BPMN^+I approach. The obtained S _pre ratio in the BPMN++ approach of 3.4 to 2.44 (on-site ($ \alpha $)/design office ($ \beta $)) is more significant (about 39%) due to the nonlinear weighting, and the consideration of additional aspects such as tool-coupling and data backbone. This result shows for the presented use case that a more significant result can be obtained with the BPMN++ approach by covering more aspects in comparison to the BPMN^+I approach.

For this use case, the total score was not affected by the origin of the data: whether they are just estimated by experts, proven by empirical studies or project controlling history. However, in future work, an uncertainty value would be considered to describe the relation between historical and empirical data on the one hand and experts estimation on the other.

In summary, BPMN++ modelling can be used to make a qualitative evaluation of a workflow. In cases where qualitative evaluation cannot be done easily, a quantitative evaluation system can be executed based on BPMN++, whose basis is an existing BPMN model, which is enriched with identified cooperation processes (BPMN^+I model). Furthermore, the BPMN++ diagram provides a better overview of the cooperation processes and can be used to check the workflow.

5.2. Threshold analysis of BPMN++

In the following, the thresholds of the ratings are discussed for two teams to ease understanding. The evaluation of the cooperation aspects is based on the ratings, which are converted to points using the proposed weighting system (Table 4). According to Section 3.4, the points for the respective ratings of team-, inter-team- and organisational aspects are added and the sum is divided by the number of ratings. Consequently, the preliminary score is dependent on the assessed quality of teams and cooperations but not on the number of them. Therefore, this evaluation scheme is scalable and allows for a comparison of workflows with different numbers of cooperations. In case, the numbers of cooperation processes and teams increase a lot it implies that the teams never worked together and, thus, have a lower inter-team rating according to our approach. The risk to fail is included by the rating of the ITA.

In the following, the calculations of the highest and lowest obtainable scores are introduced. Corresponding BPMN++ models are attached (cf. Figure A1). For an ideal cooperation process involving two teams with best ratings for all team-, inter-team- and organisational aspects as well as tool-coupling and data backbone, a minimum preliminary score of 0.75 can be obtained [1 point for each of the 12 ratings (6 ratings for team aspects of two teams, 3 ratings for ITA and 3 ratings for organisational aspects), subtraction of 3 points for ITA: therefore 9 points divided by 12 (ratings)]. This is the lowest (best) possible score.

In contrast to that, for a cooperation process involving two teams with the worst possible rating of all aspects and incompatible team structures, a maximum preliminary score of approximately 11.17 can be obtained (132 points from 12 ratings, 2 points added for incompatible team structures). It should be noted that an even higher (worse) score can be obtained for cooperation processes with a higher number of teams. Since all three team aspects have the highest weighting (13 points for poorly rated team aspects, cp. Table 4), the preliminary score for a cooperation process approaches 13 asymptotically as the number of poorly rated involved teams increases up to infinite. Depending on the chosen time criticality factor, a higher final score can be obtained when time is considered according to Section 3.4.

If a cooperation aspect cannot be rated due to lack of information, it can be disregarded in the evaluation, since the sum of the points from the ratings is divided by the number of effectively applied ratings. However, the influence of the other ratings on the obtained score grows and the balance between team-, ITA and organisational aspects may be affected. Therefore, it should be considered to rate such cooperation aspects as medium (‘o’ out of ‘+/o/−’), especially if there is a lack of information regarding several aspects. In the case that all aspects are rated as medium for a cooperation process with two teams, a preliminary score of 4.33 is obtained.

5.3. Comparing the proposed integrated graphical BPMN++ with BPMN^+I

The BPMN^+I approach proposed a table to evaluate aspects of the identified cooperation, which is then used to calculate the score. In contrast, in the BPMN++ approach, we use a graphical representation included in the proposed submodels. The advantages of the BPMN++ approach are discussed in the following.

The division into the two submodels Cooperation Diagram and Network of Actors in BPMN++ enables a meaningful separation of cooperation aspects that are team/process-dependent and team/process-independent. The psychological aspects of a team, such as MTS-mindset and communication, differ, that is, they depend on the type of the task as well as cooperation partners. In comparison, most organisational aspects can be applied to all teams of a company – regardless of the task types or cooperation partners – and remain largely unchanged at the relevant time of decision support (e.g., degree of formal organisation). Hence, the cooperation diagram has the ideal structure to be enriched with technical/psychological aspects and the Network of Actors with organisational aspects.

In the cooperation diagram, teams are assigned to the companies, whereas in the network of actors, they are assigned to the locations at which they operate. Thus, two different perspectives are provided, which can prove to be an advantage, especially in larger projects with many participants. For instance, if material flows play an essential role, the Network of Actors can quickly provide an overview of the locations between which material transport needs to be established.

The scalability is given with both BPMN++ and BPMN^+I approaches. However, the overview in the table can become problematic for more complex workflows, as evaluated aspects are not directly visible in the diagram though the aspects have to be traced in the checklist. In contrast, the evaluation aspects are visually linked to the corresponding cooperation and teams in the diagram in the BPMN++ approach, and thus, are immediately identifiable.

The graphical representation in BPMN++ allows easy extensibility of the evaluation aspects. The newly introduced elements in BPMN++ allow to insert additional criteria to be considered: for example, time expenditure could be used as an indicator of the complexity and importance of a cooperation process. Tool-coupling and common data backbones can significantly increase the effectiveness and quality of the cooperation process. Also, the introduction of further aspects in future projects is conceivable, for example, distances between relevant locations in the network of actors. Another important point is the quantitative evaluation system. The table in the BPMN^+I approach already limits the flexibility in the evaluation. The weighting of the evaluation aspects is the only available medium to adjust the evaluation system. In comparison, in the BPMN++ approach, the two subdiagrams are independent from the evaluation system. For the evaluation of the use case, we use a BPMN^+I-based approach with non-linear weightings (cf. Section 3.4), but any other kind of calculation method using an according algorithm is possible. Furthermore, paradigm neutrality is ensured by the possibility of adopting new evaluation systems while the BPMN++ model remains the same. This ensures the transferability of new empirical models and, thus, the applicability of the BPMN++ approach in a dynamic, fast-changing business environment. The development of a suitable ‘basic framework’ for adaptable and evolvable evaluation systems and the automatic import of the evaluated models will be tasks in future work.

The apparent disadvantage of the integrated graphical BPMN++ approach is the more elaborated approach via the intermediate step of the submodels, while the table is directly derived from the BPMN^+I (BPMN enriched with cooperation arrows) diagram. However, this disadvantage could be reduced in the future work: based on annotated BPMN cooperation representations, an evaluation support will be developed that automatically parses the BPMN diagrams and identifies the cooperation criteria based on its meta model.

5.4. Constraints for a successful application: comparability of similar scenarios (and individual processes in context) and integration of existing partial models

To obtain a meaningful result from the BPMN++ evaluation approach, the proposed cooperation aspects need to be best rated objectively. However, we aim to support the comparability of the subjective assessments and their validity by a clear definition of assessment aspects. In future work, guidelines and checklists may be used to allow for a more precise and more balanced rating of psychological and sociological aspects.

In order to establish the BPMN++ evaluation method, the comparability of similar projects is required to allow feedback loop and sensitivity analysis. In order to optimise evaluation and decision-making for new projects based on postcalculation of finished project data, it would be useful to issue a guideline that defines, for example, the separation between communication and cooperation or the level of detail in BPMN/BPMN++ modelling. This way, it could be ensured that the modelling (as the basis for the evaluation) is done in a uniform style.

The comparison of individual cooperation processes from similar workflows can also offer a great advantage. Here as well, a uniform modelling style satisfies essential requirements for comparability. The question as to what extent the context of the cooperation processes can be compared (e.g., previous and subsequent processes) makes the comparison difficult. This should be addressed in the future work.

Using the BPMN++ model requires an existing BPMN model in which the goal of a project is clearly defined, and the workflow is roughly planned. BPMN++ modelling can support decisions during the planning phase of a project or in case of unforeseen events that force a change in the planned workflow, as shown in the use case software change during start-up. In the latter case, an already existing BPMN-Diagram from internal company process descriptions like ISO 9000 can be used.