1. Introduction
The products and services of tomorrow are becoming smarter, more autonomous and connected, which creates opportunities for new business models and innovations. However, the added complexity of these advanced systems calls for new approaches to enable more efficient development to ensure future competitiveness in bringing these increasingly complex products and services to market (Reference Dumitrescu, Albers, Riedel, Stark and GausemeierDumitrescu et al., 2021). A promising answer is Advanced Systems Engineering, particularly Artificial Intelligence (AI) augmented Systems Engineering (SE).
SE offers a structured approach to handling complex interdisciplinary design processes, offering numerous benefits such as early error detection and cost-reduction (Reference HonourHonour, 2013; INCOSE, 2023). However, broad adoption of SE practices in the industry has been slow outside avant-garde deployments such as Aerospace and Defence, despite SE being seen as a promising, if not necessary, response to manage the increasing developmental complexity (Reference Dumitrescu, Albers, Riedel, Stark and GausemeierDumitrescu et al., 2021). SE is traditionally document-based, with artefacts stored in documents such as requirements specifications and interface control documents; however, synchronising and maintaining the information in the documents is often a challenge (INCOSE, 2023).
Model-based systems engineering (MBSE) is an enhancement to traditional SE approaches by integrating the previously document-based information into an integrated MBSE model repository (INCOSE, 2023). Previous documents are views of this repository, ensuring changes in one view are reflected everywhere. Thus, MBSE improves the ability to manage and use the information, resulting in benefits over traditional SE, such as improved communications, the ability to manage complexity and improved product quality (INCOSE, 2023). MBSE also assists with traceability between artefacts from different stages of the developmental lifecycle and between subsystems and components (Carroll & Malins, Reference Carroll and Malins2016; Wilking et al., Reference Wilking, Horber, Goetz and Wartzack2024). However, MBSE tends to be more difficult to adopt than traditional SE approaches (Reference Henderson, McDermott and SaladoHenderson et al., 2023), as it requires significant efforts to establish and maintain the relevant models within the integrated model repository (Reference Wilking, Schleich and WartzackWilking et al., 2020).
An emerging approach addressing this challenge is Advanced Systems Engineering (ASE), which focusses on using digitally driven technologies, such as AI, to enhance and streamline current SE and MBSE practices (Reference Dumitrescu, Albers, Riedel, Stark and GausemeierDumitrescu et al., 2021). ASE provides a way forward in a landscape where SE and MBSE are increasingly recognised as key to building competitive advantages in advanced systems, yet often being perceived as too complex and challenging to adopt, due to factors such as the additional expenses and expertise required for modelling (Reference Dumitrescu, Albers, Riedel, Stark and GausemeierDumitrescu et al., 2021). In parallel, AI is emerging as a disruptive force, particularly after the emergence of generative AI applications such as ChatGPT, which accelerated AI technology and uptake due to their performance and user-friendliness (Reference Kanbach, Heiduk, Blueher, Schreiter and LahmannKanbach et al., 2024), unlocking unprecedented opportunities for analysing datasets, detecting patterns and other sophisticated analyses (Reference Khaleel, Ahmed and AlsharifKhaleel et al., 2023). In the extant literature on SE and AI, two different labels have emerged: SE4AI and AI4SE (Reference McDermott, DeLaurentis, Beling, Blackburn and BoneT. McDermott et al., 2020). SE4AI focuses on applying SE approaches to manage the complexity associated with developing AI. AI4SE, which is the focal topic of this paper, focuses on applying AI, such as machine learning techniques or augmented intelligence, to support SE processes (Reference McDermott, DeLaurentis, Beling, Blackburn and BoneT. McDermott et al., 2020). It is expected that ASE and AI4SE will offer new tools that simultaneously optimise system design and engineering while also balancing Human-Centred Design (HCD) by minimising workload and increasing situational awareness (Reference Fouad, Raz, Llinas, Lawless, Mittu, Lawless, Llinas, Sofge and MittuFouad et al., 2021). However, as outlined by McDermott et al. Reference McDermott, Pepe and Clifford(2024), AI integration into SE is still in its infancy, with AI’s impact on SE processes not yet being fully comprehended. Thus, this paper addresses the following research questions:
-
RQ1) In what way can AI technologies be utilised to enhance SE activities?
-
RQ2) What are the key barriers preventing the effective integration of AI into Systems Engineering practices?
To answer these questions, a systematic review of pertinent studies and theorisations in literature is conducted to identify current and emerging research on AI augmentation, its potential benefits, and associated barriers. The focus is on understanding gaps in adopting AI4SE . The paper is structured as follows. Section 2 outlines the methodology of the literature review. Section 3 presents findings related to RQ1, while Section 4 addresses results related to RQ2. Section 5 discusses the findings with emphasis on future research directions, and Section 6 concludes with a summary and discussion of limitations.
2. Methodology
To analyse current research and adoption barriers, this paper used a systematic literature review approach. Searches were performed in the two established bibliographic databases Scopus and Web of Science (WoS) as suggested by Paul & Criado Reference Paul and Criado(2020). The search strategy focuses on AI4SE. The search terms were kept broad to capture the two topics (AI and SE) in the broadest sense, using terms commonly recognised in literature. Therefore, we used the terms Systems Engineering, Artificial Intelligence and the abbreviation AI. Furthermore, we used a proximity operator to narrow the search results to papers that focus on both AI and SE, assuming the co-occurrence of the search terms within limited proximity indicates a stronger thematic relationship. A proximity of 15 words was chosen as this is the “default” proximity operator in Web of Science using the operator “NEAR” (Search Operators, 2023), and this was then mirrored in Scopus using the W/15 operator. Based on this, the following search strings were developed for Scopus and WoS.
-
WoS Core Collection: TS=((AI OR “artificial intelligence”) NEAR (“systems engineering”))
-
Scopus (TITLE-ABS-KEY): (ai OR “artificial intelligence” ) W/15 “systems engineering”
The search was conducted, analysed and repeatedly updated over a 3 months period in September to November 2024. The following inclusion criteria were developed: Papers (1) should focus on AI integration or support for SE practices, (2) should be in English, and the (3) paper format should be journal articles, conference proceedings and book chapters. In total 284 papers were found in the search after merging the results and removing duplicates using the procedure developed by Lim et al. Reference Lim, Kumar and Donthu(2024). The papers were screened by reading the abstracts, emphasising identifying papers covering AI integration in SE processes. After screening the papers and applying the inclusion criteria, a total of 33 papers were selected for in-depth review. The selected publications were analysed and coded thematically following the approach outlined by Elo & Kyngäs Reference Elo and Kyngäs(2008).
For the role of AI in enhancing SE, we used a deductive coding process, basing the codes on the V-model core activities of VDI2206: (1) Requirements Elicitation, (2) System Architecture, (3) Implementation of System Elements, (4) System Integration and Verification and (5) Validation and Transition (Reference Graessler and HentzeGraessler & Hentze, 2020). It also incorporates “Modelling and Analysis” and “Requirements Management” as additional activities due to their integral role across the various phases of the V-Model. For simplicity, “Requirements Management” was integrated in the Requirements Elicitation code. We started with a deductive approach using the abovementioned codes but combined it with inductive to refine the analysis framework iteratively. The VDI 2206 framework was chosen for its emphasis on modelling and its relevance to the cyber-physical systems domain, this is discussed in more detail by Graessler and Hentze Reference Graessler and Hentze(2020). The analysis of barriers for AI integration used an qualitative inductive thematic approach since it is suitable when there is not enough knowledge yet about the phenomenon (Reference Elo and KyngäsElo & Kyngäs, 2008).
3. The role of AI in enhancing core activities of systems engineering
The following analysis provides an overview of how AI technologies can support and improve the core SE activities derived from the VDI2206. Figure 1 provides an overview of existing, emerging, and currently empty research areas of AI4RE, which are described in detail in the following. In general, current research seems to focus on early SE activities and Modelling and Analysis.
Figure 1. Examples of potential AI use cases in systems engineering activities
3.1. Requirements elicitation
Requirements elicitation forms the foundational activity of capturing and documenting stakeholder needs. It involves structuring and analysing requirements to develop “a description of the system characteristics which are to be fulfilled” (Graessler & Hentze, Reference Graessler and Hentze2020, p. 319). We found that requirements elicitation is the third most frequently addressed core activity, with 22 out of 33 papers covering it.
AI offers strong potential in enhancing the elicitation and writing of requirements. Existing tools and techniques such as “RAPID” are designed to analyse requirements and derive UML diagrams, while “Circe” supports automated requirements analysis (Reference Patel, Maheshwaran and SanthyaPatel et al., 2024). Moving to LLMs, Crabb and Jones Reference Crabb and Jones(2024) used OpenAI’s GPT3.5 model, to develop requirements based on specific scenarios. More detailed sample prompts for tasks such as requirements generation, specification and analysis are also provided by Arora and Grundy Reference Arora, Grundy and Abdelrazek(2023), who also provide SWOT analysis of LLM for requirements and find that LLM can bridge a gap in identifying ‘unknown’ requirements. Future possibilities highlighted by Bruneliere Reference Bruneliere, Muttillo, Eramo, Berardinelli, Gómez, Bagnato, Sadovykh and Cicchetti(2022) include the integration of AI and Machine Learning (ML) to support requirements writing by providing recommendations aligned with domain-specific standards and guidelines. Automated reasoning could also be applied to check the consistency of requirements during early design stages. Additionally, Natural Language Processing (NLP) tools can play a vital role in generating requirements from unstructured data in large documents (Reference Patel, Maheshwaran and SanthyaPatel et al., 2024), offering new possibilities for managing complex information.
AI can address the challenges of traceability and integration of requirements with subsequent phases of the V-model, a process that is often resource-intensive and prone to errors (Reference Chami and BruelChami & Bruel, 2015). Going into more detail, Rudolph Reference Rudolph(2024) used a machine-executable V-model coupled with graph-based languages to link various models within core activities of the V-model. He suggests natural language requirements can be encoded using object-oriented programming languages such as Java or visual modelling languages like UML to develop ontologies, modelled with graph-based languages and linked to later phases of V-model using SysML. Another approach, suggested by Chami and Bruel Reference Chami and Bruel(2015), involves incorporating Gaussian noise into modelling processes to represent uncertainties in requirements values and properties, thus enabling broader solution spaces and facilitating better alternatives compared to conventional SysML modelling techniques. Despite these advancements, challenges remain in encoding natural language requirements to enable integration into system-wide models and in capturing interdisciplinary information for use in AI algorithms (Reference Chami and BruelChami & Bruel, 2015). Providing a different perspective than other articles, Barry and Doskey Reference Barry and Doskey(2020) point out that AI can also facilitate sociotechnical modelling to understand stakeholders relationships and alignments. Early understanding of stakeholder alignment can reduce risk is useful as differences in beliefs and expectations are common causes of project failure (Reference Barry and DoskeyBarry & Doskey, 2020).
3.2. System architecture and design
The system architecture and design phase represent the core interdisciplinary effort in product engineering, where the overall cross-disciplinary solution structure, known as the system architecture, is established. This architecture is then decomposed into implementable units for subsequent design and implementation (Reference Graessler and HentzeGraessler & Hentze, 2020). System architecture is identified as the second most frequently occurring theme in the sampled literature, with 17 of 33 papers addressing it. Topics related to system architecture in modelling are further discussed in section 3.3.
AI can assist within the system architecture and design phase by automating processes and supporting interdisciplinary tasks. To automate SysML modelling and traceability, Bonner et al. Reference Bonner, Zeller, Schulz, Beyer and Olteanu(2023) developed a Polarion integrated tool for semi-automated creation and maintenance of trace links between requirements and MBSE, ensuring requirements are implemented correctly. Similarly, Cederbladh et al. Reference Cederbladh, Berardinelli, Bruneliere, Cicchetti, Dehghani, Di Sipio, Miranda, Rahimi, Rubei and Suryadevara(2024) proposed creating toolchains based on knowledge graph kernels to link various modules and tools, offering another approach to automating SysML modelling and traceability. Given logical architectures, various AI algorithms can be used to create physical architectures (Reference RudolphRudolph, 2024). In addition, Hsu et al. Reference Hsu, Clymer, Garcia and Gonzalez(2009) provided an example of how agent-based modelling using neural networks can be employed to understand emergent behaviour in systems and integrate this understanding into architectural models. While many studies focus on specific AI techniques or applications, Batista and Monsuez Reference Batista and Monsuez(2020) emphasised that many essential AI capabilities for creating AI assistants already exist, but integration into systems engineering processes remains underdeveloped. To accelerate this process, they promote modular and incremental implementation to reduce costs and time consumption.
3.3. Implementation of system elements
The implementation of system elements involves developing, dimensioning, designing, and detailing the specified system components (Graessler & Hentze, Reference Graessler and Hentze2020, p. 319), and is addressed by 11 of the 33 papers. AI techniques have shown the potential to reduce the design effort required in this phase, particularly by automating complex and time-intensive tasks. Rudolph Reference Rudolph(2024) demonstrates the use of graph-based languages to develop an executable V-model, enabling the use of algorithms Particle Multiple Swarm Optimisation (PMSO), simulated annealing (SA), and A* algorithms to automate design tasks of packing, piping and wire routing under design constraints. This highlights how AI be used to automate the step between logical and physical architectures. Having a different focus, Aranburu et al. Reference Aranburu, Justel, Contero and Camba(2022) explore the usefulness of AI in parametric modelling, indicating that studies have shown some initial success in automatically reconstructing 3D models based on human modelling sequencing, although with poor quality and robustness due poor-quality training data. The data quality stems from poor modelling practices, leading to inefficient design sequences entering the training datasets. This indicates that although AI has the potential to assist with automated design based on architectures and can be used for parametric modelling, design teams should be encouraged to develop high-quality datasets, which then can undergo rigorous quality checks. In the long-term, Aranburu et al. Reference Aranburu, Justel, Contero and Camba(2022) envision significant advancements in AI-enabled CAD modelling support, such as autocompletion, auto creation of primitives and relations from 2D sketches, and the use of ML algorithms trained on historical data to improve accuracy and reduce human intervention in the design phase.
3.4. System integration and verification
System integration involves step-by-step integration of systems into next product-hierarchy level, while also verifying that the system meets the relevant requirements (Reference Graessler and HentzeGraessler & Hentze, 2020). This phase is the second least addressed, with only 8 of the 33 papers discussing relevant themes. Bruneliere et al.Reference Bruneliere, Muttillo, Eramo, Berardinelli, Gómez, Bagnato, Sadovykh and Cicchetti(2022) discuss the AIDOaRt project’s intent to develop an AI-enabled framework for continuous validation, with expected improvements in productivity and quality. They also propose using ML techniques to generate unit tests and identify high-quality test suites with high probability of detecting failures. Madni (2023) supports this by suggesting that ML, particularly reinforcement learning, may be used to enhance SE verification by enabling real-time updating of models to ensure correctness. To improve verification, Hsu et al. Reference Hsu, Clymer, Garcia and Gonzalez(2009) demonstrate that agent-based modelling can be used for the early identification of emergent behaviour in the developed system, providing a proactive approach to ensuring intended system behaviour.
3.5. Validation and transition
In the final phase of the V-model, the fully integrated system is validated against stakeholder needs and then handed over to a new entity such as a client. Validation of individual elements and subsystems may also take place (Reference Graessler and HentzeGraessler & Hentze, 2020). AI techniques can be used to enhance the validation and transition process; however, this phase is the least commonly discussed, with only 7 of the 33 papers discussing relevant themes. Machine learning (ML) can detect performance issues and system errors by analysing historical data and trace analysis (Reference Bruneliere, Muttillo, Eramo, Berardinelli, Gómez, Bagnato, Sadovykh and CicchettiBruneliere et al., 2022). Additionally, requirements validation can be supported using NLP techniques (Reference Patel, Maheshwaran and SanthyaPatel et al., 2024). In complex systems-of-systems, where unpredictable or emergent behaviours often arise, agent-based simulation techniques can identify both desirable and undesirable emergent behaviours early in the validation phase (Reference Hsu, Clymer, Garcia and GonzalezHsu et al., 2009). Highlighting that the potential use of AI driven analysis continues into the operational stage, Verma and Singhal Reference Verma, Singhal, Das, Saha, Coello Coello and Bansal(2024) discuss the potential of using AI driven predictive maintenance and analysing customer behaviour thereby closing the V-model by understating customer needs in the first phase.
3.6. Modelling and analysis
Graessler and Hentze Reference Graessler and Hentze(2020) define modelling and analysis as an overarching activity involving digital representations of the systems across various engineering disciplines. From the SE discipline, MBSE connects the different models on a system level, requiring active analysis and synthesis.
The reviewed literature suggests that AI has the potential to support MBSE efforts. A key benefit is reducing the learning curve for inexperienced users, making MBSE more accessible (Reference Bader, Vereno and NeureiterBader et al., 2024). To reduce the MBSE adoption barrier, Schräder et al. Reference Schräder, Bernijazov, Foullois, Hillebrand, Kaiser and Dumitrescu(2022) propose running physical modelling workshops that use sketches instead of digital modelling tools. These workshops prioritise technical discussions over formal modelling, enabling team members to participate without prior training. Using various algorithms sketches from such sessions can later be digitised into models like UML (Castellanos-Paes et al., Reference Castellanos-Paes, Hili, Albore and Pérez-Sanagustín2022; Schräder et al., Reference Schräder, Bernijazov, Foullois, Hillebrand, Kaiser and Dumitrescu2022). AI tools can also support modelling directly. Large language models (LLMs) can generate activity diagrams from requirements (Reference Crabb and JonesCrabb & Jones, 2024), while NLP and ML techniques can automate or semi-automate the creation of SysML elements such as actors, use cases, associations, and blocks (Reference Patel, Maheshwaran and SanthyaPatel et al., 2024). Additionally, AI assistants being developed to enhance model quality by checking the consistency of design and models (Reference Bruneliere, Muttillo, Eramo, Berardinelli, Gómez, Bagnato, Sadovykh and CicchettiBruneliere et al., 2022). This highlights the potential of developing AI support for modelling, for example when designing system architectures based on textual requirements. On a broader system-of-systems level, agent-based modelling can be used to predict and analyse emergent behaviours in system-of-systems (Reference Hsu, Clymer, Garcia and GonzalezHsu et al., 2009). This highlights a strong potential of developing a wide range AI support for modelling, but also that current use cases are not sufficiently mature to use in production environments. Coupled with other technologies, e.g. graph-based models, and ontologies, (Reference Orellana and MandrickOrellana & Mandrick, 2019; Reference RudolphRudolph, 2024) AI also shows strong potential to assist with interoperability across the V-model phases. This highlights that organisations adopting MBSE but with siloed implementation are likely be a step ahead in utilising AI and other digital technologies to streamline and enhance the application of MBSE.
4. Barriers to Artificial Intelligence integration in systems engineering
This section outlines the barriers identified in the literature that impede integration of AI into SE. The analysis revealed three primary categories of barriers: 1) Data-related barriers, 2) Trust, explainability, and interpretability, and 3) Technical limitations of AI systems and algorithmic issues. These barriers are summarised in Table 1 and detailed below. Phase-specific barriers, such as siloed tools and lack of interoperability between AI and SE tools, further hinder integration. A deeper understanding of human-machine interaction is also crucial to enable the full utilisation of AI as a “colleague”.
Table 1. Overview of future research priorities
4.1. Data related barriers
AI relies on high-quality datasets. AI for SE faces several data-related barriers for training algorithms such as machine learning (ML) and neural networks remains insufficient (Reference Schräder, Bernijazov, Foullois, Hillebrand, Kaiser and DumitrescuSchräder et al., 2022), and fine-tuning LLMs for SE activities (Bader et al., Reference Bader, Vereno and Neureiter2024; Du Plooy & Oosthuizen, Reference Du Plooy and Oosthuizen2023; Schräder et al., Reference Schräder, Bernijazov, Foullois, Hillebrand, Kaiser and Dumitrescu2022). Aside from an actual lack of data, data often exist in industry but is not easily accessible to due to privacy concerns (Reference Verma, Singhal, Das, Saha, Coello Coello and BansalVerma & Singhal, 2024), restricting researcher access to legacy data (Reference Cederbladh, Berardinelli, Bruneliere, Cicchetti, Dehghani, Di Sipio, Miranda, Rahimi, Rubei and SuryadevaraCederbladh et al., 2024). Another challenge is the need for but lacking access to labelled datasets (Reference Bonner, Zeller, Schulz, Beyer and OlteanuBonner et al., 2023). While an overarching barrier, data availability also affects specific phases: for example, inconsistent data annotation, lack of labelled data, and dataset diversity in requirements elicitation, leading to potential incompatibility of business and sub-/system requirements (Reference Norheim, Rebentisch, Xiao, Draeger, Kerbrat and WeckNorheim et al., 2024).
Fragmented and inconsistent data can lead to poor information sharing across the system lifecycle (Orellana & Mandrick, Reference Orellana and Mandrick2019; Verma & Singhal, Reference Verma, Singhal, Das, Saha, Coello Coello and Bansal2024). Data often reside in siloed tools disrupting continuous data flows (Reference Batista and MonsuezBatista & Monsuez, 2020) required for generating SE models with specific language and taxonomy requirements (Reference Cederbladh, Berardinelli, Bruneliere, Cicchetti, Dehghani, Di Sipio, Miranda, Rahimi, Rubei and SuryadevaraCederbladh et al., 2024). Varying data quality creates further reliability and applicability problems, such as biases in datasets caused by training or adaptive learning over time (Reference McDermott, Blackburn and BelingT. A. McDermott et al., 2021) transferring training between datasets (Reference Bonner, Zeller, Schulz, Beyer and OlteanuBonner et al., 2023).
Thus, researchers need to ensure better access to high-quality, interoperable datasets, and improvements in data sharing practices across the SE lifecycle and between academia and industry.
4.2. Trust, explainability and interpretability
For AI to be adopted by (systems) engineers, it must first gain their trust; however, AI currently engenders distrust (Reference MadniMadni, 2023). One of the primary reasons for this is that AI often operates as a “black box” producing outputs with limited explainability and interpretability (Reference Verma, Singhal, Das, Saha, Coello Coello and BansalVerma & Singhal, 2024). This lack of transparency is especially concerning when AI is used to support decisions with potentially significant consequences, as the ability to understand and trust the outputs is critical. LLMs are particularly fraught with explainability issues due to their strong black-box nature and instability (Reference RudolphRudolph, 2024). Although techniques such as prompt engineering can reduce the problem, a key issue is their tendency to “hallucinate”, generating incorrect or nonsensical text (Reference Du Plooy and OosthuizenDu Plooy & Oosthuizen, 2023). Despite these shortcomings, generative AI such as LLMs will likely play an important role in the future due to their user-friendliness, versatility, and remarkable power in generating outcomes quickly.
4.3. Technical limitations of AI systems and algorithmic issues
Although AI capabilities have progressed significantly, technical limitations and challenges remain. AI development interfaces primarily target data scientists, requiring skills not widely available in the SE community. AI applications often lack explainability, robustness, and protection from adversarial attacks (Reference McDermott, Blackburn and BelingT. A. McDermott et al., 2021). AI systems also struggle to adapt to dynamic or unexpected contexts without human intervention, leading to ethical challenges, such as Uber’s surge pricing during a siege in Sydney (Reference MadniMadni, 2023). These technical challenges highlight the need for further research to enhance AI reliability and adaptability in SE. Specific limitations in SE include generative AI’s difficulty handling mathematical model components and producing interoperable outputs (Reference Du Plooy and OosthuizenDu Plooy & Oosthuizen, 2023). Crabb and Jones Reference Crabb and Jones(2024) faced issues with LLMs outputting interoperable files without manual corrections, even with correct file types (XML) for Cameo Systems Modeler import. Memory issues also lead to context constraints, resulting in unreliable outputs when inputs become too complex or conversations too long (Reference Bader, Vereno and NeureiterBader et al., 2024). This underscores the importance of better utilizing existing AI technologies within their limitations while maturing current AI technologies.
5. Discussion
Our findings highlight the potential of using AI to augment SE, but also underscore its adoption barriers. Aside from individual research fields, our analysis revealed critical overarching issues (Table 1) to be discussed in the following. These should be addressed before or in parallel to field-specific topics and, therefore, drive future research endeavours.
5.1. Enhancing data sharing and collaboration
The availability and quality of training data remain significant challenge AI in SE. Collaboration between industry and academia could facilitate shared datasets through initiatives such as direct data sharing, co-creating new datasets, and labelling training data with the help of experts. However, privacy concerns remain a significant barrier. Techniques such as data obfuscation (Reference Bonner, Zeller, Schulz, Beyer and OlteanuBonner et al., 2023) offer potential solutions. Norheim et al. Reference Norheim, Rebentisch, Xiao, Draeger, Kerbrat and Weck(2024) call for the creation of public datasets for training, testing and benchmarking new AI models. Enhancing interoperability between tools and datasets across the SE lifecycle is crucial. Siloed tools and data hinder AI integration, necessitating solutions like knowledge-graphs and ontologies for seamless linkages. Future research should address privacy concerns by developing frameworks and techniques for confident data sharing. Additionally, research should focus on interoperability, including data structuring, linking technologies, and developing use cases to drive progress.
5.2. Developing trustworthy and explainable AI
The black-box nature of AI systems undermines trust and hinders adoption in SE. Developing trustworthy AI systems requires addressing three core elements: validity, explainability, and accountability (Reference Alix, Lafond, Mattioli, Heer, Chattington and RobicAlix et al., 2021). Explainable AI (XAI) suggests AI outputs need to be with human cognitive processes, such as humans using contrastive explanations, generating explanations based on analogical reasoning, selecting the arguments used in an explanation, performing poorly in manipulating probabilities, and putting explanations within a dialogue (Reference Alix, Lafond, Mattioli, Heer, Chattington and RobicAlix et al., 2021). To build trust and confidence in AI, future studies should prioritise research that integrates XAI techniques and considerations. Understanding the dynamics of human-AI interaction is critical, particularly how transparency and high-quality outputs can build user confidence in AI systems. Thus, success hinges on creating explainable and accountable AI systems, providing clear, actionable explanations to build user confidence (Bruneliere et al., Reference Bruneliere, Muttillo, Eramo, Berardinelli, Gómez, Bagnato, Sadovykh and Cicchetti2022; Castellanos-Paes et al., Reference Castellanos-Paes, Hili, Albore and Pérez-Sanagustín2022). SE4AI can provide valuable support by using SE principles to, for instance, ensure traceability from AI model input via processing to output.
5.3. Advancing human-AI team research
Instead of replacing humans, literature emphasises the development of AI support or Human-AI teams combing their complementary strengths (Reference MadniMadni, 2023). However, a better understanding of Human-AI teams is needed, such as resulting team strengths and limitations, team roles, required human oversight mechanisms (Fouad et al., Reference Fouad, Raz, Llinas, Lawless, Mittu, Lawless, Llinas, Sofge and Mittu2021; Verma & Singhal, Reference Verma, Singhal, Das, Saha, Coello Coello and Bansal2024). Trust in AI team members remains a critical factor (Reference Alix, Lafond, Mattioli, Heer, Chattington and RobicAlix et al., 2021). Team design requires specifying where AI is restricted while integrating ethics, privacy, and risk considerations (Reference Lawless, Mittu, Sofge, Shortell, McDermott, Lawless, Mittu, Sofge, Shortell and McDermottLawless et al., 2021), along with AI usability (Reference Fouad, Raz, Llinas, Lawless, Mittu, Lawless, Llinas, Sofge and MittuFouad et al., 2021). Thus, future research should continue investigating Human-AI team dynamics and team design along with general frameworks for ethical and privacy-conform AI (Reference Verma, Singhal, Das, Saha, Coello Coello and BansalVerma & Singhal, 2024). This includes unifying the current spectrum of definitions and concepts (Bruni, 2024 Madni, Reference Madni2023; Du Plooy & Oosthuizen, Reference Du Plooy and Oosthuizen2023).
5.4. Leveraging MBSE for AI integration
From the literature investigated in this review, it emerges that MBSE provides a foundation for AI integration, indicating that organisations using MBSE are better positioned to adopt AI as well. However, the literature offers limited insights into this dynamic. Future research is suggested to identify successful MBSE practices that enable AI integration, particularly to identify “low-hanging fruits” and develop best practices and guidelines for successful AI augmentation of MBSE.
5.5. Addressing technical and algorithmic limitations
AI systems face technical limitations such as hallucination, instability, and context limitations in LLM, which require improved algorithm design and improved use, such as better prompt engineering (Rudolph, Reference Rudolph2024; Du Plooy & Oosthuizen, Reference Du Plooy and Oosthuizen2023). Research should focus on the adoption of techniques to improve performance of current AI such as Retrieval-augmented generation (RAG) and graph-RAG approaches to address limitations of LLMs (Reference Edge, Trinh, Cheng, Bradley, Chao, Mody, Truitt and LarsonEdge et al., 2024). Furthermore, addressing the interoperability of AI outputs with SE modelling tools is essential to reduce manual pre-processing and corrections to streamline workflows(Bader et al., Reference Bader, Vereno and Neureiter2024)
6. Conclusion
This study identified several promising avenues for enhancing SE with AI support. Although research shows potential across all stages of the SE V-model, the outlooks are particularly promising in requirements engineering, systems architecture and modelling/MBSE. However, AI integration is currently hindered by barriers such as lack of training data, poor explainability and a need for more knowledge about Human-AI teams. Thus, future research is suggested to focus on how to overcome these barriers, for example, by collaborating with industry to co-create datasets, developing data and tool interoperability, and conducting further research into Human-AI teams related themes such as explainability, responsibility and roles.
This study contributes to SE and AI literature by increasing the understanding of current research findings and providing a catalogue of research directions to facilitate the integration of AI into SE practices. Thus, this study contributes valuable research priorities to the emerging research field of ASE. In the long run, this study also contributes to industry capabilities to build SE capabilities by augmenting it with AI. This study has some limitations, the first pertains to the search string, which focuses only on AI and SE in broad terms, introducing the risk of excluding relevant publications with a specialised focus. Another limitation relates to the research methodology and how the systematic literature review was conducted. To reduce bias and increase the reliability of results, a higher number of experts could have participated in the selection and coding of publications. Finally, the last limitation relates to the databases used. Although we used two established databases, it is possible that relevant papers were excluded if not indexed in one of the databases.
Acknowledgement
This research is supported by an Australian Government Research Training Program (RTP) Scholarship.
Open access
