Introduction
This article analyzes publicly available military doctrinesFootnote 1 focusing on artificial intelligence (AI)-driven warfare, and examines whether this new technology poses a threat to or reinforces the principle of humanity during armed conflict. In contemporary warfare, including during targeting decisions, there is an increasing use of AI decision support systems (AI-DSS).Footnote 2 These systems assist soldiers in aspects such as the selection of targets, which means and methods of warfare to use, and when the attack should take place. While the defenders of such human–machine partnerships argue that the use of AI in these contexts allows for “surgical warfare”, in practice, the use of AI can undermine the principle of humanity, jeopardizing an application and interpretation of international humanitarian law (IHL) which focuses on the protection of the individual in times of war.Footnote 3
The principle of humanity,Footnote 4 a foundational pillar of IHL, seeks to limit the suffering caused by armed conflict and to protect human life and dignity.Footnote 5 This principle not only prohibits inhuman treatment, but also serves as a qualifier of a certain standard of human behaviour, commonly referred to as humaneness.Footnote 6 Moreover, this principle allows for moral and ethical considerations to be taken into account when construing and ascertaining the normative content of specific rules. Thus, for instance, the principle of humanity underpins the duty to take precautions, ensuring that military operations are conducted with the utmost care to avoid unnecessary harm to civilians and civilian objects.Footnote 7
By examining how AI-DSS are used in the conduct of hostilities, this article underscores issues related to lack of human agency, the high speed at which operators must function due to the incredibly high number of targets that these systems generate, and the influence that the AI-DSS have on the human operator’s perception of reality. While acknowledging the potential of AI-DSS to enhance civilian protection and mitigate harm by gathering information from a great number of sources for intelligence and analysis, this article offers a critical examination of how the lack of incorporation of considerations mandated by the principle of humanity impacts compliance with IHL rules. The article concludes by calling for a set of principles and guidelines to ensure the responsible use of AI under IHL. It emphasizes a human-centred approach to integrate AI-DSS into the military domain, ensuring alignment with the principle of humanity and promoting a more ethical and responsible development and deployment of AI.
The use of AI in the conduct of hostilities
AI is generally recognized as a disruptive technology,Footnote 8 with a significant impact on armed conflicts both today and in the future.Footnote 9 For instance, the United Nations Office for Disarmament Affairs (UNIDIR) has indicated that AI has the potential to accelerate conflict dynamics, intensifying crises before humans can intervene.Footnote 10 The increasing use of AI in current armed conflicts is part of a broader trend of the datafication of warfare and automation of targeting.Footnote 11 As technologies develop at a very high speed, there is the risk of focusing too much on the tool and less on the effects. Therefore, “it is not the bullets, it is the speed at which they come” that must be addressed.Footnote 12
In the past few years, States have progressively integrated AI systems into the military domain in several ways, ranging from decision-making support to autonomous weapons. In 2017, the US Department of Defence launched Project Maven, an initiative designed to use AI for military targeting. Unlike earlier-generation systems without AI capabilities, Maven uses sophisticated algorithms to analyze large amounts of reconnaissance data, including video feeds, satellite imagery and radar signals. Project Maven not only identifies targets but also recommends optimal strategies for interacting with them.Footnote 13 More recently, the United States has reportedly deployed a new AI-powered system that enables faster decision-making and pre-emptive action against the Houthis in the Red Sea. This system uses AI to analyze satellite imagery and other intelligence sources, making it a critical tool for tactical operations.Footnote 14
China is reportedly also developing similar militarized AI technologies, utilizing its expertise in facial recognition and object detection. China’s military application of AI includes developing unmanned intelligent combat systems, improving battlefield situational awareness and decision-making, conducting multi-domain offence and defence, and facilitating advanced training, simulation and wargaming practices.Footnote 15
The armed conflict between Russia and Ukraine has been called the “first AI war” and an “AI war lab”.Footnote 16 It is considered to be the first armed conflict in which the opposing sides have actively developed and used AI for military purposes.Footnote 17 Ukraine has reportedly been using AI on unmanned systems mainly to minimize human exposure to direct combat,Footnote 18 employing a “robots first” strategy and a “tech navy” of maritime drones.Footnote 19
While AI has been used primarily in weapons systems in the Russia–Ukraine armed conflict,Footnote 20 in the Israel–Hamas conflict, Israel has reportedly been using AI-DSS applications such as Lavender, Gospel and Where’s Daddy? for targeting in the conduct of hostilities.Footnote 21 These AI-DSS have become a “force multiplier”, producing “targets at a fast pace” by analyzing large sets of data from signals and human intelligence, geographical intelligence, satellite imagery, drone feeds, facial recognition databases and other sources to identify targets for elimination.Footnote 22 The widespread use of AI-DSS in the Israel–Hamas armed conflict, in particular, is calling into question the effectiveness of these tools in reducing armed conflict-related destruction, given the high rates of civilian harm.Footnote 23
AI applications include intelligence, surveillance, reconnaissance; autonomous weapons systems (AWS) and drone swarms; military logistics and maintenance; command and control decision support systems; military training, simulation and modelling; and cyber warfare and defence.Footnote 24 AI is not a form of intelligence, but rather consists of “algorithms designed to recognise patterns and from these make decisions based on set rules”.Footnote 25 What is unique about this technology is its ability to “learn” based on the data through which it is trained and to generate solutions to particular tasks.Footnote 26 This means that in the context of targeting, for instance, it relies on “predictions based on pattern recognition and classification, through the generalization of data used during the system’s training”.Footnote 27 AI may respond differently to the same situation over time because it continuously learns and adapts based on the data fed into the system.Footnote 28 However, it is important to highlight that this continuous adaptation does not occur because AI systems can “think”, or because they are “intelligent”.Footnote 29 AI uses mathematics, processes large amounts of data and suggests possible outputs, but it has no capacity to understand or critically engage with the content that it produces or the data that it works with in the same way humans do.Footnote 30
From a humanitarian perspective, there are three areas in which the use of AI by conflict parties is particularly relevant: the integration of AI on AWS, the use of AI in the development of cyber weapons and misinformation operations, and the use of AI in “decision support” or “automated decision-making” systems.Footnote 31 Global debates on AI in the military domain have focused mainly on the use of AI in weapons systems, especially AWS,Footnote 32 but there are also other AI-enabled systems that go beyond their application in weapons and have a broad potential for integration into military decision-making processes – in particular, AI-DSS, which consist of “computerized tools that are designed to aid humans in making complex decisions by presenting information that is relevant for the decision or by proposing options for the decision maker to choose from in order to achieve a goal”.Footnote 33 Unlike AWS, these decision support systems do not, in themselves, autonomously engage in the use of force, but instead support or assist military personnel in decision-making, including for targeting processes.Footnote 34
The growing integration of AI-DSS in the means and methods of warfare is giving rise to so-called “algorithmic warfare”, in which AI-driven decision-making, surveillance and AWS play a central role in military strategy.Footnote 35 These algorithmic systems are accelerating the pace as well as the lethality of war, which could, in turn, increase civilian harm. AI-driven warfare raises concerns about compliance with IHL as meaningful human oversight is increasingly eroded in favour of technological efficiency.Footnote 36 This article argues that the IHL principle of humanity can serve as a guide for the responsible use of AI in the military sphere, strengthening compliance with IHL and improving the protection of the civilian population.
The principle of humanity in IHL
The principle of humanity imposes certain limits on the means and methods of warfare; its purpose is to protect life and health, and to ensure respect for the human being.Footnote 37 Although there is a generalized agreement that humanitarian considerations have become increasingly important (especially following the drafting of the 1949 Geneva Conventions, which were primarily driven by such concerns),Footnote 38 the legal meaning and scope of the principle of humanity in IHL continues to be subject to discussion. This is even revealed in the name used to refer to this branch of law, which has evolved from “the law of armed conflict” to “international humanitarian law”.Footnote 39 In this regard, Meron noted how, through the influence of international human rights law (IHRL) and the principle of humanity, IHL has been undergoing a process of “humanization”, which means that this branch of law has been changing and acquiring a more “humane” face.Footnote 40
Defining humanity
Humanity can be defined as “restraining the capacity for armed violence and limiting its effects on security and health”.Footnote 41 Nevertheless, the concept of “humanity” remains ambiguous, and it is not defined in any international legal instrument. Mention of the “laws of humanity” appears in instruments as early as the 1868 St Petersburg Declaration; the 1899 Hague Convention II and 1907 Hague Convention IV also use this phrase in the Martens Clause.Footnote 42 These instruments had a limited protective scope, as they mainly dealt with the protection of soldiers, especially when sick or wounded, and not civilians.Footnote 43
The 1949 Geneva Conventions similarly refer to the “laws of humanity” in their “denunciation” clause inspired by the Martens Clause.Footnote 44 Additional Protocol I (AP I) of 1977 reaffirms the Martens Clause in Article 1(2) regarding general principles, keeping the original wording except for the reference to “principles” instead of “laws” of humanity.Footnote 45 According to the Commentaries on the Additional Protocols of 1977, there were two reasons for including this clause again in AP I. First, the Martens Clause avoids the assumption that anything not explicitly prohibited by the relevant IHL treaties is permitted. Second, the clause should be conceived as a dynamic element that declares the applicability of the principles despite subsequent developments in the types of circumstances or new technologies.Footnote 46
The fourth paragraph of the preamble of Additional Protocol II (AP II) establishes that “in cases not covered by the law in force, the human person remains under the protection of the principles of humanity and the dictates of the public conscience”, while the Commentary on AP II considers that “the principles of humanity actually constitute a universal reference point and apply independently of the Protocol”.Footnote 47 Therefore, the “principles of humanity” can be considered the minimum humanitarian threshold that parties to a conflict must respect in times of armed conflict, whether international or non-international, regardless of any regulation.
Legal scope of the principle of humanity
The principle of humanity’s legal scope has also been subject to various interpretations. Most scholars acknowledge that the principle of humanity is a guiding principle which permeates the whole body of IHL, underlying and informing it;Footnote 48 therefore, it serves as a guide for interpreting the more specific provisions of IHL governing the conduct of hostilities.Footnote 49 Some argue that the principle of humanity includes categorical imperatives like “do not perform this military action (in this way, at this time, and so on) regardless of your aims”.Footnote 50 Thus, the principle of humanity is seen as a universal principle in which “humanitarian responsibility extends to all parties involved in war and with war”,Footnote 51 and the International Law Commission has referred to it as, together with the principle of military necessity, “one of the two cardinal principles of the law of armed conflict”.Footnote 52 However, contrary to the foregoing approach, some scholars attribute a residual role to the principle of humanity, understanding it as mere “considerations of humanity” which, while “inspiring and instrumental”, are “no more than considerations”, reducing the principle’s normative value to extra-legal conditions.Footnote 53
Thus, the principle of humanity can serve an “architectural or axiological function” as well as, simultaneously, an “auxiliary function”.Footnote 54 By “architectural or axiological function”, this article refers to the understanding of the principle as a foundational principle which informs the whole corpus of IHL, as it has an overarching nature.Footnote 55 Therefore, it cannot be reduced to its famous epitomizations, such as the principle that prohibits unnecessary suffering and the Martens Clause, because as a foundational principle, it transcends these manifestations, which are representations of its overarching scope. Moreover, although it cannot supersede specific rules of IHL, the principle’s auxiliary function serves as assistance in their interpretation and development, especially in relation to the indeterminacies they contain, or when their application is abstruse.Footnote 56 This means that when there is no rule in IHL covering a specific situation, the parties to the conflict must abide by the principle of humanity, which must guide their actions,Footnote 57 and that when there is a rule that lends itself to an unclear interpretation, the principle serves as a general interpretative guideline.Footnote 58
The principle of humanity can also be understood as a safeguard of human dignity.Footnote 59 Pictet framed IHL as “that considerable portion of international law which is inspired by a feeling for humanity and is centred on the protection of the individual in time of war”;Footnote 60 additionally, IHRL and IHL share the ideal of the “protection of the dignity and integrity of the person”.Footnote 61 The International Criminal Tribunal for the former Yugoslavia (ICTY) also supported such an understanding, as it considered that both IHL and IHRL “take as their starting point the concern for human dignity which forms the basis of a list of fundamental minimum standards of humanity”.Footnote 62 The idea of dignity further reflects how people should be considered as separate, irreplaceable individuals.Footnote 63
Applicability of the principle of humanity
Although the principle of humanity can be interpreted in different ways,Footnote 64 it has been invoked primarily in relation to the prohibition of unnecessary suffering and to the humane treatment of civilians and persons hors combat, and as a counterbalance to the principle of military necessity. It has also been invoked, especially in the ICTY’s jurisprudence, by reference to the Martens Clause.Footnote 65 The principle of humanity, then,
provides the foundation for both the prohibition against the subjecting of an opponent to superfluous injury or unnecessary suffering – injury or suffering beyond that which is necessary to bring about the opponent’s prompt submission – and the obligation to treat humanely those who are no longer, or never were, active participants in armed hostilities.Footnote 66
Military manuals also tend to refer to the idea of humanity, but they do so in the context of the limitation of violence. For example, the Australian Defence Force’s Law of Armed Conflict guidance indicates that the avoidance of “[u]nnecessary suffering is often also referred to as the principle of humanity”,Footnote 67 while the United States’ Insurgencies and Countering Insurgencies manual understands humanity as a requirement of armed forces to avoid “inflicting gratuitous violence on the enemy”.Footnote 68
Additionally, in customary IHL, humanity is invoked regarding humane treatment of civilians and persons hors combat,Footnote 69 treatment of displaced personsFootnote 70 and reprisals against protected persons,Footnote 71 although it is not expressly mentioned in the customary rule’s definition. International tribunals have mostly referred to the principle of humanity when discussing inhumane treatment to condemn acts of torture and inhuman or degrading treatment.Footnote 72
Moreover, the principle of humanity is often considered in conjunction with the principle of military necessity as the “counterbalanced framework that underpins IHL”,Footnote 73 which means that States “need not allow for the notions of humanity or military necessity as separate, free-standing, considerations from the rest of IHL”.Footnote 74 These restraining considerations were defined by the International Court of Justice (ICJ) as a “fundamental principle” and “overriding consideration”, though the Court also noted that they were not absolute.Footnote 75 This is because, as a realistic body of law,Footnote 76 IHL balances the principle of military necessity, which permits certain levels and acts of violence in armed conflict, with humanitarian considerations, which restrain the levels of violence allowed during the conduct of hostilities. Therefore, IHL’s aim is to limit the destruction and suffering caused by armed conflict by ensuring that some minimum standards of humanity are respected.Footnote 77
In addition to the principle of distinction, which requires parties to an armed conflict to distinguish at all times between civilians and combatants and between civilian and military objectives, the rule of proportionality in attack establishes that “launching an attack which may be expected to cause incidental loss of civilian life, injury to civilians, damage to civilian objects, or a combination thereof, which would be excessive in relation to the concrete and direct military advantage anticipated”, is prohibited.Footnote 78 AP II, which covers non-international armed conflicts, does not refer explicitly to the principle of proportionality in attack, but it has been argued that the principle also applies in this type of armed conflict because it is inherent in the principle of humanity as mentioned in the preamble of the Protocol. As noted by Bothe, Partsch and Solf, “the reference to the principles of humanity in the Martens clause of [paragraph 4 of the] Preamble inherently prohibits the destruction of values which are not relevant and proportionate to the military advantage anticipated”.Footnote 79
Accordingly, the principle of humanity can be considered a fundamental principle that permeates the entire body of IHL and restrains the available means and methods of warfare in order to protect life and health and respect for the human being.Footnote 80 The principle of humanity, originally referred to as the “laws of humanity” in the Martens Clause, encompasses the prohibition of unnecessary suffering and the right to humane treatment of civilians and persons hors de combat and serves as a counterweight to military necessity in armed conflict. Furthermore, it serves as a guiding principle for the interpretation of the rights and duties of parties to armed conflicts.
AI-DSS and the principle of humanity: Risks and potential
The use of AI-DSS in armed conflict raises many ethical and legal concerns. In the current armed conflicts in Russia/Ukraine and Israel/Gaza, where AI-DSS are reportedly being widely used in weapons and as support for decision-making of human operators, these technologies are often presented as enhancing precision and efficiency; however, it remains highly contested whether their use has led to a reduction in civilian harm in compliance with IHL. This section addresses the challenges and opportunities that the use of AI-DSS in the conduct of hostilities presents for the fulfilment of the principle of humanity in compliance with IHL. While most of the literature has focused on the risks of using AI-DSS in the military domain, which are of course a cause for great concern, limited attention has been paid to the potential of these technologies to enhance the protection of civilians in wartime and mitigate harm, which is one of the purposes of the principle of humanity.
AI-DSS as a threat of the principle of humanity
The following analysis examines the main risks linked with the use of AI-DSS in the military domain – namely, the dehumanization of victims, the erosion of operators’ human agency as a result of automation bias, the introduction of bias within AI-DSS, and the opacity of so-called “black box” systems – and argues that these risks collectively threaten the principle of humanity in the conduct of hostilities. In particular, they undermine efforts to prevent civilian harm and unnecessary suffering, eroding the human dignity of both victims and operators and weakening the principle of humanity as a counterbalance to the principle of military necessity.
One of the most apparent impacts of the use of AI-DSS and other algorithmic systems in armed conflict is the dehumanization of victims, who, in the targeting process, are reduced to data points used to power the algorithms.Footnote 81 Dehumanization entails the deprivation of victims of their human status, as well as the reduction of their moral status, as they become objects.Footnote 82 In the case of digital dehumanization, this is specifically defined as
the process whereby humans are reduced to data, which is then used to make decisions and/or take actions that negatively affect their lives. This process deprives people of dignity, demeans individuals’ humanity, and removes or replaces human involvement or responsibility with automated decision-making in technology.Footnote 83
This form of digital dehumanization not only deprives victims of their human dignity but also risks undermining compliance with the principle of proportionality inherent in the principle of humanity, particularly in relation to the assessment of the concrete and direct military advantage anticipated.
At the same time, there is a risk of dehumanizing operators by using AI-DSS; this goes beyond mere functionality and refers to the ability to assess whether these systems comply with fundamental principles of IHL, such as the principle of humanity.Footnote 84 Use of AI-DSS can undermine IHL compliance as it can lead to a gradual removal of operators from targeting decisions and, therefore, a removal of monitoring of IHL observance, which is also hindered by the way AI-DSS are employed.Footnote 85 Moreover, the physical distance detaches operators, who can easily experience no regret as there is limited awareness of the consequences of their use of these systems, which are nothing less than the killing of human beings.Footnote 86 It also challenges the principle of humanity’s manifestation related to human dignity, since as indicated above, both potential victims and operators can be objectified, the former as a dataset and the latter as a “rubber stamp”.Footnote 87
The current use of AI-DSS in human–machine partnerships to conduct warfare may also undermine the operators’ human dignity. This is so because their role is easily reduced to a box-ticking function, leaving little room to exercise human agency through the displacement of discretionary legal reasoning. Importantly, dignity encompasses the ability to be a moral agent, meaning, to be autonomous and capable of exercising moral choices and being responsible for their outcomes,Footnote 88 all of which the use of AI-DSS may hinder. In this regard, operators’ agency when using AI-DSS should be exercised in a contextual, qualitative and value-laden fashion.Footnote 89 As is reflected in the literature that discusses how human and AI-DSS partnerships operate, the erosion of human agency is mainly produced by the increased speed and scalability that AI-DSS generate, the biases occurring in this type of relationship, and the “black box” nature of AI-DSS’ functioning.Footnote 90
One of the biggest added values of the incorporation of AI-DSS in the targeting cycle is their capacity to process large amounts of data which humans, by themselves, would be unable to analyze. This feature is a double-edged sword, however, because the high speed at which AI-DSS produce outputs, and the scalability that this entails, can easily overwhelm the human operator and therefore prompt him or her towards a logic of expediency, adopting an approach based on speed and optimization rather than moderation.Footnote 91 This poses a significant challenge when it comes to compliance with the obligation of meaningful human oversight, a consideration mandated by the principle of humanity, and can result in an increase in civilian harm, prompted by the AI-DSS–human partnership’s logic of expediency.
This dynamic is also conducive to the operator’s automation bias, the human tendency to rely on the output suggested by the AI-DSS without meaningful interrogation or critical engagement, which is especially present in high-pressure or stressful settings and typical in armed conflict.Footnote 92 The large volume of potential targets produced by AI-DSS may create pressure for more strikes due to this cognitive action bias, which leads operators to over-trust the AI-driven target recommendations. Moreover, the use of AI-DSS in the military domain makes it more difficult to detect erroneous outputs, as humans increasingly struggle to understand how and why these systems generate specific analyses, recommendations or predictions. This is an issue closely related to the so-called “black box” phenomenon,Footnote 93 which will be discussed below. Thus, this automation bias, or over-trust, can result in an over-reliance on the machine output, “especially if the output fits with the users’ expectations”.Footnote 94 This modus operandi challenges the principle of humanity in two ways: firstly, it undermines meaningful human oversight, since human operators, due to cognitive automation bias, may be inclined to accept the AI-DSS’s suggestions without meaningful questioning; and secondly, automation bias may also place the dignity of victims at risk if individuals are treated as mere data points rather than as human beings.
Compounding the operator’s bias, it is also important to highlight algorithmic bias, which occurs when machine learning modules under- or over-include certain types of data, leading to faulty outcomes.Footnote 95 Furthermore, through the annotating, labelling or classification of data by human operators in the training phase, the algorithmic bias can be exacerbated.Footnote 96 As has been observed in facial recognition software, AI-DSS’ accuracy rate is higher when it comes to identifying male faces compared to female faces, and people with lighter skin tones compared to people with darker skin.Footnote 97 This relates to the quality of the data used to train the algorithm, as its outcomes are only going to be as accurate as the data inserted; the question then arises as to what extent the data used to train the algorithms can be accurately used in armed conflict, as “conflict environments are harsh, dynamic and adversarial, and there will always be more variability in the real-world data of the battlefield than in the limited sample of data on which autonomous systems [or AI-DSS] are built and verified”.Footnote 98 Moreover, only quantifiable data can be included, as AI-DSS cannot as yet undertake a qualitative analysis. This means that concepts which cannot be quantified, such as the principle of humanity, risk being left out of this human–machine partnership altogether when warfare becomes increasingly data-driven.Footnote 99 This, in turn, endangers the human dignity of the potential victims, which is safeguarded through the principle of humanity, as their reduction to data points fails to capture their complexity as human beings. Lastly, it can also enhance civilian harm, especially if the inserted data and/or the system’s training process are flawed, as it can lead the system to suggest military targets which are in fact civilian persons or civilian objects.Footnote 100
Lastly, the “black box” phenomenon,Footnote 101 compounded by the high speed and scale of outputs, reduces human agency even further. This term “black box” refers to the lack of human explainability of AI-DSS’ outputs,Footnote 102 which means that, even if the environment in which the human–AI-DSS partnership operated was not stressful and there was time for a meaningful engagement with the system’s outputs, the capacity of the operator to understand the AI-DSS’s mechanism would be limited.Footnote 103 The term “explainability”, commonly used in discussions on AWS, usually refers to technical measures that “explain” black-box AI systems, while the term “understandability” refers to the degree to which AI-DSS can be understood by their human operators.Footnote 104 Without an understanding of why an AI-DSS is doing what it does, it may be hard to evaluate if it is operating in accordance with the principle of humanity, and it may be difficult to anticipate the system’s future actions. Moreover, the black box phenomenon challenges the requirement of meaningful human oversight mandated by the principle of humanity, as the AI-DSS is unable to communicate to humans the rationale behind its recommendations and suggestions, and because the AI’s process may be founded on patterns which humans cannot perceive or understand.Footnote 105
The use of AI-DSS in the context of targeting decisions thus raises a wide range of issues related to the intrinsic characteristics of these technologies or the way in which they are employed, which, as indicated, pose significant challenges to compliance with the principle of humanity and the considerations that it mandates, especially those related to meaningful human oversight, the safeguarding of human dignity and the reduction of civilian harm. The opacity of AI-DSS contributes to a form of digital dehumanization that affects the ability of human operators to adequately assess the requirements of IHL in terms of military necessity and proportionality, thereby eroding the principle of humanity as an important constraint on target selection decisions in the conduct of hostilities.
AI-DSS as a guardian of the principle of humanity
One of the main concerns around the use of AI-DSS in the military domain is whether such systems are inherently indiscriminate and unable to distinguish between lawful and unlawful targets.Footnote 106 Considering the principle of humanity, and specifically its impact on the obligation to mitigate civilian suffering, our analysis now turns to the extent to which AI-DSS can enhance the protection of civilians from direct and indiscriminate attacks and mitigate civilian harm and destruction. To this end, the following paragraphs will examine the potential of AI-DSS to improve general civilian protection, the way in which they can facilitate intelligence-gathering and analysis, their potential to mitigate civilian harm, and the automation of manual tasks to enhance civilian protection.
AI facilitates quicker and more widespread collection and analysis of data, which can, in turn, assist in the minimization of risks for civilians and increase general civilian protection.Footnote 107 Such a data-driven approach would not focus on identifying potential targets – which could reduce individuals to mere data points – but would rather aim at, for example, collecting information that would alert conflict parties to the presence of transient civilians whose appearance would otherwise most likely be missed, or collecting data that would improve the accuracy of the estimation of the anticipated collateral damage.Footnote 108 According to the surveyed practice, it seems that when developing and using AI, governments have focused on investing in faster and more precise targeting decisions in armed conflict instead of improving the protection of civilians more broadly.Footnote 109
This is important to underscore as AI-DSS can only be as accurate as the data through which they learn and which they use to deliver recommendations. Indeed, the integration of AI and machine learning technologies into decision support systems is considered to improve these systems by making them more adjustable to changing situations through their ability to “learn” from large volumes of data.Footnote 110 Unfortunately, as noted above, the data currently available to support the enhancement of civilian protection during armed conflict may be insufficient to effectively train AI-DSS. Furthermore, even if the existing data proves adequate, it may only be applicable to specific contexts or narrowly defined protective purposes. This means that its ability to be generalized or extrapolated to other situations would be rather limited.
This issue is essential as the value of AI systems rests on their ability to support human decision-making in an accurate fashion, thereby improving the overall outcome. Therefore, in order for AI-DSS to constitute a set of promising avenues to advance the protection of people affected by conflict, it is crucial to ensure the quality of the data through which these systems are trained and applied. To achieve this outcome, States ought to compile information related to civilian protection, which could help militaries to understand the way in which civilian harm happens in the first place.Footnote 111 Thus, militaries could collect data related to how civilian harm occurs (for instance, because of misidentification or as collateral damage) and also track the “date, time, location of incident, unit involved, type of operation, number of [incidents of] civilian harm, and other details that could be used for consequence management and trend analysis”.Footnote 112 It should be acknowledged that parties to the conflict may face significant difficulties in terms of the availability of and access to such information; in order to overcome these challenges, States should invest in developing methods to account for civilian harm, which is also a requirement derived from the obligation to take constant care.Footnote 113 This obligation relates to the principle of humanity in the sense that it acts as a reminder that States are required to mitigate risk and prevent harm.Footnote 114
Throughout the targeting process, the gathering and analysis of information is of paramount importance to complying with the targeting rules. This is because “IHL requires the decision-maker to have a reasonable level of certainty that a target is a valid military objective”.Footnote 115 For instance, Article 57(1) of AP I imposes on the parties the obligation to take constant care to spare the civilian population and civilian objects from the effects of hostilities. This obligation reflects the principle of humanity insofar as it establishes that legitimate military operations must consider the protection of human life and dignity in order to minimize harm to the civilian population. It constitutes an ongoing obligation of conduct, applicable throughout the conduct of hostilities (not only during attacks, but also, for instance, in ground or search operations),Footnote 116 aimed at sparing civilians and civilian objects to the greatest extent possible,Footnote 117 but not in absolute terms; rather, this provision requires commanders to take reasonable steps to eliminate or, if not, minimize the negative effects of military operations on the civilian population.Footnote 118 In this regard, AI-DSS could play a crucial role in assisting commanders in the collection and assessment of information available to them at the relevant time,Footnote 119 and as indicated, this could in turn result in a mitigation of civilian suffering, which is a basic manifestation of the principle of humanity.
In the context of the targeting process, some experts have concluded that AI can be used to help address patterns of destruction in order to reduce the likelihood of harm, such as by alerting conflict parties to the presence of transient civilians or a potential miscorrelation, or recognizing protected symbols.Footnote 120 Taking into account the principle of humanity’s architectural function, which refers to its role as a guiding principle that permeates, underlines and informs IHL’s specific rules, AI-DSS could be used in the context of the principle of proportionality to detect a change in the estimation of collateral damage or to accurately assess the foreseeable indirect effects that a specific planned attack might generate, which could be particularly useful in urban warfare contexts. This, in turn, would satisfy the principle of humanity’s purpose of reducing civilian harm.
However, as underscored in the context of the use of explosive weapons in urban contexts, it has proven particularly difficult to take into consideration these indirect, “knock-on” or reverberating effects. This is because they include not only loss of civilian life, injury to civilians and damage to civilian objects, but also, for example, the “incidental destruction of civilian houses [that is likely to] result in large-scale displacement”.Footnote 121 In this context, AI-DSS could prove particularly useful as they could include large quantities of data from across different sources related to the potential indirect effects of a planned attack which would be more difficult for the human operator to take into consideration.
Furthermore, developing technologies oriented to mitigate civilian harm aligns with States’ obligation to take precautionary measures, which is also one of the strongest manifestations of the principle of humanity. According to their financial, technical and human capacity to develop such technologies, States ought to ensure that AI-DSS are put to good and diligent use.Footnote 122 For instance, attacking parties are generally obliged to issue “effective advance warnings”,Footnote 123 except in cases where the surprise nature of the attack precludes it. Effectiveness refers to the fact that warnings must be timely and must provide safe routes and destinations;Footnote 124 in this regard it can also be particularly useful to employ AI-DSS to assist the human operator in ensuring the effectiveness of the warning, basing its recommendations on the characteristics of the civilian population, obtained through a fast and broad collection and analysis of intelligence from various sources.
Lastly, given the limitations of human decision-making, AI systems can help to automate manual tasks, identify patterns in complex datasets and accelerate the overall decision-making process.Footnote 125 Although not related to the targeting process, AI-DSS are already being used for humanitarian purposes in States like Ukraine, where the government has developed AI-DSS to advise on issues related to demining, such as how to schedule demining requests and customize alerts.Footnote 126 In conclusion, while AI-DSS in the military domain can advance the principle of humanity’s protective scope, especially in relation to the mitigation of civilian suffering, this potential can only be realized if it focuses on improving civilian protection, enhancing intelligence collection and analysis, and effectively reducing civilian harm in military operations.
The need for a normative framework for AI-DSS to comply with IHL
While AI systems may be a significant element in modern warfare, they are not a “catch-all solution to political and military problems”.Footnote 127 Since the growing use of AI in war is inevitable, it is essential to develop guidelines and normative frameworks for the responsible use of AI in armed conflict in order to ensure compliance with the principle of humanity and, ultimately, IHL. In recent years, some initiatives have already been launched in different global forums.
In 2021, the North Atlantic Treaty Organization (NATO) adopted its first AI Strategy, setting out as a core component the Principles for the Responsible Use of AI in Defence – namely, responsibility and accountability, explainability and traceability, reliability, governance, and bias mitigation.Footnote 128 In its 2024 Revised Strategy, NATO included generative AI and AI-enabled information tools and reaffirmed the implementation of the six Principles of Responsible Use as a priority.Footnote 129 Although the NATO AI Strategy does not explicitly refer to the principle of humanity, it requires that “clear human responsibility shall apply” to the development and use of AI applications, that such applications should include “appropriate human-machine interaction”, and that they should comply with IHL and IHRL. Additionally, in a 2024 report, the NATO Parliamentary Assembly highlighted the ethical and legal implications of AI in military applications in relation to compliance with the fundamental principles of IHL, including distinction, military necessity, humanity and proportionality.Footnote 130
In September 2024, a Blueprint for Action was adopted at the second Responsible AI in the Military Domain (REAIM) Summit in Seoul, South Korea, which gathered more than 2,000 people from more than ninety countries, representing governments, industry, academia and international organizations. The Blueprint for Action was formally endorsed by sixty-one countries, including the United States, the United Kingdom, Japan and Australia, and stressed the need to apply AI systems in the military domain “in a responsible manner thought their entire life cycle and in compliance with applicable international law, in particular, international humanitarian law”.Footnote 131 To ensure the responsible use of AI in the military domain, the Blueprint for Action establishes that AI applications should be ethical and human-centric; should respect national and international law; should have human responsibility and accountability; should have reliability and trustworthiness to reduce risks or unintended consequences; should have appropriate human involvement in the development, deployment and use of AI in the military domain; and should have explainability and traceability. Although it includes fundamental principles like guaranteeing that AI applications comply with international law and have a human-centric approach, around thirty States did not endorse the Blueprint at the Seoul Summit. The absence of consensus among States on common minimum standards for the responsible use of AI in the military sphere demonstrates that this issue remains controversial internationally.Footnote 132 As with NATO’s AI strategy, the Blueprint makes no explicit reference to the principle of humanity and only refers to the need to respect IHL in general.
At the European level, in 2019, the independent High-Level Expert Group on AI, set up by the European Commission, presented its Ethics Guidelines for Trustworthy Artificial Intelligence. The Guidelines put forward a set of seven key requirements that AI systems should meet to be deemed trustworthy, namely human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental well-being; and accountability.Footnote 133 These requirements are essential in the context of military applications, given the emphasis on trust and human oversight in high-risk domains.Footnote 134 Trustworthy AI systems in the military sphere are conceived to give “confidence among commanders, soldiers, and policymakers by ensuring that AI technologies operate predictably, securely, and ethically in complex and dynamic environments.”Footnote 135 Therefore, “ethics must be applied in practice and cannot remain as a set of abstract concepts, lists or tick-box exercises that stakeholders do not fully understand”, and everyone has to understand how the life cycle of AI-DSS impacts “the preservation of life and peace for current and future generations”.Footnote 136
In 2024, the European Union (EU) adopted the EU AI Act establishing a regulatory framework for AI applications within the EU.Footnote 137 This regulation has taken a risk-based approach, ensuring that requirements are proportional to the identified risks, thus maintaining a balance between innovation and safety. While the EU AI Act covers a wide range of AI applications, it explicitly excludes the military domain.Footnote 138 However, there are AI systems that can be considered dual-use goods which can be used for both military and civilian purposes, such as drones or biometric recognition systems. Insofar as these systems have a civilian use available on the European market, they may still fall within the EU AI Act’s scope.Footnote 139 Despite the exclusion of the EU AI Act in the military domain, the general framework as well as the horizontal obligations of the Act may be relevant to AI systems in the military sphere and may encourage some convergence of relevant standards.Footnote 140
In December 2024, the UN General Assembly adopted a resolution on the military use of AI and its implications for international peace and security and stressed the need to ensure responsible deployment of AI in the military domain, calling for “human-centric, accountable, safe, secure and trustworthy artificial intelligence used in compliance with international law”.Footnote 141 Along the same lines, the recommendations made by the International Committee of the Red Cross (ICRC) to the UN Secretary-General highlight the need to use AI-DSS in accordance with IHL and humanitarian principles and establish as a guiding principle the need to “preserv[e] human control and judgment”.Footnote 142 The ICRC has stressed that while IHL does not explicitly prohibit the use of AI in military applications, it does restrict its development and use, as in the case of the prohibition of indiscriminate AI-enabled weapons and AI-enabled biological and chemical weapons.
As States are increasingly adopting national strategy plans on the use of AI in the military domain,Footnote 143 and in light of the growing ethical, legal and operational concerns of such use, this article argues that adopting a comprehensive set of guiding principles and standards at the international level could aid States in the development of AI-driven means and methods of warfare in compliance with IHL. The adoption of an international treaty often seems optimal in terms of its binding nature, but the discussions on the adoption of an international legal instrument to regulate AWS have lasted more than a decade without reaching a consensus.Footnote 144 Therefore, at this stage, it seems unlikely that an international treaty regulating AI in the military field which can simultaneously consider the extremely rapid advances in AI systems will be adopted any time soon.
At the core of the framework proposed by this article lies the principle of humanity, which prohibits the infliction of unnecessary suffering, encompasses the right to humane treatment of civilians and requires constant regard for the protection of human dignity. The adoption of a soft-law instrument can contribute to consolidating opinion around the need for action in the responsible use of AI in the military domain, filling the gap in existing IHL norms and providing guidance for domestic laws.Footnote 145 Soft-law initiatives based on policy and good practice “constitute a pragmatic response to humanitarian concerns when faced with the reluctance of States to engage in lawmaking clarification processes”.Footnote 146 Also, when these instruments are endorsed by States, they can lead to positive changes in military manuals or in State practice in the conduct of hostilities.Footnote 147
Existing instruments such as the REAIM Blueprint for Action (2024), UN General Assembly Resolution 79/239 on the military use of AI (2024) and the NATO Principles for the Responsible Use of AI (revised in 2024) stress the importance of human control and compliance with IHL, but they generally fail to articulate a substantive, human-centred standard capable of guiding the design choices, operational constraints and decision-making architectures of AI-enabled systems. Incorporating the principle of humanity as a core element of these standards would require States and armed forces to ensure that AI systems are designed, tested and used in ways that preserve human oversight, prevent the dehumanization of victims, and reduce harm to civilians that may result from those systems. This framework would not only strengthen existing IHL obligations but would also provide a normative framework for emerging military technologies, ensuring that technological innovation remains compatible with the protection of people affected by armed conflict.
The adoption of this set of guiding principles and standards should involve different stakeholders, such as international organizations, governments, militaries, scientists, engineers, academics, lawyers, and humanitarian and human rights organizations, among many others. A holistic approach is essential for the regulation of AI-DSS, as technologies advance at a high speed and imply numerous ethical, legal and military concerns.
Using the principle of humanity as the overarching guide and informed by the emerging international regulatory frameworks mentioned above, this article proposes the following set of guiding principles and standards for the responsible use of AI under IHL:
• Appropriate human involvement and oversight: humans must retain ultimate responsibility for targeting decisions and, more broadly, for all decision-making in order to preserve human agency and responsibility in warfare and promote compliance with IHL.
• Respect for human dignity through the integration of safeguards in the design and use of AI-DSS to prevent the digital dehumanization of operators and improve protection of the civilian population during hostilities.
• Responsibility and accountability: establishment of mechanisms that can attribute responsibility in case of malfunctioning or unlawful damage caused by AI systems.
• Reliability: development of AI systems that can be trusted considering the constantly changing and complex environment of armed conflict.
• Explainability, understandability and traceability: AI-enabled decisions should be explainable, and it should be possible to trace the whole cycle of AI-enabled decision-making in the conduct of hostilities.
• Bias mitigation: efforts should be made to prevent and correct bias in data and algorithms, as well as in the way they are used, that can lead to discriminatory or incorrect decisions in the conduct of hostilities.
• Prevention of harm: use of AI-DSS that prioritizes protection of civilians and reduces civilian harm.
• Ethical and human-centric application of AI systems: actors should guarantee that the design, deployment and use of AI are guided by ethical values and focused on human well-being.
While this is not intended to be an exhaustive list, the proposal put forward in this article can help to identify the key elements to advance in the regulation of the responsible use of AI in the military domain, from a human-centric perspective and based on the principle of humanity, in order to encourage compliance with IHL.
Conclusion
This article concludes that AI-DSS, when used in the conduct of hostilities, can, on the one hand, pose a threat to the principle of humanity by eroding meaningful human oversight and increasing civilian harm. On the other hand, AI-DSS has the potential to improve general civilian protection and to mitigate civilian harm, but States and private companies that design and develop AI systems are not necessarily focusing their investment in this direction. Therefore, it is necessary to produce a set of guiding principles and standards to help States advance in the responsible use of AI under IHL, with a human-centred approach, in order to integrate AI-DSS into the military domain in a way that respects the principle of humanity. The proposed international instrument allows for adaptation to the constant evolution of AI systems in the military field and helps States to adopt their national strategic AI defence plans and military manuals in accordance with IHL. To this end, the principle of humanity can be a guiding beacon for the responsible use of AI in the military sphere, like a lighthouse in the night, reminding us of our duty to protect human life and dignity during armed conflict.
Open access
