1. IntroductionFootnote 1
As AI systems become more pervasive and more reliable, government leaders may delegate some of their state’s resort-to-force decisions to those systems – perhaps in the form of autonomous defensive systems intended to respond instantaneously to incoming attacks. Such delegations will be controversial: such systems may make mistakes, and delegating use of force decisions raises familiar questions about where to situate accountability for those decisions. This article analyzes what international legal responsibility arises when a state introduces AI or autonomy into a resort-to-force system and the system acts in an unintended, harmful way.Footnote 2 To make the questions more concrete, the article offers three scenarios to show different ways that things could go wrong, tease out the legal challenges, and offer some tentative answers.
The literature on state responsibility for errors in the jus ad bellum (JAB) and, relatedly, on the standards of care that states must follow when making self-defense decisions is thin. The literature on the legal standards of care that states must follow when deploying AI or autonomous systems in self-defense appears to be nonexistent. This is true even though many states and scholars are concerned that the use of AI in military settings may make it difficult to hold any one actor accountable for AI-driven errors. This article sets aside criminal law (which would apply to the crime of aggression and to individual war crimes under international humanitarian law (IHL)). It focuses exclusively on state responsibility for internationally wrongful acts and, more specifically, on when mistakes by a state’s AI system would be internationally wrongful. It tentatively concludes that the baseline standard of care under the JAB requires a state to act in good faithFootnote 3 and in an objectively reasonable way, and it identifies some steps that a state could take to demonstrate objective reasonableness when deploying AI or autonomy in its resort-to-force systems. If states were to conclude that those steps were unnecessary under an “objective reasonableness” test, then this article suggests in the alternative that states should consider following a higher “abundance of caution” standard in this setting. And because a use of force in self-defense must also comply with IHL, this article analyzes the general standard of care for IHL in this setting. It concludes that, unless a particular provision of IHL contains a specific standard of care, the general standard of care under IHL is objective reasonableness.Footnote 4
Identifying the correct JAB and IHL standards of care offers three benefits. First, reaching conclusions about the responsibility of a state for an internationally wrongful AI-driven act might affect whether and how states decide to use such systems ex ante. Second, understanding what legal standards of care do (or should) apply to force initiation by AI systems will provide greater guidance to states that deploy such systems. Third, if states reach a rough consensus in advance about how the law allocates responsibility in a scenario in which an AI-driven act of self-defense goes awry, victim states may react more predictably in the face of AI mistakes and consequently reduce the chance that AI accidents unnecessarily escalate into full-blown armed conflicts.
Because there is so little state practice and existing scholarship on the standard of care in these situations, this article’s analysis and conclusions must necessarily be preliminary and tentative. However, because these issues are both important and pressing, the goal of the piece is to provide a starting point for a broader conversation.
2. “That was not supposed to happen”: three scenarios
This Part sets out three resort-to-force scenarios that might arise if states deploy autonomous AI-driven systems in self-defense. In each scenario, the autonomous system acts in an unexpected manner in a setting that implicates the JAB (i.e., the UN Charter and customary rules that include necessity, proportionality, and immediacy). Further, when a state undertakes an act of self-defense, that action must comply with the IHL rules of distinction, proportionality, and precautions (Kretzmer, Reference Kretzmer2013). Therefore, the “accidents” in the scenarios implicate both the JAB and IHL.
2.1. Scenario 1: the flawed AI system
Imagine that State A builds and deploys a fleet of armed autonomous underwater vehicles (AUVs) to serve as self-defense systems that can respond in the wake of an armed attack or to an imminent threat thereof. The systems are intended to stay embarked at sea indefinitely and to launch missiles at the source of attack or threat of attack when the systems perceive that State A is coming under attack by air or sea. Assume that State A’s military put the AUVs through the military’s standard testing process, which includes testing the systems’ performance while in development and then testing again under realistic operational conditions. Due to a malfunction of their AI systems while deployed, however, the AUVs misinterpret an act of innocent passage through State A’s territorial sea by two of State B’s warships as posing an imminent threat of attack on State A and fire missiles at the warships, sinking them.Footnote 5
The errors that led to this accident might occur for a number of reasons. It might be, for example, that the individuals who built the systems used flawed or insufficient quantities of training data; that those testing the systems performed insufficiently robust tests or misinterpreted the test results; that the systems were trained to be overly sensitive in identifying and responding to perceived threats; or that those who deployed the systems failed to obtain adequate information about what situations the systems were suited for or deployed them in an unsuitable environment. It might even be that the system developers are unable to establish why the systems made the mistakes they did.
2.2. Scenario 2: the poisoned AI system
In Scenario 2, State A builds and deploys a fleet of AUVs to serve as self-defense systems. However, State B poisons the data on which State A’s AUVs are trained, with the goal of causing State A’s AUVs to incorrectly perceive threats.Footnote 6 State B has knowledge that, as a result of the poisoning, the AUVs may launch missiles at non-threatening objects, although it does not necessarily intend that result. As a result of the poisoning, the AUV systems perceive a 1,000-passenger cruise ship flagged to neighboring State C to be a naval vessel from State C that is about to launch an attack on State A. State A’s AUVs launch missiles at both the cruise ship and a naval operations center in State C’s territory (Hiebert, Reference Hiebert2024).
One version of Scenario 2 is that State B poisons the data before State A completes its weapons review and testing. Another version is that State B hacks State A’s AUVs after State A completed its testing, reverse-engineers the model, and deliberately changes its functioning (Boulanin, Reference Boulanin2015; Crootof, Reference Crootof2016).
2.3. Scenario 3: the competitive AI system
Scenario 3 involves two different AI systems – State A’s and State B’s – that were built and trained in isolation from each other.Footnote 7 Assume that each state tested its systems in what the state believed to be reasonable operational conditions, but neither state anticipated that its AUV would encounter another unmanned system during its deployment. State A’s AUVs detect another state’s AUVs (which turn out to be owned by State B) just outside State A’s territorial sea. State A’s AUVs slowly move closer to State B’s AUVs to use active sonar to collect information about the latter’s electronic signatures and weapon systems. However, State B’s AUVs have been trained to conduct acoustic jamming against any adversarial active sonar they encounter.Footnote 8 In turn, State A’s AUVs have been trained to treat as hostile any acoustic jamming they encounter and to fire warning shots near the source of that jamming. State B built its AUVs to defend themselves against any object that shoots in their direction, and they thus fire missiles at each of State A’s AUVs, sinking them.Footnote 9
3. What bodies of law apply to the scenarios?
Each scenario in Part 2 invites us to consider which actor was legally responsible for bringing about the use of force and whether the use of force was unlawful, such that state responsibility attaches. It is helpful to break down the analysis into discrete steps. First, to determine whether there was an internationally wrongful act, we must consider what substantive international law regulated the state’s (or states’) behavior. Second, in assessing whether an act was wrongful, we must ask what standard of care the state needed to follow to comply with those international legal rules. Third, we must ask factually whether the state met that standard of care and, even if it did not, whether any circumstances existed that precluded the wrongfulness of the act. This part addresses the first two steps, while Part 4 addresses the compliance question.
3.1. State responsibility rules
When a state violates an obligation under international law, it engages in an internationally wrongful act. Under the International Law Commission’s (2001) Draft Articles on State Responsibility (DASR), generally thought to reflect customary international law, a state bears responsibility for that violation (DASR Article 2). These secondary rules, which envision obligations to (for example) provide reparations to the injured state or allow an injured state to take countermeasures, do not specify the content of the primary rules or their interpretation (DASR Article 2 General Commentary, p.31, 34). Thus, to determine whether a state’s purported act of self-defense violated the JAB and triggered state responsibility, we must look to the Charter and customary international law (Dinstein, Reference Dinstein2017).
That said, the DASR reflects that there are situations in which the wrongfulness of a state act may be precluded. In particular, Article 23 identifies “force majeure” as a circumstance precluding wrongfulness. It defines force majeure as “the occurrence of an irresistible force or of an unforeseen event, beyond the control of the State, making it materially impossible in the circumstances to perform the obligation” (DASR Article 23(1)). However, a state may not rely on force majeure if “the situation of force majeure is due, either alone or in combination with other factors, to the conduct of the State invoking it; or the State has assumed the risk of that situation occurring” (DASR Article 23(2)). And of course, the wrongfulness of an act “is precluded if the act constitutes a lawful measure of self-defense” (DASR Article 21).Footnote 10
Also relevant for Scenario 2, a state that “coerces another State to commit an act is internationally responsible for that act if: (a) the act would, but for the coercion, be an internationally wrongful act of the coerced State; and (b) the coercing State does so with knowledge of the circumstances of the act” (DASR Article 18). Where a victim state is coerced and a third state is injured as a result, the coercing state bears responsibility for the injury to the third state (DASR Article 19, Commentary note 1).
3.2. Substantive law
Each scenario culminates in the resort to military force by one state against another state, which implicates the JAB. The substantive JAB rules are contained in the UN Charter (Articles 2(4) and 51), as well as the customary rules of necessity, proportionality, and immediacy. Article 2(4) states, “All Members shall refrain in their international relations from the threat or use of force against the territorial integrity or political independence of any state, or in any other manner inconsistent with the Purposes of the United Nations.” The three relevant exceptions to this prohibition are self-defense in response to an armed attack (including, in the view of some states, anticipatory self-defense); authorization by the Security Council (which is not present in the scenarios); or consent by the territorial state (also not present). When a state resorts to force in self-defense, that use of force must be executed consistent with customary IHL targeting rules applicable in all conflicts – notably, distinction, proportionality, and precautions. Further, a state that is party to Additional Protocol I must determine in advance whether any weapon it uses would, in some or all circumstances, be prohibited under IHL consistent with Article 36 of that treaty.Footnote 11
3.3. Standards of care
Many rules of international law articulate primary rules but are silent on the standard of care a state must follow in attempting to comply with the primary rule. By “standard of care,” this article refers to the level of effort a state must exercise to try to comply with the primary rule in order to avoid committing an internationally wrongful act, even if that effort ultimately is unsuccessful. Like other scholarship on this question, this section draws from analogous concepts in tort and criminal law that describe an actor’s mental state, such as negligence, recklessness, and intentionality – as well as the concept of strict liability, which holds an actor responsible for their act regardless of their mental state when undertaking the act. This section reviews possible standards of care under the JAB and IHL.
3.3.1. Jus ad bellum
Under what conditions does a state’s factual mistake when resorting to force render that use of force unlawful? The answer is surprisingly understudied. Only a few scholars have assessed what standard of care attaches to a state’s act of perceived self-defense (Milanovic, Reference Milanovic2020a; Milanovic & Shah, Reference Milanovic and Shah2023; Schmitt, Reference Schmitt2017). Further, as discussed below, several other scholars have explored a related question that arises under IHL: what standard of care must a state follow during an armed conflict, particularly in relation to aerial strikes? However, none of these scholars addresses whether the same standards of care should attach to the use of AI systems for JAB or IHL decisions (Hathaway & Khan, Reference Hathaway and Khan2024). The use of AI in making resort-to-force decisions (unlike the use of human judgment) raises additional complications when the AI system is hacked by an adversary or confronts another AI system.
One possibility is that the UN Charter imposes a strict liability standard, such that a state commits an internationally wrongful act whenever it erroneously uses force in self-defense, even if the state follows a high standard of care. Another possibility is that there is a certain standard of care that, if met by a state, will render its act legally permissible even if it makes a mistake. Neither the primary rules of the JAB nor the DASR specifies what standard of care a state must undertake to meet its legal obligations when seeking to comply with the JAB.
It is possible to read the Charter to impose strict liability (or an obligation of result). One might read Article 51 (which accepts the right of self-defense “if an armed attack occurs”) to reflect that the right attaches only when there has objectively been an armed attack (Milanovic, Reference Milanovic2020b). Likewise, one might read the DASR’s treatment of self-defense as a circumstance precluding wrongfulness as being subject to the objective existence of an armed attack (DASR, Article 21, Commentary para. 1). But in the context of shooting down a civilian aircraft in 1988 in a mistaken exercise of self-defense, the United States suggested that a mistake of fact about the existence of an armed attack might not trigger legal responsibility, and a range of other states – including the United Kingdom, Italy, France, Argentina, Japan, Germany, and Brazil – seemed to concur in that setting (Milanovic, Reference Milanovic2020b).Footnote 12 Indeed, the scholar who has considered the question most extensively has tentatively concluded that “a limited, honest and objectively reasonable form of mistake of fact might be appropriate for on-the-spot defensive reactions in situations in which to the state concerned the ‘necessity of self-defence was instant, overwhelming, leaving no choice of means, and no moment of deliberation’” (Milanovic, Reference Milanovic2020b, quoting Daniel Webster’s letter to Mr. Fox regarding the 1837 case of The Caroline).Footnote 13 The Tallinn Manual 2.0 similarly invokes a reasonableness standard:
[T]he right to engage in self-defense is also subject to the existence of a reasonable determination that an armed attack is about to occur or has occurred… . These determinations are made ex ante, not ex post facto. Their reasonableness will be assessed based on the information available at the time they were made, not in light of information that subsequently becomes available (Tallinn Manual 2.0, Rule 71, para. 23).Footnote 14
Under this approach, a state that acted negligently or recklessly in mistakenly resorting to force would not meet the “objective reasonableness” standard and would incur international liability, whereas a state that acted reasonably from both a subjective and objective perspective would not.
3.3.2. IHL
The primary rules of IHL provide somewhat more guidance than the JAB about what standards of care a state must follow. As Lubin (Reference Lubin2022, p.134) writes, “[T]ribunals and military manuals guide us to rely on the ‘reasonable commander’ test in determining the lawfulness of a particular strike.” That is, we must ask whether a hypothetical reasonable commander would have made the same decision that a particular real-world commander did, given the information available to her. (This inquiry must try to avoid the hindsight bias that accompanies knowledge about the actual results of the strike.) A commander might make a mistake, for example, about the identity of a target or about the proportionality assessment, even if his state undertook feasible precautions in advance of an attack, as is required under customary IHL. Milanovic (Reference Milanovic2020a) suggests that an “honest and reasonable” mistake of fact made during armed conflict would not produce an international law violation. Schmitt and Schauss (Reference Schmitt and Schauss2019, p.163) indicate that “the greater the value of a target, the more uncertainty may be countenanced in an attack, and vice versa.” Copeland et al. (Reference Copeland, Liivoja and Sanders2023) argue that the standard for targeting decisions is due diligence and that “this standard represents the minimum that would be required of a machine undertaking this function on behalf of a human.” Most restrictively, Ronen (Reference Ronen2009) and Crootof (Reference Crootof2016) have proposed a strict liability regime that would cover serious violations of IHL that give rise to state responsibility. To date, however, states have not adopted a strict liability approach.
With regard to weapons reviews, IHL is silent about what level of confidence a state must have that a new weapon will not be prohibited by IHL. As a SIPRI report noted, “A [t]est may indicate that a weapon with an automated target recognition function recognizes a lawful target 30, 80 or 100 percent of the time. Determining the data-match acceptance criterion is uncontestably a sensitive issue … [that] can only be made on a case-by-case basis” (Boulanin, Reference Boulanin2015, p.16). A former advisor to the International Committee of the Red Cross argues for a rigorous standard of care for AI weapons reviews, noting, “Since the commander or operator must make an assessment of the lawfulness of an attack using an autonomous weapon system at an earlier stage than if the selection and attack of targets were under direct human control, the legal review must demand a very high level of confidence that, once activated, the autonomous weapon system would predictably and reliably operate as intended” (Davison, Reference Davison2018, p.10). However, there is little information available about precisely what confidence level states demand in their weapons reviews. It seems likely that states would conclude that the standard of care again is reasonableness – that a state must make a good faith and objectively reasonable judgment about the ability of the weapon to comply with IHL, based on the state’s testing and evaluation.
3.4. Analysis
Although there is a paucity of law and limited state practice here, there seems to be a rough consensus that the appropriate standard of care regarding IHL targeting is good faith and objective reasonableness. For the JAB, Milanovic (Reference Milanovic2020b) suggests that the standard might be an “honest and objectively reasonable” judgment that force would meet international legal requirements (including the existence of an armed attack or imminent threat thereof and the necessity of using force).
What does an objectively reasonable judgment look like when a state is deploying an autonomous AI-driven system that has the capacity to resort to force? First, a reasonable state would take all feasible measures to avoid data-driven or technological mistakes as it builds the system. It would make sure that it has trained the system on sufficient (and sufficiently high-quality) data, and it would subject the system to robust developmental and operational testing, keeping in mind that other states also are likely to be developing autonomous systems that the first state’s systems might encounter in the wild. A state would also need to ensure that the system used force only when necessary and that any force it used was proportional to the imminent or actual armed attack. Second, a reasonable state would subject its systems to thorough weapons reviews to ensure that the fact of the AUVs’ use, as well as the AUVs’ specific operations, complies with IHL.
Third, and perhaps more controversially, because the use of AI systems in JAB operations is both nascent and risky, an objectively reasonable state would undertake a series of additional checks before deployment, such as requiring very high confidence levels for system performance, mandating white hat efforts to hack the system, and pitting the AI system against an (internal or allied) adversarial system before deeming it fit for deployment.
It is possible that states, if surveyed, would agree that these additional checks are actually necessary to meet a “good faith and objective reasonableness” standard. But if states perceive this level of thoroughness as exceeding how an objectively reasonable state would act, there is a good normative argument for raising the standard of care to require this level of thoroughness when states use AI systems for resort-to-force decisions. Given the grave costs that follow from errors in resorting to force and the serious risks today of deploying AI and autonomous systems in resort-to-force decisions, states should consider adopting a more rigorous, “abundance of caution” standard that would entail the types of additional checks described earlier. In other areas of international law, states sometimes adopt an “abundance of caution” (or “precautionary”) approach in enacting regulations to reduce potential hazards or risks, even when scientific knowledge about the risks is uncertain (Druzin et al., Reference Druzin, Boute and Ramsden2025). In this case, using an “abundance of caution” standard and applying the additional checks that such a standard entails would be a normatively desirable approach, even though doing so might exceed what reasonable states would demand of themselves today.
4. What state responsibility attaches in the scenarios?
Parts 2 and 3 show that there are several ways in which the use of AI in resort-to-force systems may lead to unpredictable outcomes and that the law is not well settled on how to assess state responsibility in this setting. These parts apply the legal standard that states seem most likely to accept to the facts of the scenarios: a requirement to act in good faith and in an objectively reasonable way. It also briefly judges each scenario under the proposed “abundance of caution” standard that would embrace the additional checks described in Section 3.4. Ultimately, because international law lacks a standardized approach to burdens and standards of proof (Tomka & Proulx, Reference Tomka and Proulx2015), determining the existence or absence of an internationally wrongful act is a complicated, fact-intensive exercise.
4.1. Scenario 1
In Scenario 1, State A’s AUVs malfunction and misinterpret an act of innocent passage as an imminent attack, sinking two of State B’s warships in a mistaken act of self-defense.
4.1.1. Jus ad bellum
If the test for a violation of the JAB is strict liability, State A clearly committed an internationally wrongful act against State B. However, if State A took objectively reasonable precautions to ensure that its AUVs would use only necessary and proportional levels of force in the face of truly imminent threats, and if the AUVs’ error is equivalent to an “honest and objectively reasonable” mistake of fact about the actions that State B’s vessels were undertaking, then State A may not have committed an internationally wrongful act under the JAB (Milanovic, Reference Milanovic2020b). The same result presumably would attach if State A had used an AI-driven decision support system to initiate force. The burden should be on State A to demonstrate after the fact that its error constituted an “honest and objectively reasonable mistake.” Further, although this is subject to debate, State A may be financially responsible for the damage it produced even if it can show the objective reasonableness of its error (Ronen, Reference Ronen2009, citing DASR Article 27(b)).Footnote 15 Finally, in the face of the AUVs’ error, a reasonable state would deactivate the AUVs and subject them to thorough re-testing. If an objectively reasonable state would have followed the additional checks described in Section 3.4, or if states coalesced around a higher “abundance of caution” standard, we would need more information about how carefully State A trained and tested its AUVs, but it seems unlikely that State A would meet that standard. Robustly testing and red-teaming the system and requiring a very high confidence level for system performance should have prevented the AUVs’ error.
4.1.2. IHL
State A targeted naval vessels of State B, an act that would not (under wartime conditions) violate distinction, proportionality, or precautions. However, unlike with a purely human decision to authorize force in self-defense using a standard missile system, the assessment of the lawfulness of the AUVs’ actions under the JAB bleeds into an inquiry about the sufficiency of State A’s weapons review of the AUVs (Milanovic, Reference Milanovic2020a). If it were objectively reasonable for State A to believe that the weapons systems would only target military objectives in self-defense and the systems failed, there would be no violation of IHL and no state responsibility for an internationally wrongful act. As with the JAB analysis, the burden lies with State A to demonstrate that its training and deployment of the system were objectively reasonable.
4.2. Scenario 2
In Scenario 2, State B poisons the data on which State A trains its AUV systems or hacks State A’s systems after they are trained, which leads the systems erroneously to use force against State C’s cruise ship and a military target. It is well understood that AI systems are brittle. State B takes advantage of this weakness, recognizing there is a likelihood that the poisoning or hacking will lead the AUVs to commit errors, such as misapprehending the nature of a cruise ship.
4.2.1. Jus ad bellum
State A: Did State A comply with the JAB standard of care? As with Scenario 1, we would assess the objective reasonableness of State A’s testing of the AUVs, but also the objective reasonableness of State A’s security/anti-hacking measures for those systems and the lack of a self-neutralization mechanism in the event of a hack or other disruption. We would also want to know at what stage State B poisoned the UAVs’ data. If State B poisoned the data before State A conducted its weapons review, State A probably incurs state responsibility for the use of force, assuming an objectively reasonable state (1) would have established security features to prevent such poisoning from happening and (2) would have conducted tests that would have revealed the results of the poisoned data.
If State B poisoned or hacked the system after State A tested, and State A had established reasonable security/anti-hacking features on the system, State A might assert that it acted consistent with the JAB standard of care or claim that State B’s poisoning constituted force majeure. Some might conclude that, because State A took objectively reasonable precautions, it did not contribute to the error through its own conduct and thus may invoke force majeure. Others might reject the application of force majeure under the theory that State A, in using autonomous systems, should have foreseen the risk of such errors and poisoning occurring. In addition, State A might also argue that its systems were subject to coercion by State B. It is not clear whether a state may be subject to coercion without its knowledge, but forcing an AI system, as an extension of a state’s military, to initiate an act against its programming does not fit neatly into a “coercion” paradigm. Instead, State B seems to have coopted or hijacked State A’s system, effectively taking it out of State A’s control.
If the JAB standard of care were an “abundance of caution” standard or if states accepted the need for the additional checks as part of an “objective reasonableness” test, State A’s actions likely would constitute an internationally wrongful act unless it had employed a very high level of cybersecurity and a reasonable state would not have foreseen that another state had the requisite skill to interfere with systems such as the AUVs.
Even if State A did not commit an internationally wrongful act, State C (having suffered a kinetic strike on its naval facility and a cruise ship flagged to it) may be tempted to respond in self-defense. If State C were cautious, it would first examine whether a use of force in self-defense was necessary – and if it believes State A’s report, it should deem a forcible response unnecessary. However, State A would need to immediately contact State C after the strikes and possibly provide State C with access to forensic information about the AUVs’ operations and poisoning to demonstrate objective reasonableness (or force majeure or coercion) (Lubin, Reference Lubin2022).
State B: We first should ask whether State B directly engaged in a use of force or armed attack against another state. Few would characterize State B’s act of poisoning or hacking as a violation of Article 2(4) as such. However, if State B could foresee that the poisoning might cause the AUVs to launch missiles (at State A itself, at another state, or at random private vessels), the argument that State B caused a 2(4) violation to occur against State C is stronger. Absent such foreseeability, State B’s act seems to constitute a violation of the customary international rule of non-intervention against State A, which falls below the use of force threshold.
4.2.2. IHL
State A: As before, we would assess the objective reasonableness of State A’s development and testing of the AUVs, including under a weapons review. If it were objectively reasonable for State A to believe (and State A subjectively believed) that its AUVs would act consistent with IHL (including by not targeting civilian objects) and that those systems were secure from foreign interference, State A’s launch of missiles would not be internationally wrongful (or, if wrongful, might be excused under force majeure or coercion). If such a belief were objectively unreasonable, State A would have failed to comply with the IHL standard of care and would bear responsibility for an internationally wrongful act.
State B: It does not necessarily violate international law to sabotage an adversary’s weapons systems, but sabotage must comply with IHL. To assess whether State B violated IHL, we would ask whether State B took objectively reasonable steps to ensure that the poisoned system would comply with distinction, proportionality, and precautions – steps not in evidence here. State B thus arguably committed an internationally wrongful act against State C by enabling an indiscriminate use of force that killed civilians.
4.3. Scenario 3
In Scenario 3, the AUVs of State A and B enter an unplanned escalatory cycle, which culminates in State B’s AUVs sinking State A’s AUVs.
4.3.1. Jus ad bellum
For both states, we must ask first whether the system – at any stage of its operations – would engage in a patent violation of the JAB if it operated consistent with its training. If neither system was programmed or trained to undertake acts that would be per se violations of the JAB, we must ask whether each state acted objectively reasonably in selecting the algorithm and training that it did.
State A: Building an autonomous weapon system that responds inside a state’s territorial sea with warning shots when facing acoustic jamming would not inherently violate the JAB.Footnote 16 The U.S. view, contained in the U.S. 2022 Commander’s Handbook on the Law of Naval Operations, is that “[u]nder international law, warning shots do not constitute a use of force” (U.S. Navy, 2022, sec. 3.11.5.1). Under the current U.S. Department of Homeland Security use of force policy, maritime and aviation law enforcement personnel may use warning shots to signal a vessel to stop (or leave U.S. airspace) after other means of signaling have failed (U.S. Department of Homeland Security, 2023, sec. V.B.). Assuming that other states agree with the U.S. characterization of warning shots, State A’s AUVs do not seem inherently designed to violate the JAB.
If warning shots are not inherently unlawful, we still should ask whether State A acted objectively reasonably in instructing its AUVs to fire warning shots in response to acoustic jamming. Here, State A might not have acted reasonably in treating jamming as a trigger (because jamming is not particularly threatening) or in failing to require the AUVs to issue other types of warnings before proceeding to warning shots. Further, if it is clear that various states have deployed AUVs and it is reasonable to assume that those AUVs may not accurately interpret the signal that a warning shot is intended to convey, then the use of warning shots by any AUVs may cease to be objectively reasonable.
State B: Allowing soldiers to use force in self-defense in response to a hostile act or demonstrated hostile intent would not violate the JAB. Less clear is whether building an AUV that fires defensively at anything that shoots at (or near) it inherently violates the JAB, given that no human life is at risk (Deeks & Anderson, Reference Deeks and Anderson2019). In Scenario 3, State B’s AUVs were in international waters, not in State A’s territorial sea. It may be objectively reasonable to perceive underwater warning shots in international waters as a hostile act that triggers a right of self-defense. In sum, it is hard to judge whether State B’s AUVs acted reasonably in self-defense or engaged in an unwarranted use of force themselves.
In light of the ambiguity presented in this AI-driven escalation, one possibility is that both States A and B faced a situation of force majeure, which would excuse any international law violations that might have occurred here. However, in light of the growing use of AUVs by states today, the better argument is that both states acted objectively unreasonably in failing to foresee the likelihood of such unintended escalation. A situation that in 2015 may have been force majeure because it was so rare is now eminently foreseeable. The adoption of the requirement to apply additional checks as part of an “objective reasonableness” test or the use of the higher “abundance of caution” standard makes this conclusion even easier to reach. Had the states followed that approach or standard, they would have tested their systems extensively against adversarial models and would have increased the threshold at which their AUVs would respond to purported threats from other AUV systems.
4.3.2. IHL
Neither State A nor State B violated IHL in Scenario 3. State A’s warning shots did not hit anything; had they hit State B’s AUVs, those are military objectives; and State B’s missiles sank military targets. Assuming each state’s weapons reviews were conducted in an objectively reasonable way, neither state committed an internationally wrongful act under IHL.
5. Conclusion
As far as we can discern the outlines of the required standard of care necessary to comply with the JAB – and there is limited practice to guide us – a state must act in good faith and in an objectively reasonable way in attempting to assess the facts. If a state uses an AI or autonomous system to directly inform or execute acts of self-defense, the state should expect that its use of that system will be held at least to this standard. In light of the high stakes of JAB decisions, this article argues that the “objective reasonableness” standard should require the application of an additional set of checks before deploying AI systems in this setting or, in the alternative, proposes a higher, “abundance of caution” standard of care that contains those checks. It adopts this approach because of the potential ease of poisoning or hacking those systems and the unpredictable interactions that may occur between competitive AI systems. Under this higher standard or more robust set of checks, the mistakes set out in the three scenarios very likely would have been avoided. At bottom, this area of international law is insufficiently fleshed out and worthy of further exploration by states.
Acknowledgements
Thanks to Jenny Davis, Toni Erskine, Asaf Lubin, Marko Milanovic, Michael Schmitt, and participants in the 2nd Workshop on Anticipating the Future of War: AI, Automated Systems, and Resort-to-Force Decision Making, Australian National University (23–25 July 2024), for helpful comments.
Funding statement
I did not receive funding to write or publish this article.
Competing interests
The author declares none.
Ashley Deeks is the Vice Dean and Class of 1948 Scholarly Research Professor at the University of Virginia Law School. She is an elected member of the American Law Institute, a member of the State Department’s Advisory Committee on International Law, and a contributing editor to the Lawfare blog. In 2021–22, she served as Special Assistant to the President, Associate White House Counsel, and Deputy Legal Advisor to the National Security Council. Before joining UVA, she worked for ten years in the Office of the Legal Adviser at the U.S. Department of State.
Open access
