Introduction
Artificial intelligence (AI) systems are being integrated into military functions across the air, land, maritime, electromagnetic, cyber and space domains. They now support filtering sensor data, accelerating intelligence analysis, informing logistics and generating targeting assessments that shape how commanders perceive operational environments and decide on the use of force. This article addresses what States, armed forces and technology developers must do before systems are deployed to establish conditions under which their use can comply with international humanitarian law (IHL).
IHL requires that parties to armed conflict respect the principles of distinction, proportionality and precaution.Footnote 1 Commanders remain responsible for ensuring that all uses of force comply with these obligations regardless of the technological systems that inform them.Footnote 2 As AI systemsFootnote 3 become more integrated into defence operational workflows, ensuring that force is used lawfully and is informed by appropriate human judgement becomes more complex. The challenge is whether conditions that inform human judgement have been sufficiently established in advance.
This challenge is most visible in the growing practice of human–machine teaming (HMT), in which humans and AI systems jointly contribute to shared tasks:Footnote 4 AI systems can process sensor data at speeds and volumes beyond human capacity, while human operators supply contextual judgement, legal reasoning, and accountability for decisions. In military operations, HMT structures are now shaping how commanders perceive the operational environment and decide on the use of force. AI systems can identify patterns across multiple intelligence streams simultaneously, reduce human cognitive bias in target assessment and support identification of military and civilian objectives.Footnote 5 They also introduce new risks, including automation bias, information overload, adversarial manipulation, degraded performance in complex environments and opacity in how outputs are generated.Footnote 6 Urban environments heighten these risks, as military objectives are often intermingled with civilians and civilian objects. Coalition operations are also a challenge, as differences in doctrine, training and interoperability complicate shared assessments.Footnote 7
The Geneva Conventions of 1949 and their Additional Protocols of 1977 and 2005 impose positive duties on States to respect and ensure respect for IHL before and during armed conflict.Footnote 8 Preparatory measures in this regard include dissemination of IHL, integration of legal obligations into military doctrine and training, review of new weapons and establishment of accountability mechanisms.Footnote 9 The increasing integration of AI into military operations heightens the importance of these duties. Ensuring that human–machine teams can operate in a manner consistent with IHL cannot be deferred until hostilities arise; rather, AI system design, testing, governance and training must be structured in advance to enable commanders and operators to exercise appropriate human judgement consistent with applicable legal obligations. For the purposes of this article, “appropriate human judgement” means that commanders and operators have access to sufficient, reliable and actionable information; understand and can apply the applicable IHL rules independently of system outputs; can think critically rather than deferring to system recommendations; and operate within institutional frameworks that require and enable informed rather than procedural authorization.
This article proposes a cross-disciplinary framework for ensuring that AI-enabled human–machine teams can operate in compliance with IHL. It does so by mapping IHL obligations onto three established AI governance frameworks – the National Institute of Standards and Technology AI Risk Management Framework, the Institute of Electrical and Electronics Engineers Standards Association Research Group’s Autonomous and Intelligent Systems life-cycle framework, and the Alan Turing Institute’s Defence AI Assurance methodology – to form a traceable chain of legal and technical responsibility. Building on this mapping, the article proposes an HMT Assurance Card that integrates IHL requirements, civilian harm pattern analysis, contested-environment testing, operational training, and audit documentation into a unified standard. The Assurance Card shifts the inquiry from human presence in the decision chain to whether that human is technically and institutionally enabled to exercise genuine judgement, helping users to translate IHL obligations into measurable standards and audit documentation applicable across the AI system life cycle, defence institutions and coalition environments.
Legal and policy frameworks applicable to AI-enabled military systems
IHL and governance of AI in armed conflict
Protection of civilians during armed conflict is both a legal obligation and an ethical imperative. IHL applies to the conduct of hostilities in both international and non-international armed conflict and is technology-neutral, as it governs means and methods of warfare regardless of the platform used or system attributes. Whether force is applied through conventional weapons or AI-enabled systems, the key principles of distinction, proportionality and precaution are applicable.Footnote 10 The increasing integration of AI into military operations creates challenges to ensuring that those deploying these systems and acting on AI outputs can exercise sufficient human judgement.
There remains broad consensus among States that existing international law continues to govern AI-enabled capabilities in the military domain. In December 2024, the UN General Assembly adopted Resolution 79/239, affirming that IHL applies “throughout all stages of the life cycle of artificial intelligence in the military domain” and calling for appropriate safeguards to keep human judgement and control at the heart of military decision-making.Footnote 11 The 2023 Political Declaration on Responsible Military Use of Artificial Intelligence and Autonomy (endorsed by over sixty States) echoes the primacy of IHL governing AI-enabled weapons and systems.Footnote 12 The multi-stakeholder Summit on Responsible Artificial Intelligence in the Military Domain (REAIM) has produced outcome documents since 2023, endorsed by attending States, recognizing IHL’s applicability to military AI and multilateral coordination.Footnote 13
Critically, States have peacetime obligations in the form of preparatory duties to ensure that when hostilities arise, armed forces are institutionally and operationally equipped to comply with the law. Article 1 common to the four Geneva Conventions (common Article 1) requires States Parties “to respect and to ensure respect for the present Convention[s] in all circumstances”;Footnote 14 this has been understood to encompass wide dissemination of IHL, incorporation of IHL into military doctrine and practice, assignment of legal advisers to armed forces, establishment of accountability mechanisms, and review of new weapons, means and methods of warfare.Footnote 15 These obligations form the institutional framework for lawful military operations, and as AI systems become integral to military operational decision-making, such State preparatory duties extend to the design, testing, procurement and governance of AI-enabled capabilities.
IHL rules
In targeting, IHL reflects an attempt to balance military necessity with the principle of humanity. Under the principle of precaution, the law requires that once an attack on a verified military target is judged permissible, those responsible for planning and deciding upon that attack must apply precautionary measures to avoid or minimize incidental civilian harm.Footnote 16
The principle of distinction requires that parties to a conflict distinguish between combatants and civilians as well as between military objectives and civilian objects.Footnote 17 Civilians and civilian objects must not be the object of attack, although civilians may lose immunity from attack if and for such time as they directly participate in hostilities.Footnote 18 Additional Protocol I (AP I) defines military objectives as objects which by their nature, location, purpose or use “make an effective contribution to military action and whose total or partial destruction, capture or neutralization” offers a “definite military advantage”.Footnote 19
AI systems, such as computer vision models, are increasingly used to support targeting activities (as seen in Ukraine, Gaza, Iraq, Iran and Syria), extracting information from digital images, videos and other visual inputs.Footnote 20 These systems can detect objects by locating them within an image or video frame and classify them according to trained categories.Footnote 21 Reliability of the outputs for their intended use and IHL compliance must be demonstrated through rigorous testing and evaluation and data validation across varied operational environments, and against adversarial interference.Footnote 22 A system that works well in controlled conditions but degrades in urban terrain, adverse weather or contested electromagnetic environments has not been validated for its intended use, and a commander relying on the outputs in those conditions may not have access to sufficient and reliable information.
Using AI classification models to assess whether a person is directly participating in hostilities also raises reliability concerns. Unlike object detection or sensor fusion tasks, the determination of direct participation in hostilities requires contextual and behavioural judgement in a specific environment. This determination depends on factors that classification models are poorly equipped to make reliably;Footnote 23 for instance, datasets that reflect the physical traits or behavioural signatures of combatants in one operational context may not transfer to another and may embed biases that result in misidentifying civilians not directly participating in hostilities as threats.Footnote 24
The rule on proportionality prohibits attacks that may cause incidental loss of civilian life, injury to civilians or damage to civilian objects that would be excessive in relation to the concrete and direct military advantage anticipated.Footnote 25 This rule “is not a balancing test with the scales resting at equilibrium”, but requires the attacker to take a systematic or analytical approach that evaluates both the expected harm to civilians and civilian objects and the anticipated military advantage in order to ensure that the former is not excessive in relation to the latter.Footnote 26 Where AI systems generate target assessments or risk estimates that inform this analysis, the lack of explainability of the outputs may mean that the operator and/or commander may not have the necessary information to conduct the proportionality assessment.Footnote 27
AP I also sets out rules on feasible precautionary measures to be taken to avoid or minimize civilian harm. These include verifying that targets are lawful military objectives, selecting means and methods of attack that minimize incidental harm, and cancelling or suspending non-compliant attacks.Footnote 28 Furthermore, in the conduct of “military operations”, IHL requires that “constant care[Footnote 29] shall be taken to spare the civilian population, civilians and civilian objects”.Footnote 30 States and commanders must factor civilian protection measures into operational planning and mission execution not only during armed conflict but also in preparation for conflict.Footnote 31
In the AI context, the constant care principle is applicable to intelligence-gathering, data collection, and system design and manufacture.Footnote 32 It may also apply to private contractors or civilian intelligence entities involved in developing or deploying AI-enabled systems for combat missions.Footnote 33 Those who plan and initiate AI-enhanced operations must ensure that systems will function reliably in the circumstances of intended use, taking into account how systems have performed in testing and in recent operations, particularly in degraded, contested and urban environments where risks to civilians are the greatest.
Article 36 of AP I obliges States Parties to review new weapons, means or methods of warfare, reflecting a recognition that legal assessment of weapons must occur before deployment and not after civilian harm has materialized.Footnote 34 The integration of AI into weapons systems complicates the review function, as traditional weapons reviews were not designed to assess systems whose algorithmic output may adapt to the operating environment and during use. McFarland and Assaad argue that review of such AI-enabled weapons must extend beyond conventional weapons pre-deployment review to address the degree of human input into targeting decisions, and should expand to post-deployment review for IHL compliance.Footnote 35
Finally, IHL places a duty on States and commanders to ensure compliance.Footnote 36 Military commanders remain responsible for verification and validation of targets relying on AI tools, must ensure that anticipated civilian harm is weighed against the expected military advantage, and must ensure overall constant care to limit effects on civilians. States must also ensure that responsibility for lawful use of AI systems is distributed across the full chain of institutional preparationFootnote 37 – this means that engineers and manufacturers must design systems that perform as intended; procurement authorities must conduct thorough testing in realistic operational conditions; legal reviewers must identify lawful and unlawful uses; and operators must be trained appropriately.Footnote 38 Failures at any point in this chain may affect the reliability of operational judgement when deploying AI-enabled capabilities.
Civilian harm mitigation: From policy to operational requirement
Treaty obligations establish the legal floor for civilian protection during armed conflict. In practice, States are increasingly recognizing, based on examination of data from operations, that more needs to be done to improve operational approaches in order to reduce civilian harm, and they have enacted civilian protection policies and practices to address this concern. Civilian harm mitigation (CHM)Footnote 39 has emerged as an operations capabilities framework that when implemented and resourced can enable armed actors to better anticipate risks to civilians and civilian objects, take steps to mitigate those risks, and respond to incidents of civilian harm.
This risk management framework has been developed from operational experience. Militaries view CHM as a strategic, legal and ethical imperative that reinforces the legitimacy of military operations and contributes to better operational outcomes.Footnote 40 The North Atlantic Treaty Organization (NATO) enacted its Protection of Civilians Policy in 2016 following examination of its operations in Afghanistan and Libya, defining civilian protection as exercising approaches to avoid or minimize harm in compliance with the principles of IHL, including measures to prevent, identify, investigate and track incidents of civilian casualties and provide amends and post-harm assistance when civilians are harmed.Footnote 41 The United States,Footnote 42 Nigeria, the Netherlands and Iraq have also enacted CHM-related policies.Footnote 43
CHM refers to proactive measures, policies and procedures taken by armed forces to avoid, minimize and address harm to civilians during military operations.Footnote 44 It integrates civilian harm considerations into operational planning and mission execution and ensures that civilian harm data is tracked to inform operational adjustments and institutional learning. Critically, this approach enables intelligence preparation of the operating environment to include sufficient elements of the civilian environment, such as civilian presence and the resources, services, structures, infrastructure and systems which sustain life, so as to better estimate civilian harm risk, including by modelling indirect harm and potential civilian reactions to hostilities.Footnote 45 This analysis should be integrated into the common operating picture prior to hostilities and supplemented with real-time information in order to incorporate changes throughout operations.
Emerging research is showing how AI systems can also be deployed to support CHM efforts, including through decision aids embedded in weapons systems and situational awareness platforms that can help identify risks to civilians in target areas and alert operators to possible misidentification of civilians as military objectives.Footnote 46 Unmanned sensors and robotic platforms can also collect information about civilian presence in urban environments, including inside compounds and tunnels where human observation is impractical.Footnote 47 Data fusion capabilities can support the development of a comprehensive understanding of the civilian environment, including the locations of protected objects such as hospitals, schools and civilian infrastructure. Modelling and simulation tools can aid in planning, estimate civilian harm from both direct and indirect effects and assist in post-strike assessments.Footnote 48
As AI becomes embedded in operational workflows, the central issue is whether deployers of such tools retain the capacity for appropriate human judgement consistent with IHL and informed by effective CHM practices. This capacity depends on the institutional, technical and dynamic conditions under which decisions are made. Where AI systems inform or shape use of force actions, preparedness requires incorporating IHL and CHM into system design, testing, documentation and training before deployment. Whether those considerations are effectively integrated into operational decisions depends on how the human–machine team is structured, technically, institutionally and operationally. That structure is examined in the following section.
Human–machine teaming: Operational realities
HMT is emerging as a dominant operational model for integrating AI-enabled capabilities into military practice. HMT is defined as “the partnership of humans and machines collaborating to achieve a shared objective through interaction and communication” in which each contributes distinct strengths to shared tasks.Footnote 49 In military contexts, AI-enabled systems can provide speed, sensing and data processing that exceed human capacity, while humans supply contextual nuance, ethical and legal judgement, policy and strategy alignment, and responsibility for operational decisions.Footnote 50 This division is critical as under IHL, commanders and operators bear responsibility for ensuring that attacks comply with rules on distinction, proportionality and precaution. Crucially, HMT depends on predictability, transparency, trust in systems, cognitive workload management, robust communication in degraded environments, and the ability for humans to intervene or modify system behaviour.Footnote 51
HMT in practice
HMT is already shaping military operations across multiple domains. In Ukraine, widespread use of first-person-view drones is reported as illustrating how human visual navigation combined with machine speed can impact manoeuvrability and target homing in ways that preserve human control over engagement decisions.Footnote 52 In other operational contexts, AI agents and co-pilots are being tested for navigation and threat detection;Footnote 53 autonomous surface vessels patrol maritime routes under human supervision;Footnote 54 unmanned ground vehicles support explosive-ordnance disposal and casualty evacuation;Footnote 55 and AI-enabled intelligence, surveillance and reconnaissance platforms assist operators in filtering and interpreting vast quantities of sensor data.Footnote 56
These developments do not alter the allocation of legal responsibility under international law – commanders and operators remain responsible for ensuring compliance with the rules on distinction, proportionality and precaution regardless of the degree to which AI systems contribute to the decision support.Footnote 57 However, when AI systems recommend courses of action, they shape the informational basis upon which operational decisions are formed, and use of force decisions depend not solely on final human authorization, but also on whether the commander/operator had access to sufficient, reliable and actionable information to exercise appropriate judgement.Footnote 58 This means that human–machine teams, when using AI systems, must address how system outputs can be interrogated, how transitions between automated and human-controlled functions are structured, and how operators are trained to recognize the limits of the systems they work with. These conditions should be addressed prior to military operations.
AI failure modes and IHL obligations
AI-enabled systems can enhance situational awareness, detect patterns that humans might overlook and reduce certain cognitive biases.Footnote 59 Improved pattern recognition and data fusion may assist in identifying military objectives or supporting precautionary measures. However, risks arise from automation bias, over-reliance on system outputs, degraded performance in contested environments, adversarial manipulation and opacity of model behaviour.Footnote 60 Each of these failure modes has direct implications for specific IHL obligations.
Automation bias, where the operator defers to machine outputs without adequate critical scrutiny, may result in the rigour needed to ensure distinction and proportionality requirements during armed conflict not being met. Opacity in AI system output generation compounds this risk. Where an AI system provides a confidence score,Footnote 61 such as classifying a target at high confidence, without communicating the uncertainties caused by sensor noise, environmental interference or adversarial manipulation,Footnote 62 the commander may lack the information necessary to assess whether the output is reliable enough to act upon. Where the basis for AI system outputs cannot be interrogated, decisions made as a result of those outputs do not constitute informed human judgement.Footnote 63
Transparency and explainability are therefore essential conditions for informed human judgement. For example, US Department of Defense (DoD) Directive 3000.09 on “Autonomy in Weapon Systems” requires that the most consequential AI systems be designed with “technologies and data sources that are transparent to, auditable by, and explainable by relevant personnel”.Footnote 64 A confidence score alone does not satisfy this requirement: it is not sufficient for an AI system to indicate high confidence that a lawful military target is present without also indicating on what grounds that conclusion was reached – whether through sensor data, pattern-of-life analysis or intelligence – so that the commander can assess whether the basis is reliable in the applicable operational environment.Footnote 65
Degraded performance in complex environments poses challenges in ensuring compliance with precautionary obligations. Computer-vision systems that function well in clear, static or sparsely populated settings often degrade sharply in urban environments characterized by clutter, partial occlusion by terrain and infrastructure, and the presence of civilians and their movements.Footnote 66 For example, if a civilian is carrying an object that looks like a weapon from a specific angle, or if a target is partially obscured by a structure, a model may provide a high-confidence incorrect output.Footnote 67
Decision support tools trained on structured data sets may struggle with irregular operations or deceptive tactics. Adversarial actors may use camouflage, decoys or deliberate manipulation of training data to exploit specific patterns that AI systems have learned to recognize, producing high false negative rates in target identification.Footnote 68 Thus, miscalculations could occur due to insufficient or poor-quality data, data poisoning, human bias transferred to the AI system during the development stage, and/or cyber attacks,Footnote 69 with operational consequences.
Humans are not immune to the same failures, though adversarial deception confounds human operators and AI targeting systems differently. AI may detect patterns across large volumes of data that a human operator would miss, while an operator brings contextual judgement about adversary behaviour and/or civilian environment and pattern of life of civilians during armed conflict that a system not trained or updated with real-time information cannot replicate. For example, both Russia and Ukraine are reported to have deployed inflatable armour, false artillery installations and painted decoys specifically designed to exploit the limits of automated target recognition and human visual assessment simultaneously.Footnote 70
These risks are heightened in densely populated urban settings, where military objectives are frequently intermingled with civilians and civilian objects, and in coalition operations, where divergent rules of engagement, differing data standards and interoperability constraints complicate shared situational awareness.Footnote 71
The relevant question, then, is not whether humans or machines are more reliable under adversarial conditions but whether the HMT unit is structured so that each compensates for the other’s failure mode. CHM best practices, specifically broad analysis of the civilian environment rather than a narrow focus on target classification outputs, can also provide that structural check. A commander who understands the civilian environment independently of system outputs is better positioned to interrogate a high-confidence classification that does not cohere with the operational picture than one who relies on the system output alone.
Notably, in coalition operations, use of AI-enabled systems for targeting can introduce failure modes arising from differing legal and institutional frameworks across partners. When an AI-enabled system developed by one State generates recommendations that are acted upon by operators from another, questions on aligning appropriate human judgment, rules of engagement and evidentiary thresholds for target validation may differ. This also creates ambiguity across the human–machine team with regard to the attribution of responsibility – does responsibility remain with the system developer, the recommending authority, the operator executing the strike, or the commander authorizing it? These challenges are compounded by the opacity of AI-enabled decision support, which may obscure the basis on which recommendations are relied upon in fast-paced environments. Coalition AI use exposes a gap in existing AI governance frameworks, which are designed for national systems and do not account for distributed decision-making across allied forces.
Patterns of civilian harm and implications for AI system design
Precautionary obligations under IHL require that forces take feasible measures to minimize civilian harm in military operations.Footnote 72 Deploying AI systems for targeting would therefore require not only a general accounting for AI failure modes but also an understanding of how civilian harm occurs in armed conflict and integration of those civilian harm patterns into dataset design, testing, and operational training in order to enable better planning and risk mitigation approaches for operators and commanders.
Beyond intentional or reckless conduct, civilian harm during armed conflict may also arise from misidentification and incidental harm.Footnote 73 Misidentification takes two forms: misassociation occurs where information about a valid military target is incorrectly ascribed to a civilian person or object, while misperception occurs where a civilian is mistaken for a threat based on appearance, behaviour or other information.Footnote 74 An examination of US military operations, undertaken by DoD commissioned studies, showed that misidentification accounted for half of US-caused civilian casualties arising from errors in intelligence about specific individuals and misinterpreting behaviour of civilians under time pressure, ambiguous visual data and/or incomplete sensor data.Footnote 75 These are the types of conditions under which AI classification systems are most likely to produce high-confidence incorrect outputs. Thus, AI systems trained on datasets that inadequately represent the diversity of civilian population, clothing or behaviour in the operational environment risk reproducing the misidentification patterns that drive engagement decisions and cause civilian harm.
Civilian harm through incidental harm occurs where civilian presence is not observed by military forces or sensors – meaning, for example, that civilian movements or the presence of civilians in buildings or vehicles are not reflected in data – until after the engagement,Footnote 76 or where civilian presence is known but the anticipated harm is not viewed as excessive in relation to the military advantage expected. It is also possible to miss the presence of civilians in dynamic operations, where the targeting timeline is compressed compared to deliberate strikes.Footnote 77 Another cause of civilian harm is weapon malfunction or delivery errors that lead to the target being missed, resulting in civilian harm at the affected location.Footnote 78 In weaponeering, the choice of ordnance size and payload can cause secondary explosions that can impact civilians outside the intended range or occur directly in adjacent areas or structures where civilians are present.Footnote 79
Examination of military operations demonstrates that reducing these risks requires front-loading earlier in the targeting process a comprehensive analysis of the civilian environment, pattern-of-life analysis, tactical alternatives or adjustments, and improved situational awareness rather than deferring to the final engagement decision.Footnote 80 This analysis would need to be updated with new information in real time in order to reflect changes in the civilian environment and adversary movements and tactics. AI systems designed to provide support at this planning stage rather than at the trigger-pull decision are better aligned both with IHL’s precautionary obligations and with the CHM lessons that military operations have generated. Systems that accelerate or automate the decision without equivalent support for the planning and verification stages that precede it increase rather than reduce civilian harm risk.
Civilian harm patterns have direct implications for how AI systems used to support target identification and risk assessments must be designed and tested. Datasets should reflect the types of environments in which systems will be deployed: urban terrain with its population density and infrastructure, maritime environments where military and civilian vessels share the same waterways, and cyber and space domains where dual-use technology affects critical civilian infrastructure including hospitals, navigation systems and communications networks. Periodic testing should evaluate system performance against each harm pattern, assessing whether systems correctly identify military objectives, can detect civilian presence, and can communicate uncertainty under degraded, contested and urban environments. The HMT Assurance Card proposed in this paper recommends integration of civilian harm pattern analysis and domain-specific performance requirements as minimum pre-deployment standards to mitigate against AI failure modes.
Data integrity and the limits of machine outputs
The reliability of AI-enabled systems depends fundamentally on the quality, integrity and traceability of the data on which they operate.Footnote 81 In conflict environments, these dependencies can create significant vulnerabilities. Operational data may be incomplete,Footnote 82 outdated, held in legacy systems, buried in classified systems that are not interoperable across commands or coalition partners,Footnote 83 or degraded by electronic warfare, adverse weather or destruction of sensor infrastructure. Each of these conditions can compromise target identification, selection and engagement in ways that human operators may not be able to detect without adequate transparency into how the system outputs were generated.
Adversarial manipulation of data represents a distinct and compounding risk. An adversary may actively seek to poison training data, manipulate sensor inputs or introduce deceptive signatures which exploit patterns that an AI system has been trained to recognize.Footnote 84 Under such conditions, high system confidence may be due to deception.Footnote 85 When AI systems draw on multiple sensor inputs, a single corrupted source can distort the entire assessment without any indication to the operator.Footnote 86
These vulnerabilities in AI-enabled systems must be fully understood in order to define the conditions for appropriate human judgement. More data does not necessarily result in better decisions, but relevant, trusted, traceable data does. Humans who understand the source, limitations and potential manipulation of the data that the system is relying on are better positioned to interrogate outputs, identify anomalies and exercise appropriate human judgement when using AI systems. This understanding should be built into training, embedded in human–machine interface design, and validated through testing under the adversarial and degraded conditions that characterize real operational environments.
Commanders and IHL
Finally, IHL grants commanders discretion to exercise decisions based on a good-faith interpretation of the information reasonably available at the time.Footnote 87 As AI systems influence that information environment, preparedness requires structured testing of system performance under contested conditions and across multiple domains, including patterns of civilian harm, and transparent documentation of system limits. This needs to be done before deployment to ensure that human–machine teams can behave consistently with applicable law and policies during armed conflict.
Appropriate human judgement can thus be better understood as an operational capability supported by technical design, testing and institutional safeguards. Realizing this requires that human operators be trained on the AI system and its limits, be conversant in key IHL rules and CHM best practices, and operate within procedures that hold them accountable. Translating these conditions into criteria that can be evaluated requires AI governance architecture that is calibrated to military operations. Existing AI risk management frameworks offer structural reference points, but they need to be adapted to human–machine teams operating under the demands of armed conflict.
AI governance frameworks and implementation gap
The challenge of aligning HMT with IHL and CHM best practices requires analysis of AI governance frameworks across civilian technical standards, defence life-cycle models, multilateral consultations and academic scholarship. A systems engineering life-cycle approach provides the most appropriate analytical lens for this inquiry. Ensuring that human–machine teams can operate in compliance with IHL is not a single-point determination; it spans design, procurement, testing, training, deployment and post-deployment monitoring, and requires frameworks applicable across that full life cycle rather than at discrete points within it.
The present paper analyzes three frameworks that together supply the functional architecture that the HMT Assurance Card requires. The National Institute of Standards and Technology AI Risk Management Framework (NIST AI RMF), which is widely used in US federal agencies and is referenced in the DoD’s AI governance framework,Footnote 88 provides a risk governance structure adaptable across institutional roles. The Institute of Electrical and Electronics Engineers Standards Association Research Group’s Autonomous and Intelligent Systems (IEEE AIS) life-cycle framework was explicitly designed for autonomous and AI systems in defence applications across the AI life cycle, giving it relevance to defence procurement and acquisition activities. The Alan Turing Institute’s Defence Assurance methodology, developed in collaboration with the UK Ministry of Defence and Defence AI Centre, provides an auditable evidence generation mechanism not provided for in the NIST AI RMF or IEEE AIS framework. Other frameworks, such as the 2024 European Union AI Act and the 2024 Council of Europe Framework Convention on Artificial Intelligence, do not apply to AI systems used exclusively for defence or national security.Footnote 89 The gap that the proposed HMT Assurance Card seeks to address is the absence of a single instrument integrating legal, technical, operational and civilian harm considerations into a unified evaluative standard applicable across the full AI life cycle, defence institutions and coalition environments.
The NIST AI RMF offers a structured model made up of four functions – govern, map, measure and manage – for identifying, assessing and mitigating risks across the AI life cycle.Footnote 90 The govern function addresses organizational roles, responsibilities and risk culture corresponding to the duty to integrate legal obligations into command structures and review mechanisms.Footnote 91 The map function requires contextualization of AI systems within their operational environment and aligns with the obligation to assess foreseeable risks to civilians prior to deployment.Footnote 92 The measure and manage functions address testing, evaluation and ongoing risk response in ways that correspond to the precautionary obligations to verify targets and take all feasible measures to minimize civilian harm.Footnote 93 Mapping the NIST AI RMF to HMT systems provides a governance architecture, while IHL provides normative standards against which each function can be assessed.
The IEEE AIS framework, proposed in 2024, was explicitly designed for defence applications.Footnote 94 It requires legal, policy and ethical evaluation across nine life-cycle stages, with ongoing activities addressing responsibility, training, test and evaluation, and risk assessment. The framework highlights the need to analyze failure modes, human–machine interactions and safety constraints early and throughout the life cycle.Footnote 95 It recommends ethical and legal evaluations, including in relation to IHL and other legal requirements, across the nine life-cycle stages from the outset.Footnote 96
The Alan Turing Institute’s Defence Assurance methodology offers a complementary mechanism focused on demonstrating trustworthiness of AI systems in national security contexts prior to deployment. It emphasizes the production of evidence that an AI system is understandable, behaves as intended and can be evaluated by reviewers who were not involved in development.Footnote 97 Its system-card framework requires suppliers and defence organizations to jointly document evidence of legal compliance, performance under realistic conditions, supply chain integrity and ethical safeguards in a single auditable record.Footnote 98 This emphasis on transparent evidence generation is relevant to IHL, given the importance of understanding system behaviour for oversight, command responsibility and post-incident investigations.
Multilateral initiatives reinforce that IHL applies to military AI and that compliance must be integrated into the AI development life cycle. The Global Commission on Responsible AI in the Military Domain’s Responsible by Design report advocates embedding ethical and legal considerations across the AI life cycle and emphasizes that responsibility must be integrated from the earliest stages of development.Footnote 99 The International Committee of the Red Cross (ICRC), in its submission to the United Nations pursuant to General Assembly Resolution 79/239, reaffirmed that IHL fully applies to AI in the military domain and emphasized that legal reviews and effective human oversight are essential.Footnote 100
The practical effect of this normative convergence is constrained by the political conditions shaping current military AI development. Strategic competition creates structural incentives to prioritize AI deployment at speed over multilateral governance maturity.Footnote 101 The absence of binding multilateral AI instruments applicable to the military domain means that implementation depends on choices of individual States. In this environment and given the accelerated integration of AI systems into military operations, there is an urgent need for States, defence institutions and developers to adopt AI life-cycle governance frameworks.
Military AI governance discourse highlights the difficulty of translating notions of human control into measurable system properties. Discussions on lethal autonomous weapons (LAWS) have generated calls for human-in-the-loop constraints grounded in the principle of meaningful human control.Footnote 102 In contrast, the United States requires “appropriate levels of human judgement”Footnote 103 under DoD Directive 3000.09, emphasizing procedural safeguards – such as system testing, operator training and robust design – rather than requiring direct human control over every lethal force application. Both standards correctly identify a requirement, but neither provides an evidentiary framework for demonstrating that it has been satisfied.Footnote 104 This gap has proven difficult to close even within alliance settings: a NATO technical research programme, published in 2025, found that alliance members agree that human judgement is essential, but perspectives differ on what this requires in practice, leaving it unclear what specific design guidelines should be derived from the standard.Footnote 105
The practical consequences of this gap are significant. Post-deployment AI drift (where model performance degrades over time as operational conditions diverge from training data), brittle generalization across environments (where a system performs well in training conditions but fails when those conditions shift), adversarial manipulation and over-reliance on opaque outputs can occur under any governance model.Footnote 106 Formal human involvement, however, does not guarantee human judgement that is sufficiently informed to make a decision. Effective oversight requires rigorous validation, including task-specific performance benchmarks, integration of civilian harm patterns, failure audits, stress testing under domain shift and post-deployment monitoring.Footnote 107
This implementation gap between principle and measurable practice underscores the need for a structured framework capable of translating IHL obligations and doctrinal standards into operationally verifiable criteria for human–machine teams. Assaad and Dorsey similarly identify this gap, recommending integrating the IEEE AIS life-cycle framework into targeting processes and autonomous weapons systems, demonstrating that technical standards can inform legal review.Footnote 108 The present paper builds on current research and proposes an HMT Assurance Card to integrate legal, technical, operational and civilian harm considerations into an evidentiary standard.
The HMT Assurance Card
Current legal and AI governance frameworks address different points in the AI system life cycle without integrating them into a unified evidentiary standard,Footnote 109 and existing AI frameworks also do not integrate legal, technical, operational and civilian harm considerations into an evaluative instrument. The proposed HMT Assurance Card is designed to close this gap. It is a structured evaluative instrument that operationalizes appropriate human judgement as an assessable institutional capability, which shifts the inquiry from whether a human is formally present in the decision chain to whether that human is technically and institutionally enabled to exercise genuine judgement consistent with IHL obligations and CHM best practices. It does not replace legal review but complements it by providing structured evidence that human–machine teams are prepared to support lawful conduct across the system life cycle.
The card’s architecture: Four conditions
The card comprises eight components organized around four conditions that must be present for a commander or operator to exercise appropriate human judgement consistent with IHL obligations and CHM practices and to operate within institutional frameworks that require informed and not procedural authorization. Humans and AI systems make errors, albeit different ones – as previously noted, an AI system can detect patterns across large volumes of data that would be missed by a human operator, while an operator brings contextual and legal judgement that a system cannot replicate because it is not in the training data.Footnote 110 A well-structured HMT unit is more operationally reliable than either operating alone. The four proposed conditions are designed around the complementarity of HMT in order to ensure that the human side of the team is technically and institutionally enabled to exercise genuine judgement.
The eight components are legal requirements mapping; civilian harm pattern analysis; domain-specific performance requirements; contested environment and adversarial testing; human–machine interaction criteria; operator training and institutional readiness; audit trail and documentation; and post-deployment monitoring and review. The first four components establish the evidentiary conditions for lawful decision-making, the following two address interface design and operator preparedness, and the final two address institutional accountability and post-deployment assurance.
The first of the four conditions requires that the commander has access to sufficient and reliable information to support the legal assessment. For example, Article 57 of AP I requires that those who plan or decide upon attacks must take all feasible precautions to verify that targets are lawful military objectives, must select means and methods that minimize civilian harm, and must cancel or suspend attacks when harm would be excessive. Each of these obligations is information-dependent; the commander must be able to assess at time of decision whether the available information is sufficient to discharge the obligation. The informational standard that this condition seeks to address is whether the commander is reasonably able to interrogate the output and not merely receive it.
This means that system outputs must communicate uncertainty in terms that a commander can assess, that failure modes are understood and documented,Footnote 111 and that performance has been validated where the risks to civilians are greatest.Footnote 112 When a system presents high-confidence output without communicating the basis for assessment (such as sensor quality, pattern-of-life analysis or all-source intelligence), the commander may lack the information needed to evaluate whether output is reliable enough to act upon. A confidence score presented without disclosure of the model’s calibration performance – that is, whether its stated probabilities correspond to its observed accuracy across operational conditions – can be considered an unanchored number;Footnote 113 the commander would not be able to distinguish between a well-calibrated 75% confidence output and one generated by a model that is systematically overconfident by 20 or 30 percentage points under the conditions of use.Footnote 114 To support informed human judgement, system documentation should include calibration metrics against domain-relevant test conditions such as degraded environments, contested electromagnetic settings and conditions that differ materially from the training data, in order to address this condition of the Assurance Card.
Legal requirements mapping establishes the informational standard that the system should meet to support compliance with IHL. Civilian harm pattern analysis ensures that the system has been evaluated against the specific ways through which civilian harm occurs in its intended domain. Performance requirements should be validated against worst-case rather than average conditions.Footnote 115 It is questionable whether a system that performs adequately in controlled testing but degrades in urban terrain, adverse weather or contested electromagnetic environments can support the precautionary obligation to take feasible measures to minimize civilian harm.
Contested environment and adversarial testing under conditions of electronic warfare, sensor degradation and adversarial manipulation help evaluate system behaviour in real operational environments.Footnote 116 A commander relying on a system that has not been validated under conditions of intended use does not have access to sufficient, reliable and actionable information. For example, the DoD’s Human Systems Integration Test and Evaluation framework illustrates a case where a full-motion video classifier programmed to always return its best guess leads an operator to believe that the system is confident in its classification and consequently to authorize fires on a school bus because the operator was unaware that the system could not express uncertainty.Footnote 117
Notably, a system that performs adequately in controlled conditions may appear misleadingly reliable if the test dataset is not sufficiently operationally realistic. An AI-enabled capability trained on poor-quality or non-representative data will likely be ineffective in the field even where controlled testing suggested otherwise.Footnote 118 For purposes of the Assurance Card, contested environment and adversarial testing should therefore evaluate system performance against specific conditions, including where civilian harm risks are greatest, and not the conditions under which the system was designed to succeed.
The second condition requires that the operator understands the applicable IHL rules and can apply them independently of system outputs. The specific risk that this condition seeks to address is that repeated reliance on AI recommendations can erode the operator’s independent analytical capacity over time, including how to recognize when IHL requires them to override or interrogate a system recommendation. Proportionality analysis cannot be delegated – the assessment of whether anticipated civilian harm is excessive in relation to expected military advantage requires contextual judgement by a commander that a system output can inform but not replace.Footnote 119 Operator training and institutional readiness ensures that operators and commanders know the applicable IHL rules, apply them to system-generated outputs and reach an independent assessment. Training should be validated against realistic scenarios that require operators to exercise judgement under conditions of time pressure, incomplete information and system uncertainty.Footnote 120
The third condition requires that system design and operational conditions support and not replace the commander’s judgement. Automation bias, time pressure and cognitive load create structural incentives for deference to machine outputs. Human–machine interaction criteria require that interface design mitigate the risk of automation bias rather than reinforce it.Footnote 121 This means that operators should be presented with uncertainty sources such as sensor degradation, data gaps or environmental interference as a required step before a confidence score is shown, and that authorization should be structured so that approval requires active input, enabling operators to actively engage with, rather than passively accept, system outputs.Footnote 122 Put another way, an interface design which provides an operator with outputs without identifying sources or uncertainties implies that approval becomes procedural rather than a result of informed judgement. The third condition seeks to address these structural concerns.
The fourth condition requires that the commander operate within an institutional framework that makes appropriate judgement possible, rather than treating human authorization as a procedural formality. The specific risk that this condition addresses is the normalization of rubber-stamp authorization, where human approval becomes a procedural step – especially in AI-enabled targeting, where operational tempo is high, there are large volumes of system-generated outputs, and there is pressure to act. Operator training and institutional readiness would thereby ensure that standard operating procedures create clear accountability pathways and that training is validated against realistic operational scenarios. The audit trail ensures that a complete, auditable record of how the system was designed, tested, reviewed and used is available to post-incident reviewers assessing both system behaviour and the adequacy of human judgement at each life-cycle stage.Footnote 123 Post-deployment monitoring and review treats the Assurance Card as a living instrument, incorporating evidence of actual system performance, civilian harm incidents and material changes in operational conditions as they arise.Footnote 124 This is consistent with lessons learned processes that CHM best practices recommendFootnote 125 and reflects the reality that AI system performance can degrade as operational conditions change.
Table 1 summarizes each component, the condition it addresses, the key performance indicator that demonstrates compliance, and the minimum documentation that must exist in the audit trail. The HMT Assurance Card aims to give engineers, policy-makers, legal advisers, commanders and operators a shared instrument for demonstrating, in terms that are traceable and institutionally auditable, that human–machine teams can exercise informed and appropriate human judgement before they are deployed.
HMT Assurance Card: Components, conditions, performance indicators and minimum audit documentation
Table 1 Long description
The table links four assurance conditions and a consolidated audit trail to required card components, what must be demonstrated, key performance indicators, and minimum audit documents. For sufficient, reliable and actionable information, it requires legal requirements mapping, civilian harm pattern analysis, domain-specific performance requirements, and contested and adversarial testing. Performance is demonstrated through quantified harm pathway error rates and domain thresholds met before deployment. Minimum audit documentation includes risk assessments, validated test results, deployment authorization records, and documented failure modes visible to operators. For understanding applicable IHL rules and civilian harm mitigation, it pairs legal mapping with operator training and readiness, measured by passing pre-deployment IHL knowledge assessments and showing operators can apply rules independently, supported by training curricula, operator training records by system version, assessment results, and legal adviser review. For system design and operational conditions that support human judgement, it emphasizes human–machine interaction criteria, uncertainty communication, and validation under degraded, time-pressured conditions, with indicators such as correct operator interpretation and monitored override and query rates, backed by interaction assessments, stress testing, and operational monitoring records. For an institutional framework for informed judgement, it requires standard operating procedures, monitoring protocols, accessible audit trails, and timely incident reporting and re-evaluation after trigger events, documented through SOPs, incident reports, and lessons-integration records. A final row consolidates all evidence into a single version-controlled life-cycle record with assigned responsibilities, verified at design completion, procurement authorization, pre-deployment validation, and post-incident review, using an integrated master system card plus version control and retention records.
Conclusion
The integration of AI-enabled systems into military operations – tools that filter sensor data, generate targeting assessments and shape the informational basis upon which commanders make decisions – is already reshaping how armed forces exercise judgement during armed conflict. IHL rules on distinction, proportionality, precaution and constant care continue to apply, and commanders remain responsible for ensuring that uses of force comply with those obligations regardless of the technological systems that inform them. What has changed is the complexity of demonstrating such compliance and the urgency of establishing, through peacetime preparation, that human–machine teams are institutionally and technically capable of meeting the standard that IHL requires.
This paper proposes the use of an HMT Assurance Card as a practical instrument for translating IHL obligations and CHM best practices into measurable system requirements across the life cycle of AI-enabled HMT. It argues that appropriate human judgement can be operationalized through four conditions: that commanders and operators have access to sufficient, reliable and actionable information; that they understand and can apply applicable IHL rules and CHM best practices independently of system outputs; that system design supports rather than displaces their independent judgement; and that institutional frameworks require and enable informed rather than formal authorization. The card’s components address these conditions across the system life cycle, drawing on the IEEE AIS life-cycle framework, the NIST AI RMF and the Alan Turing Institute’s Defence Assurance methodology, and mapping each component to specific IHL obligations, performance indicators and minimum audit documentation. In doing so, it shifts the question of IHL compliance from a retrospective assessment of whether a specific decision was lawful to a prospective assessment of whether the institutional and technical conditions for lawful decision-making were established in advance.
The HMT Assurance Card is offered as a starting point for dialogue among developers, defence institutions and policy-makers. Structured engagement between these communities would also seek to address topics such as threshold-setting authority, responsibility between developers and deployers, interoperability issues, and adversarial testing in sensitive target applications. The proposed standards will also need refinement as AI technologies evolve and to reflect operational lessons learned. The card is intended to provide a common language across these communities and is structured around shared performance criteria and a common evidentiary standard so that IHL compliance and CHM best practices are built in before conflict begins. Ultimately, however, the proposed framework can only be effective if there is political will to apply it. Closing the gap between principle and practice in military AI governance requires States to treat AI life-cycle governance as a strategic imperative and to ensure compliance with IHL when AI-enabled technologies are deployed in armed conflict.
Open access
