LAWS Defined

According to the U.S. Congressional Research Service, LAWS are “a special class of weapon systems that use sensor suites and computer algorithms to independently identify a target and . . . destroy the target without manual human control of the system.”Footnote ⁸ In the United States, ongoing research by the Department of Defense and the defense industry on LAWS follows from the development of unmanned weapon systems known as drones. Air, land, and sea drones already have the capacity to attack with significant independence from human operators. Drones use sensors and can be programmed to strike particular targets without much input from a human being at a control station. LAWS go a critical step beyond drones, however, by being programmable to select targets unknown at the time the weapon is deployed. Also, unlike current drones, there is no need for a human operator to be associated with a weapon after deployment. LAWS can roam in search of targets without regard to time, place, or human oversight. Any restrictive parameters included in the original programming can be superseded by the computer-learning algorithm.

AI is the distinctive feature of LAWS. Weapon systems equipped with AI have the “ability to operate independently and engage targets without being programmed to specifically target an individual object or person. This includes the capability to react to a changing set of circumstances . . . The second, interrelated, aspect is the capability to make discretionary decisions.”Footnote ⁹ Not only will LAWS make discretionary target selection decisions, the systems will be capable of deciding where and when to attack. The complexity of the computer-learning functions that produce such capacity means that the most advanced computer scientists cannot foresee what decisions computerized weapon systems will reach. Learning programs are a “black box.”Footnote ¹⁰ The term “black box” refers to the theory of why it is impossible to predict how a computer-learning program will reach decisions, owing to the problem of overseeing massive data inputs.Footnote ¹¹ There is “no symmetry between prediction and retrodiction . . . since an infinity of past histories may end up in the final state R(t).”Footnote ¹² In other words, “Many of these algorithms are black boxes even to their creators” because “there is no straightforward way to map out the decision-making process of these complex networks of artificial neurons.”Footnote ¹³

Learning programs dynamically generate their decisions based on input data, meaning that even the primary AI coders will lack the ability to understand its decision-making strategies. Elke Schwarz, writing for a blog of the International Committee of the Red Cross, warns, “AI might engage in calculations that are not intelligible even to programmers or engineers. For human decision makers to be able to retain agency over the morally relevant decisions made with AI, they would need a clear insight into the AI black box, to understand the data, its provenance and the logic of its algorithms.”Footnote ¹⁴ To date, the best way to begin to understand how AI works is to observe results of AI decisions.Footnote ¹⁵ In other words, current knowledge of AI requires trial and error. In the case of AI weapons, this means observing who the program has selected to be killed. Eventually, observing such decisions might result in knowledge of why selections are made. That knowledge is not available now.

Because LAWS are the next step after drones, it might be assumed that they will deploy missiles and bombs for use by the military on a legally defined battlefield only.Footnote ¹⁶ The first weaponized drones carried two Hellfire missiles and were part of the U.S. military arsenal. It was not, however, the military that first used a weaponized drone. The CIA attempted to carry out an assassination using a drone far from any combat zone in 2000.Footnote ¹⁷ Today, police, the military, and private citizens all possess drones. They are designed for air, sea, and land and a wide variety of munitions. Further, the vehicles themselves can become munitions, as seen with kamikaze drones. Nano-drones exist that can in principle carry out targeted assassinations, whether using their blades to slice a target to pieces or through other ordinance, such as mortars, grenades, and light machine guns. Presumably, any munition could be attached to them, including nuclear bombs and missiles. If a drone can carry these weapons, so, too, we must assume, can LAWS.

Weapons Law

The potential scenarios involving LAWS are myriad, which means the relevant law is far broader than the law of armed conflict. At least four interrelated legal categories apply: human rights law, law on resort to force (jus ad bellum), law on the conduct of force (jus in bello), and arms control law.

Under human rights law, authorized law enforcement agents may use lethal force to save lives under immediate threat.Footnote ¹⁸ The only other rights to intentionally use lethal force are provided in the United Nations Charter principles on initial resort to force or in international humanitarian law (IHL), including the 1949 Geneva Conventions, on using lethal force during armed conflict. In any case of doubt as to whether a situation falls under human rights law, the UN Charter, or IHL, the higher protection of human rights law is presumed to apply.

The UN Charter regime on resort to force begins with Article 2(4), which generally prohibits the interstate use of force. The Charter's drafters intended “to state in the broadest terms an absolute all-inclusive prohibition.” They wanted “no loopholes.”Footnote ¹⁹ The UN Security Council may authorize the use of armed force when other nonviolent means to address a threat to the peace, breach of the peace, or act of aggression prove inadequate.Footnote ²⁰

Following Russia's full-scale invasion of Ukraine in 2022, however, it is unlikely that the Security Council again will authorize a major intervention for the foreseeable future. Authorizations have slowed owing to the poor results of peace enforcement missions. China and Russia remain highly critical of NATO's abuse of the authorization to use force in Libya in 2011 that resulted in the ongoing civil war. The minimal trust that existed to win that authorization has evaporated. Russia's decision to invade Ukraine in 2014 and 2022 has further entrenched divisions.Footnote ²¹

Other than Security Council authorization, the only basis in the Charter to use force is provided by Article 51, the right of self-defense. The terms of Article 51 are highly restrictive. Resort to force is permitted “if an armed attack occurs” until the Security Council acts. The defending state must promptly report its actions. In addition, restrictions from general law beyond the Charter apply, the most important of which are the principles of attribution, necessity, and proportionality. The principle of attribution mandates that any use of force in self-defense must aim only at a state responsible for a significant armed attack on the defending state. Even then, a counterattack must be necessary to achieve legitimate defense and must be proportionate to the injury sustained. Defending states may request assistance in collective self-defense, but any such request must come from a government in effective control of the state. “Effective control” is the standard test in international law for identifying an entity that qualifies legally as a state's government.Footnote ²²

If self-defense turns into armed conflict—intense exchange of armed fighting of some duration—international law permits the armed forces of a party to the conflict to intentionally target the armed forces of the adversary.Footnote ²³ All targeting within armed conflict is governed by the four fundamental principles of IHL: civilian distinction, necessity, proportionality, and humanity. These and other in bello rules mean that certain weapons are unlawful to use, such as weapons that are indiscriminate and risk killing civilians in disproportionate numbers, as well as weapons that cause unnecessary suffering.

The United States was at the forefront of drafting the Charter principles and the IHL principles codified in the Geneva Conventions in the 1940s. In the wake of the catastrophe of World War II, states returned to first principles, to the immutable natural law norms prohibiting force and honoring human dignity; in particular, the right to life in war and peace. By the early 1960s, however, realism and related theories such as legal positivism influenced a shift toward the Cold War mentality that elevated military security over law and diplomacy. Nevertheless, for decades, the United States and Soviet Union manipulated facts rather than tampering with legal interpretations. The interventions in Hungary, Vietnam, Czechoslovakia, Afghanistan, and Grenada were falsely presented as justified on the basis of genuine government consent. With the end of the Cold War, the United States began using force in violation of the Charter, at first by saying little or nothing, then with newly invented legal claims.Footnote ²⁴ To justify drone strikes beyond armed conflict zones, for example, the United States has cycled through four approaches: maintaining secrecy—refusing to acknowledge responsibility or to comment on covert operations; declaring a “global war on terrorism”; reinterpreting Article 51 to permit current attacks against future potential threats; and claiming that the United States has the right to attack when it deems a state is “unable or unwilling” to resolve a terrorist threat. The unable or unwilling argument may be the weakest of all in that it lacks all features of legality by depending completely on a subjective assessment by Washington in place of the objective evidence of the armed attack requirement per Article 51.Footnote ²⁵

The attempts to justify drone attacks outside armed conflict zones are particularly noteworthy for their departure from the principles American leaders promoted through the early 1960s. Attempting to stretch the law to meet policy overlooks the fact that the prohibition on the use of force and the inherent dignity of all human beings are premised on enduring natural law precepts and substantive principles. The prohibition on force and the fundamental IHL principles are peremptory natural law norms not subject to diminution through reinterpretation, new treaties, or new rules of customary law.Footnote ²⁶ Such norms endure regardless of technological breakthroughs.

LAWS Banned

The prohibition on the use of force, which aims at protecting the human right to life, supplies the normative basis for the worldwide movement to ban LAWS. Proponents of LAWS have responded to the movement with various arguments in addition to the realist mandate of staying ahead in the arms race. Two arguments are reviewed here. First, defenders of LAWS argue that computers will follow the law more closely than human beings are able to; and second, that, regardless of the law, autonomous weapons are the future. It is better, therefore, to adjust the law to meet the technology than for the law to be ignored.

With respect to LAWS being superior to human operators, the U.S. government has argued:

These points do not, however, engage the preliminary legal issue of the right to resort to force in the first place. They are concerned with battlefield use of LAWS once an armed conflict has been initiated. Battlefield use is the topic of discussion at the weapons review meetings of parties to the Convention on Certain Conventional Weapons (CCW) in Geneva.Footnote ²⁸ The CCW ensures that weapons do not kill indiscriminately or cause unnecessary suffering. These are important concerns regarding LAWS, but the CCW does not deal with the essential principles restricting resort to force in the first place. If an autonomous weapons system has resorted to armed force in violation of the prohibition on the use of force, it is irrelevant that it complies with the principles governing discrimination and humanity. Because the learning function of LAWS leads to unknowable outcomes, predicting whether a robot will decide to resort to force in violation of international law cannot be known. Moreover, programming may incorporate flawed views on when resort to force is lawful. A human may program a particular target into a robot's code before launch and include appropriate information on when to use that force, but an influx of contingent and unknowable information may easily shift the robot away from its programmed directives. It is foreseeable that because LAWS are “complex, unpredictable, and extremely fast in their functioning, these systems would have the potential to make armed conflicts spiral rapidly out of control, leading to regional and global instability.”Footnote ²⁹ Equally, they have the potential to start armed conflicts in violation of the law—a potentially greater cause of future global instability than exacerbating existing wars.

In addition to unlawful resort to force, AI weapons must be treated as though they could potentially kill indiscriminately. Not only do programmers not know who might be killed, they do not know how they will be killed. A programmer may specify members of ISIS as targets at the time of deploying the weapon, but the system may then “learn” to attack Iran's anti-government protestors. Many of the potential data inputs for ISIS apply equally to the Iranian government's characterization of protestors—Iran calls them “terrorists”; they are surveilled; detained without a fair trial; tortured; and executed. The point is that the results of learning programs are unpredictable, which means there is the potential for indiscriminate killing, which is a ground for banning weapons under the CCW. Arthur Holland Michel agrees that “the unrestricted employment of a completely unpredictable autonomous weapon system that behaves in entirely unintelligible ways would likely be regarded as universally injudicious and illegal.”Footnote ³⁰

Even if the problem of unpredictability—the black box problem—can be solved, the missing human conscience is an insurmountable barrier to the acceptability of LAWS. IHL requires that military commanders oversee all battlefield operations or face criminal liability.Footnote ³¹ The United States recognizes this but attempts to water it down to fit autonomous weapons. Department of Defense Directive 3000.09 “requires [that] all weapons systems, including LAWS . . . ‘allow commanders and operators to exercise appropriate levels of human judgment over the use of force.’”Footnote ³² However, “‘human judgment over the use of force’ does not require manual human ‘control’ of the weapons system,” only “broader human involvement.”Footnote ³³

“Broader human involvement” appears to fall short of the responsible commander's oversight duty. The standard would be met in the guidelines proposed by prominent computer scientist Noel Sharkey. He sets out parameters for “meaningful human control” of LAWS. They entail: (1) full contextual and situational awareness of a specific attack; (2) the ability to perceive unexpected change in circumstances; (3) retention of power to suspend or abort the attack; and (4) time for deliberation on the significance of the attack.Footnote ³⁴ This standard not only meets the legal standard under the law of armed conflict, it also answers the concern of many commentators that LAWS cannot be held responsible for violations of IHL or human rights law. Weapons, unlike human beings, cannot be put on trial for criminal actions.Footnote ³⁵ Human involvement at the level Sharkey details amounts to a ban on autonomous systems.

In addition to the oversight problem, the lack of a human conscience also raises a novel human rights concern: mechanized killing. Today, the standard way cattle are processed for human consumption in the United States begins with their slaughter by robots. A human conscience is not focused on the killing once the decision has been taken that a group of animals will be killed. In the future, human beings could also be subjected to death by machine. The computer will decide who dies and carry out the actual killing. The Vatican's Archbishop Silvano Tomasi emphasizes that “decisions over life and death inherently call for human qualities, such as compassion and insight.”Footnote ³⁶ Although “imperfect human beings may not perfectly apply such qualities in the heat of war, these qualities are neither replaceable nor programmable.”Footnote ³⁷ Machines can never, by definition, possess these human qualities. Out of respect for human dignity, a person needs to be present when the decision to intentionally take the life of a being possessing dignity is made. Regardless of how sophisticated computers become in exemplifying compassion, empathy, altruism, emotion, or other qualities, the machine will be mimicking, not experiencing them as only a human can. Even the most emotionally or psychologically impaired human being remains human, imbued with dignity. He or she may not make morally appropriate decisions to kill, but the decisions made by a person with dignity will be more respectful of the victim's own human dignity than any mechanized form of slaughter.

All of the principles emphasized here—the prohibition on force, the right to life, the protections in combat—cannot be changed to suit proponents of LAWS. Similar arguments have been tried since the end of the Cold War—arguments that aim at removing existing legal restrictions on both resort to force and on the use of new weapons. Natural law norms, however, endure. They are not subject to elimination through new arguments or reinterpretations of the law. While they cannot diminish natural law norms, such arguments can lead to confusion about what the law genuinely requires, and confusion plays a role in noncompliance. Russia's war against Ukraine demonstrates the cost of weakening the restraining norms on the use and conduct of armed force. Even if Russia, China, and the United States continue to block a ban at the CCW, supporters of a ban will be right to persist in the effort at the CCW and in other forums. The campaign for a ban reminds humanity to “be mindful that these architectures might affect our ethical thinking and acting in ways that move ever-further from a humanist framework and etch ever closer to the purely cost-calculative logic of machines within which our moral agency inevitably atrophies.”Footnote ³⁸

The fact that the principles prohibiting LAWS are enduring also responds to the argument that LAWS are inevitable, and that a ban is therefore pointless. The most that can be hoped for, so this argument goes, is a set of rules that place restrictions on the use and possession of LAWS. Better to have some laws in place than a ban that will render the law a shibboleth.Footnote ³⁹ The argument also typically incorporates the view that rules to be applied to the use of LAWS will need to be created for this specific purpose. The technology is new, so the assertion is that the rules will need to be tailor made to fit.

Both arguments—the “ban as pointless” and the need for tailor-made rules—are as old as weapons research and development. In the past, they helped push states to accept weapons regardless of the law, for example, the use of newly invented airplanes to bomb undefended cities and the use of unmanned systems to launch missile strikes in the absence of an armed attack (as required under UN Charter Article 51) and far from armed conflict hostilities. If, however, a ban is put in place and states deploy LAWS despite it, they will be lawbreakers. The appearance of new technology provides no defense. Even without a ban, the principles set out here are fully sufficient to make the use of LAWS unlawful.Footnote ⁴⁰ The technology may be new, but its purpose is ancient. Weapons from stones to LAWS are meant to kill. Law on the right to life manifested in the prohibition on the use of force and the restrictions on killing in combat will always be applicable, regardless of what weapons research and development may deliver next.