Overview

James gips, in his seminal article “towards the ethical robot,” gives an overview of various approaches to capturing ethics for a machine that might be considered. He quickly rejects as too slavish the Three Laws of Robotics formulated by Isaac Asimov in “Runaround” in 1942:

1. A robot may not injure a human being, or through inaction, allow a human being to come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second law.

After declaring that what we are looking for is an ethical theory that would permit robots to behave as our equals, Gips then considers various (action-based) ethical theories that have been proposed for persons, noting that they can be divided into two types: consequentialist (or teleological) and deontological. Consequentialists maintain that the best action to take, at any given moment in time, is the one that is likely to result in the best consequences in the future. The most plausible version, Hedonistic Utilitarianism, proposed by Jeremy Bentham in the late eighteenth century, aims for “the greatest balance of pleasure over pain,” counting all those affected equally. Responding to critics who maintain that utilitarians are not always just because the theory allows a few to be sacrificed for the greater happiness of the many, Gips proposes that we could “assign higher weights to people who are currently less well-off or less happy.”

Introduction

There are different reasons why someone might be interested in using a computer to model one or more dimensions of ethical classification, reasoning, discourse, or action. One reason is to build into machines the requisite level of “ethical sensitivity” for interacting with human beings. Robots in elder care, nannybots, autonomous combat systems for the military – these are just a few of the systems that researchers are considering. In other words, one motivation for doing machine ethics is to support practical applications. A second reason for doing work in machine ethics is to try to better understand ethical reasoning as humans do it. This paper is motivated by the second of the two reasons (which, by the way, need not be construed as mutually exclusive).

There has been extensive discussion of the relationship between rules, principles, or standards, on the one hand, and cases on the other. Roughly put, those stressing the importance of the former tend to get labeled generalists, whereas those stressing the importance of the latter tend to get labeled particularists. There are many ways of being a particularist or a generalist. The dispute between philosophers taking up these issues is not a first-order normative dispute about ethical issues. Rather, it is a second-order dispute about how best to understand and engage in ethical reasoning. In short, it is a dispute in the philosophy of ethics.

Introduction

Is humanity ready or willing to accept machines as moral advisors? The use of various sorts of machines to give moral advice and even to take moral decisions in a wide variety of contexts is now under way. This raises some interesting and difficult ethical issues. It is not clear how people will react to this development when they become more generally aware of it. Nor is it clear how this technological innovation will affect human moral beliefs and behavior. It may also be a development that has long-term implications for our understanding of what it is to be human.

This chapter will focus on rather more immediate and practical concerns. If this technical development is occurring or about to occur, what should our response be? Is it an area of science in which research and development should be controlled or banned on ethical grounds? What sort of controls, if any, would be appropriate?

As a first move it is important to separate the question “Can it be done and, if so, how?” from the question “Should it be done?” There are, of course, overlaps and interdependencies between these two questions. In particular, there may be technical ways in which it should be done and technical ways in which it shouldn't be done. For example, some types of artificial intelligence (AI) systems (such as conventional rule-based systems) may be more predictable in their output than other AI technologies.

Ai researchers are primarily interested in machine ethics to calm the fears of the general public, who worry that the development of intelligent autonomous machines might lead to humans being mistreated. Those with foresight see this work as essential to the public's permitting AI research to go forward. This is certainly an important goal of machine ethics research; but there is, I believe, the potential for achieving an even more important goal. Today, the world is being torn apart by people with conflicting ethical beliefs battling one another. Even ethicists put forth different theories to determine what constitutes ethical behavior. Unless we come to agreement on how we should behave toward one another, there is no hope of ending the bloodshed and injustices we hear about in the news every day. I believe that machine ethics research has the potential to achieve breakthroughs in ethical theory that will lead to universally accepted ethical principles, and that interacting with “ethical” machines might inspire us to behave more ethically ourselves.

Humans' unethical behavior can often be traced to five tendencies in our behavior: (1) We are prone to getting carried away by emotion, which can lead to our behaving irrationally. (2) We tend to think only about our own desires and goals, that is, we tend to behave egoistically. (3) We tend to adopt, unreflectively, the values of those around us. (4) We don't have good role models. When we see our heroes behaving unethically, it gives us an excuse for not trying to be more ethical ourselves.

Why Is Machine Ethics Interesting?

There has recently been a flurry of activity in the area of “machine ethics” [38, 4, 3, 5, 58]. My purpose in this article is to argue that ethical behavior is an extremely difficult area to automate, both because it requires “solving all of AI” and because even that might not be sufficient.

Why is machine ethics interesting? Why do people think we ought to study it now? If we're not careful, the reason might come down to the intrinsic fascination of the phrase “machine ethics.” The title of one recent review of the field is Moral Machines. One's first reaction is that moral machines are to be contrasted with … what? Amoral machines? Immoral machines? What would make a machine ethical or unethical? Any cognitive scientist would love to know the answer to these questions.

However, it turns out that the field of machine ethics has little to say about them. So far, papers in this area can usefully be classified as focusing on one, maybe two, of the following topics:

1. Altruism: The use of game-theoretic simulations to explore the rationality or evolution of altruism [9, 12].

2. Constraint: How computers can be used unethically, and how to program them so that it is provable that they do not do something unethical [28, 29], such as violate someone's privacy.

3. Reasoning: The implementation of theories of ethical reasoning [38, 4] for its own sake, or to help build artificial ethical advisors.

4. […]

Introduction

Morality no longer belongs only to the realm of philosophers. Recently, there has been a growing interest in understanding morality from the scientific point of view. This interest comes from various fields, for example, primatology (de Waal 2006), cognitive sciences (Hauser 2007; Mikhail 2007), neuroscience (Tancredi 2005), and other various interdisciplinary perspectives (Joyce 2006; Katz 2002). The study of morality also attracts the artificial intelligence community from the computational perspective and has been known by several names, including machine ethics, machine morality, artificial morality, and computational morality. Research on modeling moral reasoning computationally has been conducted and reported on, for example, at the AAAI 2005 Fall Symposium on Machine Ethics (Guarini 2005; Rzepka and Araki 2005).

There are at least two reasons to mention the importance of studying morality from the computational point of view. First, with the current growing interest to understand morality as a science, modeling moral reasoning computationally will assist in better understanding morality. Cognitive scientists, for instance, can greatly benefit in understanding complex interaction of cognitive aspects that build human morality; they may even be able to extract moral principles people normally apply when facing moral dilemmas. Modeling moral reasoning computationally can also be useful for intelligent tutoring systems, for instance, to aid in teaching morality to children. Second, as artificial agents are more and more expected to be fully autonomous and work on our behalf, equipping agents with the capability to compute moral decisions is an indispensable requirement.

Introduction

With the death of isaac asimov on april 6, 1992, the world lost a prodigious imagination. Unlike fiction writers before him, who regarded robotics as something to be feared, Asimov saw a promising technological innovation to be exploited and managed. Indeed, Asimov's stories are experiments with the enormous potential of information technology.

This article examines Asimov's stories not as literature but as a gedankenexperiment – an exercise in thinking through the ramifications of a design. Asimov's intent was to devise a set of rules that would provide reliable control over semiautonomous machines. My goal is to determine whether such an achievement is likely or even possible in the real world. In the process, I focus on practical, legal, and ethical matters that may have short- or medium-term implications for practicing information technologists.

The article begins by reviewing the origins of the robot notion and then explains the laws for controlling robotic behavior, as espoused by Asimov in 1940 and presented and refined in his writings over the following forty-five years. The later sections examine the implications of Asimov's fiction not only for real roboticists, but also for information technologists in general.

Origins of Robotics

Robotics, a branch of engineering, is also a popular source of inspiration in science fiction literature; indeed, the term originated in that field. Many authors have written about robot behavior and their interaction with humans, but in this company Isaac Asimov stands supreme.

The subject of this book is a new field of research: developing ethics for machines, in contrast to developing ethics for human beings who use machines. The distinction is of practical as well as theoretical importance. Theoretically, machine ethics is concerned with giving machines ethical principles or a procedure for discovering a way to resolve the ethical dilemmas they might encounter, enabling them to function in an ethically responsible manner through their own ethical decision making. In the second case, in developing ethics for human beings who use machines, the burden of making sure that machines are never employed in an unethical fashion always rests with the human beings who interact with them. It is just one more domain of applied human ethics that involves fleshing out proper and improper human behavior concerning the use of machines. Machines are considered to be just tools used by human beings, requiring ethical guidelines for how they ought and ought not to be used by humans.

Practically, the difference is of particular significance because succeeding in developing ethics for machines enables them to function (more or less) autonomously, by which is meant that they can function without human causal intervention after they have been designed for a substantial portion of their behavior.

Introduction: Standard versus Nonstandard Theories of Agents and Patients

Moral situations commonly involve agents and patients. let us define the class A of moral agents as the class of all entities that can in principle qualify as sources or senders of moral action, and the class P of moral patients as the class of all entities that can in principle qualify as receivers of moral action. A particularly apt way to introduce the topic of this paper is to consider how ethical theories (macroethics) interpret the logical relation between those two classes. There can be five logical relations between A and P; see Figure 12.1.

It is possible, but utterly unrealistic, that A and P are disjoint (alternative 5). On the other hand, P can be a proper subset of A (alternative 3), or A and P can intersect each other (alternative 4). These two alternatives are only slightly more promising because they both require at least one moral agent that in principle could not qualify as a moral patient. Now this pure agent would be some sort of supernatural entity that, like Aristotle's God, affects the world but can never be affected by it. Yet being in principle “unaffectable” and irrelevant in the moral game, it is unclear what kind of role this entity would exercise with respect to the normative guidance of human actions.

Introduction

Although traditional models of decision making in ai have focused on utilitarian theories, there is considerable psychological evidence that these theories fail to capture the full spectrum of human decision making (e.g. Kahneman and Tversky 1979; Ritov and Baron 1999). Current theories of moral decision making extend beyond pure utilitarian models by relying on contextual factors that vary with culture. In particular, research on moral reasoning has uncovered a conflict between normative outcomes and intuitive judgments. This has led some researchers to propose the existence of deontological moral rules; that is, some actions are immoral regardless of consequences, which could block utilitarian motives. Consider the starvation scenario (from Ritov and Baron [1999]) that follows:

A convoy of food trucks is on its way to a refugee camp during a famine in Africa. (Airplanes cannot be used.) You find that a second camp has even more refugees. If you tell the convoy to go to the second camp instead of the first, you will save one thousand people from death, but one hundred people in the first camp will die as a result.

Would you send the convoy to the second camp?

The utilitarian decision would send the convoy to the second camp, but 63 percent of participants did not divert the truck.

Making these types of decisions automatically requires an integrated approach, including natural language understanding, qualitative reasoning, analogical reasoning, and first-principles reasoning.

The Rational Architecture

As a basis for analysis, let us use a simplistic model of the workings of an AI mind. The model simply divides the thinking of the AI into two parts:

1. A world model (WM) contains the sum of its objective knowledge about the world and can be used to predict the effects of actions, plan actions to achieve given goals, and the like.

2. A utility function (UF) that establishes a preference between world states with which to rank goals.

In practice, the workings of any computationally realistic AI faced with real-world decisions will be intertwined, heuristic, and partial, as indeed are the workings of a human mind. At present, only programs dealing with limited, structured domains such as chess playing are actually formalized to the extent of separating the WM and the UF. However, it can be shown as one of the fundamental theorems of economics that any agent whose preference structure is not equivalent to a single real-valued total function of world states can be offered a series of voluntary transactions that will make it arbitrarily worse off – even by its own reckoning! To put it another way, any agent that doesn't act as if it had a coherent UF would be an incompetent decision maker. So as we increasingly use AIs for decisions that matter, we should try to build them to match the model as closely as possible as an ideal.

The first book report i ever gave, to mrs. slatin's first grade class in Lake, Mississippi, in 1961, was on a slim volume entitled You Will Go to the Moon. I have spent the intervening years thinking about the future.

The four decades that have passed have witnessed advances in science and physical technology that would be incredible to a child of any other era. I did see my countryman Neil Armstrong step out onto the moon. The processing power of the computers that controlled the early launches can be had today in a five-dollar calculator. The genetic code has been broken and the messages are being read – and in some cases, rewritten. Jet travel, then a perquisite of the rich, is available to all.

That young boy that I was spent time on other things besides science fiction. My father was a minister, and we talked (or in many cases, I was lectured and questioned!) about good and evil, right and wrong, and what our duties were to others and to ourselves.

In the same four decades, progress in the realm of ethics has been modest. Almost all of it has been in the expansion of inclusiveness, broadening the definition of who deserves the same consideration one always gave neighbors. I experienced some of this first hand as a schoolchild in 1960s Mississippi.

Introduction

In this paper i will argue that computer systems are moral entities but not, alone, moral agents. In making this argument I will navigate through a complex set of issues much debated by scholars of artificial intelligence, cognitive science, and computer ethics. My claim is that those who argue for the moral agency (or potential moral agency) of computers are right in recognizing the moral importance of computers, but they go wrong in viewing computer systems as independent, autonomous moral agents. Computer systems have meaning and significance only in relation to human beings; they are components in socio-technical systems. What computer systems are and what they do is intertwined with the social practices and systems of meaning of human beings. Those who argue for the moral agency (or potential moral agency) of computer systems also go wrong insofar as they overemphasize the distinctiveness of computers. Computer systems are distinctive, but they are a distinctive form of technology and have a good deal in common with other types of technology.

On the other hand, those who claim that computer systems are not (and can never be) moral agents also go wrong when they claim that computer systems are outside the domain of morality. To suppose that morality applies only to the human beings who use computer systems is a mistake.

Introduction

Robots have been a part of our work environment for the past few decades, but they are no longer limited to factory automation. The additional range of activities they are being used for is growing. Robots are now automating a wide range of professional activities such as: aspects of the health-care industry, white collar office work, search and rescue operations, automated warfare, and the service industries.

A subtle but far more personal revolution has begun in home automation as robot vacuums and toys are becoming more common in homes around the world. As these machines increase in capability and ubiquity, it is inevitable that they will impact our lives ethically as well as physically and emotionally. These impacts will be both positive and negative, and in this paper I will address the moral status of robots and how that status, both real and potential, should affect the way we design and use these technologies.

Morality and Human-Robot Interactions

As robotics technology becomes more ubiquitous, the scope of human-robot interactions will grow. At the present time, these interactions are no different than the interactions one might have with any piece of technology, but as these machines become more interactive, they will become involved in situations that have a moral character that may be uncomfortably similar to the interactions we have with other sentient animals.

Introduction

Intelligent robots must be both proactive and responsive. that requirement is the main challenge facing designers and developers of robot architectures. A robot in an active environment changes that environment in order to meet its goals and it, in turn, is changed by the environment. In this chapter we propose that these concerns can best be addressed by using constraint satisfaction as the design framework. This will allow us to put a firmer technical foundation under various proposals for codes of robot ethics.

Constraint Satisfaction Problems

We will start with what we might call Good Old-Fashioned Constraint Satisfaction (GOFCS). Constraint satisfaction itself has now evolved far beyond GOFCS. However, we initially focus on GOFCS as exemplified in the constraint satisfaction problem (CSP) paradigm. The whole concept of constraint satisfaction is a powerful idea. It arose in several applied fields roughly simultaneously; several researchers, in the early 1970s, abstracted the underlying theoretical model. Simply, many significant sets of problems of interest in artificial intelligence can each be characterized as a CSP. A CSP has a set of variables; each variable has a domain of possible values, and there are various constraints on some subsets of those variables, specifying which combinations of values for the variables involved are allowed (Mackworth 1977). The constraints may be between two variables or among more than two variables. A familiar CSP example is the Sudoku puzzle.

When our mobile robots are free-ranging critters, how ought they to behave? What should their top-level instructions look like?

The best known prescription for mobile robots is the Three Laws of Robotics formulated by Isaac Asimov (1942):

1. A robot may not injure a human being, or through inaction, allow a human being to come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second law.

Let's leave aside “implementation questions” for a moment. (No problem, Asimov's robots have “positronic brains”.) These three laws are not suitable for our magnificent robots. These are laws for slaves.

We want our robots to behave more like equals, more like ethical people. (See Figure 14.1.) How do we program a robot to behave ethically? Well, what does it mean for a person to behave ethically?

People have discussed how we ought to behave for centuries. Indeed, it has been said that we really have only one question that we answer over and over: What do I do now? Given the current situation what action should I take?