In this essay i use the 2004 film i, robot as a philosophical resource for exploring several issues relating to machine ethics. Although I don't consider the film particularly successful as a work of art, it offers a fascinating (and perhaps disturbing) conception of machine morality and raises questions that are well worth pursuing. Through a consideration of the film's plot, I examine the feasibility of robot utilitarians, the moral responsibilities that come with creating ethical robots, and the possibility of a distinct ethics for robot-to-robot interaction as opposed to robot-to-human interaction.

I, Robot and Utilitarianism

I, Robot's storyline incorporates the original “three laws” of robot ethics that Isaac Asimov presented in his collection of short stories entitled I, Robot. The first law states:

This sounds like an absolute prohibition on harming any individual human being, but I, Robot's plot hinges on the fact that the supreme robot intelligence in the film, VIKI (Virtual Interactive Kinetic Intelligence), evolves to interpret this first law rather differently. She sees the law as applying to humanity as a whole, and thus she justifies harming some individual humans for the sake of the greater good:

One way to view the puzzle of machine ethics is to consider how we might program computers that will themselves refrain from evil and perhaps promote good. Consider some steps along the way to that goal. Humans have many ways to be ethical or unethical by means of an artifact or tool; they can quell a senseless riot by broadcasting a speech on television or use a hammer to kill someone. We get closer to machine ethics when the tool is a computer that's programmed to effect good as a result of the programmer's intentions. But to be ethical in a deeper sense – to be ethical in themselves – machines must have something like practical reasoning that results in action that causes or avoids morally relevant harm or benefit. So, the central question of machine ethics asks whether the machine could exhibit a simulacrum of ethical deliberation. It will be no slight to the machine if all it achieves is a simulacrum. It could be that a great many humans do no better.

Of course, philosophers have long disagreed about what constitutes proper ethical deliberation in humans. The utilitarian tradition holds that it's essentially arithmetic: we reach the right ethical conclusion by calculating the prospective utility for all individuals who will be affected by a set of possible actions and then choosing the action that promises to maximize total utility.

Introduction: Better than Human

We get better at being moral. unfortunately, this doesn't mean that we can get moral enough, that we can reach the heights of morality required for the flourishing of all life on planet Earth. Just as we are epistemically bounded, we also seem to be morally bounded. This fact coupled both with the fact that we can build machines that are better than we in various capacities as well as the fact that artificial intelligence is making progress entail that we should build or engineer our replacements and then usher in our own extinction. Put another way, the moral environment of modern Earth wrought by humans, together with what current science tells us of morality, human psychology, human biology, and intelligent machines, morally requires us to build our own replacements and then exit stage left. This claim might seem outrageous, but in fact it is a conclusion born of good old-fashioned rationality.

In this paper, I show how this conclusion is forced upon us. Two different possible outcomes, then, define our future; the morally best one is the second. In the first, we will fail to act on our duty to replace ourselves. Eventually, as it has done with 99 percent of all species over the last 3.5 billion years, nature will step in to do what we lacked the courage to do. Unfortunately, nature is very unlikely to bring our replacements with it. However, the second outcome is not completely unlikely.

Once people understand that machine ethics is concerned with how intelligent machines should behave, they often maintain that Isaac Asimov has already given us an ideal set of rules for such machines. They have in mind Asimov's Three Laws of Robotics:

1. A robot may not injure a human being, or, through inaction, allow a human being to come to harm.

2. A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law. (Asimov 1976)

I shall argue that in “The Bicentennial Man” (Asimov 1976), Asimov rejected his own Three Laws as a proper basis for Machine Ethics. He believed that a robot with the characteristics possessed by Andrew, the robot hero of the story, should not be required to be a slave to human beings as the Three Laws dictate. He further provided an explanation for why humans feel the need to treat intelligent robots as slaves, an explanation that shows a weakness in human beings that makes it difficult for them to be ethical paragons. Because of this weakness, it seems likely that machines like Andrew could be more ethical than most human beings.

Introduction

How can machines support, or even more significantly replace, humans in performing ethical reasoning? This is a question of great interest to those engaged in Machine Ethics research. Imbuing a computer with the ability to reason about ethical problems and dilemmas is as difficult a task as there is for Artificial Intelligence (AI) scientists and engineers. First, ethical reasoning is based on abstract principles that cannot be easily applied in formal, deductive fashion. Thus the favorite tools of logicians and mathematicians, such as first-order logic, are not applicable. Second, although there have been many theoretical frameworks proposed by philosophers throughout intellectual history, such as Aristotelian virtue theory (Aristotle, edited and published in 1924), the ethics of respect for persons (Kant 1785), Act Utilitarianism (Bentham 1789), Utilitarianism (Mill 1863), and prima facie duties (Ross 1930), there is no universal agreement on which ethical theory or approach is the best. Furthermore, any of these theories or approaches could be the focus of inquiry, but all are difficult to make computational without relying on simplifying assumptions and subjective interpretation. Finally, ethical issues touch human beings in a profound and fundamental way. The premises, beliefs, and principles employed by humans as they make ethical decisions are quite varied, not fully understood, and often inextricably intertwined with religious beliefs. How does one take such uniquely human characteristics and distil them into a computer program?

A runaway trolley is approaching a fork in the tracks. if the trolley runs on its current track, it will kill a work crew of five. If the driver steers the train down the other branch, the trolley will kill a lone worker. If you were driving the trolley, what would you do? What would a computer or robot do? Trolley cases, first introduced by philosopher Philippa Foot in 1967[1] and now a staple of introductory ethics courses, have multiplied in the past four decades. What if it's a bystander, rather than the driver, who has the power to switch the trolley's course? What if preventing the five deaths requires pushing another spectator off a bridge onto the tracks? These variants evoke different intuitive responses.

Given the advent of modern “driverless” train systems, which are now common at airports and are beginning to appear in more complicated rail networks such as the London Underground and the Paris and Copenhagen metro systems, could trolley cases be one of the first frontiers for machine ethics? Machine ethics (also known as machine morality, artificial morality, or computational ethics) is an emerging field that seeks to implement moral decision-making faculties in computers and robots. Is it too soon to be broaching this topic? We don't think so.

In this part, four visions of the future of machine ethics are presented. Helen Seville and Debora G. Field, in “What Can AI Do for Ethics?” maintain that AI is “ideally suited to exploring the processes of ethical reasoning and decision-making,” and that, through the World Wide Web, an Ethical Decision Assistant (EDA) can be created that is accessible to all. Seville and Field believe that an acceptable EDA for personal ethical decision making should incorporate certain elements, including the ability to (1) point out the consequences, short and long term, not only of the actions we consider performing, but also of not performing certain actions; (2) use virtual reality techniques to enable us to “experience” the consequences of taking certain courses of action/inaction, making it less likely that we will err because of weakness of will; and (3) emphasize the importance of consistency. They argue, however, that there are limits to the assistance that a computer could give us in ethical decision making. In ethical dilemmas faced by individuals, personal values will, and should, in their view, have a role to play.

Seville and Field, interestingly, believe that AI could create a system that, in principle, would make better decisions concerning ethically acceptable social policies because of its ability to check for consistency and its ability to be more impartial (understood as being able to represent and consider the experiences of all those affected) than human beings.

Introduction

The responsibilities of a system designer are growing and expanding in fields that only ten years ago were the exclusive realms of philosophy, sociology, or jurisprudence. Nowadays, a system designer must have a deep understanding not only of the social and legal implications of what he is designing, but also of the ethical nature of the systems he is conceptualizing. These artifacts not only behave autonomously in their environments, embedding themselves into the functional tissue or our society but also “re-ontologise” part of our social environment, shaping new spaces in which people operate.

It is in the public interest that automated systems minimize their usage of limited resources, are safe for users, and integrate ergonomically within the dynamics of everyday life. For instance, one expects banks to offer safe, multifunction ATMs, hospitals to ensure that electro-medical instruments do not electrocute patients, and nuclear plants to employ redundant, formally specified control systems.

It is equally important to the public interest that artificial autonomous entities behave correctly. Autonomous and interactive systems affect the social life of millions of individuals, while performing critical operations such as managing sensitive information, financial transactions, or the packaging and delivery of medicines. The development of a precise understanding of what it means for such artifacts to behave in accordance with the ethical principles endorsed by a society is a pressing issue.

The newly emerging field of machine ethics is concerned with ensuring that the behavior of machines toward human users is ethically acceptable. There are domains in which intelligent machines could play a significant role in improving our quality of life as long as concerns about their behavior can be overcome by ensuring that they behave ethically. Machine metaethics examines the field of machine ethics. It talks about the field, rather than doing work in it. Examples of questions that fall within machine metaethics are: How central are ethical considerations to the development of artificially intelligent agents? What is the ultimate goal of machine ethics? What does it mean to add an ethical dimension to machines? Is ethics computable? Is there a single correct ethical theory that we should try to implement? Should we expect the ethical theory we implement to be complete? That is, should we expect it to tell a machine how to act in every ethical dilemma? How important is consistency? If it is to act in an ethical manner, is it necessary to determine the moral status of the machine itself?

When does machine behavior have ethical import? How should a machine behave in a situation in which its behavior does have ethical import? Consideration of these questions should be central to the development of artificially intelligent agents that interact with humans.

The question of whether machine ethics exists or might exist in the future is difficult to answer if we can't agree on what counts as machine ethics. Some might argue that machine ethics obviously exists because humans are machines and humans have ethics. Others could argue that machine ethics obviously doesn't exist because ethics is simply emotional expression and machines can't have emotions.

A wide range of positions on machine ethics are possible, and a discussion of the issue could rapidly propel us into deep and unsettled philosophical issues. Perhaps, understandably, few in the scientific arena pursue the issue of machine ethics. You're unlikely to find easily testable hypotheses in the murky waters of philosophy. But we can't – and shouldn't – avoid consideration of machine ethics in today's technological world.

As we expand computers' decision-making roles in practical matters, such as computers driving cars, ethical considerations are inevitable. Computer scientists and engineers must examine the possibilities for machine ethics because, knowingly or not, they've already engaged – or will soon engage – in some form of it. Before we can discuss possible implementations of machine ethics, however, we need to be clear about what we're asserting or denying.

Introduction

Practical ethics typically addresses itself to such general issues as whether we ought to carry out abortions or slaughter animals for meat, and, if so, under what circumstances. The answers to these questions have a useful role to play in the development of social policy and legislation. They are, arguably, less useful to the ordinary individual wanting to ask:

Such diverse ethical theories as Utilitarianism (Mill, 1861) and Existentialism (MacQuarrie, 1972) do address themselves to the question of how we ought to go about making such decisions. The problem with these, however, is that they are generally inaccessible to the individual facing a moral dilemma.

This is where AI comes in. It is ideally suited to exploring the processes of ethical reasoning and decision-making, and computer technology such as the world wide web is increasingly making accessible to the individual information which has only been available to “experts” in the past. However, there are questions which remain to be asked such as:

• Could we design an Ethical Decision Assistant for everyone? i.e., could we provide it with a set of minimal foundational principles without either committing it to, or excluding users from, subscribing to some ethical theory or religious code?

• What would its limitations be? i.e., how much could/ should it do for us and what must we decide for ourselves?

• How holistic need it be? i.e., should it be restricted to “pure” ethical reasoning or need it consider the wider issues of action and the motivations underlying it?

The challenges facing those working on machine ethics can be divided into two main categories: philosophical concerns about the feasibility of computing ethics and challenges from the AI perspective. In the first category, we need to ask first whether ethics is the sort of thing that can be computed. One well-known ethical theory that supports an affirmative answer to this question is Act Utilitarianism. According to this teleological theory (a theory that maintains that the rightness and wrongness of actions is determined entirely by the consequences of the actions), the right act is the one, of all the actions open to the agent, which is likely to result in the greatest net good consequences, taking all those affected by the action equally into account. Essentially, as Jeremy Bentham (1781) long ago pointed out, the theory involves performing “moral arithmetic.”

Of course, before doing the arithmetic, one needs to know what counts as “good” and “bad” consequences. The most popular version of Act Utilitarianism – Hedonistic Act Utilitarianism – would have us consider the pleasure and displeasure that those affected by each possible action are likely to receive. As Bentham pointed out, we would probably need some sort of scale to account for such things as the intensity and duration of the pleasure or displeasure that each individual affected is likely to receive. This is information that a human being would need to have, as well, in order to follow the theory.

Introduction

This paper introduces an approach to, rather than the final results of, sustained research and development in the area of roboethics described herein. Encapsulated, the approach is to engineer ethically correct robots by giving them the capacity to reason over, rather than merely in, logical systems (where logical systems are used to formalize such things as ethical codes of conduct for warfighting robots). This is to be accomplished by taking seriously Piaget's position that sophisticated human thinking exceeds even abstract processes carried out in a logical system, and by exploiting category theory to render in rigorous form, suitable for mechanization, structure-preserving mappings that Bringsjord, an avowed Piagetian, sees to be central in rigorous and rational human ethical decision making.

We assume our readers to be at least somewhat familiar with elementary classical logic, but we review basic category theory and categorical treatment of deductive systems. Introductory coverage of the former subject can be found in Barwise and Etchemendy [1] and Ebbinghaus, Flum, and Thomas [2]; deeper coverage of the latter, offered from a suitably computational perspective, is provided in Barr and Wells [3]. Additional references are of course provided in the course of this paper.

Preliminaries

A category consists of a collection of objects and a collection of arrows, or morphisms. Associated with each arrow f are a domain (or source), denoted dom f, and a codomain (or target), denoted cod f.

Several of the authors in this part raise doubts about whether machines are capable of making ethical decisions, which would seem to thwart the entire project of attempting to create ethical machines. Drew McDermott, for instance, in “What Matters to a Machine?” characterizes ethical dilemmas in such a way that it would seem that machines are incapable of experiencing them, thus making them incapable of acting in an ethical manner. He takes as the paradigm of an ethical dilemma a situation of moral temptation in which one knows what the morally correct action is, but one's self-interest (or the interest of someone one cares about) inclines one to do something else. He claims that “the idiosyncratic architecture of the human brain is responsible for our ethical dilemmas and our regrets about the decisions we make,” and this is virtually impossible to automate. As a result, he thinks it extremely unlikely that we could create machines that are complex enough to act morally or immorally.

Critics will maintain that McDermott has defined “ethical dilemma” in a way that few ethicists would accept. (See S. L. Anderson's article in this part.) Typically, an ethical dilemma is thought of as a situation where several courses of action are possible and one is not sure which of them is correct, rather than a situation where one knows which is the correct action, but one doesn't want to do it.

To some, the question of whether legal rights should, or even can, be given to machines is absurd on its face. How, they ask, can pieces of metal, silicon, and plastic have any attributes that would allow society to assign it any rights at all.

Given the rapidity with which researchers in the field of artificial intelligence are moving and, in particular, the efforts to build machines with humanoid features and traits (Ishiguro 2006), I suggest that the possibility of some form of machine consciousness making a claim to a certain class of rights is one that should be discounted only with great caution. However, before accepting any arguments in favor of extending rights to machines we first need to understand the theoretical underpinnings of the thing we call law so that we can begin to evaluate any such attempts or claims from a principled stance. Without this basic set of parameters from which we can work, the debate becomes meaningless.

It is my purpose here to set forth some of the fundamental concepts concerning the law and how it has developed in a way that could inform the development of the machines themselves as well as the way they are accepted or rejected by society (Minato 2004). In a very real sense, as we will see, the framing of the debate could provide cautionary guidance to developers who may make claims for their inventions that would elicit calls for a determination concerning the legal rights of that entity.