It is fitting that the last example we introduced in the book was about the Internet Research Agency’s (IRA) use of social media, analytics, and recommendation systems to wage disinformation campaigns and sow anger and social discord on the ground. At first glance, it seems odd to think of that as primarily an issue of technology. Disinformation campaigns are ancient, after all; the IRA’s tactics are old wine in new boxes. That, however, is the point. What matters most is not particular features of technologies. Rather, it is how a range of technologies affect things of value in overlapping ways. The core thesis of our book is that understanding the moral salience of algorithmic decision systems requires understanding how such systems relate to an important value, viz., persons’ autonomy. Hence, the primary through line of the book is the value itself, and we have organized it to emphasize distinct facets of autonomy and used algorithmic systems as case studies.

A little after 2 a.m. on February 11, 2013, Michael Vang sat in a stolen car and fired a shotgun twice into a house in La Crosse, Wisconsin. Shortly afterward, Vang and Eric Loomis crashed the car into a snowbank and fled on foot. They were soon caught, and police recovered spent shell casings, live ammunition, and the shotgun from the stolen and abandoned car. Vang pleaded no contest to operating a motor vehicle without the owner’s consent, attempting to flee or elude a traffic officer, and possession of methamphetamine. He was sentenced to ten years in prison.

This chapter connects our arguments about agency and autonomy in chapters 2-4 to conceptions of freedom and its value. We argue that freedom has two fundamental conditions: that persons be undominated by others and that they have an adequate degree of autonomy and agency. We then explain that algorithmic systems can threaten both the domination-based and the agency-based requirements, either by facilitating domination or by exploiting weaknesses in human agency. We explicate these types of threats as three sorts of challenges to freedom. The first are “affective challenges,” which involve the role of affective, nonconscious processes (such as fear, anger, and addiction) in human behavior and decision-making. These processes, we argue, interfere with our procedural independence, thereby threatening persons’ freedom by undermining autonomy. The second are “deliberative challenges.” These involve strategic exploitation of the fact that human cognition and decision-making are limited. These challenges also relate to our procedural independence, but they do not so much interfere with it as they exploit its natural limits. A third sort of challenge, which we describe as “social challenges,” involve toxic social and relational environments. These threaten our substantive independence and thus, our freedom.

This chapter outlines the conception of autonomy that grounds the arguments throughout the book. We begin with a basic definition of autonomy as self-government, distinguish global and local autonomy, and explain how autonomy may be understood as a capacity, as the exercise of that capacity, as successful self-government, and as a right. We then describe a key split in the philosophical literature between psychological autonomy and personal autonomy. We offer an ecumenical view of autonomy that incorporates facets of both psychological and personal autonomy. Finally, we rehearse some key objections to traditional conceptions of autonomy, and explain how contemporary accounts address those criticisms.

In this chapter, we address some distinctively epistemic problems that algorithms pose in the context of social media and argue that in some cases that epistemic problems warrant paternalistic interventions. Our paternalistic proposal to these problems is compatible with respect for freedom and autonomy; in fact, we argue that freedom and autonomy demand some kinds of paternalistic interventions. The chapter proceeds as follows. First, we discuss an intervention that Facebook has run in hopes of demoting the spread of fake news on the site. We explain why the intervention is paternalistic and then, using the framework of this book, defend the intervention. We argue that while Facebook’s intervention is defensible, it is limited. It is an intervention that may pop some epistemic bubbles but will likely be powerless against echo chambers. We then discuss heavier-handed interventions that might be effective enough to dismantle some echo chambers, and we argue that at least some heavier-handed epistemically paternalistic interventions are permissible.

When agents insert technological systems into their decision-making processes, they can obscure moral responsibility for the results. This can give rise to a distinct moral wrong, which we call “agency laundering.” At root, agency laundering involves obfuscating one’s moral responsibility by enlisting a technology or process to take some action and letting it forestall others from demanding an account for bad outcomes that result. We argue that the concept of agency laundering helps in understanding important moral problems in a number of recent cases involving automated, or algorithmic, decision-systems. We apply our conception of agency laundering to a series of examples, including Facebook’s automated advertising suggestions, Uber’s driver interfaces, algorithmic evaluation of K-12 teachers, and risk assessment in criminal sentencing. We distinguish agency laundering from several other critiques of information technology, including the so-called “responsibility gap,” “bias laundering,” and masking.

This chapter addresses autonomy’s role in democratic governance. Political authority may be justifiable or not. Whether it is justified and how it can come to be justified is a question of political legitimacy, which is in turn a function of autonomy. We begin, in section 8.1, by describing two uses of technology: crime predicting technology used to drive policing practices and social media technology used to influence elections (including by Cambridge Analytica and by the Internet Research Agency). In section 8.2 we consider several views of legitimacy and argue for a hybrid version of normative legitimacy based on one recently offered by Fabienne Peter. In section 8.3 we explain that the connection between political legitimacy and autonomy is that legitimacy is grounded in legitimating processes, which are in turn based on autonomy. Algorithmic systems—among them PredPol and the Cambridge Analytica-Facebook-Internet Research Agency amalgam—can hinder that legitimation process and conflict with democratic legitimacy, as we argue in section 8.4. We conclude by returning to several cases that serve as through-lines to the book: Loomis, Wagner, and Houston Schools.

One important criticism of algorithmic systems is that they lack transparency, either because they are complex, protected by intellectual property, or deliberately obscure. There is a debate about whether the EU’s General Data Protection Regulation (GDPR) contains a “right to explanation” This chapter addresses the informational component of algorithmic systems. We argue that information access is integral for respecting autonomy, and transparency policies should be tailored to advance autonomy. We distinguish two facets of agency (i.e., capacity to act). The first is “practical agency,” or the ability to act effectively according to one’s values. The second is “cognitive agency,” which is the ability to exercise what Pamela Hieronymi calls “evaluative control”. We argue that respecting autonomy requires providing persons sufficient information to exercise evaluative control and properly interpret the world and one’s place in it. We draw this distinction out by considering algorithmic systems used in background checks, and we apply the view to key cases involving risk assessment in criminal justice decisions and K-12 teacher evaluation.

Chapter 3 takes the conception of autonomy outlined in chapter 2 and explains how it grounds moral evaluation of algorithmic systems. It begins by offering a view of what it takes to respect autonomy and to respect persons in virtue of their autonomy, drawing on a number of different normative moral theories. The argument starts with a description of a K-12 teacher evaluation program from Washington, DC. It then considers several puzzles about the case. Next, the chapter provides an account of respecting autonomy and what that means for individuals’ moral claims. It explains how that account can help us understand the DC case, and we will offer a general account of the moral requirements of algorithmic systems. Specifically, we offer the Reasonable Endorsement Test, according to which an action is morally permissible only if it would be allowed by principles that each person subject to it could reasonably endorse. The chapter applies that test to the Loomis, Houston Schools, and Wagner cases. Finally, the chapter explains why the book does not focus directly on “fairness.”