Algorithmic Decision-Making

An algorithm is “a procedure for solving a mathematical problem . . . in a finite number of steps that frequently involves repetition of an operation; broadly: a step-by-step procedure for solving a problem or accomplishing some end especially by a computer.”Footnote ¹⁷ As such, algorithms are nothing new: they are omnipresent in mathematics, but also in guidebooks for tasks such as cooking a meal. Past experiences about how to do these things are distilled into step-by-step instructions that others can follow. My focus, however, is on algorithms that are implemented in software.

What differentiates such algorithms is that they need to be formulated in a language that computers can understand. Natural language needs to be translated into formal language.Footnote ¹⁸ Another crucial difference is that with increasing computing power, computers can go through a large number of steps very quickly, and process amounts of data that human beings could never process (“big data”). Algorithms can be very simple or extremely complex; up to a certain point, however, it is possible to understand, step by step, what they do and how different inputs influence the outputs.Footnote ¹⁹

This changes when elements of machine learning are involved. What they have in common is that the steps of the algorithms are not all defined beforehand; rather, the algorithms can find their own solutions for the given problems. They proceed by trial and error and are “taught” which strategies are promising, by receiving feedback on whether they have provided correct solutions. For example, when an image recognition program is “taught” to recognize horses in pictures, the programmers do not know what exactly the program does; they can only observe whether the success rate increases. This is what is usually meant when an algorithmic system is called an “artificial intelligence.”Footnote ²⁰ In such cases it can be become very difficult, even for the programmers themselves, to comprehend how the algorithms arrive at their results. The algorithms manage, for example, to categorize an email as “spam” or “not-spam,” but they do so in ways that are not following the same logic as human intuition. For example, they classify frequencies of words in texts (the most frequent ones being “our,” “click,” “remov,” “guarantee,” and “visit”), without any attention to the meaning of messages, and yet arriving at a relatively high percentage of correct decisions.Footnote ²¹

A major challenge for the application of algorithms, and in particular machine learning, is the quality of data. There can be various problems, for example, differential error rates in different categories (such as between men and women who change their names when getting married), which can lead to distortions.Footnote ²² And depending on how algorithms are used, the feedback they receive might be distorted. For example, if a program is used for sorting applicants into those who get a job interview and those whose applications are immediately rejected, there is no possibility of getting feedback about the rejected candidates.Footnote ²³ This is why, in order to arrive at meaningful evaluations, one has to consider algorithmic decision-making in the social contexts in which it is used: it is the combination of various social factors and the programs that leads to certain outcomes.Footnote ²⁴

Plurality, Natality, Judgment

To ground the contrast with algorithmic decision-making, in this section I briefly review three core elements of Arendt's political throught: the notions of plurality, natality, and judgment. Plurality, for Arendt, is an ontological condition of human life and a key feature of the political realm. In her distinction between labour, work, and action in The Human Condition, Arendt holds that “Human plurality, the basic condition of both action and speech, has the twofold character of equality and distinction.”Footnote ²⁵ In the public realm of politics, individuals become visible in their individuality, which is always more than the “what” of their different features. They answer the question “Who are you?” by action and speech.Footnote ²⁶ This openness and mutual visibility of the public realm stand in stark contrast to the darkness of the private realm in which labor and work take place. Hence, Arendt's rejection of a model of politics that exclusively follows the model of “work” or “production”Footnote ²⁷ also implies that politics requires that individuals see and hear each other.

The existence of such a realm is important for Arendt not least because it is the space in which natality, in its political form, has its place. Philosophers have emphasized throughout the centuries that humans are mortal. Arendt emphasizes that they are also “natal”: with each human being who is born, something new comes into the world, and human beings have the power to make new beginnings. “The miracle that saves the world, the realm of human affairs, from its normal, ‘natural’ ruin is ultimately the fact of natality. . . . It is . . . the birth of new men and the new beginning, the action they are capable of by virtue of being born.”Footnote ²⁸

Natality implies an openness towards the future that stands in sharp contrast to the ever-sameness of biological processes. Human beings can build new worlds; together, they can take their fate into their own hands. Beginning, “the supreme capacity of man,” is crucial for understanding what it means to act in the shared human world—and it is “identical with man's freedom.”Footnote ²⁹ She uses the metaphor of a second birth: “With word and deed we insert ourselves into the human world, and this insertion is like a second birth, in which we confirm and take upon ourselves the naked fact of our original physical appearance.”Footnote ³⁰

Lastly, there is Arendt's “elusive”Footnote ³¹ notion of judgment, the capacity made possible by the ontological conditions of plurality and natality, which has led to some controversy among commentators. Of interest in the present context is Arendt's emphasis on the social nature of judging, even when done by a single individual: in judging, one evaluates new phenomena against a background of comparable phenomena. It takes into account how other human beings, from their specific perspectives, see these phenomena, in an appeal to “common sense” (sensus communis). In her discussion of “Socratic dialogue” in Thinking and Moral Considerations, Arendt describes the internal dialogue, the “two in one,” that human beings hold when they reflect about their actions. This inner dialogue does not provide us with positive prescriptions what to do.Footnote ³² It tells us what not to do and undermines “our fixed habits of thought and the accepted rules of conduct.”Footnote ³³

This creates the space for judgment in the proper sense: a kind of judging that does justice to particulars and yet arrives at intersubjectively valid results. Individuals need to adopt what Arendt, drawing on Kant's Third Critique, calls an “enlarged mentality”: one that stands in a dialogue with how other individuals would describe a specific phenomenon. For her, “without ‘the test of free and open examination,’ no thinking and no opinionformation are possible.”Footnote ³⁴ When we judge a particular thing (the beauty of a flower, the virtue of an individual, the greatness of a historical moment, etc.), we imagine the perspectives of others and the sensus communis about it, which allows us to transcend our own limited perspective.

This is what connects judgment—which might, from what has been said so far, be understood purely as a moral practice—to the political realm. Judgment is, for Arendt, “one of the fundamental abilities of man as a political being insofar as it enables him to orient himself in the public realm, in the common world.”Footnote ³⁵ As such, judgment is a “democratic world-building practice”:Footnote ³⁶ it creates the shared world in which human beings can live together. Some commentators have argued that Arendt seems to shift from a notion of judgment of actors to one of spectators in The Life of the Mind.Footnote ³⁷ Spectators are better able to take on a plurality of perspectives, while actors, in the heat of the moment, might lose the detachment that is required to imagine the perspectives of others.Footnote ³⁸ Moreover, actors only rarely find themselves in the egalitarian but contestatory scenario that Arendt's understanding of judgment presupposes.

Nonetheless, the difference between actors and spectators should not be understood as a categorical one.Footnote ³⁹ After all, human beings often take on both roles: even those who are busy actors at day can think about their actions at night. Those who seem to be mere spectators can decide to become active when they feel that they can add a specific perspective. Moreover, there is always the possibility of a dialogue between actors and spectators. In fact, Arendt herself mentions the possibility that “actor and spectator become united” in moral judgment.Footnote ⁴⁰ Similarly, spectatorship and action can go hand in hand in what I call “Arendtian practices”: human beings can both observe and act, emphasizing the one or the other depending on the occasion. This notion of Arendtian practices provides my point of reference for constrasting human and algorithmic decision-making.

Arendtian Practices beyond Politics

Plurality, natality, and judgment are related and complement each other in fleshing out a specific vision of how human beings “do politics.” For Arendt, they have their primary place in the political sphere, which she famously contrasts with the private sphere of labor and work. But clearly, her notion of politics cannot be equated with the institution and processes of representative democracies. Arendt's frequent references to the Greek polis might be thought to suggest she was hopelessly nostalgic, while her discussion of the French and American RevolutionsFootnote ⁴¹ might create the impression that politics in her emphatic sense is only, if ever, possible in extraordinary circumstances.

One might therefore be tempted to dismiss her vision of politics as an ideal type that is hardly ever instantiated in real life. Given Arendt's many pessimistic remarks about her own period, one could indeed wonder whether plurality, natality, and judgment can ever be realized in modernity. The focus on labor and the rise of “the social,” or what Arendt calls the “victory of the animal laborans,”Footnote ⁴² threaten to submerge the realm in which action and speech would be possible.Footnote ⁴³ With the rise of consumerism, and the organization of work along Fordist lines, society is understood—and organized!—as a realm of predictable processes that stifle individuality and the creative search for new beginnings.Footnote ⁴⁴ Politics degenerates into administration, and where it leaves space for something akin to action, it does so only for a handful of privileged individuals.

There is another possible way of reading Arendt, however, which clings to her vision of politics in a way that might be more optimistic than she herself was. Such a reading can follow Seyla Benhabib in seeing judgment—and with it, ultimately, also the conditions of plurality and natality—as not belonging exclusively to the political realm, but rather as concerning the possibility of moral judgment in general.Footnote ⁴⁵ It can also include the dimension of “world building” inherent in judgment that Linda Zerilli emphasizes,Footnote ⁴⁶ which has a more political dimension, but is, arguably, also not limited to “political” arenas in the traditional sense.

This reading holds that opportunities for judgment and action can arise in many social spheres (and questions the way in which Arendt is sometimes read in a “territorial” of separate social spheres, as Patchen Markell puts it).Footnote ⁴⁷ Human beings are spectators and actors, not only in politics, but also in many other areas of life in which they encounter others: as colleagues at work, as members of neighborhood associations or NGOs, as parents of children who go to the same schools, and so forth. While such areas are often not organized in a formally democratic way, there is at least the potential of encountering others as equals, without a presumption of hierarchy. And often, in order to arrive at judgments in these spheres, we do imagine the perspectives of others, in ways that resemble the “enlarged mentality” that Arendt describes. This also means that the possibility of beginning something new is not limited to one social sphere; what matters for this to happen is “an agent's attunement to its character as an irrevocable event, and therefore as a new point of departure,” as Markell puts it.Footnote ⁴⁸

Of course, the fact that individuals reflect and form judgments about questions in various spheres of life does not, by itself, mean that there are also social spaces for real exchanges of opinions, or “common worlds.” Many decisions are not at all taken in a way that resembles Arendt's concepts of judgment or action. Often, unequal power relations make this hard or impossible. Moreover, if encounters follow the logic of what Arendt calls “the social,” standardized behavior is expected, with no room for deviation from the preestablished patterns. This holds in particular for the realm of capitalist wage labor.

And yet, time and again, forums for open discussions on an equal footing do emerge, at least in societies that preserve the right political conditions, such as freedom of speech and freedom of association. It can then be possible to encounter others along the lines described by Arendt: presenting our opinions to one another, trying to understand each other's position, and coming to shared judgments. As Keith Breen notes, “Arendtian freedom concerns the vocal ability to engage and participate with others in matters of common significance whose status is uncertain and thus amenable to debate.”Footnote ⁴⁹ Encountering each other as equals and exchanging perspectives, so that something new can emerge, is what I see as the core of “Arendtian practices.”

In a discussion about Arendt and Habermas, Craig Calhoun emphasizes that Arendt does not “tie her idea of public space to the state in the way Habermas does his notion of public sphere.” This allows for the possibility that “the occasions of public action may be multiple, each involving different mixes of people.”Footnote ⁵⁰ Such freedom can appear in various spaces, not only in the political realm but also, for example, when patient groups push for change in the health system.

“Arendtian practices” are valuable both in themselves, as a core element of human sociability, and in their role in shaping the habits and practices of democratic societies. Arguably, no democratic society can flourish, and remain democratic over extended periods of time, without spaces in which citizens can have this experience. The joint “management of collective affairs” can shape the habits and develop the skills that citizens need for democracies to flourish.Footnote ⁵¹ The rulers of nondemocratic societies, in contrast, often try to suppress such practices, out of (probably well-founded) fears that individuals could get a taste of what it means to act together.

One contested question is whether, for Arendt, such practices could also arise in the world of work (in a non-Arendtian sense)—which is, after all, where many algorithms are currently being used. Arendt famously distinguishes between “labor,” or the cyclical, repetitive activities tied to biological necessities, and “work,” or the creation of the material world around us. In some places, she seems to reject the realms of “labor” and “work” as potential public spaces.Footnote ⁵² Some have criticized her “bifurcation of freedom and necessity and equation of necessity with labour and nature, an equation that results in labour and, by association, the economic being erroneously denied a uniquely human status.”Footnote ⁵³ Arendt does indeed describe animal laborans as incapable of “action and speech”;Footnote ⁵⁴ the same seems to hold for homo faber, where speech serves merely for the giving of orders for instrumental purposes.

But human beings are always more than animal laborans, even when they enter their workplaces. What Arendt has in mind when talking about the modern workplace seem to be Fordist and Tayloristic factories. But many workplaces are different, and even Fordist factories have canteens and changing rooms and permit union meetings and common festivities.Footnote ⁵⁵ And while individuals may have work together as homogeneous equals, as Arendt notes,Footnote ⁵⁶ such spaces also offer opportunities for meeting one another as distinct individuals and deliberating in a noninstrumental way.

Some readers have indeed rejected a reading of Arendt that sees the different spheres she describes as completely separate.Footnote ⁵⁷ Klein argues that what Arendt feared was not so much the replacement of politics by economics as the disappearance of certain institutions that make it possible for economic issues to be debated and negotiated in the ways that she describes by drawing on the notions of judgment and action.Footnote ⁵⁸ Arendt can thus indeed be read as holding that “all spheres of life, including labor and work, can be informed by an appreciation of the human need to be recognized and known through engaging in meaningful, nondeterministic, personally creative, and ultimately ‘public’ endeavors.”Footnote ⁵⁹

Numerous studies in economic sociology confirm that the account of work as a soulless “system” driven by instrumental reason is too simplistic. Isabelle Ferreras, in an ethnographic study of supermarket cashiers, has underlined the degree to which work has an expressive, noninstrumental dimension.Footnote ⁶⁰ Lyn Spillman studied the role of business associations as social spaces in which complex intersubjective processes of meaning making take place, and in which sociability and collegiality can develop.Footnote ⁶¹ When Ronald Beiner holds that “the real danger in contemporary societies is that the bureaucratic, technocratic, and depoliticized structures of modern life encourage indifference and increasingly render men less discriminating, less capable of critical thinking, and less inclined to assume responsibility,”Footnote ⁶² it seems that one has to take into account all spheres in which judgment and joint action are, at least potentially, possible. These spaces matter both individually—because each one is valuable for the human individuals who take part in it; and in sum—because the fewer such spaces exist, the more the democratic ethos of a society is threatened.

However, one important reason why such spaces could continue to exist, even within the hierarchies of workplaces, was what organization research has called the “problem of control”: controlling workers who execute complex tasks is difficult and costly, hence there is hardly ever full control.Footnote ⁶³ Often, one needs to give individuals and groups a certain degree of autonomy in order for them to organize themselves and to react to the specificities of new cases or new localities; sometimes, this can even lead to a rejection of the given instrumental logic and to the formulation of new goals. Arguably, the bureaucratic and technocratic tendencies that Arendt deplored in her own time still left open quite some space in which individuals could encounter each other on an equal footing, as distinct individuals, exchange their opinions, and form judgments. For example, when colleagues discuss their work, they can arrive at shared judgments about how to reorient that work. In other words, they can not only reflect, instrumentally, about how to achieve certain goals, but also reflect about new goals.Footnote ⁶⁴ At least this holds in workplaces that are not completely algorithmically governed, such as certain warehouses in which workers’ activities are entirely determined by software.

With the advent of algorithms, many such spaces that offer the potential for creating Arendtian pratices might disappear, and individuals might be conditioned, even more than is already the case, towards instrumentalist instead of deliberative, or hierarchical instead of egalitarian, modes of encounter.Footnote ⁶⁵ In the next section, I show that algorithmic decision-making is fundamentally different from such practices, with regard to the three dimensions of plurality, natality, and judgment.

An Arendtian Critique of Algorithmic Decision-Making

When human beings encounter each other in Arendtian practices, their distinctness as individuals and their equal status shape their interaction—as captured in Arendt's notion of plurality. Take, for example, a number of book lovers who run into each other in a book shop, engage in a discussion about a recent bestseller, start recommending other novels to each other, and finally decide to start a book club. They might convince individuals to start reading something that is completely different from their previous reading habits. This process is rather different from the way in which algorithms provide buying recommendations for books. Here, individuals appear as statistical constructs, as “whats” not “whos”: sums of data points that may include things like age, sex, location, and in particular previous purchases. The algorithms compare these data points to others, categorizing, comparing, and aligning them and then spitting out recommendations.Footnote ⁶⁶

It might be objected that the more data points are available, the more the algorithmic programs can tailor their recommendations to each specific individual. But this is, emphatically, not the same as the kind of individuality that matters for Arendtian practices. It is, rather, a more fine-grained cross-sectional analysis of data in which individuals continue to be treated as members of specific categories—and only of those categories that are captured by the algorithms. It can hardly capture the true quality of social relations and the meaning that a certain book can have for an individual who recommends it to someone else. Or take the example of job applications—here, algorithms will often look for similarities with existing employees, but they can hardly capture the more nuanced questions about how candidates would fit into the social relations of existing teams.

Another element that is missing in the algorithmic decision is the openness, the space of mutual visibility, that is constitutive of human plurality. Users may or may not know what inputs go into a system, and they often also do not know what exactly the algorithmic system is supposed to produce. For example, applicants may have no reliable information about what a company considers a “good employee” and has, hence, encoded in its hiring programs. Indeed, the company may carefully guard the list of criteria, since if it were to become public knowledge, individuals might start to game the system by tweaking the information they share. To be sure, lack of transparency and strategic behavior often mar human hiring processes as well—but in these, there is at least the possibility of challenging decisions and demanding an explanation, in a frank encounter, which cannot be done if an algorithm made a decision and even the company itself maybe does not quite understand what it is doing.

Algorithmic decision-making is also diametrically opposed to what Arendt describes as natality. Instead, it is tied to the past. The potential here, as described earlier, is the possibility of encoding past knowledge so that current users do not have to repeat the same errors over and over again. But the danger is to thereby tie the present to the past in a way that makes newness impossible. Catherine O'Neill puts this point strongly: “Big Data processes”—including the algorithms used to analyse the data sets—“codify the past. They do not invent the future.”Footnote ⁶⁷ Algorithms work with past data (which in some cases reach up to the present but might also be much older) and they are based on the assumption that the same correlations that held in the past will also hold in the future. For example, if applicants with a certain university degree were a good fit for the job in the past, it is assumed that this will also be true for the next round of applicants. An organization that uses algorithmic systems “must assume that its future applicant pool will have the same degree of variance as its current employee base.”Footnote ⁶⁸ This is also why structural biases, e.g., discrimination against minorities, are reproduced by algorithms.Footnote ⁶⁹

The way in which Arendt describes judgment is also very different from the way algorithms make decisions. One difference has to do with the lack of transparency mentioned above. When humans are confronted with an algorithm, they often do not know how the outcomes are produced. In contrast, in processes of judgment—especially in ones that are truly intersubjective, not just imagined—we can see how the process of mutual adjustment of perspectives comes about. In “The Crisis in Culture,” Arendt mentions the notion of “wooing” others, which appears in similar ways in Kant and in Greek political philosophy.Footnote ⁷⁰ One of its features is that it is an open, inviting mode of trying to convince others, which excludes, for example, manipulative rhetorical tricks. In the Lectures on Kant, Arendt emphasizes the importance of “holding oneself and everyone else responsible and answerable for what he thought and taught.”Footnote ⁷¹ This is, of course, an idealization. But the way in which algorithmic decision-making works is at the very opposite end of the spectrum. This can have to do with business secrecy, lack of knowledge about coding, and the intrinsic intransparance of many algorithmic systems,Footnote ⁷² but also with the ways in which algorithms are embedded in social hierarchies in which those who are subject to the algorithms’ decisions are often not able to hold those who use them accountable.Footnote ⁷³

One might object, here, that there can be forms of “newness” brought about by algorithms. There are examples of AI generating “art,” for instance, composing music. But these follow patterns that were already established and recombine them, usually producing mediocre versions of more of the same.Footnote ⁷⁴ And natality cannot be reduced to creativity; it also has an intersubjective dimension and includes the will to do something new. Take, for example, IBM's Watson's recipe creation function, which recombines flavors, sometimes creating “new” recipes.Footnote ⁷⁵ But what it cannot do is to completely switch categories—for example, to realize that the friends you have invited over are not in the mood for trying out new recipes, and that it would be better to order pizza and talk through their relationship problems—which requires judgment.

Thus, there are important limits on the “newness” that algorithms can create. They also have to do with one important difference between human and artificial intelligence, which is that the latter is domain specific.Footnote ⁷⁶ Hence, it cannot recognize the need for new decisions that cut across domains. It needs to be taught beforehand what matters for its task (and it may well lead to a self-fulfilling prophecy in which these features come to matter most). For many decision-making situations, however, “what matters” cannot be determined beforehand, but needs to be found out in the process. Algorithms might be able to find new means but cannot find new ends.

It seems no exaggeration to say that where algorithms make decisions, there is no “common world” in the Arendtian sense, in which individuals jointly interpret the reality they encounter, exchange opinions, and come to judgments.Footnote ⁷⁷ When one encounters an algorithmic decision-making system, it does not make sense to try to convince it—even less so than in the case of a rule-bound bureaucrat, who exemplifies the “rule by nobody” that Arendt warned against.Footnote ⁷⁸ If one wants to bring about a different outcome, the only thing to do is try to game the system, by second-guessing which parameters one has to insert in order to get the results one wants. The mindset is a purely strategic one, and while such maneuvers also happen between human beings, algorithms do not leave any other choice but the manipulative road. What is lost when we are faced with algorithms is the constitutively shared nature of our human reality that we are all responsible for interpreting together.Footnote ⁷⁹ Beiner captures well what is at stake here: “judgment has the function of anchoring man in a world that would otherwise be without meaning and existential reality: a world unjudged would have no human import for us.”Footnote ⁸⁰

Another feature of human agency that gets lost in algorithmic decision-making are the various checks on the unpredictability of action. Of particular relevance here are Arendt's reflections on the role of promising and forgiveness, and the dangers of acting upon the realm of nature—e.g., through nuclear technology—where the possibility of “undoing” actions does not exist in the same way.Footnote ⁸¹ Arendt discusses the role of modern science at quite some length in The Human Condition.Footnote ⁸² Some aspects of her concerns also apply to algorithms, notably the rise of instrumental thinking and the fact that scientists take decisions that affect societies as a whole.Footnote ⁸³ As she writes about “the action of the scientists”: “since it acts into nature from the standpoint of the universe and not into the web of human relationships, [it] lacks the revelatory character of action as well as the ability to produce stories and become historical, which together form the very source from which meaningfulness springs into and illuminates human existence.”Footnote ⁸⁴ Similarly, algorithms “going wild” can have the same unpredictability as human action, but without the corrective mechanism provided by promising and forgiveness.Footnote ⁸⁵

Thus, some spaces for judgment need to be maintained, from an Arendtian perspective—but are there reasons to fear that algorithms would make this impossible or even just less likely? After all, not all social spheres need to be, or indeed could be, governed by what I have called “Arendtian practice.” And if algorithms could contribute to liberating individuals from labor or work, they might indeed help create more space, or, more concretely, more time, for Arendtian practices.Footnote ⁸⁶ So, do we not run the risk of overlooking the emancipatory potentials of algorithms?

This is a valid point, but it misses the thrust of my argument. My point is not to reject algorithmic decision-making; there are many places where algorithms can be very useful tools. Instead, I want to warn against them crowding out, often for the sake of alleged cost reductions,Footnote ⁸⁷ spaces for Arendtian practices, which have specific, irreplaceable value. This concern, which is in the spirit of Arendt's critique of bureaucracy, economism, scientism, and other deterministic approaches, takes two forms.

The first is the overall space for Arendtian practices in society; this point is structurally similar to her admonition of the rise of the “social” that crowds out spaces for political action. We might see a phenomenon of “algorithm creep”: of algorithms slowly taking over far more space than they should.Footnote ⁸⁸ Such a scenario could come about because each single choice to introduce an algorithmic decision-making system, taken by itself, may seem harmless—and there may be a coordination problem among those who decide about the introduction of algorithmic decision-making systems. The summative effect might be considerable, and it might be far harder to return to nonalgorithmic decisions once the programs have been introduced. One factor to take into account in this context is what psychologists call “automation bias”: humans tend to trust automated systems more than may be appropriate.Footnote ⁸⁹ When Joseph Weizenbaum developed an early form of artificial intelligence, a program called ELIZA that could mimic a therapist, he was shocked to see the extent to which humans shared intimate details with it.Footnote ⁹⁰ Even programs that are meant to be merely auxiliary tools might be given far too much power, with humans giving up their own responsibility and simply going with “what the computer said.”

The second reason for concern is that algorithms might be applied to questions for which they are simply not suitable. Here, Arendt's notions of plurality, natality, and judgment can in fact help develop criteria for thinking through various cases. These may not be the only relevant criteria—for example, in a situation of emergency there may be reasons to prioritize efficiency and speed in order to satisfy urgent needs—but they provide pro tanto reasons.

With regard to plurality, one key question is whether it is appropriate to treat individuals according to predefined categories, or whether their distinctness needs to be taken into account. In a discussion of “technological due process,” Danielle Citron draws a basic distinction between rules, standards, and combinations of both: “A rule prescribes ex ante an outcome for a particular fact scenario. . . . On the other hand, a standard requires decision makers to exercise discretion, applying ex post policies to events.”Footnote ⁹¹ While rules have the advantage of predictability, standards permit decision makers to tailor an outcome to the facts, increasing the likelihood of an “ideal” ruling.Footnote ⁹² Algorithms, however, are rule based—therefore, “decisions best addressed with standards should not be automated.”Footnote ⁹³ While this is certainly true, it may be questioned whether it is sufficient as a guideline. One important question is that of alternatives: Will those who make discretionary decisions really make them according to the requirements of the situation, or is it likely that considerations that should be irrelevant (e.g., some form of private gain) will intrude into the decision-making process? Importantly, the notion of plurality, in conjunction with the notion of judgment, implies that the most adequate alternative may often not be decisions by single individuals with their own idiosyncracies and biases. Rather, an appropriate way of judging particulars may be to bring together a group of diverse people who can discuss face to face, on an equal footing, and try to achieve an “enlarged mentality” together.

With regard to natality, an important question is whether it is appropriate to “go on” with existing patterns, or whether there might be a need to take a new tack. Hence, at a minimum, the desirability of the existing patterns needs to be scrutinized: Are they such that we can, with a clear conscience, continue with them? Many data sets reflect past injustices, such that algorithms come to be biased against women or ethnic minorities;Footnote ⁹⁴ such phenomena could, however, be corrected if the data were prepared, and the algorithms trained accordingly. But in other cases, completely new approaches might be needed—think, for example, about the shift in the treatment of drug addicts that turned away from seeing it as a mere medical problem, and started to take the social contexts into account. Such a shift could not have been initiated by an algorithmic program; it required human agency and political struggles.Footnote ⁹⁵

Relatedly, there is the question of judgment. Judgment is not needed, or if it is, only minimally, when one can proceed with given ends in mind (or rather, in the code), in one single domain, and when all that matters is finding the best strategy for getting there. But one may well ask how many decisions in the realm of human affairs are like this. Earlier, I quoted Breen's formula about “matters of common significance whose status is uncertain and thus amenable to debate.”Footnote ⁹⁶ In such situations, the ends are not predefined, and there are no clear-cut, uncontroversial standards that one could apply for describing outcomes as successes or failures.

In contrast, algorithmic decision-making is ideal for situations in which the status is certain, there is no need for debating the goals, and the criteria of success are clear-cut. This is why computers and artificial intelligence are extremely good at games: there, the “set of rules” is “complete and consistent.”Footnote ⁹⁷ But many social situations do not have the character of games. Think, for example, about many challenges of social work, where the aim is not—or should not be—to simply tick boxes and achieve a certain outcome, but rather to pay attention to the specific situation of each individual and to reflect, together with them, what the goal of the intervention is actually supposed to be.

In stark contrast to games, many decisions that humans have to take are multidimensional and we cannot expect all parameters to fit into predefined categories—we may not even know yet which categories will be relevant for understanding them. In other words, we do not yet have general categories under which to subsume the particulars, but rather need to find the right—new!—categories for grasping what matters about the particulars; and here, judgment is needed. Again, social work is a case in point: there might be biographical issues that do not fall under the predefined categories of a questionnaire. The danger, obviously, is to resort too quickly to algorithms, thereby closing down the avenue for judgment, and for a deeper understanding of the problems. In such cases, we try to fit particulars into general categories that are not right for them, and maybe even impose on them inadequate value jugments that are built into the algorithms.Footnote ⁹⁸

Seeing algorithms as ready solutions for various economic, organizational, and civic tasks requires understanding these tasks as mere problems of administrative weighing and prioritization—but they might, in fact, arise out of deep, and maybe even tragic, value conflicts. Weizenbaum, one of the early pioneers of AI research, warned against such a reductionism of rationality to “logicality” early on, mentioning Arendt's skepticism towards formalistic analyses of politics.Footnote ⁹⁹ The technocratic, model-based approach to conflict that appeared in the “rational choice” view of the Cold War, and in technocratic approaches to conflicts in general, amounts to a denial of “the very possibility of the collision of genuinely incommensurable human interests and of disparate human values, hence the existence of human values themselves.”Footnote ¹⁰⁰

Here, an interesting connection can be drawn to Arendt's essay “Truth and Politics.” Arendt contrasts the “element of coercion” in truth with the “representative thinking” of citizens in which the opinions of others are taken up. It would be a misunderstanding to read this essay as holding that “truths”—or, to use a less highly charged, but by now also controversial, term, “well-established facts”—are irrelevant for politics; Germany did invade Belgium and not the other way round.Footnote ¹⁰¹ Knowledge about facts that are so clearly established that they require no discussion is indeed the kind of knowledge that could be encoded into algorithms: for example, political agents can and should use pocket calculators whose software encodes certain mathematical laws. The problem, however, is that few facts are as clear, and as clearly recognizable, as mathematical laws; even “scientific” facts are discovered in ongoing discussions and interpretations of the evidence by scientists.Footnote ¹⁰² Their significance and relevance in evaluating situations and possibilities must also be a matter of judgment. This means that the boundary between the realms of facts and opinions is notoriously contested—and hence the question of which facts could be embedded in algorithmic programs and which ones need to remain debatable in the realm of opinions is equally contested.

In summary, what is needed is a conscious, responsible delineation of the kinds of decisions that we can hand over to algorithms with a clear conscience, the ones where algorithms and humans can collaborate in productive ways, and the ones where it would be irresponsible to let algorithms take over, because the potential for “Arendtian practices,” and the need for attention to the dimensions of plurality, natality, and judgments, would be lost. When decisions about the introduction of algorithms are made in practice, however, this often happens against a background in which there is already too little space for Arendtian practices, for example because the logic of “the social” had already been given far too much space. In that sense, the problem is not new, but with new technological possibilities of algorithmic decision-making it gains new urgency.

These questions lead back to the issues of justice I alluded to in the introduction. In addition to the problem of reinforcing old, or creating new, injustices through algorithmic decision-making, one crucial question is for whom algorithms are used. Given the way in which algorithmic systems blend into existing hierarchies of power and status, we should hardly expect these systems to take away decision space in the executive suites and other realms in which the rich and the powerful meet. It is much more likely that such spaces will shrink for the middle and lower ranks of workplace hierarchies, and for those who receive services, e.g., medical services, from overburdened and understaffed welfare-state institutions. This is another sense in which many algorithms are unlikely to bring about anything new, but might rather reinforce existing injustices.Footnote ¹⁰³

Think, for example, about a team of street-level bureaucrats whose daily meeting in which they distribute tasks and discuss cases is replaced by an algorithmic allocation system; this takes away an opportunity in which they could not only discuss instrumental questions, but also deliberate together about goals that need to be rethought, or new starts that need to be made. Not even the very few moments in which workers, patients, or the recipients of public services could encounter one another as individuals, exchanging opinions and forming judgments together, might be preserved if their work is constantly algorithmically allocated, tracked, and controlled. An experience that is, arguably, crucial for maintaining a democratic ethos would then become a privilege for the few.