2.1 Judging from the Syntactical Stance: A Thought Experiment

Consider now a syntax monster who is to the syntactical stance what a Martian is to the physical stance (the Martian is a Laplacean omniscient physicist). Just as the Martian inhabits a physically deterministic universe governed by causal laws, the syntax monster inhabits a syntactically deterministic universe governed by statistical correlations. That is to say that by hypothesis, the Martian and the syntax monster can predict the decisions of every human judge of their respective universes word for word and with one hundred percent accuracy. Now suppose that in their respective universes, people decide to delegate the jobs of human judges to these creatures instead. Consider now how these creatures would go about their job as judges.

The physical stance relies on the constellation of physical particles and the causal forces among them to predict future physical events. Thus, the Martian judge predicts the stable arrangement of ink particles on the paper (or keyboard strikes, light projections on monitors, etc.). The syntactical stance similarly relies on the constellation of syntax in existing texts and their statistical relations in order to predict the constellation of syntax in future texts. And so, the syntax monster judge predicts the constellation of syntax to be printed on the paper, projected on monitors, etc. The Martian judge explains each judicial decision, be it their own or otherwise, in terms of the arrangement of physical forces and particles elsewhere in the world. The syntax monster judge explains them in terms of the arrangement of syntax elsewhere in the world.

Note that we can describe the behaviors of the Martian and the syntax monster from various stances, including from what Dennett calls the “intentional stance.”Footnote ¹⁶ Taking the intentional stance treats the entity whose behavior is to be predicted as a rational agent, ascribes to it a set of beliefs and desires (given its place in the world and its purpose) and, finally, predicts that this rational agent will act to further its desires in light of its beliefs.Footnote ¹⁷ But nothing of significance follows from this, because we cannot draw any conclusions about whether the entity in question actually is rational or whether it actually has beliefs and desires. As Dennett emphasizes, we can adopt the intentional stance to describe the behavior of a thermostat just as we describe the behavior of human beings. But this is no reason to think that either humans or thermostats literally have beliefs or desires. So too, the fact that the intentional stance can be adopted to describe the behaviors of the Martian and the syntax monster is no evidence about whether they have beliefs or desires.Footnote ¹⁸ I bring this up to emphasize that the reason we should conclude that the Martian and the syntax monster will judge from the physical and syntactical stances respectively is not the fact that I have chosen to explain their behaviors from those stances. The latter fact is no evidence for the former. Rather, I am assuming that this is how they go about their jobs per hypothesis and as part of the thought experiment’s setup. This, and only this, is how they will go about their jobs because, for the purposes of a thought experiment, I have supposed that this and no more is what they are capable of. What is of interest to the discussion here is only the stance that they adopt, and in fact have no choice but to adopt, when they go about predicting (and explaining) the textual output of each court decision. What varieties of stances we could adopt to describe each of their respective behaviors is wholly irrelevant.

2.2 Lack of Rationale in Decisions

Suppose that the relevant legal outcome in a judicial decision is that Tom should pay Chris ten shillings. A problem arises when we inquire into the basis of that outcome. Consider the Martian judge first. If in response to our inquiry, he could have explained his decision to us, he would have said something along the following lines: Tom should pay Chris ten shillings because such and such physical forces and particles were arranged in such and such a way at such and such point in time and so forth, such that a judgment to that effect would have been printed out of the court’s printer, had humans been judging. This sort of explanation, however, is completely unresponsive to our inquiry.

Similarly, the syntax monster could only explain its decision on the basis that such pattern of English characters formed such and such constellations in such and such regularity in such and such sources of text and so forth. That sort of explanation too is entirely unresponsive to our inquiry.

The reason such explanations prove unresponsive is that our inquiry is not about what causes or correlations underpin the outcome but about its normative basis. A human judge could of course also offer a causal or correlative explanation. Social scientists offer this kind of explanation all the time. For instance, not having had lunch seems to cause or correlate with judges handing harsher sentences.Footnote ¹⁹ But this sort of answer would also be entirely unresponsive to our inquiry. What is more, if this sort of answer was all that a human judge offered, we would find it reasonable for Tom to leave with a chuckle and impunity. That is because, like a human judge, we understand that in this context, our inquiry is not a causal or correlative inquiry but a normative one.

What goes unmet in the cases of non-human judges is the expectation that the judge offers a correct rationale for her decision. We need not assume of course that this rationale actually justifies the decision. It may not. Nevertheless, it must be responsive to our normative inquiry. Even if it fails to actually justify the outcome, it must be at least in the business of justifying it; it must be the right sort of response. Meanwhile, the explanations that non-human judges offer for their decisions do not ring in the right register because they do not even hope to justify them.

Contrast our discussion now to two cases in which the same problem does not arise. First, suppose that instead of enlisting the syntactical monster to be a judge, we enlist them to become an economist and replace a human economist’s daily market predictions. The monster’s report can replace the actual report because the actual report does not need a normative explanation. The economist also bases her report on causal and correlative factors. Thus, the monster economist’s output would not be deficient in the ways that a monster judge’s outputs would be.

Second, suppose that instead, we task the monster with predicting what Anna, a literature critic, will write for tomorrow’s newspaper about Jane’s new novel. The same sort of deficiency does not arise if Anna’s editor decides to circumvent Anna and send the monster’s output to print instead, because the rationale behind it does not matter. Where the monster fails is when the output’s rationale is both of interest and expected to be responsive to normative inquiry.

4.1 AI Outcomes Appear to Have Rationales to Us

In the thought experiment, I granted that AI judges can produce output that is in content indistinguishable from content created by human judges. This consists in every word that appears in regular judicial decisions, including what looks like the decision’s rationale to human onlookers. Why is that not enough?

Eugene Volokh argues that “when AI judges become highly effective at crafting persuasive legal arguments, there will be little reason to prefer human judges to AI judges, at least for the overwhelming majority of legal questions, including the law development questions that reach the Supreme Court.”Footnote ²⁵ By “crafting persuasive legal arguments,” Volokh does not mean that AI judges actually base their decisions on persuasive normative rationale. Rather, he means it in exactly the terms I have granted and consistent with our assumptions about AI judges: AI judges produce output that is in content indistinguishable from content created by human judges, including what looks like a rationale to onlookers. Indeed, Volokh adds: “I have carefully tried to avoid saying that the AI judge should explain why it reached the result it reached, saying instead that the AI judge should articulate reasons supporting the decision.”Footnote ²⁶ Volokh is careful to exclude the basis of AI’s decision, presumably because he implicitly recognizes that such an explanation would be unresponsive to normative inquiry. His suggestion is therefore that the existence of supporting reasons for legal judgments is sufficient for them to be valid. However, although the existence of supporting reasons is likely necessary for the validity of legal judgments, it is likely not sufficient for it in any desirable legal system.

Obviously, according to our assumption, the AI presents the reasoning supporting its judgment as its rationale in exactly the same manner that a human judge presents hers. Thus, to onlookers, this part of the output appears as though it really were the AI’s rationale. Without knowing the assumptions we have made earlier, there would be no way of telling the difference. It is only thanks to what we know to be true generally about the sort of AI technologies we are considering, and in light of what our thought experiment shows about correlative and causal inferences, that we can be sure that what the AI presents to us is not in fact the basis of its decision. In fact, for all that observers can see, the rationale of the human judge may be no more than the same. But we are certain about the rationale of the AI judge what we can only suspect about that of human judges.Footnote ²⁷

Once we recognize this fact, it becomes evident that the supporting reasons, which the AI presents as its rationale, are in fact no such thing and therefore quite irrelevant to meeting the legal ideal of our concern. This is because we know that—contrary to appearance—such content would have had nothing to do with the judgment’s actual basis. The judgment’s actual basis, as Volokh seems to acknowledge, was the correlation of various factors that the AI considered, just as the arrangement of stars and planets was the actual basis of the Martian’s decisions.

So Volokh’s appeal to the sufficiency of the output is an appeal to the sufficiency of the decision’s ex post justifiability. But this is surely not enough. For consider that even bare-bone AI judgments may be justifiable ex post, say, by onlookers. The decision in both cases, though justifiable ex post, lacks a normative basis, and thereby fails to meet the relevant legal ideal. That the ex post justification is produced by the AI rather than onlookers does not change anything.

But Volokh’s stance may be that ex post justification is all we need insofar as it has a hope of justifying the outcome. For after all, it is responsive to our normative inquiry “why.” But requiring a rationale for a decision is not simply requiring that there be some content in the world with a hope of justifying it. It is rather that the basis on which that decision was made has had a hope of justifying it. Obviously, there may be ex post justifications even for the decisions of the Martian and the syntax monster judges. Nevertheless, it seems plausible that Tom can still walk away with a chuckle after pointing out the obvious fact that the judgment of these creatures, though justifiable, was made on the basis of things like the arrangements of planets or syntax. Presumably, Tom can point to this mere fact and walk away with a chuckle, because considerations such as the arrangements of planets and syntax are not valid bases for legal decisions.

Finally, perhaps Volokh is contesting this last point. Perhaps he is inviting us to consider a system of judging in which decisions made on the basis of the arrangements of planets are valid, so long as they are accompanied by ex post justificatory content. Put another way, Volokh may be inviting us to jettison the ideal for legal judgments to have a normative basis, if we ever become successful in designing a system that can produce justifiable outcomes via other means. We must take this proposal extremely seriously, given the extraordinary benefits that per hypothesis flow from effortless, instantaneous, and accurate judicial decisions by Martian, syntax monster, or AI judges. Because it is precisely in light of such extraordinary benefits that Volokh suggests that even decisions made on the basis of the arrangements of planets should be valid, so long as they are accompanied by persuasive ex post justificatory content. This proposal seems quite reasonable because the qualification made by justification is quite strong, which is something we have granted by stipulating that the content of the decisions of these imaginary judges will be identical to human decisions. This stipulation entails that their decisions will be equally ex post justifiable.

Volokh’s view, so interpreted, may strike some, especially non-American readers, as an extreme Holmesian line of thought.Footnote ²⁸ This Holmesian stance is widely suspected to be implausible as a model of the concept of law.Footnote ²⁹ But its conceptual (im)plausibility should not concern us on this occasion.Footnote ³⁰ What should concern us rather is a purely normative question: whether or not we should adopt such an institution of judging (be it in accordance with the concept of law or the concept of some other system of social control) in pursuit of the promised benefits of judicial automation. The correct answer to this purely normative question, I argue, is in the negative.

My reason is that jettisoning the ideal that decisions have a normative basis makes for an objectionable institution of judging, even when the decisions of such an institution are ex post justifiable. And this is all the grounds we need to reject it. Consider that the rationale judges offer under the status quo may actually fail to fully justify their decisions. The full justification of decisions under the status quo, if they can indeed be fully justified, is institutional and rests in part on the ex ante legitimacy of the process that includes, among others, a constraint that judicial outcomes be reached on the basis of a normative rationale rather than syntactical correlations or physical causes. But by rejecting this constraint, the Holmesian view cannot avail itself of the institutional justification that could (at least in theory) close the gap between the human judge’s rationale of her decision and its actual justification. It is not clear moreover, whether and how, an institution of judging on the basis of the arrangement of planets can make up for the points it loses in the ability to claim legitimacy over cases where the supplemented justificatory content fails to actually justify the decision ex post facto.

Indeed, this problem runs deeper and affects even cases where the supplemented justificatory content does happen to actually justify the decision ex post facto. Consider that in many legal cases, we just do not know what the right outcome is, even if several outcomes would be ex post facto justifiable (or conversely, if none would be). In other words, there may be no right outcome ex ante. We would then need a legal process to settle what outcome is right. The rightness of the outcome, in other words, will in part rest on it being the outcome of processes that can be defended against alternative processes ex ante.

Existing practice under the status quo can be (at least somewhat) defended on grounds that its process decides cases on the basis of reasons. To the extent this defense succeeds, the outcomes of this process can be regarded as right. But an institution of judging on the basis of the Holmesian view could not be ex ante defended, at least not as an alternative to the status quo, because its process decides cases on the basis of causes and correlations. So, its outcomes cannot be regarded as right in the same sense (at least not when the status quo process is more choice-worthy), even in cases where supplemented justificatory contents do happen to justify these outcomes.

We must therefore reject a Holmesian institution of judging, that is, we must reject an institution of judging that regards the arrangements of planets and syntax as valid bases to decide legal cases, even if such an institution is qualified by the requirement that the resulting decisions be justifiable ex post facto.

At this juncture, Volokh may try to qualify his stance by maintaining that we should only treat those decisions of AI judges as law that are actually independently justified ex post facto to the parties involved. Perhaps if the outcomes are actually justified to the parties, albeit ex post facto, their shortcomings will be overcome. Perhaps. Even so, the doors to such a qualification are closed. Because sincerely justifying each outcome would require someone like a human judge to review AI decisions de novo. Barring practical issues, this may compensate for the lack of rationale in the initial AI-generated decisions, but it also renders them entirely redundant.

Note that the door to limiting de novo reviews to only those decisions that are appealed is also closed. For there is no reason to think any party who receives adverse decisions by an AI judge would not appeal on the grounds that the decisions lack rationale. Indeed, one could regard this very paper as a class action appeal on their behalf.

Finally, practically speaking, a right to appeal and de novo review may not in reality fully protect the affected parties, because de novo review may be biased in favor of affirming or overturning AI-generated decisions. For one, human judges, depending on their attitude to technology, may show more or less deference to AI-generated decisions than to decisions made by a mortal judge down the hallway. But they may also be too languid, over-worked, or apathetic to review the matter sincerely de novo. Practically speaking, the temptation to simply affirm on the basis of the justificatory content AI generates may be too hard to resist.

4.2 The Argument Rests on Controversial Assumptions

4.3 Human Judges Also Do Not Know the Reasons Behind Their Decisions

It may be tempting now to think that where AI judges fail, human judges fail too. I have argued that AI judges ultimately make decisions based on statistical correlations which bear no relevance to normativity. It may be argued that human judges also ultimately make decisions based on biological or neurophysiological processes that bear no relevance to normativity.

To begin with, this argument proves too much. Showing that humans are also incapable of making legally valid demands on one another is no salvation for replacing them with AI judges. Instead, it undermines the legal process in both its human and machine-operated forms. As such, it is not something I need to worry about in my critique of AI judges. Nonetheless, I think the argument is mistaken too.

First, this objection rests on a defunct analogy between neurophysiological processes of the brain and the statistical processes of an algorithm. First, consider how this analogy conflates two levels of descriptions: if the analogy is supposed to operate at lower-level processes that give rise to computation, then the analogue to neurophysiological processes of the brain would be the electrical activity in a computer’s circuits. The statistical processes in algorithmic decision-making are instead themselves the higher-level or emergent phenomenon of such electrical activity. The analogues to statistical processes are not the neurophysiological processes of the brain but the emergent phenomenon they give rise to, namely normative reasoning (and other kinds of mental phenomena).

Second, even if the analogy were apt in its level of description, it flaunts the widespread consensus that mental phenomena such as normative reasoning are not reducible to the neurophysiological phenomena they supervene on, unlike algorithmic processes that are reducible to the electronic processes occurring in their circuits.Footnote ⁴⁸ The objection urges that if normative justification cannot obtain by way of electrical activity or algorithmic calculations, it could also not obtain by way of biological or neurophysiological activity. That point is of course true. But it is irrelevant because I have not claimed that the normative reasoning which human judges are capable of obtains at the biological or neurophysiological levels. Instead, the idea here is that this sort of justification happens at the level of conscious human reasoning. For the objection to cast any doubt on the matter, it should either show that normative reasoning cannot obtain by way of conscious human reasoning, or that conscious human reasoning is reducible to the neurophysiological phenomena it supervenes on.

Alternatively, this objection can be cashed out as a worry that human reasoning is opaque to us, and perhaps even to the judge herself. The claim here may be that ‘black box’ arguments (which we met earlier) also apply to human reasoning. The worry would then be that human judges could act for the wrong reasons behind our backs, even unbeknownst to themselves. For instance, they could make distribution decisions as a result of implicit bias. Worse yet, the claim against human judges may be even stronger, in the same way that my critique of AI judges is stronger than black box arguments. Here, the worry would be that even when human judges decide correctly, their rationale is unknown, unknowable, or nonexistent, even though what they write down as their rationale looks like conscious legal reasoning to onlookers. In other words, the objection may be that what I have claimed of AI judges may be true of human judges as well.

This iteration of the objection again proves too much (showing that human judgment also lacks rationale is no salvation for AI judges). Nonetheless, I think its assumptions about human limitations are mistaken. It is of course important for us as humans to be realistic about the extent of our own capacity in normative thinking and about the limitations of the rationales of the demands we make on one another, sometimes as judges and under the banner of law. A grain of humility should make us pause, especially when the burdens and consequences of such demands are significant and grave. This level of humility and the skepticism that underlines this iteration of our objection, may ground a strong case, say, against the death penalty. But this kind of humility should not lead us to give up on institutional normativity altogether. For even if we fail to make the right choices in all the close calls, human experience attests that we can easily distinguish between significantly better or worse outcomes not just in hindsight but also ex ante.

Finally, this objection may be directed at juries rather than judges. Thus far, I have only discussed judges making decisions. But in many cases, including more serious cases, the verdict is decided not by a judge but by a jury. And one thing a jury cannot do is explain its rationale. After all, jury members vote and may have different reasons for their vote. And in any case, their deliberations are kept secret. A plausible iteration of our objection then may be that jury verdicts also lack rationales. To my lights, this is the only objection we may need to continue to worry about, because it is powerful. But several avenues of addressing it come to mind, which I shall briefly sketch without fully defending.

The first avenue would be to appeal to the distinction between questions of fact and questions of law. The idea here is that juries are determinants of fact rather than determinants of questions of law. The response would then rely on this distinction and maintain that only answers to questions of law require normative rationales. After all, causal or correlational explanations are apt for determining matters of fact. The idea here is that what jury members do is like what the economist of my earlier discussion does (see section 2.2). But if this is right, then we will have to be ready to also maintain that the AI-jury-member’s vote would not be deficient in the same way that the AI-economist’s output was not. This may not be a terrible outcome (see section 4.5, below).

The second avenue of response would be to insist that the decision of each jury member, and hence each vote, has a normative rationale. Thus, each vote meets the requirement that it be cast on the basis of a normative rationale, though not being accompanied by it. There obviously still remains a gap between each vote having a rationale and a jury verdict as a whole having one. This gap could be bridged either by loosening the constraint of what we expect of valid legal decisions or fictionally ascribing a rationale to the collective verdict (along similar terms that we ascribe responsibility or agency to, say, corporate entities).

The third avenue would be to distinguish the ex post facto rationalization of the jury verdict (by the judge) from the ex post facto rationalization of AI. The distinction here will have to rest on the fact that jury members do have a normative basis for their vote, albeit one that is secret and perhaps differs from the rationale of other members, and so forth.

I am not myself entirely sure that these avenues, if pursued, will yield a fully satisfactory response to the objection’s iteration against jury verdicts. But they might. And there may be other avenues. Else, we may have to qualify the scope of the case against AI judges to non-jury trials.

4.4 Access to Justice

What about the purported advantages of AI judges? Can they offset the lack of rationale in their decisions? The purported gains will be in accuracy, efficiency, or both. It is no secret that court dockets are full, the legal process is slow, humans are fallible (their decisions are prone to errors as a result of things like not having had breakfast), there is significant inconsistency in human-generated results, and so forth.Footnote ⁴⁹ It is also no secret that the poor and marginalized get the brunt of all this. But these shortcomings arise from underfunding and understaffing the courts, which is a political choice. And as Wilmot-Smith forcefully argues, there is much that we can and should do to address the shortcomings that does not come at the cost of giving up any social, moral, or legal ideals.Footnote ⁵⁰ The impetus behind Volokh’s intuitions and the techno-optimist enthusiasm about judicial automation, therefore, is a false dichotomy, as if we had no choice but between backlogs and AI judges. The hard question, however, is not about picking between backlogs and AI Judges. It is rather about the price we are willing to put on normative rationales for judicial decisions.

4.5 Breathalyzers and Traffic lights

It might be thought that a host of electronic devices like traffic lights and breathalyzers already do what an AI judge would do, albeit being much simpler and making much simpler decisions. If my argument outlawed such banal legal instruments, it would be overinclusive. Conversely, it may appear that such instruments amount to counterexamples to my analysis: if traffic lights can decide who should stop or go, then an AI judge could decide who should go to jail or walk free. But such objections fail, because they rest on a mistaken analogy between instruments like traffic lights and AI judges. Importantly, unlike AI judges, traffic lights and breathalyzers do not make any legal demands but merely feature in demands made by the law. The objection appears tempting if and when we think of the role of a stop light as deciding who should stop or go, or as demanding that we stop when it is red. But traffic lights make no such decisions or demands. It is rather the law that make such demands as ‘stop when the light is red’. Thus, traffic lights and other such instruments merely feature in what the law demands.

5.1 Not Denigrating AI Technology

Starting from my characterization of AI technology and its future. Readers may find themselves in agreement with my rejection of the futurologist claim that ‘in some future, AI can be capable of anything’, but nevertheless worry that in some future, AI systems will be capable of reasoning in a way that is responsive to normative inquiry. So, at the risk of somewhat repeating what I have said above in section 4.2.3, let me explain in more detail why the door to such a future is closed.

All AI technology (of the relevant computational type) stems from assuming a form of determinism about the universe, including human behavior. The assumption, call it modest correlative determinism, is that all future events, including all future human behavior, are at least to an extent a correlative function of some past events. It then discerns (or ‘learns from’) the events in the past to make predictions about the future. To the extent that the universe and human behavior are not deterministic (i.e., to the extent they deviate from past correlative patterns), AI will be out of luck. But to the extent they are deterministic, they will be (in principle) predictable by AI, given sufficient relevant data, computational power, and advancements in the technology.

In practice, AI does not have access to all the correlative data. Thus, AI’s predictive (in)accuracy will be a function of the training data and often falls short of what we can call ‘the ideally attainable level of accuracy’. The challenge to reach the ideally attainable accuracy arises either from not knowing exactly what data about which past will be sufficient input for the AI to reliably predict future events, or not having it. Having is a matter of data collection. But more is not always better. What AI needs is enough of the relevant data. If you have AI predict consumer shopping behavior on the basis of astrological data, the predictions will not be good because astrological data are not strong predictors of human shopping behavior. A better predictor may be factors like their shopping history, income bracket, geographical location, height, weight, or some combination of several such factors. The relevant data is the right combination of factors that amounts to the best possible prediction. When we talk of the ideal AI technology of the future, we are talking about AI systems that in addition to having sufficient computing power, have all the relevant data for our target prediction domain, such that they can predict events in that domain at the ideally attainable rate of accuracy.

What about AI that is designed to do things rather than predict them? I have been talking of AI judges that decide legal cases. I have suggested that the ideal embodiment of this AI would predict what a human would do. But perhaps this suggestion is unreasonably restrictive. It may be tempting to think instead that the ideal embodiment of this AI just decides the cases itself rather than predicting what a human judge would do. This temptation is wrongheaded, however. To understand why, it is important to disentangle reality from metaphor. When we talk of AI deciding to do xyz, we are often speaking in metaphor. What we mean is that AI will predict that doing xyz will be regarded as the right thing to do by humans. So, when we say that AI decides legal cases, what we must mean is that it spits out an outcome that it predicts humans will regard as correct. Remember, that we are not talking about Artificial General Intelligence (AGI) from science fiction that has a mind of its own.Footnote ⁵³ Metaphors aside, all that the (relevant kind of) AI does is predict.

Let me pull all of this together in a relatable example. Consider an AI system that moderates social media posts. Suppose it is tasked to flag hate speech on a social media platform. How does it go about the business of judging which posts to flag? It tries to predict which posts a human moderator would flag as being in violation of a content policy and will go about flagging those. Its ‘judging’ then consists in no more than predicting what a human moderator would judge in the same circumstances.Footnote ⁵⁴ Its design, therefore, rests on the modest correlative determinism assumption about human flagging behavior, which is to say, its design assumes that whether a human would flag each post in the future will be, at least to an extent, a correlative function of some past events or behaviors. This assumption happens to be, to a reliable extent, true. Experience shows that whether someone would flag a post is to varying degrees a correlative function of various past events. This is why an AI can fulfill this function to varying levels of accuracy. The attained level of accuracy depends on which past events or whose past behaviors the AI is basing its predictions on.Footnote ⁵⁵ If the AI made its predictions on better predictors, it would make better predictions; and if it made its decisions on the best predictor, which may be a combination of several predicting factors, its accuracy can in principle reach the ideally attainable level of accuracy.

Much current technology research focuses on improving AI performance by fine-tuning training data. Speaking without metaphor, this research focuses on improving the accuracy of AI prediction by discovering better predictors. For instance, Balagopalan et al. have recently discovered a distinction in the strength of two predictors that helps improve the kind of AI we have just been considering.Footnote ⁵⁶ The distinction is one between past normative judgments (e.g., judgments about whether something is violative of a rule) versus past descriptive judgments (e.g., judgments about whether something exhibits features that a rule prohibits). Both are reliable predictors of future normative judgments; however, Balagopalan et al. were the first to show that the former is a much better predictor than the latter.

In the context of our social media example, which Balagopalan et al. also discuss, past normative judgments are judgments about whether a particular post violates a rule against hate speech. Past descriptive judgments are about whether a post exhibits hate speech. Both sets of data could be used to train an AI to make normative decisions, that is to say, decide whether a human would regard a new social media post as violative of a rule against hate speech. If trained on descriptive data, the AI will first try to predict whether a human, as a descriptive matter, would regard the post as exhibiting hate speech and then implements the rule with a simple ‘if then’ logic (hence flagging posts that it predicts humans would regard as exhibiting hate speech). If trained on normative data, the AI will directly try to predict whether a human would, as a normative matter, regard the post as violative of a rule against hate speech (hence flagging posts that it predicts humans would regard as violating a rule that prohibits hate speech).

Both AIs would perform quite well, that is to say, the posts they each flag as violating a rule against hate speech will significantly overlap with those posts that humans would have flagged. But the AI trained on normative data performs much better (the posts it flags will have more overlap with those humans would flag). Why? Because, as it turns out, humans tend to be more lenient in their normative judgments than in their descriptive judgments, meaning that they are less likely to say that a post violates a rule (against hate speech) than saying that it exhibits a rule-prohibited feature (hate speech).Footnote ⁵⁷ As such, the AI trained on descriptive data will incorrectly flag cases where a human would regard the post as exhibiting hate speech though not violative of a rule against it. The AI trained on normative data, meanwhile, will not make such mistakes and will therefore reach a higher level of accuracy in its prediction of human normative judgment.

In short, Balagopalan et al. show that AI can reach a greater accuracy in detecting whether a human would regard a social media post to violate a rule against hate speech if it is trained on data that are labeled as violative/non-violative of the said rule rather than data labeled as exhibiting/non-exhibiting hate speech.Footnote ⁵⁸ Such a finding can be presumably generalized to other contexts, to the extent that humans show a tendency in those other contexts to not find a rule violation with the presence of a rule-prohibited feature below a minimum threshold. In computer science metaphor, this sort of finding will be dubbed ‘enhancing AI’s performance by fine-tuning training data.’ In abstract terms though, the idea is just that discovering better predictors can improve AI predictions about ‘what a human would do’.

Note that whichever type of data the content moderator AI is trained on, its starting assumption is that future human judgment regarding rule violation is a correlative function of some past events. In the state of the art, the assumption is that they are a correlative function of past descriptive human judgments. In the improved experimental systems designed by Balagopalan et al., the assumption is that they are a correlative function of past normative human judgments. Both assumptions are to an extent true, though Balagopalan et al. show that the assumption of their experimental systems are to a greater extent true, meaning that future human normative judgments are more a correlative function of past human normative judgments than they are a correlative function of past human descriptive judgments.Footnote ⁵⁹

The point of all this discussion is this: at the most abstract level, and irrespective of their level of accuracy, all AI systems, be they of the present or the future, share the same basic assumption and design. The assumption is what I have called modest correlative determinacy with regard to any target domain. The design rests on predicting events in the target domain on the basis of its expected correlation with some past events. The essential differences in various AI systems only lie in which past events they rely on more to predict the future events of the target domain.

It should be clear, then, that while advancements in the technology (better data, better labeling, better design, more computational power, or what have you) can increase the accuracy of AI systems’ predictions about the universe, including human behavior (and including human normative behavior), they could not change the mode of AI reasoning. All that such advancements could hope to do is get us closer to what I have called the ideally attainable rate of accuracy in AI predictions. To stack the cards in favor of even the AI systems of the future, the argument of this paper assumed one hundred percent accuracy, which is in any case, either equal to or greater than the ideally attainable rate of accuracy (depending on whether the target human behavior is one hundred percent correlatively deterministic or not). It should be clear then that no AI, past, present, or future, is immune to the critique of this paper. This is because the critique targets an inherent feature of AI, namely correlative reasoning, which is the very form of reasoning that all AI is enabled by.

5.2 Not Idealizing Human-Run Institutions

There may be good reason, including risk aversion, to hold AI to heightened standards. But this is something we should explicitly deliberate on so that good reasons can be set apart from mere bias. Because absent good reasons, holding AI actors (or AI-run institutions) to standards we do not normally hold humans to could risk halting automation when we should not. The danger for irresponsible techno-pessimism arises when we pretend to hold humans to standards that we actually do not, just to show that AI actors fail to meet them. So let us turn to my characterization of human judging. I said that it would be a good thing for judges to make their decisions on the basis of normative reasons rather than purely causal or statistical reasons. Indeed, I said that parties can reject adverse outcomes with a chuckle if the basis for them was solely causal or correlational reasons. But have I relied on a naïve or romanticized picture of human judging? Or is this a standard to which we can hold human judges?

In human-run institutions of judging, many cases are decided primarily by reference to a rule (be it a statute or precedent). Many are decided on procedural rather than substantive measures. So, at least in many cases, judging in human-run institutions is shallow, formalistic, and constrained quasi-mechanically by precedent. But my critique of AI judges is compatible even with such a modest and realistic view of human judging.

To see why, suppose human judgments consisted in nothing more than mechanistic rule-following. Even so, these judgments would be perfectly responsive to normative inquiries in the narrow sense that I claimed judicial decisions must be. This is because rules, even when mechanistically applied, are quintessential examples of normative material that are in the business of justification. And a decision made on the basis of a rule is normative in the sense that it specifies what ought or ought not to be done. Thus, an explanation of the kind: ‘This court finds that xyz because there is a rule that we should do so’ for a decision, though by no means capable of guaranteeing its justification, is sufficient to meet the standard I have set. This is because that kind of explanation is at least in the business of justifying legal judgments, which is all that the standard asks.Footnote ⁶⁰

But note how the Laplacean scientist could not even do this much: instead of reasoning that he should hold Tom liable for paying Chris because there is a rule that he should so hold, the scientist determines—on the basis of the causal effects of the constellations of physical material and forces—that the human judge would have held Tom liable, had she been presiding. So too, the syntax monster will statistically determine the arrangement of text coming out of the human judge’s printer, on the basis of the statistical correlations between all the texts in the world. This means that neither the Martian nor the syntax monster are even up to the task of mechanistic rule-following. Given the argument of the paper, the same goes for all AI systems.

As we saw in section 4.1 above, even explanations that are responsive to normative inquiry may very well fall short of actually justifying their decisions. Mechanistic rule-following is no exception. For one, this sort of explanation can always be contested, à la legal positivists, on grounds that the rule was pointless or corrupt. So, I am by no means claiming that mechanistic rules, much less any other normative material used as the basis of legal judgments, must themselves be justified or justifiable in some further terms. All I am claiming is that they already ring in the right register.

It is extremely important then, to appreciate just how low this paper’s argument sets the normative bar: all that the rationale of a judgment needs in order to pass it is having some normative material in its mix, be it a rule, a precedent, or some other legal value. The bar is so low that any real or hypothetical human-run legal system, on any conception of law, could easily meet it. This is indeed what makes this critique incredibly powerful.

Of course, it may be possible to imagine a human-run legal system that also falls prey to the critique. Human judges already make their decisions partly on the basis of non-normative material, many of which are indeed causal or correlative reasons (not all of which are necessarily illicit).Footnote ⁶¹ And it may be possible for an institution where judges solely rely on such bases for their judgments to still count as a legal institution on some conceptions of law. The critique only claims that such an institution is also undesirable. The fact that my argument applies such hypothetical legal systems is not a problem for it, because avoiding what I say is undesirable is so easy for human-run decision-making institutions, that any real or hypothetical legal regime could meet it.Footnote ⁶² Yet, it is a feature that no AI-run decision-making institution could possibly avoid. To summarize then:

In short, in no way does the argument here idealize human judges or existing practice in human-run institutions. Rather, the argument adopts an incredibly modest standard that any judicial system worthy of our attention should meet and then shows why any AI-run institution necessarily fails to meet it.