2. Humility

According to Whitcomb et al. (Reference Whitcomb, Battaly, Baehr and Howard-Snyder2017), humility in general is having the right stance toward one’s limitations. Intellectual humility in particular, then, is having the right stance toward one’s intellectual limitations (p. 8). What is the right stance? It depends on the limitations, but in general, having the right stance toward one’s intellectual limitations will involve both understanding what those limitations are and calibrating one’s beliefs and behavior in light of them. One intellectual limitation common to all humans is fallibility. As Descartes reminded us, there are few, if any, topics or questions on which we cannot inadvertently slide into an error. What’s involved in taking the proper stance toward our fallibility? In Whitcomb et al.’s terms, what does it take to “own” our fallibility? A natural thought is that owning one’s fallibility is incompatible with being certain in any fallibly formed beliefs. If my eyesight is fallible, then I should not be certain my eyes aren’t deceiving me. If my powers of inference and deduction are fallible, then I should not be certain that some conclusion I’ve drawn using those powers is correct. Owning one’s fallibility, then, requires open-mindedness about the possibility that one has made a mistake.

And if fallibility is ubiquitous, then there will be few, if any, topics about which we can be certain, compatible with our being intellectually humble. Christensen (Reference Christensen, Gendler and Hawthorne2007) offers an argument very much along these lines, though he puts things in terms of a tension between certainty and rationality, rather than certainty and intellectual humility:

A closely related argument that intellectual humility—or rationality, or reasonableness, or open-mindedness—is incompatible with certainty appeals to the need to make room for learning. Roughly, the idea is that if you are certain that some hypothesis is false, then you will not be in a position to recognize anything as evidence in favor of it. But any coherent hypothesis could turn out to gain support from one’s future evidence, so one should not rule it out to start with. Lewis (Reference Lewis and Jeffrey1980) gives a pure version of this argument:

And Carroll (Reference Carroll2017) offers a sort of hybrid of the argument from fallibility, and the argument from learning:

These arguments can seem pretty compelling. What resistance have they met with in the philosophical literature? By and large, objections have concerned difficulties with infinity.Footnote ⁵ To use a well-worn example, imagine a fair lottery on the natural numbers—one number will be picked, and each number has an equal chance of being picked. For any given number, you cannot be sure it will not be picked. You can imagine learning that it was picked, after all. So by the arguments above, your degree of belief that some number, say 100, is picked but cannot be 0. But if this means your degree of belief that 100 is picked must be some positive number, then we quickly face a paradox. Either you make invidious distinctions among numbers that by stipulation have the same chance of being pickedFootnote ⁶ or you assign each number the same positive probability $ k $ of being picked, and end up with a probability that some number is picked which is, incoherently, greater than 1. After all, however, small $ k $ is, there’s some finite $ m $ such that $ km>1 $ .

Advocates of regularity who discuss this problem, such as Lewis (Reference Lewis and Jeffrey1980, p. 271), accordingly tend to appeal to infinitesimal probabilities—probabilities that are greater than 0, but smaller than any real number $ k $ . They then hold that in a fair lottery on the natural numbers, each number has an infinitesimal probability of being picked. Their opponents argue that even the appeal to infinitesimal probabilities cannot save the defender of regularity from the paradox.Footnote ⁷ My own sympathies are with the opponents of regularity; I do not see any way to “quarantine” the difficulties with infinities that clearly beset straightforward statements of a regularity principle. But I also recognize that many readers will find this line of objection unconvincing. Infinities are weird! They lead to paradoxes! Maybe we should be strict finitists.Footnote ⁸ We certainly should not reject otherwise compelling principles just because they break down in infinitary cases. I discuss this debate only to set it aside. The objection to regularity that I’ll ultimately raise does not crucially depend on any reasoning about infinity. But to get there, we need to take a detour via a more general objection not to regularity, but to representing degrees of belief using probabilities in the first place. Ultimately, my response to that objection will invoke a picture of probabilistic reasoning that thoroughly rejects regularity.

3. Complexity

An influential objection to the Bayesian, probabilistic framework for representing belief and learning is that it is psychologically unrealistic because updating even a modestly complex body of belief by Bayesian conditionalization is computationally intractable. This line of thought is often associated with Harman (Reference Harman1986):

Harman supports this assumption with an argument to the effect that the Bayesian conditionalizer faces a problem of combinatorial explosion; as her corpus of beliefs gets bigger, the task of updating it by conditionalization gets much harder, very quickly. While Harman’s argument for this claim was relatively informal, a few years later, computer scientist George Cooper (Reference Cooper1990) proved that exact Bayesian inference is NP-hard, arguably vindicating Harman’s suggestion.

Within philosophy, friends of probabilistic frameworks for thinking about belief and learning have tended to respond to considerations about the intractability of Bayesian inference by talking about ideals and idealization. Even if learning by conditionalization is not feasible for limited agents like us, it still represents a kind of rational ideal, and one that is useful for us to the extent that we can understand our own cognition as better or worse approximating it.Footnote ⁹ As I’ll ultimately offer a very different response to worries about the intractability of probabilistic reasoning, I first want to say a bit about why I do not find this response satisfying.

There’s an old joke that runs as follows. A tourist lost in a village in Scotland asks a local how to get to Aberdeen. The local responds: “Well, I wouldn’t start from here.” I suggest that if Harman is right that belief is an all-or-nothing matter for humans, then ideals concerning how to reason with degrees of belief are unhelpful in much the same way as the local’s advice to the tourist is. It’s no accident, perhaps, that David Christensen—one of the main philosophical defenders of the picture of Bayesian inference as a normative ideal—does not accept that humans do not have degrees of belief. Rather, as I understand him, he thinks we do have degrees of belief, and what’s humanly intractable is ensuring that those degrees of belief are probabilistically coherent, and/or updated over time by conditionalization.Footnote ¹⁰ With this metaphysical picture of belief, we can imagine how a set of degrees of belief that does not satisfy various coherence constraints might still get closer to satisfying them over time, and thus, how a picture of fully rational reasoning with degrees of belief could be a helpful kind of aspirational ideal.Footnote ¹¹

But here I want to offer a reply on Harman’s behalf, and which I take to be congenial with what he wrote elsewhere.Footnote ¹² In both economics and philosophy, the conceptual apparatus of subjective probabilities and utilities was introduced via representation theorems, which say, roughly, that if people’s choices obey certain constraints, then they can be understood as having a probability function that represents their opinions, and acting so as to maximize the expectation (relative to their probability function) of a utility function that represents their values.Footnote ¹³ While this approach is not without opponents, and is probably more popular in economics than philosophy,Footnote ¹⁴ I think it’s fair to refer to it as the orthodox approach to decision theory. And it requires that degrees of belief satisfy both synchronic and diachronic coherence constraints. Roughly, what this means is that if we want a broadly functionalist metaphysical picture of degrees of belief where representation theorems characterize the relevant functional roles, then we cannot easily accept that we have degrees of belief but they do not behave (even approximately) the way representation theorems say they do.Footnote ¹⁵ And if Harman is right that it’s utterly intractable to update a large corpus of belief (which we have) by conditionalization, we should worry that any degrees of belief we might have could not behave even approximately the way representation theorems say they do.

So if we think the lesson to draw from Harman is that we do not have mental states that come anywhere near satisfying the constraints they’d need to be degrees of belief, then we should also think that norms about how to ideally reason with degrees of belief just aren’t that relevant for us. Developing the skeptical line of thought, it’s hard to see why a framework that describes an ideal of reasoning with degrees of belief should have any lessons concerning how to evaluate our own reasoning, which uses very different basic materials. To use a perhaps strained metaphor, knowing what ideal flight with flexible, flapping bird wings looks like might tell you almost nothing about how to design an airplane with rigid, immobile wings.

The response to Harman I’ll consider is in important ways concessive; I want to grant to Harman the idea that norms for how an ideally rational agent would update an intractably vast corpus of degrees of belief are so far removed from human feasibility as to be uninteresting. But we can remain interested in norms of belief updating and decision-making that make extensive use of probabilities without accepting the picture Harman attacks. There is a tight connection between Harman’s target—the picture of Bayesian norms describing how an ideal reasoner updates a vast corpus of belief—and regularity, because regularity forces the corpus of beliefs that an agent is updating to be vast. If it’s coherent to suppose that I’m a brain in a vat, then regularity requires me to assign some small probability—perhaps infinitesimal—to this possibility. But quite a lot of hypotheses are coherent. If all coherent hypotheses must be assigned a positive probability, then by necessity the event space over which probabilities are distributed will be huge.Footnote ¹⁶ So, if we take Harmanian worries about computational tractability seriously, and try to respond to them by describing how probabilistic reasoning could be done using tractably small probability spaces, we are certain to end up describing a species of probabilistic reasoning that does not conform to a regularity principle.

Ultimately, that’s where I’ll go. I’ll first try to show that everybody—not just the Bayesian—faces the problem of describing how updating beliefs can be tractable. In particular, I’ll argue that the notorious “frame problem” is fruitfully viewed as a more general version of the problem of the complexity of updating probabilistic beliefs. In this more general setting, the problem is how an agent identifies a representation of her practical situation—one that contains enough relevant information for solving it to be fruitful while still being compact enough to be tractably analyzable—as the one to subject to some decision-making rule. This problem arises no less for an agent who reasons with all-or-nothing beliefs than for an agent who reasons with degrees of belief; reasoning with a large enough corpus of all-or-nothing beliefs is also intractably complicated. But if the problem can be solved for all-or-nothing belief, it can arguably be solved in a very similar way for degreed belief.Footnote ¹⁷ Ultimately, however, the solution will be incompatible with insisting that degrees of belief conform to a regularity principle.

While the original statement of the frame problem is due to McCarthy and Hayes (Reference McCarthy, Hayes and Morgan1981), I’ll work with the broader formulation in Dennett (Reference Dennett1990), who presents it in the form of a parable, which I’ll quote in full:

While Dennett’s discussion is rich and hard to summarize, one of the main points he makes is that positing that humans have a language of thought (LOT), a la Fodor (Reference Fodor1975), does less work in explaining how we manage to intelligently cope with our environments than it might initially seem. R1, R1D1, and R2D1 all have languages of thought that they can use to represent propositions about their actions and the consequences of their actions. Still, they cannot intelligently act. While this is not meant to be an argument that an LOT is not a necessary piece of the puzzle, it is meant to show that in the absence of a solution to the frame problem—the absence of some story about how an agent manages to use their LOT to pose and analyze the right questions—we have not rendered our ability to flexibly respond to our environments in real time all that much less mysterious by positing an LOT.

I want to use the parable to make a similar point to Dennett’s, but where the target is not the LOT hypothesis, but instead the hypothesis that human beliefs are all-or-nothing, rather than degreed. In Dennett’s parable, each of R1, R1D1, and R2D1 is working with all-or-nothing and yes-no beliefs, rather than probabilities. Still, they either miss important implications of their beliefs or get bogged down in trivialities. It’s not at all obvious what a solution to the frame problem would look like—or even whether it might be somehow circumvented rather than solvedFootnote ¹⁸—but a natural thought is that, somehow, intelligent agents manage to focus their cognitive resources on the right questions, such that answering those questions and acting on that basis is both computationally feasible, and practically fruitful. In effect, this natural thought amounts to positing a two-stage model of cognition in which the first stage is a black box that somehow solves the frame problem—that is, it identifies some set of questions as the relevant ones to answer—and the second stage analyzes the questions that the first stage has outputted, and then uses the answers to those questions as the basis for action. Once we conceive of things this way, I suggest that it’s not clear why the second stage could not involve something that looks very much like textbook decision theory and expected utility maximization. If everyone—whether they think we have all-or-nothing beliefs, or degrees of belief—needs to appeal to a cognitive black boxFootnote ¹⁹ that somehow identifies a tractable set of questions for analysis, it’s not clear why those questions could not concern probabilities and expected utilities, rather than yes-no questions about the consequences of actions.

Before moving on, I want to preempt a natural concern. On a natural interpretation of Harman’s argument, the problem is that probabilistic reasoning becomes very hard very quickly, so that even not-very-complex bodies of degreed belief are intractably hard to update by conditionalization. So, one might think that even if our frame-problem-solving black box gave us reasonably small problems of Bayesian inference to work with, they would still be intractable; so it must instead give us some yes-no questions to answer. This does not fit nicely with the lesson Cooper himself drew from his proof that computing exact posterior probabilities is NP-hard, however. Rather than concluding that probabilistic inference is hopeless, he concluded that “research should be directed away from the search for a general, efficient probabilistic inference algorithm, and toward the design of efficient special-case, average-case, and approximation algorithms” (Cooper, Reference Cooper1990, p. 393). And the subsequent literature in computer science has vindicated this suggestion—approximate Bayesian inference is now a rich field.Footnote ²⁰ Roughly, the lesson, as I understand it, is that so long as you are willing to settle for methods for updating degrees of belief that give you output probabilities that are close to the true Bayesian posterior probabilities, then reasoning with probabilities need not be computationally intractable. So, there’s no obvious obstacle to the suggestion floated in the previous paragraph, as long as we are content to settle for the idea that the second stage of cognition is approximately, rather than perfectly, Bayesian.

In the next section, I’ll flesh out the idea of a two-stage model of cognition, drawing on the distinction between small worlds and grand worlds, drawn by Savage (Reference Savage1954).

4. Small Worlds

Long before Harman or Cooper worried about the tractability of Bayesian updating, Savage (Reference Savage1954) grappled with very similar concerns in the context of decision theory. Savage endorses a policy of contingency planning, summarized by the proverb “look before you leap.” But he immediately qualifies his endorsement:

Savage does not appeal to considerations about combinatorial explosion or computational complexity to defend the necessity of confining one’s attention to “small worlds,” nor does he identify probabilistic reasoning in particular as the source of the intractability of grand-world reasoning. And he’s right not to. To stick with one of his examples, grand-world reasoning about chess would not involve working with any probabilities. Ultimate consequences in chess—checkmates and stalemates—follow with certainty from ultimate board positions, and intermediate board positions follow with certainty from player decisions. But it’s still intractable—for humans or computers—to play chess in the grand-world style, by considering the entirety of the game tree and using backward induction to select the optimal move at each point.

So, the lesson I take from Savage is that some kind of two-stage strategy for solving a decision problem—first, artificially confine one’s attention to a tractable simplification of the problem and then solve it—is forced on us regardless of whether the decisions we face concern probability or not.

But—and we are now finally circling back to regularity—in the special case where the tractable problems we solve in the second stage do involve probability, I’ll suggest that our probability assignments will inevitably violate a regularity principle, at least in spirit. Why should this be? One thought—too quick, but on the right track—is that the space of possibilities we consider in our small-world problem will have to rule out some coherent hypotheses. For instance, if we are deciding whether to call a bet in a game of poker, we are likely to think about a small world that includes each of the standard possibilities for what cards our opponents are holding, but which does not include possibilities in which they are holding nonstandard cards (e.g., an earl of quills, or a countess of clovers). Since it’s coherent to imagine that someone is holding a card other than one of the standard 52, assigning such possibilities probability zero amounts to violating a regularity principle. One might object that there’s a difference between assigning such possibilities probability zero, and simply ignoring them—leaving them out of the model. I think this objection rests on a misunderstanding. It’s an axiom of probability theory that the state space, $ \Omega $ , must be assigned probability 1. It’s a quick consequence of the axioms that the complement of the state space, $ 0/ $ , must be assigned probability 0. In any given probability model, the event $ 0/ $ can be interpreted as “none of the above.” And the fact that it must be assigned probability 0 means that a poker model that “ignores” nonstandard possibilities does assign them probability 0, albeit as a class rather than case by case.

A natural suggestion at this point is that small-world probability assignments can avoid violating regularity by using a “catch-all” event—one which, by stipulation, occurs when none of the rest of the events considered in the small world occur.Footnote ²¹ As long as the catch-all is assigned some positive probability, then it would seem as if we have not violated regularity—if it turns out that your opponent was playing some elaborate practical joke which involved using nonstandard cards, that hypothesis wasn’t ruled out—it was already accounted for, indirectly, in the catch-all.

There are familiar reasons to be unsatisfied with the use of catch-all hypotheses in the context of Bayesian confirmation theory. In general, it’s very hard to motivate any specific assignments of probability to a catch-all hypothesis, let alone assignments of conditional probabilities of evidence given the catch-all; while we can offer considerations that bear on how likely we should be to make various astronomical observations given that Newtonian mechanics is true, or given that general relativity is true, it’s much harder to say what the probabilities for observations should be given that some hypothesis other than any of the ones we have yet thought of is true. Similarly, in a decision theoretic context, it’s very hard to motivate any assignments of payoffs to actions, given that the true state is the catch-all.

But I think we can sidestep these debates to argue that, whatever the merits of using catch-all hypotheses to leave room for hypotheses we have not imagined, they still do not really amount to a way of saving the spirit of the regularity principle. Imagine a small-world decision problem in which an agent A is considering two options, a $ {}_1 $ and a $ {}_2 $ , with different payoffs depending on two states of the world, S $ {}_1 $ and S $ {}_2 $ , along with a catch-all hypothesis C, for which she reserves 1% probability. This situation is represented below, with the question marks representing the difficulty of saying how fruitful various actions should be, conditional on some unimagined possibility turning out to be true.

While there are obvious difficulties with using such a small-world problem as a guide to action—how are we supposed to calculate expected utilities and decide what to do, given the unknowns?—I think we can also argue that the problem does not really respect the spirit of regularity. The decision table represents it as certain that if a $ {}_1 $ is chosen and the state of the world is S $ {}_1 $ , the agent will receive a payoff of 1. But it’s hard to see how we could interpret a $ {}_1 $ and S $ {}_1 $ such that the possibility of a different payoff, given that act and that state, is incoherent or impossible. Certainly, if the states are familiar states like, for example, what cards the opponent is holding, and the actions are familiar actions like calling a bet, it will be coherent to imagine that even though your opponent is holding worse cards than you, calling his bet will result in a loss (e.g., maybe he’ll get so angry that he shoots you).

It seems to me the only way to avoid this problem is to pack enough detail into our states and actions that they really do jointly fix outcomes (which will also have to be impossibly detailed) with necessity. But once we do this, states will be so specific and fine-grained that it will no longer be appropriate to call what we are working with a “small” world; any decision problem where states are individuated sufficiently finely to avoid the worry in the previous paragraph will have so much detail as to be intractable to solve.Footnote ²² So to insist for the sake of respecting regularity that actions, outcomes, and states should be specified with enough detail that we are not ignoring any possibilities by treating actions and states as together necessitating outcomes is itself to fail to take adequate account of our cognitive limitations.

Moreover, I want to suggest that once we accept what I’ve called the two-stage sketch of decision-making—that decision-making involves first somehow settling on a small-world problem, and then solving it—the motivation for thinking that small-world problems themselves should assign probability to a catch-all hypothesis is substantially diminished. Assigning some probability to a catch-all is meant to be a way of guaranteeing that we do not wrongly rule anything out. But a more attractive response to the risk of wrongly ruling things out, I suggest, is openness to revising which small-world model we are using, rather than insisting that all belief updating should be broadly like Bayesian conditionalization—updating within a model that’s already made some allowance for the situation we find ourselves in. To insist that all updating should be modeled by a process like conditionalization—perhaps somehow involving a catch-all hypothesis—is to fail to respect our cognitive limitations. An agent is in a position to update by conditionalization when she’s already anticipated some piece of evidence and has formed a contingency plan for how to respond to it. An agent who is limited in what evidence she’s able to anticipate thus will not always be able to update by conditionalization. Because we cannot anticipate all possibilities, sometimes, rather than conditionalizing on evidence we have already made room for in our small-world model, we need to go back to the drawing board and come up with a new small-world model that can adequately account for the force of the previously unanticipated evidence.

This suggests me a more promising place to locate the kind of the open-mindedness characteristic of human intellectual humility that we have seen cannot be captured by a formal regularity principle. An intellectually humble human will not be too wedded to the small-world framing assumptions she uses to tackle a decision problem. That is, she’ll have the right stance toward the fact that she is cognitively limited in what possibilities and evidence she can imagine. I admit that this is frustratingly vague as an account of the virtue of open-mindedness. But that vagueness is an unavoidable consequence of the fact that we do not, as yet, have an adequate account of what I’ve called the first stage of decision-making; our various formal decision theoretic methods tell us how to solve a given small-world decision problem, but not how to decide which small-world problem to solve, or when to revise such decisions. If the kind of the open-mindedness characteristic of intellectual humility manifests itself in this hard-to-formalize stage of inquiry, rather than in the stages that are better modeled using extant formal tools, it’s no surprise that we lack a crisp characterization of it.

An optimist might think there’s still an attractive, at least semiformal characterization of intellectual humility that survives the concession that obeying regularity is computationally intractable. While it’s not the case that a rational agent never assigns coherent hypotheses probability zero, it is the case that for any coherent hypothesis a rational agent assigns probability zero, and there’s some circumstance she could find herself in that would lead her to reconsider and start assigning it positive probability.

This thought strikes me as a potentially promising line of investigation, but no more than that. In the absence of a story about how rational agents do or should construct small-world decision problems—it’s not even clear what we should characterize as the inputs to the process—it seems premature to insist on constraints like the one just mentioned. Maybe if we had a clear picture of how we do solve the frame problem, we’d also see that some hypotheses are “unreachable” for usFootnote ²³—we’d never start assigning them positive probability—but we’d also regard that as acceptable, because a different approach to solving that frame problem that would let us start assigning positive probability to those hypotheses would be, on the whole, worse for us. I have no idea.