Ann, a philosopher, isn’t sure if God exists. One day, Ann meets another philosopher whom she regards as equally smart and equally familiar with the evidence and arguments that bear on the existence of God. Ann asks the other philosopher, “Do you believe in God?”
“No,” the other philosopher says.
“How confident are you in your judgment?”
“Utterly.”
“You don’t think that there’s any possibility that God exists?”
“That’s right,” the other philosopher says.
Nancy, a schoolteacher, is confident that the Republican will lose the next election. One day, Nancy meets another schoolteacher whom she regards as equally smart and equally familiar with political matters.
“I think that the Republican has a 20% chance of winning–what do you think?” Nancy asks.
“100%.”
“You don’t think that there’s any possibility that she’ll lose?”
“None whatsoever,” the other schoolteacher says.
These are two cases of disagreement with an epistemic peer.Footnote 2 Otherwise unremarkable, they are distinguished from the myriad cases in the epistemology of disagreement literature by the extreme confidences that the interlocutors have in their beliefs.Footnote 3 In the first case, the other philosopher has perfect confidence that God exists is false; in the second case, the other schoolteacher has perfect confidence that the Republican will win the next election is true.
My purpose in this essay is to argue that cases of excessive confidence like these imperil conciliatory theories of disagreement, as currently construed. These theories call for a person to modify her confidence in a proposition when faced with disagreement from an epistemic peer.Footnote 4 This is usually sensible advice, but it is too quick when an epistemic peer is sure, or almost sure, that his opinion is correct.
Excessive confidences must be handled differently by our conciliatory norms. The rational response in these cases is—roughly speaking—to remain “steadfast” in one’s belief to some degree. This is because excessive confidences are irrational, and we should take an interlocutor less seriously—perhaps not seriously at all—when he expresses one. Since the most popular conciliatory norms (e.g., those discussed at the beginning of Section 1) treat extreme confidences (say, 0.9999) and temperate confidences (0.7) the same, none is fully general, nor fully correct.
I proceed as follows: In Section 1, I describe the conciliatory position and present three conciliatory belief revision norms. I also say a few words about why excessive confidence is irrational. In Section 2, I show that cases of excessive confidence (i) are not mere mathematical pathologies, (ii) are orthogonal to the “Independence” debate, and (iii) cannot be easily dismissed by claiming that anyone who expresses an excessive confidence is not, ipso facto, an epistemic peer. In Section 3, I present two test cases that any approach to excessive confidence should be able to contend with. In Section 4, I describe how our conciliatory norms may be modified to deal with the problem of excessive confidence. I argue that a Bayesian approach to the epistemology of disagreement handles the problem elegantly and effectively. I discuss some ramifications of the preceding analysis for the general disagreement debate in Section 5. I conclude in Section 6.
One preliminary note. To this point, I’ve talked about cases of disagreement with an epistemic peer. But as I have argued before (Mulligan, Reference Mulligan2021), the peerhood case is an artificial one. It seems that many, maybe most, of our epistemic interactions involve people who are at least a little less, or a little more, “competent” or “accurate” (etc.) This essay’s argument is perfectly general. If we ought, normally, to defer to an epistemic interlocutor who is our superior to a degree
$ \delta $
, then—all else equal—when that interlocutor is excessively confident, that degree of deference should drop to
$ \gamma <\delta $
. This raises the possibility that we might show more deference to a non-excessively confident epistemic peer than to an excessively confident epistemic superior.
1. Conciliationism and Excessive Confidences
A disagreement norm is a function
$ f:\left[0,1\right]\times \left[0,1\right]\to \left[0,1\right] $
.Footnote 5 Given her own confidence
$ c $
in a proposition and her interlocutor’s expressed confidence
$ x $
, a person may compute a unique value
$ f\left(c,x\right) $
to adopt as her new, post-disagreement confidence in the proposition.
A number of disagreement norms have been proposed in the conciliatory literature. For example, there is the “Equal Weight View” (or what Jehle and Fitelson [Reference Jehle and Fitelson2009] call “Straight Averaging”):
There is Easwaran et al.’s (Reference Easwaran, Fenton-Glynn, Hitchcock and Velasco2016) “Upco” (“Updating on the Credences of Others”):Footnote 6
There is my Bayesian preference (Mulligan, Reference Mulligan2021), which was derived by Genest and Schervish (Reference Genest and Schervish1985):
where
$ \mu $
and
$ \lambda $
are parameters to be discussed in Section 4.
Although each norm will typically produce a different post-disagreement confidence, they share a feature relevant for this article: A single function applies to the entire domain. That is, disagreement norms are not piecewise functions.
Why should we distinguish excessive confidences, such as
$ x=0 $
and
$ x=1 $
, from other values in the interval? And why should we not conciliate as usual when we disagree with someone who expresses an excessive confidence? There are four reasons.
First, excessive confidence is evidence of cognitive bias on the part of the person expressing it. The existence of God is, to put it mildly, a difficult and complicated matter. Our brightest minds have struggled with it for millennia, without resolution (some would say without progress). Even the most devout among us ought to profess to something less than certainty in his belief in God.
Similarly, predicting elections is a tricky business. The most sophisticated models, fed with ample data, regularly get things wrong. Thus, when one hears an interlocutor say that he is absolutely sure that God exists (or that He does not), or that the Republican will lose the election (or win it), we learn that he is badly afflicted by wishful thinking.
Second, excessive confidence does not make sense from a Bayesian point of view. To say, for example, that a person has a confidence of 1 in some proposition is to say that there is no future evidence that could shake him from that belief. But that should not be the case in the strongest real-world circumstances. So it should not ever be the case.Footnote 7
The strongest real-world circumstances are mathematical proofs. If there’s a case in which we can be confident that a proposition (theorem) is true, it is when it has been demonstrated rigorously. But even when we have such a proof, that does not justify perfect confidence in the proposition. This is because we might one day discover that the proof contains an error.
For example, it was “proven” (by Euler, no less) that for all polyhedra, the number of vertices of a polyhedron, plus the number of its faces, minus the number of its edges, equals 2. This result was widely accepted by mathematicians for years. If you had asked them, some probably would have professed perfect confidence in it because it had been demonstrated via the method of rigorous proof.
The problem, however, is that the proof was erroneous (this formula, the so-called “Euler characteristic,” only holds for convex polyhedra).Footnote 8 Although an eighteenth-century mathematician might not have been able to point to the flaw in Euler’s proof, he could have, and should have, known that the existence of a flaw was possible. And he should not have had perfect certainty in the theorem as a result.
Third, there are skeptical considerations that speak against excessive confidence. In the Meditations, for example, Descartes raises the possibility that we are being manipulated by an evil demon to believe (what strike us as) obvious truths, like 2 + 3 = 5.
Now Descartes, of course, believes that knowledge of this proposition can be recovered through the cogito. But even if you buy that reasoning, or other anti-skeptical arguments, you might be wrong. And even if the chance that you are wrong is tiny, that is sufficient to make excessive confidence untenable.
Finally, fourth, the problem of excessive confidence is not merely theoretical. It is a ubiquitous feature of human judgment which will taint many instances of real-world disagreement. In the words of one social psychologist, “no problem in judgment and decision making is more prevalent and more potentially catastrophic than overconfidence” (Plous Reference Plous1993: 217).
For example, one classic study (Fischhoff et al., Reference Fischhoff, Slovic and Lichtenstein1977) found that experimental subjects expressing odds of 1,000:1 that their answers were correct (
$ \sim $
99.9% confidence) were nevertheless wrong 12–19% of the time, across three experiments. The calibrated odds for their actual accuracy were about 6:1.Footnote 9 Because excessive confidences are frequently miscalibrated, it is a mistake to reflexively let them influence our beliefs more than temperate confidences do. In sum, any theory of disagreement that handles excessive confidences badly will handle a significant fraction of real-world cases of disagreement badly.
2. What the Problem Is Not
Three immediate and easy resolutions to the problem spring to mind. None works.
It might be thought, first, that this is just a mathematical pathology—a result of drawing confidences from the closed interval
$ \left[0,1\right] $
rather than the open interval
$ \left(0,1\right) $
.
However, very low (but
$ \ne $
0) and very high (but
$ \ne $
1) confidences pose the same problem. There is nothing relevantly different about an interlocutor reporting a confidence of 0.0001 in the Democrat will win the election and a confidence of 0. The former is to say that the probability of a Democratic victory is akin to the probability that you’ll be struck by lightning,Footnote 10 which is ridiculous.
Indeed, in some contexts even more modest confidences—say, 0.9—can qualify as excessive. If the proposition in question is the stock market will go up tomorrow—a notoriously hard thing to predict—then a confidence of 0.9 may suggest irrationality.Footnote 11 As will be discussed, context matters critically.
Second, it might be thought that the problem raised in this essay is subsumed by the “Independence” debate in the epistemology of disagreement. In fact, it is independent, as it were, of that debate.
The Independence Principle holds that one must not use the fact of disagreement to discount an interlocutor’s opinion. As David Christensen puts it, Independence “prevent[s] blatantly question-begging dismissals of the evidence provided by the disagreement of others. It attempts to capture what would be wrong with a P-believer saying, e.g., ‘Well, so-and-so disagrees with me about P. But since P is true, she’s wrong about P. So however reliable she may generally be, I needn’t take her disagreement about P as any reason at all to question my belief.’” (Reference Christensen2011: 2).
In the cases under consideration here, however, it is not disagreement per se, or the mere fact that an interlocutor has a confidence different from our own, that leads us to discount his opinion. It is that there is something peculiar about excessive confidence; namely, it is strongly suggestive of irrationality. When we (with a confidence of, say, 0.4 in some proposition) ignore an interlocutor who expresses an extreme confidence (0.9999), we do not do so because he disagrees with us. (Observe that we would not dismiss him if he had a confidence of 0.9. We would conciliate as usual.) The disagreement is besides the point. It is the confidence, rather, and what it implies, that generates the dismissal.
Third, it is tempting to say that these cases of disagreement involving excessive confidence are moot since there is no disagreement with an epistemic peer in the first place, since no epistemic peer can be excessively confident. After all, I have argued that interlocutors who express an excessive confidence are not equally thoughtful, nor equally free from bias. And on a plausible definition of epistemic peerhood (cf. n. 2), that disqualifies them from peerhood status. Alas, this does not solve the problem.
For one thing, I think some will intuit the opposite: that people whom we regarded as epistemic peers before disagreement remain peers after they express an excessive confidence. What is unique about the situations we are considering is that it is the confidence itself—the value of
$ x $
—that is problematic. This is categorically different from typical facts that speak to peerhood status, all of which involve the character of an interlocutor, the way he expresses confidence, and so on. For example, a significant difference in education or in intellectual capability may disqualify someone from peerhood status. So may the manner in which
$ x $
is asserted: If our (otherwise-qualified) interlocutor says “x” with slurred speech, glassy eyes, and breath redolent of alcohol, that may mean that he is not an epistemic peer after all (because he is drunk). But if we imagine a real-world case akin to the first vignette—we meet another philosopher whom we believe understands the relevant issue as well as we do, has published on it equally, seems about as smart as we are, has never expressed bias as best we can tell, and so on–well, some may retain the intuition that this person is a peer.Footnote 12
That there may be something to this intuition may be clearer with a case of epistemic superiority. Suppose we meet an accomplished mathematician, a number theorist, and we ask her, “What is your confidence in Fermat’s Last Theorem is true?” She replies that she is perfectly confident in this proposition; after all, the theorem was proven in Reference Taylor and Wiles1995 by Andrew Wiles (with help from Richard Taylor).
In this case, one cannot plausibly argue that our interlocutor is not at least our epistemic peer. Indeed, it seems clear that she is our epistemic superior. Yet there is still something wrong with her extreme confidence. After all, the two relevant papers (Taylor & Wiles, Reference Taylor and Wiles1995; Wiles, Reference Wiles1995) are 129 pages long and contain methods only a handful of mathematicians understand. Even though he was confident it was sound, Wiles’s original proof contained a fatal error (caught by referees at the first journal he approached). It is possible that another, subtle error snuck through. Moreover, Wiles’s proof relies on many results from other mathematicians, which in turn rely on others’ results, and there could be an error in one of those papers. If any of that is possible, then perfect confidence in the proof is irrational.
The point, then, is that the irrationality of excessive confidence is different than the irrationality that implicates peerhood. To fully explore this matter would require a conceptual analysis of epistemic peerhood (on that see, e.g., Gelfert, Reference Gelfert2011; Lougheed, Reference Lougheed2020), but that is unnecessary, since even if one does think that excessive confidence imperils peerhood status, that does not eliminate the problem I have raised.
The reason is that, as alluded to in the introduction, it is rarely the case that a person is a peer simpliciter. Rather, our interlocutors tend to be a little smarter or a little less smart, a little more knowledgeable along this line but less knowledgeable along that one, and so on. But no matter what our interlocutor is—epistemic inferior, peer, or superior—we have reason to be skeptical of his opinion if it involves excessive confidence. So, merely disqualifying him as a peer for expressing excessive confidence is not enough, since we would still end up modifying our belief too much (albeit using a different norm) as a result of disagreement.
Indeed, if we consider the case of Fermat’s Last Theorem, above, it seems that we should give some deference to our interlocutor’s opinion when she expresses perfect confidence in Fermat’s Last Theorem is true. We should increase our confidence in this proposition. But we should not increase our confidence as much as we would have, had she given a high but not excessive confidence (say,
$ x=0.98 $
).
3. Two Test Cases
I will now consider two objections to the general approach to dealing with excessive confidence which I have outlined.Footnote 13 I shall not resolve them here; rather, we shall use them, in the following section, as test cases as we consider how to rigorously treat the problem of excessive confidence.
Objection 1. Suppose confidences above 0.9 are excessive when it comes to proposition X, but your confidence, 0.85, is not. You meet two people, one of whom has a confidence of 0.2 in X and the other of whom has a confidence of 0.91. Although excessively confident, the second person’s opinion cannot be wildly off (being so similar to your own view, that would mean that your own confidence is wildly off—which is epistemically untenable). So the second person’s opinion should count for something.
Objection 2. Suppose confidences above 0.9 are excessive when it comes to proposition Y, but your confidence, 0.2, is not. You meet five people, all of whom have confidences of 0.91 in Y. Even if these five people are excessively confident, their consensus surely counts for something. Indeed, it seems epistemically important.Footnote 14
4. Discussion
It is irrational to be excessively confident in one’s beliefs. Thus, when we encounter an excessively confident interlocutor within the context of disagreement, we gain insight into his epistemic status. That insight, in turn, justifies an extra degree of skepticism.
How, then, should Conciliationists modify our theories to deal with the phenomenon of excessive confidence? A straightforward approach is to simply ignore those confidences that you regard as excessive. Suppose, for example, you think that Conciliationism’s Equal Weight View (equation (1)) is correct in its broad strokes. That is, except for cases involving excessive confidence, we ought to “meet in the middle” with our interlocutor. Then we could define our disagreement norm piecewise, as:
where
$ \varepsilon \in \left(\mathrm{0,0.5}\right) $
defines excessive confidence in the given context. For example, suppose that we take
$ \varepsilon =0.02 $
. Then, if our interlocutor says he is
$ >98\% $
sure that the proposition is true, we ignore his opinion and remain steadfast. If he says he is
$ <2\% $
sure we ignore his opinion and remain steadfast. For any other
$ x\in \left[\mathrm{0.02,0.98}\right] $
, we conciliate as usual.Footnote 15
One problem with this norm is that it blunders into the two objections outlined in the last section. In those cases,
$ \varepsilon \hskip1.5pt := \hskip1.5pt 0.1 $
. Considering the first objection: This norm fails to grant the (slightly excessive) interlocutor (with confidence 0.91) any epistemic influence at all. Whether he is there or not is irrelevant—we end up with a confidence of 0.53. And in the case of the second objection (five interlocutors with confidences of 0.91 in Y), there is no confidence change at all—which, as mentioned, seems wrong.
Further, this norm is too crude. It stands to reason that one should be “more steadfast” in the face of a confidence like
$ x=1 $
than in the face of
$ x=0.99 $
, even if both are above the threshold
$ 1-\varepsilon $
. Equation (4) does not capture that dynamic.
More generally, defenders of approaches like the Equal Weight View should aspire to a norm that captures gradations in epistemic peerhood. If the norm can do that, then degrees of excessiveness (and, thereby, degrees of irrationality) can be captured as well.
Because I think that ultimately this is not the most promising route for the Conciliationist to take, I shall not explore the development of such a norm in detail. However, an example will illustrate what I have in mind.
First, define a scaling function
where
$ \varepsilon \in \left(0,\infty \right) $
is interpreted similarly to above: The greater
$ \varepsilon $
be, the larger the range of confidences we wish to regard as excessive in the given context (see Figure 1). (
$ K(x) $
is the scaled Bernoulli variance raised to
$ \varepsilon $
.)
Scaling function
$ K $
Now modify equation (1) as follows:
This is the Equal Weight View, but modified such that excessive confidences do not lead to as much belief revision as more temperate confidences do. What counts as “excessive” and “temperate” is determined by the
$ \varepsilon $
parameter. In the most extreme cases, in which
$ x=0 $
and
$ x=1 $
, there is no belief revision at all; we remain steadfast.
Figure 2 describes the norm for
$ c=0.7 $
and
$ \varepsilon =0.8 $
. The dotted segment represents the classic Equal Weight View (i.e., equation (1)):
Modified Equal Weight View (
$ c=0.7 $
;
$ \varepsilon =0.8 $
)
The modified Equal Weight View
$ {f^{{\prime\prime}}}_1 $
approximates the classic Equal Weight View
$ {f}_1 $
well for most of its domain. It diverges, however, at the extremes—which is the desired behavior. For example,
$ {f^{{\prime\prime}}}_1\hskip0.24em \left(\mathrm{0.7,0.7}\right)={f}_1\left(\mathrm{0.7,0.7}\right)=0.7 $
;
$ {f^{{\prime\prime}}}_1\hskip0.24em \left(\mathrm{0.7,0.4}\right)\approx {f}_1\left(\mathrm{0.7,0.4}\right)=0.55 $
;
$ {f^{{\prime\prime}}}_1\hskip0.24em \left(\mathrm{0.7,0.99}\right)=0.71 $
(whereas
$ {f}_1\left(\mathrm{0.7,0.99}\right)=0.85 $
);
$ {f^{{\prime\prime}}}_1\hskip0.24em \left(\mathrm{0.7,0.01}\right)=0.67 $
(whereas
$ {f}_1\left(\mathrm{0.7,0.01}\right)=0.36 $
).Footnote 16
How does the modified Equal Weight View deal with the two objections of §3? Let us begin with Objection 2. Since we are dealing with multiple interlocutors, we cannot directly apply equation (6). But since they all share the same confidence (
$ {x}_1 $
=
$ {x}_2 $
=
$ {x}_3 $
=
$ {x}_4 $
=
$ {x}_5 $
=0.91), there is a natural modification:
Choosing
$ \varepsilon =0.2 $
, this yields a post-disagreement confidence of 0.67, which seems reasonable (i.e., going from 0.2 to 0.67 when faced with five highly confident peers whose confidence is, nevertheless, irrationally high.)
Indeed, this looks like a good result when compared to the simplest norm for dealing with multiple interlocutors (and the natural generalization of the classic Equal Weight View)—linear averaging. That yields a post-disagreement confidence of 0.79. For the reasons given in this article, that value is too high because of the fact, not contemplated by linear averaging, that these peers’ confidences are excessive. Equation (7) gets you the haircut you are looking for.
Now consider Objection 1. In this case, our interlocutors do not share the same confidence (unlike in Objection 2), and a tempting modification of equation (6) is:Footnote 17
where
$ {x}_2 $
is the excessively confident interlocutor (i.e., the one with a confidence of 0.91).
Again choosing
$ \varepsilon =0.2 $
, we have a post-disagreement confidence of 0.65, which happens to be the same (with rounding) as the result of linear averaging. So this, too, seems sensible.
But now there is a conceptual problem. Equation (8) creates an asymmetry between yourself and
$ {x}_1 $
which is not justified. To illustrate, suppose we are in a context in which almost no weight should be given to the opinion of a highly confident peer. This may be expressed by setting
$ \varepsilon =7 $
, for instance (which essentially eliminates the third term on the right-hand side of equation (8)).
Then equation (8) yields a post-disagreement confidence of 0.63 (whereas the theoretically appropriate value is 0.53, splitting the difference between yourself and your (non-excessively confident) interlocutor, who has a confidence of
$ {x}_1=0.2 $
). The problem is the
$ 3 $
in the denominator in the second term on the right-hand side of equation (8). All of the influence which
$ {x}_2 $
would have exerted, had he not been excessively confident, is in effect replaced by your own opinion, when it should be split up equally between you and
$ {x}_1 $
. So in fact, despite being explicitly ignored,
$ {x}_2 $
continues to exert latent epistemic influence, reducing the weight you put on
$ {x}_1 $
. That is uncalled for.Footnote 18
Taking a step back, one might object that all these modifications lack the elegance of the classic Equal Weight View, so discussed in the epistemological literature. But to my mind, there is no reason to expect that a simple, parameter-free norm like the classic Equal Weight View will capture the richness of real-world disagreement. Conciliationists attracted to generalizing their norms in this way should not be cowed by this critique.
In any event, I come down on the side of the Bayesians, in part for how they can handle these complexities. Consider again equation (3):Footnote 19
The parameter
$ \mu \in \left(0,1\right) $
represents the confidence that a person expects her interlocutor to report. (And thus, as is the Bayesians’ wont,
$ {f}_3\left(c,x\right)=c $
when our interlocutor reports what we expect her to,
$ x=\mu $
. For in such a case, our interlocutor’s belief
$ x $
was baked into our prior.)
The parameter
$ \lambda $
has several interpretations, but can be thought of as a measure of our interlocutor’s reliability. When
$ \lambda =0 $
, we assess our interlocutor’s judgment as wholly unconnected to the true state of the world. Thus, we do not allow that judgment to affect our belief. When
$ \lambda $
is negative, we assess that her judgment varies inversely with the true state of the world. When
$ \lambda $
is positive, we conciliate in our interlocutor’s direction. And the more sensitive our interlocutor’s judgment is to the true state of the world, the larger
$ \lambda $
be.Footnote 20
Thus, the Bayesian may simply choose a lower
$ \lambda $
than she otherwise would when her interlocutor reports an excessive confidence. If she thinks it appropriate to remain steadfast in the face of excessive confidence, then she sets
$ \lambda =0 $
. If she thinks some conciliation is appropriate, she chooses a positive
$ \lambda $
. In either case, equation (3) works.
There are two complications. First, we would like to select a value for
$ \lambda $
before
$ x $
is revealed to us; that way, Independence-type worries (Section 2) do not arise. But now this cannot be done; we cannot finalize
$ \lambda $
until we hear our interlocutor’s opinion.
Second, West and Crosse (Reference West and Crosse1992) suggest a promising approach to the selection of
$ \lambda $
. Observe from equation (3) that we can work backwards, computing
$ \lambda $
by deciding on an appropriate posterior probability,
$ {f}_3\left(c,x\right) $
, for a given value of
$ x $
. This single assessment fixes the putatively proper value of
$ \lambda $
no matter what our interlocutor actually says.
West and Crosse suggest (p. 288) using the extreme cases of
$ x=1 $
or
$ x=0 $
, or
$ x=1-\varepsilon $
or
$ x=\varepsilon $
(where
$ \varepsilon $
is small and positive). Put in the lingo of this essay, they suggest that we choose
$ \lambda $
by considering what we ought to believe, were we to hear an excessive confidence.
This approach will not work. Suppose that, following the argument of this essay, we judge that if we hear one of these excessive confidences, then our opinion should be unchanged. Notice, then, that we have
In other words, by following this advice for selecting
$ \lambda $
, we are driven to Steadfastness no matter what our interlocutor actually reports. But that is not right: if, say,
$ c=0.6 $
and
$ x=0.8 $
, then we should conciliate (assuming
$ x\ne \mu $
). Now that cannot happen.Footnote 21
Neither complication is serious. To dispose of the first, we simply choose two values of
$ \lambda $
before our epistemic interaction: one is to be used in the regular case (in which the confidences we receive are temperate) and the other is to be used if we face an excessively confident interlocutor. Formalism aside, this is probably good practice given the qualitative concerns about excessively confident peers raised here and the real-world prevalence of this behavior.
More generally,
$ \lambda $
need not be modeled as two fixed parameters (one for the case of temperate confidence and one for the case of excessive confidence). Rather,
$ \lambda $
can be a function of our interlocutor’s reported confidence:
$ \lambda =\lambda (x) $
. In this case, equation (3) becomes
which is now nonlinear in
$ x $
in general. Independence concerns are still avoided, though, provided that
$ \lambda (x) $
is specified ex ante. Not every such function will be admissible;
$ \lambda (x) $
must be chosen so that the resulting posteriors are probabilistically coherent.
Second, West and Crosse’s suggestion for choosing
$ \lambda $
can still be used for the majority of cases, involving temperate confidences. And we can still use very high and very low (though not excessive) values of
$ x $
. Then we can analyze how that value ought to change in the case of excessive confidence and take an appropriate haircut. That is, for cases of excessive confidence, we can choose a (lower)
$ \lambda $
, sensitive not just to the fact that
$ x $
is excessive, but to the degree of excessiveness as well.
Suppose we decide that, in some context, confidences
$ <0.1 $
and
$ >0.9 $
are excessive. We have a confidence of 0.6 in the proposition at issue and expect our interlocutor to agree with us (
$ \mu =0.6 $
). We reckon that if our interlocutor says that his confidence is in fact 0.9, then the rational post-disagreement confidence is 0.75. We have now fixed
$ \lambda =0.5 $
, and this value can be used for any non-excessive confidence,
$ 0.1\le x\le 0.9 $
, that we hear.
And we can use lower values of
$ \lambda $
for excessive confidence. Perhaps we think that
$ \lambda $
should be reduced by 0.02 for confidences of 0.09 and 0.91, reduced by 0.04 for confidences of 0.08 and 0.92, and so on, with
$ \lambda $
bottoming out at 0.3 (for confidences of 0 and 1).Footnote 22
We have now selected
$ \lambda $
before any confidences are revealed to us. And we can take the firm line—if contextually appropriate—that the most excessive confidences call for Steadfastness on our part, while not locking us into that result if a temperate confidence is in fact uttered.
Before explaining how this Bayesian approach deals—effectively—with the objections of Section 3, I want to stress how important the issue of interdependence is, and that care must be exercised in accounting for this possibility.Footnote 23 When it comes to real-world disagreement, dependence is the rule rather than the exception. There may be dependence between interlocutors, dependence between yourself and one or more of the interlocutors, or both. Many disagreement norms (e.g., the Equal Weight View) do not contend with the important nuance of this issue for the simple (and in my view fatal) reason that they are unable to incorporate dependence in the first place.
In the Bayesian model discussed, dependence is handled via the
$ \lambda $
parameter. It would take us too far afield to discuss the modeling of dependence in detail. Observe, though, that in this model, one may take two approaches to incorporating interdependence. One may take a “top down” approach, explicitly considering the details of the scenario (asking and answering questions like: “What epistemic capacities are the same?”; “What common training have we received?”; “What evidence do we share?”) and then selecting appropriate
$ \lambda $
s. In some cases, this may be relatively easy; in others, onerous.
Or, one may take a “bottom up” approach, considering what the appropriate post-disagreement confidence should be given such-and-such simple, fixed, and tractable inputs. That contemplation will yield
$ \lambda $
s that both appropriately incorporate interdependence and reveal its extent.
Let us move on to the two objections of Section 3. Both involve multiple interlocutors and the appropriate modification of equation (3) is (Genest & Schervish Reference Genest and Schervish1985: 1207).Footnote 24
First, we consider Objection 1. Our two interlocutors are indistinguishable, so we have, preliminarily,
$ {\lambda}_1={\lambda}_2 $
. We compute this parameter as suggested above: by considering what the rational confidence would be if our interlocutors uttered such-and-such. But we do not consider extreme values of
$ {x}_1 $
and
$ {x}_2 $
for the reasons given, but rather temperate ones.
One example (of many) potential temperate confidences to use would be to consider what we ought to believe if one interlocutor disagrees with us with similar intensity (
$ {x}_1=0.15 $
) and one is wholly unsure (
$ {x}_2=0.5 $
). For many contexts, a natural thing to say is that equal weight would be appropriate:
$ {f}_4\left(\mathrm{0.85,0.15,0.5}\right)=0.5 $
. It is straightforward to compute
$ {\lambda}_1={\lambda}_2=0.33 $
.Footnote 25
Now our interlocutors’ confidences,
$ {x}_1=0.2 $
and
$ {x}_2=0.91 $
, are revealed to us. Before considering the issue of extreme confidence, we can see that
$ {f}_4\left(\mathrm{0.85,0.2,0.91}\right)=0.66 $
, which is intuitively sensible and in fact almost the same as equal weighting (
$ 0.65 $
).
We are not quite done, though, because
$ {x}_2 $
is excessively high. So we take a haircut off
$ {\lambda}_2 $
, as described—to, say, 0.30. With rounding, the post-disagreement confidence drops very slightly, to
$ 0.65 $
, and this interlocutor continues to exert significant epistemic influence over us–as is intuitive and theoretically desirable.
Next, Objection 2. Following similar reasoning to the above yields, preliminarily,
$ {\lambda}_1=\cdots ={\lambda}_5=0.17 $
(i.e., if it ends up that one person agrees with us, assigning a probability of 0.2, two people disagree, and two people are wholly unsure, then we ought to split the difference,
$ {f}_4\left(\mathrm{0.2,0.2,0.8,0.8,0.5,0.5}\right)=0.5 $
).
Now all five reveal to us their (slightly excessively high) shared confidence, 0.91. We take our haircut and obtain
$ {\lambda}_1=\cdots ={\lambda}_5=0.15 $
. This yields a post-disagreement confidence of 0.73, which is again sensible: Our initial confidence in the proposition was
$ 0.2 $
, and we met five people with (slightly) excessive confidences of 0.91. Qualitatively, we go from being highly skeptical of the proposition to being confident in it. (In this case, equal weighting yields 0.79.)
So the Bayesian approach surmounts the two objections raised in Section 3. It also offers an intuitively compelling motivation for Conciliationism, along with flexibility, via the
$ \mu $
and
$ \lambda $
parameters, in grappling with the critical complexities of real-world disagreement (the most important being gradations in competence and ubiquitous interdependence). And, relevant here, it deals with excessive confidence sensibly.
5. Lessons for the Disagreement Debate
We have diagnosed a problem: Excessively confident interlocutors (epistemic peers, inferiors, and superiors) are currently taken more seriously, from the point of view of the theory of disagreement, than they ought to be. I have argued that a Bayesian approach to disagreement, operationalized in equations (3) and (9), is the cure. I now want to offer some broader comments about why the formal Bayesian approach that has been elaborated in this article is so attractive within the philosophical setting of the epistemology of disagreement.Footnote 26
Before I do, though, I want to address Benjamin Anders Levinstein’s (Reference Levinstein2015) apparently contrary claim that “for reliable agents, weight should tend to grow with opinionation” (p. 9). For the reasons given in this article, that cannot be categorically correct (it could, though, be true up to extreme confidences).
Levinstein’s focus is not what concerns us here; namely, how to rationally update our beliefs in light of disagreement (what Levinstein calls the “micro-theory” of disagreement). Levinstein’s argument, rather, putatively constrains our updating procedures. It would take us too far afield to analyze his argument in detail, but consider the following case, which he calls Sunrise:
I consider Tom–and indeed, nearly everybody I ever come in contact with–roughly a peer when it comes to the question of whether the sun will rise tomorrow. The reason I hold such an egalitarian view is almost entirely based on my expectations about what these people think. My distribution
$ \mathfrak{b} $
(Tom’s credence is
$ x $
) over potential credences he may have in the proposition that the sun will rise tomorrow has nearly all its weight right around 1… . And, conditional on him having a credence right near 1, I assign him very low expected inaccuracy. If he surprises me and tells me that his credence is close to .5, I’ll legitimately ignore him (almost) entirely, since reality diverged so extremely from my expectations. (p. 9)
This is incompatible with the Bayesian approach because we Bayesians update our beliefs only if others’ reports diverge from our expectations. What Levinstein views as a reason to remain steadfast (
$ x\ne \mu $
) is, to my mind, grounds to conciliate.
And while I certainly agree that “nearly everybody I ever come in contact with [is] roughly a peer when it comes to the question of whether the sun will rise tomorrow,” this is not “based on my expectations about what these people think.” It is because nearly everybody I ever come in contact with is roughly equally “smart” when it comes to the proposition the Sun will rise tomorrow, equally experienced in the relevant ways, equally free from relevant bias, and so on. And, critically, there are never contextual features when this question comes up that might provide my interlocutors some independent insight into the phenomenon.
Of course,
$ x\ne \mu $
is a necessary but not a sufficient condition for conciliating. The reason we give little heed to a typical interlocutor who expresses a relatively low confidence in the Sun will rise tomorrow is that, in this context, such interlocutors provide almost no independent insight above and beyond what we already possess. So
$ \lambda \approx 0 $
, and we should indeed “ignore him (almost) entirely.”
I’m 99.9% sure that the sun will rise tomorrow. I expect my epistemic peer interlocutor will also report a confidence of
$ \mu =0.999 $
. I compute
$ \lambda $
by supposing that, if he in fact reports a confidence of
$ x=0.5 $
, my confidence should barely be shaken, perhaps falling only to 0.990. Then we have
$ \lambda =0.02 $
, and, no matter what my interlocutor says, he has little epistemic impact on me. After all, he’s almost certainly joking, or he’s distracted (and thus not really responding to the question at all), or he’s depressed and expressing a general disquiet (again, not really responding to the question), etc.
Footnote 27
These examples are, I think, not terribly helpful because they are so thin on context—and again, context matters crucially. Consider this case:
Sunrise*
Your neighbor Bob works for the Federal Emergency Management Agency (FEMA). Bob is not particularly bright; he’s certainly less knowledgeable than you are on most topics, and he suffers from various common and mild biases. His duties are secretarial, but they do enable him access to high-level discussions.
You had Bob over for dinner last night. He seemed distracted. When you asked him what was wrong, he made some brief and vague comments about his job, about existential risk, and about solar physics. It didn’t make a lot of sense. In the middle of dinner, while he was in the bathroom, you saw his cell phone light up. It was a text from “John Doe.” You know that’s the FEMA Chief of Staff’s name. The message said that Bob was urgently needed to attend an interagency meeting. The message noted that the meeting would be classified at the Top Secret level. After returning from the bathroom, Bob saw the message, quickly made an apology, and rushed out.
This morning, you see Bob while taking out the trash. You ask him how confident he is in the Sun will rise tomorrow.
In Sunrise*, it seems that Bob’s your epistemic inferior (being less intelligent, less knowledgeable, and more biased). But here—unlike in Sunrise—there are contextual features which imply that this interaction is an important and unusual one. Bob brings something epistemically new to the question at hand.
It would be reasonable to reason thus: “Look, it’s probably nothing, but it’s possible that something’s up with the Sun, or our planet, or something, and Bob’s become aware of it through his work. Now that I think about it, he’d probably know about that stuff before nearly anybody else. And whatever Bob’s defects, he’s a stable and straightforward guy. He wouldn’t make a joke about something like that. And he’s a teetotaler, so his response isn’t going to be muddled by drugs or alcohol. I guess it’s possible that he could have unknowingly consumed some drug, but that’s very unlikely. So let’s see. What if he reports a relatively low confidence in the Sun will rise tomorrow? Then it’s likely he knows something, or thinks he knows something. Of course, even then, it’s probably not a serious cause for alarm. But if Bob reports, say, a confidence of 0.75, I should drop my confidence from 0.999 to 0.95.”
Now we have fixed
$ \lambda =0.2 $
. When Bob ends up saying that the probability that the Sun will rise tomorrow is only 50%, you reduce your confidence from 0.999 to 0.9. That does not seem unreasonable. Even though “reality diverged so extremely from my expectations” (I was almost sure that my interlocutor’s confidence would be right near 1, but in fact it was 0.5), I do not, and should not, “ignore him (almost) entirely”—pace Levinstein.
Anyways, the point is not that that particular reasoning or choice of parameters is correct. The point is that contending, rationally, with each case of disagreement (including even the simplest and least controversial cases) requires thinking carefully about the context in which it occurs and selecting parameters suitable to that context. The main contextual feature we have been considering in this essay is (the possibility of) excessively high or low reported confidence. But one must also assess one’s interlocutor’s epistemic credentials, attend to the possibility of dependence, and more. It’s remarkable that the Bayesian model, containing only two parameters, provides all this flexibility.
Some of Levinstein’s findings are correct and compatible with the Bayesian approach. Notably, he points out that “strong opinions require high self-regard. By extension, if you think a bunch of your friends will end up about as accurate as you are, they’d better tend to have pretty strong opinions too, since a moderate credence can only be so accurate. But if you expect them to be opinionated and as accurate as you, you also can’t expect that they’ll generally be too far from your credence.” (pp. 13–14).
Recall our consistency conditions for the single-interlocutor case (n. 20). Your opinion
$ c $
and your judgment about your interlocutor’s “accuracy”
$ \lambda $
do indeed constrain your expectation about his credence
$ \mu $
.
To illustrate, let us suppose that your “strong opinion” is
$ c=0.95 $
and that your interlocutor is “about as accurate as you are,”
$ \lambda =0.5 $
.
Then we have
$ {\displaystyle \begin{array}{l}\max \left\{\frac{0.95}{\mu -1},\frac{-0.05}{\mu}\right\}\le 0.5\le \min \left\{\frac{0.95}{\mu },\frac{0.05}{1-\mu}\right\},\end{array}} $
or
$ \mu \in \left[\mathrm{0.9,1}\right) $
. So he’s indeed not “too far from your credence.”
Scenarios like Sunrise come up frequently in the disagreement literature. Adam Elga, in trying to defend the Equal Weight View against some obvious counterexamples, recommends:
Think of your state of mind before [disagreement]. We have assumed that, conditional on the two of you disagreeing, you think that your advisor is just as likely as you to be right. But it is also natural to assume that, conditional on the two of you disagreeing and your finding her answer utterly insane, you think that you are much more likely to be right. If so, then when that circumstance arises the equal weight view instructs you to favor your own answer… . What makes the above answer work is an asymmetry in the case. You find your advisor’s answer insane. (Reference Christensen2007: 491)
In these “insane” cases (e.g., Sunrise, but not Sunrise*), one should not significantly modify one’s belief (let alone afford one’s interlocutor equal weight). And, as we have discussed, that’s precisely what occurs, in a nuanced and principled way, in the Bayesian approach (with
$ \lambda $
very small).
Note (and this is important for the Bayesian, and arguably different than how this issue has been discussed in the literature to this point)Footnote 28 that our interlocutor is “insane” not because we are highly confident in our belief, but because that is contextually appropriate. Indeed, it is not hard to imagine cases in which we are deeply uncertain about some proposition but yet would disregard any disagreement on grounds of “insanity.”Footnote 29
The way Jennifer Lackey (Reference Lackey, Gendler and Hawthorne2010b) frames the central problem of the epistemology of disagreement is that the “nonconformist” (i.e., adherent of Steadfastness) cannot grapple with the One Against Many Problem, and the “conformist” (adherent of Conciliationism) cannot grapple with the Many Against One Problem.
The One Against Many problem is: “If enough epistemic peers disagree with me in an appropriately independent fashion, clearly I should significantly revise my belief” (pp. 278–79). We have seen (Objection 2) how the Bayesian approach solves the problem, and how this caveat of “appropriately independent” may be made precise.
The Many Against One problem is: “If many epistemic peers independently agree with one on the answer to a question, yet only one epistemic peer disagrees, surely substantial doxastic revision is not required; however, it is not clear how [Conciliationism] will provide a principled explanation of this.” I hope it is clear that, in such a case, equation (9) will protect one’s initial belief, and the sole dissenting peer will not substantially affect it.
That said, if we are to take Lackey’s scenario literally—if these many peers are truly “independent”—then substantial doxastic revision may well be required, but away from the dissenter, not toward him. For instance, if one has a pre-disagreement confidence of 0.7, and one receives reports from five independent peers, one of whom reports a confidence of 0.4 and the other four of whom report a confidence of 0.7, then one’s post-disagreement confidence may be
$ > $
0.7! (This phenomenon is sometimes known as “risky shift”.)
And Christensen has lamented that we are “a long way from offering a detailed conciliationist recipe for accommodating the evidence provided by the disagreement of others” (Reference Christensen2011: 17). He says that what is needed is a “general Conciliatory principle” whose prescriptions are sensitive to “the likelihood that [one’s interlocutor’s] expressed disagreement is sincere,” “her degree of well-informedness,” and “the likelihood of her having reasoned correctly from the evidence she has” (p. 17). We have seen that the Bayesian approach is such a principle, satisfying these, and indeed other, essential desiderata.
Even with such a principle, Christensen says that
many questions remain open: How far, in general, should one revise one’s belief? How does the principle extend to cases where initial confidence is distributed differently–e.g., where both parties are more confident that P than
$ \sim $
P but to different degrees? … The initially attractive idea of uniformly splitting the difference in credences does not sit well with the motivations for Conciliationism. So we are a long way from having a formula, or even a recipe with quantities, describing in general how one should react to disagreement. (p. 17)
None of this is aspirational. We have seen that there is a simple and intuitive way to handle confidences that are “distributed differently” and that “uniformly splitting the difference” (i.e., the Equal Weight View) is appropriate only in special, usually highly stylized, cases. Equation (9) is the formula, the “recipe with quantities” desired.
6. Conclusion
A salient, and to me distressing, part of contemporary life is the certainty that people have in their beliefs. This is true even (especially?) when these beliefs concern difficult matters of morality and politics. Whatever other deleterious effects excessive confidences have, they are dangerous from the epistemic point of view. We must take care not to let them affect our beliefs too much. This may be accomplished by modifying our conciliatory norms as suggested. It is a small change, but an important one.
Thomas Mulligan is a Visiting Scholar at Georgetown University and an Adjunct Researcher at the RAND Corporation. His research has spanned political philosophy, epistemology, decision theory, and the philosophy of artificial intelligence. He is the author of Justice and the Meritocratic State (2018, Routledge) and is currently working on the political economy of AI.
Open access
