Weighing versus Counting?

Deliberative democrats often insist that, whereas voting is about counting heads, deliberation is about weighing arguments. Deliberation, they say, requires us to consider arguments or truth claims on their own merits—and we should do so regardless of the number of people voicing them. We should recognize the force of the better argument when we encounter it.^{Footnote 12}

The classic deliberative model rests on protracted, dynamic discursive exchanges among a small(ish) set of people, however. The forceless force of the better argument is at its best in such settings, which allow back-and-forth exchanges, reciprocal prodding, and questioning among deliberators. Yet most of the information we receive everyday does not come through such structured deliberations or ideal speech situations. Instead, we have to assess what weight we should give it solitarily. In the absence of protracted, discursive engagement with our informants, it becomes harder to assess the “betterness” of the information and arguments we come across. And that may be especially so if what is at stake is some matter of fact rather than of logic or moral conviction.

Perhaps the epistemic merits of some truth claims are plain on their face, such that we immediately recognize their correctness as soon as they are uttered.^{Footnote 13} The merits of others are not so clear, however. Whether those claims are true or some arguments are better than others is far from self-evident to us. In such cases, the force of the better argument is too feeble to provide clear guidance.

Suppose I am confronted with some such proposition, and suppose I have already assessed its substance and used whatever relevant knowledge I have to form a judgment about it. Suppose I then learn that one hundred people, each of whom I regard as my epistemic peer, came to a judgment different from mine. Shouldn’t that fact figure in my internal deliberations as well? For a (probably quite large) subset of claims based on facts, counting heads can also aid us in weighing the truth of these claims correctly. If multiple people are making the same claim independently from one another, then we ought to revise our beliefs in light of that fact. The more independent assertions there are of the same claim, the more epistemic weight we should ascribe to it, updating our beliefs in light of each independent assertion.

This does not mean that we should “follow the herd” mindlessly, of course. The point is just that if many independent thinkers report that they believe X, we should count each of those reports as evidence for X’s credibility, and we should take that evidence into account when revising our credence in X. How much weight we attach to that evidence is an open question. Yet each new piece of independent evidence should count for something in our internal deliberations, and if many pieces of independent evidence point toward the same conclusion, the result of that updating will be that we will ourselves be more inclined toward that conclusion.^{Footnote 14}

Hence if informants are epistemically independent, then something akin to counting has its place in internal deliberation, just as it does externally in voting. But that is true only if those informants are speaking independently of one another.

What Weighing Truth Claims Ideally Entails

Something like Bayesian updating ought ideally to be at the heart of internal deliberation.^{Footnote 15} In Bayesian terms, each speaker’s claim can be seen as a new piece of evidence in light of which we revise our beliefs, according it epistemic weight in proportion to its credibility. In simple terms, the epistemic weight that one accords to a claim depends on how reliable one believes the information it conveys to be. In the first instance, we assess the epistemic merits of a proposition on the basis of our prior knowledge: we form an initial judgment of a claim on its substance, as it appears to us. Then, as additional evidence comes to light, we revise our belief in the claim in light of how credible we believe that new evidence to be.

Take the claim, “Russia interfered in the 2016 American elections.” In my internal deliberations, I would attribute some credence—a prior probability—to this claim on the basis of what I already know of Russian-American relations, of Russia’s technological capacity to leverage such attacks, of how susceptible social media and voting technology are to such interference, and so on. If I then hear the same claim again from a reputable security analyst, I would adjust upward my credence in this claim, given the relatively high conditional probability that I attach to that claim being true if that expert says it is. My confidence in that claim increases further with each additional credible agent making the same claim. More generally, the more people who endorse a claim, the more likely I will judge that claim to be true (assuming I regard those people’s reports as credible at all).^{Footnote 16}

Of course, the number of sources may be used as reliable evidence in our internal deliberations only insofar as those sources act independently of one another, exercising critical independent judgment rather than merely slavishly repeating or reporting what others have said. And just as we would not want to count twice the same person’s vote in an election, so too should we steer clear of “deliberative double-counting.” Directly or indirectly taking into account the same person’s claim twice in our internal deliberations would be to give it undue epistemic weight. And that, of course, is one way the Russians attempted to manipulate the American election—by having their many bots and trolls say the same thing.

Boundedly Rational Agents Being Misled by Message Repetition

The preceding section described what our processes of internal deliberation should ideally entail. Well-known cognitive limitations may hamper our judgements, however, precluding us from weighing others’ claims as accurately as we should. Our limited memory capacity and bounded rationality, in particular, can undermine our assessments. The way we react, as listeners, to message repetition is one such cognitive limitation of particular relevance here.

Psychological studies show we are very vulnerable to message repetition, quite generally. Hearing the same message multiple times creates an impression of both familiarity and commonality. The feeling of familiarity with a proposition increases with the number of exposures, regardless of their source: that familiarity (or “exposure”) effect is much the same whether someone hears the same thing three times from the same person or from three different people. But hearing the same report from multiple different people also has a commonality effect. It leaves listeners with the impression that what they hear is the general public opinion across their community. Crucially, both familiarity and commonality effects serve to make them more likely to believe that the proposition is true (Rothbart et al. Reference Rothbart, Fulero, Jensen, Howard and Birrell1978; Weaver et al. Reference Weaver, Garcia, Schwarz and Miller2007).

Theories of public opinion formation based on models of bounded rationality account for citizens’ susceptibility to repetitions of the same information via what is called “persuasion bias.” Such studies have shown that individuals’ influence on group opinions is not uniquely determined by the accuracy of the information they provide, but also by how well-connected they are; that is, by their position in the social network. Being well positioned in the network allows their information to be repeated many times across the network and have an echo effect that is not epistemically warranted (DeMarzo, Vayanos and Zwiebel Reference DeMarzo, Vayanos and Zwiebel2003, 914).

These findings are extremely relevant for developing a normative account of internal deliberation and belief formation. The deeper patterns of interconnectedness among individuals today are likely to make information repetition ubiquitous in the public sphere. Taking others’ claims into account without knowing if they are epistemically independent, or counting claims without being certain of their origin,^{Footnote 17} could thus undermine our internal deliberations and lead to flawed individual (and indirectly collective) epistemic judgments.

This effect of information repetition is well-known and put to good use by some political and economic agents such as political parties and corporations. Both political propaganda and advertisements intentionally use message repetition to exploit their audiences’ cognitive vulnerabilities and mislead their beliefs. Yet not all agents engage in message repetition with the malevolent intent of misleading the others’ beliefs. Message repetition may occur accidentally in discursive exchanges with a benign intention: to inform, to justify a belief, and so on. Furthermore, in some cases speakers may not even be aware that they are parroting information from another source; they may have internalized that information and forgotten its source—a case of epistemic appropriation.^{Footnote 18}

Regardless of the intention behind it, however, message repetition distorts the judgements of listeners if they are unaware that the message is a mere repetition of something they have already taken into account. In what follows, I discuss why grave violations of epistemic independence are problematic, regardless of the intentions of agents engaged in message repetition.^{Footnote 19}

Clone Claims

As senders of information, agents sometimes repeat information they have themselves received from other agents, without adding any new information of their own in the process. In so doing, they violate epistemic independence. In communicating that repeated information to others, those agents expose yet other agents as receivers of information to the replicated information, which may mislead the latter’s judgments if unaware of this replication, making them less epistemically competent.

Take the following example. A chemistry lab runs a series of experiments. Twenty scientists conduct the same experiment, each one reporting the results of their experiment to the team leader, who compiles the lab’s findings on the basis of the evidence received from all of the experimenters. Eighteen of them are working alone, but two (trainee Jim and his mentor Kate) are working together. Instead of observing the experiment Jim plays poker on his iPhone. In need of a report to send, he then copies Kate’s report and sends it as his own. Kate sends her report in as well, unaware of what Jim has done.

The example involves a flagrant, complete violation of epistemic independence on Jim’s part, one that could have been easily avoided. How should the team leader count Jim’s replica report? If Jim had independently observed the experiment himself, then it would have made sense for the team leader to weigh both Jim’s and Kate’s findings in deciding the lab’s overall conclusion. But because Jim’s report is simply a replica of Kate’s, the team leader should take no account whatsoever of Jim’s report, after having already taken Kate’s into account.

Analogously, while internally deliberating we can erroneously attribute too much epistemic weight to a claim someone has just repeated^{Footnote 20} after hearing it from another person, whose own claim on this topic we have already fully taken into account (cf. Goodin Reference Goodin2001, 122, fn. 6, and 123–24; Lehrer Reference Lehrer and Ryle1976; Reference Lehrer2001; Lehrer and Wagner Reference Lehrer and Wagner1981). If someone is only reporting others’ claims (as Jim was doing with Kate’s observations), then there is of course a danger of double-counting the same claim (as the team leader risks doing with Kate’s report), which would distort our final judgment.

Epistemic Overcounting Short of Double-Counting

Strict double-counting occurs when we take into account the same information twice in our internal deliberation. That might easily happen, if we cannot distinguish new information from old information that we have already incorporated.

Next let us consider something short of that: cases in which a new assertion provides some genuinely new information while also containing some old information that we have already taken into account. Two reports might come from sources that are subject to some common causes, but nonetheless report independently of one another. Or the second report might repeat—but then add to—information that we have already taken into account. To treat the second report as fully on a par with the first one would be to overcount the first report. But it would not be literally to double-count it, because the second report also adds some genuinely new information.

1. Common Causes, Independent Assessments

We all get much of our information from the same sources, and we are all influenced by what others are thinking: “our cognitive lives are never self-made” (Beerbohm Reference Beerbohm2012, 154). In this sense, we can never be fully epistemically independent. Yet we can nonetheless exercise independent judgement both in processing the information we receive from others and when passing along that information to others. Hence, an agent can be said to be “relatively epistemically independent” if that agent “keeps some distance” from and “is not excessively reliant” on what others think (Beerbohm Reference Beerbohm2012, 154).

Consider a variation of the earlier example. Suppose that Jim and Kate conducted their experiment together, but each wrote a separate, independent report of what they observed. While their write-ups were independent of one another’s, however, the experiment was same for both. Both were reporting on what happened in the same test tube.

When compiling the results of what happened in the lab that day, the team leader should give weight to both Jim’s and Kate’s reports. Each brought some independent judgment to bear on their report, unlike in the first version of the example. But the team leader should accord less weight to each of their reports than to the reports of other 18 scientists in the lab reporting on what happened in their different test tubes.

Such “common causes” are ubiquitous. Indeed, even the 18 scientists’ reports observing a different test tube cannot be said to be completely independent from one another. All of the scientists were working with the same batch of chemicals, which might have been contaminated. All were working at the same altitude, under the same atmospheric conditions. Those factors were common causes affecting the outcomes of all the scientists’ experiments at the same time (Dietrich and Spiekermann Reference Dietrich and Spiekermann2013).

Still, because each of those 18 other scientists’ reports was “independent conditional on all those common causes,” each report is epistemically worth something (and more than the reports from Jim or Kate, who were observing the same test tube). They are just not worth as much, epistemically, as the reports from scientists working in different labs, using different batches of chemicals and at different altitudes.

Recent contributions to epistemic democracy distinguish various ways of violating epistemic independence that can compromise decision making to a greater or lesser extent (Dietrich and Spiekermann Reference Dietrich and Spiekermann2013; Estlund 1984; Reference Estlund1989; Reference Estlund2008; Goodin and Spiekermann Reference Goodin and Spiekermann.2018, ch. 5; Grofman and Feld Reference Grofman and Feld1989; Waldron Reference Waldron1989). The general message is that agents’ reports are often likely to be subject to common causes of various sorts. But although this reduces the epistemic value of their reports, it does not vitiate it altogether. Just so long as those reports are independent conditional on all common causes to which they are subject, they will add something unique in their assessment of the situation. In other words, their assessments will be epistemically independent from one another, bracketing all the common causes that make them non-independent of one another.

This weaker form of independence is the most we can realistically aspire to in political life. People read the same newspapers, watch the same television programs, and read the same webpages; we cannot easily avoid having our judgments influenced by the same common causes. But even this weaker form of epistemic independence is undermined if people merely repeat what they heard, without any critical reflection on their part. That is an abnegation of independence conditional on all common causes, and it is at least in principle a wholly avoidable one (a later section suggests ways in which that can be done).

If, in contrast, people critically reflect on the content before passing it along, they are exercising judgment that is independent, conditional on all common causes (Goodin Reference Goodin2003; Waldron Reference Waldron1989). Their judgment is somewhat non-independent, insofar as they are subject to the same common cause (the same newspaper, TV program, webpage, or whatever); but it is somewhat independent, insofar as they reflect on the content before passing it on (Goodin Reference Goodin2003; Waldron Reference Waldron1989). Then their repetition of the original message itself contains some new information.

Partial Repetition

Sometimes we may actually have good epistemic reasons to repeat others’ claims, and hence our repetition then conveys some new information about that claim that our audience has not taken into account. Suppose I have good reason to regard you as likely to be correct because of what I know (but my hearers do not) about your training or track record. Or suppose I have reflected on your claim and find that, on the basis of my own knowledge, I concur with it. Then my reiteration of your claim conveys some new evidence about that claim’s epistemic credibility, as well as repeating evidence already on the table. Others taking into account my reiteration of your claim would therefore lead to something less than complete double-counting—although counting my reiteration fully on a par with the original claim would still amount to overcounting it, assuming they have taken your original claim fully into account.

Everything depends on whether my repetition is epistemically warranted in some way—whether it conveys some new, additional information that the others do not have, over and above that which is already contained in the original claim being repeated. Suppose that I repeat your claim x simply because I want to be in your good graces^{Footnote 21} with no epistemic reason for repeating it. Then my repetition would not contain any new, relevant information about x’s credibility, over and beyond the information contained in your original assertion of x.

Consider retweeting or reposting in this light. The way online platforms are set up allows us to instantly observe both that someone is simply repeating someone else’s assertions and whose they are. We should thus be able to tell if we have seen that message before, and to discount it or disregard it, if so. In that respect, we should be able to easily distinguish new information from old information.^{Footnote 22} What we cannot so readily know is what to make of the fact that someone has retweeted or reposted that message. Does that fact, that she retweeted or reposted it, contain new, independent information, over and above the information contained in the original message?

Maybe it does, if the retweeter or reposter knows something that we do not about the reliability of the original tweeter or poster, for example. Then we ought to count the retweet or repost for something, even if we do not weigh it as heavily as a wholly new piece of evidence. At least in the case of retweets or reposts, however, we can reasonably doubt that the repeated message will be based on new information, simply because the architecture of online interaction is set up in a way that rewards unreflective, impulsive behavior.^{Footnote 23} Most retweets, reposts, “shares,” or “likes”—that is, the sorts of clone claims we most typically encounter online—are probably not the result of any considered judgment about those claims’ credibility and do not contain any new, value-added information. Hence, it would ordinarily be wrong for people to take impulsive retweets or reposts as representing opinions that are based, at least in part, on new information not contained in the original tweet or post and that should be given some epistemic credit in their internal deliberations.

New Discursive Norms

The least intrusive approach would try to solve the problem through the adoption of a new set of deliberative norms by citizens and particularly by influentials. Those norms generically entail doing whatever one realistically can to avoid epistemically misleading other people. Perhaps ordinary citizens should be subject to a weaker form of that requirement, trusting that hearers, once adequately informed of the existence of repeated claims, would discount them. But influentials, by reason of their power to command large audiences, should be held to a higher epistemic duty of care not to help promulgate clone claims.

Citizens, Reveal Your Sources. The problem of epistemic double-counting arises from the fact that, all too often, we do not know whether someone is making a claim as a relatively independent epistemic agent (as a firsthand source) or as a relatively non-independent epistemic agent (as a secondhand source). If that is the problem, the most obvious and most modest solution is the voluntary adoption of deliberative norms requiring speakers to reveal (1) when they are simply passing along others’ messages, (2) from whom, and (3) for what reasons. This would give listeners the information they need to increase the accuracy of their internal deliberations. It would allow them to distinguish new from old information that they have already taken into account, thereby helping them to avoid double-counting or misattributions of epistemic weight to repeated claims.

Such deliberative norms should be a preferred solution insofar as they do not pose the same sort of threat to free speech that alternative solutions might do. These norms do not prohibit speakers from saying any particular things. They actually require deliberators to say more, not less.^{Footnote 26} Speakers would have to elaborate on the things they have already said, providing some background for the claims they are advancing—details they would not have otherwise provided.

On the downside, the speakers’ commitment to these norms is essential to the success of this approach, especially if the norms are only informal and are not supported by any enforcement mechanism. Doubtless some people will fail, at least from time to time, to internalize and act on these norms. Even those who, in principle, consider the norms useful and legitimate might find, in practice, that living by them proves a nuisance. Providing a full reference for every claim one makes in a deliberation may be burdensome and disturb the natural flow of conversation in an obnoxious way.

In their study of social networks and persuasion bias, DeMarzo, Vayanos, and Zwiebel (2003, 919, fn. 16) briefly consider this solution but promptly dismiss it.^{Footnote 27} They do so precisely on the grounds that such communication would be “extremely complicated”; indeed, “the information the agents would need to recall and communicate would increase exponentially with the number of rounds” of communication. To fully eliminate repetition, agents would have to know and report the full structure of their discursive network; that is, who/what was the source of their info, who/what was the source of their source, and so on. Furthermore, to abide by full-disclosure norms, the information that agents would need to communicate would increase with each discursive interaction, imposing an unreasonable burden on them.^{Footnote 28}

Fully countering clone claims in this way might thus be infeasible. But we might at least reduce their prevalence in public discourse by inculcating a norm of revealing one’s sources at least two steps back; that is, a norm of revealing one’s own source and the source’s source. Such a norm of limited disclosure would be much less of a burden on speakers’ memories and natural conversational flows.

Some might object that even such norms of limited disclosure might be too demanding and obtrusive to be adopted voluntarily as an informal social norm. The alternative would be for them to be formally adopted and socially enforced. Yet, in some environments the norms may be virtually unenforceable. The online sphere is one example, because of the sheer volume of communications that need monitoring. The social media platforms themselves would have to step in to monitor whether the norms are being respected and impose some kind of sanctions when they are breached. To be sure, they are already occasionally penalizing users for some offenses, but usually they can do so only by relying on other users to report the offense. And if the norms are seen as too demanding to begin with, and few users comply with them, then the rest of the user community may not bother reporting norm violations insofar as noncompliance is pervasive. One may well ask, then, whether it is naive to consider such deliberative norms as a solution.^{Footnote 29}

This objection, however, overlooks the interests and motivations that all speakers and listeners naturally have in conversation. Both listeners’ and speakers’ incentives are mutually compatible with the deliberative norms that I propose. Speakers have an interest in being listened to, and listeners have an interest in listening to new information. This means listeners will have an incentive to listen to information whose “newness” they can assess, in preference to information whose newness they cannot assess. If so, speakers who have an interest in being listened to will have an incentive to abide by the norms and reveal their sources, so that others will listen to what they have to say.

Thus, we should not be unduly skeptical about the uptake these deliberative norms would enjoy. Provided speakers care about being listened to, and listeners care about listening particularly to new information, there are incentives for all parties to adopt and abide by this modest version of the deliberative norms.

Influentials, Know Your Sources. It would suffice to hold ordinary citizens to a limited epistemic standard (insofar as it is practical to do so), because any given one of them has a limited capacity to influence public debate and distort collective rationality. But there may be reasons to hold influential public officials and prominent opinion leaders to higher epistemic standards, because they can influence a great many people at once: those whom we know to be boundedly rational agents who may not necessarily be able to keep track of the source (and the source’s sources) of everything they hear.

Non-independent agents like bots can come to dominate public debate when their claims are amplified by persons who have many followers, such as the U.S. president. As an avid retweeter, President Trump has routinely amplified tweets from dubious sources (Timberg, Dwoskin, and Entous Reference Timberg, Dwoskin and Entous2017).^{Footnote 30} He is not the only person to undermine public deliberation in this way, to be sure. Various other political figures, celebrities, and media personalities have amplified clone claims coming from the fake accounts run by Russian operatives (Timberg, Dwoskin, and Entous Reference Timberg, Dwoskin and Entous2017), making it more likely for citizens to come across what in fact are clone claims they can easily double-count, not being aware of their sub rosa common origin.

High public officials such as the U.S. president, media outlets, and other high-profile opinion leaders are in a privileged position to influence a great many people at the same time. Given their prominent epistemic position and the (apparent) epistemic authority they enjoy, they should be under a particularly stringent epistemic duty of care to exercise independent judgment and to pay careful consideration when passing along information from other unchecked sources, like bots—which makes their audience more vulnerable to double-counting. Some political theorists have already discussed at length the nature and extent of citizens’ epistemic obligations (Beerbohm Reference Beerbohm2012, ch. 6). Perhaps it goes without saying that these should extend, in an even more demanding way, to influential political decision makers whom we should expect not to be easily swayed by non-independent agents deployed in public deliberation at someone else’s command.

Minimally, the norms of public deliberation ought to impose a special epistemic duty of care on influentials to “know their source”—and, as in the banking analogy, not to accept things from dubious sources when passing along information to their many followers.^{Footnote 31} With power comes responsibility. There is much to be said in favor of imposing such duties, particularly on prominent public persons, as safeguards against the falsification of public opinion, the corruption of public deliberation, the distortion of collective reasoning, and ultimately the undermining of popular sovereignty.

Not only are there good epistemic reasons to do that but there are also good democratic reasons. Allowing claims made by non-independent agents to infiltrate our deliberations is likely to increase existing power imbalances and exacerbate structural inequalities. Powerful and resourceful agents will be the first to try to amplify their voices by using other non-independent (human or nonhuman) agents as mouthpieces. There are myriad examples of powerful lobby groups and individuals (from tobacco companies to Monsanto to the Koch brothers to President Putin) resorting to such tactics—whether astroturfing,^{Footnote 32} deploying bots and trolls, or financing scientifically dubious research—to increase their power or protect their advantaged status (Oreskes and Conway Reference Oreskes and Conway2010). They all try in those ways to exaggerate the weight and support of their claims.^{Footnote 33} The deployment of epistemically completely non-independent agents may distort our internal deliberations to the point where our individual judgments and ensuing collective decisions are not only epistemically flawed but also can no longer genuinely claim the citizens’ meaningful consent. At least in principle then, epistemic hazards can have serious political consequences, especially when we are dealing with problems (complete epistemic non-independence) and processes (internal deliberations) that are invisible to the naked eye.

How exactly can such norms requiring influentials to know their sources and not pass on dubious information stick? One solution would be for online platforms to enforce them as part of their “conditions of use.”^{Footnote 34} Or, perhaps, they could be merely enforced by social pressure as in the previous section. Even in the absence of external enforcement, however, assuming once again that listeners have an interest in distinguishing good quality information from bad, and that influentials have an interest in being listened to, then influentials will naturally have an incentive to reassure their listeners as to the quality of the information they communicate. More specifically, influentials have an interest in (1) checking carefully the origin of information they intend to promulgate and the credibility of its source, before passing along that information to their listeners, and (2) signaling clearly to their listeners that they have done epistemic due diligence in this respect. The norms requiring influentials to exercise a special epistemic duty of care would thus be fully compatible with listeners’ and speakers’ interests.

Excluding or Removing

In addition to these suggested discursive norms, two other approaches may counteract epistemic overinclusion. Both promise to be equally epistemically effective in preventing clone claims from impinging on our internal deliberations, although they come with differential costs in other respects. The first, more extreme version works by banning anyone voicing any such claims from public sphere discussions altogether. The second, less extreme approach removes clone claims from the deliberative space. The first approach bans the speaker, whereas the second bans only the specific offending clone claim.

Both come with democratic costs. The former approach amounts to an act of social exclusion, banning a person from the public forum altogether,^{Footnote 35} which would be democratically unacceptable (at least in the case of real people, if not bots). The latter approach amounts, at most, to a violation or a limitation of free speech. That too would generally be regarded as democratically problematic, although perhaps less so once we realize how these limitations are similar to others we already allow and consider legitimate.

Excluding the Offenders. The first exclusionary approach is one that social media networks such as Twitter and Facebook have begun implementing. They have started, albeit hesitantly and belatedly, weeding out fake user accounts associated with bots that mimic real, independent citizens participating in public deliberation (Wu Reference Wu2017). Considering the magnitude and political implications of clone claims coming from these accounts recently and their potential for undermining internal deliberation and collective rationality, that is surely good news.

Although their exact number is unknown, existing estimates indicate that a large number of non-independent agents are currently influencing public debate. On some calculations, as many as 48 million Twitter active users or “nearly 15 percent—are automated accounts designed to simulate real people.” Facebook has disclosed that it may have many automated accounts; its initial estimate in 2017 was 60 million,^{Footnote 36} but in November 2018, Facebook announced that it removed more than one billion fake accounts, many of them bots. That shows that online platforms have the capacity to identify and remove bot accounts (Romm and Dwoskin Reference Romm and Dwoskin2018).

There are at least two different problems with managed bots: they are spreading deliberately false information, and they are mimicking one another in spreading the same false information. Both obviously pose epistemic problems, but it is the latter that gives rise to risks of epistemic double-counting and overcounting.

While some are individual accounts (e.g., the account of citizen John Doe), a number of them pretend to represent collective organizations or groups, hence larger numbers of individuals. One example is @Ten_GOP, an account claiming to speak for Tennessee Republicans that was recently shown to be set up by Russian operatives (Timberg, Dwoskin, and Enous Reference Timberg, Dwoskin and Entous2017). This type of fake accounts claiming to represent interest groups have a greater potential of undermining internal deliberation by giving the impression that a view is endorsed by a myriad of independent epistemic agents—when in fact it can be traced to a sole agent. The phenomenon of “followers” is similar. Many real users take pride in having large numbers of followers, who are supposedly citizens independently associating themselves with the user’s opinions. Yet in fact many are bots, deployed to make it look like the user’s opinions are epistemically weightier than they really are (Confessore et al. Reference Confessore, Dance, Harris and Hansen2018). Buying bots and fake user accounts has unsurprisingly become a profitable business.

Removing fake automated user accounts is minimally problematic democratically. Excluding real human agents from public deliberations (online or otherwise) is much more so.^{Footnote 37} Banning citizens^{Footnote 38} from public debate on the grounds that they engage in message repetition would constitute a serious violation of their civic rights^{Footnote 39}—rights that robots do not have.^{Footnote 40} Hence we should look instead for other ways to counteract the negative effects of message repetition, options that do not infringe on citizens’ freedoms or gravely stifle public debate.

One such solution is suggested by my earlier argument that the online architecture structuring public debate rewards impulsive rather than reflective behavior, which makes it unlikely that humans’ online message repetition is the result of much independent judgment. Furthermore, even if users can be aware of every single instance of message repetition occurring via retweeting or reposting, they may well be unable to keep track of all instances of message repetition they come across and to remember all claims and their firsthand sources as they have been disclosed to them. Both factors make people vulnerable to the exposure effects of message repetition, causing them to overestimate the epistemic weight of particular claims, simply because their frequency creates the illusion that they are widely endorsed, when in fact they are not.

To remedy this, we could simply disable retweeting or reposting, thereby removing one all-too-easy way for citizens to clone others’ claims. Or we could flag to the audience those users whose content contains a large proportion of repeated claims: just as we display the number of followers, for example, we could display the proportion of retweets/reposts of the user’s total number of tweets/posts. At the very least, this would put their readers on notice that their content might well be unreliable. This might also disincentivize citizens from behaving non-independently, insofar as a large number of reposts or retweets would signal to the audience that a user does not post original content and therefore might not be worth “following.” Displaying the count of repeated messages might thus undermine the profiles of agents who are incorrectly perceived by the masses as “opinion leaders”—whose, by definition, independent judgment deserves particular attention—but who in fact are “followers” themselves, because most of their content comprises retweets or reposts.

Removing Claims. The second exclusionary approach to counteracting clone claims would be to simply expunge such claims from the deliberative environment. In the online space, this would require removing certain contributions.^{Footnote 41} Just like the previous approach, this approach would also impinge on the democratic value of freedom of expression, although less so, because what is being banned is the post or claim rather than the person making it. We might find even that practice problematic at first, on grounds of free speech. But perhaps it might appear less problematic if we came to see the contamination of public discourse through the promulgation of clone claims as being epistemically analogous to “falsely shouting fire in a crowded theater.”

Some restrictions on freedom of speech are considered legitimate by courts and political theorists alike. In the judgment just alluded to, Supreme Court Justice Oliver Wendell Holmes argued that limitations on free speech are legitimate, where speech is bound to have dangerous consequences.^{Footnote 42} J. S. Mill goes further, arguing that if one’s statements (true or not) will cause harm to others, then it is acceptable for the state to limit one’s liberty to express them at least in certain circumstances.^{Footnote 43} Clone claims could be viewed in an analogous light.

Even where their content is true, clone claims are nonetheless deceptive. They mislead the public into attaching more confidence to those claims than is epistemically warranted, given that they are copycat claims. Corrupting other people’s judgment—or even worse, a nation’s collective judgment—constitutes a grave moral and epistemic harm. Depending on how debilitating its effects are for citizens and their interests, we might consider the making and propagation of clone claims epistemically analogous to causing bodily harm, and we might, on that basis, justify limitations on the freedom of such speech in special cases as well.