The Dilemma of Disinformation Regulation: Two Competing Ends

In July 2023, Taiwan’s Ministry of National Defense issued a public clarification in a press conference: no, its annual Han-Kuang drill was not a ‘rehearsal’ for the then-President Tsai to ‘flee’ to the United States (Wang Reference Wang2023). Fact-checking organizations traced the viral claim and found that the original post, which alleged Tsai of ‘intending to flee with Taiwanese assets’ after ‘provoking the Chinese government’, came from Weibo and resurfaced on Facebook every year during the drill (MyGoPen 2021; Taiwan FactCheck Center 2023). Similar fabrications have unsettled the democratic world. For example, AP News now has a dedicated page of ‘Not Real News: A look at what didn’t happen this week‘. One recent entry confirmed that – no, the federal government does not give each migrant a $5,000 gift card at the border. Another one verified that – no, there was not a crowd chanting ‘F–k Joe Biden’ at a football game. The video has been digitally altered to insert the chant (Goldin Reference Goldin2023).

Disinformation can take various forms. Common examples include false connections (the headlines or visuals do not match the content), false context (genuine content is shared with false contextual information), imposter content (genuine content is impersonated to mislead), and fabricated content (the story is completely made up to deceive) (Wardle Reference Wardle2020). A 2021 European Parliament report catalogs a few common strategies of disinformation actors: spreading false content through automated bot networks, masking content sponsorship (to give the false impression that it comes from real grassroots activists), impersonating authoritative media and public figures, and digitally altering or fabricating images and videos (Colomina et al. Reference Colomina, Sánchez Margalef and Youngs2021).

For definitional clarity, this article joins the growing body of media studies to identify disinformation as deliberate false messages intended to mislead and deceive, which differentiates it from unintentional falsehoods as misinformation (Freelon and Wells Reference Freelon and Wells2020; Jack Reference Jack2017; Wardle Reference Wardle2020; Wardle and Derakhshan Reference Wardle and Derakhshan2017). Specifically, I refer to disinformation as false content deliberately created to mislead and manipulate the political judgments of its audience. This means that a piece of disinformation contains at least three elements: (i) the speaker believes that a statement is factually erroneous, (ii) the speaker consciously presents as what they believe to be factually true, and (iii) the purpose of this misrepresentation is to misguide the audience’s political reasoning.Footnote ^{1 ,} Footnote ² Although the third element is not always included in the definition of disinformation studies, there is no doubt that most disinformation campaigns intend to impact political events (for example, Colomina Reference Colomina, Sánchez Margalef and Youngs2021; Engler Reference Engler2019). Disinformation raises questions in normative political theory, not because it misinforms listeners on any random matters. Rather, as Bennett and Livingston (Reference Bennett and Livingston2020, 3) point out, disinformation actors aim to ‘advance political goals’, and their strategies often involve ‘discrediting opponents, disrupting policy debates, influencing voters, inflaming existing social conflicts, or creating a general backdrop of confusion and informational paralysis’. This article thus follows Bennet and Livingston’s observation and limits its focus to misleading content that has a political orientation. The scope of analysis thereby excludes, for example, commercial marketing ploys or a friend’s mischievous pranks.

Social media and internet platforms allow information to be transmitted at an unprecedented rate, making democratic discourse vulnerable to manipulation. Content regulation (the removal of false content or fake accounts, specifically) has since become a common defensive tool. In the United States, such measures are largely voluntary initiatives undertaken by private platforms in the absence of statutory mandates. By contrast, the United Kingdom’s Online Safety Act 2023 obliges platforms to curb disinformation and misinformation following the directives of Ofcom (Coe Reference Coe2024). France’s ‘bills against manipulation of information’ empower electoral candidates to seek judicial removal of false content that could compromise electoral integrity, with judges required to issue rulings within forty-eight hours (Craufurd-Smith Reference Craufurd-Smith2019). While these interventions can impede the spread of deceptive materials and enhance the costs of disinformation campaigns (Clifford and Powell Reference Clifford and Powell2019; Fielitz and Schwarz Reference Fielitz and Schwarz2020), their legitimacy remains ambiguous.

In legal studies, opinions are divided over the constitutionality of falsehood regulation (Bromell Reference Bromell2022; Samples Reference Samples2019; Susskind Reference Susskind2022). In the United States, critics argue that any governmental intervention infringes the First Amendment, whereas others hold that a careful reading of Section 230 of the Communications Act could justify a more expansive role of the government (Franks Reference Franks, Brison and Gelber2019). In Europe, legislative restrictions on speech appear more readily embedded in constitutional frameworks. Article 10 of the European Convention on Human Rights clearly states that exercising the freedom of speech ‘carries with it duties and responsibilities, [and] may be subject to such formalities, conditions, restrictions or penalties as are prescribed by law and are necessary in a democratic society’. Nonetheless, legal disputes persist. For example, in 2022, Germany’s Court of Cologne ruled that its Network Enforcement Act (NetzDG, which requires social media platforms to moderate false information) was in partial violation of the EU law – in part because the Federal Office of Justice was deemed not ‘independent enough’ to supervise media services (Loebbert Reference Loebbert2022). Despite extensive discussions in legal literature, a focused normative investigation remains lacking.

On the normative grounds, two interrelated questions beg more clarity: how does disinformation differ from other false accounts, and does the difference justify its removal? Without consistent principles that distinguish disinformation from other false expressions, even well-intended regulation can end up censoring speech that ought to be protected. In fact, opponents of regulation often question whether a consistent distinction is even tenable. The CATO Institute argues that the definition of disinformation is too vague to translate into regulating guidelines (Samples Reference Samples2019). Even parties that see regulation in a more positive light, such as the European Parliament, conceded that ‘It is not yet clear where the dividing line is between legitimate political persuasion and illegitimate manipulation of thoughts’ (Colomina et al. Reference Colomina, Sánchez Margalef and Youngs2021). And yet, as our political life is increasingly mediated by digital platforms, false speech has a much higher chance of inflicting systemic harm. Outright rejections of interventions can render democracies defenseless against co-ordinated attacks. Addressing the threats of disinformation, therefore, requires a careful re-articulation of the boundaries of free speech in contemporary contexts.

If falsity alone is insufficient to justify removal, to answer the question of legitimacy, we need to first examine what is at stake when liberal democracy does not protect false speech. In the next section, I review three principal justifications for free speech and evaluate how each responds to the challenge of disinformation.

Free Speech, False Opinion, and Lies

Free speech is often defended for three primary values: the pursuit of truth, individual freedom, and the functioning of a democratic state (Barber Reference Barber1984; Brison Reference Brison2021; Bromell Reference Bromell2022; Moon Reference Moon and Macfarlane2021). In other words, speech is not protected simply to have ‘more speech’, but because its protection brings some essential goods for liberal democracy (Brison Reference Brison2021). Hence, a defense of free speech must explain how it advances the targeted value in a given context. This section reviews three free speech defenses in the context of disinformation. I then argue that the democratic defense is the one that poses the strongest objection to falsehood regulation – and yet, even on this account, disinformation as deliberate falsehoods still falls outside the ambit of protection.

Truth-seeking is one of the most commonly referenced reasons to defend expressive freedom in liberal societies. John Stuart Mill (Reference Mill, Philp and Rosen2015[1859]) famously argues that even factually erroneous opinions should be freely aired, since public debate and deliberation may expose neglected truths. In the same spirit, Justice Brandeis characterizes free speech as a ‘means indispensable to the discovery and spread of political truth’, and insists that the remedy for fallacies is ‘more speech, not enforced silence’ (Whitney v. California Reference Whitney1927). However, this truth-seeking optimism is subject to empirical challenges. Recent studies find that false beliefs have a strong stickiness in our belief systems (Baekgaard et al. Reference Baekgaard, Christensen, Dahlmann, Mathiasen and Petersen2019; Gilbert Reference Gilbert1991; Kunda Reference Kunda1990; Taber and Lodge Reference Taber and Lodge2016). Some experiments even show that counterspeech can entrench rather than dislodge errors (Nyhan and Reifler Reference Nyhan and Reifler2010; Lewandowsky et al. Reference Lewandowsky, Ecker, Seifert, Schwarz and Cook2012).Footnote ³

Taking account of today’s media environment, Wu (Reference Wu2018) notes that the First Amendment was written in a time when speech was highly censored and scarce, and many of its assumptions no longer hold. Now the amount of speech often overwhelms individuals and impedes meaningful engagement with one another. Similarly, Napoli (Reference Napoli2019) points out that online platforms do not provide the conditions for truth discovery as Millians envisioned, since the filter bubbles and algorithms rarely expose users to diverse opinions or motivate high-quality debates.Footnote ⁴ This means that if truth were the main concern, tolerating blatant falsehoods can be counterproductive. Restrictions on disinformation can potentially relieve information fatigue and create space for productive discussion. Therefore, the truth defense of free speech does not seem to give definitive reasons to refuse to limit disinformation.

Another common defense frames free speech as an essential means to self-realization. Indeed, the ability to verbalize internal thoughts is central to personal freedom. But the right to free speech is never absolute. Defamation, fraud, and false advertising are well-established domains where deception incurs legal sanction. On the other hand, as social media and new technologies enable new expressive activities, it can be argued that we never had the ‘liberty’ to broadcast our words to a global audience. Regulating online spaces is thus different from policing in-person conversations. Given the level of influence of online platforms, one can argue that the scope of expressive freedom in online spaces should be more restrictive.Footnote ⁵ No matter where the judgment falls, unless one endorses free speech absolutism, it is well recognized that expressive freedom can be subject to reasonable constraints. For those who seek to contain disinformation, the ‘freedom’ to disseminate deceptive information is simply not a meaningful one to protect. Especially when regulation targets online platforms and leaves expressions in other domains intact, the self-realization defense does not seem to give sufficient reasons to deny content regulation.Footnote ⁶

Finally, free speech is defended as a component of democratic life (for example, Post Reference Post2011). The freedom to articulate one’s views and convictions allows citizens to communicate with one another. It enables the key functions of collective self-government, such as agenda-setting, the defense of interests, negotiation, and persuasion (Barber Reference Barber1984, 178–79). Post (Reference Post2011) maintains that an opinion can be factually erroneous, but having the freedom to express it and impact public discourse is what affirms a person’s equal standing in the collective decision-making process. Free speech is therefore constitutive of democracy. Sanctioning some viewpoints while permitting others jeopardizes the political equality of citizens (Heinze Reference Heinze2016). Through these lenses, the democratic defense gives rise to a compelling objection to falsehood censorship. If the purpose of free speech is to allow citizens to communicate freely as political equals, factual accuracy alone is not a legitimate reason to deny protection. Restricting false content online will deprive individuals of equal opportunities to participate in the formation of public opinion and to influence collective decisions in the way they see fit.

Yet condoning the democratic harm of disinformation can run into what Bonotti and Seglow (Reference Bonotti and Seglow2022) call the problem of ‘internal balancing’. That is, safeguarding the speaker’s right to utter problematic information can preclude others from enjoying the same freedom effectively. In reply to Post, Brown (Reference Brown2015) points out that hate speech often silences victims and undermines their ‘fair and real opportunities’ to impact public opinion. If equal rulership is what is at stake, democracies might have a positive reason to limit hate speech rather than leaving it unchecked (Brown Reference Brown2015, 194–96). In a similar vein, extensive deceptive information can counteract the democratic purpose of free speech – to allow meaningful exchanges between citizens in collective governance.

I argue that breaking this impasse requires that we draw a fine line between respecting diverse opinions and being indifferent to all kinds of non-facts. In this regard, Hannah Arendt’s Truth and Politics offers a helpful roadmap. On the one hand, Arendt (Reference Arendt and Kohn2006 [1967], 232–36) insists that factual truth is unshakable, independent of interpretation, and distinct from opinion. The stubbornness of factual truth is manifest in a statement such as ‘Germany invaded Belgium in August 1914’, whose validity can be affirmed even by ‘the most extreme and sophisticated believers in historicism’, and a contrary claim is ‘clearly an attempt to change the record’ (Arendt Reference Arendt and Kohn2006 [1967], 235–45). On the other hand, Arendt (Reference Arendt and Kohn2006 [1967], 236) warns that truth ‘peremptorily claims to be acknowledged and precludes debate, and debate constitutes the very essence of political life’. This is not to say that truth stifles democracy. Quite the contrary; for Arendt, truth is what underpins the formation of sound opinion. At the same time, political communication requires more than stating facts, but also making such facts relevant to our community.

Imagine that one says, ‘It rains’. On Arendt’s account, the meteorological fact that ‘water drops from the sky’ will not have any public bearing unless it gives rise to a shared implication for the community (that ‘the river wall needs repair’ or ‘the water-restriction policy should end’). Arendt acknowledges that in forming an opinion, ‘our thinking is truly discursive, running . . . through all kinds of conflicting views’, and some opinions will be mistaken. But an opinion is always an attempt to communicate to our fellow members: this is how ‘I’ see the world (Arendt Reference Arendt and Kohn2006 [1967], 236–37). Re-reading Arendt, Zerilli (Reference Zerilli2016, 265) thus stresses that having diverse opinions is the foundation of a democratic life – a democratic common world emerges only ‘when there is a plurality of worldviews’, since ‘[o]ur sense of what is common, “the sameness of the object,” can appear only when it is seen from multiple perspectives’.

Competing claims naturally arise. Two individuals can witness the same rain, and one attributes it to supernatural power, the other to meteorology. Arendt offers no conclusive formula for reconciling opinions (that might or might not be informed by truth), but she does suggest that ideal opinion formation involves envisioning oneself in the standpoints of others (Arendt Reference Arendt and Kohn2006 [1967], 237). The more positions that one can inhabit, the better quality the opinion is. The implicit lesson here, I argue, is that effective political communication requires a willingness to move beyond material disagreements and recognize the lived experience that an interlocutor seeks to convey. By drawing on Arendt, I do not mean that citizens must be able to practice this ideal mode of communication at all times. Nor do I imply that every opinion deserves equal intellectual weight in political reasoning. What I mean to stress is that since opinions are the vehicles of personal beliefs and preferences, members of a democracy only get to create a shared world with others as political equals when their opinions can enter freely into the public realm and be subject to consideration or objection.

Recognizing this, it is only reasonable that Arendt draws a sharp distinction between opinion and lie. If the equal moral weight of an opinion stems from respecting a speaker’s shared authority in collective governance, deliberate falsehoods lose their bearings. Regardless of its empirical accuracy, an opinion is an attempt to describe ‘how things appear’ to one’s eye. A lie, by contrast, does not ‘reflect upon personal truthfulness’ and, instead, is a conscious misrepresentation of the speaker’s view (Arendt Reference Arendt and Kohn2006 [1967], 245). While self-deception is always possible, disinformation does not fall into that category. Rather, as deliberate falsehoods created to mislead, disinformation belongs to what Sissela Bok (Reference Bok1978, 17) classifies as ‘clear-cut lies’ – whose ‘intention to mislead is obvious, where the liar knows that what he is communicating is not what he believes, and where he has not deluded himself into believing his own deceits’.

Revisiting Arendt’s account allows me to draw two implications for this inquiry. First, indeed, as Post contends, the freedom to participate in the formation of public opinion is what affirms one’s equal authority in democratic governance. But the purpose of this freedom is to ensure that personal viewpoints and convictions can enter the public realm and shape collective decisions. A deceptive message that carries no authentic views does not give rise to the same moral claim. Second, if we take free speech’s democratic function seriously, democratic government may even have a positive duty to protect collective decisions from outright manipulation. Bok (Reference Bok1978, 20) has a sharp description of how lies undermine our agency as decision makers: ‘All our choices depend on our estimates of what is the case; these estimates must in turn often rely on information from others. Lies distort this information and therefore our situation as we perceive it, as well as our choices’. In the same way, disinformation deprives individuals of the chance to make meaningful choices among viable options.

Here, Shiffrin’s critical analysis of the relationship between free speech and lies offers a valuable lens to analyze the threat that disinformation poses to citizens’ political agency. Shiffrin (Reference Shiffrin2014, 82–87) argues that dominant free speech theories tend to only emphasize our interests as listeners (who seek unrestricted access to information) or speakers (who wish to express without interference). What these frameworks neglect, she contends, is our interests as thinkers – who listen in order to process and apply information, and who speak, not merely to express, but to formulate and update our thoughts through the conversations with others. On this account, thinking, the ability to interpret interactions and develop judgment, is what unites the purpose of both listening and speaking. By feeding others with untruthful information, what lies erode is a person’s moral agency as a reflective thinker (Shiffrin Reference Shiffrin2014, 23–24, 90–92).

Departing from Shiffrin, I do not claim that free speech is only meant to serve our role as thinkers. What I mean to stress is that individuals are listeners, speakers, and thinkers all at once. And these roles can come into tension. Protecting a speaker’s right to disseminate falsehoods can compromise others’ ability to receive credible information and exercise judgment. In the case of disinformation, deceptive speech denies others as co-founders of a common world by distorting the epistemic basis on which their judgments rest. In this light, free speech becomes counterproductive if it only ensures participation in public discourse without enabling meaningful collective governance. Regulating disinformation is not only compatible with free speech’s democratic function, but also desirable – if we are to protect authentic exchanges in the creation of a shared world.

Toxic Persuasion and Norm-Disrupting Speech

I have argued that disinformation differs from factually false opinion since it does not convey authentic personal beliefs and, therefore, is not entitled to protection. However, if a speaker’s authenticity is what determines the normative status of false speech, a further question arises: where would persuasive rhetoric fall within the very same framework? To mobilize an audience, political speakers often knowingly simplify complex events, exaggerate the benefits of a policy, or downplay anticipated costs. These tactics very much blur the line between an honest opinion and a lie. Nonetheless, persuasion plays an important role in political speech, and it seems troubling to conflate it with disinformation. In this section, I differentiate disinformation, as norm-disrupting speech seeking to disturb public deliberation, from ordinary persuasion. I argue that, although both disinformation and persuasion can misrepresent how the speaker perceives the objective world, only the former warrants restrictions.

Some persuasion strategies are indeed toxic for meaningful deliberation. In Gorgias, Plato famously warns that mobilizing rhetoric can conceal a speaker’s convictions. It is a ‘knack’ rather than a ‘craft’, ‘one that perpetrates deception by means of shaping and coloring, smoothing out and dressing up’ (Plato and Zeyl Reference Zeyl2012 [1987], 24–25). Such strategic rhetoric seeks to mobilize persuadees without full disclosure of the speaker’s underlying rationale. For similar reasons, Keith Dowding (Reference Dowding2016, 13–14) argues that non-manipulative persuasion is only possible when the speaker and the listener both agree on some criteria on how to verify the evidence in a persuasive claim, even at some abstract level. Absent such an agreement, a prompted behavior change nonetheless involves manipulation since the speaker will have to appeal to reasons that they do not genuinely share to mobilize the listener. (For example, imagine a science believer convinces a magic believer to take a pill not because of its scientific benefits but because of its superpower.)

It is helpful to identify the moral defects in persuasion when striving for better communication. But one might question whether a ‘purified’ mode of communication is achievable, or even desirable, for our political life. As Markovits (Reference Markovits2006, 258–59, 262) points out, strategic language is a natural human experience: ‘In different contexts, the same statements can mean very different things; the same delivery affects different audiences in diverse ways [. . .] and the wrong voice was unduly limit the prospects of being heard’. Not to dismiss the potential harm of toxic persuasion, its moral murkiness, as I see it, is that by presenting something consistent with the listener’s pre-existing view, the speaker bypasses a conscious (and often critical) evaluation process of the audience, leaving their genuine reasoning unexposed and unexamined. And yet, if democratic communication values two-way exchange rather than one-way preaching, strategic rhetoric can actually help to explore common grounds and enhance mutual understanding when it makes audiences feel more seen and heard. More importantly, conflating persuasion with disinformation overlooks important differences between them and the normative issues that ensue.

Indeed, common persuasion can involve conscious overstatement or understatement, such as ‘we’ve created more jobs than any government in the last two years’ or ‘a thousand billionaires only pay 3 percent in taxes on average’, when the data only covered 400 families, or oversimplification and ambiguity, such as ‘voting for me is voting for gender equality’. But are these claims morally indifferent to a statement such as ‘a suitcase of ballots was brought into the polling station’ or ‘the federal government is giving immigrants $5,000 gift cards’? The key distinctions, I maintain, lie in the way in which the speech is used to penetrate the political world and the way that the speaker responds to the shared norms of public discourse. When persuasion, rather than deception or manipulation, is the primary goal, a speaker usually avoids completely baseless claims and is willing to provide evidence or respond to questions when challenged. In contrast, a disinformation speaker tends to (i) make baseless assertions without credible evidence, (ii) refuse to justify or correct claims when pressed, and (iii) urge listeners to disengage from other sources and rely solely on the speaker’s authority.

To illustrate these distinctions more concretely, consider first the content of a narrative. Toxic persuasion tends not to be completely baseless and can retain a measure of validity in certain contexts. Imagine a self-proclaimed feminist candidate making a case that ‘voting for me is voting for gender equality’. While exaggerated, such a claim may still be reasonable if the candidate has paid sustained attention to related issues. The situation is very different when, for example, during the campaign period of the marriage equality referendum in Taiwan, opponents of same-sex marriage asserted that if the new legislation is passed, ‘we are no longer allowed to call our mom “mom” or our dad “dad’‘, and that these familial terms would be replaced with gender neutral “parent 1” and “parent 2“’ (Liang Reference Liang2016). This kind of statement has no grounds in any given context. The claim may be somewhat less troubling if there were precedents of policing this language, which was not the case.

The second distinction concerns the speaker’s responsiveness to questions or challenges. A persuader might employ a selective or misleading framing. But we can generally expect them to provide further justification, or even correction, when pressed. For example, in defending a tax policy, a persuader might highlight only favorable statistics yet still be willing to explain how the numbers were calculated, reference credible sources, or acknowledge mistakes. Whereas a disinformation speaker often remains unresponsive, simply reiterates the same claim, or supplies fabricated ‘evidence’. This question-avoidant attitude also marks the third distinction: while a persuader might selectively cite evidence in favor of their view, they are generally willing to respond to counter-sources and explain the discrepancies. Disinformation speakers, on the other hand, tend to delegitimize all the rival claims, calling them fake or politically tainted, and urge listeners to disengage from other sources and rely exclusively on the speaker’s authority.

Beerbohm and Davis (Reference Beerbohm and Davis2023) describe this latter conduct as a form of epistemic abuse (what they call ‘political gaslighting’) – in which the speaker induces the audience to suspend cognitive exercises in their belief-forming process, and makes them reliant on the speaker’s authority for political judgment. Speech in this manner is different from mobilization. While both mobilizers and gaslighters may present information selectively, it is the latter who attempts to limit listeners’ thought process, discourage engagement from alternative sources, and tether ‘the listener’s loyalty’ to the speaker and political leaders (Beerbohm and Davis Reference Beerbohm and Davis2023, 875). Beerbohm and Davis (Reference Beerbohm and Davis2023, 874) also suggest that persuasion can be somewhat ‘innocuous’ when one can ‘reasonably expect’ that the speaker may ‘spin on the facts’ for a given purpose. In their example, a swimmer might anticipate that their coach will exaggerate the odds of winning as a motivational strategy; the exaggeration becomes less problematic when there is a shared understanding that speech may be used in this way.

Likewise, in our political life, misrepresentation is often anticipated. We thus expect public speakers to be ready to face questioning. Under this premise, misleading persuasion becomes less troubling, not because dishonesty is blithely welcomed, but because every public speech is subject to a process of scrutiny. Challenges, debates, and interrogations are the cornerstone of political communication, since it is only through these mechanisms that citizens test a speaker’s truthfulness, with a shared understanding that some lies will inevitably sail under the radar. While what constitutes a credible justification is always subject to debate, this process becomes meaningless if the speaker simply evades the scrutiny altogether – repeating baseless claims, dismissing all counter-evidence, refusing to answer questions, and urging audiences to detach from competing information. In doing so, the speaker not only avoids justifying their claims but also denies the norms of public discussion. It is in these respects that disinformation, as norm-disrupting, manipulative speech, differs from ordinary persuasion.Footnote ⁷

By making these arguments, I do not suggest that there is a perfect system to differentiate disinformation from toxic persuasion. Some cases will inevitably fall in the gray area. However, there are two implications that this analysis seeks to foreground. First, these criteria help to resist the relativist tendency that equates speech with very different orientations as morally indifferent. Second, disinformation, particularly as norm-disrupting, manipulative speech, is not entitled to the same protection as persuasion or mobilization. Taken together, my analysis is meant to show that the differences between persuasion and disinformation warrant not only conceptual separation but also differentiated normative treatment.

Indeed, this section suggests that distinguishing between toxic persuaders and disinformation disseminators requires observing the course of action of a speaker. As indecisive as this sounds, this is also how the court assesses problematic speech, such as libel or slander: by examining how the speech in question is deployed in a context. Democratic decision making is a multistep process. To detect a manipulator hence requires looking closely at how speech is employed to serve the speaker in a sequence of events. An obvious example is when the speaker self-contradicts, such as denying the safety of vaccines but receiving them secretly. Seeing speech as a political act, a disinformation speaker often reveals that their goal is not to persuade on a matter of genuine concern, but to mislead – by discrediting opponents, delegitimizing other sources of authority, their primary goal is often to obstruct public communication.

When disinformation is widely condoned, it generates democratic costs beyond distorted policy choices; it leads to the paralysis of the public sphere. In Truth and Politics, Arendt (Reference Arendt and Kohn2006 [1967], 246) offers a vivid foresight of what a society dominated by ‘organized lying’ looks like: ‘The result of a consistent and total substitution of lies for factual truth is not that the lies will now be accepted as truth, and the truth be defamed as lies, but that the sense by which we take our bearings in the real world – and the category of truth vs. falsehood is among the mental means to this end – is being destroyed’. Pervasive disinformation breeds what McKay and Tenove (Reference McKay and Tenove2021, 708) call ‘epistemic cynicism’, where citizens lose trust in any information and withdraw from public discussion. If enabling democratic communication is the primary reason we protect false speech, reducing the amount of disinformation is, in fact, crucial to preserving the very space where political discourse takes place.

Muting the Liars: A Democratic Model of Regulation

I have argued that disinformation differs from factually false opinion and toxic persuasion and thus falls outside of the scope of free speech. Nevertheless, to say that restricting a category of speech does not inherently infringe on expressive freedom is different from saying that a specific regulation is legitimate. Designing appropriate regulation raises a separate set of concerns. Given how fluid and ubiquitous speech is, the primary challenge for policy makers is to ensure that disinformation controls do not stray beyond legitimate targets and suppress other utterances. Since different constitutional frameworks allow different forms and degrees of interventions, the goal of this section is not to offer jurisdiction-specific prescriptions, but rather, to articulate the normative considerations behind institutional design and suggest potential responses. Following earlier theorization, I propose a ‘reduction model’ and a two-track identification system to identify content that can be subject to removal. While I leave the choice of regulating entity open, I outline factors that favor legislative interventions.

Disinformation poses a serious threat to democratic governance. However, if what distinguishes disinformation from false opinion and toxic persuasion is its deceptive orientation, accurate detection seems to require reading the speaker’s mind. A speaker’s genuine intent can be murky and disguised, and misjudgments can end up censoring legitimate speech and endangering political communication. For this reason, I follow Susskind’s suggestion (Reference Susskind2022) that regulatory models should aim to reduce the amount of disinformation rather than eliminate it.Footnote ⁸ More precisely, I argue, a reduction model should make two important concessions. First, it should recognize that detecting every piece of disinformation is impossible, and neither should this be the goal. Reduction rather than elimination is a necessary trade-off to protect expressive freedom. Second, removal decisions should rest on active indicators of manipulation. This means that the range of content that can be legitimately removed will be smaller than the actual scope of disinformation.

How exactly does this reduction model operate? My suggestion is that a regulatory framework should distinguish between two categories of information and assign different procedures. The first category includes materials where manipulation is self-evident: a bot account, an edited video, a fabricated news headline, or a doctored image. In addition, since disinformation is usually generated by malicious entities, there are often traces of co-ordinated attacks. Security analysts and fact-checking companies often use Coordinated Inauthentic Behavior (that is, multiple inauthentic accounts pushing identical content), IP tracing, or linguistic traits to identify a connection with disinformation campaigns (for example, Rastogi and Bansal Reference Rastogi and Bansal2022). In these cases, the evidence of distortion is explicit, and the content does not represent sincere belief. Here, media platforms or regulators can proceed with removing the content.

In contrast, some information can be completely inaccurate or baseless, but the signs of manipulation are not immediately discernible. This kind of content falls into the second category, which requires a more careful detection process. Identifying disinformation in these cases requires observing closely the context in which the speech is made and the speaker’s course of action – whether they dismiss the need for evidence, refuse to answer questions, and discourage audiences from engaging alternative sources. This process demands significant time and resources, and can risk chilling public discourse if applied to every user. Limiting the targets is therefore necessary.Footnote ⁹ In practice, one can envision restricting such scrutiny to high-visibility accounts, the accounts of celebrities, posts and messages that garner wide attention, or content that covers significant events (such as a presidential debate). In France, for instance, the ‘bills against manipulation of information’ focus on false information spread within three months before elections that has the potential to undermine ‘the reliability of the elections’ (Craufurd-Smith Reference Craufurd-Smith2019).Footnote ¹⁰

In terms of regulating entities, when all else is equal, a legislative framework is likely to yield more effective outcomes. This is because, firstly, legislation can provide consistent standards across platforms and better counteract the issue of account migration, whereby users shift to less-regulated venues to evade restrictions (Innes and Innes Reference Innes and Innes2023). For example, the case of Gettr, founded by the team of Donald Trump after his suspensions from Facebook and Twitter (McGraw et al. Reference McGraw, Nguyen and Lima2021), underscores the importance of consistent regulation across sites. Legislative interventions that apply to all platforms can more effectively close these regulatory gaps.

Secondly, authenticating contested content may require systemic co-operation across platforms. For example, an original photo might appear on Facebook and then be modified and reposted on Twitter. Decision makers might need access to the original image to identify the sign of manipulation. When and how information is circulated can also be of significance. The same message can have different implications posted at different times. Judging whether the content carries manipulative intent may require cross-analysis of data across sites. Policy interventions that facilitate such co-operation (such as the European Union’s Code of Practice on Disinformation, which establishes networks across platforms to ensure swift fact-checking during election periods) are likely to achieve more effective outcomes than social media self-regulation (European Commission 2025). Conversely, platform self-regulation can have the advantage of being flexible and proactive, and responds faster to new disinformation strategies. Policy makers and regulation advocates have to weigh the relative merits within their legal and political systems.

In addition to the dual-track system, I propose three guidelines to ensure that regulation facilitates, rather than hinders, the democratic function of free speech. First, democracies should refrain from penalizing users who inadvertently share false information. Even if a post shows the intent to deceive (say, a fabricated headline), the act of sharing should not incur sanctions, unless there is clear evidence that the user knowingly shares the content to mislead. This draws a complex line: while such content may be removed for self-evident manipulation, individual users should not face penalties for failing to fact-check, nor have their accounts suspended. This restraint is to ensure that citizens’ rights to free speech are robustly protected. As argued earlier, the freedom to share personal perspectives, even when factually mistaken, is the foundation of collective governance. This freedom should not be unreasonably curtailed unless there are demonstrable signs of manipulation.

Second, decisions regarding content moderation should be subject to sufficient oversight, periodic review, and public transparency. Some mechanisms must be in place to ensure that removal decisions are not left to the whims of private companies or governmental organs. One potential strategy to establish checks and balances is to compose an independent advisory committee to oversee removal decisions and develop regulatory guidelines, with or without legislative mandates or penalties. More importantly, any content removal should undergo regular review and be transparent. For instance, social media platforms may be required to report all removed content to an independent committee or official department on a monthly basis, and these reports should be publicly accessible. Finally, an open and fair appeal channel should be in place. Appeals against removal decisions can be adjudicated by a court or an ad hoc committee, ideally by a body distinct from the original decision maker. These mechanisms are crucial to enable civic oversight and provide avenues for remedy in cases of misjudgment.

Before the conclusion, let me address a common concern raised by the allies of regulation proponents. Some might argue that the framework I propose is too permissive towards harmful information shared by those who genuinely believe it. For example, one might genuinely perceive vaccines as ineffective and the coronavirus as a hoax. Proponents of content regulation may argue that such falsehoods should be removed regardless of the speaker’s intent to deceive – and to some extent, I agree. Some false information invokes significant public health concerns or threatens national security and may be legitimately removed to protect the community. For example, one can envision a restriction that applies to all COVID-related falsehoods, and the signs of manipulation are irrelevant. However, in these cases, we are not relying on a democratic argument for restricting false accounts, but a national safety or exigency one. The investigatory focus shifts from ‘whether regulating some false speech is justifiable from the standpoint of democracy’ to ‘whether the emergency in question outweighs our commitment to democratic values’. For the latter, it is clear that regulation incurs democratic costs, and the purpose is not to protect a meaningful democratic space but to safeguard the safety or stability of a community. Restrictions of this kind of information fall in a different normative realm and demand their own justifications. Proponents will have to explain why the particular case warrants a violation of democratic freedom, which is beyond the scope of this article.