14.1 Introduction
Disinformation, despite being around for centuries,Footnote 1 is a growing concern in the digital age. Defined as ‘false or misleading content that is spread with an intentionFootnote 2 to deceive or secure economic or political gain and which may cause public harm’,Footnote 3 disinformation has become increasingly prevalent on social media platforms and instant messenger groups.Footnote 4 In view of the potentially significant societal and political implications that its spread can have, including the erosion of public trust in institutions, exacerbation of social polarisation, and manipulation of democratic processes,Footnote 5 policy-makers have discussed and probed different regulatory approaches to tackle such ‘information disorders’.Footnote 6 Traditional approaches, such as state regulation and self-regulation by intermediary platforms and service providers, have shown significant limitations. State regulation, for instance, raises concerns about potential infringement on freedom of speech and the risk of political manipulation of public communication.Footnote 7 On the other hand, self-regulation by platforms often leads to inconsistent enforcement and lacks public accountability.Footnote 8 Co-regulation is being seen as a more balanced and effective approach to disinformation governance in general:Footnote 9 involving a collaborative process between the state and the private sector, this approach combines the strengths of state regulation and self-regulation while mitigating their weaknesses.
While the advantages of co-regulatory settings are indeed generally agreed upon, the approach shows challenges in such areas as disinformation, misinformation,Footnote 10 and malinformation,Footnote 11 where the statements in question might be dubious (and sometimes misleading) but still consist of lawful forms of expressions and are therefore protected by freedom of speech.Footnote 12 Moreover, current co-regulatory approaches aiming at disinformation implicitly assume that false or true statements as well as misleading expressions can always be clearly identified.Footnote 13 This, however, is not the case,Footnote 14 since many forms of relevant information consist of half-truths, decontextualised quotes and facts, or simply opinions.Footnote 15 Without focusing on the notion of disinformation and the process of determining truth – and particularly who has the final say in this process – current co-regulatory frameworks still carry the risk of assigning to either state bodies or private parties the power over who gets to say what online. Given these challenges, this chapter aims to explore potential and necessary advancements in co-regulatory governance structures for dealing with forms of ‘awful but lawful’ content – that is, socially undesired but nevertheless licit expressions – from a social constructivist perspective.
To do this, Section 14.2 discusses the limitations of classic regulatory approaches to governing disinformation online. Section 14.3 shows that while generally better suited to tackle disinformation in a human-rights-considerate way, current co-regulatory schemes carry with them underlying structural risks of interfering with freedom of expression and basic democratic principles and could be prone to misuses of power. Against this background, Section 14.4 develops the basic, procedural, and institutional requirements when it comes to advancements in co-regulation that can overcome the identified structural risks. Finally, Section 14.5 concludes the chapter by summarizing the findings and offering recommendations for future policy development.
14.2 Deficits of State-Based and Self-Regulatory Approaches to Regulating Disinformation Online
The challenge of disinformationFootnote 16 online has traditionally been addressed through two primary regulatory approaches: either state regulation or self-regulation by platforms. However, both these strategies present significant limitations that either hinder their effectiveness or bear risks of misuses of power over public communication.
14.2.1 State Regulation and the Questionability of Defining a ‘State Truth’
State regulation, which involves the government and/or the legislator setting and enforcing rules to control disinformation, has the advantage of legal authority and the ability to impose penalties or other sanctions.Footnote 17 However, this approach raises concerns about potential infringements on freedom of speech – especially in areas where the relevant expression is not illegal, for example forms of disinformationFootnote 18 or legal hate speech.Footnote 19 There is a risk that state regulation of admissible information can lead to censorship of minority or oppositional views or of expressions that are not in line with the views of the people in power.Footnote 20
The reason for these risk potentials is that any legal provision must define disinformation in a way that allows legal consequences to be applied to relevant statements. Regularly, legislators choose to attach sanctions to information that is either false, or misleading, or both, sometimes complemented by a requirement to bear the potential to inflict harm to individuals, to society, or to public order.Footnote 21 In all such cases the state – a regulatory body, a public authority, or, finally, a court – must decide in each case whether a relevant statement is false and/or misleading. This challenges the principle of a consensus-based theory of truth, as the state can unilaterally determine the truth (see Section 14.2.4). This can undermine democratic principles and limit the diversity of ideas and opinions within society.
Moreover, approaches that depend on case-by-case hearings, reasonings, and decisions do not scale. The enforcement of any state regulation on disinformation requires significant resources and often specific domain expertise. Governments would need to invest in monitoring and evaluating information to determine its accuracy and – if required – its potential for harm. This usually is a complex and resource-intensive task, as we can, for instance, observe in court hearings and proceedings.Footnote 22 To cope with huge numbers of cases, the competent public bodies would have to develop heuristics for faster decision-making, potentially undermining an encompassing societal search for truth. Another significant challenge for classic state-led regulation is the global nature of the internet, which means that individual states may have limited ability to regulate content or enforce national laws in cases where an expression originates from outside their jurisdiction.
14.2.2 Self-Regulation as a Liberal Take on the Social Responsibility of Private Actors
Where states have refrained from setting legal provisions to deal with online disinformation, high expectations towards self-regulatory initiatives have been formulated – at times, self-regulatory measures have been almost coerced by public bodies (see Section 14.2.3). Often based on references to their ‘social responsibility’, large platform providers are expected to counter any developments that are publicly discussed as social problems or seen as risks for either individual or collective harm. However, as long as there had been no legal framework obliging the providers to tackle such issues, these indications of responsibility can be seen as ethical arguments.Footnote 23 Platforms may follow these expectations based on voluntary moral considerations, strategic avoidance of future legal provisions,Footnote 24 or economic worries about a platform’s brand value.
Basically, self-regulationFootnote 25 involves platforms setting their own rules and mechanisms to control disinformation. This approach has the advantage of leveraging the platforms’ technical expertise and understanding of their user base.Footnote 26 Plus, the platform providers have the practical means to interfere with both individual communications and the information streams within their services. However, self-regulation is theoretically and empirically known for its significant limitations, including when it comes to tackling disinformation. Where no legal definition of disinformation is given, the platforms will have to come up with definitions themselves, or, alternatively, with alternative criteria in their community standards that allow them to delete or downrank the content in question.Footnote 27 Where a platform chooses to rely on a definition of prohibited content that is based on the dichotomy of true/false, it may become a powerful player that decides on the truth or falsehood of any expression and might be prone to intended or unintended bias, resulting in risk potentials for an open public discourse (see Section 14.2.4). Especially when it comes to large-scale automated decision-making algorithms used by platform providers,Footnote 28 there will be no opportunity to realistically judge every platform decision on a case-by-case basis.Footnote 29
Moreover, platforms have a financial incentive to maximize user engagement,Footnote 30 which can conflict with the goal of reducing disinformation, especially where disinformation’s effectiveness is based on emotionalising content.Footnote 31 This underlying conflict of interest can undermine the effectiveness of self-regulation.Footnote 32 Additionally, platforms might apply their own rules inconsistently, further eroding the credibility and effectiveness of self-regulation. Finally, the lack of public accountability can lead to less transparency in how rules are applied,Footnote 33 undermining trust in the platform and its self-regulation efforts.Footnote 34
14.2.3 Coerced Self-Regulatory Initiatives as Self- or as Co-regulation?
The first systematic approach to tackling disinformation at the EU level was not based on strict industry initiatives but rather on a form of self-regulation that can be seen as somewhat coerced:Footnote 35 the European Commission’s activities in initiating the EU Code of Practice on Disinformation.Footnote 36 Proclaimed as the ‘first time worldwide that industry has agreed, on a voluntary basis, to self-regulatory standards to fight disinformation’, the 2018 Code aimed at implementing measures that the Commission had presented beforehand.Footnote 37 These measures were based on recommendations by a High-Level Expert Group that issued their report in April 2018, making the Code of Practice on Disinformation more of an EU-set proposal that companies could or should sign than an industry initiative.
While the provisions had been introduced by the Commission, the implementation took place within the sphere of the companies that signed the Code. The monitoring of implemented measures based on the Code’s commitments was based on self-reporting mechanisms and a name-and-blame approach; there was neither a dedicated monitoring body nor any options for sanctioning non-compliance with the Code.Footnote 38 In sum, the 2018 Code could be seen as a form of more or less ‘induced self-regulation’,Footnote 39 developed by a political actor, while the implementation and respective monitoring processes were solely up to the signatories.Footnote 40
During the 2020 evaluation of the Code, the assessment report pointed out the success of the instrument (compared to a situation where no measures had been taken at all), but also criticised some aspects: inter alia, the absence of clear definitions and objectives, lack of transparency and accountability, and lack of adequate oversight or opportunities for input from public bodies or other stakeholders. Among the recommendations for further optimisation of the Code has been, inter alia, establishing a co-regulatory framework for the Code 2.0.Footnote 41 And yet, while the updated Code was proposed and signed in 2022 by more than thirty signatories, it still lacks any systematic co-regulatory fundament. While co-regulatory approaches depend on a legal framework and/or a public supervisory authority (either monitoring compliance with the code or supervising a co-regulatory body implementing the Code’s provisions), the updated Code neither shows these aspects, nor is it explicitly embedded in current legal frameworks.Footnote 42 Where the Commission refers to the relevance of signing and adhering to the Code in the context of Art. 34 and 35 DSA, these provisions do not oblige very large online providers (VLOPs) to submit to it, but rather the other way round: VLOPs signing and implementing the Code might use this as a legal argument in discussions about their compliance with Art. 35 DSA. However, they might leave the group of signatories at any time without any direct legal consequences.Footnote 43 And while the Code allows for provisions that aim at better definitions (without touching the notion of disinformation), more transparency, better data access, and more monitoring commitments including key performance indicators,Footnote 44 it still puts the decision on what is disinformation into the sole hands of private actors, co-opting them to regulate legal speech.Footnote 45
14.2.4 The Societal Process of Negotiating Truth and Its Fundamental Relation with Democracy
A situation where powerful actors decide on what is truth contradicts major concepts of truth and neglects its underlying epistemological process. But in both state-based regulation and self-regulatory platform governance, centralised arbiters of truth are emerging who decide on which statements are true and which ones are not. As shown, these decisions may be subjective, bearing a risk of unintended consequences.Footnote 46 A social constructivist perspective can show the reason that such approaches not only are prone to bias and misuse of power but also interfere with democratic theory itself. The consensus theory of truth suggests that statements are considered true because people generally agree upon them: truth is not something that exists independently of human collective decisions, but it arises out of a relationship between a proposition and the way the world is perceived by people.Footnote 47 Acknowledging that truth is subjective and can vary from person to person, this approach recognises that individuals have different perspectives, beliefs, and experiences that shape their understanding of truth. Therefore, society becomes the arena where these different perspectives are discussed and debated and where society ultimately reaches a common understanding.Footnote 48 This perspective underscores the idea that truth is not a static or absolute construct but is continually refined and negotiated through societal communication: truth is socially constructed. Societal consensus is seen as the ultimate arbiter of truth, and social constructivist theory requires society itself to negotiate this consensus.Footnote 49 If truth is the output of collective decision-making, single nodes of decision-making power might weaken or replace this societal negotiation process.
This has implications for democracy. When, in a democratic society, decisions are made through a process of discussion, debate, and agreement among individuals and societal groups, the consensus theory highlights the importance of public discourse in shaping our understanding of truth.Footnote 50 The public sphere, as conceptualized by Habermas, serves as a space where citizens deliberate on matters of common concern, forging a collective will.Footnote 51 The vitality of the public sphere rests on its inclusivity, ensuring that diverse voices, perspectives, and narratives converge to negotiate and thus shape societal truths. However, the efficacy of the public sphere is contingent upon its autonomy from both state and market forces, ensuring that the process of truth negotiation remains organic and uncoerced. While false information may have negative effects on rational discourse and trust in public institutions, so do measures that prune allegedly false statements from public discourse, limiting the remaining statements to those that are deemed true by one single actor (e.g., the state, a public body, or a company). This underlines that the societal process of negotiating the truth value of a dubious claim is a necessary and central part of democratic discourse and collective decision-making.Footnote 52 Otherwise, single actors can shape public discourse according to their own worldviews, forcing their own truth upon everybody else by hindering differing perspectives, beliefs, and experiences from entering the societal negotiation process, potentially resulting in the erosion of public trust.
Consensus-based truth and democracy are deeply intertwined concepts, both emphasizing the collective over the individual, participation over exclusion, and the process over imposition. At the heart of democratic societies lies the principle of pluralism: the diversity of opinions and the free exchange of ideas.Footnote 53 Were a single entity to dictate what is true and false, it would undermine the very essence of democracy. Open public discourse and with it the societal process to negotiate truth can be seen as a structural protection against authoritarianism: by distributing power among its citizens, democracy inherently resists authoritarian tendencies for power to be concentrated in the hands of a few.
As an alternative approach to basing the definition of disinformation on the dichotomy of true/false, some definitions of disinformation name either potentially misleading expressions or statements that might cause harm to either individuals or society.Footnote 54 However, these criteria reveal structurally comparable challenges, as they imply a reality from which information is leading people away,Footnote 55 granting the decision-making entity the power to (indirectly) define reality.
Basing the definition on harm, in contrast, can theoretically be seen as a good starting point for legal measures,Footnote 56 since an infringement of rights empowers the state to take proportionate measures of corrective action.Footnote 57 However, where dubious information is not infringing any individual right or endangering the public order, it can be seen as a legally protected form of expression. When it comes to forms of public harm such as licit racist or offensive speech, expressions that undermine social cohesion, or expressions that weaken trust in democratic or journalistic institutions, the process of defining such harm and applying those interpretations to relevant statements delegates uncertainty about truthfulness toward uncertainty as to whether and to what extent a statement inflicts harm.Footnote 58 The operationalisation of public harm,Footnote 59 or its ‘severity’ or ‘toxicity’,Footnote 60 and the proof of whether harm has been done gives almost as much power to discourse as does the declaration of what truth is, with the single exception that there is no discourse-based necessity to have the society negotiate harm.Footnote 61
In essence, delegating the power to single actors to decide what is ‘true’ and what is ‘false’ in these muddy areas of speech can be seen as rocking the foundation of any societal communication-based consensus theory of truth. Regarding state-controlled or self-regulatory approaches to tackle disinformation, one will either end up with a ministry of truth or a private monopoly that decides at scale about what is true and what is false, especially where state-targeted guardrails are bypassed by delegating decision-making power to private entities.Footnote 62
14.3 Potentials and Risks of Co-regulatory Approaches to Tackling Disinformation Online
Given the limitations and structural challenges discussed in Section 14.2, state regulation and self-regulation fall short as strategies for governing disinformation online. Many authors, therefore, propose the exploration of co-regulation as a better solution:Footnote 63 co-regulation, which involves a collaborative process between the state, the private sector, and sometimes civil society, could combine the strengths of state regulation and self-regulation while mitigating the issues identified here.
14.3.1 Co-regulatory Frameworks: Looking Auspicious, but with a Problematic Heritage
Co-regulation has emerged as a particularly prominent approach in European (audiovisual) policies,Footnote 64 particularly in areas such as commercial communication, protection of minors, and video-sharing platforms.Footnote 65 This regulatory approach relies on indirect state oversight through ex ante assessments of voluntary codes of conduct or via acknowledged self-regulatory bodies supervised by regulators.Footnote 66 One of the key benefits of co-regulation is its potential to balance the power dynamics inherent in the regulation of online media content. By involving multiple stakeholders in the regulatory process, co-regulation can help prevent the concentration of regulatory power in a single entity, whether it be the state or a private platform. This can help mitigate the risks associated with state regulation and the conflicts of interest inherent in self-regulation.
Co-regulation also benefits from the combined expertise of the state and of private platforms. The state brings its legal authority and experience in policy-making, while platforms contribute their technical and content-governance expertise as well as better understanding of their user base than the state possesses, leading to more effective and nuanced regulations that are better suited to the complex and evolving nature of information governance.Footnote 67 Moreover, co-regulation can provide a flexible and adaptive approach to content governance: unlike state regulation, which can be slow to adapt to technological changes, co-regulation allows for regular review and adjustment of regulations.Footnote 68
While co-regulation is deemed a European ‘success model’, and with the Audiovisual Media Services Directive (AVMSD) explicitly emphasizing this approach,Footnote 69 it faces challenges when dealing with lawful but socially undesirable content. First, co-regulatory systems delegate the decision-making power to private entities (which can be seen as problematic in itself), but where public bodies have the final say about a service provider’s (and/or a self-regulatory body’s) compliance with the law when governing disinformation, it is again in the hands of the state to decide on what is true and what is not. Such a decision would address either disinformation content directly, or aspects of the decision-making process of a provider or a co-regulatory body, and thus the content, indirectly. This shows the potential for abuses by either public or private entities – or both, in cases of deliberate or inadvertent collusion – constricting the societal process of negotiating truth.
14.3.2 Consequences for Freedom of Expression and Platform Governance
The problematic aspects shown here have implications for freedom of expression, freedom to inform oneself, and platform governance that is considerate of human rights. From a social constructivist, public-discourse-oriented perspective, it has become clear that neither state-based nor private-company-controlled decision-making regarding the admissibility of a dubious statement is in line with the premise that negotiating truth must be a societal process and must first and foremost lie with societal discourse.
This is not a general issue of all co-regulatory frameworks; in fact, distributed power is a good thing to have when it comes to communication control and content governance. The distinctive aspect in disinformation governance, though, is the missing legal framework that deems specific expressions as illegal (in contrast to, e.g., the advertising of products that are high in fat, salt, and sugar products, content impairing the development of children, commercial communication exploiting the inexperience of minors). Where the legislator explicitly limits freedom of speech for specific kinds of expressions, statements, and opinions, co-regulatory codes or bodies implement, concretise, and enforce those prohibitions. In dealing with disinformation, though, there is no general ban on stating false information or expressing opinions that contain dubious or generally misleading information. Here, the intervention interfering with freedom of speech is based on an act of setting up a state or private entity to rule over truth itself. This calls for an exploration of advanced co-regulatory approaches that can better navigate the complexities of regulating awful but lawful online media content. Such advancements must focus on how to build a co-regulatory governance structure that is based on the outcome of societal negotiations of truth (see Section 14.2.4).
To be clear: there are of course many forms of outright and undoubtedly false or misleading content, and the concept of harmful communication might be a good starting point for tackling mis-, mal-, or disinformation.Footnote 70 The instruments currently chosen to tackle disinformation also seem to reflect awareness that it is better to implement measures on an infrastructural level than to govern single pieces of content.Footnote 71 But even then, all such measures are based on an implied understanding of what ‘false’ or ‘misleading’ content is, where the process of finding or determining consensus regarding the assumed reality and truth is prerequisite for applying such definitions. Leaving this process to single entities is endangering public discourse, and with it democracy. It is understandable that current (co-)regulatory frameworks in disinformation governance circumvent this issue when quick solutions (and political successes) are needed. But it is neglectful not to reflect on the fundamental flaws in current approaches: first, the implication that true and false information is a clearly delineated dichotomy, and second, the misconception that single entities can have the legitimate power to decide on what is true and what is false.Footnote 72
14.4 Requirements for the Advancement of Co-regulatory Frameworks in the Area of Disinformation Governance
As this discussion has shown, shaping co-regulatory frameworks to tackle disinformation is an intricate task that requires careful consideration of power dynamics with a specific focus on the societal process of negotiating truth. Here, a governance approach is needed that combines the strengths of co-regulation with strong links to this societal process. This section looks into the basic, institutional, and procedural requirements that underpin such an advanced co-regulatory framework in the area of disinformation, drawing on concepts from hybrid governance and its intertwined norm-making between legislators, platform providers, regulatory bodies, NGOs, and society at large.
14.4.1 Basic Requirements: Components of an Advanced Co-regulatory Framework
The first provision an advanced co-regulatory framework must set up is the clear differentiation between illegal and legal expressions. Where there is an existing statutory and opinion-agnostic prohibition of specific expressions (e.g., information that is provably false and obviously leads to harm, such as incitement to violence or harmful health information during a pandemic), the above-mentioned challenges for legitimate speech are not applicable. For any form of dubious information that infringes existing laws – for example, personal rights, criminal law provisions, or both – the lawmaker has issued clear directives. A co-regulatory framework might very well include provisions for how to identify and proceed against illegal forms of speech; however, this is not the peculiarity of governing dubious but legitimate statements. When it comes to tackle legal forms of dubious information, the framework proposed in Figure 14.1 should make this circumstance very clear.
Forms of dubious information and their handling in advanced co-regulatory frameworks.
Figure 14.1 Long description
Overview of provisions that a co-regulatory framework might include to identify and proceed with illegal forms of speech. Some of the possible measures involve user comments, user-based tagging or labelling, reporting mechanisms, external fact checks, information correction, or other forms of providing objection or other context information. To encourage and support open public discourse, these measures should always be designed in a way to foster constructive dialogue and debate, for instance, by providing users with tools to evaluate the credibility of sources and information, rather than making unilateral decisions on content removal.
For contested but nevertheless legitimate statements, an advanced co-regulatory framework seems even more complex than current approaches. Based on the concept of hybrid governance, several norm-making players are tightly interacting with each other in these advanced governance structures. Neither the state nor private corporations are ideally positioned to ensure the societal negotiation of truth. As we have shown, the democratic imperative is to decentralize this process, ensuring that the negotiation of truth remains a collective, societal endeavour, free from undue influences on legitimate expressions, be they political or commercial, constructive or shocking. The basic premise of an advanced co-regulatory framework therefore is to appreciate this societal process, decoupling it from handling provable untrue and harmful expressions. This means of establishing and safeguarding public discourse, especially when it comes to disputed but not provably untrue expressions, calls for a legal framework and an infrastructure that enable public discourse to make doubts visible when it comes to digital information.Footnote 73 Where discourse participants can openly discuss controversial statements, and contested information can be clearly identified as such, the recipients of such information can be consciousFootnote 74 of the dispute regarding the truth or alleged falsehood of a statement – enabling the negotiation process that is needed.
Making doubts visible in digital environments means implementing technical features and interfaces that allow for contradiction and discussion.Footnote 75 Some of the possible measures involve user comments, user-based tagging or labelling, reporting mechanisms, external fact checks, information correction,Footnote 76 or other ways of expressing objections or providing contextual information.Footnote 77 To encourage and support open public discourse, these measures should always be designed in a way as to foster constructive dialogue and debate: for instance, by providing users with tools to evaluate the credibility of sources and information, rather than making unilateral decisions on content removal.
This alone will not suffice to tackle potential negative effects. Three other contextual factors play a central role when it comes to the societal handling of contested information: (1) the skills to critically evaluate information in general, (2) the (theoretical) opportunity for citizens to take part in the discourses that negotiate truth, and (3) the ability to comprehend and critically reflect on perceivable doubts or ongoing discussions regarding the truth value of a relevant statement. For the first factor, promoting media literacy from an early age will result in a society that is more discerning and less susceptible to dubious information. For the second, a co-regulatory framework can provide underlying principles when designing features that enable discourse, such as openness, low thresholds for participation, and transparent handling of any reacting user comment. For the third, measures might be clearly visible visual markers (including machine-readable labels), information about the contesting users(s), and highlighting of contesting parties that adhere to specific duties of care, such as fact checkersFootnote 78 or journalistic outlets. Moreover, encouraging the proliferation and accessibility of diverse information sources seems helpful where individuals are exposed to a range of perspectives, leading to them becoming better equipped to (co-) negotiate truth.
Another (albeit indirect) prerequisite for advanced co-regulatory settings are basic standards regarding the transparency of the discourse taking place (see Section 14.4.3). Where platforms have access to data – either specific statements and their distribution networks or articulated doubts regarding a content piece – such data can be used to support the process of societal discourse, for example by providing debates regarding specific statements or naming the accounts publishing them. Open data initiatives could set an example for accessible data regarding public discourse,Footnote 79 as data that is open to the public almost always makes for better and wider control, examination, and dispute.
14.4.2 Institutional Requirements: Decoupling Truth-Finding from Regulatory Interventions
In classic co-regulatory settings, we can observe two different governance structures. In code-based co-regulation, industry players agree on a code that is then referenced in the legal framework and/or monitored by a supervisory body. In an alternative co-regulatory set-up, the lawmaker establishes legal provisions that are then concretised and enforced by an approved co-regulatory body that is monitored by a regulator. In hybrid speech-governance environments, public and private communication rules overlap and interact to a significant degree: the lawmaker’s requirements shape private platform rules, while private platforms interpret and concretise these legal requirements in community standards from a strategic and economic perspective. However, the terms not only implement the legal provisions but also often contain provisions and guidelines that go beyond. Hybrid governance concepts recognize such inextricable links between private and public communication rules in speech regulation on platforms.Footnote 80
In the case of disinformation governance, both the lawmaker and the platforms should refrain from defining disinformation based on the dichotomy of true or false. Instead, one of the basic requirements for co-regulatory frameworks in advanced disinformation governance is to keep the societal process of negotiating truth open and to make doubts visible; any legal framework should therefore focus on providing the foundation for a governance structure and concrete measures that take this into account, either refraining from interfering with this process or supporting and helping society to accomplish it.Footnote 81 The institutional requirements of such a co-regulatory approach involve one or more new entities that facilitate this process by developing and supporting respective measures and by supervising their implementation by providers and/or any third parties (discourse-oriented bodies, or DOBs). While one potential configuration of current hybrid governance systems is the establishment of independent social media councils, these are structurally a little different from the envisaged DOBs. Social media councils usually act as impartial, third-party institutions responsible for overseeing content moderation decisions, developing guidelines and best practice and ensuring that content moderation practices adhere to principles of human rights and due process. By contrast, DOBs identify and oversee tools and measures to facilitate public discourse, enabling communities to make doubts about a specific statement visible. DOBs have an enabling role on an infrastructural level rather than in a case-by-case decision-making body; they support a critical public discourse on private platforms with a focus on negotiating truth.
Besides these entities, the need for classic actors in co-regulation will remain: the state providing the legislative framework, and co-regulatory bodies supervising the implementation of measures in cases of illegal expressions or where false statements have been identified, as shown in Figure 14.2.
Governance structure of an advanced co-regulatory system for disinformation governance.
Figure 14.2 Long description
A co-regulatory approach involves one or more new entities that enable facilitating the process of negotiating truth by developing and supporting respective measures and by supervising their implementation on side of providers or/and any third parties, namely: Discourse-Oriented Bodies. Besides these entities the need for classic actors in co-regulation will remain, i.e., the State providing the legislative framework, and co-regulatory bodies supervising the implementation of measures in cases of illegal expressions or where false statements have been identified.
To fulfil its role in such a setting, a DOB should be free from potential influences from either the state or private companies; it has to be independent from external influences, political agendas and organisations, and financial, knowledge-based, or factual dependencies.Footnote 82 Such an institution should be bound to the common good: either through its composition, with internal pluralism or multiple stakeholders,Footnote 83 or by procedure, with for example participatory approaches.Footnote 84 Institutional approaches known from network governance could be fitting for DOBs as well: concepts that emphasise horizontal interactions among a network of equal stakeholders from civil society, academia, the tech industry, user communities, and other relevant sectors, where decision-making relies on consensus, collaboration, and mutual agreement. Alternatively, one could assess whether a form of hybrid institution might work as a DOB: that is, by combining elements of both public and private governance structures a hybrid institution might formally be a public body but still operate with sufficient autonomy.
As a DOB’s main task is to help with and observe the implementation of features on the infrastructural level and of processes on the providers’ organisational level as well as to supervise the discourse-oriented application of such processes, it has to operate with a strong backlink to public interest. This might be realised through strong legal requirements regarding the public remit of such a body, or through organisational requirements concerning public representation or participatory approaches. The active involvement of civil society can provide valuable insights into the societal impacts of disinformation and help ensure that the co-regulatory approach is responsive to the needs and concerns of the public.
While a state- and company-independent, public-interest-bound third party such as a DOB is crucial to overseeing the enabling and support of societal negotiation processes by making doubts perceivable, there will still be a need for a co-regulatory body that monitors compliance, enforces rules, resolves disputes, and ensures transparency and accountability once dubious information has been objectively falsified or has been deemed false on the basis of negotiation. Here, traditional content moderation measures can be foreseen by the legal framework (deletion, downranking, warning labels, profile-related limitations), comparable to existing areas of co-regulation. This also entails supervisory tasks on the part of a regulator that monitors and controls the co-regulatory body’s decisions and actions.
In such a co-regulatory framework aiming at enabling the wider communicative ecosystem, some institutions and stakeholders can be identified to which additional requirements should apply. Where fact-checking organisations can evaluate claims and provide evidence-based assessments of relevant statements, these bodies are an important actor when it comes to identifying and assessing allegedly false claims.Footnote 85 To minimise any structural risk that comes with such a position of power in negotiating truth, fact-checking bodies have to be independent and transparent.Footnote 86 There should not be any dependencies on either state bodies or private companies, and their remit should have a strong public interest orientation to be in line with the societal objectives of public discourse. Stakeholders in media education and awareness-raising, aiming at educating the public about critical thinking, source evaluation, and the basics of journalistic integrity, should also be free from (hidden) political agendas.
When it comes to norm-making and functions within the proposed governance structure, the lawmaker has to set up a legal framework to establish the public interest body explained here as well as a co-regulatory body that supervises the providers’ handling of content that has been identified as false or misleading.Footnote 87 The lawmaker should state the general principles of the framework and should name exemplary measures to facilitate the societal negotiation of truth and making doubts visible. Moreover, the legislator should either stipulate how providers should handle information that has been identified as false or establish a framework where industry develops a code that after an agreement becomes binding within the legal framework.
The DOB is the competent player to (1) concretise, optimise, and extend the legally foreseen instruments and measures that support the public process of negotiating truth, (2) supervise the implementation of negotiation-supporting measures on the side of the providers in a way that is aligned with public interest (both ex ante and ex post), and (3) set up requirements for and approve independent and neutral fact-checking bodies and/or supervise their compliance. On the institutional level the DOB might also consult the regulator as well as the co-regulatory body on relevant topics. This separation of truth-identifying powers ensures that truth-related processes run independently of both state and company interests and interventions.
The regulatory body is needed to supervise the compliance of the providers with the provisions concerning their handling of content that has been identified as false and/or misleading, if the system is based on an industry code (in this case, the regulatory body also would have to approve such a code in advance). Alternatively, the regulator will have to approve and monitor an industry-led co-regulatory body to monitor the providers’ compliance with either legal norms or a code concerning platforms’ governance of content identified as false and/or misleading. In all cases, the supervisory function would encompass ex ante checks of standard operating procedures and technical configurations as well as ex post monitoring of their content governance in practice.
The providers themselves have to implement the requirements and measures developed by the DOB regarding the facilitation of the societal process of negotiating truth, and they also have to implement the legal requirements, measures, and processes regarding content that has been identified as false and/or misleading. These tasks comprise steps to provide technical access not only to infrastructural tools but also to their documentation, as well as access to platform data to be able to assess both the functioning of the negotiation process and the impact of any intervening measures. In this way, a structural decoupling can be established between the societal process of negotiating truth and decisions regarding content that has been identified as being false.
14.4.3 Procedural Requirements: Transparent, Inclusive, and Accountable
The success of such an advanced co-regulatory model is contingent upon a framework of procedural prerequisites. Considering their societal and human rights aspects, these requirements have to ensure that all processes are, inter alia, transparent and all measures fair and accountable.Footnote 88
Transparency involves the open and accessible dissemination of rules, decision-making processes, and outcomes to all relevant stakeholders and the public in general.Footnote 89 This includes clear guidelines on when the truth value of any given information is deemed low or non-existent, the ideal process that leads to such a decision, the responsibilities of each actor in the governance structure, and the sanctions in cases of non-compliance, either in implementing the discourse-enabling measures or in applying countermeasures in cases of statements that have been identified as false. Transparency should also be considered on the part of the regulator, the co-regulatory body, and the providers;Footnote 90 their methodologies, decisions, and justifications should be open to public scrutiny as well as to that of any affected individual.Footnote 91 Providers should present encompassing reports about their implemented measures for supporting the negotiation of truth as well as their application of any measures against statements identified as false. As a public-interest-oriented body, the DOB would also be obliged to act in the most transparent way possible, since it is focused on facilitating open discourse.
Where upholding the societal process of negotiating truth is one of the key objectives, considering inclusiveness as a procedural requirement is pivotal to counter any aggregation of power, especially on the side of the DOB. For public-interest-oriented governance to truly reflect the societal negotiation of truth, it must be participatory in nature. This means actively involving a diverse range of stakeholders, from civil society groups and academia to user communities and marginalized voices. The more stakeholder groups and public interest representatives take part in the DOB’s procedures, the bigger the chance of impartiality. It remains to be seen whether the body could also consider direct forms of participation by the public.
Finally, accountability mechanisms are necessary to ensure that all parties adhere to the rules and that there are consequences for non-compliance.Footnote 92 This would involve public reporting, regular audits, and evaluations. Hybrid governance models, by their very nature, distribute options to review and to point out irregularities among multiple stakeholders, ensuring that stakeholders remain answerable to the public. Platforms could provide feedback mechanisms whereby users can voice concerns about content governance policies, ensuring that platform policies evolve based on collective user input; the same goes for the DOB’s procedures to identify tools and measures to support this societal process. Accountability also directly applies to fact-checking bodies; they have to publish their methodologies and should be open to scrutiny to ensure trustworthiness.
Finally, even an advanced co-regulatory system as proposed here is not immune to errors or oversights. Recognizing this, it is necessary to establish clear redress mechanisms whereby individuals or groups can challenge decisions, seek clarifications, or appeal against perceived injustices.Footnote 93 Regarding appealed content moderation decisions, the usual addressees for redress would be the platform provider first (or a social media council), then the co-regulatory body, then the regulatory body, and finally, the competent court.
14.4.4 Advantages, Challenges, and Open Questions
An advanced co-regulatory framework, where the process of truth negotiation is decoupled from any state or private decisions-making powers, offers advantages over current regulatory approaches. It provides a more nuanced approach to tackling contested information, leaving the process of identifying false statements to society. The collaborative nature of such a hybrid governance model allows for a more adaptable and flexible regulatory framework,Footnote 94 capable of responding to the evolving challenges posed by awful but lawful content. By leaving the process of identifying a statement’s truth-value with society and fact-checking institutionsFootnote 95 and delegating necessary follow-up content decisions to bodies vested in decentralised co-regulatory settings, lawmakers can focus on procedural rules, mitigating the risk of state-controlled limitations on freedom of expression and companies’ decision-making power on what is true and what is not. However, this approach, intended as a starting point for further discussion on disinformation governance, leaves open three major questions.
Who has the final say? When decentralising the decision on whether a statement is true or false, it might happen that the result of the public discourse is ambivalent; the statement is contested but could not be proven wrong. In such cases, one approach could be to give freedom of expression the benefit of the doubt and leave the statement untouched (while still making doubt visible). Another approach could be to give platform providers leeway in how to handle such situations; in this case, redress mechanisms become especially important, but if any potential infringement of freedom of expression would evolve out of such procedures, the affected person would still have the right to approach a court considering their right to be heard. In cases of such disputes or contentious decisions, it would still be a court that has the final say, albeit only after clear hierarchies of decision-making.
How does it scale? As we have shown, current state-based approaches do not scale due to the requirement of context-sensitive, proportional decisions. The sheer volume of digital content coupled with the rapidity of its dissemination pose significant challenges to any governance model in view of dubious claims.Footnote 96 The same goes for decentralized models: participatory frameworks might face comparable hurdles in scaling up to address the vast number of individual communications.Footnote 97 Public discourse is able to tackle larger numbers of posts, but requires time, human resources, and expertise as well,Footnote 98 setting a natural limit of messages that can be checked and/or debated publicly. It will be crucial to address any evolving disparities in the allocation of attention to prevent inconsistencies or power imbalances.
How to consider cross-border issues? Where digital platforms operate in transnational settings that transcend geographical boundaries, the proposed governance model works independently of unique cultural and political environments. Harmonised standards concerning discourse-facilitating measures and procedural guidelines will need to work consistently across different regions. As these apply on an infrastructural level, the challenge of different value systems remains:Footnote 99 the public discourse in one area might differ significantly from another,Footnote 100 and hence it cannot be ruled out that these processes result in different assessments when it comes to the truth value of a statement. However, since rational public discourse uses the same epistemic sources and methods worldwide, it seems that differing results in different public discourses will be the exception. What becomes evident, though, is that the proposed system not only acknowledges but manifests local, national, and regional discourse spaces.
14.5 Summary: DOB is a Free Elf
This chapter has explored the challenges and opportunities associated with disinformation governance, with a particular focus on the potential of advancing current co-regulatory approaches. The central issue with current approaches to tackling disinformation is the implicit assumption that disinformation is something that can be easily identified. Current approaches accumulate decision-making power on the side of either the state or private platform providers, but the democratic ethos, deeply embedded in the annals of political theory, hinges on the principle of open discourse. It is through public debate that societies negotiate their truths, shaping collective realities and shared understandings. This process is not merely a by-product of democratic governance but a foundational pillar upon which democracies are built. Where state bodies or digital platforms are centralised actors deciding on what is true and what is false, regulatory frameworks run into a profound democratic dilemma.
The path forward requires a model that decouples truth-related deliberation from central decision-making, giving the process of negotiating truth (back) to society and ensuring that the power to influence public discourse is not concentrated in the hands of a few. Delving into potential governance structures, this chapter has identified that advancing co-regulatory settings can strike a balance between freedom of expression, freedom of information, and the prevention of societal harms. Structurally decoupled governance models, grounded in democratic principles and societal interests, emphasise the collective negotiation of truth, thus restoring the democratic ethos of open discourse.
The approach suggested here combines elements of state regulation, self-regulation, and the active participation of civil society, creating a form of hybrid governance. The model centrally relies on a public-interest bound DOB that identifies and supervises the implementation of discourse-oriented tools, measures, and procedures with the central objective of enabling the public to make doubts visible. The application of measures regarding content that is deemed false as a result of the negotiation process lies within a traditional co-regulatory set-up: platform providers apply measures, and co-regulatory bodies and/or regulatory bodies supervise their actions.
The co-regulatory approach offers a promising path forward, but it is not without its challenges. Implementing this approach will require careful design and ongoing adjustment to ensure it remains effective and responsive to the evolving nature of disinformation. However, as the digital environment and disinformation policies continue to evolve, there is an increasingly pressing need for innovative regulatory approaches to address the challenges posed by awful but lawful content. Hybrid governance offers a promising solution here that can help strike the right balance between freedom of expression and the mitigation of societal harms. By embracing hybrid governance and fostering public–private cooperation and public-interest-bound institutions, policy-makers, service providers, and stakeholders can work together to build more effective, balanced, and rights-preserving governance structures.
Open access