4.1 Introduction
Announced in 2018 and operational since 2020, the Oversight Board immediately created quite a buzz. Since then, the Board’s practices left a lasting impression on debates about how to control social media platforms’ immense rulemaking and rule enforcement capacities. Some criticized the Board as a presumptuous Silicon Valley legitimacy-grab.Footnote 1 Others highlighted the ambiguous and potentially performative effects of the “Supreme Court” label that Meta had managed to pin on the Board.Footnote 2 Others again consider the Board an imperfect but innovative and potentially positive step for more platform accountability.Footnote 3 Scholars and practitioners have analogized the Board to the US Supreme Court,Footnote 4 US courts,Footnote 5 international courts,Footnote 6 administrative tribunals,Footnote 7 human rights tribunals,Footnote 8 “courts” in authoritarian regimes designed only to deflect responsibility,Footnote 9 human rights grievance mechanisms,Footnote 10 customer- service,Footnote 11 “fourth-branch” institutions, or transnational hybrid adjudicators.Footnote 12 The Board’s founders have described it as a “non-judicial grievance mechanism” as envisaged by the terms of the UN’s Guiding Principles on Business and Human Rights.Footnote 13 Meta also compares the Oversight Board to human rights grievance mechanisms, presenting it as “a new mechanism for independent review of certain content decisions.”Footnote 14 Meta figures have also mentioned the possibility of future emulation of the Board by other actors as the company “hope[s] [the Board] serves as a useful model for other companies and initiatives.”Footnote 15
However, this self-description is merely stating the obvious. Clearly, the Board is not a traditional judicial institution but an organizational unit established and financed by a private entity, in this case a social media company. Arguably, it will manage to remedy a few particular grievances. But the need to remedy individual grievances does not fully account for the Board’s characteristics, origins, context, and function. Instead, and in contrast to the EU’s out-of-court dispute settlement bodies (ODSs) discussed in the previous chapter, the Oversight Board firmly aims to elicit structural change at Meta beyond the resolution of (very few) disputes between the company and its users. The disputes and the Board’s dispute resolution capacity must be understood mainly as a means to that end. This should come as no surprise, since the Board bends existing categories and draws bits and pieces from various role models that draw from both public- and private-law roots. However, viewing the Board solely from a legal perspective, especially through the lens of human rights, risks neglecting the legitimacy issues pertinent to it from an institutional perspective. In turn, dismissing the Oversight Board as a PR gimmick, a superficial take on content moderation, or even as a solely private body with little or no public relevance risks underestimating the possibly novel paradigm the Board (and bodies like it) may represent. In our increasingly globalized and interwoven world, private organizations wield ever more power.Footnote 16 This power manifests especially through rulemaking and rule enforcement.Footnote 17 Controlling this power requires new mechanisms that warrant at least experimentation beyond traditional, private law-inspired control instruments like liability or competition law. Notwithstanding Meta’s self-serving interests, we can approach the Board as a pioneering experiment in bringing into being a new kind of control mechanism – even though, or precisely because, many of its features and underlying ideas replicate preexisting ones. Therefore, reduced to its essentials, the Board can best be characterized as an emulation (albeit imperfect) of individual rights adjudication. The Board adjudicates claims about individuals’ rights to curtail Meta’s power, particularly regarding rule enforcement and, increasingly, with respect to rulemaking. Board members exercise impartial judgment, and the body presents itself as operationally independent in many (if not all)Footnote 18 respects, decides disputes between individuals and a dominant power structure, and bases its decisions on norms, especially those enshrined in international human rights law. And yet it is a private body with a circumscribed adjudicatory authority. In other words, the Oversight Board is also an Emulated Guardian.
The key takeaway from this case study is as dialectic as it is cautiously optimistic. On the one hand, the Oversight Board, comparable to the Digital Services Act (DSA)’s ODSs, reflects the idea of using individual rights adjudication to control platforms’ private bureaucracies, in this case Meta’s content moderation machinery. But although Meta promoted the Oversight Board as “a Supreme Court for Facebook,” the corporation never designed it to be a potent adjudicator. One could therefore dismiss the Board as a merely ceremonial appropriation of symbolism that is meant to publicly legitimate Meta’s scolded content-moderation behemoth. However, and this is the dialectic aspect, as the case study shows, the Board’s design, staffing, and the public’s perception of it as an adjudicator enabled the Board to incrementally expand its guardianship over Meta. In other words, the Oversight Board is enough of an adjudicator to (somewhat successfully) play with the perception of being an adjudicatory guardian and thereby performatively construct its authority. To be clear, the Board is far from being a fully fledged adjudicative controller of Meta’s administration-like rulemaking and rule enforcement. Yet the same mechanisms – juridification, judicialization, and individual rights adjudication – that provided the conceptual soil for the Board’s creation in 2018, later enabled it to go beyond its in-built restrictions. The Board is a more “activist” adjudicator than initially thought – with all the positive and negative consequences this entails. These issues have become even more pronounced since Meta CEO Mark Zuckerberg announced in early 2025 that the company would scale back certain policies, including its collaboration with external fact-checkers and its rules on hate speech. How this will affect the Oversight Board remains to be seen, but a deep understanding of the Board’s operation up to this point is essential for assessing future developments in what appears to be a more conservative and libertarian era of online governance emerging in the mid-2020s.
The case study progresses in six steps. It recapitulates the Board’s origins, analyzes the details of its institutional organization, the rules on which it bases its decisions, some of its decisions themselves, and whether Meta in fact implements them.
Before we begin, one last word on terminology. In 2021, Facebook, the parent company of Facebook, Instagram and Threads (social networks), WhatsApp (a chat app), and several other companies renamed itself Meta. Many of the Oversight Board’s governing documents were crafted before said rebranding and therefore speak of Facebook instead of Meta. To represent existing textual material as directly as possible, the case study, therefore, uses sometimes also the term Facebook (especially, when directly quoting or referencing governing documents with the old terminology).
4.2 Origins
Others have already told the story of the Oversight Board’s genesis at length.Footnote 19 This section therefore builds on existing accounts and reconstructs three aspects of the Board’s origins which in turn highlight three factors driving its inception. Firstly, the Board was based from the start on the idea of emulating individual rights adjudication. Irrespective of the company’s presumably self-serving (because profit-driven) motives, the organization its executives created utilizes personnel, institutional designs, normative material, and practices known from the traditional context of adjudicative control over (public) power. Second, this section sheds light on the decisive role played by lawyers and legal academics in the Board’s origin (and practice). The Board is the offspring of these juridical arguments – that Meta executives decided to establish an adjudicator and not, say, an ombudsperson is no coincidence. In other words, we will examine the performative effects of juridical concepts presented in internal debates about how to make the platform’s content moderation more accountable. Third, the section draws attention to excluded perspectives and initial reactions from outside actors, some of whom were challenging the presumed ‘rationality’ of lawyers working for Meta who were defending the company’s right to emulate an adjudicatory body. Contrasting these two perspectives highlights one major side effect of the Board’s adjudicative design: a usurping of the public legitimacy attached to juridical arguments, personnel, and institutions. Fourth, we take a step back and look at the discursive effects of Meta’s use of juridical concepts and constitutional metaphors in general. Discussions surrounding the Board’s justification and purpose can be seen as a discursive intervention that shifted debates away from then prevalent discussions about election interference and privacy (remember Cambridge Analytica) and toward content moderation and accountability mechanisms. This external performative effect of the Board yields considerable path dependencies as debates about privacy and content moderation are separated – even though they share the same underlying problem – that is, “business models based on data collection, behavioral advertising, and other aspects of surveillance capitalism.”Footnote 20
4.2.1 Emulation as the Board’s Leitmotif
Since the mid-2010s, social media platforms and especially Facebook, the world’s largest social media network, have faced growing criticism. Politicians, journalists, and scholars started questioning platforms’ role in a series of (arguably interrelated) political events, like Brexit, and the election of Trump as US President, or, later, the Rohingya’s violent persecution in Myanmar.Footnote 21 Further, a string of scandals revealed social media companies’ troubled relationship with privacy, content moderation, and potential external effects arguably exacerbated by their products.Footnote 22 In particular, the way platforms regulated the communication of their users, a process often euphemistically called content moderation, was characterized as unaccountable, arbitrary, and opaque.Footnote 23 These political events and platform-related scandals have led to an (ongoing) public relations crisis for Meta and have increased public pressure on regulators to ramp up their regulatory regimes for tackling possible abuses on (and by) platforms, which eventually resulted in the drafting and passing of the DSA (a complex process discussed in Chapter 3). These events also resulted in numerous hearings where company executives were brought before the US Congress and the European Parliament.
The story of the Oversight Board begins a few days before Mark Zuckerberg, Meta’s founder and chief executive, was summoned to Capitol Hill in mid-April 2018. Anticipating angry lawmakers questioning the company’s fumbled response to allegations of interference by the Russian regime in the 2016 Presidential election, its monopoly-like market power and, importantly, privacy concerns, Zuckerberg outlined his vision of “building a fair system” granting individual users “a right to appeal [Facebook’s] decisions on individual posts.”Footnote 24 Zuckerberg proposed emulating one of the oldest ideas in public law: individual rights adjudication. In other words, Meta was taking it upon itself to put in place its own system of adjudicative control over its content moderation bureaucracies. In the phrasing of this book, Meta vowed to establish its own Emulated Guardian. In an interview just a few days before his hearing on Capitol Hill, Zuckerberg said:
[My goal is to create some] sort of independent appeal process. Right now, if you post something on Facebook and someone reports it and our community operations and review team looks at it and decides that it needs to get taken down, there’s not really a way to appeal that. I think in any kind of good-functioning democratic system, there needs to be a way to appeal. And I think we can build that internally as a first step.
But over the long term, what I’d really like to get to is an independent appeal. So maybe folks at Facebook make the first decision based on the community standards that are outlined, and then people can get a second opinion. You can imagine some sort of structure, almost like a Supreme Court, that is made up of independent folks who don’t work for Facebook, who ultimately make the final judgment call on what should be acceptable speech in a community that reflects the social norms and values of people all around the world.Footnote 25
These statements are striking for several reasons. First, the company’s founder is outlining the idea of his own company devising an adjudicatory entity that is somehow to be “almost like a Supreme Court.” This “supreme court” metaphor is what the interview is famous for. However, there are at least two other notable remarks. That is, on the one hand, how Zuckerberg portrays democratic government as the role model for his company’s governance of user communication. He draws inspiration from “any kind of good-functioning democratic system.” Further, not only did he make reference to a “democratic system” as an inspiration and announced a “Supreme Court” as a governance solution, but he also mainly talked about introducing “a way to appeal” content moderation decisions – a thinly veiled reference to administrative governance.
Such references to legitimizing concepts like “supreme court” or democracy are by no means random. Speaking about his private company in public-law terms, his co-opting of constitutional vocabulary served to throw a veil of publicly legitimizing imagery over the at the time toothless accountability structures in place. Many media outlets picked up on the “supreme court” metaphor and started talking of the Oversight Board as Facebook’s “Supreme Court.”Footnote 26 However, despite nods to the notion of constitutional governance, the idea behind the Oversight Board’s genesis and resulting practices indeed emulates policy choices and organizational designs that have historically been diffused as individual rights adjudication.Footnote 27 To be clear, the Board’s actual powers are far more limited than initially foreseen, and allusions to a Supreme Court-like body were merely meant to be taken as metaphorical, a fact admitted by Zuckerberg himself.Footnote 28 However, even though initial plans were scaled down significantly, the Board is a tentative step to establishing a privately run adjudicatory body whose own goal is to incrementally increase its guardianship over one of the world’s largest private governance operations: the way Meta “moderates” content on Facebook, Instagram, and Threads.
This distinction between, on the one hand, the Board’s formal mandate and, on the other hand, the Board’s practice is important. As we will see, due to performative self-legitimation by savvy lawyers, the Board managed to relatively quickly sewer its own institutional interest – that is, essentially, to do its own thing – from that of the company, whose executives arguably liked the idea of experimenting with the Board without giving it too much power.
The normative and institutional structures required to install such a novel guardian are reminiscent of public-law tenets and, especially, the principles underlying administrative law.Footnote 29 To assume this essentially public function, the Oversight Board combines, by way of emulation, public- and private-law elements.
4.2.2 Lawyers, the Supreme Court Label, and Juridification
The second context-related aspect of the Board’s inception is the dominance of lawyers, legal scholars, and “constitutional metaphors” (in my coauthors’ and my phrasing) or “juridical discourse” (in Thomas Kadri’s phrase) in its creation.Footnote 30 From the start, the Board was conceived as a juridical, rights-focused, public law-inspired adjudicative body.Footnote 31 Even though many ambitious initial plans were scrapped, the underlying rationale seemingly always was to build a body that would be (perceived as) independent, norm-based, and with (some) authority to decide on disputes. In other words, creating the Board was always directed toward emulating an adjudicator.Footnote 32 Meta is a big and multilayered company. Presumably, different people working for it had differing visions for the Board at various times. What is remarkable though is that internal documents show that the perception of whatever entity would emerge from this experiment seems to have been a crucial factor for Meta.Footnote 33
4.2.2.1 From Supreme Court to Amorphous Adjudicator
The first to say that Facebook should subject its content moderation to a private adjudicative body was Noah Feldman. Feldman is a Harvard-based law professor specializing “in constitutional studies, with particular emphasis on power and ethics, design of innovative governance solutions, law and religion, and the history of legal ideas,” to quote from his profile on the university’s website.Footnote 34 As he later recalled, it all began on “a long bike ride up in the hills behind Palo Alto.”Footnote 35 After talking to legal scholars at Stanford University as well as people working for Facebook, Feldman had the idea of reproducing institutional designs from liberal democracies for online platforms.Footnote 36 He says that
platforms could benefit by the introduction of a governance mechanism that has traditionally been used, and intensively, in the last 50 or 60 years in liberal democracies, to manage the social conflict around what speech should be allowed and what should not be allowed, namely the constitutional court or the supreme court model. That is, in its essence, a model where there is an independent body that is not directly answerable to the primary, first-order decision-maker that has a set of principles that are clearly articulated on which it relies to make decisions.Footnote 37
In late-January 2018, roughly two months before Zuckerberg announced his “Supreme Court” plan, Feldman sent a memorandum to his old college friend Sheryl Sandberg, Facebook’s number two at the time.Footnote 38 Sandberg forwarded the memo to Zuckerberg.Footnote 39 The two-page document was titled “A Supreme Court for Facebook.”Footnote 40 In this memo, Feldman framed Facebook’s problem with hate speech as a battle between committed advocacy groups who want Facebook to ban certain types of speech and the general, societal interest in free speech.Footnote 41 In a binary interpretation, he argues that the “problem is the asymmetry between those who seek censorship and those who favor freedom.”Footnote 42 Implicitly, Feldman seems to view the company as on the side of societal interests in safeguarding free speech. In Feldman’s eyes, Facebook lacked a committed voice and a specific institution to help it defend free speech.Footnote 43 Without such an institution, well-organized lobbying groups demanding “censorship” could easily advance their agenda as “nobody is pushing hard on the other side of the door.”Footnote 44 However, Feldman argues, “there is a solution, one borrowed from the free-speech lessons experienced by governments over the last hundred years.”Footnote 45 This solution, Feldman says, is for Facebook to adopt free expression as its most fundamental normative basis, as a kind of “constitutional rule,” and to establish an adjudicative body to defend it.Footnote 46 Courts, he clarifies, unlike majoritarian institutions such as legislators, “can say ‘no’ … [to] demands of advocates seek[ing] to limit speech.”Footnote 47
Feldman portrays adjudication as the model for Facebook to counter demands for “censorship” and to upgrade how it exercises power over its users.Footnote 48 His memo advocated that platforms should set up “their own quasi-legal systems” and that “we need a Supreme Court for Facebook” to mitigate the “dangers social media platforms pose to free speech.”Footnote 49 In March 2018, Feldman sent the company a second memo, titled “Facebook Supreme Court: A Governance Solution.”Footnote 50 This outlined how “a Facebook Supreme Court … [a]long with lower appeals courts” should adjudicate free speech on Facebook and “create a durable institution to deliver principled, reasoned decision-making that would be widely understood as legitimate.”Footnote 51 One interviewee, who worked at Facebook at the time, said that the “supreme court framework” was there “because of Noah Feldman and because we lawyers tend to think in adjudicative terms.”Footnote 52 But apart from his attempt to pass off his metaphorically named entity as something “constitutional” and his argument that individual rights adjudication could be emulated, Feldman’s influence seems to have gradually waned once more people got involved. One interviewee, who had an advisory role in the Board’s creation, pointed out that various ideas were discussed and that
[t]here are other models of self-regulation. One could be a human rights tribunal, for example, instead of a supreme court. One might be a self-regulatory body. There are many ways to describe tribunals that work along principles other than those of a supreme court.Footnote 53
Nowadays, many interviewed staffers reject the portrait of the Oversight Board as a “supreme court.”Footnote 54 Further, internal memos from Facebook from late 2018 and early 2019 indicate a wider search for the ideal type of organization. As one interviewee involved in the Board’s creation at Facebook remembers, they looked at “the universe of possible solutions.”Footnote 55 Correspondingly, an internal memo identified different “families” of control regimes for large, administrative power structures, namely “investigative institutions; supervisory institutions; arbitral adjudication processes, administrative adjudication bodies, national judicial systems, including both European continental-style appellate courts and American appeals courts, and international judicial systems.”Footnote 56 This same memo listed the benefits and downsides of “judicial and quasi-judicial” control of power.Footnote 57 Examples range from the European Court of Justice, to Human Rights Courts, French appeals courts, the US Supreme Court, and, notably, to “the [US] system of administrative appeals in federal executive agencies.”Footnote 58
This reference to “administrative appeals” in US administrative law is worth commenting on as it plays into the point about the increasing “administrification” of platform governance in Chapter 2.Footnote 59 To recapitulate, Meta’s rulemaking and rule enforcement, just like administrative law, concerns a powerful institution (Meta), a group of individuals (users), and rights that are traditionally understood as coming under the purview of public law (freedom of speech). Further, the main function of all normative material governing Meta’s governance of communication is supposed to ultimately delineate how Meta exercises its power vis-à-vis a growing and anonymous (indeed, anonymized) group of users. This picture structurally resembles the function of administrative law.Footnote 60 Controlling such a complex “system of administration” (following Douek’s description) would seem to require a panoply of guardians.Footnote 61 As outlined in Chapter 2, various guardians of administrative or administration-like power exist, ranging from traditional, generalist courts to other adjudicatory institutions to various notionally independent operators like ombudspersons or auditors.
4.2.2.2 Juridification and the “Appropriate” Policy Choice
However, since Meta tasked mainly lawyers with building the edifice that would eventually become the Oversight Board, the “appropriate” policy choice to emulate, in the legal team’s eyes, was individual rights adjudication. As one interviewed legal scholar, who advised Meta on creating the Board, put it:
Personnel matters in these kinds of things too. When you have a bunch of lawyers … work[ing] on this thing, inevitably, what’s going to come out is going to be something familiar to lawyers, and just the way in the world that lawyers work.Footnote 62
This is a telling explanation. Because lawyers dominated the Board’s creation process (and, so far, its later practice), the project to design a guardian to oversee Meta’s massive power quickly became funneled into emulating bits and pieces from different models of adjudicatory oversight over public power. This is why the Oversight Board, despite its generic and inoffensive-sounding name, is more than a mere panel of experts, an ombudsperson, or some other type of “guardian.” From the start, its creators were perhaps subconsciously united by their respective preconceptions of what adjudicative control over power might look like. In turn, the structure that eventually emerged reflects the Board’s limitations as a private law-inspired entity.Footnote 63
From the perspective of organizational sociology, one cannot overstate the preeminence of this mélange of lawyer-dominated staff, uncertainty, and the woolly aim to “create an independent appeals mechanism.” Uncertainty is the seed of emulation.Footnote 64 As DiMaggio and Powell said, “[w]hen goals are ambiguous, or when the environment creates symbolic uncertainty, organizations may model themselves on other organizations.”Footnote 65 Emulating preexisting institutions, like individual rights adjudication, to increase an organization’s public legitimacy is a common driver of new formal structures. As explained in Chapter 2, emulating institutions and formal structures is relatively straightforward and an “easy sell” to the public. In contrast, it is much harder to implement more deeply rooted changes in policy, strategy, or practice. These are often harder to comprehend for laypeople and thus cannot be legitimized in people’s perceptions in the same way as the idea of emulating existing institutions by putting in place formal structures.Footnote 66 As Evelyn Douek noted, the “fundamental purpose [of the Oversight Board] is to bring greater legitimacy to Facebook’s content moderation ecosystem.”Footnote 67 According to Douek and many other critics, the Board is only a means to an end for Meta to increase its own public legitimacy in societal discourse.Footnote 68 Thomas Kadri points out that platforms “exploit” juridical arguments and structures like the Oversight Board to buttress their public legitimacy.Footnote 69 Similarly, DiMaggio and Powell argued already in the 1980s that private, for-profit enterprises “tend to model themselves after similar organizations in their field that they perceive to be more legitimate or successful.”Footnote 70 As discussed in Chapter 2, individual rights adjudication is one of the most legitimate and successful institutions controlling wide-reaching rulemaking and rule enforcement enterprises.Footnote 71 Hence, corroborating the theorization about emulation outlined in Chapter 2, Meta’s “Governance and Strategic Initiatives” team followed a common pattern in modeling the company’s Board after existing institutions. This team was the main driver behind the Board’s development.Footnote 72 It was run by Brent Harris, a lawyer who joined Facebook that year.Footnote 73 Apart from Harris, it consisted of ten people.Footnote 74 Further, “dozens” of “cross-functional” people would contribute to the governance team’s work.Footnote 75 The number of people involved in creating the Board, either as employees or outside consultants, was described by interviewees as in the low double-digits.Footnote 76 All of the lawyers involved with Facebook were highly educated and most had received law degrees in the US.Footnote 77 One interviewee, who was an in-house adviser to the company, recollected the internal debates at Facebook as follows:
I think that understandings of the way that courts relate to the people over whom they exercise authority definitely were in the room. Whether they were in the room because anybody wanted to consciously think about them, or whether they were in the room just because of the personnel and communicative context of the thing is not as clear.Footnote 78
How far this emulation went – and whether it can reproduce at least some of the success of adjudicatory control over power in the context of national states or nation-spanning conglomerations like the EU – follows later in this chapter.
4.2.3 Excluded Perspectives
Building an edifice with the help of (mostly) US-educated lawyers will necessarily exclude a range of options. To include more diverse perspectives but also to buy time, in 2019 Meta organized a series of workshops focusing on ideas for the Board’s design. This global consultation included a series of consultations with stakeholders such as nongovernmental organizations (NGOs) and civil-society advocacy groups.Footnote 79 The company also held several “workshops and roundtables” in, to name just a few places, Singapore, Berlin, Oklahoma City, New York, Paris, Nairobi, Mexico City, and Taipei.Footnote 80 Presumably these roundtables were the first occasions when the ideas advanced by Meta’s lawyers (presumably with Zuckerberg’s sanction) were presented to activists, stakeholders, and practitioners from outside the company.Footnote 81 The first roundtable, held in Singapore, featured a moot court-like debate about specific cases.Footnote 82 Participants were split into groups and received a hypothetical case upon which they had to decide on the basis of predefined norms.Footnote 83 As one interviewed activist, who participated in the Berlin workshop in the summer of 2019, recalls:
[T]hey gave us test case studies … These were blunt cases, and the people invited to this thing were, again, Westerners. There was one done in Latin America and there was maybe one in Africa, but I know people who were at both of them, and they said that there were still a lot of white people at both. … I would say it was more like a college debate team than a court. It felt nothing like a court to me. It was me sitting around a table with a bunch of other overprivileged white people, discussing a case where actual harm hadn’t been done. … I don’t think it set them up well for understanding the cases that would come to them or what they would mean for people on the ground. I think it already came out, so to be clear, they focused solely on European, American right-wing propaganda kind of cases.Footnote 84
Although it is impossible to identify who participated in these workshops, we can reasonably assume that Facebook wanted participants to discuss how they would like an adjudicative institution to look like. In contrast, if an adjudicative institution was even the right idea in the first place was not under discussion.
Further, the critical remarks by the participant indicate that Meta’s governance team’s preconceptions diverged from the needs of affected users. Especially in non-“Western” parts of the world, the same interviewee recalls that
[T]here’s just a lot of eye-rolling at the Oversight Board, to be honest … I think that there’s a lot of “What’s the purpose of putting all of this money behind the Oversight Board when we’ve been saying these things all along?” which is a fair critique. … I think the Oversight Board’s most useful function … is, in fact, that it formalizes and makes explicit a lot of those recommendations, synthesizes them, really. To be honest, from my colleagues in the Middle East I haven’t heard that many positive things about the Oversight Board. It’s not so much negative as just “What’s the point?”Footnote 85
What the activist is alluding to here is important. The Oversight Board was constructed by those with a specific mindset governed by ideas about individual rights, adjudicative control over power, and shot through with the largely metaphorical language of “supreme” courts and “constitutions.” As we will see, the Board somewhat falls short of these expectations. Most importantly, its authority is severely restricted, and it decides too few cases to offer any meaningful judicial protection to the users of its services. This reasonably brings us to question the institution’s overall validity. As the activist put it: “What’s the point?”
4.2.4 Foreclosed Debates
The dominance of lawyers in the discourse, as well as the predominance of constitutional metaphors and juridical framings, construed the Board as an adjudicative solution to Meta’s unaccountable methods of content moderation, which was itself portrayed as an issue of how to police billions of free-speech acts. In other words, the question the Oversight Board was designed to handle was that of free speech.Footnote 86 However, when we jump back to the beginning of the Oversight Board’s story in early 2018, we see an unusual discursive shift. Back then, content moderation was arguably not Meta’s biggest, let alone its only, problem. Instead, Meta was drawing strong criticism from scholars, journalists, and politicians for its handling of personal data and its interface design.Footnote 87 To begin with, Meta was using algorithms to “personalize” user experiences, leading to the potential creation of “echo chambers” and an entrenching of attitudes and opinions.Footnote 88 Then, Meta’s flawed privacy governance schemes were allowing outside parties to access and retrieve huge amounts of personal data. The most infamous of these data breaches was orchestrated by a company called Cambridge Analytica.Footnote 89 Cambridge Analytica managed to gain access to the personal data of eighty-seven million US citizens and later allegedly used at least some of that data to tamper with the 2016 Presidential election.Footnote 90
Zuckerberg announced the setting up of the Board a few days before his hearing on Capitol Hill, and around the time those data breaches became public, in April 2018. Although the hearing was also about content moderation, fundamental rights, and accountability, its main concerns were election interference and privacy in the context of the Cambridge Analytica scandal.Footnote 91 The Congress hearing took place under the title “Facebook, Social Media Privacy, and the Use and Abuse of Data.”Footnote 92 Senators pressed Zuckerberg mostly on his company’s “disturbing” privacy-governance systems that had ended up leaving the door open for Cambridge Analytica to steal personal data.Footnote 93
As Ari Ezra Waldman noted, there are “numerous discourses at play in informational capitalism,” some focusing on innovation and possible technological solutions to real-world problems, others highlighting social networks’ effects on free speech.Footnote 94 Therefore, one can view Meta’s project of the Oversight Board – and its metaphorical narration in constitutional-law terminology – as a performative shift in the discourse surrounding the wider role of social media companies in society.Footnote 95 Discussing how the Oversight Board might bring accountability to Meta’s moderation of speech shifted debates away from Meta’s still flawed handling of personal data and the morality of its business model. As revelations by whistleblower Frances Haugen in late 2021 indicated, Meta did not substantively improve its products, its handling of private data, and its overall business model.Footnote 96 Instead, the intensifying debates about content moderation approach platforms through the prism of individual rights adjudication but are increasingly detached from discussions regarding social media platforms’ business models, ways of handling data, and privacy.Footnote 97 Debates surrounding intrusive regulatory intervention when it comes to platforms’ business models – or even arguments for breaking up social media companies altogether – are still rife today, although the academic and political winds seem to have shifted toward more reform-oriented approaches.Footnote 98 In other words, platforms’ wide-ranging regulatory powers seem to be so entrenched that minimizing them, let alone eliminating them, appears increasingly impracticable. This tacit discursive premise in turn paved the way for various reformist ideas to come to the fore, the Oversight Board being just one of the most prominent.
To conclude, we shall recapitulate the two – slightly stylized of course – centripetal forces shaping the Board’s creation and its later practice. These are, on the one hand, Meta’s striving for public legitimacy and, on the other hand, as a consequence thereof, the nascent adjudicators’ unique position and ability to incrementally build on the public’s perceptions of what adjudicative authority entails. In other words, the fact that Meta decided to create the Board as an adjudicator and not as, say, an ombudsperson, potentially enables the Board to performatively play with the perception of being an adjudicator and thereby provide some accountability for Meta. Even though the next sections show the serious limitations built into the Board’s institutional framework, the story of the Board’s genesis and practice are thus evidence of the shortcomings, and dangers, but also of the potentially paradigmatic potential of Emulated Guardians.
4.3 Institution
This section investigates the Board’s institutional characteristics. To emulate individual rights adjudication, Meta needed to construct the Oversight Board as a private adjudicator that would adjudicate the public rights of users or, in other words, provide accountability for Meta’s private governance by using public-law mechanisms. Now we look at the Board’s organizational structure and personnel, as well as its jurisdiction, before reflecting on its impartiality and independence.
4.3.1 Organization and Personnel
Owing to its entirely private nature, the Board’s institutional structure differs significantly from many other adjudicative bodies controlling traditional administrative power.
The Oversight Board does not exist as a uniform entity.Footnote 99 Instead, it consists of four actors working together, governed by a set of founding documents. These are the “Oversight Board Trust,” an irrevocable Delaware noncharitable trust with several trustees (“the trust”).Footnote 100 The trust funds the Board’s operations and makes management decisions concerning its operation. The trust created a limited liability corporation (LLC) under Delaware law that rents office space and hires staffers.Footnote 101 Further, the trust filed the individual contracts of the Board members who bear ultimate responsibility for the Board’s adjudication. At the time of writing, the Board had between seventy and ninety staffers, who are employed by the LLC and who run the Board’s operations.
4.3.1.1 The Founding Documents
Four documents govern the Oversight Board. These are the charter, a vague and lofty text setting out the Board’s purpose and function; the trust agreement, a detailed contract establishing a Delaware noncharitable trust; the LLC agreement, a contract establishing a limited liability corporation; and a set of detailed bylaws.Footnote 102 The charter, bylaws, and trust agreement were drawn up exclusively by Meta’s lawyers. Meta then drafted the bylaws; the Board approved them and may amend parts of them subject to approval by Meta executives.Footnote 103 Further, several later crafted documents emerged as important throughout the case study and will be discussed in greater detail below. These are the code of conduct for Board members and staff as well as the Board’s self-given Rulebook for Case Review and Policy Guidance of November 2020.Footnote 104
In organizational terms, the trust agreement in particular has been described as perhaps the most consequential because the trust agreement is a formal contract, whereas the charter was just “declared” by the company.Footnote 105 The trust agreement, LLC agreement, and bylaws are much more detailed – and restrictive – than the charter. Yet, the exact relationship and order of precedence between those documents is complex. Especially in terms of organizational or operational hierarchy, it is not intuitively clear how some of the documents relate to one another. For example, the bylaws restrict the Board’s jurisdiction over subject matter to removal of particular types of content, whereas the charter uses general language, allowing “people who disagree with the outcome of Facebook’s decisions … a request for review.”Footnote 106 Intuitively, the use of terms like ‘charter’ and ‘bylaws’ would imply that rules established by the charter rank more highly than those set out in the bylaws. Similarly, the bylaws describe the charter as “the Board’s primary governing document” whereas the bylaws only “detail the operational procedures.”Footnote 107 Further, the bylaws explicate that “where there is a potential conflict between interpretations of the charter and bylaws, the charter shall prevail.”Footnote 108 The trust agreement reaffirms that the charter sets forth the “framework for creating the Oversight Board” and that only the charter “specifies the Oversight Board’s authority and jurisdiction.”Footnote 109 But at the same time, the charter states, in Article 6(2), that “the Board’s operational procedures will be outlined in its bylaws. The charter and the bylaws will act as companion documents.” Whether the charter’s vague allusion to the bylaws as a ‘companion document’ indicates normative equality between charter and bylaws, primacy of the bylaws as lex specialis and/or lex posterior, or primacy of the charter as lex superior, remain an entirely unsettled matter.
The best way to understand the relationship between the founding documents is to keep in mind the documents’ different drafting histories.Footnote 110 The charter was pinned down first and reflects the initial, high-flying goals from the early “Supreme Court” times.Footnote 111 It speaks of “people” instead of users, delineates the Board’s “authority to review” and establishes that the “Board’s resolution of each case will be binding.”Footnote 112 The trust document and the LLC agreement, in turn, were crafted as waterproof legal documents by a probably larger und more specialized group of lawyers scrambling to find a private-law form operationalizing the lofty public announcements of Zuckerberg and the terms of the charter.
4.3.1.2 The Trust
The trust was set up in October 2019.Footnote 113 According to Section 2.1 of the trust agreement, the “purpose of the Trust … is to facilitate the creation, funding, management, and oversight structure that will permit and protect the operation of an Oversight Board.” To achieve this, “the Trustees,” of which there are six, “shall form and fund a limited liability company (‘the LLC’) of which the Trust will be the sole member through its Trustees,” according to Section 2.1 paragraph 2 of the trust Agreement. The purpose of the LLC is to “establish, administer, and attend to the ongoing operation of the group of individuals who make up the Board Members,” the paragraph continues. In 2019, Meta announced a contribution of 130 million dollars to the trust.Footnote 114 In 2022, Meta allocated another 150 million dollars to the trust, bringing the total amount to 280 million dollars.Footnote 115
4.3.1.3 The Board Members
The most public and perhaps the most important component of the Board are its members. Members are individual contractees of the trust, meaning they are not employed by the LLC, to maintain their independence. Their compensation, described as “six-figure salaries,” is not disclosed and might differ from member to member as “each Board member contract shall be based upon provisions suggested by … [the trustees] and the Oversight Board.”Footnote 116 Members do not work full-time for the Oversight Board but are contractually expected to work fifteen hours a week for it.Footnote 117 However, several interviewees indicated that members often spend considerably more time preparing and deliberating.Footnote 118 Members need not be physically present at the Board’s offices around the world. Many of their deliberations happen online, spanning several time zones and continents. The members bear ultimate responsibility for the Board’s decisions, recommendations, and policy advice.Footnote 119 In November 2024, the Board had twenty-one members, though it can have up to forty.Footnote 120 According to the charter, Board members are appointed for a fixed term of three years, renewable twice.Footnote 121
Powers
The members are vested with several “collective powers,” according to Article 1(4) of the charter. They can request Meta to provide information, instruct Meta to allow or remove content and to explain its own decisions. According to Article 1(4)(2) of the charter, members may collectively “interpret Facebook’s community Standards and other relevant policies … in light of Facebook’s articulated values.” The Board can issue “policy guidance” but “will have no authority or powers beyond those expressly defined by this charter,” according to Article 1(4) of the charter. Later sections – in Article 2 of the charter – refer to the Board’s “authority to review.”
Qualifications and Selection of Members
Qualifications to become a member are vaguely expressed. According to the charter, members shall “possess and exhibit a broad range of knowledge, competencies, diversity and expertise.”Footnote 122 They should be able to work in a team and have “familiarity with matters relating to digital content and governance.”Footnote 123 The Board appears to be relatively diverse when compared to traditional adjudicative bodies but not necessarily when compared to international adjudicators, which often have strict rules safeguarding proportional representation on their bench. According to its bylaws the Board must always include a “globally diverse set of members” representing all world regions.Footnote 124 Naturally, these regions are broadly defined – mainly by continent – and members might tend not to be familiar with the particular local contexts of appeals or the jurisdictional realities applying in the regions from where the appeals emanate. Nevertheless, the panels tasked with drafting decisions must include at least one member from the specific region.Footnote 125
More than half of the Board’s twenty-one members as of November 2024 have a law background.Footnote 126 Among the Board’s members are several journalists, a former politician, a Nobel Peace Prize laureate, and a media and communications scholar.Footnote 127 As of November 2024, the membership’s composition appears almost gender-balanced (eleven male, ten female) and features members from all world regions, however with a US dominance (six members), followed by Asia (five members), Europe (four members), South and Central America, the Middle East (two members each), and Africa and Oceania (one member each).Footnote 128 The initial four members of the Board were selected by Meta.Footnote 129 The later members were selected by the Board’s cochairs, who are also members, and Meta.Footnote 130 Since the Board became operational, its members autonomously decide who joins their ranks (but seemingly only based on a shortlist reviewed by Meta).Footnote 131 From a comparative perspective, this is a rather unusual structure, as appointment and nomination procedures tend to tie adjudicators to the entities they are tasked with controlling. Given the spotlight on the Board’s independence and Meta’s seeming inability to offer a more legitimate appointment or nomination process, this model appears to be a sensible one. We should not forget that many such arrangements owe their existence to the fact that Meta remains a private company – with all the structural reluctance toward external oversight that comes with it. If ever, external, meaningful guardianship over the world’s largest content moderation machinery will build incrementally, not overnight.
Juridified Practice
Although the Board needs to function as an adjudicative body, its members do not have to be lawyers. Yet, as mentioned above, more than half of its twenty-one members active in November 2024 either studied law and/or work in law-related fields like human rights activism or are legal scholars. The dominance of lawyers among the Board members has several consequences.Footnote 132 For instance, the company’s executives presumably viewed lawyers as particularly capable of building the internal procedures, substantive reasoning, and external legitimacy of a body that was supposed to function and be perceived as adjudicative. Intuitively, this makes sense. Who else but lawyers could (or would) build an adjudicative institution? It also mirrors the dominance of lawyers in the Board’s creation at Facebook.Footnote 133 US lawyers dominated the internal debates at Facebook’s “Governance Team” that rolled out the Board’s institutional design between 2018 and 2020. Arguably, this lawyer-dominated group contributed to the Oversight Board being “an adjudicative body” (and not, say, one with a more parliamentary flavor). As one interviewee, who advised Meta’s governance team when creating the Board, recalls:
The fact of the matter is that the Oversight Board clearly started down one path. They could have started totally different paths. They could’ve issued three-page decisions, like the court of cassation does. It could’ve done that. It didn’t. There’s nothing in the charter or whatever that will tell you that it didn’t do. It didn’t do because of sociological factors, because they put a bunch of US lawyers in charge. What the fuck happens when you put a bunch of US lawyers in charge? Well, they started doing things that they’re familiar with.Footnote 134
Similarly, most people staffing the institution are lawyers as well. Whereas no member can have held a prior position at Meta, some high-level staffers at the Oversight Board were part of Facebook’s effort in creating the Board in the first place.Footnote 135
The dominance of lawyers among members pushed two normative aspects higher up the Board’s agenda than initially foreseen. These are, first, human rights and, second, public- and administrative-law principles. The expertise in public law and human rights law of many Board members explains why the Board took a different turn than the company, or at least some people in the company, had presumably intended at its inception. The charter posits that the “Board will review content enforcement decisions and determine whether they were consistent with Facebook’s content policies and values.”Footnote 136 In contrast, the Board should only “pay particular attention to the impact of removing content in light of human rights norms protecting free expression.”Footnote 137 Therefore, based on a strict reading of the charter, it seems as though Meta initially envisaged normative material unilaterally produced by Meta itself (its policies and values) and not human rights law as the key normative yardstick for the Board’s decision-making.Footnote 138 Of course, many people were involved in creating the Board, and competing visions may have circulated at Meta and within the governance team.
That said, the Board seems to focus particularly on international human rights law and to a much lesser extent on Meta’s own devised rules. As we will explore in detail in Section 4.5 below, much of the Board’s normative reasoning revolves around, and is framed in, the language of international human rights law, most notably Article 19 of the International Covenant on Civil and Political Rights (ICCPR).Footnote 139 For example, the Board allows for restrictions of freedom of expression only if “provided by law” and “necessary” to protect the rights of others, national security, public order, public health or morals, in the sense of Article 19(3) of the ICCPR. This external and arguably stricter normative framework can be understood as going beyond the initial design in the charter. The human rights background of many Board members (and staffers, as we will see in the following section) likely contributed to the Board’s tendency to focus on international human rights law instead of Meta’s vague and self-given “values,” “charter,” or “community standards.” The policies and values that Meta executives had devised are still referenced in all decisions, but most of the Board’s ink is dedicated to human rights-related reasoning.Footnote 140 This focus on human rights also constitutes, as we will soon see, one of the key differences from the DSA’s ODSs, the other Emulated Guardian examined in this work.
Further, the Board used this human rights framework to infuse administrative-law principles into their recommendations based on legality and proportionality assessments, as specified in Article 19 of the ICCPR. It often deliberates on the basis of arguments like normative clarity and accessibility, users’ right to remediation, the duty of the company to give reasons, and to translate its decisions and rules, for example. In this way, the Oversight Board aims to incentivize Meta to improve the normative framework for its content moderation. As one interviewee put it:
The Board is attempting to infuse content moderation with juridical norms and procedures. We might call this a process of juridification … In partnership with Meta, and in light of existing human rights norms, the Board articulates norms by which Meta’s content moderation decisions should be settled. It seeks to ensure that these norms will be enforced according to public law procedures. The upshot is to take a private for-profit corporation like Meta and to juridify its content-moderation operations.Footnote 141
The rules that are supposed to “juridify” Meta’s content moderation are public-law rules. Legality, proportionality, right to a remedy, and the duty to give reasons for a decision, for example, are basic, almost universal administrative-law principles. The interviewee continues:
To me, the issue is the infusion into management of public law principles. Now, I don’t know whether that’s the precise idea of juridification, but that seems to me the precise challenge of the Oversight Board.Footnote 142
Similarly, in its decision regarding whether Facebook was justified in banning former US President Trump from its platform after his role in the attack on the Capitol on January 6, 2021, the Board assessed whether Facebook’s rules were “clear and accessible.”Footnote 143 In classic administrative-law fashion, the Board “reiterates that the patchwork of applicable rules makes it difficult for users to understand why and when Facebooks restricts accounts, … [this] raises legality concerns.”Footnote 144 In the same vein, the Board argued that “[a]ppropriate limits on discretionary powers are crucial to distinguish the legitimate use of discretion from possible scenarios around the world in which Facebook may unduly silence speech not linked to harm or delay action critical to protecting people.”Footnote 145 The discursive might of such public law arguments is remarkable. While the Board had used general public law arguments to contain (and justify) Meta’s power in its 2021 Former President Trump’s suspension decision, the Board further expanded this approach in 2023. Then, when faced with incendiary posts by another head of state, the Board used more substantive human rights arguments and weighed, in quite some detail, the conflicting arguments for and against shutting down the profile of the Cambodian Prime Minister, who had used his Facebook account to incite violence in the run up to an election. Both cases dealt with Facebook’s infamous “newsworthiness allowance,” a rule that essentially excludes important people from otherwise enforced rules against hate speech and the incitement of violence. In 2023, the Board demanded that the company would not only delete content posted by the Cambodian Prime Minister but recommended to Meta to temporarily suspend his profile altogether.Footnote 146 That head of state, Hun Sen, had disseminated incendiary rhetoric through his Meta accounts, which arguably led to physical violence.Footnote 147
Such clear borrowings from public law would be impossible if not for the preponderance of lawyers among the Board’s members (and staff). Hence, the dominance of lawyers among the Board’s creators, members, and staffers aligned its design and practice closely with what lawyers are familiar with – that is, adjudicatory institutions. This again led to direct borrowings from public- and especially administrative-law principles to “juridify” Meta’s content management operations.
4.3.1.4 The Staff
Apart from its members, the Board relies on the efforts of a considerable number of staff. Between seventy and ninety people work in offices around the world. In 2024, the London office was the largest, with roughly forty staffers working there.Footnote 148 As Board members only work part-time for the Oversight Board and are scattered around the globe, the staff have assumed a key – and largely overlooked – role in shaping the Board’s strategic vision and practice. Staffers fulfill a dual function. On one hand, staffers are sherpas, helping the Board members navigate intricate legal and technical terrain. On the other, staffers are strategists themselves. By dint of their full-time and long-term positions working for the Board and their expertise in the field, they know many cases by heart and infuse their own normative and strategic considerations into the Board’s deliberative remit. Staff members have thereby played a crucial part in fashioning the Board as a formal structure independent of Meta.
Permanent Position and Expertise
In light of many social theorists’ focus on practice and cultural perspectives,Footnote 149 recent sociological and legal scholarship has highlighted the often overlooked but prime importance of staffers, secretariats, and “back-room personnel” in the business of international adjudication.Footnote 150 Especially in diversely staffed international adjudicative institutions, advisers, lawyers, and staffers are an indispensable pillar for organizational efficacy. Typically, their role is particularly prominent wherever adjudicating personnel work in globally dispersed, ad hoc, or part-time conditions. Yet most courts, tribunals, and other adjudicators remain coy about the precise roles their staff play. Instead, adjudicators often seek to “maintain the ‘fiction’ of rulings being delivered from on high …, whereas in fact, a large team of permanent staff is devoted to handling increasingly complex disputes.”Footnote 151 Other scholars have suggested that this applies for other international and transnational adjudicators like the European Court of Justice and the World Trade Organization (WTO)’s appellate bodies, and this is equally true for the Oversight Board.Footnote 152
The main reason for the staff’s prominent role in the Board’s practices is that, in contrast to Board members, staff are full-time employees. Plus, there are a lot more staffers than Board members. To repeat the figures, while the staff totals between seventy and ninety people, the Board has only roughly twenty members. Around thirty-five staffers, deployed to the so-called “content team,” work directly on all cases. The content team has several subteams. These are: a case selection team; a project management team; a case and policy team; and the data and implementation team. All staffers work under the supervision and at the direction of Board members. Yet, the content team’s permanent positions, expertise, and involvement with all aspects of each case seem to allow staffers to dig deep into issues they consider relevant and bring questions to Board members’ attention.Footnote 153 Staffers contribute – and thereby shape – the Board’s practices and strategies in various ways. They create longlists for consideration by a subteam of members, the Case Selection Committee, which then draws up a shortlist of, in their view, the most serious cases.Footnote 154 Staff also provide evidentiary material to inform deliberations at the direction of the Board members and help with fact-finding in relation to complex technical and normative questions. During the deliberations of Board members, staffers are also allowed to respond to questions about the briefing materials when asked.Footnote 155
Further, Oversight Board staffers are highly qualified. Many staffers are lawyers, and many have a human rights background.Footnote 156 This enables staffers to contribute to the Board’s adjudicative practice while reflexively reinforcing the Board’s adjudicative nature and perceptions thereof. Appropriately enough, staffers and members are subject to the same code of conduct.Footnote 157 Requirements to disclose conflicts of interest or the obligation to be independent, for example, amount to virtually the same thing as the organization’s code of conduct always refers to “members and staff.”Footnote 158 This indicates the staff’s importance for the decision-making process and the Board’s overall strategy. Another factor reinforcing the staff’s position within the institution is staff members’ identification with the Oversight Board’s mission. A strong sense of commitment and the perceived societal relevance of their work was quite prevalent in every interview and supplementary discussion. Many of the interviewed staffers working in content teams perceive their work as helping to resolve problems for society.Footnote 159 Many have a background in NGO work, courts, or other judiciary-related jobs.Footnote 160 One staffer describes the staff as having a “very strong sense of identification” with the Board and its mission,Footnote 161 while another mentioned their work’s main objectives to “protect individual rights” and to “mitigate the negative systemic effects on the basis of individual cases.“Footnote 162 Even though these qualitative insights are inherently subjective and reflect, at best, the perspective of the interviewed staffers and not the entire group of employees, we should consider them tentative indicators of many staffers’ attitudes to their work.
However, staff expertise also raises one problematic issue. Several staffers are former Facebook employees, some even from the Governance and Strategic Initiatives team that created the Oversight Board.Footnote 163 On the one hand, intimate knowledge of how Meta works is crucial for the Board’s success. On the other, staffing a guardian institution with former employees of that same “guarded” organization brings with it connotations of personal connections between controller and controlled. In that sense, the questions surrounding the Oversight Board’s institutional design are not at all new or unprecedented. In fact, they remind us of Iuvenal’s classic “quis custodiet ipsos custodes?” (who guards the guardians?) question.Footnote 164 Correspondingly, the Oversight Board’s code of conduct considers “being a former … employee of Facebook, Inc.” a conflict of interest for “members and staffers.”Footnote 165 Further, Board members and staffers must disclose several potentially disqualifying facts, one of them being previous employment at Facebook.Footnote 166 However, former employment does “not necessarily automatically disqualify” someone from working for the Board, as these cases would “be reviewed by the trustees to determine the appropriate course of action.”Footnote 167 A reading of the applicable governance documents indicates that such conditions (and the accompanying potential conflicts of interest) were reviewed and validated by the trustees in several cases. From a pragmatic perspective this makes sense, because outside hires would presumably be less familiar with the inner workings of Facebook’s content-moderation activities. However, this episode adds to the impression that the normative material applicable to the Board is met by considerable centripetal forces of seeking legitimacy (by the use of constitutional metaphors and strict codes of conduct, for example) while maintaining practical operability (in other words, needing people who actually know what Meta is doing).Footnote 168
“Fishing Mode”
Driven by what appeared like a high degree of personal motivation, the staffers working on the Board’s adjudicative processes operate and help fine-tuning the Board’s focus, “jurisprudence,” and procedures. Each of the content subteams consists of several staffers, and there are usually fewer than ten on each subteam.Footnote 169 These content subteams fulfill specific tasks concerning the Board’s adjudication and are present at every step while a case makes its way through the Oversight Board.Footnote 170 Staffers in the Case Selection Team, which is made up of several lawyers and political scientists, aid in selecting cases at the direction and supervision of the Case Selection Committee, which comprises a rotating membership of Board members.Footnote 171 Staffers also prep Board members in substantive, procedural, and factual matters.Footnote 172 Even though staffers do not write formal drafts of the Oversight Board’s decisions or advisory opinions, it is plausible that the extensive reports written by the Case and Policy Team lay the groundwork for the later draft produced by Board members.Footnote 173 At least some of the “content team” staffers are present when Board members deliberate but do not contribute to deliberations and are not encouraged to share their opinions.Footnote 174
And so, even though they are not vested with formal decision-making authority, staffers are an indispensable part of the decision-making process in a broader sense. One striking example of the staff’s value and strategic vision was what one interviewed staffer called “fishing mode.”Footnote 175 This describes how a group of fewer than ten staffers draws up a longlist of a few dozen cases from the many thousands submitted to the Board by users of Meta’s services. Board members provide staffers with criteria, which staffers then apply when filtering appeals.Footnote 176 A high-ranking staffer described these case-selection criteria as a blend of specific content, like “breasts of transitioning people that are deleted as female nudity,” with general aspects like “posts from conflict regions.”Footnote 177 Hence, filtering appeals works in an aim-oriented fashion. “Fishing mode” depicts the case selection team’s angling for “good cases” after first filtering the cases based on criteria like metadata, region, language, views, and violated community standard.Footnote 178 Like Captain Ahab and the crew of the Pequod, the case selection team stands in the crow’s nest looking for tell-tale ripples in the water, hinting at a whale under the surface. But how to spot a metaphorical whale in this ocean of cases? The same interviewee mentioned that a key consideration in longlisting is the user statement.Footnote 179 It increases the chances of selection where the user shows serious engagement with the matter and an understanding of what “the problem” is.Footnote 180 This has a practical background. It makes it more likely that the user will keep their post up and does not delete it once informed that the Oversight Board is investigating it.Footnote 181 The staffer described finding good cases as follows:
It’s not like finding the needle in the haystack. It’s more like finding a particularly nice-looking piece of hay in the haystack.Footnote 182
Clearly, one key question is what makes a “good” case. It is not only the criteria but also the structural issues underlying Meta’s decision-making processes that the case is potentially addressing.Footnote 183 Yet the staffer points out that it is hard to interpret these structural aspects from the limited amount of information they are able to glean from the submissions.Footnote 184 Often, the Board recommends Meta to change its rules to be more concise and transparent and align its procedures more with principles of public law, administrative law, and good governance. To buttress its argument, the Board refers to public-law standards as they appear in the wording of legislation like Article 19 of the ICCPR, for example – standards like legality and proportionality.Footnote 185 Further, the Board repeatedly recommended that Meta should notify users about content-moderation decisions, that it should state reasons and, where necessary, provide timely translations of its rules and decisions.Footnote 186
4.3.1.5 The LLC
Lastly, there is the limited liability company, which is the legal person at the center of much of the Oversight Board’s contracting. The purpose of the LLC is solely organization related. The sole member of the LLC is the Oversight Board Trust, according to Article 2.2 of the LLC agreement. The trustees serve as the LLC’s managers. The corporation is obliged to rent office space, draw up contracts with staffers (as employees), and draw up individual contracts with Board members.Footnote 187
4.3.2 Jurisdiction
Because Emulated Guardians are adjudicators, they must have jurisdiction. To be effective controllers of organizational power, said adjudication must also be compulsorily applied to the organization these guardians are tasked with controlling.
4.3.2.1 Subject Matter Jurisdiction
The Board’s purpose is to control Meta’s content-moderation activities. To achieve this aim, the Board’s founding documents outline the Board’s “jurisdiction.” The charter states that Meta users “may bring forward content for Board review” whenever they “disagree with the outcome of Facebook’s decision and have exhausted appeals.”Footnote 188 In contrast, the bylaws are formulated more narrowly. They had initially restricted the Board’s jurisdiction to review only “individual pieces of content, such as specific posts, photos, videos, and comments” that had been “removed for violations of … ‘Community Standards’ or ‘Community Guidelines.’”Footnote 189 Because of this, the Board appeared to be barred from reviewing a multitude of other actions and design choices impacting how Meta exercises power over its users. Scholars have repeatedly criticized these reduced terms of reference as the “biggest disappointment” of the Board.Footnote 190 Allowing users other than the original uploader to appeal moderation decisions is essential, as it will typically be the former whose rights may be infringed. For example, to protect the rights of victims in cases of defamation, libel, or revenge porn, victims must be able to challenge moderation decisions. Correspondingly, respective provisions were included in the list of possible appellants to ODSs in the EU’s DSA.Footnote 191 Since April 2021, and pursuant to a clause in the bylaws,Footnote 192 which opened the door for future expansions of jurisdiction, the Board now accepts appeals for the removal of content from Facebook and Instagram (that is, appeals against Meta’s decision to leave content up).Footnote 193 In 2024, the Board expanded its jurisdiction to Meta’s Threads, a social network the company set up in 2023 to compete with X after Elon Musk’s takeover.
Interestingly, the bylaws already indicate that the Board might move toward a more comprehensive interpretation of its powers in the future.Footnote 194 Apparently, this issue emerged during the global consultation period, prompting the company to concede, in an attachment to the global consultation report, that “[o]ver time, the Board may look to decide upon other actions (e.g. downranking or applying interstitial warnings).”Footnote 195 Correspondingly, Article 3 Section 1.1.2 of the bylaws states that “in the future, people will have the opportunity to request the Board’s review of other enforcement actions.” As mentioned previously, the Oversight Board has already made use of that provision and expanded its jurisdiction. Further, in October 2022, the Board ventured further down the path of these more graded forms of moderation as it demanded Meta to put up warning screens like “sensitive or disturbing content.”Footnote 196 In several interviews, staffers and Board members indicated their willingness to focus on “other enforcement actions” like the amplifying or demoting of content, which may highlight or restrict the visibility (and therefore harmfulness) of content without deleting it.Footnote 197
Therefore, prominent criticism that the Oversight Board would exemplify a simplistic, “stylized” understanding of content moderation as an endless string of customized decisions seems not entirely fair.Footnote 198 Although such a viewpoint may have colored the earliest visions for the Board, its practice indicates a nuanced and comprehensive understanding of the structural challenges of content moderation.Footnote 199 As we will shortly see, the Board’s key normative output is not made up of individual decisions but general, structure-focused recommendations, which aim at improving Meta’s governance procedures and structures instead of deciding only on a few paradigmatic cases.Footnote 200 Thus, the Oversight Board is not a solely binary adjudicator allowing “individual appeal to a human in every case.”Footnote 201 Quite the opposite, because the Board decides only very few individual appeals. Instead, the Board uses the few cases it adjudicates to formulate general normative sentences that its members believe should apply to Meta. Based on these general normative sentences, the company might then improve its governance and decision-making practices.
In conclusion, the Board’s jurisdictional focus rests on one important, and publicly visible, aspect of Meta’s content moderation: the removal or retention of individual pieces of content for violation of community standards. However, the company exercises power over its users in a more wide-ranging way than this. Meta retains content even though other users may want it removed. The company shows content according to algorithmic metrics and has, after all, designed the technical infrastructure that allows users to communicate in the first place. All these decisions, one way or another, “moderate” the communication of users on the platform. For the time being, the Board’s “jurisdiction” seems not to have plunged further into those deeper layers of moderation. Ultimately, those deeper and darker waters is where the whale is. Yet, on the evidence of decisions made up until now and the tentative expansion of the Board’s jurisdiction following the remit allowed by the bylaws’ provisions, it appears that the Board might incrementally venture toward a more comprehensive jurisdiction over the various aspects of content moderation which go beyond individual decisions that enforce rules relating to individual pieces of content.
4.3.2.2 Compulsory Jurisdiction
In principle, the Board’s jurisdiction is compulsory for Meta for the appealable content outlined above. According to Article 2(1) of its charter, the company “will commit to the Board’s independent oversight.” However, appeals may be ineligible for Board review if they concern content that is “criminally unlawful” or if a Board decision was likely to trigger “criminal liability” allegations or “adverse government reactions” for Facebook, its employees, or the Board.Footnote 202 For such pieces of content, Meta can contest the Board’s “jurisdiction.” The Board and the company found a somewhat peculiar way to ensure that the Board stays within the bylaws’ and charter’s limits. The Board typically shows the company the shortlisted cases prior to selecting the appeals for review. At this prior stage, the company might then object to the Board choosing, or technically: shortlisting, a specific case for the above-mentioned reasons.Footnote 203 Neither the charter nor the bylaws had stipulated any requirement for this prior “legal review” by the company.Footnote 204 But it appears to be now standard practice that the Board shares shortlisted cases for prior “legal review” with Meta. The company’s legal department then may “exclude” certain appeals from the list due to legal problems in the cases’ respective jurisdictions. As stated in the Board’s Rulebook for Case Review and Policy Guidance, which Board members adopted in November 2020: “shortlisted cases will be sent to Facebook for legal review. Facebook may exclude from the shortlist cases that are not eligible for review by the Oversight Board, in accordance with the Bylaws, such as cases that could result in criminal liability or adverse government action if reviewed.”Footnote 205
This is remarkable. In principle, Meta’s interest in avoiding negative legal consequences by subjecting itself to the Board appears legitimate. Despite all the metaphorical bravado, the Board remains an Emulated Guardian that resides, in contrast to the DSA’s ODSs, outside the realms of democratically set law, and only time will tell whether the Board will incrementally build adjudicatory authority.Footnote 206 However, the fact that the company has retained at least the possibility of weighing in on the Board’s case selection is a blot on the latter’s authority. From the perspective of individual rights adjudication, Meta should be able to contest the eligibility of cases but not decide whether they are ineligible. Thus, at the end of the day, we see how Meta’s interest in sustaining autonomy collides with the Board’s public image and function as (adjudicatory) guardian. Yet, historically, adjudicatory control over bureaucratic power structures grows gradually and incrementally.Footnote 207 Thus, possibly existing communicatory channels leading from the company to the Board should warn us not to enthusiastically drape the Board with the ceremonial garb of “supreme court,” as the Board is still in the process of building up its organizational muscles.
4.3.2.3 Appeals, Referrals, and Policy Advice
Users whose content has been deleted by Meta or who filed an unsuccessful notice for other users’ content to be deleted have “standing” to file an appeal to the Board.Footnote 208
Further, Meta may “directly refer cases to the Board that are significant and difficult.”Footnote 209 For example, in January 2021, the company referred the decision to the Board as to whether it was justified in permanently deleting the Facebook and Instagram accounts of then US President Trump after his role in the January 6, 2021 storming of the Capitol. Using Article 19 of the ICCPR’s criteria for legality, the Board harshly criticized the company’s incomplete and imprecise sanctioning policies. Ultimately, the Board argued that the company had no right to permanently ban Trump’s accounts because such a sanction was not stipulated in its terms of service at the time. However, the Board allowed that a temporary ban on Trump’s accounts would have been a proportionate response given the threat of real-world harm emanating from many of the posts on his accounts.
Lastly, separate from individual case review, the company may request “policy guidance” and a “policy advisory statement from the Board.”Footnote 210 These are general normative sentences on structural issues. Even though the various documents and the Board’s decisions themselves do not use consistent vocabulary, we can distinguish between policy advisory statements and specific recommendations following general requests by the company, and policy advisory statements and “recommendations” concerning the underlying governance issues at the company, but which are retrospectively issued as a byproduct of individual decisions.Footnote 211
4.3.3 Impartiality and Independence
To emulate adjudicative control over administrative power, the Oversight Board must be impartial and independent. As outlined in Chapter 2, independence refers to the adjudicating institution’s structural design, whereas impartiality signifies the fair-mindedness and objectivity of the person adjudicating.Footnote 212 Adjudicators must be externally autonomous – that is, not bound by hierarchical command or factual constraint – and internally objective and fair-minded in terms of the procedure’s outcome and the rules applied.Footnote 213 When applying these standards, we should remind ourselves of the words of Martin Shapiro who, based on extensive historical work, cautioned: “persons and institutions to which we normally award the titles judges and courts, … in reality … are simply at one end of a spectrum rather than constituting an absolute distinct entity.”Footnote 214 For example, specialized adjudicators that control administrative power often have a legal, organizational, and practical nexus to the organization they control. Often, sub-court administrators like UK administrative tribunals, American Article-I courts, or EU Boards of Appeal may even operate attached to specific administrative agencies. Yet they all have structural safeguards in place to shield their adjudicative decision-making processes and people from undue influence by the controlled entity. Therefore, certain organizational overlaps are often necessary in practice or, for the purposes of gaining greater expertise, even desirable. Adjudicatory impartiality and independence hence call for an appropriate institutional, personal, and procedural setup that helps to safeguard externally autonomous decision-makers and internally objective decision-making while maintaining operability.
Viewed against this yardstick, I argue that the Board is – albeit organizationally and financially intertwined with Meta – in practice sufficiently autonomous and its members fair-minded to be considered impartial. However, when it comes to the Board’s independence we see a more blurred picture. On the one hand, the (irrevocable) trust installed a seemingly effective organizational buffer between Board and company. On the other, Meta retained avenues of influence over who gets nominatedFootnote 215 and may “exclude” cases from review if “they are not eligible for review by the Oversight Board.”Footnote 216 To what extent Meta treads these paths of influence is unclear and it should be noted that during interviews nobody indicated any undue influence being applied.Footnote 217 However, these two organizational and procedural aspects are clear “no-go’s” from a classic perspective on adjudicatory independence. The crux is: the Board is only of value to Meta if it is publicly perceived as sufficiently independent. This is arguably why, for example, Meta’s preliminary “legal review” of the cases entering the Board’s docket is not explicitly stipulated in the provisions of the charter or in the bylaws but only in a rather ominous sounding “rulebook,” which “nonbindingly” specifies the bylaws.Footnote 218
The following section analyzes the decisive aspects of independence and impartiality for adjudicators controlling administration-like operations. The Board is not entrenched in Meta’s hierarchy, though the company nevertheless retains some influence over its docket. Further, we reflect on the trust’s role as mediator between the Board and Meta, and on how “dangerous” the Board effectively is for Meta. We then look more deeply into the Board’s nomination and removal procedures and investigate its operational and financial autonomy.
4.3.3.1 No Hierarchy, but Docket Influence?
For the Board to be independent, its members must not be subject to hierarchical command governing their decision-making. The Board’s independence was a key concern during its creation.Footnote 219 Article 1 Section 1 of the bylaws, as well as Article 1 of the charter, stipulate that members “will exercise neutral, independent judgment and render decisions impartially.” Even though the founding documents retain some veto powers with the company when it comes to certain management decisions by the trust (for example, changing one or more bylaws), Meta cannot issue any “commands” to Board members.Footnote 220 Correspondingly, no interviewee voiced concerns about company overreach. Even though their responses do not guarantee that there was no overreach by the company,Footnote 221 the way the Board’s decisions developed suggests that it indeed operates largely autonomously from the company’s influence. In many decisions, the Board has thrown light on and condemned Meta’s governance practices. For example, in the Trump decision, the Board criticized the company’s request to formulate rules. It stated:
[t]he Board rejects Facebook’s request for it to endorse indefinite restrictions, imposed and lifted without clear criteria. Appropriate limits on discretionary powers are crucial to distinguish the legitimate use of discretion from possible scenarios around the world in which Facebook may unduly silence speech not linked to harm or delay action critical to protecting people.Footnote 222
The Board’s forceful repudiation of the company’s formally referred request, the founding documents’ focus on the Board’s independence, as well as the Board members’ status as independent contractees, makes hierarchy-based subordination of the Board to the company unlikely. Similarly, several interviewees said that they had seen a drop in public criticism of the Board’s purported independence after the Trump decision.Footnote 223
Yet even though the company does not appear to be exercising undue influence on the Board’s decision-making and especially its deliberations, Meta retains at least the possibility of exerting influence over the Board’s docket. Board members who are part of the Case Selection Committee are in theory free to compile a shortlist of possible cases to take forward. However, before the Case Selection Committee decides which cases to adopt for decision, that shortlist, according to the Board’s internal “rulebook” must be sent to Meta. It is unclear what would happen if longlisted cases were not sent to legal review by Meta. As mentioned previously, the company “may exclude from the shortlist cases that are not eligible for review by the Oversight Board, in accordance with the Bylaws, such as cases that could result in criminal liability or adverse government action if reviewed.”Footnote 224
Naturally, if Meta ends up removing cases from the shortlist, the Case Selection Committee cannot select them, and the Board cannot decide whether to take them forward. Neither the bylaws nor the charter regulate this “Facebook Legal Review,” although the company’s intervention is supposed to be just part of the “non-binding … practical guidance … detailing case review and policy guidance procedures.”Footnote 225 Judging from transparency reports, Meta indeed conducts such legal reviews for each case and, in early 2021, Meta did exclude at least one case, and possibly more, from the shortlist through its use of this “legal review.”Footnote 226
An adjudicator whose docket is subject to any caveats emanating from the entity it is supposed to be controlling does not match up to the traditional standards of independence familiar in other adjudicative bodies. Even nonadjudicative guardians like ombudspersons or other “fourth branch” actors often enjoy more autonomy in selecting and rejecting cases. As Klonick observed, Meta’s “delegation of authority to the Trust [and the Board in general] is far from absolute.”Footnote 227 In sum, Meta granted the Board independence in the most literal sense, as the Board is not formally entrenched into Meta’s hierarchical structure. However, depicting the Board as independent in the traditional, adjudicatory sense of the word would, at the moment, be inaccurate for the simple reason that Meta retains some influence over the Board’s docket.
4.3.3.2 The Role of the Trust
Even though the Board does not receive directives from the company, organizationally speaking, the two bodies are intertwined. Central to these ties – but also, simultaneously, to the Board’s far-reaching practical autonomy vis-à-vis the company – is the trust. The irrevocable, noncharitable “Oversight Board Trust” (established in Wilmington, Delaware) serves as a buffer between Meta (as a private company) and the Board, which, although privately organized, performs the public law-like function of controlling Meta’s administration-like content moderation. Six trustees are obliged to isolate the Board’s adjudicatory decision-making from the company as much as possible. Board members and trustees communicate mostly in regular (though none too frequent) meetings, where they discuss organizational requirements and administrative issues, for instance.Footnote 228
According to Article 2.2 of the trust agreement, the trust and the individual trustees have to “protect the independent judgment of the Board Members and their ability to fulfil their stated purpose.” To do so, many organizational aspects are delegated to the Board itself. For example, the Board can propose new members, establish new procedural rules (within the limits of the bylaws), or engage in new practices like issuing quarterly reports to measure the impact of its decisions and policies. In turn, the trustees are obliged to allocate resources to the LLC for office space, travel, accommodation, workshops, and work retreats for teams scattered around the globe. The trust also handles contracts for Board members and employees. Trustees do not participate in case selection, neither do they take part in deliberations, voice opinions, or anything else that could directly influence the Board’s adjudicatory work. One trustee describes their involvement with the Board’s day-to-day work as follows:
We [the trustees] stay completely independent of the Board when it comes to selecting and deciding cases. But with respect to the actual administration of the OSB entity, there are channels of communication. There must be, because we need to set up an organization that works for the Board members, but we must do so in ways that are consistent with our own fiduciary responsibilities.Footnote 229
Hence, beyond formal channels of communication, informal avenues between trustees and Board members arguably exist. After all, the group of Meta executives and employees creating the Board, its members, and staffers is fairly tight knit. Most of them went to (and sometimes teach at) elite US law schools. The effects of these personal and habitual connections among practitioners, lawyers, and legal scholars have been identified as a common factor for fostering the emergence of normative orders.Footnote 230 Such “field effects” (borrowing from Pierre Bourdieu) should not be underestimated when it comes to the makeup and operations of the Oversight Board.
Correspondingly, even though most people working for the Board, or the trust seem to have, at some time or other, shared their strategic ideas for how the Board should operate, the Board’s practice isolates its decision-making from the company and even the trustees. Trustees seem largely isolated from personal or strategic considerations relating to the people working for the Board. One trustee said:
Of course, I stay completely away from the decision-making processes. That has nothing to do with me. But the larger strategic positioning and functioning of the Board is an entirely different question. We need to establish a model that will provide criteria of success with respect to these larger strategic issues. I myself have many ideas, but in the end the Trustees and Board members will need to reach a common agreement.Footnote 231
This trustee alludes to a thread emerging during the conversations and interviews conducted for this present work: the Oversight Board’s ongoing soul-searching. The Board’s complicated institutional and internal structure, the public demands on it (some of which Meta and the Board itself drummed up), and the challenge of using private-law mechanisms to accommodate its own emulation of individual rights adjudication – at heart an intrinsically public function – complicates the Board’s soul-searching. In short, controlling one of the largest experiments in (private) global governance while being an entirely private and relatively small institution is a conflicting, ambiguous, and perhaps impossible task. Again, the same trustee explained that:
We’re using the structures of private law to create an entity that’s essentially responsible for establishing public law norms. We have to work out exactly how these two distinct legal paradigms intersect. The role of the trustees is to guarantee the independence of the Board. That obligation is clear enough when it comes to actually selecting and deciding cases. But what about if the Board decides that to be independent it requires certain staffing levels or administrative assistance? How do we reconcile that kind of independence with our own fiduciary responsibility to the res of the trust? These kinds of issues raise complicated and unresolved questions. We’re trying to feel our way towards solutions.Footnote 232
Reflecting on the trust’s role in setting down a procedural framework for the Board’s external autonomy and internal fair-mindedness, we arrive at some (partly conflicting) conclusions. For a start, the trust constitutes a formal as well as informal link between the Oversight Board as an organization, its members as individuals, and Meta as a company. Communication exists and, all symbolic distance aside, it would be naïve to assume that the Board operates in isolation from the company. The issue is then whether such communication impairs the Board’s external autonomy and internal fair-mindedness. Reasonably assessing the fair-mindedness of Board members (and staffers), if it were possible at all, would require an array of psychological surveys that is far beyond the scope of this book. However, if this author’s subjective impressions based on the conducted interviews are of any value, a “pro-Meta bias” was not detected among the interviewed staffers or Board members. Some interviewees even described the Board’s internal culture as expressly “adversarial” toward Meta,Footnote 233 while others compared it to a “human rights NGO.”Footnote 234 At the same time, interviewees characterized the Board’s impact as relatively limited and rejected the idea that it had any “supreme court”-like influence over Meta.Footnote 235 But they seemed confident that their adjudicatory practice, based largely on human rights and administrative law, could be valuable for the interests of users – and that they were not in their jobs just to do Meta a favor.Footnote 236
This leaves us with the question of the Board’s external autonomy. This autonomy would be impaired if the trust or the company could somehow pressure the Board to decide in a certain way. The trustees’ fiduciary duty is to enable the Board to exercise “independent judgment.”Footnote 237 Trustees have no obligations vis-à-vis the company, and none of the interviewees suggested that the company exerts influence via the trustees. Further, baseline communication between adjudicator and controlled entities seems normal and does not necessarily impair external autonomy or internal fair-mindedness. Based on the limited impressions from the conducted interviews, therefore, the trust seems to work reasonably well as a buffer between company and Board.
In that context, it is important to keep a realistic perspective when it comes to how big a “threat” the Board really is for the company.Footnote 238 Apart from individual decisions on specific pieces of content, Board decisions are not binding. Arguably, and given the hundreds of millions of dollars Meta already invested, it would not make sense for Meta to weigh in the Board’s decision-making process because the Board’s main normative feedback to Meta, the nonbinding recommendations (see at Section 4.6.2), require Meta’s involvement for them to be implemented anyway. If Meta policymakers want to circumvent the Board, it would be much easier to delay and tacitly thwart the implementation of the Board’s decisions, recommendations, and policy advisory statements.Footnote 239 Direct meddling with the Board tends to result in disastrous publicity and thwarts the (assumed) aim of Meta to (also) use the Board as a publicly legitimizing formal structure.
Take as example an episode about Meta’s withdrawal of a policy advisory opinion on the company’s handling of hate speech in the context of Russia’s war in Ukraine.Footnote 240 The company requested a policy advisory opinion from the Board, whose importance several interviewees described as equal to that of the Trump case. Yet, very late in the process, Meta reversed and withdrew its request. In an unusual press release, the Board subsequently criticized the company stating that it “believe[s] that the request raises important issues and … [the Board is] disappointed by the company’s decision to withdraw it.” Press reactions to Meta’s withdrawal were equally negative, some depicting the move as “undercutting” the Board’s authority whose “only role is to handle the easy questions,” prompting the question “why bother with it at all?” Neither charter nor bylaws explicate that Meta had the right to withdraw an already referred request for a policy advisory opinion. Article 2 Section 2.1.3 of the bylaws states that the company “may request” such statements and that the Board “may accept or reject” such requests.
4.3.3.3 Nominations and Removal of Members
Another factor relevant for independence are the nomination and removal procedures for Board members. A body consisting only of people hand-picked or waved through by the controlled entity raises concerns about independence. Moreover, adjudicators’ personal sense of fair-mindedness requires that they should not worry about being sacked for any decisions they might make. The removal aspect does not appear particularly problematic. According to the charter and the bylaws, Board members can be prematurely removed from the Board only for just cause but not “due to the content decisions they have made.”Footnote 241 Instead, removing Board members requires a “two-thirds vote of the Board (not counting the member(s) in question), subject to approval of the trustees …; and may be considered only for a violation of the code of conduct.”Footnote 242
The code of conduct regulates in detail how members and staff “must conduct themselves, both in their professional and personal capacities” which means “avoiding even the appearance of impropriety and flagging all potential conflicts through proper channels.”Footnote 243 Therefore, members, staff, and their immediate family should not, for example, receive gifts, loans, or financial assistance on the basis of their position on or relationship with the Board.Footnote 244 Further, the idea of removal just because Meta wants a Board member to be removed appears unlikely. The trustees’ fiduciary duty is to protect the Board’s independence, not Meta’s interests, and Board members themselves would hardly vote against their own institutional interests. Hence, the Board’s removal regime protects Board members from removal by the company.
However, nomination procedures for members are not fully beyond Meta’s grasp. Neither the charter nor the bylaws detail the process of nominating new Board members. Article 4 Section 2 of the bylaws stipulates that “trustees will formally appoint” Board members. None of the governing documents further specifies the nomination procedure except for allusions to the preferred professional qualities of potential nominees. According to an article by Steven Levy, Meta has wielded its influence over the Trust to prevent people its executives perceived as too critical from becoming Board members.Footnote 245 According to Levy, a candidate’s previous and repeated criticisms of Meta resulted in some of its management (internally of course) voicing “reservations,” which ultimately “put the kibosh on the nomination.”Footnote 246 Such reporting is perhaps even more damaging for Meta than it is for the Board, as it emphasizes how reluctant the company is to cede authority to the Board and subject itself to truly independent external feedback. That is, however, neither surprising nor particularly worrying – authority of external guardians emerges incrementally, through continuous wrestling and discourse, rather than immediate submission by the controlled power structure.
4.3.3.4 Operational Autonomy and Budget
When focusing on its day-to-day work, the Board is simultaneously autonomous – as the company does not meddle with the Board’s decision-making – and dependent on operational support by the company. In particular the Board’s IT relies heavily on Meta’s support. From a pragmatic perspective, such organizational semi-dependence is inevitable. Yet at times, it may be delicate as highlighted by severe disruptions of the Board’s web infrastructure in 2021. For example, the bylaws state that “Facebook will maintain responsibility for the development and maintenance of tools that the Board uses to review and decide upon cases, including the Board’s website and case management tool.”Footnote 247 This web infrastructure is crucial for the Board to be able to receive user requests and handle its massive caseload. This “case management tool” refers to “the platform used by the Oversight Board to receive and review case submissions and collect and store case files.”Footnote 248 What at first sounds mundane in fact generally shapes case selection. One staffer remembered that the case management tool malfunctioned when the Board started to accept the first appeals in early 2020.Footnote 249 Another staffer, in May 2022, mentioned that the case management tool, albeit working well overall, was still having occasional technical issues.Footnote 250 Therefore, organizationally, the Oversight Board, with its workforce of only seventy to ninety people, remains a relatively small entity for controlling the rulemaking and rule enforcement of the world’s largest social network, an organization with tens of thousands of employees. This necessarily leads to a considerable degree of dependence when it comes to day-to-day operations because the Board often requires Meta’s engineers to provide, for example, IT support.
Lastly, the Board depends on Meta’s contribution to the irrevocable trust.Footnote 251 The Board itself cannot raise any money.Footnote 252 As mentioned before, in 2019 Meta contributed 130 million dollars to the trust. Meta initially estimated that the first tranche would last six years, but in July 2022 the company announced putting another 150 million dollars into it.Footnote 253 Presumably, the Board spent more money than anticipated but this supposition could not be confirmed by any of the interviewees.
The trustees retain prerogative over all major spending decisions.Footnote 254 The trustees are independent of Meta and have a fiduciary duty to “facilitate the creation, funding, management, and oversight” of the Board.Footnote 255 Therefore, trustees have the power to invest in and sell property, establish the LLC, open accounts and exercise transactions, enter into contracts, and anything else necessary to keep the Board running.Footnote 256 To date, the biggest investment seem to have been the fifteen million seed funding for the newly founded Appeals Centre Europe – an Oversight Board spin off that was certified as an ODS under Article 21 DSA in late summer 2024 and might become an independent funding stream for the Board.Footnote 257 However, forming additional “companies for the purpose of effectuating the purpose” of the trust requires Meta’s consent.Footnote 258 Even though Meta initially conceived of and initiated the Board as a six-year experiment, the early recapitalization of the trust in July 2022 is a sign of the Board’s financial independence for the foreseeable future. However, in line with 2023 cost-cutting measures at Meta, the company appeared poised to gradually limit the Board’s financial basis, prompting the Board to moderately consolidate in 2024.Footnote 259
4.4 Rules
This section introduces the different normative layers governing content moderation and the Board’s adjudication. We can differentiate between internal rules devised by Meta and external international human rights law. By way of context, the Board has so far not substantively interacted with EU law. The key takeaway of this section has two parts. We find out that the practically relevant internal rules of content moderation are, in contrast to what is often argued, not the terms of service but a variety of subsidiary rules that detail the normative regime. Then, we also discuss Article 19 of the ICCPR, which provides the Board with a normative scaffolding to evaluate what Meta is doing.
4.4.1 Internal Rules: Values and Content Policies
The initially intended normative focus of the Board were the rules that Meta (or Facebook) had unilaterally and internally devised. These were a set of “values,” and Meta’s “content policies” which are terms of service and, crucially, the huge list of rules governing user communication known as “community standards.”
4.4.1.1 Meta’s “Values”
In early 2018, when the specific shape of the Oversight Board was still unclear, Meta executives considered setting up the company’s “own quasi-legal system where clearly stated policy rules can be applied to the plethora of real-world expression challenges.”Footnote 260 The normative core (a privatized Kelsenian Grundnorm so to speak), should have been an “iconic, one-sentence values commitment Facebook would adopt.”Footnote 261 To a degree, the Board’s foundational documents still reflect this idea, stating that the Board “will review content enforcement decisions and determine whether they were consistent with Facebook’s … values.”Footnote 262 However, identifying these values is not that simple. Neither the charter nor the bylaws explain what these values are. Instead, both charter and bylaws premise that Meta “articulate” them.Footnote 263 There is no public process or document of perceptible legal quality establishing what these values ought to be. The only source naming and detailing the values is a blog post by Meta’s then Vice President for Global Policy Management on the company’s own public relations blog.Footnote 264 The executive writes that the company was “expanding the values that serve as the basis for [its] decision-making.”Footnote 265 According to this post, the “paramount” value is “voice,” whereas other values – “authenticity,” “safety,” “privacy,” and “dignity” – might limit the free expression of that “voice.”Footnote 266 The text goes on to briefly explain what these values mean. “Voice” relates to a far-reaching, US-informed interpretation of free expression.Footnote 267 Of course, voice and expression on Meta are exercised by way of users posting content, which is the basis of Meta’s business. “Authenticity” means ensuring that users do not “misrepresent who they are or what they’re doing.”Footnote 268 “Safety” prohibits intimidating other users, as well as the dangers of excluding, or silencing content.Footnote 269 “Privacy” aims to “give people the freedom to be themselves,” while “dignity” refers to the executive’s assertion that Meta “believe[s] that all people are equal in dignity and rights.”Footnote 270
In short, various Meta employees’ own (unexamined) definitions of “values” should serve as the primary normative framework for the Board’s assessment of Meta’s rule enforcement and arguably also its rulemaking.Footnote 271 Given the vagueness of their definitions, as well as the limited public visibility and legitimacy of those expounding the ideals upon which these “values” are purportedly based, “values” cited in a blog hardly provide appropriate normative material to control what Meta is doing.
4.4.1.2 Content Policies
As argued above, the Board is supposed to assess whether Meta’s enforcement actions comply with its “content policies.” The charter mentions that “Facebook’s Community Standards and other relevant policies” are “collectively referred to as content ‘policies.’”Footnote 272 These are unilaterally devised rules that manifest in at least three layers.
Terms of Service
The top layer of social media companies’ rules are their terms of service. These represent nothing less than the contractual basis governing the relationship between Meta and the users of its services. The terms of service are relatively brief and do not regulate user communication substantively. When it comes to content moderation, the terms of service only establish that users must not violate the “community standards” and “other terms and policies.” These community standards and other terms and policies make up a second layer of rules. These are a variety of normative texts, appear under different titles, and are not immediately obvious as a part of the terms of service. Instead, the terms of service only reference the community standards and other rules. Therefore, in general, the actual terms of service are only germane because they signpost the much more decisive second normative layer, which are the community standards (and the other, unspecified, rules).
Community Standards and Other Policies
The decisive normative layer governing online communication in Europe and beyond consists of unilateral rules devised by platforms detailing what communication is allowed on a given platform. These community standards are binding with respect to users because the platforms’ terms of service (which the user has to agree to) reference them. In the case of Meta, community standards claim to “apply to everyone, around the world, and to all types of content.”Footnote 273 They are split by topic into more than twenty different “standards,” dealing with issues such as “hate speech” to “bullying and harassment” to “cybersecurity.”Footnote 274 Each standard has a “policy rationale” which outlines the aim of the specific community standard. The community standards – with varying concision and specificity – then provide definitions and examples of prohibited communication acts. To illustrate this, we can look at Facebook’s community standard covering hate speech. As of November 2024, almost thirty updated versions of the standard were published on Meta’s website. The first hate-speech community standard published by Facebook dates to May 25, 2018.Footnote 275 Since then, Meta has amended the hate-speech standard roughly every few months.Footnote 276 The hate-speech standard’s “policy rationale” states that people should be able to “use their voice and connect more freely when they don’t feel attacked on the basis of who they are.” There then follows a “definition” of hate speech. Facebook defines this as
a direct attack against people – rather than concepts or institutions—on the basis of what we call protected characteristics: race, ethnicity, national origin, disability, religious affiliation, caste, sexual orientation, sex, gender identity and serious disease. We define attacks as violent or dehumanizing speech, harmful stereotypes, statements of inferiority, expressions of contempt, disgust or dismissal, cursing and calls for exclusion or segregation.Footnote 277
The practical complexity of such a definition is obvious. How to draw a line between “direct” and “indirect” attacks against people? How to handle necessarily subjective, value-laden, and context-dependent adjectives like “violent,” “harmful,” or “dehumanizing”? Which diseases are “serious”? Does only the content of a communication act make it “hate speech” or also the attitudinal stance(s) of the person who expresses it? What about the temporal and territorial space in which an utterance was made?Footnote 278 Are we even capable of identifying the past, current, and future social, political, and cultural contexts of an utterance with sufficient certainty? What about contexts that are culturally foreign to Meta’s predominantly North American rule makers? Interviewed practitioners described content moderation as “phenomenally difficult,”Footnote 279 and scholarsFootnote 280 and tech journalistsFootnote 281 have even pronounced content moderation an impossible endeavor. Platforms’ increasing reliance on automated moderation (despite the well-known shortcomings of natural-language processing)Footnote 282 only exacerbates these difficulties.
To drain the seemingly endless swamp of possible contextual layers, each community standard therefore provides further, generalized explanations. These general “explanations” often consist of normative statements relating to those whom the company wishes to protect. For example:
We also prohibit the use of harmful stereotypes, which we define as dehumanizing comparisons that have historically been used to attack, intimidate, or exclude specific groups, and that are often linked with offline violence.Footnote 283
The sections then elaborate – in a section entitled “do not post” – on specific examples of the types of content that meet the above-mentioned definitions. The “do-not-post” content subdivides into several “tiers.” A “tier 1” attack includes, for example,
(d)ehumanizing speech or imagery in the form of comparisons, generalizations, or unqualified behavioral statements … to or about … insects, animals that are culturally perceived as intellectually or physically inferior, … sub-humanity.
In sum, Meta’s community standards reflect the reactive and incremental nature of their creation. They are lengthy, convoluted, sometimes contradictory, and highly subjective. As one interviewee recalls, “[t]he process for making content rules is basically a room full of US lawyers sits down and makes the rules after the latest crisis.”Footnote 284 Accordingly, “ad-hockery,” imprecise terminology, and structural deviations between (and within) standards characterize the resulting “community standards.” Simply put, if you want to take down a communication act (usually a post), you are likely to find a community standard that prohibits this post. Yet most standards and their cascading “explanations” are crammed with adjectives and imprecise terminology that can be interpreted in many ways. To an extent, such normative uncertainty is inevitable – especially if you are aiming to regulate the communications of two or more billion people. However, like any sanctioning approach, restricting “speech” on social media requires a minimum level of normative foreseeability and consistency if it is not to be totally arbitrary. We will re-encounter these deeper levels of platform rulemaking in Chapter 6 because they point to one practical shortcoming of many scholarly or regulatory takes on reforming content moderation.Footnote 285 Just infusing fundamental rights, which are broad normative categories, without specifying how this affects these sublayers of specific rules, will not solve content moderations’ normative shortcomings.
4.4.2 External Rules: International Human Rights Law
The third set of norms the Board uses to control Meta is international human rights law. Meta is a private company and therefore cannot, of course, be a signatory of international human rights treaties like the ICCPR. Furthermore, despite scholarly criticism, international law does not provide for the immediate application of human rights law to private actors even if they are powerful transnational companies.Footnote 286 This section briefly outlines how the charter largely omits human rights, introduces the UN Guiding Principles as the tool that makes international human rights law relevant for corporations like Meta, and introduces the ICCPR, and especially Article 19, as the normative fulcrum that underpins the Board’s reasoning.
4.4.2.1 The Charter’s (Superficial) Reference to Human Rights
The charter only superficially references human rights. In Article 2 Section 2 – entitled “basis for decision-making” – the charter explains that “the Board will review content enforcement decisions and determine whether they were consistent with Facebook’s content policies and values.” In other words, the charter implicitly excludes human rights as a normative standard for the Board to “review” actions by the company.Footnote 287 Instead, it merely stipulates that “[w]hen reviewing decisions, the Board will pay particular attention to the impact of removing content in light of human rights norms protecting free expression.”Footnote 288 The other relevant documents, especially the bylaws, only partially reproduce the charter’s wording regarding the community standards while entirely omitting human rights law in the context of case review.Footnote 289 Therefore, Meta’s initial vision for the Board did not include a prominent role for anything but rules made by Meta executives themselves, especially not for international human rights law.Footnote 290 Instead, at least one strong idea in the creation process was to base the Oversight Board’s control entirely, or at least predominantly, on rules and “values” devised by the company itself. Hence, just as with Article 21 of the DSA’s review of the criteria governing “incompatibility,” Meta’s idea for the Oversight Board was to create an adjudicator that would, however, only apply Meta’s internally devised rules to control the company.Footnote 291
4.4.2.2 UN Guiding Principles on Business and Human Rights
Although international human rights law is not binding for private companies and even though the Board’s foundational documents do not grant it a prominent place, normative bridges between international human rights law and bodies like the Oversight Board do exist. The main bridge connecting corporations and international human rights are the United Nations Guiding Principles for Business and Human Rights (UNGP).Footnote 292 According to its author, UN Special Representative John Ruggie, this UN normative framework rests, apart from states’ (and the EU’s) duty to protect individuals from human rights abuses by corporations, largely on a voluntary “responsibility” of corporations “to respect human rights” as well as on “remediation through legitimate processes.”Footnote 293 Also, the UNGP demand “judicial and non-judicial remedies” by states.Footnote 294 Further, they stipulate that “[i]ndustry … should ensure that effective grievance mechanisms are available.”Footnote 295 However, as betrayed by its title, the UN’s “principles” in this context are “guiding,” not legally binding. They also tend often to be vaguely formulated, asserting for example that “[b]usiness enterprises should respect human rights. … [which] refers to internationally recognized human rights.”Footnote 296 How stringently corporations are expected to honor their “responsibility to respect” in this UN framework is not a question of legal accountability but of managerial competence and operational feasibility against the backdrop of the profit motive.
4.4.2.3 The ICCPR, Especially Article 19
Among these internationally recognized human rights mentioned in the UNGP is the ICCPR. The ICCPR contains several human right provisions. Crucial for our context is Article 19, which establishes an individual right to freedom of expression. A close reading paired with a word-count analysis of the Board’s decisions taken up to November 2024 shows that Article 19 of the ICCPR – and not Meta’s values or content policies – is the prime norm-oriented focus of the Board. The justification requirements of Article 19 of the ICCPR inform the Board’s core normative considerations. The decisive parts of Article 19 of the ICCPR read:
2. Everyone shall have the right to freedom of expression; …
3. The exercise of the rights provided for in paragraph 2 of this article carries with it special duties and responsibilities. It may therefore be subject to certain restrictions, but these shall only be such as are provided by law and are necessary:
(a) For respect of the rights or reputations of others;
(b) For the protection of national security or of public order (ordre public), or of public health or morals.
Based on these paragraphs, the Oversight Board came up with a three-pronged test for Meta’s compliance with its “human rights responsibilities,” as the Board refers to them. The subject of this test can be either Meta’s rule enforcement (for example, because it deleted a post), or Meta’s rulemaking (because it prohibited a post). Often, both aspects are tested together, lacking a formal distinction.Footnote 297 The three prongs are: legality; legitimate aim; and necessity and proportionality. While the “legality” section mainly tests the quality of Meta’s rulemaking, the “necessity and proportionality” section often mixes rulemaking and rule enforcement.
4.4.3 Detached from EU Law
Despite its engagement with international human rights law, the Board does not engage with regional human or fundamental rights regimes. Notably, neither does it reference or apply the EU’s Charter of Fundamental Rights (CFR) to cases dealing with content posted within the bloc or by users residing in the EU.
In principle, the CFR binds the EU and Member States when applying EU laws.Footnote 298 In principle, it does not bind corporations. However, the European Court of Justice (ECJ) applies several CFR provisions and other guarantees that could be understood as fundamental rights – such as nondiscrimination and equal pay – to private actors.Footnote 299 Further, various member-state domestic courts have regularly infused the principle of fundamental rights into their assessment of relevant contractual relations, subjecting the corporation then known as Facebook to due-process requirements.Footnote 300 As we saw in the previous chapter, at least one newly certified ODS in the EU outrightly applies the charter between social media platforms and individuals.Footnote 301 This descriptive part is not the place to discuss whether the Board might be under a legal obligation to apply EU law, and particularly the CFR, in cases legally located in the EU. For the purposes of this section, it is sufficient to note that the Oversight Board operates entirely detached from the normative layers of supranational or domestic law. For example, cases “where the underlying content has already been blocked, following the receipt of a valid report of illegality, and not removed for Community Standards violation” are inadmissible. In other words, the Board does not aim to interfere with regimes like Germany’s now defunct Network Enforcement Act (NetzDG) or the DSA. Rather it aims to operate free from such legal constraints, using only Meta’s own standards as well as international law. From the global perspective in which Meta operates, such an approach makes sense, especially for a private company. From an economic standpoint, diversifying standards for different regions across the world would make Meta’s governance structure more costly.Footnote 302 But it also means that the Board’s norm-based reasoning does not reflect regional (specifically EU) legal frameworks.
4.5 Decisions
After the previous section introduced the normative layers guiding the Board’s decision-making – Meta’s values and content policies as well as international human rights law – this section investigates how the Oversight Board applies these rules in its decisions. To do so, the section presents qualitative insights as well as a quantitative analysis of the word count in different parts of Board decisions. To anticipate: almost all Board decisions follow the same overall structure for their different normative layers as well as, where applicable, within the human rights assessment it produces. This allows us to compare how much the Board writes and argues on certain aspects by counting and comparing the word distribution among these sections. Word count does not necessarily equate to the quality or “importance” of a text’s normative reasoning. However, insights from interviews and a close reading of Board decisions indicate that international human rights law, and especially Article 19 of the ICCPR, is the normative focus of the Board. Thus, a thorough understanding of how many words the Board allots to analyzing Article 19 of the ICCPR, in comparison to other normative phenomena like values or community standards, might hint at the relevance of human rights law for the Board’s practice.
After a brief introduction to the general layout of Board decisions, this section then engages with some of the Board’s reasoning to gain an impression of how the Board uses human rights thinking in its deliberations. After outlining how using word count as the basis for a quantitative analysis may corroborate these qualitative insights and statements made by interviewees, the section proceeds to examine how much the Board writes in terms of the three normative layers – content policies, values, and human rights – as well as in the three subsections within the human rights assessment. This provides us with a quantitative basis upon which to draw several conclusions.
4.5.1 What Board Decisions Look Like
The Board published its first decisions on January 28, 2021. Since then, it has issued more than 100 decisions (as of November 2024). Initially, all decisions followed the same procedure and textual structure. However, after some criticism that the Board was too hesitant and slow, the Board began issuing three different types of decisions. These three types are “standard,” “summary,” and “expedited.” Standard decisions are in-depth reviews of specific and often systemic issues, whereas the function of summary decisions largely seems to be to check on the implementation of prior standard decisions. Expedited decisions are relatively rare and offer fast-tracked procedures in situations of imminent real-world harm, for example regarding videos of abducted Israelis in the context of the October 7 attacks or the aftermath of an Israeli strike on Al-Shifa hospital in Gaza. Whereas summary decisions offer only cursory analysis and reasoning, expedited decisions go into considerable detail. Both do not include a public comment section. The following largely focuses on standard decisions. Until the time of writing in November 2024, the Board issued sixty-eight standard decisions (one of which was essentially empty as the user withdrew their appeal, so it is effectively sixty-seven decisions).
All decisions are published on the Board’s website.Footnote 303 They are published in English and, at least in standard and expedited decisions, the language(s) of the original dispute. The texts have a serial number and a title.Footnote 304 Titles are crisp and reference the subject matter of the case, for example “Former President Trump’s suspension” when the Board decided whether Meta was right to indefinitely suspend Trump’s accounts from its platforms after his role in the 6 January 2021 riots.Footnote 305 Other titles point to the wide range of topics and contexts the Board deals with, for instance “Alleged crimes in Raya Kobo” (about communication related to ethnic tensions in Ethiopia), “Colombian protests” (about criticisms of the Colombian President), “Mention of the Taliban in news reporting” (covering reporting on terrorist regimes), “Claimed COVID cure” (relating to misinformation about the Covid-19 pandemic in France), or “Cambodian Prime Minister” (when the Board recommended to Meta to suspend the account of Hun Sen).
All standard decisions contain roughly the same sections, typically in the same sequence, with slight variations over time. Usually, decisions begin with a case summary and a summary of the Board’s key findings. The decision’s text is separated into several parts, typically nine or ten. These are, a decision summary, a description of the case, a few remarks on the authority and scope of the Board’s review, and a description of the relevant normative standards. In this part, the Board typically cites the relevant paragraphs of the community standard and sets out the human rights-related norms it sees as pertinent to the case. Here, the Board does not engage in substantive reasoning. Further, the decisions summarize the statements made by the user in their appeal, as well as Meta’s explanation for its decision, and submissions by third parties. Then, the Board engages in a normative assessment called “Oversight Board Analysis.” In earlier decisions this was always part eight and since mid-2024 part five. This is the only part of the decision where the Board assesses facts, weighs different interpretations, and engages in normative reasoning. Standard decisions usually conclude with one or two brief parts consisting of the Board’s nonbinding recommendations and its binding decision. These last two parts are sometimes combined.
Two things are noteworthy about this structure. To begin with, it is quite striking that the Board rather strictly adheres to it. All sixty-seven standard decisions published at the time of this writing follow that structure, with only minor deviations along the way. That a transnational body – one that deliberates via video call and whose decision-makers are spread across five continents – can commit to such a uniform working method already tells us something about the Board’s strategic vision and discipline. Further, the sequencing of the parts appears to be deliberate. Outlining facts and normative standards before engaging in normative reasoning to arrive at a conclusion consisting of a decision and several recommendations implies that the Board members see themselves – or at least want to present themselves – as appliers rather than creators of the law. This viewpoint is reminiscent of Montesquieu’s famous observation that “Le juge est le bouche de la loi,” and mirrors the practice of many international adjudicators.Footnote 306 Setting out a decision only at the very end of a twenty- to thirty-page document resembles the style of decision-framing known from practitioners at the European Court of Justice, the International Court of Justice, and the European Court of Human Rights but also EU Boards of Appeals. In contrast, domestic courts like those in Germany or the United States begin their judgments by presenting the decision; they only explain it later.
In its decisions, mostly within the “Oversight Board Analysis” part, the Board engages with various normative materials. These are, international covenants and authoritative commentary, most notably the ICCPR and the UN Human Rights Committee’s General Comments.Footnote 307 Further, the Board quotes decisions of international courts (especially the European Court of Human Rights),Footnote 308 the US Supreme Court,Footnote 309 and legal scholars.Footnote 310 It engages with decisions and other normative material produced by a wide range of adjudicators, including the UN Human Rights CommitteeFootnote 311 and the Committee on the Elimination of Racial Discrimination (CERD) Committee.Footnote 312 In particular, the Board’s legality and proportionality assessments engage in some detail with the acquis of international human rights law, most notably Article 19 of the ICCPR.Footnote 313 In several decisions the Board reiterated criticism of the company’s rulemaking and rule enforcement.
4.5.2 Tracing the Board’s Turn to Human Rights
Several interviewees pointed to human rights as determinative normative category in their day-to-day work.Footnote 314 Many staffers and several Board members have a background in human rights, and the Board’s first Director, Thomas Hughes, previously worked for an NGO literally called “Article 19.”Footnote 315 Hughes would later run the Oversight Board spin off, the Appeals Centre, in Dublin.Footnote 316 Because Meta’s community standards and values are so convoluted and vaguely worded, they are of little help in deciding cases. Therefore, what began as a mission to control the enforcement of rules quickly faced the reality that, not only was enforcement problematic, its underlying rules were not fit for purpose as well. To enhance the quality of Meta’s rules, especially its community standards, the Board would have to resort to external standards. Especially from a legitimation perspective, citing and thereby tying oneself to international law autonomizes and legitimizes the Board vis-à-vis the company as well as in relation to national regulators. From the Board’s perspective, this hits two birds with one stone. One interviewed Board member described the system in this way:
[A] purely formalist review system is meaningless. Yes, there are important things to do in making sure that Facebook is applying its own rules correctly and consistently, but the big issues are substantive. … We need some sort of external standards here. … Assuming we’re not dealing with the legal and we don’t have any democratic footing, it seems that human rights is the best bet … for … a scope of externally justifiable values that have received a lot of consensus-based legitimacy and been drawn out over 50 years.Footnote 317
A close reading of the Board’s decisions corroborates important aspects of this member’s claim. The Board confines most of its argumentation, reasoning, and citing of other adjudicatory material – for example, by referring to material from the UN Human Rights Committee or the European Court of Human Rights, other texts, such as reports by UN Special Rapporteurs, or the “Rabat Plan of Action,” and the arguments of legal scholars – to the “Oversight Board Analysis.” In one subsection of that “Analysis” section, the Board analyzes whether Meta is complying with its human rights responsibilities in the case at hand.Footnote 318 Virtually every standard decision features this human rights-focused subsection. Citing other adjudicative decisions, primary and secondary legal material as well as scholarship are key ingredients of adjudicative reasoning.Footnote 319 This material situates and thereby legitimizes normative utterances by an adjudicator in the more general system comprising the norms, applications, institutions, and practices of legality.Footnote 320 The Board also sometimes discloses internal disagreement in this human rights section. Several decisions feature passages contrasting majority and minority positions among Board members in the Board’s human rights assessment.Footnote 321 For example, in a decision concerning communication in the context of the Azerbaijani-Armenian conflict, the Board stated:
The majority noted that the post, when read as a whole, made clear the user’s choice of slur was not incidental but central to the user’s argument that the target group was inferior. Moreover, the post in question was widely disseminated at the height of an armed conflict between the user’s State and the State whose nationals the post attacked. … A minority found Facebook’s deletion of the post was not proportionate, on the basis that the risks cited by the majority were too remote, and were not foreseeable.Footnote 322
Contrasting minority and majority positions within a multiperson adjudicator has several effects.Footnote 323 It might delegitimize the adjudicator as its decisions could then be perceived as the outcome of bargaining or as a subjective interpretation and not as a clear evidence-based consequence of the interpretation of a particular legal text. On the upside, it could be argued that admitting to norm-related disagreements among adjudicators paints a more realistic picture of their actual deliberations and offers a broader variety of perspectives on the decision. This might bolster an adjudicator’s legitimacy as it increases transparency and presents adjudication as what it is: a process of reasoning that will usually allow for more than one conclusion.
In contrast, the Board only occasionally engages with Meta’s values. For example, in its first closely reasoned decision, the Board wrote 320 words on Meta’s compliance with its own community standards, 691 words on compliance with international human rights law, and a meager forty-six words regarding compliance with Facebook values.Footnote 324
4.5.3 Introducing Word Count as a Quantifiable Unit
But how does the Board reason, and on what issues? And how can this be studied objectively? This section draws on several quantitative analyses of Board decisions to show that the Board increasingly grounds its reasoning in international human rights law—rather than Meta’s self-made rules—and subjects Meta’s rulemaking itself to growing scrutiny.
Based on the interviews, a close reading of the decisions, and academic works looking at the value of argumentation and citation in adjudicative texts, we can assume that the Board writes most about what it considers normatively important or otherwise relevant.Footnote 325 Writing more will increase word count. Hence, within the Board’s normative reasoning (which is always confined to the “Oversight Board Analysis” part), each section’s word count, that is, that of the section on Meta’s community standards, the company’s “values,” or crucially, international human rights law, may serve as a proxy indicator for that section’s importance.Footnote 326 Hence, analyzing the word count aims to answer two questions. First, how much does the Board write about Meta’s self-given community standards, “values,” and international human rights law respectively? Second, how much does the Board write that relates to the different subsections (i.e., legality, aim, necessity, and proportionality) within its human rights assessment? This again may tell us something about which issues and norms the Board considers particularly important – and which arguments it uses to lend weight to its largely nonbinding normative input vis-à-vis the company.
4.5.4 Quantifying the Board’s Focus on Human Rights
Let us begin with the first question: How much does the Board write when it comes to each of the three sets of normative material? That is, how many words does the Board allot to community standards, values, and human rights in its decisions? To repeat, all standard decisions follow the same structure. In every decision, the Board’s normative reasoning is contained to one specific part labeled “Oversight Board Analysis.” Initially, this Oversight Board Analysis part cascaded into three sections, one for each normative level, that is, Meta’s community standards, Meta’s values, and international human rights law. These “values” are presented by way of five vaguely worded normative statements unilaterally made by the company,Footnote 327 whose purpose is to “guide” Meta’s enforcement and rulemaking.
However, before digging further, we may have a look at the overall development of the word count of the “Oversight Board Analysis” section. They key takeaway is: on average, it grew considerably. Figure 4.1 shows the total word count of each decision in columns and the trend (exponential regression) as a dotted line. Thus, the Board wrote more and more over time in its normative analysis.
Word count Oversight Board Analysis total (2021–11/2024)
Figure 4.1 Long description
The x-axis ranges from 1 to 67 in increments of 2, while the y-axis ranges from 0 to 6,000 in increments of 1,000. The line starts from 1,000 at 1, and ends with about 3,000 at 67, going through large fluctuations throughout. The best-fit line stretches from about 2,000 at 1 to about 4,000 at 67.
Now, we unpack the different subsections of the “Oversight Board Analysis.” As previously noted, the Oversight Board Analysis contains the Board’s normative reasoning and initially divided into three, since late 2022 into two parts. That missing third element are Meta’s values, which were the shortest section from the very beginning. In all but one of its decisions, fewer than 500 words of text were devoted to discussing Meta’s values. Often it was considerably lower, averaging out at 181 words. Later, as those values were ostensibly useless to properly review Meta’s exercise of power, the Board largely ceased referencing them. Since late 2022, decisions do not contain a specific section dealing with Meta’s “values” anymore.
In turn, the Board’s engagement with Meta’s community standards and international human rights law seems to have grown consistently over the years. In the first two years, the Board usually allotted between 500 and 1,000 words to its engagement with Meta’s community standards. Since 2023, that number climbed considerably to often more than 1,500 words. The overall average as of November 2024 being 855 words. While the growth in words allotted to the community standards appears, on average, steady but not exponential, the human rights section ballooned over the years. Here, the average word count was much higher from the beginning, being roughly 1,500 words in the first two years, reaching almost 2,000 words overall in November 2024 and 2,350 if one looks at only the decisions since 2023. In many decisions, the section on human rights comprised more than 2,000, 3,000 and in one case even more than 5,000 words. In all decisions, human rights standards “received” more words than the sections regarding Meta’s values; and almost always more than in the section relating to Meta’s community standards.
Figure 4.2 shows the different word counts distinguished by applied normative standard, namely Meta’s values and policies, as well as international human rights law. Further, comparing the trends of Figures 4.1 and 4.2 we see that the Oversight Board Analysis and especially its section on human rights became longer over time. In contrast, the section on Meta’s values was largely stable and then disappeared while the section where the Board analyzes only Meta’s community standards, grew but by far not as strongly as the human rights section. Thus, the expanded word count in the human rights section appears to be the main driver for the (over time) greater amount of text devoted to normative reasoning in the Board’s decisions.
Word count of Meta values, Meta policies, and human rights law in the normative reasoning part of Oversight Board decisions (2021–11/2024)
Figure 4.2 Long description
The y-axis ranges from 0 to 6,000 in increments of 1,000. The x-axis starts with 2021-003, 2021-004, 2021-005, and so on, and continues till 2024-041. There are 5 lines in the graph. Community Standards (starting at about 350 and ending at about 400) and Human Rights (starting at about 1,100 and ending at about 2,500) show high fluctuations throughout. The line for Values, although fluctuating, stays much lower (below 500) throughout and ends abruptly midway. The straight exponential line for community standards starts at 300 and ends at 1,050. The straight exponential line for human rights starts at 1,250 and ends at 2,700. All values are approximate.
In total, the values received roughly 6,000 words, the content policies roughly 60,000 words, and human rights a bit more than 130,000 words. Even though such total values mix together lengthy and more briefly worded decisions, the disparity indicates that the Board’s normative reasoning mainly takes place in that part of the decision that focuses on human rights law.
What can we conclude from Figure 4.3? The increase in words may not necessarily indicate that the Board considers human rights more “important” and that therefore the decision’s authors take it upon themselves to write more. The text may simply be more imprecise, lengthy, or repetitive. However, as a close reading of the decision shows, most of the text allotted to the human rights subsection does indeed read like legal argumentation. For example, in its human rights analysis in the decision relating to the banning of former US President Trump, the Board cautiously constructed a normative standard against which Meta’s actions could be measured:
In international law on freedom of expression, the principle of legality requires that any rule used to limit expression is clear and accessible. People must be able to understand what is allowed and what is not allowed. Equally important, rules must be sufficiently clear to provide guidance to those who make decisions on limiting expression, so that these rules do not confer unfettered discretion, which can result in selective application of the rules. In this case, these rules are Facebook’s Community Standards.Footnote 328
After outlining this normative standard by recourse to Article 19 of the ICCPR, the Board applies the facts, cites experts (like a UN Special Rapporteur, for instance), and refers to previous decisions. The same decision argues:
The clarity [of Meta’s community standard] leaves much to be desired, as the Board noted in a prior decision (case 2020–005-FB-UA). The UN Special Rapporteur on Freedom of Expression has also raised concerns about the vagueness of the Dangerous Individuals and Organizations Standard (A/HRC/38/35, para 26, footnote 67). As the Board has noted previously in case 2020–003-FB-UA, there may be times in which certain wording may raise legality concerns, but as applied to a particular case those concerns are not warranted. Any vagueness under the terms of the Standard does not render its application to the circumstances of this case doubtful. The January 6 riot at the Capitol fell squarely within the types of harmful events set out in Facebook’s policy, and Mr. Trump’s posts praised and supported those involved at the very time the violence was going on, and while Members of Congress were calling on him for help.Footnote 329
In no decision does the Board employ comparable argumentative resources to assess Meta’s values and only in a few decisions (and to a lesser extent within those decisions) to Meta’s community policies. Although the Board’s engagement with Meta’s community standards grew over time, the main forum seems to remain human rights law. Therefore, the qualitative insights from the close reading of paragraphs like the above and the insights from the interviews substantiate the claim that we can plausibly assume that the Board has over time shifted its attention away from Meta’s own rules and toward human rights. Whether this is because human rights are perceived as more legitimate, more pertinent, or simply because many employees of the Oversight Board previously worked in a human rights context, is uncertain.
Total word count “Oversight Board Analysis” per normative category (2021–11/2024)
So, to answer the first question, we can conclude that the Board allots by far the most words to the human rights section of its decisions. It writes considerably less in both sections dealing with either Meta’s community standards of the later defunct section on Meta’s values.
4.5.5 The Two Forums: Legality and Proportionality
Now let us move to the second question: how many words does the Oversight Board allot to the three subsections within its assessment of Meta’s human rights responsibilities? Does the Board write more in the sub-section dealing with the normative basis of a measure, which the Board calls based on the framing of Article 19 ICCPR “legality?” Or does it focus more on a measure’s “legitimate aim,” or the “necessity and proportionality” of a measure? And what can this tell us about the Board’s normative focus?
In Figure 4.4 we see that the Board writes the least in the subsection dealing with the “legitimate aim” of a measure. Here, it typically allocates less than 200 words, averaging out at only roughly 140 words. More usually, but not as a consistent trend across all the decisions, the subsection on “legality” has a greater amount of text. This averages at roughly 440 words but peaks at 1,457 words and dips as low as thirty-five words. In several cases the legality subsection had the highest number of words devoted to it and, when this occurred, this was, at least in the first two years of practice, often in inverse proportion to the word count of the subsection covering a measure’s “necessity and proportionality.” In other words, when the part of a decision dealing with legality is longer, then the part focusing on issues of necessity and proportionality tends to be shorter, and vice versa. However, this correlation seems to have largely ceased. Decisions of 2023 and 2024 appear to have a more equidistant relationship of the sections on legality and necessity and proportionality. This might be plausibly explained through incrementally manifesting practices within the Board as to its approach and style of reasoning.
Word count of legality, legitimate aim, necessity, and proportionality subsections in the “Oversight Board Analysis” part (2020–11/2024)
Figure 4.4 Long description
The y-axis ranges from 0 to 3,500 in increments of 500. The x-axis starts with 2021-003, 2021-005, 2021-007, and so on, and continues till 2024-041. The legality line starts at about 250 and ends at about 300, going through major fluctuations and showing a peak of 1,500 at 2022-009-010. The necessity and proportionality line starts at about 700 and ends at about 1,100, also going through major fluctuations, with a peak of 2,500 at 2022-009-010 and 3,400 at 2024-004-005-006. The legitimate aim line also goes through fluctuations, but stays below 500 throughout, only showing a peak of about 750 at 2022-009-010.
Furthermore, we see a trend occurring over time in changes to the word count in the subsections covering legality, aim, and necessity and proportionality.Footnote 330 Whereas the (legitimate) aim’s word count remains largely stable, the word count increases much stronger over time for the legality and, particularly, the necessity and proportionality sections. Simply put, the Board writes increasingly more about legality and necessity and proportionality than about the legitimate aim.
This could mean that the legality and the necessity/proportionality parts are the two main forums for normative argumentation within the “Oversight Board Analysis”. If that were the case, this would be remarkable. Whereas the necessity and proportionality section typically concentrates on an actor’s enforcement action, the legality section focuses solely on this enforcement action’s legal basis. Traditionally, Art 19 ICCPR requires any restrictions speech to be based on a legal rule such as an anti-libel or anti-hate speech law passed by parliament. The Board, however, resorts to Meta’s self-made rules as “legal” basis for speech restrictions. In other words, when the Board devotes a lot of words to the legality of Meta’s actions and not only their proportionality, this might suggest it is engaging in greater scrutiny of Meta’s rulemaking. Simultaneously, the increasing word count in the necessity and proportionality section might also indicate the Board’s increased scrutiny of Meta’s enforcement procedures.
We now proceed with a close reading of the Board’s conclusions about the “legality” of Meta’s moderation efforts and their necessity and proportionality.
4.5.5.1 The Board’s “Legality” Test
Let us now have a look at the legality section. Legality, a concept rooted in Article 19 of the ICCPR, feels somewhat out of place in the entirely private context we find ourselves navigating here. As envisaged by the United Nations’ Guiding Principles, the Board requires Meta’s restrictions of free expression to be “provided by law.”Footnote 331 But what is Meta’s “law” in the context of Article 19(3) of the ICCPR? Meta has huge problems producing normative material that works on the worldwide scale on which the company operates. Normative material regulating human behavior must be concise and clear. As we have seen above, most of the rules established by Meta are neither concise nor clear. Further, the notion of “law” in Article 19(3) of the ICCPR is traditionally understood in a rather narrow sense. Based on the Human Rights Committee’s “General Comments,” this “may include laws of parliamentary privilege … [but not] traditional, religious or other such customary law.”Footnote 332 The Board, however, never discussed whether notions like “law” or “legality” are applicable for entities like Meta. It just applied them. Meta does not feature a rule-based let alone participatory process of rulemaking.Footnote 333 How can its privately devised and ad hoc rules be “law” within the meaning of Article 19 of the ICCPR?
Frictions will always occur when transplanting intrinsically state-centered rules into a private-law context, but this awkward grafting is still striking. On the one hand, the Oversight Board, for good reasons, generally avoids discussions of “sovereignty” or whether an entity like a transnational company or a social media firm is “state-like.”Footnote 334 On the other side, Article 19 of the ICCPR’s legality requirement offers the Board an opening to review not only Meta’s “enforcement decisions” but also its rulemaking. In several decisions, the Board’s evaluation in the legality section assessed the content and quality of Meta’s community standards. Often such assessments result in comprehensive recommendations to Meta outlining what aspects of the community standards the company should change. For example, in a 2021 decision regarding support for the – later deceased – Russian opposition politician Alexei Navalny, the Board demanded a complete overhaul of one of the then twenty-four community standards.Footnote 335 Its criticism was that the different subsections and rules of the standard were unclear, that it included many words with unclear definitions (what is the difference between bullying and harassment?) and obscured rather than elucidated the true meaning of the standard. Part of its decision read:
Facebook should amend and redraft its Bullying and Harassment Community standard to … explain the relationship between policy rationale and the “Do Nots” as well as the other rules restricting content that follow it. … Distinguish between bullying and harassment and provide definitions that distinguish the two acts. … Clearly define its approach to [different groups of users] … Include illustrative examples of violating and non-violating content.Footnote 336
Such criticism is often renewed in different, subsequent cases, especially in following “summary” decisions. The Board regularly criticizes the company for offering too few examples, for its slow progress in translating standards and decisions, and its unclear and incoherent definitions. For example, in a 2022 decision regarding communication about the Taliban, the Board reiterated that it considers the relationship between two definitions in one particular standard to “remain unclear to users.”Footnote 337
In sum, the Board extensively uses its decisions’ legality subsection to control – or at least comment on – Meta’s rulemaking. The increase in word count for this part of its decisions, coupled with statements by interviewees, therefore indicates that a prime focus of the Board’s deliberation is Meta’s rulemaking.
4.5.5.2 Necessity and Proportionality
Whereas the legality section focuses on the mostly formal requirements (clarity and consistency) of Meta’s rulemaking, the necessity and proportionality section are where the Board scrutinizes Meta’s policies of rule enforcement. Typically, the Board’s necessity and proportionality assessment has a double focus. On the one hand, it concentrates on the specific enforcement action as it was conducted for the case it is reviewing. For example, in a case dealing with the wrongful deletion of a post promoting breast cancer awareness, the Board established that: “The Board is concerned that the content was wrongfully removed by an automated enforcement system and potentially without human review or appeal. … In this case, these technologies failed to recognize the words ‘Breast Cancer’ that appear at the top left of the image in Portuguese.”Footnote 338
However, the Board normally embeds assessments of a specific enforcement act in more general statements pertaining to the issue at stake. Therefore, assessing an action’s necessity and proportionality allows the Board to say something about the company’s procedures in general. For example, in the breast cancer awareness case, the Board inductively drew several general conclusions from its individual assessment of the particular case. Drawing attention to a specific failure of Meta’s automated moderation system, the Board criticized Meta’s over-reliance on automated tools and pointed out the “limitations” of the technology, especially its “limited ability to understand context.” This part of the decision reads:
This reflects the limitations of automated technologies to understand context and grasp the complexity of human communication for content moderation. … The Board accepts automated technologies are essential to the detection of potentially violating content. However, enforcement which relies solely on automation, in particular when using technologies that have a limited ability to understand context, leads to over-enforcement that disproportionately interferes with user expression. … Regardless, automated removals should be subject to both an internal audit procedure explained under section 9.2 (I) and appeal to human review should be offered, allowing enforcement mistakes to be repaired. Automated content moderation without necessary safeguards is not a proportionate way for Facebook to address violating forms of adult nudity.Footnote 339
What is striking is how the Board references its own recommendations in a later section of the same decision. It states that “automated removals should be subject to … an internal audit procedure explained [below].”Footnote 340 In the referenced section, the Board issued a series of recommendations regarding how Meta should improve its overall procedures. Among these recommendations we see again an increasing degree of generalization. The Board issued some recommendations with a specific connection to the case. For example, the company should “[i]mprove the automated detection of images with text-overlay to ensure that posts raising awareness of breast cancer symptoms are not wrongly flagged for review.”Footnote 341 However, other recommendations are more general. The company should, for example, notify users about the specific rule of the community standard Meta considers to be violated, and it should put in place an “internal audit procedure to continuously analyze a statistically representative sample of automated content removal decisions to reverse and learn from enforcement mistakes.”Footnote 342 Similar arguments and connections are made in other cases. As will be argued in Section 3.6.2 below, the Board’s recommendations constitute a crucial part of its normative output.Footnote 343 Therefore, tying them to arguments made in the necessity and proportionality section highlights that section’s importance.
In sum, a close reading of the decisions shows that the Board invests considerable argumentative resources on the necessity and proportionality section. Even though it is typically longer than the legality section, this is not always the case, which might indicate the beginnings of a trend toward stricter scrutiny of Meta’s rulemaking by the Board. However, the heart of the Board’s human rights assessment is making recommendations for Meta to tighten up its content moderation. In other words, the Board “uses” the individual case to inductively generate procedural advice for the company. This advice might be restricted to a particular subject or community standard in question. However, it can often be of a more general nature and based on principles of good governance and administration.
4.5.6 Interim Conclusion
This section detailed how the Board uses various normative categories – Meta’s community standards, Meta’s values, and international human rights law – in its decisions and arrives at several conclusions. First, the Board does not engage substantively with Meta’s “values.” Second, the Board does however engage in some detail with the company’s community standards as a way to control Meta’s enforcement actions. Third, the Board’s main normative focus – in every single decision and increasing over time – is international human rights law, especially Article 19 of the ICCPR. This is remarkable since the charter only mentions human rights law in passing and the bylaws do not mention it as a basis for review at all. Fourth, as part of its human rights assessment, the Board engages in an increasingly detailed analysis modeled on elements of Article 19(3) of the ICCPR. Typically, this includes a “legality” subsection in the Board’s written decisions that tests the normative “clarity” and “accessibility” of Meta’s rulemaking, as well as an extensive “proportionality” assessment split into “(legitimate) aim” and “necessity and proportionality” subsections. Sixth, whereas in its earliest decisions the Board tended to focus on the “necessity and proportionality” subsection, we have seen a relative increase over time in words allotted to the “legality” subsection. In the latter, the Board assesses whether Meta’s own rules, especially the “community standards”, comply with international human rights law. Even though it is too early to speak of a stable trend, we can tentatively conclude that the Board’s attention focuses both on controlling Meta’s enforcement (by questioning the proportionality of Meta’s enforcement) and controlling Meta’s rulemaking (by questioning the clarity and consistency of the rules Meta enforces). So far, the focus on controlling rulemaking has remained at the mainly formal level (relating to the clarity and consistency with which standards are formulated, for instance).
The Board’s turn to human rights aligns with remarks by staff members in the interviews who said they were limited in their ability to control individual decisions but wanted to “change how Meta makes these decisions.”Footnote 344 This turn to human rights might also be explained by the prevalence of people with a background in human rights among staffers (who draft a first legal analysis of each case) and among Board members.
4.6 Implementation
We now explores how (and whether) Board decisions are implemented. The empirical basis for assessing the degree to which Meta implements Board decisions is thin because the company remains coy about how exactly (and if at all) it implements the Board’s normative output. However, based on transparency reports by Meta and the Board, several interviews, and a close reading of Board decisions, we might be able to pin down the contours of how Meta implements, or chooses not to implement, Board decisions.
The Board generates several distinct kinds of normative output. Formally, it “decides” individual cases and offers “policy advisory opinions” based on general questions referred to it by Meta.Footnote 345 Its decisions feature an operational part that is binding for Meta for that specific case. But the document – known as the “decision” – also includes “policy advisory statements” and/or “recommendations.” These are nonbinding, generalized normative recommendations relating to how Meta should improve its rulemaking and rule enforcement. A close reading as well as remarks from several interviewees indicate that these policy advisory statements are the Board’s primary normative contribution to Meta. Further, even though policy advisory statements are not binding, many interviewees felt them to be more effective than the (necessarily individual) decision due to the general terms in which these statements were formulated and because of their focus on the company’s operations. The Board has begun to build up recurring normative patterns that could be interpreted as doctrines and call for “naming and shaming”-inspired measures to influence the company’s behavior.
Due to Meta’s largely untransparent implementation of Board decisions, the latter’s efficacy cannot be assessed comprehensively. For an adjudicator, this lack of transparency is hard to accept. Even though the Board has quickly adapted – bringing out increasingly sophisticated quarterly transparency reports and, later, a “recommendation tracker” – questions remain regarding Meta’s transparency and accountability when it comes to how, or even whether, it implements nonbinding policy recommendations and advice from the Board.
4.6.1 Set Board Decisions Binding Precedent?
Each Board decision features a brief “operational part” that assesses whether a specific piece of content should stay on Meta’s services or not. This decision is “binding” for Meta, but only for that specific enforcement action relating to the case under review.Footnote 346 Decisions do not have precedential status beyond the individual case. Meta should only evaluate whether it is “technically and operationally feasible to apply the Board’s decision” in cases with “identical content and parallel context,” according to Article 4 of the charter. What “parallel context” means is obscure, but “identical content” can perhaps be pinned down more successfully. Further, even if these criteria are met, the resulting obligation is nothing more than the requirement on Meta’s part to exercise due diligence.Footnote 347 Thus, Meta is not bound by the Board’s decision in cases other than the one decided – their executives need only “take action by analyzing” whether parallel implementation would be feasible. Hence, the “binding” power of the Board’s decisions is negligible for the Board’s practice. Especially so given the Board’s leeway in case selection (subject to Meta’s approval) and its relatively small caseload.
4.6.2 Infusing Public Law Principles Into Private Governance?
Almost all the Oversight Board’s decisions feature nonbinding, more general normative statements called either “recommendations” or “policy advisory statements.” These statements inductively generate principles for Meta’s rulemaking and rule enforcement. Interviewees described these statements as the normative core of the Board’s decisions.Footnote 348 One interviewee emphasized that “albeit the Board makes its decisions in a way that can be understood as adjudicative, our main way of exercising our power are the policy recommendations.”Footnote 349 These are attached to each decision in their own section, typically the tenth. In these recommendations, the Board formulates general guidelines and principles recommending what the Board “should” do to improve its rulemaking and rule enforcement. As one trustee interviewed for this book put it:
The Oversight Board is going to decide maybe 40 or 50 cases in a year, out of billions of posts. It follows that the Board itself is not the manager of disputes. The Board is not like a police force. Instead, it’s infusing the content moderation operations of Meta with public law principles and procedures. It’s articulating substantive norms that interpret Meta’s community standards in light of human rights principles.Footnote 350
What the interviewee describes as an infusion of “public-law principles” into Meta’s managerial way of governing its users are the nonbinding recommendations and advisory statements that the Board attaches as supplements to most of its decisions. In its third decision, the Board explicitly formulated such recommendations for the first time.Footnote 351 Since then, each decision has featured at least one such “policy advisory statement.” These statements draw upon normative or factual assessments made by the Board in its analysis, often with a special focus on human rights-related implications.
For example, in a case concerning posts referring to the – then still alive – Russian opposition figure Alexei Navalny, the Board used the legality assessment of Meta’s bullying and harassment community standard to assess this standard’s normative coherence. It found that the whole standard was constructed in an overly complex way, making it hard for users to know “what is and what is not allowed.”Footnote 352 The Board criticized several categories Meta introduced in the standard as it were not clear if they “serve[d] simply as a guidance for the specific rules that follow, or if they must be interpreted conjunctively with the rules.”Footnote 353 To back up this criticism, the Board then formulated a ‘policy advisory statement’ outlining what Meta should do. The Board recommended that “[t]o comply with international human right standards, Facebook should amend and redraft its Bullying and Harassment Community Standard.”Footnote 354 It then formulated six tangible reforms. For example, the Board should “explain the relationship between the policy rationale and the ‘Do nots’ as well as the other rules restricting content that follow it.”Footnote 355 Further, the standard should differentiate between bullying and harassment and “should clearly explain to users how bullying and harassment differ from speech that only causes offense and may be protected under international human rights law.”Footnote 356 Meta should “[c]learly define its approach to different target user categories and provide illustrative examples for each category (i.e. who qualifies as a public figure).”Footnote 357
Most of the Board’s recommendations in fact reflect rationales, principles, and notions familiar from public-law contexts. Regarding Meta’s rule enforcement, the Board recommended, for example, that the company should provide reasons for its decisions.Footnote 358 It should ensure that users are notified that their content has been removed and which particular violation had led to its removal.Footnote 359 It should also provide timely translations so that its rules are intelligible to users.Footnote 360 Users must have the right to appeal, sometimes directly linked to the ICCPR’s right to an effective remedy. Several recommendations explicitly referred to the principle of proportionality.Footnote 361 Further, the Board recommended specific procedures to Meta for investigating whether its enforcement algorithms might be biased.Footnote 362
In terms of Meta’s rulemaking, the Board regularly recommends that the company includes more examples,Footnote 363 as well as rules that are worded in a clearer and more concise way.Footnote 364
In sum, the Board’s policy advisory statements synthesize principles from external normative material – human rights law and secondary sources (like General Comments from the UN Human Rights Committee), non-governmental or international organizations i documents (the Rabat Plan of Action), and scholarship – and the statements offer recommendations for Meta to implement in order to improve its rulemaking and rule-enforcement efforts.
4.6.3 Developing Doctrine
The Board also began to develop normative sentences that could be described as doctrines. Although conceptions of what counts as doctrine differ greatly among legal cultures, several characteristics seem identifiable.Footnote 365 The function of doctrine is to stabilize a normative order by establishing hierarchy, clarity, and foreseeability. Doctrine creates general normative sentences guiding future behavior.Footnote 366 The Board uses citations and references to incrementally construct “doctrines.”
On the one hand, the Oversight Board increasingly returns to issues it brought up in earlier decisions but considers insufficiently remedied. The Board clarifies or reiterates prior normative statements and criticizes Meta for insufficient implementation. Such repetitions often include citations of earlier Board decisions. For traditional courts, citing decisions’ titles or excerpts is widely considered as a useful tool for maintaining normative stability and developing precedential law and bodies of legal doctrine.Footnote 367 Further, the Board references international courts and tribunals. For example, in a case concerning hate speech in the context of ethnic conflicts in Ethiopia, the Board referred to prior decisions as well as to judgments by the International Criminal Tribunal on Rwanda (ICTR). The Board said that it “came to a similar conclusion in decision 2020–003-FB-UA. [And that] … cumulative [hate speech] can amount to causation through a ‘gradual build-up of effect,’ as happened in Rwanda where calls to genocide were repeated [then quoting an ICTR case].”Footnote 368
Citing older cases may not automatically mean that an adjudicator is accurately recreating earlier normative assessments.Footnote 369 Citations are also sometimes used “ritualistically.”Footnote 370 However, for a young but ambitious organization like the Oversight Board, such “ritualistic” practices are important for gaining public legitimacy.
Some of these “doctrines” directly challenge traditional conceptions of international law. The Board increasingly formulates abstract interpretations of Meta’s normative material and human rights law. Perhaps the most striking example of such an abstract “doctrine” fashioned by the Board is its reference to a state of “public emergency” and Meta’s resulting recourse to “emergency powers” to suspend its fundamental rights obligations. We will discuss said “doctrine’s” ripple effects in greater detail in the next chapter.Footnote 371 For descriptive purposes, we will just briefly look at that decision’s decisive paragraph. In a case dealing with potentially incendiary speech about Tigrayans in Ethiopia. The content in question was posted by an individual user, but the Ethiopian government was reported to post similar content. The Board asserted:
The Board finds that in a country where there is an ongoing armed conflict and an assessed inability of governmental institutions to meet their human rights obligations under international law, Meta may restrict freedom of expression that it otherwise would not (see ICCPR Article 4 on derogations in times of public emergencies).Footnote 372
In other words, the Board emboldens a private, for-profit company to step in whenever it considers governmental institutions to be failing in their human rights obligations in situations of armed conflict. But not only that. The Board essentially emboldens Meta, again, a private company, to declare a state of emergency vis-à-vis a government to justify policing that government tighter than before. We will discuss this in more detail in the following chapter. Yet, I take the liberty of highlighting the audaciousness of such doctrines already here. Arguments like these directly challenge core tenets of international law. They are reminiscent of the earliest plans for the Board, outlined in an internal memo which argued that “by acting to a degree like a sovereign in establishing courts, Facebook presents itself as a responsible decision-maker, prepared to use a method of decision-making deemed valid and legitimate (at least in principle) by most governments on earth.”Footnote 373 The Board follows this path – even though its actual authority falls far short of earlier ambitions expressed by some within the company.Footnote 374 Asked about the audacity of such state-of-emergency arguments, one Board member interviewed, who was not party to the deliberations relating to that specific decision, affirmed that “the Board members are acutely aware of the potential effects of statements like that.”Footnote 375 Time will tell what the effects will ultimately be, but the significance of the potential downstream effects is hard to overstate.
4.6.4 The Implementation Conundrum
Yet, despite intermittently emerging doctrines, the Board’s recurring focus on Meta’s governance, and the Board’s lists of recommendations, the question remains: does it work? Does Meta actually implement the Board’s nonbinding recommendations?
As argued in Chapter 2, nonbinding recommendations and doctrines can also have lasting effects. When an adjudicator manages to convince the controlled actor, this may lead to more deeply rooted policy changes than the exercise of adjudicatory force through issuing binding decisions.Footnote 376 Because adjudicators seek to “adjust future behavior, not compensate for past wrong,” their involvement is prolonged as they construct an “ongoing regime of performance.”Footnote 377 Accordingly, the Board regularly returns to issues it raised in earlier decisions to “guide” the company. Dialogue-based and nonbinding adjudicatory techniques have been extensively explored by legal scholars and, to a degree, might also work for the benefit of the Oversight Board.Footnote 378 However, many dialogue-based forms of adjudicatory control are exercised by actual courts, which still have the ultimate tool of comprehensive and binding decisions at their disposal. In contrast, as shown above, the Board’s authority to issue binding decisions exists mainly on paper, so the Board – although seen by some as a potent adjudicator – cannot bindingly adjudicate at scale. To make its normative output more effective, the Board could draw on the perceived legitimacy attached to its partly adjudicative status but ultimately it must convince Meta executives to implement its recommendations. Such “good faith”-based implementation reflects the private nature of the Board and of Meta. After all the Board can revert to neither the institutions nor the norms of the rule of law.
As a consequence, Meta executives regularly choose not to put in place the Board’s recommendations in a verifiable way. The story is of fumbled implementing efforts, and the operational complexity of the recommendations. Meta staffers argue that the
size and scope of the Board’s recommendations go beyond the policy guidance that we first anticipated when we set up the Board, and several require multi-month or multiyear investments.Footnote 379
This aligns with statements from interviewed Board staffers who said their counterparts at the company were surprised and even unhappy with the depth and intrusiveness of Board recommendations – which arguably would be a good thing.Footnote 380
To track implementation of its recommendations, the Board first began publishing quarterly transparency reports that assess Meta’s implementation efforts. Especially since 2022, the Board scrutinizes how Meta implements its recommendations and uses its quarterly published transparency reports as a “mechanism to force better behavior from Meta.”Footnote 381 In the first quarterly report of 2022, the Board published a comprehensive assessment of Meta’s implementation of all recommendations until then. The report distinguishes six categories of implementation, ranging from “implementation demonstrated through published information” to “recommendation omitted, declined, or reframed.”Footnote 382 This was later expanded into a section called “recommendation tracker” on the Board’s website. The recommendation tracker contains two tables, one listing all published recommendations and one adding two more categories to the six categories just mentioned.
Neither the 2022 quarterly report, which introduced those six implementation categories, nor the later introduced recommendation tracker on the Oversight Board website publish their underlying methodology or the used data. So, the Board’s assessment that implementation in these two categories has indeed been “demonstrated” remains at times rather untransparent to outside observers.
Of the 288 recommendations issued by the Board up until November 2024, only fifty-seven recommendations were demonstrably and completely implemented according to the Board. Another forty-one recommendations assessed the Board to be verifiably partially implemented. For many recommendations, Meta reported progress (sixty-seven) or reported implementation without providing corroborating information (forty-six). Meta declined to put in place several recommendations after a feasibility assessment (fourteen), omitted or reframed recommendations (twenty), or declined to implement the recommendation out of principle (thirty-one).
According to Figure 4.5 all but the two top-right categories (implementation demonstrated; partial implementation demonstrated) show recommendations whose normative content was not verifiably adopted. According to the Board, Meta verifiably implemented 21 percent of its recommendations as of November 2024. Further, the Board considers another 15 percent of its implementations to be verifiably “partially” implemented. However, the exact efficacy and relevance of “partial” implementation remains unclear. Also, the Board does not disclose its verification of Meta’s implementation. Rather, until this day, the Board and Meta remain in a constant struggle toward, hopefully, incremental improvements in transparency. For example, in a 2023 decision the Board still deemed it necessary to explicate that it “expects Meta to share more data to enable the Board to assess improvements in performance over time.”Footnote 383 Therefore, this category may be treated with some skepticism. Nonetheless, when comparing the data available in November 2024 with that of the first comprehensive transparency report in 2022, we see that the full implementation indeed increased over time, climbing from 14 percent in 2022 to 21 percent in November 2024; “verifiable partial implementation” more than doubled from 7 percent in 2022 to 15 percent in November 2024.Footnote 384
The Board’s assessment of Meta’s implementation of Board recommendations (as of 11/2024)
Figure 4.5 Long description
The pie chart is titled Board’s Assessment of Meta's Implementation Q2 2024. It has 8 sections. Moving clockwise, they read as follows. Implementation demonstrated: 17 percent. Partial implementation demonstrated: 11 percent. Progress reported: 30 percent. Implementation lacking corroborating data: 16 percent. Recommendation declined after feasibility assessment: 6 percent. Recommendation declined: 12 percent. Recommendation omitted, reframed: 6 percent. Awaiting first response: 2 percent.
Between the Board’s inception in 2021 and November 2024, Meta reported – without corroborating their self-reporting – either “progress” or even “implementation” on another 113 Board recommendations. However, “progress” only means that “Meta has made a commitment to implementing this recommendation but has not yet completed all necessary actions.”Footnote 385 Whereas modifications to its publicly accessible normative material can be assessed by the Board, Meta’s post-recommendation handling of automated enforcement procedures remains obscure. In particular, it is still unclear how significant an implementation must be to count as “progress” and whether “implementation” always pertains to a comprehensive, “word-for-word” implementing of a recommendation.
The success or otherwise of transparency reports as enforcement mechanisms is beyond the scope of this study. However, judging from the anecdotal insights from the interviews as well as a close reading of Meta’s updates to its community standards, the tentative answer must be that the Board’s recommendations have had mixed success. To some extent, they have been verifiably implemented. Yet 178 out of 276 recommendations until November 2024 were either not fully implemented, their implementation has not been corroborated by any (publicly accessible) data or otherwise failed to materialize. Only roughly a fifth (21 percent) of the total are verifiably implemented recommendations. Conversely, for almost four-fifths (79 percent) of the recommendations made in that period Meta did not corroborate full implementation. Even if one deducts here the 15 percent verifiable partial implementations, one is left with roughly two-thirds of Oversight Board recommendations whose implementation cannot be seriously assessed or is even flat-out rejected by the company.
These figures present a paradox. On one hand, if only one out of five recommendations is fully implemented, it would be premature to declare success. In short, Meta remains obstinate and opaque, a fact the Board itself repeatedly criticized in its decisions. It is also important to look at which recommendations Meta implemented and which it rejected. It seems that at least some very salient recommendations were flat-out rejected. Remember the 2023 case on the Cambodian Prime Minister we discussed above?Footnote 386 The Board recommended that
Meta should immediately suspend the official Facebook page and Instagram account of Cambodian Prime Minister Hun Sen for a period of at least six months under Meta’s policy on restricting accounts of public figures during civil unrest. The Board will consider this recommendation implemented when Meta suspends the accounts and publicly announces that it has done so.Footnote 387
And indeed Mr. Sen’s Facebook page went dark shortly after the Board issued its recommendation to Meta. However, that was not because the company disabled the account. Quite the opposite. The Cambodian Prime Minister appears to have shut down his Facebook account himself, arguably in response to the Oversight Board’s decision.Footnote 388 The company, in contrast, expressly “declined” the Oversight Board’s recommendation to suspend Mr. Sen’s account.Footnote 389 Presumably, that is because the company does not wish to cede decision-making capacity in such a vital question to the Board – a remarkable rollback from the stance the company took in its 2021 Trump referral. Back then, the Meta asked the Board to come up with guidelines how the company should handle malevolent heads of state – and the Board tarnished the company for its lack of clear rules on that matter.Footnote 390 That Meta now expressly rejects a Board recommendation – based on rules Meta had devised in the meantime – to suspend an account of a government figure who was arguably inciting real-world violence seems deeply troubling. Especially so as all executives at the company must be aware that Facebook infrastructures played – according to the United Nations – a significant role in the displacement and murder of many Rohingya people in Myanmar in 2018. At the same time, the fact that the Cambodian Prime Minister self-suspended his account could be understood as testament to the normativity of Board decisions. Yet, alas, it did not last. Only a month after his exit, shortly before Cambodians went to the polls for a general election, Mr. Sen reactivated his account.Footnote 391 In the following election Mr Sen’s party won almost all seats – 120 out of 125 – in the Cambodian Parliament.
On the other hand, there is some positive movement. The rate of verifiable implementation in 2024 has significantly improved compared to 2022, which is a crucial indicator of progress. Perhaps, the Board’s focused and detailed efforts – such as the implementation tracker, or even external factors such as this very book – are exerting pressure on the company to improve its adherence to recommendations and overall improvement of its content moderation regime.
Looking ahead, two key points warrant attention. First, the Oversight Board was astute in transforming a vice into a virtue by publishing Meta’s initially dismal implementation rates. Highlighting how uncooperative the company was during the Board’s early years, and likely still is in some respects, appears to have prodded Meta into demonstrably adopting more recommendations. At present, the Board openly calls out Meta’s shortcomings in fully implementing its guidance, a strategy that paradoxically risks undermining the Board’s own credibility. However, Meta has a clear interest in sustaining the perception of the Board as a (somewhat) effective oversight mechanism. As long as the company seeks to benefit from the legitimacy bestowed by external, adjudicative oversight, it remains incentivized to comply – at least enough to avoid entirely eroding the Board’s authority. Therefore, it seems entirely plausible that we will see a further uptick in implementation. This would then warrant further, more qualitative research as the mere number of implemented recommendations tells us little about the actual quality of the reforms.
Second, some patience is warranted. The Oversight Board is still a young organization, and many well-established human rights bodies within the UN system arguably have similarly low implementation rates, despite decades of experience. The core challenge, as consistently highlighted in interviews with Board members, staff, and trustees, lies in the social norms and internal culture at Meta – factors that do not change overnight. Consequently, the gradual but steady implementation of Oversight Board decisions is a hopeful sign.
Looking to the future, however, it will be critical for the Board to be transparent about how it assesses “verifiable” implementation, so that independent third parties – particularly researchers – can effectively scrutinize the compliance process.
To summarize, having limited jurisdiction or relying on the good faith of digital corporations may not be immediately fatal for Emulated Guardians. Especially so in post-state governance where many power relations are regulated in vague, nonbinding arrangements. However, a “verifiable” success rate of only 21 percent cannot be seen as representing effective implementation.Footnote 392 Maybe Meta implements more than that – but the company’s unwillingness to be transparent about measures it puts into practice constitutes a legitimacy problem for the Board, although the latter’s transparency reports are bold and innovative in their approach. Eventually, these transparency reports may either gradually push Meta into compliance or, as a result of that transparency itself, their contents might end up chipping away at the Board’s image as a credible guardian of power, and thereby undermine its authority.
4.6.5 Prospects
Assessing possible improvements in the Board’s efficacy and Meta’s implementation will be the work of further studies. To a large degree, this change will depend on a change of culture at Meta. We know how hard and unlikely such a cultural change may be.Footnote 393 Also, the whole tech sector shifted or appears poised to shift to the political right, a trend epitomized by but not restricted to Elon Musk’s role in the 2024 US presidential election.Footnote 394 How lasting and impactful such shifts are remains to be seen. Nonetheless – or perhaps particularly since that election – the Oversight Board, despite its ceremonies, can be seen as a first step toward important yet contested cultural change. In addition, the fact that Meta increased its funding of the trust earlier than it needed to indicates a long-term and perhaps enhanced commitment to adjudicatory oversight. In contrast, other developments foreshadow the long journey still ahead for Emulated Guardians. Evolving such a culture, reflexively, requires institutions and rules to pave the way. The Board and the references to human rights in its decisions may be encouraging such cultural change from the outside. But they will not and cannot compel this change.
If, ten years from now, we consider the Oversight Board a failure (or do not recall it at all), it will likely not have been the Board’s fault but the company’s unwillingness to subject itself to adjudicative oversight. However, for the time being, impatience seems misguided. If the Board turns out to be the next chapter in the saga of adjudicative control over organizational power, assuming authoritative guardianship will take time. Nonetheless, it seems equally possible that the Board – even if it persists in its current form – will become yet one more quaint guardian-like experiment sidelined by more dynamic and powerful organizations.
In any event, I argue the Oversight Board should not be entirely forgotten, as emulation inevitably casts ripples. As we saw in the other case study, we already witness policy diffusion of aspects of the Oversight Board in Europe, most notably through the Appeals Centre Europe but arguably also in User Rights.Footnote 395 I suspect this trend will extend beyond content moderation, likely reappearing, mutatis mutandis, in future governance frameworks. Anthropic, a leading artificial intelligence company, has established a so-called Long-Term Benefit Trust, which is supposed to infuse (some) public-interest considerations into both the company’s management and the composition of its board. For future corporations and post-state power structures, someone might once again come up with the same, apparently simple, idea to emulate individual rights adjudication or other canonical ideas of public law.
4.7 Conclusion
This chapter examined the genesis and practice of the Oversight Board, a privately established Emulated Guardian that was set up ostensibly to control the world’s largest social network’s rulemaking and rule-enforcement efforts. It looked at the ways in which lawyers and juridical concepts shaped the Board’s creation as an adjudicatory institution, and examined its institutional structure and personnel. We have seen that the dominance of lawyers among Board members and staffers enabled the Board to attempt to incrementally overcome some of its in-built handicaps. Among these are a complex relationship between, on the one side, an able, impartial group of members and staffers and, on the other, several backdoor channels used by Meta to influence the Board’s adjudicative practices, which eventually undermined the Board’s hoped-for independence. Crucially, the Board turned to international human rights law as an external normative yardstick against which to measure at least some of Meta’s internally devised rules. Although the Board’s attempts to infuse public-law principles into a private-law context represent sometimes bold challenges to the status quo, for example when developing a doctrine of “public emergency,” its recommendations on the basis of these “infusions” often end up not being verifiably implemented by Meta. Nevertheless, the Board’s skillful adaptation of juridical personnel, argumentation, practices, and reasoning might indicate that its incremental build-up of authority has, perhaps, only just begun.
Open access