In the sixth month, Gabriel the angel was sent from God to a town in Galilee called Nazareth, to a virgin engaged to a man called Joseph, from the family of David. […] ‘Don’t be afraid, Mary’, said the angel to her. ‘You’re in favour with God. Listen: you will conceive in your womb and will have a son; and you shall call his name Jesus […] The Lord God will give him the throne of David his father, and he shall reign over the house of Jacob for ever. His kingdom will never come to an end.’Footnote 1
4.1 Introduction
The annunciation by JessupFootnote 2 of the birth of what would become transnational law conceived within the womb of Western (and then global) jurisprudence imagined a transformed juridical order in which the notion of law was broadened beyond the state (at least with respect to a definable set of activities). These legalities, in turn, were to be entangledFootnote 3 to solve what before had been issues the resolution of which could be undertaken solely by reference to the law of a state.Footnote 4 But this ‘good news’ did not immediately produce transformations in the halls of the priestly castes charged with the preservation of the jurisprudential order carefully nurtured in recognizable form from the time of Martin Luther and centred on the state as the principal expression of regulatory power through law.Footnote 5 For this caste and the states they served, the solution was, is and remains the law of conflicts and traditional private international law.Footnote 6 However, the twenty-first century has seen a reluctant acceptance in theory of the decentring of the state, and consequentially, of the recognition of the rise of multiple centres of governance with multiple forms of law.Footnote 7 That conceptual recognition comes at least a century behind early modern studies of its realities in some states.Footnote 8 The reluctance arises from the ideological consequences of such a conceptual acceptance. At its limit, the fear produces a modernist panicFootnote 9 about the state of the state, and of law as its official language. Despite the need expressed by some,Footnote 10 there remains a reluctance to give up ‘the ultimately law-focused epistemological mechanism still at work’,Footnote 11 and thus to forgo the post-1945 ambitions for a transformation of global politics based on the self-reflexive state as the highest legitimate form of communal political expression,Footnote 12 the expression of which could be made legitimate only when undertaken through the language of law.Footnote 13 Indeed, ‘[t]he main shortcoming of the extended conception of law advocated by anthropological and sociological approaches is the one pointed at by the so-called pan-legalist objection: the problem of the distinctiveness of law from other social normative orderings has been haunting the theorists of legal pluralism until today’.Footnote 14 In short, what factions of the leader class feared most in the twenty-first century was precisely the consequences of legal entanglements that might bring down their tightly woven conceptual house.Footnote 15
Yet that very rejection provides strong evidence not merely of its existence but also of its effects, principal of which are the ruptures that entanglement produces between these emerging centres of law/norms/governance,Footnote 16 both within and among the conventional nation state.Footnote 17 It is within these spaces that one might seek both the meaning and manner in which what Luhmann once described as structural coupling might occur.Footnote 18 These spaces without a space, these in-between spaces of law (and governance), these fragmented but entangled legalities, have assumed a spatial dimension.Footnote 19 It is now understood as both a connector (the tramesFootnote 20 through which spaces connect and communicate) but also as its own normative territory within which those communications and connections are not merely mediated but managed through complex entanglements,Footnote 21 or dynamic processes of communicative irritations.Footnote 22
The budding focus on the interspatial carries with it both promise and challenge. The promise: an interspatial gap filling in all of its complexities and theoretical possibilities. This has been the central exploration of entanglements among law, norms, rules and habits with coercive effect.Footnote 23 The challenge is centred on the risk of boundaries without end: of the permanent and quite dynamic cacophony of borders that is produced by the obsession with the interfaces between bodies of norms that themselves create borders within the space between norm systems for which other interfaces are necessary.Footnote 24 The problem of the never-ending spaces between spaces, where every law system defines its own inter-spaces, becomes self-entangling, and might itself be undergoing an extra-spatial transformation. That extra-spatial form of governance – in which space loses its centrality and law changes its forms and function – is the object of the exploration here.
More specifically, the chapter considers the emergence of data-driven analytics and the algorithmic techniques of imposing consequences (some of it machine driven, on the basis of artificial intelligence (AI))Footnote 25 as defining not just new modalities of governance but also reshaping the conception of spatiality within which entangled governance happens, national and transnational, public and private. So reshaped, these data-driven governance legalities entangle with traditional modes of governance through law in what may be new and interesting ways. These ‘social credit’ ratings, risk management, assessment, accountability or compliance systems have been established as a means to aid traditional governance;Footnote 26 yet they have the potential to displace the structures of governance they are meant to serve. But more than that, they may well change the landscape and language in which one encounters legal entanglements and its operational effects. The thesis of this chapter is simple: legal entanglement has moved beyond the two-dimensional space for engagement envisioned by Jessup and his successor. Entanglement was once confined to states, other governance institutions and between them all. That engagement was built around and with the traditional language of law and jurisprudence. Increasingly, the emerging quantitative legalities built around AI and big data management systems (e.g. social credit initiatives) that are neither dependent on the forms and techniques of law nor on the bureaucratic apparatus of state entangle more decisively with conventional and plural law and norm systems. These add an additional layer of actor (the ‘market’, the analyst, the modeller, the systems engineer, the coder) and seek to displace the language of law and jurisprudence with the language of data-based analytics applied towards a comprehensive management of behaviour. The resulting entanglement may reshape the meaning and making of entanglement itself. At its limit, this reshaping will have a particular effect on the way in which conventional plural legalities, until now the singular feature of globalized law frameworks, may be assembled through dynamic and sometimes unstable entangled linkages and rationalized by a regulatory algorithm that may come to manage them all.Footnote 27
This contribution, then, considers governance entanglement between the entangled legalities of law-norm systems and the legalities emerging from data-driven systems of behaviour management. The realities of emerging legalities that exceed the capacity of law to express their form require a three-dimensional analysis of entanglement and a broader view of legality, one that reconsiders data-driven, machine-administered regulatory systems more than an enhanced form of property.Footnote 28 One deals here not just with the flattened inter-legalities of the traditional structures of ordering power by rules. Instead, one must now understand the way those clusters of entanglement themselves are entangled with emerging modalities of law/regulation/norms which have come to form the centre of what is understood as plural legalities.
Section 4.2 briefly sketches the characteristics of emerging legalities that neither embrace the form nor the language of law. The two sections that follow examine the nature of inter-systemic entanglements which occur when data-driven governance legal ordersFootnote 29 are thrown into the already plural mix of legalities. Two principal forms of data-driven governance have emerged that are increasingly linking to traditional legalities.Footnote 30 The first is the Chinese ‘social credit’ initiative, which emerged in its current form in 2014, and is an undertaking by the present administration of the Chinese government that is meant to produce an all-around approach to ensuring compliance with law and social responsibility under the guidance of the state. The second are US and Western private initiatives around emerging markets for data. These are framed around principles of governance, risk management and compliance. With respect to each, the nature and textures of entanglement that encounters between traditional plural legalities and data-driven governance systems produce is considered. These entanglements present a distinct challenge, the challenge of linguistic disjunction, for the management of human organization. While law- and norm-based systems speak the same language, data-driven governance does not. Communicative disjunction may have profound effects on the nature and quality of entanglement, producing a competition for the lingua franca that may affect the way in which law is expressed, and may threaten the plurality of law.Footnote 31 Each also exhibits quite distinct characteristics and therefore quite different forms and qualities of entanglement, suggesting a more complex fracture and interaction among ever more different systems of legalities that are emerging in fractured political systems arranged around global trade regimes.
4.2 The Construction of Data-Driven Operating Systems
The governance consequences of data-driven compliance and risk management systems, and of the informal systems of ratings and assessment, has only recently exploded into the popular imagination of academics and others.Footnote 32 What started as an effort to rationalize the emerging techniques of indicators,Footnote 33 as a means of governance, quickly became a study of these techniques, increasingly systematized, as governance.Footnote 34 The recognition of algorithmic governance appeared like a direct and hidden threat to the carefully constructed public structures of law and governance systems.Footnote 35 Just as it seemed that theory could come to grips with the possibility of law (governance) systems beyond the state, the entire framework of law systems themselves seemed to be sidelined by data-based algorithmic systems to which law and norms appeared to be a stranger. And these challenges have come to the West in forms that appear to conflate the operations of Marxist-Leninist government with Western democratic markets-driven polycentric (plural) governance.Footnote 36 Yet these judgements ought not to deter from considering the (inevitable it seems) rise of these systems in both China and the West, and the additional layer of entanglement they add to the emerging formal and public systems of governance that constitute global legal polycentric (pluralist) governance.Footnote 37 To that end, a brief exploration of the shape of social credit initiatives fuelled by AI and machine learning-enabled algorithms is worth considering.
The rise of social credit initiatives (ratings and data-driven governance in the West) and big data management systems (as a means to implement these governance frameworks) appears to further entangle legalities in perhaps unexpected ways. It takes as its starting point the significant drive towards accountability and measurement in governance incarnated through rankings and ratings. Administration of rankings and ratings then devolves to the institutions that administer and along with that devolution goes the power to determine what behaviours will be valued, and in what amount, to rank and rate. In effect, the indicator acquires regulatory autonomy. Thus characterized as a regulatory system, it is then entangled with traditional law-norm systems.Footnote 38 If law and norms – traditional governance – are grounded in the supremacy of space, of territory, then how is one to approach governance orders that might themselves be detached from traditional spatial limitations? That is the principal focus of this chapter. It considers the emergence of what might eventually be understood as a legal/normative order that has arisen not from the need to organize a territory, within which people and things are corralled, but generated from and centred on the objects of regulation themselves. The emerging systems that have been referenced in this chapter as ‘social credit’ initiatives offer a glimpse at a governance operating system quite distinct from the legal/normative systems that have served as the building blocks first of the Westphalian state system (in its domestic and international elements), and thereafter the polycentric systems that mark this transnational age.Footnote 39
In place of legal/normative systems driven through the construction of an apparatus of government within territories (physical or abstract) it fashions a system driven through data generated by people and things.Footnote 40 Data substitute for custom and tradition; data-driven governance substitutes the language of counting for the more qualitative descriptive language of practice. Governing organs substitute, in place of custom and tradition, the quite precise data warehouse, one with a vocabulary and rule of normativity all of its own.Footnote 41 It can be filled constantly with the detritus of daily activity – at the market, on the web, in the car, on the street, in the store, etc.Footnote 42 These can be used for a variety of ends, some of them public, but in the process entangling the normative decisions about character of data for the content of norms.Footnote 43 In the place of policy and principle one encounters data analytics. Choice ceases to be politically potent only within the electoral field; all choices become political as they point behaviour managers to desires and habits of the masses producing data. In lieu of debate there is analytics. The framing of that analysis, that is the judgements and principles embedded in those analytics, is not subsumed within the algorithm. The algorithm itself is the expression of the sum of the objectives and perspectives of those for whose objectives the algorithm is deployed. In lieu of principles, then, there are presumptions and the self-created limitations of data fields that create the boundaries within which choices are cabined. These are the structures of conventional governance, but now deployed in a quite different space.
But these are not normative orders in the traditional sense. That fundamental characteristic suggests the opportunities and challenges that these emerging systems represent. Social credit initiatives – data and algorithmic orders – change the relation among governance actors, decentring the judge, the regulator and the social actors, in favour of those who can manage, organize and apply metrics to some end or other (that is to the same ends of law). Yet they do not mediate relations among norms so much as absorb them all in furtherance of extra-spatial objectives – to focus on behaviours in a self-reflexive loop founded on behaviour-generated data, the vectors of which may be modified through systems of real-time rewards and punishments (of individuals and institutions, including governmental and economic institutions). These orders, then, are not so much enmeshed as they are ubiquitous – serving both as techniques of management, and as the primary structures of regulation themselves. Absorption does not mean elimination; it posits an entanglement of a different order – between conventional governmental orders, the entanglements of which continue to produce norms, but their interpretation and application, that is the way these norms are given meaning, entangle conventional legalities with the analytic legalities of the algorithm and the entanglement of choices made between techno-experts, coders and public or private officials. These characteristics became much more visible in the context of the Covid-19 pandemic where the entanglement of simulation and conventional political legalities became the central regulatory element of institutional responses to the virus.Footnote 44
It follows then that the anticipated entanglements become more confused where one set of systems (law/regulation) continues to posit that metrics-based governance is merely technique, rather than a regulatory system in its own right. The interspatial, and its data-driven forms, produces a quite distinctive template for the conceptual construction of law/norms between law/norm systems; and it emerges with its own language and sensibilities in ways that are not yet completely clear. Every political culture approaches the emerging realities of ‘artificial intelligence’, machine learning and the data-driven governance that is sometimes its object, in ways that tend to affirm cultural expectations and points of view. Where cultures are obsessed with particular conceptions of privacy and property, the result is an engagement that is centred on political constraints against states and enterprises, on the management of markets for data and at the same time on an enhanced methodology of data-driven analytics to deepen cultures of compliance, assessment and accountability.
In liberal democratic states, that focus also has constitutional and political dimensions.Footnote 45 Thus, the regulatory machinery of the European Union has lately been tasked to manage data under the presumption of a hierarchy of authority that further presumes the normative nature of law and the mechanical nature of data and their analytics. Data and data processing ‘should be designed to serve mankind. The right to the protection of personal data is not an absolute right; it must be considered in relation to its function in society and be balanced against other fundamental rights, in accordance with the principle of proportionality.’Footnote 46 There is a particular aversion to the forms of data-driven intrusions from the economic sector (the ‘for hire’ fields) on the formalities and rituals of exogenous democratic expression.Footnote 47 And, of course, there is an equal obsession with theft of informationFootnote 48 and its misuse.Footnote 49 And yet, even as its elites offer the protection of law, it manages that data which itself manages behaviour.
A tiny microchip inserted under the skin can replace the need to carry keys, credit cards and train tickets. […] The small implants were first used in 2015 in Sweden – initially confidentially – and several other countries. Swedes have gone on to be very active in microchipping, with scant debate about issues surrounding its use, in a country keen on new technology and where the sharing of personal information is held up as a sign of a transparent society.Footnote 50
Thus combined, there is a tension between the centrality of the individual as the incarnation of data, and the utility of data in the production of economic goods with great disciplinary effects. This debate has been refined in the context of the Covid-19 pandemic, which has produced a substantial entanglement between data-driven indicator systems, national and international law and regulation in ways that will take some time to untangle.Footnote 51
On the other hand, where cultures are obsessed with communal solidarity and the protection of stability around socio-political core values as the basis for legitimate social organization and operation, then the result is an engagement that is centred on issues of surveillance, of detection, and of management of behaviour around key societal principles. There is a particular aversion to antisocial behaviour as a politically destabilizing force.Footnote 52 Feng Xiang, a law professor at Tsinghua University, recently spoke about the end of markets: ‘[i]f AI rationally allocates resources through big data analysis, and if robust feedback loops can supplant the imperfections of “the invisible hand” while fairly sharing the vast wealth it creates, a planned economy that actually works could at last be achievable’.Footnote 53 He might not be wrong about AI and markets; even the Cubans have sought to turn their economic planning into the rudiments of complex algorithms.Footnote 54 Yet this culture shares with the other a mania for observation – and for punishment and reward based on that power to observe,Footnote 55 one which is incompatible with the values of property and rights-based legal orderings.Footnote 56 And yet, even as there is a movement towards the overt management of behaviour through data, there is as well an obsession to ground that regulatory approach within the structures of regulation – in this case a regulation that protects the integrity of data and its use by the state and its delegees. Data and consequence (accountability for business, incentive to encourage preferred behaviours and punish violations of behaviour norms) thus combined, focus on stability and order in social development.
4.3 Chinese ‘Social Credit’ Systems: The State at the Centre
Social credit is a deeply entangled system; it is a system that will eventually apply to all legal actors, persons, institutions, officials and foreigners in or seeking to engage with China and Chinese actors.Footnote 57 It is more than that – it has been created as a nexus point between the constituting role of law (as well as the normative guides that are its rules) and the evaluative function of data-driven analytics.Footnote 58 It sits at the borderlands between a rating-evaluation system and systems of legal regulation. It is also entangled with the core political premises that drive Marxist-Leninist political economy in China under the leadership of the Communist Party of China (CPC).Footnote 59 Its autonomous and systemic character was emphasized from the beginning:
It is founded on laws, regulations, standards and charters, it is based on a complete network covering the credit records of members of society and credit infrastructure, it is supported by the lawful application of credit information and a credit services system, its inherent requirements are establishing the idea of an sincerity culture, and carrying forward sincerity and traditional virtues, it uses encouragement to keep trust and constraints against breaking trust as incentive mechanisms, and its objective is raising the honest mentality and credit levels of the entire society.Footnote 60
Yet one notes the entanglement – the system is constituted through law, but is established as autonomous from the law system around which it operates. The object is to establish a structure for managed entanglement within the state. Each, in turn, requires the production of a self-reflexive legality whose operation is entangled with those of the normative political, societal and economic order. Its mechanism, machine learning and AI, the entanglement of a rule system for AI and AI as law itself was also elaborated by the State Council in 2017.Footnote 61 ‘Social credit systems are centered on ratings. Ratings are derived, in turn, from data generated by what is being rated—individuals, businesses, public and private institutions, and eventually even [CPC] members. To that end, it is necessary to manage data production as it is to manage the analytics and consequences drawn from the data.’Footnote 62
Social credit initiatives have as their object the development of a national reputation system, assigning a rating that reflects a qualitative judgement of relevant data gathered about the subject. Reputation, itself, embraces notions of sincerity, and of integrity and compliance, in accordance with the standards and objectives overseen by the state. Four areas are identified: ‘sincerity in government affairs’ (政务诚信), ‘commercial sincerity’ (商务诚信), ‘societal sincerity’ (社会诚信) and ‘judicial credibility’ (司法公信). AI and machine learning focus on the generation of mechanisms for the management and integrity of data and for its marketization.Footnote 63
The objectives of both are grounded in economic and social development and support for national security.Footnote 64 The project of a comprehensive and nationally integrated programme of credit ratings of virtually all aspects of organized life in China, built pursuant to rules and laws, administered by public and private bodies and overseen by the CPC remains a work in progress, though one increasingly structured by the highest Chinese state organs.Footnote 65 It is self-consciously constructed as an alternative legality, both within China and against the forms of plural legalities in the West that the Chinese leadership increasingly find difficult to enmesh with their own.Footnote 66
Nearly from its inception, social credit was understood as a new kind of law, separate from but entangled with traditional law systems, which had been themselves the product of the fusion of traditional approaches to law and Western concepts of law, rule of law and legal mechanics.Footnote 67 At the same time, social credit expressed a form of governance that also deeply entangled systems of social ordering through rules (laws) and the paramount authority of the CPC to guide such lawmaking and its application through its own political-economic model.Footnote 68 The determination to adopt a social credit initiative (and its operationalization through AI-enhanced algorithmic governance techniques) was itself a function of the determination that such data-driven governance would enhance the long-term Chinese political-economic objectives embedded in their concept of socialist modernization, a core policy of the Chinese state since the era of Deng Xiaoping. Social credit was to evidence the transformation of techniques that marked a new phase of the socialist market economy system and of the social governance system, one which received its more definitive political form after the announcement of the ‘New Era’ political line of the CPC in the wake of the 19th CPC Congress of October 2017.Footnote 69
But its genesis also represented a practical response to a long-term problem that both state and private elements of society found increasingly burdensome.Footnote 70 These included a number of issues that threatened not merely the orderly progress of socialist modernization, but also inhibited the progressive advancement of social and cultural objectives. These included grave production safety accidents, food and drug security incidents, commercial swindles, the manufacture and sale of counterfeit products, tax evasion, fraudulent financial claims, academic impropriety and gaps between the extent of integrity in government affairs and judicial credibility and the expectations of the popular masses. If neither law nor regulation appeared to produce conformity, and if the transaction costs of deploying a vast enforcement network was counterproductive to the long-term goal of a self-regulating society that was efficient and productive in socially approved ways, then a different approach to the management of societal factors was necessary.
This ambitious set of objectives was to be guided by a set of core premises and constraints. Social credit initiatives must conform to and advance the objectives of the CPC Basic Line, including the development of economic forces for policy ends.Footnote 71 The Chinese social credit system also has a moral dimension, which deeply informs its regulatory and enforcement dimensions. Its object is to steer the culture and practices of people in virtually every aspect of their lives. To those ends, the Twelve Core Socialist Values unveiled in 2012 play an important role.Footnote 72 Entanglement is meant to be conscious and coordinated rather than organic and serendipitousFootnote 73 – highlighting the difference between a central-planning (public and administrative) versus a markets-based (private and demand driven) political order.
The social credit initiative was comprehensive. It was envisioned that when completed, social credit systems would manage key operations in four sectors: government, commercial activities, social integrity and judicial credibility.Footnote 74 With respect to the role of social credit in government, the focus was on the use of data-driven analytics tied to algorithms that produced the basis for accountability to result in rewards or punishments around a variety of governmental actions. These included: administrative permissions, government procurement, tendering and bidding, labour and employment, social security, scientific research management, cadre promotion and appointment, management and supervision, application for government financial support and other such areas and fostering the development of a credit services market. Clearly, data-driven analytics and its resulting assessment system would produce an immediate effect on those whose conduct triggers measured responses. For those who met credit minima, access to benefits would be enhanced. Those assessed at a low enough level would be placed on an effective blacklist that would make functioning in a modern society substantially more difficult and costly, absent readjustment. Moreover, social credit was also to be used as a tool for intra-governmental accountability, and to monitor civil servants.
Social credit mechanisms were also directed towards a very broad range of commercial activities – whether by state-owned enterprises or the private sector. Private behaviour by individuals was also to be managed through social credit systems under the umbrella of enhancing ‘Social Integrity’. An area of particular note for the application of data-driven analytics was that of the judicial system. The specific focus was on judicial creditability and on the integrity of the judicial function and the performance (and accountability) of judges for their work. The State Council identified a number of areas: proceedings transparency, prosecutorial and public security services conduct, the operation of judicial administrative systems and law enforcement standardization. Here one notes the likelihood of a vertical entanglement. Judges are the objects of law and the agents of procedures reflected in the legalities of law and regulation. Yet the content of the way those functions are assessed (and consequentially the way that the implementation of legal duties is understood and measured, and thus interpreted) become a function of a distinct legality – that of the social credit system applied to the judge. Data-driven projects might well include the production of data-driven algorithms to guide judicial decision-making or to develop guidelines for charging and prosecution. Here one encounters an entanglement in which the discretionary scope of one system is constrained by the operation of the other.
The societal effects of social credit programmes were to be enhanced through the application of these mechanisms on education and culture projects. Social credit in the development of education system reform was to be tied to the construction of the socialist core value system. Education was to be a means for the socialization of social credit mentalities and its general acceptance.Footnote 75 Assessment would be built around parameters for judging the development and operation of ‘moral’ classrooms. These are to be built around the establishment of models of appropriate conduct.Footnote 76 Appropriate conduct, in turn, is to be assessed against the twelve socialist values that have been established after the CPC’s 18th Congress: ‘Core socialist values comprise a set of moral principles summarized by central authorities as prosperity, democracy, civility, harmony, freedom, equality, justice, the rule of law, patriotism, dedication, integrity and friendliness.’Footnote 77 To that end, state organs are encouraged to oversee special campaigns in focus sectors, and to ‘persist in correcting unhealthy trends and evil practices of abusing power for personal gain, lying and cheating, forgetting integrity when tempted by gains, benefiting oneself at others’ expense, etc., and establish trends of sectoral sincerity and integrity’.Footnote 78
All of this is to be accomplished by building social credit baseline systemsFootnote 79 and their mechanisms.Footnote 80 These systems were then to be operationalized through blacklists. Blacklists – made up of the names of people whose social credit scores fall below certain thresholds – have already begun to have a substantial effect in everyday life. The system is manifested through ratings, and more importantly from the collection of blacklists produced as a function of ratings. Blacklists then affect the availability of goods and services.Footnote 81
These, then, will serve as the systems through which entanglement will be coordinated. Baseline systems were identified as (1) sectoral credit information systems; (2) local information systems; (3) credit investigation systems; and (4) uniform credit investigation platforms in the financial sector. In addition, government in cooperation with the private sector were to develop credit information exchange and sharing. These shall provide the regulatory structures for the operationalization of data-driven assessment and punishment/reward systems. The forms of entanglement are also identified as (1) incentive structures and punishments for deviations; and (2) legal, regulatory and standards systems for credit.
The entanglements of social credit within the Chinese context are as comprehensive as its ambitions. Social credit is meant to provide a new language for law, at least as it is meant to serve to control behaviour. At the same time, the constitution of social credit is driven by law. That is, law serves as a constituting element of regulatory systems that themselves are grounded in forms and practices that are not law. At the same time, those forms and practices of social credit then drive law as it is applied. They do so by giving meaning to the objectives and expectations written into law by the way that social credit is administered through the process of identifying behaviour, analysing its meaning and attaching consequences. At the same time, social credit provides a bridge between law, compliance and assessment systems. In the process it also helps shape the cultures within which expectations are shaped and cultural habits formed, which then lend themselves to expression in the normative content of law. Here social credit moves entanglement into an ecology of interrelated subsystems all deployed to move forward the political project of the CPC.
4.4 ‘Social Credit’ in the West: A Governmentalized Private Sector around Markets for Data
To speak of ‘social credit initiatives’ in the West is to consider an initiative that does not exist – as such. And yet, a more careful consideration reveals the outlines of the forms of social credit outside of the state. To speak to the development of social credit and its operationalization through machine learning and AI-enhanced algorithms, is to understand how the fracture and diffusion of power has produced something more than the aggregation of governance spaces and their ‘in-between’ spaces arranged in some manner or other. It evidences the organization of power beyond the orthodox space of law, in the sense that these new data-driven legalities use conventional polycentric governance spaces themselves as fuel for the generation of behaviour-managing ‘incentives’ or ‘punishments’ that function as regulation while avoiding anything like its traditional forms.
Social credit in the West is made possible not merely by the availability of spaces within spaces, but by the effective borderlessness of the market itself. Those extra-spatial zones have been enhanced through the governmentalization of the private sphere, the privatization of the public sphere and the migration from law and regulation to systems and systems management.Footnote 82 To speak about social credit in the West is to identify a host of fractured and market-driven projects by a large number of actors.Footnote 83 These actors also exercise governance authority in traditional ways (through rules, laws and the exercise of political and economic power). But for them, increasingly, assessment and accountability regimes, born of compliance objectives within the spaces and inter-spaces of conventional law/normative systems,Footnote 84 provide incentives to manage populations above the multiple governance spaces to which they might have to account.
What emerges in the West are systems of ‘governance, risk management, and compliance’.Footnote 85 Where in China the government (under the guidance of the CPC) pushes data-driven governance, in the West it is the market, and the delegation of managerial authority (compliance), that tends to drive these oversight and control systems – the literature highlighting this dynamic is by now well developed.Footnote 86 Anyone can rate and assess – but there are markets for rating as well; the most successful producers of rating enhance their profitability in markets for ratings. They focus on creating everything from hierarchies of value to assessment of conformity to a variety of corporate social responsibility obligations.Footnote 87 Less well developed is the conception of data, and the ratings that draw on them, as a system with regulatory effect. ‘Simply put, because of big data, managers can measure, and hence know, radically more about their businesses, and directly translate that knowledge into improved decision making and performance.’Footnote 88 Data-driven governance is essential for a variety of private sphere activities, and has become a business in its own right.Footnote 89
For both, data-driven analytics and the algorithms through which data-based judgements can be formed and consequences processed have become important elements of risk and compliance systems in five critical areas. The first is law enforcement (state entities) and compliance (private entities). Data-driven analytics here is presented as a method for complying with legal duty or responsibility. Administrative regulation and law provide the objectives, but the implementation occurs within automized data-driven systems. Examples include regimes for the distribution of governmental funds through revenue sharing and other programmes.Footnote 90 The second is transparency regimes: ‘It is used within an organization or community to enhance its operation and discipline its members; it is used externally to enhance legitimacy (norm) and accountability (technique) among stakeholders who have an interest in but not a direct participation in the operation of the enterprise.’Footnote 91 The third is in controlling behaviour. For economic enterprises this was driven in part by law,Footnote 92 and in part by changes in the way that administrative officials exercised authority.Footnote 93 Data-based analytics may be essential in the exercise of prosecutorial discretion in the USAFootnote 94 and the UK.Footnote 95 Businesses are increasingly using data-driven analytics to control behaviours through health and wellness programmes. The fourth, shaping cultures, is possible when monitoring (micro-surveillance) is tied to transparency and enforcement. Smoking campaigns are a well-known example.Footnote 96 Here one entangles moral value systems (about smoking) into political action (anti-smoking regulation) which is then entangled within medical and quantitative measures of harm which contribute to health rankings that affect private markets for insurance, and rankings that may affect individual access to credit or education.Footnote 97 Fifth is accountability as assessment, self-assessment and accountability regimes at the individual and entity levels.Footnote 98 These entangle both legal systems through compliance regimes and markets-based systems through data-driven ratings systems.
Entanglements between data and conventional governance orders have only recently emerged more clearly. The recent controversy over the inclusion of questions about citizenship on the US census provides a case in point.Footnote 99 Inclusion of the question would generate data, a core operational principle of census taking. That generation is coupled with a host of conventional law which relied on the governance of census data generation for its own operation. But the generation of data has significant consequences precisely because the system of regulation into which those data are injected is itself based on the way data are curated. Here is the point of entanglement between census as a system of data generation, census as the jurisdiction of compiling a quantitative narrative image of the American population, and census as a necessary predicate for the operationalization of a number of conflicting political objectives expressed though the law of states and the federal government. It’s no surprise to see census questions shifting with the times: ‘Lots of questions go off the census when they’re not very important anymore.’Footnote 100 The conflict within data governance became famously entangled with the legal regulation of the administrative state in a US Supreme Court opinion most notable for its recognition of conflict expressed in rules governing the exercise of political discretion within a web of legislation where the interests of secondary sovereigns are affected.Footnote 101
Another important point of entanglement between data-driven and traditional legalities centres on the scope and principles through which data may be harvested. Most aggressive – again voluntary and on a bargained for basis – are chip implants for employeesFootnote 102 by merchants (including the state). This is a system grounded in consent, in value added as an inducement for participation. Increasingly, however, states have sought to use their legislative authority to restrict, or at least manage, this essential feature of data-based rule systems, by legislating the legal effects (and limits) of consent, especially within an employment relationship.Footnote 103 More passive are the seamless systems of cameras, credit card transactions, turnstiles, passes for highway tolls, key stroke and internet tracking systems and the like that can effectively track individuals and record their activities on a continuous basis. Their regulatory effect is now well known.Footnote 104 Yet their entanglements with plural legal systems have only just invaded the consciousness of regulatory stakeholders. Emerging issues include privacy rights;Footnote 105 and rights to be forgotten.Footnote 106 But the focus has also turned to the application of issues of race, gender and other non-discrimination law and policy to the structures of data-driven governance bound up in the extraction of information for behaviour management ends.Footnote 107 Conversely, data-driven systems themselves have entangled with law systems in the area of discrimination law by providing a complementary system, the products of which are used by law systems as techniques of proof of discriminatory intent or in forming or assessing policy and regulations.Footnote 108
Compliance itself is a data-driven exercise, but one in which the parameters are set by legal systems, administered through the actors onto which compliance is imposed, and assessed and disciplined either by the state or private actors.Footnote 109 Compliance is expressed as the private law internal governance systems that implement the delegation of responsibility (in part) from the state effected through law or regulatory directive. These include implementing administrative guidance,Footnote 110 to compliance systems built around disclosure and reporting systems, for example, the French Supply Chain Due Diligence LawFootnote 111 or the Australian Modern Slavery Law.Footnote 112 The second is crafted through data-driven assessment systems that take their objectives from the policies of private law internal governance and which rely heavily on markets for external disciplining.Footnote 113 These entanglements are clearest in the context of corporate governance.Footnote 114
Of all of the forms of data-driven governance, perhaps the closest the West has to the emerging Chinese social credit initiative are credit rating agencies. Financial credit rating agencies, for example, become actors in the governance of financial markets through their production of a standardized rating of risk (creditworthiness), which is then used for making investment decisions, and consequently to make the regulation of financial markets dependent on the risk assessed.Footnote 115 These operate within a market for services with a few big players rating business and public credit. Their profit derives from subscriptions or issuer-pays models of income. This produces a markets-driven model that mimics the effects of Chinese centralizing and public control models. These credit agencies’ businesses are grounded in data-based analytics applied to objectives and their effect is to discipline behaviours through reward–punishment systems derived from their analytics. Their systems are functionally differentiated, and they exhibit only a necessary unification even within the same field. At the same time, their activities serve as the basis for regulation, and are, to some extent, regulated by the state whose finances they in turn rate.
4.5 Conclusion
The relationship between traditional governance orders, in a context of spatially distinct but intermeshed legal/normative orders, remains to be fully explored. This chapter suggested some points of entry and on fundamental approaches. First, the use of metrics and the quantification of accountability has moved beyond an increasingly sophisticated palette of rankings and inducement to become a regulatory space in its own right. Second, that regulatory space embeds politics and law within the construction of its analytics and the determination of the meaning of rankings. China is building a centralized system tied to its political organization. Most of the elements of social credit have already been developed in the West. But the unification of the various elements, and their seamless operation, would be a great innovation. While the West approaches data-driven governance entanglements through the lens of privatization and markets,Footnote 116 China inverts the trajectories of entanglement, focusing instead on recreating within the state the universe of data-driven governance in which law becomes the instruction manual for the operation of social ordering through data-driven analytics. Third, these systems pose a challenge for the conventional understanding of entanglement among systems all characterized by qualitative approaches to regulation. Perhaps most interesting of all the consequences of these social credit systems may be their ability to absorb traditional systems, and in that process of absorption to reduce the centrality of borderlands between systems.
Hints of this trajectory are evident in the movement towards compliance and accountability. These increasingly data-driven exercises turn traditional governance systems, and the governance systems arising between and within them (conventional spatiality of governance), into the generators of data that themselves can be subject to management in accordance with principles and objectives. These objectives may themselves be drawn from the political-cultural assumptions of society or themselves may be dynamic expressions monitored through the aggregate conduct of data generators themselves. The multiplicity of legal regimes, then, is itself a source of data useful for data-driven analytics that can manage these as well as aid in the management of systems.Footnote 117 This suggests entanglement of a different order, between qualitative and quantitative regulatory measures which increasingly fold one into the other while retaining an element of autonomy based on the different regulatory spaces from which they are sourced.
What is clear is that entanglement can no longer ignore the legalities of data-driven governance even as it seeks to embed its language and structure its communication across systems that do not speak the same language. The entanglements of a law after modernityFootnote 118 become more complicated – not merely as between distinct and polycentric rule systems sharing common characteristics (the forms and functions of rules), but now between systems that do not speak the same language (words versus metrics, compliance versus assessment, etc.). Governance evolves from the language and conceptual universe of politics and principles to the language of the operating system grounded in systemic objectives. In a world of algorithms, those who would devise them will be king. And those who would be kings in Western democratic republics may well soon be scientists and not lawyers.Footnote 119 ‘When you look at the most important issues facing our country, it is climate change or healthcare policy or cyber security, the integrity of our elections. Who better to address these issues than a scientist?’Footnote 120 Who better indeed, when the state is a container for accountability systems through data-driven algorithms? That, ultimately, suggests the great challenge for legalities and their entanglements where society continues to move beyond the problems of entangling legal systems to those for which the econometrician, the statistician and the moralistFootnote 121 may have as great a voice as the lawyer or the judge, and the politician or administrator.Footnote 122
Open access