INTRODUCTION
In today’s world, data are collected on almost every aspect of our lives. Human experience has become “raw material for translation into behavioral data” (Zuboff Reference Zuboff2019, 8). Due to technological advances in big data analytics and algorithms, human dataFootnote 1 have a special appeal to political rulers, enabling them to gain better insight into, and arguably greater control over, their subjects (Feldstein Reference Feldstein2019). Transforming human experience into data and controlling such data have become crucial to the effective exercise of political power and potentially the attainment of supreme power.
The exercise of state control through data and technology is a long-standing theme in contemporary society. As early as 1987, Roger Clarke (Reference Clarke1987, n.p.) warned of the dangers of “dataveillance,” a term he defined as “the systematic use of personal data systems in the investigation or monitoring of the actions or communications of one or more persons.” More recently, Zygmunt Bauman and David Lyon (Reference Bauman and David2013, 8) coined the term “liquid surveillance” to refer to the diffusion of state power into the everyday lives of citizens through systematic monitoring, tracking, tracing, sorting, and checking. In China, the Social Credit System (SCS) appears to present an unprecedented model of data-based surveillance and control of society. Although many countries use dataficationFootnote 2 to assess their citizens in various respects,Footnote 3 the SCS has the ambitious goal of evaluating and intervening in all aspects of its citizens and businesses that are deemed meaningful to the state, including the economic, social, legal, moral, and political dimensions of both their online and offline lives. How to make sense of the SCS’s uniqueness in the accelerating trend of datafication, and how to better understand the implications of the SCS for individual autonomy so as to prepare for a possible era of increased datafication-enabled social control?
In this article, we develop the concept of the data state as a governance model enabling the state to comprehensively monitor, evaluate, and control its subjects through datafication, leaving them little room to defend their autonomy. We use the SCS as a case study to examine the intricacies and dynamics of control and submission that are intermediated by omnipresent datafication and a state monopoly on behavioral standard setting. We argue that, if the SCS fully integrates the three major functions envisioned in its official blueprint (that is, the provision and regulation of financial credit rating, market regulation and social governance, and the promotion of state-endorsed values), this would ultimately allow the data self, carefully groomed by the state, to dominate the bio-self.Footnote 4 When thoroughly connected to a series of social and economic governance initiatives utilizing big data technology, the SCS may turn China into a data state. Although it may be tempting to dismiss the SCS as an extreme example that will not be replicated elsewhere, the lesson to be drawn is that the uncontrolled development of dataveillance and liquid surveillance can result in an ecosystem that provides fertile ground for the emergence of a data state. Understanding the particular features of the data state and the data state ecosystem is essential to enabling citizens of democratic states to call a halt to any further expansion of datafication at this critical juncture. Such understanding also paves the way for further research on devising a system of effective checks and balances against the ever-expanding reach of datafication.
The first part of the article commences with a discussion of the evolution of the SCS from 2014 to 2020. In view of the diverse national policies steering this multidimensional system and the various experiments contributing to its construction, we focus on the core functions and implementation mechanisms of the SCS and critique the system’s inherent defects. The second part analyzes how the SCS paves the way for the emergence of a data state with two peculiar features by fulfilling the state’s ambitious and ambiguous policy goals and integrating other data-driven governance initiatives. The first feature is comprehensive access to, and evaluation of, citizens’ lives, and the second is the dominance of the data self over the bio-self. The third part discusses the dangers of the uncontrolled development of datafication in the West by drawing on Byung-Chul Han’s (2017) work on the digital panopticon and the writings of John Cheney-Lippold (Reference Cheney-Lippold2011) on algorithmic identity.
CHINA’S SCS (2014–20)
Overview
China’s SCS, currently serving as a versatile instrument not only of economic regulation but also of social governance and the promotion of morality, has undergone prolonged construction. The SCS was first introduced as a financial credit system in the early 2000s; however, its functions have gradually transformed since then. In 2014, the central government released the Planning Outline 2014–2020 for the Construction of a Social Credit System (Planning Outline), which provided the blueprint for the system as the world now sees it.Footnote 5 Party-state authorities at the national level have since issued numerous related policy documents rather than enacting laws. In these documents, multiple themes stem from the broad and vague concept of “credit.” Over a six-year period, sectoral regulators and regional administrations not only built subsystems toward the centrally specified policy goals but also introduced pilot programs to accommodate their respective regulatory priorities. Only limited aspects of these subsystems and programs have been endorsed by local legislation (Chen and Cheung Reference Chen and Cheung2017; Mo Reference Mo2020). As of the end of 2020, regulations governing the SCS had been enacted by the legislatures of only nine provincial-level regions and were pending in another seven.
The volume of official rules related to the SCS and the variety of local experiments pose a challenge for any systematic examination of the system. Indeed, fragmentation and inconsistency in its implementation have been well documented (Ohlberg, Ahmed, and Lang Reference Ohlberg, Shazeda and Bertram2017; Arsène Reference Arsène2019; C. Liu Reference Liu2019). This is not to say that the SCS is simply a patchwork of uncoordinated building blocks and presents merely mundane techniques of statecraft. At the conclusion of 2020, marking the end of the (first) construction phase according to the blueprint, the institutional and infrastructural targets of the SCS appeared to have been achieved. These targets include “the formulation of a set of standards for social credit,” the completion of a credit evaluation system that “covers the entire society and is based on the sharing of data resources about credit,” and “giving full play to the mechanisms of incentivizing trustworthiness and punishing untrustworthiness.”Footnote 6 Adjustments have been made in the SCS’s policy orientation, but it is important to note that state authorities, rather than market or social entities, play a persistently dominant role in steering the SCS and carrying out its functions (Backer Reference Backer2019a, 209; Lin Reference Lin2020, 3).
In this article, we identify the prominent functions of the SCS as performed by the state over its citizens from 2014 to 2020, as well as the essential mechanisms to carry out these functions.Footnote 7 By examining significant national policies, local legislation, and prevalent practices, we discern a theme that emerges in the evolution of the system and may continue to sustain it. The theme can be encapsulated as an ambition of the state to monitor and evaluate the behaviors of individuals by relying on a set of state-imposed standards of “trustworthiness” and altering their behaviors by giving sanctions and incentives, all through the datafication of behaviors and the sharing and use of the data on centralized, state-run platforms.
Major Functions
To provide a pathway through the labyrinth of its policies and experiments, we outline the three most prominent functions of the SCS affecting the behaviors of citizens: (1) the provision and regulation of financial credit rating; (2) market regulation and social governance; and (3) the promotion of state-endorsed values. Given that the definitions of “credit” and “social credit” are surprisingly absent from all national policy documents concerning the SCS, we use a function-based approach, which serves to highlight the general policy thrusts and identify the differences and connections of the subsystems (see Table 1).
Major national policies concerning the SCS (2014–20)
Provision and Regulation of Financial Credit Rating
The “social credit system” was introduced in the early 2000s as a means to support China’s economic reforms by reducing loan defaults and improving the creditworthiness rating mechanism (X. Liu Reference Liu2016). Often labeled the “narrowly defined SCS” by Chinese scholars, the initial system resembled financial credit systems in the West, and its construction was overseen by the People’s Bank of China (PBoC) from 2002 to 2013 (Zhang, Cheng, and Zhang Reference Zhang2015). Since 2006, the PBoC has operated the Credit Reference Center, which aggregates payment, loan, and other financial data on both individuals and corporations and issues credit reports (L. Wang Reference Wang2018). The PBoC also regulates “credit investigations” (zhengxin 征信)—that is, the collection and analysis of credit data for the purpose of assessing the data subject’s ability to make the required payments and thus minimizing default risks. Credit rating as such can be conducted by state-owned or private institutions, according to the Regulations on the Administration of Credit Investigation Industry enacted in 2013.Footnote 8
Since 2014, financial credit rating has become a part of the expanded SCS and runs in parallel with other types of broadly defined “credit evaluation” (discussed below). The PBoC continues to be responsible for regulating credit investigation, whereas the National Development and Reform Commission (NDRC) oversees other types of credit evaluation. The PBoC and the NDRC are now the two leading authorities of the Inter-ministerial Conference for the Construction of the Social Credit System, which includes over forty central-level party and state authorities and coordinates SCS policies. The PBoC Credit Reference Center maintains the National Basic Database for Financial Credit Information, a component of the SCS’s data infrastructure. As a provider of the “fundamental service of credit investigation,” the center offers credit reports (L. Wang Reference Wang2018, 77), which not only include financial data but also record information about court judgment defaults, punitive administrative decisions, and public utility payments (Xinhua News Agency 2020). It also offers paid credit-rating services to enterprises and citizens. In 2014, the Credit Reference Center developed a credit-scoring model similar to that of Fair, Isaac and Company (FICO) in the United States. The model has been followed or considered by commercial banks (Caixin Magazine 2019).
Market Regulation and Social Governance
A distinct function of the current SCS is the regulation of behaviors both in the market and in society.Footnote 9 The leap from the financial to the non-financial realm appears not to be grounded on a coherent governance goal but, rather, framed as a response to pressing social problems. From early 2003 to 2014, the conceptual expansion of the SCS can be described as occurring in three phases. In the first phase, the meaning of “credit” was extended from financial creditworthiness to cover a trader’s reliability in terms of contract fulfillment.Footnote 10 In the second phase, in 2007, a non-contractual element was annexed to the object of evaluation of the SCS, as the policymakers intended to tackle odious business behaviors that betray the public trust, such as the production of substandard or counterfeit commodities and tax evasion (Chen, Lin, and Liu Reference Chen, Ching-Fu and Han-Wei2018).Footnote 11 This move stressed a market player’s assumption of social responsibility, as opposed to its economic capability, to earn the trust of other members of society. At this time, the term “credit” began to be used interchangeably with “chengxin” (Lei Reference Lei2014, 66), a compound word that combines honesty (cheng 诚) with keeping one’s promise (xin 信).Footnote 12 The third phase of expansion occurred under the Planning Outline, under which the concept of “credit” further covered behaviors affecting the public interest and breaking others’ trust, such as misbehaviors affecting environmental health, public safety, or social security. The Planning Outline pushed the SCS into two parallel domains: “commercial chengxin” and “social chengxin.”Footnote 13 A 2019 policy further developed a “credit-based” approach to market regulation.Footnote 14 A subsystem of the SCS focusing on the regulation of market and social behaviors can thus be identified separately from the subsystem focusing on the management of default risks.
National policymakers use subtle terminology to identify the particular subsystem of market regulation and social governance. Shouxin (守信, which can be translated as trust keeping or promise honoring) is used to refer to a quality of an individual or organization that enables them to be trusted. In contrast, shixin (失信, trust-breaking or promise breaking) denotes a quality that renders an individual or organization untrustworthy. Gonggong xinyong xinxi (公共信用信息, public credit information) refers to information generated by the authorities that indicate the trustworthiness of a subject in a non-financial context. Local legislation governing the SCS has also highlighted the non-financial dimension of trustworthiness. For example, the Shanghai Municipal Regulations on Social Credit defines social credit as the status of a citizen or organization with respect to their fulfillment of contractual commitments or performance of legal duties in social and economic activities.Footnote 15 Regulations in several other regions regard violations of law as trust-breaking behaviors (Mo Reference Mo2020).
The central government has particularly emphasized the SCS’s function of market regulation and social governance, thereby rendering the SCS, in part, an instrument of the enforcement of laws and regulatory rules (K. Shen Reference Shen2019). To ensure analytical clarity in this article, the credit evaluation to serve this function is called “trustworthiness evaluation” as opposed to “financial credit rating.”
Promotion of State-Endorsed Morality and Values
An attempt to use the SCS for the ideological control of society emerged before the Planning Outline was issued, and it has gained momentum since 2016. The Chinese Communist Party (CCP) established a connection between credit investigation and chengxin in its moral sense in a 2011 plenum decision concerning the construction of a “socialist civilization” and reasserted it in a 2013 plenum decision (Knight Reference Knight, Creemers and Trevaskes2020, 239). The CCP’s Eighteenth Congress designated chengxin as one of the twelve “core socialist values.”Footnote 16 In 2016, a national policy further integrated the core socialist values into governance of the state, positioning the SCS as an institutional device for restricting behaviors that contravene such values.Footnote 17 The CCP commands that these values be embraced in legislation and public policies and adhered to in social management. It is against this backdrop that the central government has expressed its plan to evaluate compliance with ethics in selected spheres of society under the SCS, using the terms “personal chengxin” and “chengxin system.”Footnote 18 For analytical clarity, we refer in this article to the evaluation of an individual’s compliance with officially promoted values as “integrity assessment,” as opposed to “trustworthiness evaluation,” which focuses on compliance with laws.
Integrity assessment, established in national policy documents, targets several categories of peopleFootnote 19 and emphasizes the profiling of individuals. The central government explicitly requires individualized “integrity dossiers” to be compiled on students and “credit records” on the targeted professionals.Footnote 20 Based on these credit records, a subsystem of professional profiling seems to be emerging separately from the city-level scoring programs.
The impact of each function of the SCS on individuals’ rights will not be elaborated on here.Footnote 21 Rather, we focus on the fact that policy makers have embraced a unique and vague concept of credit to consolidate the three functions of the SCS. To justify the semantic expansion of the concept, Chinese officials have resorted to “theoretical innovations.” A view raised by scholars and prevalent in official discourses defines credit broadly as the basis for a member of society to be trusted by other members in economic or social activities (Zhang and Zhang Reference Zhang and Zhang2019). Thus, through credit-based regulation, the state can alter both market transactions and social interactions. One theory explaining how trust can be broken not only by a breach of contract but also by a violation of law(s) posits that credit is an aggregation of three dimensions: chengxin (honesty and keeping promises in the moral sense) to gain general trust as a foundational quality, fulfillment of contractual commitments to gain the trust of market players as an economic quality, and compliance with binding rules to gain the trust of the authorities as a social quality (Wu Reference Wu2013). This theory of “three-dimension credit” has been hailed by many authorities as the basis for the “innovative establishment” of a credit system with Chinese characteristics (Luo Reference Luo2018, 56; Lin Reference Lin2020, 3). However, it has been questioned by a few PBoC officials who participated in designing the credit investigation system (Wan Reference Wan2017; L. Wang Reference Wang2018). The National Standards for Credit has indeed combined the three dimensions into the concept of credit, indicating the current official stance.Footnote 22
Core Implementation Mechanisms
The SCS uses two core mechanisms to enable the state to comprehensively monitor, evaluate, and alter the behaviors of citizens: (1) combined punishments and incentives carried out by the state and (2) a unitary, state-run data infrastructure.
Combined Punishments and Incentives as Wielded by the State
The mechanism of “combined punishments for shixin (trust breaking) and combined incentives for shouxin (trust keeping)” is a kaleidoscope of regulatory schemes.Footnote 23 It is called “combined” because actions are undertaken concurrently by multiple authorities across regions and sectors. Broadly, combined punishments mainly support the function of market regulation and social governance, whereas combined incentives are mainly used to promote morality. Two stages of the regulatory process precede the levy of punishments or provision of incentives: the monitoring of a citizen’s behaviors and the evaluation of their credit. State authorities at all levels determine which behaviors to monitor according to the rules of their respective jurisdictions and the broad categories provided by national policies. The results of evaluating a citizen’s credit can lead to different types or degrees of state intervention.
Combined Punishments for (Serious) “Trust Breaking”
Since a national policy was issued in 2017 to standardize blacklisting, misbehaviors captured by the SCS have generally been classified as either “blacklisted,” thus subjecting the citizens in question to cross-regional and cross-sectoral punishments, or “warranting attention,” thus placing the citizens under close surveillance by the relevant regulators.Footnote 24 At the national level, it is mainly at the SCS Inter-ministerial Conference that various authorities discuss and adopt criteria for determining whether a behavior is trust-breaking and thus warrants combined punishments. Most of these criteria are established in the memoranda of understanding (MOUs) jointly issued by various agencies of the state or the CCP. In addition, local legislation governing the SCS and the myriad MOUs agreed to by local administrative and judicial authorities provide different criteria for the determination of trust-breaking behavior. Since 2017, only the departments of provincial-level governments have been authorized to issue these criteria.Footnote 25 The prevalent pattern across regions is to define the “seriously trust-breaking behaviors” that warrant blacklisting and combined punishments within the territory in question and to subject the “(modestly) trust-breaking behaviors” to more diversified restrictions (Ye and Shi Reference Ye2021).
The types of blacklisted behavior have multiplied to accommodate the changing concerns of the authorities involved. The Guidelines on Combined Punishments outline the “seriously trust-breaking behaviors,” which fall within four “focal areas”: (1) severely endangering health, life, or safety; (2) severely undermining the market order or regular social order; (3) refusing to perform statutory duties imposed by the judicial or administrative authorities; and (4) shirking national defense duties or otherwise harming the interests of national defense.Footnote 26 The guidelines enumerate specific types of misbehaviors arising from business and not-for-profit activities within the first two areas. Notably, some appear not to directly involve the element of trust as it is commonly understood, such as “[failure to ensure] safety in fire prevention” and “disrupting the cyberspace order concerning information dissemination.”
Regarding the third area (violation of a legal duty), the Supreme People’s Court (SPC) was the first to intervene against this behavior by labeling those who refused to carry out duties specified in effective judgments as “trust-breaking persons subject to enforcement of judgement.”Footnote 27 This established the association between “trust-breaking” and “infraction.” From 2009 to 2013, lists of such violators were aggregated and published nationwide, a practice found to remarkably alleviate the enforcement difficulties plaguing China’s civil justice system for decades. Later, defaulting on a judgment became a salient type of blacklisted behavior under the SCS, to be governed under the MOU on Judgment Defaulters, a dedicated MOU reached between the SPC and certain other ministries.Footnote 28
Ministries and local governments soon realized that combined punishments were a powerful means of enforcing regulatory actions they had previously undertaken in isolation. The taxation authority took the lead in leveraging the mechanism of combined punishments.Footnote 29 The scope of “blacklisted behaviors” was subsequently extended to the violation of a variety of laws and regulatory rules, going beyond the scope of the Guidelines on Combined Punishments and resulting in a plethora of MOUs. By October 2020, over fifty MOUs had been issued by national authorities, each targeting a regulatory sector (for example, taxation, environmental protection) or industry (for example, tourism, transportation) (Wu and Liu Reference Wu and Liu2020).Footnote 30 In addition to MOUs, a few pieces of local legislation were enacted to define new types of seriously trust-breaking behaviors (Ye and Shi Reference Ye2021). The adaptability of blacklisting to regulatory priorities is exemplified by recent government responses to COVID-19. In the spring of 2020, in an attempt to restrict human mobility to contain the epidemic, numerous provinces introduced similar policies to blacklist as trust-breaking those citizens who had concealed their contact with suspected COVID-19 patients or evaded quarantine (Southern Metropolis Daily 2020).
State authorities have the discretion to disclose the names of blacklisted citizens online, with details on their credit evaluation. Blacklists are usually published on the national portal “Credit China,” relevant sectoral credit websites, and/or the provincial credit websites concerned.Footnote 31 For example, the names and identity numbers of judgment defaulters and updates on their cases can be found on a dedicated website.Footnote 32 To maximize the shaming effect of public disclosure, some local governments have broadcast blacklists through other media (Schaefer et al. Reference Schaefer, Ether, Trey, Polk, Andy, Frida and Joe2019).Footnote 33 Although the disclosure of blacklists is discretionary, blacklisting invariably triggers cross-sectoral and cross-regional actions according to predetermined plans (mainly in the form of a MOU) for “combined punishments.” Once a citizen has committed misconduct in one area, punishments are concurrently exerted by multiple stakeholders in his or her life outside that area. The prevailing motto of the SCS is: “Those who have broken trust on one matter will face obstacles everywhere.”Footnote 34 The goal is to make life difficult for offenders wherever they go.
For example, the MOU on Judgment Defaulters stipulates that the courts shall inform the relevant government agencies, CCP departments, and financial institutions of their lists of defaulters to enable those entities to take restrictive measures against the defaulters. These lists link one’s history of non-compliance with judicial decisions directly to their economic interests, personal or business reputation, transaction opportunities, and even the basics of their daily life or daily operation. MOUs that strengthen administrative authorities’ enforcement of laws (forty-five of the fifty-one national-level MOUs) cover a broader spectrum of state-imposed measures than does the MOU on Judgment Defaulters, including, for instance, (1) restrictions affecting a trust-breaking person’s economic opportunities, such as his or her ability to establish a company, raise funds, or receive government subsidies; (2) denial of access to market sectors, such as food, public utilities, and banking; (3) constraints on participation in government procurement and the transaction of state-owned properties; (4) disqualification from employment in the civil service or public institutions; and (5) rendering trust-breaking persons illegible for honorary titles. The SCS also complements state punishments with so-called “industry-based” and “market-based” punishments.Footnote 35 For example, the state urges professional communities and industry associations to take restrictive measures against their members who qualify as “seriously trust-breaking” according to state standards.Footnote 36
Combined Incentives for Observation of State-Endorsed Values
In contrast to the punishments outlined above, incentives are mainly used to encourage state-endorsed values. National authorities maintain “redlists” or lists of trustworthy citizens meriting combined incentives. Initially, the targets of redlists were law-abiding market players in a few sectors. From 2016 to 2018, redlists were expanded to include two types of exemplary persons: outstanding young volunteers and people substantially contributing financially to the alleviation of poverty and misery.Footnote 37 An attempt was made in 2019 to extend the redlists to include all citizens who receive recognition or honors for honesty or ethical leadership from party or state authorities.Footnote 38 The incentives for redlisted citizens generally fall into three categories: preferential measures in administrative processes, advantages in public services, and increased odds of being awarded a prize. In addition, the NDRC has promoted a set of initiatives called “Credit Ease +” to provide trustworthy business entities with financial support or subject them to a simplified regulatory process (Kong, Tian, and Han 2019). Some city governments have incorporated such initiatives into their citizen credit evaluation programs, thus providing the same incentives for both regulatory compliance and observation of state-defined morality (Knight Reference Knight, Creemers and Trevaskes2020).
Local Programs of Citizen Credit Evaluation
Although sector-specific blacklisting and redlisting are supervised by national authorities, local governments have leeway to design measures to address misbehaviors that do not rise to the level of “seriously trust-breaking” and to encourage trust-keeping behaviors. A growing number of provinces and cities have established locally united programs for evaluating the non-financial “credit” of their citizens. Different from the national subsystem of financial credit rating, these programs are diverse in the scope, criteria, methods, and consequences of evaluation. For example, qualifying trust-breaking behaviors include non-performance of commitments to regulatory authorities in Zhejiang Province;Footnote 39 improper conduct in the course of petition in Rongcheng City, Shandong Province;Footnote 40 unruly behaviors on the subway in Beijing;Footnote 41 and violations of professional ethics in Yulin City, Shaanxi Province.Footnote 42
The quantified programs that produce “credit scores” have garnered particular attention. Such programs existed in at least twenty-one cities as of May 2019; however, they vary widely (C. Liu Reference Liu2019; Knight Reference Knight, Creemers and Trevaskes2020). Whereas a few programs follow the FICO score scheme, a larger number of them have developed their own multidimensional indicators (Zhu, Huang, and Chen Reference Zhu, Huang and Chen2019, 11). The dominant pattern involves matching selected trust-breaking behaviors to certain indicators and deducting points according to a preestablished scale as well as adding points for designated trust-keeping deeds. Citizens are classified into ranks based on the resulting scores. Notably, most programs quantify not only business activities but also behaviors governed by public order laws and state standards of morality.
Most quantified programs offer modest benefits for high scores (for example, fast-tracked administrative processes, reduced costs for public services); they rarely elaborate on specific punishments for lower scores (C. Liu Reference Liu2019; Zhu, Huang, and Chen Reference Zhu, Huang and Chen2019). This may be a lingering effect of the 2010 public resistance against the scoring experiment in Suining County, Jiangsu Province, in which lower scores triggered negative consequences (Creemers Reference Creemers2018). Quantified metrics are by no means the only form of evaluation. Under certain local SCS legislation or standalone policies, misbehaving citizens face discriminative, if not punitive, measures (Mo Reference Mo2020).
Unitary, State-Run Data Infrastructure
Two national government databases are dedicated to the integration, sharing, and use of data derived from the surveillance and evaluation of citizen behaviors. The National Basic Database for Financial Credit Information, as discussed above, aggregates only data that the PBoC considers relevant to financial creditworthiness ratings. In parallel, the National Credit Information Sharing Platform (hereafter National Public Credit Platform), launched in 2015 and supervised by the NDRC, aggregates data related to the broadly defined trustworthiness of citizens, including blacklists, redlists, and administrative penalties and licenses. The National Public Credit Platform is connected with all of the SCS subsystems and is accessible to all authorities of the party-state as well as authorized financial and social institutions. Regional public credit platforms are established by governments of all provincial units and a number of cities, which submit required data about local citizens to the national platform and integrate other region-specific data as well (Jiang Reference Jiang2019). The national platform constructs theme-based datasets (for example, blacklists, profiles of key professionals), conducts data analytics (for example, crossmatching of blacklists and redlists nationwide), and provides credit information services (for example, generation of reports of public credit, automated data push for combined punishments) (57). The “Credit China” portal, which is part of the platform, is a window for public disclosure, enabling easy search of selected blacklists, redlists, and trustworthiness evaluation results.
The introduction of the “social credit unified code” (SC unicode), a unique eighteen-digit code given to each citizen and organization, has substantially reduced the difficulties in data aggregation and synchronization. Citizens’ SC unicodes are identical to the numbers of their resident identity cards, which are further linked to personal data held by government agencies, such as their tax payments, social insurance, and household registration. The coding, exchange, and disclosure of public credit data comply with a series of national specifications (Zhou Reference Zhou2020). Other credit data processing is also standardized by the National Standardization Technical Committee on the Social Credit, a quasi-official body established in 2015, together with other sectoral and local standardization bodies.
Inherent Defects of the SCS
The SCS has built efficient data-sharing platforms and significantly bridged data silos to enable coordinated regulation across various domains (Lin Reference Lin2020). It has also enhanced the enforcement of a wide range of legal rules governing misbehaviors deemed to be of great public concern (Hu, Gong, and Zhu Reference Hu2020). Nevertheless, legal scholars have identified various measures under the system as undermining the protection of fundamental rights (Chen and Cheung Reference Chen and Cheung2017; Chen, Lin, and Liu Reference Chen, Ching-Fu and Han-Wei2018) or otherwise contradicting government efforts to uphold the rule of law (von Blomberg Reference Von Blomberg2018; K. Shen Reference Shen2019). It is equally necessary to examine the compatibility of the evolving SCS with existing Chinese laws as it is to critique the inherent defects in the SCS that may survive technical refinements (for example, legalizing the SCS’s current functions and mechanisms through a national law) and pose persistent challenges to individual autonomy.
A Distorted Concept of Credit and Decontextualized Evaluation of Citizens
Given the recent flood of data-driven rating systems around the world (Mau Reference Mau2019; Pasquale Reference Pasquale2019), it is tempting to classify China’s SCS as an extension of financial scoring systems or a reputation-rating phenomenon (Dai Reference Dai2018; Síthigh and Siems Reference Síthigh and Mathias2019). Yet, as demonstrated above, the SCS is fundamentally different in the expansive scope of behaviors that it captures and in its statist evaluation. The SCS embraces a peculiarly pliable concept of credit, in the name of which the state turns citizens into data projects to be measured and managed. Precisely because the national policies and local legislation offer ambiguous explanations of “credit,” the evaluation of citizens under the SCS is largely decontextualized, and the results of the evaluations are artificially rendered quantifiable or comparable.
Although Chinese officials have attempted to justify the expanded concept of credit with “theoretical innovations,” such theories and the resulting policies stretch the common understanding of “credit” to their limits. Credit rating, as commonly understood, assesses the degree of trust between market players. The SCS, however, adopts a more elusive concept of trustworthiness, attempting to evaluate the trust that members of society should place in evaluated individuals. In reality, however, this concept of trustworthiness represents party-state authorities’ trust in the citizenry. Furthermore, the theory of three-dimension credit has shifted the main criterion of trustworthiness from honoring one’s promises in bilateral relations to complying with rules imposed or values endorsed by the state, with little respect for autonomy.
The evaluation process is thus statist, as it relies on criteria defined by the state and in the state’s interest. A substantial portion of the moral misconduct identified under the SCS was originally governed by social norms not defined according to the will of the state but, rather, formed spontaneously among social communities and professional associations. By defining contraventions of morality and professional ethics as trust-breaking behaviors, the SCS has replaced relatively autonomous social norms with state-imposed standards. The SCS’s standards thus do not necessarily relate to ethical excellence, as it claims. Furthermore, in the name of assessing integrity, the state has set behavioral standards without going through any legislative process.
The state-imposed evaluation criteria of trustworthiness pertaining to market regulation and social governance are also incoherent and often disconnected from what they purport to be. The infractions appearing on most SCS blacklists are denounced by authorities appealing to moral imperatives. Some infractions indeed relate to the fraud that has long plagued China, such as instances of food poisoning, substandard construction, and telecommunications fraud. These violations can be understood as breaches of implicit promises or as sheer dishonesty. However, many other infractions characterized as trust-breaking do not involve moral issues such as promise keeping or honesty (Y. Shen Reference Shen2019). Administrative authorities have frequently equated trust-breaking behaviors with violations of the laws and regulations of environmental protection,Footnote 43 tourism,Footnote 44 and health-care institutions.Footnote 45 Also, a judgment defaulter can be deemed trust-breaking for avoiding a contractual commitment as well as a state-imposed obligation. In essence, the evaluation of “trustworthiness” in these cases is in fact a reassessment of the seriousness of violations of laws or government orders rather than of the trust between parties. Labeling violators as trust-breaking has become a convenient means of addressing discrete governance problems.
More importantly, evaluation results based on an ambiguous criterion of trustworthiness are decontextualized and qualitatively incomparable. Financial reliability, promise-keeping ability, professional integrity, and honesty reflect different aspects of an individual. It is only fair to assess each of these qualities in the relevant financial, business, professional, and/or other societal context. By grouping disparate types of conduct into a single dimension of trustworthiness, the SCS has confused the evaluative methods developed for distinct purposes. This unidimensional evaluation strips an individual of interpersonal contexts, denying the plurality of judgments formed about him or her by fellow members of society. Different aspects of an individual are thus reduced to something commensurable. And his or her rich and multifaceted value to various communities is diminished to his or her bare utility or worthiness in the eyes of the state.
The consequences of decontextualized evaluation can be seen in the spillover effects of trustworthiness evaluations (made in unspecified contexts) on financial credit ratings. The State Council has encouraged government authorities to “guide” financial institutions to “raise … loan rates and insurance premium rates in respect of trust-breaking entities” or to impose restrictions on the loans and/or other financial services provided to these entities “according to the principle of risk-based pricing.”Footnote 46 Guidance as such often has de facto binding forces on financial institutions, most of which are state owned or state controlled. Although the PBoC has attempted to separate the factors affecting one’s creditworthiness from those concerning one’s morality, the NDRC is eager to generalize the non-financial trustworthiness of market players across contexts to alter their financial creditworthiness (L. Wang Reference Wang2018, 113; C. Liu Reference Liu2019, 30). The decontextualized concept of credit has apparently won top-level endorsement, as “Credit Ease Loan” schemes have flourished nationwide to provide trustworthy enterprises with preferential loan rates.
With respect to trustworthiness evaluations, the SCS also disregards differences in the nature and significance of various types of behaviors, overlooking the underlying reasons for their meritorious or blameworthy nature in their social settings. This decontextualized process artificially renders the resulting “untrustworthiness” computable. Citizens are thus classified as “seriously trust-breaking,” “trust-breaking,” and “trust-keeping” for differentiated treatment.Footnote 47 Some city SCS programs assign numerical scores to citizens. Unlike other rating systems, which measure financial creditworthiness or some single aspect of performance (for example, traffic penalty points or passenger scores on ride-sharing services), such programs measure the integral character or whole being of an individual as a citizen. Recent studies have highlighted the propensity of the big data economy to turn citizens into consumers. Similarly, these programs risk reducing citizens to data projects.
At the philosophical level, decontextualized evaluation of citizens generates a distorted representation of reality and matches it to the rigid machine dimension of the SCS. At the practical level, the motives behind conduct are discounted and disconnected from the restrictions and incentives the conduct induce. For example, in the case of a judgment defaulter, it is impossible to determine from the label of “untrustworthy” the cause of default, be it financial hardship, other extraneous circumstances, or intentional avoidance. The categorical responses of the SCS risk eliminating the discretion and reflexive understanding that are crucial to the effective operation of the rule of law and a humane society (Marx Reference Marx2015, 123).
Semi-automation of Combined Punishments and Ingrained Disproportionality
Trustworthiness evaluation results can lead to different types of state intervention. However, it is combined state punishments that manifest the SCS’s control function and its most significant impact on citizens’ autonomy. A prominent feature of the mechanism of combined state punishments is the almost seamless interface between the evaluation of a behavior and the state intervention that ensues. The semi-automation of sanction gives full play to the deterrent effects of the SCS. The concurrent imposition of punishments is based on the computerized matching of citizens with various blacklists, which is achieved through cross-regional and cross-sectoral sharing of data on “trust-breaking” behaviors on the National Public Credit Platform. This process reduces authorities’ discretion in determining punishment. Once an authority first labels a particular behavior as trust breaking and places the actor on a blacklist, other authorities that proceed to inflict punishments on the actor are not bound to follow the normal procedures of decision making. That is, because the SCS mandates mutual recognition of the results of credit evaluation,Footnote 48 authorities acting as “punishers” are not required to independently investigate the facts concerning the behavior or judge its nature and merits according to their own expertise or the laws they administer. Instead, they automatically adopt the evaluator’s findings and defer to the SCS platform’s blacklists. The punishers exercise discretion only in determining whether the “trust-breaking” label gives rise to punishment under the rules of the SCS. Certain of the combined punishments are provided for, or allowed by, the existing laws; however, some of the other punishments established by the SCS rules are at odds with laws (Y. Shen Reference Shen2019; Li Reference Li2021).
Disproportionate combined punishments under the SCS have been documented (K. Shen Reference Shen2019; Wang and Huang Reference Wang and Huang2021). The national policy allows the same range of sanctions to be applied to a wide variety of misconduct.Footnote 49 Air and high-speed train travel bans are a common form of sanction levied by the courts against judgment defaulters.Footnote 50 Although some may consider this a justifiable punishment against judgment defaulters who have failed to pay damages to a party they have wronged, it should be noted that this punishment has been extended to defaulters of liability for tax or administrative fines, thereby enforcing the decisions of administrative authorities rather than only those of the courts. Unlike judgments resulting from contestation and arbitration, however, administrative decisions are often made unilaterally, and their legality and reasonableness may be challenged through review processes. Under the SCS, many infractions identified by administrations, although of different degrees of gravity, have indiscriminately incurred severe punishments. For example, “seriously trust-breaking” people excluded from traveling on high-speed trains include not only those who have refused to pay overdue debts as ordered by the courts but also people in charge of social security service providers (for example, hospitals, pharmacies) that have violated service agreements and people who have been punished for smoking in railway carriages.Footnote 51 It is questionable whether people guilty of these three infractions carry the same degree of blameworthiness or cause similarly serious consequences.
Combined punishments not only impact on a “trust-breaking” individual’s own employment prospects (for example, disqualifying him or her from being a civil servant) but also may affect his or her family. For example, the MOU on Judgment Defaulters initiated by the SPC unequivocally bars the children of defaulters from attending high-tuition, private schools.Footnote 52 Courts in some cities have proactively advised private schools to check their applicants’ parents against the judgment defaulters list (Legal Daily 2018). Although officials try to justify this punishment by citing the need to prevent defaulters from transferring funds and evading judicially confirmed debts, they fail to appreciate its impact on the fundamental right to education of the defaulter’s children (Li Reference Li2021). More significantly, the punishment inflicted in this manner has the effect of imposing “guilt by kinship,” which prevailed in political campaigns from the 1950s to the 1970s and which contradicts the doctrine of individualized liability for violation of the law that is the foundation of the contemporary Chinese legal system.
Local SCS legislation has attempted to address this problem of disproportionality by stipulating that combined punishments should be commensurate with the nature, circumstances, and social harm of the associated trust-breaking behavior.Footnote 53 A police document issued by the State Council at the end of 2020, guiding the next-stage construction of the SCS, requires administrative authorities to respect the principle of proportionality and prevent harsh punishment for minor faults.Footnote 54 However, such policy gestures are unlikely to adequately address the problem because national policy makers continue to hold combined punishments as the core mechanism of the SCS and the distorted concept of credit as its foundation.Footnote 55
In fact, the disproportionality of punishment derives from the disconnect between the legal liability for misbehavior and the grounds for further punishing the actor. Although combined punishments are often used to enforce laws, they go beyond the “just deserts” principle that underlies the determination of criminal or administrative penalty in Chinese law. The legal liability for the violation of a legal rule may be a consideration for judging whether the violation is “seriously trust-breaking” under the SCS. Yet it is the derivative label of “being trust-breaking,” not the liability per se, that is the grounds for the consequent combined punishments. The “trustworthiness” of a citizen, as determined by the targeted infraction under the SCS, differs from his or her legal liability for that infraction. As discussed above, trustworthiness evaluation reduces those committing infractions of differing natures to a unidimensional rank of “trustworthiness” or, in a worse case, to a numerical score, with no reasoning provided for the hierarchy of behavior or the conversion. Whatever proportionality that may originally exist between an infraction and the legal liability thereof is removed in the evaluation process.
Furthermore, combined punishments are imposed by authorities with no legal jurisdiction over a targeted infraction. The objective of this mechanism is markedly different from the objective of the traditional (and legal) punishments imposed for infractions falling within an authority’s regulatory purview. Under the SCS, an infraction in Sector A is often irrelevant to the infractions in Sector B that incur lawful punishments by the regulator of Sector B. For example, under the MOU on Marriage Registration, an individual who makes false statements about his or her consanguineous relation with his or her spouse is barred from registering an enterprise as an authorized economic operator with the customs authority.Footnote 56 However, a business entity’s conduct pertaining to the customs code is not determined by its legal representative’s breach of eugenics policies. Thus, the customs authority may exert its own punitive power for an objective other than that for which this power is authorized (that is, regulating the order of customs) to maximize the deterrent effects of another authority’s decision (that is, civil affairs administration). Combined punishments are essentially designed to create a complex ripple effect extending beyond the scope of punishment delineated by legal liability. This necessarily defeats any serious attempt to improve the proportionality of the mechanism.
The SCS has evolved into a system of rules that overlaps, but is separate from, the legal system. It combines the creation of norms (that is, standards setting), implementation, enforcement, and adjudication (in the limited situations allowing a citizen to raise objections to factual inaccuracies in his or her credit records) in a single system. Its outlook is instrumental (stressing statist criteria of trustworthiness) and disciplinary (expanding punishments to coerce compliance). Empowered by data technology, the SCS conveys an aura of objectivity and neutrality as it moves laws to the sidelines, reducing them “to the means by which the factors necessary for the production of scores, or rankings, can be determined” (Backer Reference Backer2019b, 14). As bureaucracy increasingly wields its powers through the SCS, the laws upholding the principles of rationality and proportionality are gradually eroded and risk being ultimately displaced.
FROM THE SCS TO A DATA STATE
China’s SCS is still in its fledgling stage, with local experiments running, national legislation pending, and centralized policies being refined. However, it has enormous potential to reach “all aspects of organized life deemed important to the state” (Backer Reference Backer2019b, 14). Current national polices and local legislation do little to effectively address its inherent problems of decontextualized evaluation and semi-automation of disproportionate punishment. Even more worrying are signs that local SCS programs are increasingly connected to various surveillance projects, such as the communal grid management system. If the SCS fully integrates its three functions and connects them seamlessly to other data-driven governance initiatives, it is likely to turn China into a data state. As explained in the introduction, we define “data state” as a governance model enabling the state to comprehensively monitor, evaluate, and control its subjects through datafication and data-driven techniques, thereby severely restricting their autonomy. The nature of the SCS’s evolution highlights the potential for two special features of a data state in a fully-fledged SCS: (1) the comprehensive monitoring and evaluation of citizens’ data by the state and (2) the dominance of the data self over the bio-self.
Comprehensive Monitoring and Evaluation
A distorted concept of credit leads to the fluid scope and high intensity of datafication. As noted above, the SCS has the ambitious goal of monitoring and assessing an unprecedented scope of personal traits and behaviors ranging from citizens’ subjective inner traits (for example, honesty, integrity) to their engagement with other members of society. It can also capture all interactions between citizens and bureaucracy that are regulated by state rules. Further coupled with the unceasing data collection and big data analytics in the market sector, the SCS and the webs connected with it are inescapable by citizens.
Anonymity in the digital environment is almost impossible in China, given the existence of the SC unicode and the real-name registration system tied to cellphone SIM cards, Internet access points, and account-based online services, pursuant to the 2017 Cybersecurity Law.Footnote 57 Chinese citizens are incorporated into the networks woven by state-owned telecommunications companies and private Internet service providers, which have a statutory obligation to submit user data to state authorities.Footnote 58 Service contracts usually provide for government access to user data. The implications of government access are far-reaching, as the data self of a citizen includes all of the data one voluntarily discloses online (for example, by microblogging), the digital trail one leaves (for example, one’s browsing history), and one’s interactions with fellow citizens (for example, in online discussion forums). Notably, “online trustworthiness,” an important component of one’s social credit, hinges on the compatibility between the information and speeches one publishes or transmits online and the state-specified standards.Footnote 59 Cyberspace authorities that regulate online content are knitting a web of combined punishments for untrustworthy Internet users.Footnote 60 Chinese citizens have embraced the Internet and communication technologies since the dawn of the twenty-first century, only to realize now that these technologies have turned them into data segments to be evaluated by the state.
Business corporations have been and will remain actively involved in behavioral evaluation. In 2015, eight firms affiliated with influential Internet or financial groups—including the much-discussed Sesame Credit under Alibaba’s Ant Finance Group—experimented with rating personal creditworthiness via big data analysis of their users’ consumption and social behaviors. Two years later, they failed to obtain licensing to conduct personal credit investigation from the PBoC, which adopted a narrow definition of financial creditworthiness and expressed concern about the conflicts of interest posed by these groups’ roles in credit rating and loan service (Caixin Magazine 2019). Nevertheless, the PBoC subsequently licensed Baihang Credit and Pudao Credit to use records collected from Internet-based financing institutions and alternative data for credit investigation. Although the two corporations are de facto under government control, 8 percent of Baihang Credit shares are owned by each of the eight aforementioned firms, and the principal shareholders of Pudao Credit include key players in the smartphone industry, facial recognition technology, and e-commerce-related financing (Caixin Web 2020). Similarly, the NDRC, eager to promote its broad concept of credit as trustworthiness, has mobilized market forces to offer “third-party credit evaluation services.” It has reached data-sharing agreements with multiple corporations, under which they contribute their clients’ credit data to the National Public Credit Database (China Credit 2017). To further credit-based regulation, the NDRC has instructed local governments since 2019 to facilitate the evaluation and profiling of market players by big data institutions.Footnote 61 Furthermore, the NDRC and the PBoC have designated twelve and sixteen “model cities” for SCS construction in 2017 and 2019, respectively (Science and Technology Daily 2019). Some model cities have explored partnering with technology companies to calculate the untrustworthiness risks of enterprises (Guoxin Institute 2019) or to pursue the public shaming of untrustworthy citizens (Schaefer et al. Reference Schaefer, Ether, Trey, Polk, Andy, Frida and Joe2019).
In addition to the SCS, other datafication initiatives and surveillance projects have also contributed to the inescapability of Chinese citizens from the data state ecosystem. For instance, China is close to becoming a cashless society; mobile payment is now the norm (Yang and Arnold Reference Yang and Martin2018). Without access to mobile payment services, it would be difficult, if not impossible, to live in urban areas of China. Despite its convenience, the mobile payment system documents how users spend their time and money. In addition, facial recognition technology is widely used in China, for a variety of things from access to the subway system to determining eligibility for online mobile services (Xie Reference Xie2019). Businesses and public services are cultivating in Chinese citizens the habit of allowing their behaviors to be “datafied.” Against this background, the SCS is expanding the boundaries of behavior capture.
Dominance of the Data Self over the Bio-self
The inescapable identification and monitoring and the perpetual evaluation and categorization of citizens by the state through the SCS is developing a unique “data self” for each citizen. Different from the bio-self, the data self is based on the datafied segments of one’s behavior and the decontextualized rating of such segments. The data self in the SCS is a comprehensive, continually updated profile of an individual’s bio-self that is purposively compiled by the state from data generated by the state itself or provided by corporations or other social organizations. Citizens’ bio-selves are reduced to data selves for evaluation and use by the state. Data selves risk redefining and overtaking real bio-selves, as they have become determinative of the opportunities that citizens are afforded and the treatment they receive in real life.
The inescapability of the SCS is especially manifested in the public sector. All branches of the state have access to the credit records of citizens through the unitary Public Credit Platform. “Credit clearance” following the assessment of citizens’ trustworthiness is embedded in the decision-making processes of all market regulatory authorities.Footnote 62 Having a data self has thus become a necessity. Credit clearance is also planned to be, or already used in, the administration of public order and the provision of public services.Footnote 63 In some cities, the SCS is connected to the “grid management system,” which has operated for over a decade as a mechanism of precise surveillance and swift policy implementation at the grassroots level. Under the mechanism, urban neighborhoods and rural communities are divided into grids for the purpose of collecting information about the residents, providing public services, identifying risks to social stability, and enforcing corresponding measures (Bandurski Reference Bandurski2018; Mittelstaedt Reference Mittelstaedt2021). With the SCS connected to the grid management system, the residents’ credit records are automatically forwarded to government agencies that police the grid so that combined punishments and incentives can be carried out promptly (Shenzhen Special Economic Zone News 2018; Zhonghong Wang 2020). As of 2021, some developed provinces are aiming to further integrate the SCS into the grid management system to effectuate big data-driven warnings of risks pertaining to social stability and public order.Footnote 64 In other words, the data self compiled by the SCS is being consolidated as a precondition for government treatment of the bio-self in various contexts. Moreover, the combined punishments mechanism, which ensures that “those breaking trust on one matter face obstacles everywhere,” perpetuates the negative, direct impact of the data self on the bio-self.
Gamification is a byproduct of the data self becoming more “real” than the bio-self. The bio-self must strive to optimize opportunities under the dictates of the state. As citizens are judged by the data shadow that hangs over them and the digital footprints they leave behind, the bio-self has an incentive to cultivate the data self and to ensure the smooth existence of both selves. Instead of engaging in all of the state-encouraged behaviors, the bio-self may gamify the system by focusing on a few behaviors that are likely to have the greatest impact on the data self. In a scenario in which one’s online speech data are captured and analyzed as the primary basis for a behavioral understanding of one’s inner mind and to facilitate future psychographic categorizations, one is likely to deliberately utter particular types of speech to gain credit. A recent initiative undertaken by a city government in Shandong Province offers a case in point. The initiative involves connecting the SCS to the use of a smartphone app designed for ideological training. Individuals who earn high scores on the app by reading designated materials and doing well on quizzes can redeem their scores in the form of discounted interest rates on loans from local banks. Enterprises contributing to the app can also obtain positive assessments pertaining to their trustworthiness (Lightening News of Qilu Web 2020; Weifang Evening News 2020). Although the causal relationship between frequent engagement with the app and the financial creditworthiness of an individual user or the business trustworthiness of an enterprise cannot be established with certainty, a surge in popularity of the app (with over 210,000 users as of May 2020) is probably an indication of citizens’ enthusiasm to leverage the initiative to improve their credit profiles.
More importantly, while the state uses the decontextualized evaluation of trustworthiness as a tool of control over its citizens, citizens have an opportunity to exploit the system. Most of the MOUs and legislation concerning combined punishments include a mechanism called “credit repair” (xinyong xiufu 信用修复), which allows individuals to request that their names be removed from blacklists and that the restrictions imposed on them be lifted.Footnote 65 According to a national policy issued in late 2019, citizens can take such measures to repair their credit as correcting “wrong conduct,” offering “pro bono services” and engaging in “charity activities.”Footnote 66 The policy behind this mechanism appears to be that these behaviors can partly remedy the harm caused by law breaking or other kinds of misconduct. Yet this arrangement may reproduce social inequalities, as those with superior economic or social capital are better able to undertake the prescribed charity activities and thereby appear less frequently on blacklists and/or achieve higher social credit scores. It is also doubtful that the prescribed “reparative” behavior, which is unrelated to the trust-breaking conduct, can genuinely “repair” the public’s trust in the individual concerned.
Although gamification is possible, the SCS has a “looping effect,” by which it can adjust its categorization. “Looping” in this context refers to a feedback effect whereby the mechanism of classification changes the behavior of those who fall within it, which in turn causes the classifications and evaluation criteria to be redrawn (Hacking Reference Hacking2004). Guided by national policy makers’ agenda, the respective weights of the various factors considered in the assessment of a citizen’s trustworthiness can be modified by authorities in various sectors and at various levels. Regardless of whether a citizen is seemingly beating the system in the short term, the more the bio-self endeavors to win, the more it is bound to the data self and to the SCS.
Ultimately, the state has the upper hand in the construction of the data self, owing to information asymmetry and the arbitrary criteria for punishments and rewards. The end result of the data self’s dominance is the bio-self’s growing dependence on the state. It is extremely difficult for the bio-self to defend its autonomy and resist state control enabled by the SCS. Even if a Chinese citizen can secure legal remedies for an incorrect credit record or an unfair assessment by asserting certain statutory rights (whether procedural or substantive), he or she remains unable to challenge the overall “digital iron cage of subordination,” particularly when that cage has been built by a state that monopolizes coercive power and is not subject to constitutional review (Mau Reference Mau2019, 154).
In sum, Chinese citizens have few options but to engage with the SCS and, in so doing, to allow state power to be exercised over an increasing number of aspects of their lives. They are thus forced to participate in “co-active” control by “choosing to live with ever empowering surveillance technologies” (Kim Reference Kim2004, 193). In such a data state ecosystem, resistance is impractical as citizens’ livelihood and very survival are inextricably intertwined and interdependent with the continuously expanding system. Above all, the SCS operates through continuous evaluation, which may change from moment to moment. The demands made of citizens are likely not only to make them behave according to the agenda of the authorities but also to shape their lives and lifestyles. Perpetual evaluation and semi-automated intervention will result in citizenship being defined by scoring, or “cybernetic citizenship” (Orgad and Reijers Reference Orgad and Wessel2020, 16), with citizens constantly pedaling through the data trail they have left behind. At the end of the day, the digital panopticon is both within and outside them.
LESSONS TO LEARN AND THE DANGERS OF UNCONTROLLED DEVELOPMENT OF DATAFICATION
It may be tempting to dismiss China’s SCS as an extreme example unlikely to be replicated in Western democracies. In particular, compared to Western nations, China has a very different political system and a governance model that does not permit adequate checks and balances on the state’s exercise of power, despite various legal reforms (J. Chen Reference Chen2020; Fu and Dowdle Reference Fu, Michael, Chen and Fu2020). One may assume, given the concerns about privacy and personal autonomy held by citizens in Western democracies, that a democratically elected government would never attempt (or be allowed) to introduce any system akin to China’s SCS, in which the state comprehensively monitors, evaluates, and alters the behaviors of individuals by relying on state-imposed standards and a centralized data-sharing platform. The SCS, which combines norms creation, implementation, and enforcement into one infrastructure, is indeed unique.
However, we must recognize that we are all living in an era of datafication, wherein almost every aspect of our daily lives is turned into measurable data: how many steps we take, who we interact with, what we like to buy, and even how well we sleep (Mau Reference Mau2019). Moreover, since early 2020, governments have taken unprecedented measures against the spread of COVID-19 to track, trace, and contain the spread of the highly contagious virus using data-driven technologies (Edmond J. Safra Center for Ethics Reference Edmond2020). Citing public health as a justification, governments globally have collected and processed a massive amount of data on their citizens, including geolocation, proximity, social contacts, health and medical history, and biometric data. Thus, it is important to be acutely aware that the uncontrolled development of datafication can result in a data ecosystem that lays fertile ground for the emergence of a data state.
In fact, scholars have long warned that such relentless datafication would lead to a society of “dataveillance” (Clark Reference Clarke1987) and “liquid surveillance” (Bauman and Lyon Reference Bauman and David2013). Dataveillance is the direct, external monitoring of individuals by public authorities, and liquid surveillance refers to the encompassing, obscure, and seemingly less intrusive side of surveillance conducted by business entities. Surveillance becomes “liquid” when its “in/visibility … is characterized by data-flows, mutating surveillance agencies and the targeting and sorting of everyone” (Lyon Reference Lyon2010, 325). David Lyon (Reference Lyon2007, 4–5) emphasized that surveillance becomes fluid as the boundaries between online and offline become increasingly less salient to how people live their lives and the distinctions between tracking, monitoring, and redirecting people’s choices and activities by government agencies (national security agencies and the police) and private entities (corporations and other advertisers) become blurred. Shoshana Zuboff (Reference Zuboff2019, 8) further characterized present-day society as engaging in “surveillance capitalism,” where our actions are transformed into behavioral data to be recorded, collated, evaluated, and sold by private companies. She points out that our data have become the propelling force behind the new data industry and the lucrative business of behavioral advertising, which is the lifeblood of such tech giants as Google and Facebook (47).
What is unprecedented is our own participation in these phenomena; we regularly feed data, whether willingly or unwillingly, to the state and corporations alike. The relentless submission of human data and rampant advances in data technologies may culminate in a data state ecosystem that allows comprehensive access to, and evaluation of, citizens’ data by the state as well as the dominance of the data self over the bio-self, which are the two special features of a data state, discussed earlier.
The Digital Panopticon
We can hardly deny our role in contributing to the new data ecosystem. First, we intentionally and deliberately share our human data with others, whether by providing consent for their use to obtain “free services” or services necessary to get on with our daily lives or by communicating with one another on social media and cheerfully sharing our intimate thoughts and habits online. Second, we leave data trails in cyberspace, such as via our browsing habits, for advertising companies to harvest. Third, our data are shared with private companies by third parties—namely, our friends who post our photos on Facebook or other social media platforms. In sum, the data ecosystem is rich with data that we ourselves contribute. Those data are then easily harvested by private companies and later captured by the state.
This has resulted in our living in a “digital panopticon” of our own construction (Han Reference Han and Butler2017, 8). The powerful concept of the panopticon proposed by Jeremy Bentham (Reference Bentham1791) in the eighteenth century has long been the dominant metaphor for the dual attributes of surveillance: power and domination, on the one hand, and self-discipline and monitoring, on the other. Writing on neoliberalism in the twenty-first century, the influential, Korean-born German philosopher Byung-Chul Han (Reference Han and Butler2017, 40, 61) argued that we have ended up becoming our own panopticon, each of us carrying out the perpetual task of “auto-surveillance.” For Han, digital technologies, including the Internet, give us an illusion of limitless freedom and communication; however, it is exactly this feeling of freedom that lures us ever deeper into a digital panopticon of voluntary self-exposure, self-exploitation, and confession (39). In the digital panopticon, data are not surrendered under duress. Instead, we have been seduced into becoming data subjects with the lure of pleasure or convenience. Han’s portrayal is of the techno-socio landscape of a data-driven society in the Western capitalist style. Unlike China’s SCS, whereby the state steers the development of the data ecosystem and grooms its subjects, the digital panopticon emerging in Western democracies is essentially privatized, with the largely voluntary submission of our human data. People’s voluntary contribution of data may soon become an irresistible surrender to a data state ecosystem.
There are indications that the data and profiles held by corporations are accessed by the state. Contemporary scholars have highlighted the increasing commingling of the state and the market at the technological frontier (Fourcade and Gordon Reference Fourcade and Jeffrey2020). However, such commingling is far from sufficient to offer a complete picture of the new data ecosystem in which citizens are compliant, cooperative, or even active in contributing their data to the state and corporations, thereby facilitating surveillance and control. For example, in the ongoing fights against terrorism and the COVID-19 pandemic, governments worldwide have been collecting sensitive personal data, including biometric data and health data, from both citizens and visitors in the name of public health and safety. The Dubai government, to name one, announced in March 2021 that travelers’ irises will be scanned at the airport to verify their identities (Debre Reference Debre2021). The European Commission introduced a COVID-19 vaccine passport (including details of vaccination, previous COVID-19 test results, and medical statements), allowing European Union (EU) citizens to travel freely around Europe in the summer of 2021 (European Commission 2021). Moreover, it has been reported that the governments of at least twenty-seven countries currently use data from telecommunications companies to track the movements of their citizens (Privacy International 2021). The extension of state power into the private sector for easy access to data is nothing new. The Edward Snowden saga revealed that secret agencies of the US and UK governments have been tapping into private data with the participation of Google, Facebook, Microsoft, and other tech companies (Greenwald and MacAskill Reference Greenwald and Ewen2013).
We often have little choice but to comply, and some of us may even support government monitoring and control measures. Many consider health and survival to be overriding concerns, justifying the trade-off between public health and privacy or other civil rights (Cox Reference Cox2020). Some may even view the surveillance state as an efficient model for combating and ending the pandemic (Cha Reference Cha2020; Cheung Reference Cheung2020). Addressing the willing cooperation and unquestioning obedience of citizens around the globe with respect to drastic government measures in the time of the pandemic, Han argues that we have been reduced to a “society of survival that [has lost] all sense of [the] good life” (Sigüenza and Rebollo Reference Sigüenza and Esther2020). In Han’s view, we have yielded to hysteria over survival and given up many of the civil rights we hitherto held dear (Sigüenza and Rebollo Reference Sigüenza and Esther2020).
Thus, a new social condition has surfaced in which indefinite datafication is embedded in politics and governance, commercial practice and business design, and social lives and daily interactions. The rapidly growing datafication within the new data ecosystem will ultimately provide the state with comprehensive access to our data for evaluation. The data we contribute to big data analytics will facilitate the state’s ability to profile and classify us for the purposes of management, punishment, and exclusion.
Dominance over the Bio-Self by the Data Self
In the second part of this article, we observed the dominance of the data self over the bio-self through the SCS. However, the “data self” is not a new phenomenon unique to the SCS. Our contribution of data to the data ecosystem has given rise to the formation of new digital identities, which scholars have described using various terms of their own creation. For example, Roger Clarke (Reference Clarke1994, 77) used the term “digital persona” to refer to “an individual’s public personality based on data and maintained by transactions, and intended for use as a proxy for the individual.” Kevin Haggerty and Richard Ericson (Reference Haggerty and Ericson2000, 611) explain that a “data double” is formed from the data fragments resulting from discrete surveillance systems. Writing on the age of artificial intelligence, John Cheney-Lippold (Reference Cheney-Lippold2011, 165) describes the construction of a new “algorithmic identity,” defining this as “identity formation that works through mathematical algorithms to infer categories of identity.” Following Gilles Deleuze’s (Reference Deleuze1992, 5) remarks on “dividuals” (divided individuals) who are always in orbit in a continuous network, Cheney-Lippold (Reference Cheney-Lippold2011, 169) conjured an image of dividuals as pieces “of raw data obtained by Internet marketing surveillance networks, [who are] made intelligible and thus constitute the digital subject through code and computer algorithms.” In this system, new information is created about individuals, and new identities are assigned to them. What begins as particular information about particular individuals can be separated from them and recombined in ways unforeseen and beyond their control.
Cheney-Lippold (Reference Cheney-Lippold2011) expresses particular concern regarding how gender, class, and race can be determined by statistical commonality models and data gathered from our online behavior. The recombination of such data configures new identities based on criteria deemed salient by those with access to our information, be they government officials or corporate marketers. The data harvested from a person gain their own body and identity. Information about that body is then treated as if it were conclusive in determining the person’s identity. Cheney-Lippold (Reference Cheney-Lippold2011) points out that, unlike traditional census surveys, this process is not a one-off but, rather, is continuous, meaning that we are categorized through continual interaction with the system, resulting in a constantly evolving data body that is subject to measurement, analysis, prediction, and modification. Moreover, our datafied existence increasingly exerts direct impacts on the rights, interests, and even legal personalities that we enjoy in real life. This is often done without our knowledge, thus eliminating the possibility of legally challenging the outcome. For instance, Cheney-Lippold (Reference Cheney-Lippold2016) stated that “citizenship” is measured and evaluated by the US government based exclusively on algorithmic interpretations of communication data, as Snowden revealed by blowing the lid off the PRISM project. Our algorithmic citizenship may be different from our formal citizenship. Yet our algorithmic identity determines whether we are “foreigners” without privacy rights and subject to surveillance (Cheney-Lippold Reference Cheney-Lippold2016, 1722, 1734).
Although the bio-self may be affected (as discussed in the aforementioned literature), an individual can have more than one algorithmic identity, digital persona, or data double. Furthermore, these three entities may not be true representations of the bio-self. For instance, an individual’s algorithmic identity may constitute only an inference of that person, a digital persona may be created from an inadequate dataset (Clarke Reference Clarke1994), and a data double can be a misleading and/or incomplete representation of the self (Lyon Reference Lyon2007). Disturbing and menacing as PRISM is, the US government’s interest in the collection of data is limited to determining whether citizens are potential terrorists. Thus, it may be tempting to conclude that the bio-self is unlikely to be overtaken by the data self in Western democracies.
However, with the increasing datafication of our daily lives and ongoing advances in data technology, our desires, purposeful activities, spontaneous social networks, and subconscious habits have been captured by interconnected networks. Unsuspecting citizens engaging in various online social activities for gain, pleasure, and/or convenience, although vigilant, may nonetheless find themselves docile subjects with no ability to exit or resist. If the relentless growth of datafication is allowed to continue unchecked, we may soon become dependent on data-driven technologies in all arenas of our daily lives and our bio-selves overtaken by our data selves, as in the case of a fully-fledged SCS.
CONCLUSION
Above, we discuss the various salient features of China’s SCS. The most striking elements of the ambitious SCS system are the state’s potential ability to comprehensively evaluate its citizens’ datafied behaviors and the grooming of the data self, with the data self taking precedence over and steering the bio-self. In the end, the bio-self may have to learn how to co-exist with a data self that has completely redefined it in practical terms.
Our lives and our beings, it seems, are made up of data. State authorities are keen to engage with our data for easy governance. China’s SCS arguably provides an efficient model to other state authorities of how to integrate behavioral standard setting and data-based mechanisms in pursuit of a government agenda. The SCS is a complex system aimed at monitoring its citizens through multiple players, evaluating various dimensions of their lives, and intervening through semi-automated schemes of punishment and reward. In building the SCS under the decontextualized rhetoric of “trustworthiness,” China’s ruling party has defined the characteristics of a trustworthy citizen in a party-state. Under the current system, only the financial credit rating function uses big data analytics, and it is used to a limited extent, but this does not preclude the possibility of the SCS’s harnessing data-driven techniques for its higher functions and becoming integrated into other surveillance networks. Should that happen, the decontextualized evaluation process would become more organic, with the definition of a trustworthy citizen inferred, constructed, and developed through techno-social engineering. Whether currently or in the future, it is not only the meaning of a trustworthy citizen that is being defined; it is also one’s identity as an individual that is being molded by the state.
Although China’s SCS may not be replicated by other countries, the accelerating rate of datafication of various aspects of our lives can easily contribute to the formation of a data state ecosystem, particularly as the state is extending its reach to capture human data held by private corporations. We must not ignore a similar tendency of Western state authorities to extend their reach to private enterprises in the name of national security or public health. Although China’s SCS is still evolving, we call for critical awareness of how a society may be transformed into a data state to prevent the seemingly invincible and irreversible datafication of governance from resulting and to obviate the fertile soil that brews the data state. Understanding the special features of the data state is essential to enabling the citizens of democratic societies to be alert to and, if necessary, call a halt to any further expansion of datafication at this critical juncture. It also paves the way for further research on devising a system of effective checks and balances against the ever-expanding reach of datafication.
China’s SCS may be a distinct case of the emergence of a data state; however, it offers a cautionary tale that must be taken seriously. The impact of using data technology to record, evaluate, and eventually control individuals’ behavior cannot be underestimated. This tale invites us to contemplate both the temptations of data, technology, and power and the vulnerability of individuals and individuality.
Open access
