2.1.1 Algorithms

Algorithms are most commonly described as finite sequences of defined instructions, aimed at solving a problem.Footnote ¹ These instructions can be expressed in computer code, which enables the algorithm’s execution by a computer program. In this way, algorithms enable human beings to automate processes that they would otherwise need to undertake themselves, from a simple repetitive task to a complex calculation, or the analysis of heaps of information. An algorithm is hence shaped by whichever is the human-defined objective behind it, or the problem one aims to solve.Footnote ² Over time, researchers have sought to address problems of varying complexity through algorithmic systems – understood as systems that are comprised of multiple algorithms – from automatically translating text from one language to another, or distinguishing between different human voices, to calculating the individual tax rates of the entire population. Algorithmic systems are increasingly used to take over tasks which would be considered as requiring ‘intelligence’ if carried out by a human being.Footnote ³ Accordingly, the term artificial intelligence – which I will discuss further in Section 2.1.5 – can be seen as an umbrella term for different algorithmic techniques aimed at carrying out ‘intelligent’ functions.Footnote ⁴

In its simplest form, an algorithm relies on certain input that is provided, after which the algorithm – or the sequence of step-by-step instructions expressed in code – is executed, which ultimately results in an output.Footnote ⁵ Figure 2.1 on the next page provides a visual abstraction of this process.

Figure 2.1 Abstraction of an algorithm.

An algorithm can be compared to a cooking process: it requires ingredients (input), step-by-step directions of how to prepare and cook these ingredients (instructions), resulting in a – hopefully delicious – dish (output). The shape and content of the input, instructions and output will depend on the algorithm’s context and purpose, just as the ingredients, cooking instructions and resulting dish will depend on what one intends to cook.

To provide an example, I may wish to automate the issuing of a fine to highway drivers who drive too fast. They should, however, only be fined when they exceed the maximum speed limit, which on Belgian highways is 120 km/h. To this end, I could develop an algorithmic system that uses as input the measured km/h that the car drives. If I do not want to manually insert those km/h into the system, I could equip the system with a sensor that enables it to automatically measure the speed and thereby facilitate the provision of its input. In terms of instructions, I will need to write out in code that, once the speed limit is exceeded, a fine should be issued. Finally, the resulting output would be a concrete decision to issue a fine (or not) to the driver in question. Of course, I could also decide to include additional input and instructions to the system so that I not only automate the decision of issuing a fine but also the subsequent step of looking up the driver’s address, and notifying her that a fine should be paid within a certain timeframe – yet in its simplest form, this algorithmic process comes down to Figure 2.2.

Figure 2.2 Abstraction of an algorithmic system to automate the decision to issue a fine.

The outcome of an algorithmic system can thus be used to make a decision without the need for further human intervention, such as in the above example. This can be referred to as algorithmic decision-making, since it concerns a decision-making process that is automated through an algorithm.Footnote ⁶ Although I may be physically present when a driver crosses the speed limit, I will not necessarily be prompted to first verify the system’s input, nor will I need to confirm that a fine should indeed be issued. Instead, I have outsourced that decision to the algorithmic system. The outcome of such system, however, need not necessarily be a final decision. Algorithms can also be used for a range of other functions or sub-tasks, from executing a simple calculation, to computing a prediction, recommendation or categorisation,Footnote ⁷ which can then inform a subsequent human or algorithmic decision.

The above algorithmic system could, for instance, also be tweaked to produce as output a recommendation of whether or not to issue a fine, which a human being then needs to approve, rather than an actual fining decision. This is sometimes denoted as an algorithmic recommendation system. Moreover, it is also possible that an algorithmic system is used to merely support a sub-part of a decision, based on which a human being can then decide to take a particular action. For instance, an algorithmic system can be used to automatically determine the speed at which a car was driven, without also being programmed to issue a recommendation to fine the driver when that speed reaches the limit. Nevertheless, I may use that piece of information to determine whether the speed limit was exceeded and whether a fine should be issued. This can be referred to as a more general algorithmic decision-support system, since the system is used to support a decision, without making an explicit recommendation or taking a final decision.Footnote ⁸

Collectively, the algorithmic decision-making system, recommendation system and support system described above enable algorithmic regulation, since they are deployed by a public authority to inform or take an administrative act (namely, issuing a fine). In this book, I will hence deploy the term algorithmic regulation to cover all three types of systems. However, when discussing concrete examples, and whenever the distinction is relevant, I will indicate whether it concerns an algorithmic system that adopts (takes) or that informs (recommends or supports) an administrative act.

An obvious advantage of using an algorithmic system instead of manually measuring the driving speed and issuing corresponding fines is the fact that the process can be applied at much larger scale and speed, so that many more drivers can be controlled and fined where needed and unwanted behaviour can be reduced. Moreover, I no longer always need to be physically present at the place of the event, but can instead be at my office and work on another matter in the meantime. However, since the delegation of tasks to algorithmic systems also entails, to a certain degree, the delegation of responsibility for tasks originally undertaken by human beings, reliance thereon can give rise to several ethical and legal questions, depending on the task at stake. As long as I am using an algorithmic system to automatically activate my heater when the temperature falls below 20°C, this algorithmisation does not lead to much consternation. Yet the algorithmisation of tasks that have more significant consequences for individuals, collectives and society evidently needs to be handled with greater care. In the example above, for instance, all three of the systems – although differentiable in the part of the task that they automate – can entail consequences for drivers who may be subjected to a fine.

Various algorithmic approaches can be used to automate tasks, and these can be classified in several ways. A commonly made distinction concerns algorithmic systems focused on reasoning – sometimes referred to as knowledge-driven systems – and those focused on learning – sometimes referred to as data-driven systems.Footnote ⁹ Yet it must be stressed that the boundaries of these categories are not uniformly set, and that algorithmic systems can also draw on different approaches simultaneously or on approaches that do not neatly fall under one or the other category. With this caveat in mind, in what follows I will set out the main features of knowledge-driven and data-driven systems, respectively. Rather than giving a comprehensive account of the wide set of techniques falling under each of these approaches, the discussion below will be succinct, aimed only at providing the reader with a basic understanding of how these systems might operate the context of algorithmic regulation.

2.1.2 Knowledge-Driven Systems

As their name implies, knowledge-driven systems start from a certain knowledge model of reality that is represented through code, based on which the algorithm can subsequently reason. Two elements are core in this regard. The first concerns a knowledge base, which can contain facts, observations and other types of data that represent a coded model of reality. Such knowledge must be ‘translated’ so that it can be processed by an algorithmic system – also known as knowledge representation.Footnote ¹⁰ The second element concerns coded instructions about how the system can analyse, combine and draw inferences based on that knowledge to solve a problem – known as knowledge manipulation or simply reasoning.Footnote ¹¹ These instructions are typically codified in the system through symbolic rules that set out the relationships between the symbols and establish how the system can deduce information, engage in planning and scheduling activities, search through knowledge bases, and optimise solutions.Footnote ¹² How the developer of the system translates knowledge and rules into code is hence key, as is the extent to which the codified knowledge is correct or based on appropriate assumptions and beliefs. Given the important role that pre-established rules play in these systems, they are sometimes also referred to as ‘rule-based systems’.

Rule-based systems were the dominant paradigm up until the 1980s and are therefore also known as ‘traditional’ or ‘good, old-fashioned’ AI.Footnote ¹³ They have, for instance, successfully been programmed to win a game of chess from the world chess champion,Footnote ¹⁴ as the rules of chess-playing are well delineated and can hence be represented in code with relative ease. Such systems can also be used by public services to assist in decision-making on the allocation of social welfare benefits. To this end, information about the income, family situation and legal status of individuals first needs to be represented in code – along with other information that determines whether or not an individual is eligible to receive benefits. In addition, the relevant legislation that stipulates the criteria and conditions under which individuals can receive benefits also needs to be translated to symbols and rules. On that basis, the system can subsequently infer a citizen’s benefits eligibility, calculate the corresponding amount and provide as output a recommendation or decision on whether benefits should be allocated and how high the amount should be – as represented in Figure 2.3.Footnote ¹⁵

Figure 2.3 Abstraction of a knowledge-driven system to automate welfare benefits allocation.

Given their reliance on finite and discrete symbols and logical rules, reasoning processes are typically interpretable. The reasoning steps of the algorithm – which can have varying forms of complexity – can be retraced, which makes the algorithmic process relatively transparent and explainable (at least for technical experts, who should then be able to communicate about how this process works to those subjected thereto). This traceability also makes it possible to verify whether some important information has potentially been overlooked, or whether the legal rules about benefits eligibility, which were translated into symbolic rules, have been interpreted correctly. At the same time, reasoning techniques in principle presuppose that the steps or rules that need to be taken to solve a problem are – at least to a certain extent – already known.Footnote ¹⁶ Similarly, they presuppose the existence of human expertise in the domain to which the problem relates.Footnote ¹⁷ In situations where the rules that apply are pre-established by law, relatively precise and relatively easily codifiable in symbols, this need not pose major issues – even though, as we shall see in Chapter 4, the translation from legal rules to code poses fundamental challenges.Footnote ¹⁸ However, in situations where the rules that govern the situation are not (yet) known, vague or uncertain, a knowledge-driven approach that is based on simplistic rules may not be effective.

Consider, in this regard, an example in the same sphere: an algorithmic process to assess the risk that individuals commit social welfare or tax fraud. While the majority of taxpayers and welfare applicants act in good faith, in every country, there will be individuals who commit fraudulent acts to pay less than they owe, or to receive more than they should.Footnote ¹⁹ Since public resources are limited, and not every individual can or should be investigated for fraud, one may consider developing an algorithmic system that can help identify or predict the risk of fraud, in order to prioritise those cases for investigation. Yet, while I can probably reuse a lot of the information on citizens that I had gathered for the previous example, this task may be more difficult to delegate to a rule-based knowledge-driven system. What are the rules that I would need to codify to identify a potential risk of fraud? Fraudulent types of behaviour do not always occur in easily describable steps.Footnote ²⁰ I could make an attempt by codifying the rule that anyone who applies for welfare benefits yet lives together with someone earning more than 2,500 EUR/month poses a risk of fraud, as shown in Figure 2.4. However, with this type of rule, the output – namely, the decision on whether I need to further investigate that applicant – risks being both overly inclusive and exclusive. Taking a rule-based approach may pinpoint me somewhat in the right direction, but will still require me to take into account that this rule may be too general to yield effective results. Of course, the more knowledge and experience I gain, the more I can use this knowledge to formulate more tailored rules, and perhaps also to add some exceptions to the rule. However, fraudulent techniques might change over time, so I would constantly need to update those rules.

Figure 2.4 Abstraction of a knowledge-driven system to identify the risk of social welfare fraud.

To remedy the problem of uncertainty, recourse can be made to probabilistic and statistical models that can help make predictions based on the knowledge that I do have, and thereby cover certain knowledge gaps and reduce uncertainty.Footnote ²¹ These models are increasingly being researched and tailored for use in the context of knowledge-driven approaches. By relying on these models, the rigidity of more simplistic ‘rule-based’ systems that merely apply a static rule to a knowledge base can be diminished. Probabilistic and statistical models can hence be seen as lying somewhere in between knowledge-driven and data-driven systems.

Importantly, the knowledge on which the system reasons needs to be transformed into machine-readable code.Footnote ²² Depending on the size of the knowledge base, the potentially informal nature of the knowledge, and the relationships that need to be represented between the various symbols that represent the knowledge, this can be quite laborious. In the context of algorithmic regulation, whenever the legal rules that need to be translated are vague or open to multiple understandings – which is not infrequent, given the inherent openness and interpretability of language – certain interpretative decisions will need to be taken in advance, to the exclusion of others. Furthermore, if knowledge about certain aspects of the problem is lacking, it cannot be represented. Unless this is remedied through approaches that provide more flexibility, this will mean that – when a specific situation arises that deviates from the codified norm and for which no explicit alternative rule has been foreseen – the system is unlikely to provide a solution. Notwithstanding this restraint, knowledge-driven systems are used profusely in virtually all societal domains, and are able to assist in tasks of high complexity.

2.1.3 Data-Driven Systems

Researchers also explored other routes to perform ‘intelligent’ tasks, focusing on data-driven approaches instead. Rather than codifying a certain model of knowledge in advance, data-driven systems “let the data speak”Footnote ²³ and infer the model from the data instead. Accordingly, these models typically allow for more flexibility and adaptability since, by learning from the data, the system will be able to adjust to new situations. Though already explored earlier, data-driven systems primarily gained traction after the 1980s, when the capabilities of knowledge-driven systems seemed to be hitting their limits.Footnote ²⁴ Especially from the twenty-first century onwards, data-driven approaches started booking new successes and led to several major breakthroughs in the field – particularly those relying on machine-learning techniques.Footnote ²⁵ It is for this reason that some AI definitions only refer to learning-based systems.Footnote ²⁶

The recent successes of those systems are largely linked to the enhanced availability of data and in particular ‘big data’, which can be defined as “large volumes of extensively varied data that are generated, captured, and processed at high velocity”.Footnote ²⁷ A growing number of tools enable the collection and analysis of data in digital – and hence easily perusable – format, by means of sensor-embedded objects, phones and computers, and the Internet of Things (IoT) more generally.Footnote ²⁸ The fact that the processing power of computers significantly improved – thus enabling more and faster computations, and the fact that the storage of large volumes of data became more affordable, likewise contributed to the success of data-driven systems.

There are various types of data-driven techniques. Generally speaking, such techniques aim to infer the rules that should be followed from (a large number of) examples contained in a dataset, which can be labelled or not. The algorithm is thus tasked with inferring the rules (or function) behind the data and to create a model that should identify the relationship between the various datapoints provided to the system (for instance, inputs and outputs). By analysing a large set of examples (training phase), and by continuously tweaking the model based on those examples to better fit the dataset’s curve, the function can be determined with ever more accuracy. Once the model has been developed based on training data, whenever the system is confronted with new input, the model can be used to predict that new input’s corresponding output (use phase). In addition, there will typically also be a testing phase in between, to see how well the model performs before deploying it at scale.

The fact that data-driven systems are able to ‘learn’ a model from the data they are provided, does not take away the fact that these systems likewise rely on human coding. After all, the algorithms still hinge on instructions pertaining to the development of the model, and the use of the model to categorise or cluster new data. Yet the representation of reality itself will in principle not be codified in advance, nor will the instructions set out how such representation can be manipulated to achieve a certain result. This absence of the codification of knowledge and rules that set out the relationship between different knowledge elements also means that data-driven systems are typically not able to pick up causal relations between input and output (though this is a growing topic of research).Footnote ²⁹ Instead, the inferred relationship between input and output is one of correlation. As to inferring the model, different techniques exist. A distinction is often made between supervised, unsupervised and reinforcement learning techniques – each of which is briefly discussed below.Footnote ³⁰

Under supervised learning, an algorithm is instructed to analyse training data examples that consist of input–output pairs which are explicitly labelled or scored, and to infer a model therefrom. The labelling typically occurs by a human being prior to the dataset’s use. Based on the model that is inferred from the original examples, the system will have learned how to classify new examples. The performance of the system will typically increase over time, as the analysis of more data can allow it to infer more granular rules and improve the model.

For an example of such a method, I suggest revisiting the algorithmic system used to assess the risk of social welfare fraud. Instead of using a rule-based algorithm and programming rules that are likely to be under- and over-inclusive, I can instead try to teach the system to distinguish between fraudulent and non-fraudulent applications, as represented in Figure 2.5 on the next page. To develop the system, I could train it on a dataset containing information about past welfare benefits applicants, as well as an indication (or label) of whether they committed fraud in the past or not. Crucially, I would need a sufficiently large dataset, with enough examples of both types of categories.Footnote ³¹ On that basis, I would instruct the algorithm to create a model that seeks to deduce the relevant variables for each of the two categories. When a new applicant subsequently submits an application to receive benefits, I can then use that model to predict which of the two categories – fraudulent or not – best captures the pattern of the applicant’s profile. The final output would consist of the categorisation of that applicant in the corresponding category. If the applicant would be categorised in the ‘fraudulent’ category, I may treat this application with extra suspicion, and investigate it further before accepting it.

Figure 2.5 Abstraction of a supervised data-driven system to predict the propensity of fraud.

Alternatively, it may well be that I would like the system to discover fraudulent features that I did not think of myself. In that case, I can turn to unsupervised learning techniques. Under unsupervised learning, the data based on which the algorithm is trained does not contain pre-assigned labels. Instead, the algorithm analyses the unlabelled dataset by looking for potential patterns therein, and by developing a model based on those patterns. This model can, for instance, result in the clustering of data into categories with similar features, or the detection of certain anomalies within a given dataset. These techniques are hence useful to explore certain rules or clusters that were not previously identified within a dataset.Footnote ³²

When looking for welfare fraud, rather than explicitly labelling examples, I could let the algorithm create a model based on past data from citizens, and let it identify clusters of datapoints or anomalies in the dataset that may reveal potential correlations with fraud – as represented in Figure 2.6 on the next page. For instance, the data analysis might reveal that a large majority of past fraudulent applicants refused to provide public authorities with information about their financial situation when asked. A model can then be created based on the identified correlations, which I could subsequently use to assess the fraud propensity of a new applicant by verifying whether one of the identified patterns is also present there. Also in this example, the final output – based on the model’s prediction of fraud propensity or not – can be a recommendation to treat a certain application with suspicion. A civil servant may choose to follow this recommendation or not. Alternatively, the recommendation can also be formalised into a decision to automatically take further investigative steps, without leaving any margin of appreciation in this respect.

Figure 2.6 Abstraction of an unsupervised data-driven system to predict the propensity of fraud.

A beneficial feature of unsupervised learning methods is not only the reduced workload – since the training set need not be labelled in advance – but also the fact that patterns can be identified which human beings did not previously consider or perceive. At the same time, this technique requires a larger dataset to produce successful results, and it can also lead to the identification of patterns that are irrelevant, erroneous or useless, precisely because there are no prior labels that indicate what is of value to solve the problem.Footnote ³³ In the example above, it may well be that the data analysis reveals that a majority of past fraudulent applicants owned a cat, and a majority of past non-fraudulent applicants owned a dog. It would, however, be a far stretch to automatically treat all new applications from cat-owners as suspicious due to the mere fact that such a correlation was picked up. As I discuss in the next section, beyond the risk of spurious correlations,Footnote ³⁴ the lack of labels also means that the data clustering can occur in a manner that deviates from non-discrimination law, by, for instance, taking into account prohibited grounds of discrimination, such as gender or nationality.Footnote ³⁵

Finally, I should also mention a third learning technique, called reinforcement learning.Footnote ³⁶ This approach aims at enabling an algorithmic system (denoted as ‘the agent’) to learn which decision it should take through a reward mechanism, given uncertainty about the system’s environment and given an objective – represented by a value function – that needs to be maximised. The system is instructed to observe the state of an environment and to identify the optimal action to alter the environment’s state in such a way that the receipt of a reward signal is maximised. Since the system’s aim is to maximise accumulative reward signals rather than an immediate reward, it learns to identify the course of action yielding the best results over the longer term.Footnote ³⁷

Unlike supervised learning, there are no labelled pairs of input–output examples to train the system. Instead, the model considers state–action pairs, and the extent to which they yield a certain reward. This reward feedback – typically in the form of a numerical score – allows the model to be tweaked so as to increase its success in achieving the value. Each time, the system needs to balance the exploitation of knowledge it gained from previous modelling attempts, with the exploration of new action methods that were not yet tried but could yield a (higher) reward.Footnote ³⁸ By repeating the process of analysing the result of an action on the environment’s state and the corresponding reward, the model can thus come closer to maximise the value that represents the objective over the long term.Footnote ³⁹

An example of a reinforcement learning system is a chatbot deployed by a public authority to provide citizens with answers to some frequently asked questions, as represented in Figure 2.7 on the next page. It can draw on an initial dataset of past questions and answers, but also on a wider collection of information concerning the public authority’s functioning that may be of use for citizens. The chatbot can be programmed to seek feedback from the citizen at the end of the conversation to evaluate whether the provided answers satisfied the citizen’s inquiries. This feedback can be used to improve the chatbot’s answer whenever a similar question is asked by a citizen in the future.

Figure 2.7 Abstraction of a data-driven system to automate and improve the answering of citizen questions.

The upside of this technique is the fact that one does not need to undertake the laborious task of labelling input–output pairs, one does not need to have complete information about the environment and one does not need to give continuous feedback about the model’s performance, since this is only done for the totality of the outcome (in the example above, the quality of the chatbot’s final answer). An important downside, however, is the risk that the algorithm may compute a model that merely pursues ‘reward hacking’, namely, the maximisation of the reward signal, while oblivious to the fact that intermediate actions to achieve that reward may be undesirable or cause undesirable consequences.Footnote ⁴⁰ The way in which the objective – or the value function – is defined, is hence essential.

Finally, let me also make a brief note about large language models or LLMs, which are learning-based models that are designed to both ‘comprehend’ and generate text. These models are trained on a very large dataset of text examples, which in some cases span virtually all text that can be found on the internet.

The model is trained to infer and predict which sequence of words constitutes the most sensible response to a prompt provided by the user, based on the prompt’s context. Just as with other learning-based models, LLMs consist of large strings of numbers (weights) which are ‘interpreted’ and executed by code. When the model is trained, it slightly adjusts these weights based on what it has ‘learned’, allowing it to improve its predictions of the next suitable word(s) in light of the overall context. It is precisely the capability of large language models to take such context into account that explains the success of their predictions, and hence of their performativity. These models have also been referred to as foundational models, as they are typically trained on a broad range of data rendering them capable of performing various general tasks, which makes them a suitable ‘foundation’ to build new applications in a more cost-effective manner, without the need to develop a model from scratch. Indeed, the models can be finetuned to serve more particular purposes, for instance by feeding them with domain-specific datasets to adapt them to certain contexts (e.g. in the area of law or medicine) or to adapt them to specific language types (e.g. therapeutical conversations).

Since LLMs are typically further trained on the inputs they receive from their users, they heavily rely on reinforcement learning by human feedback. By virtue of their ability to predict relevant word sequences, LLMs can be used to power chatbots, such as ChatGPT (by OpenAI), Bing Chat (by Microsoft), Bard (by Google) and Llama (by Meta). Beyond text, similar approaches also drive applications that enable the generation of images, videos, and audio. Since the interfaces of these generative models are specifically designed to enable their use by non-experts, the wide availability of the systems – often free or at low cost – arguably lead to a major breakthrough in the public’s awareness of the technology’s transformative impact on society. Jumping on this hype, it should come as no surprise that public authorities too have been exploring how they can use LLM-powered chatbots to facilitate their interactions with citizens to more efficiently deliver their services. In October 2023, the UK government, for instance, announced that it would launch an AI chatbot to help Britons pay their taxes and access their pensions, based on a model that would be powered by OpenAI.Footnote ⁴¹

2.1.4 Comparing Approaches

Bearing in mind the above description of knowledge- and data-driven algorithms, we can now make a few comparisons. Firstly, when knowledge-driven systems merely rely on the codification of symbols and rules, they can be relatively inflexible, since they do not deal very well with new and uncertain situations that deviate from the coded model. This is, however, different for knowledge-driven systems that also draw on probabilistic and statistical methods, as they can extrapolate from the known information to make predictions that can contribute to a solution, despite the potential uncertainty or knowledge gaps. Data-driven systems are typically more flexible, as they enable the inference of models that can evolve over time in light of new data that is fed into the system. Yet the models established through data-driven systems or through statistical methods can subsequently be used for reasoning, thus underscoring the fact that systems need not only focus on one approach or another.Footnote ⁴²

The fact that data-driven systems do not require a coded description of what the data represent by translating knowledge into symbols and symbolic rules has been an important contributor to the successes they booked, especially in domains where the representation of codified knowledge is more difficult. Indeed, over the last decade, learning-based approaches have, for instance, been responsible for significant advances in computer vision (or methods to acquire and analyse images and videos), computer hearing (or methods to acquire and analyse sound data like music or speech), and computer touch (or methods to acquire and analyse tactile information). Furthermore, especially when using unsupervised learning techniques, learning algorithms can help identify patterns and categorisations that humans did not previously perceive or consider, which can lead to useful new insights.Footnote ⁴³

However, the use of data-driven systems also has drawbacks. First, the inherently probabilistic nature of their outcomes means that there will always be a margin of error. In some situations, such algorithms can achieve a very high level of accuracy in their predictions, yet it may be that the dataset on which the system was trained did not present an accurate representation of real-life situations, thus reducing the system’s accuracy and utility outside the lab.Footnote ⁴⁴ The output may also be false or entirely estranged from reality, which is especially problematic in the context of generative models that produce text or images which come across as highly realistic. It has been noted that data-driven algorithms can be “statistically impressive”, yet they remain “individually unreliable”.Footnote ⁴⁵ That said, knowledge-driven algorithms are not free from error either: if the knowledge that the system should reason upon is not correctly represented by the human programmer, the outcome will be equally flawed.

Second, certain data-driven algorithms – especially those relying on deep learning methodsFootnote ⁴⁶ – suffer from a lack of transparency and interpretability, which has been referred to as the so-called ‘black box’ problem. Such systems “internalise data in ways that are not easily audited or understood by humans”, for instance due to the “complexity of the algorithm’s structure”, or because they rely “on geometric relationships that humans cannot visualise”.Footnote ⁴⁷ This means that human developers and deployers of such algorithms cannot always explain how a certain outcome was achieved, and why it was achieved.Footnote ⁴⁸ This can be contrasted with the interpretability of knowledge-driven systems which, in light of their typically predetermined symbolic rules, provide better insight into how and why a certain output is provided.

Third, while data-driven algorithms can identify patterns and correlations between data points that humans might not have recognised, they are in principle unable to establish causal relationships between those points, which may lead to spurious correlations, a risk inherent to virtually all statistical models.Footnote ⁴⁹ Furthermore, the lack of symbolically represented knowledge also renders data-driven systems unable to contextualise the information they process. This means that certain logical facts that are evident for human beings (‘common sense’), may not be picked up by the system. In addition, the system can also ‘learn’ in ways that the developers did not predict or intend, which may lead to unforeseen adverse consequences.Footnote ⁵⁰

Finally, data-driven algorithms have a higher vulnerability to adversarial attacks. By inserting deceptive input into the system – which can go unnoticed by the human eye – one can deliberately confuse the model created by the system and thereby compromise its output.Footnote ⁵¹ While certain defence techniques exist against those attacks, such as data manipulation constraints, the risk nevertheless remains that the system’s vulnerabilities are exploited, and that the results computed by the system become unreliable.Footnote ⁵² The high complexity of the models and the decreased transparency mentioned above, can also render the detection of those attacks more difficult.

In sum, knowledge-driven and data-driven techniques each have a set of capabilities that enable the automation of (intelligent) tasks. When seeking to solve a problem through an algorithmic system, the choice of technique will hence be determined by the type of problem, the existing knowledge and expertise about the problem, the availability of (representative) data, the characteristics one wishes to prioritise when seeking a solution to execute the task and – most importantly – the budget. One does not necessarily need to choose between one technique or another, as various methods exist to remedy the constraints of each, by combining reasoning techniques with learning-based approaches. Data-driven models can be used to help predict how the sought-after data could look like, after which the outcome of that prediction can serve as a basis for further reasoning and analysis through symbolic rules.Footnote ⁵³ This demonstrates that the distinction between knowledge- and data-driven techniques is not always clear-cut. In fact, the most recent research trend focuses on hybrid systems, whereby knowledge- and data-driven techniques are combined.Footnote ⁵⁴

2.1.5 AI Systems

Before concluding this technical chapter, there is one more term I wish to expand on in the context of algorithmic regulation, namely artificial intelligence (AI) – the importance of which has been described as “more profound than electricity or fire”.Footnote ⁵⁵ World leaders have stated that “whoever becomes the leader in this sphere will become the ruler of the world”,Footnote ⁵⁶ and the European Parliament’s Special Committee on Artificial Intelligence in a Digital Age even claimed that it can be “thought of as the fifth element after air, earth, water and fire”.Footnote ⁵⁷ Aside from the hype that these hyperbolic statements represent, what does this technology consist of, and how does it relate to the algorithmic systems I described in the previous sections?

As noted above, AI can be understood as an umbrella term for a range of ‘intelligent’ technological applications. The pursuit of intelligent machines is almost as ancient as human history,Footnote ⁵⁸ yet the first time the term AI was used dates back to the now-famous 1956 Dartmouth Workshop, considered to mark the birth of ‘AI’ as a field of research.Footnote ⁵⁹ While all AI systems are composed of algorithms, not all algorithmic systems necessarily fall under the AI umbrella. As the term indicates, to be considered ‘artificially intelligent’, a system would need to perform a task that is typically associated with intelligence, rather than merely constituting the simple automation of a task. What is considered an ‘intelligent’ task may vary significantly based on the person to whom the question is asked, and the discipline they are trained in. A philosopher might define the term quite differently from a schoolteacher or a lawyer. More importantly, AI experts, too, tend to define this technology in different ways. In their seminal AI handbook, Russell and Norvig, for instance, identify definitions that focus on the ability of a machine to think or the ability act humanly, versus definitions focusing on their ability to think or act rationally.Footnote ⁶⁰ The latter definition is more prominently used among AI researchers nowadays, as it can more easily be formalised by virtue of an AI system’s optimisation function, through which the system can be tasked to select those actions that are expected to maximise its utility.Footnote ⁶¹

In essence, one could consider algorithmic systems on a spectrum. One extreme contains algorithms with only a very minimal set of instructions, too basic or simplistic to be considered ‘intelligent’. The other extreme contains algorithms that are highly complex and sophisticated, and would be considered ‘intelligent’ by virtually all researchers. Yet, when it comes to algorithmic systems situated in the middle part of the spectrum, researchers do not always agree on whether the system would be considered ‘intelligent’ or not, and the perception thereof also tends to change over time.Footnote ⁶² There is no fixed rule as regards how many or which type of instructions an algorithmic system must contain in order to be called an ‘AI’ system.

Since the term AI is currently widely used, and since the European Union’s regulatory initiative is titled ‘the AI Act’, let me discuss some AI definitions put forward in the context of EU policy, and examine how these compare with the description above. In this regard, it must be borne in mind that the conceptualisation of ‘AI’ has not only an academic or scientific relevance, but also a political one. The scope of the regulatory requirements that legislators may impose on ‘AI’ systems will after all depend on how they define ‘AI’. Furthermore, the definition of AI also has a societal relevance, as it contributes to framing the broader narrative around the impact of AI in society.Footnote ⁶³ Consider, for instance, the way in which the European Parliament’s Delvaux report frames AI, evoking the context of “Mary Shelley’s Frankenstein’s Monster” and “the story of Prague’s Golem”,Footnote ⁶⁴ and cautioning that “humankind stands on the threshold of an era when ever more sophisticated robots, bots, androids and other manifestations of artificial intelligence (‘AI’) seem to be poised to unleash a new industrial revolution”.Footnote ⁶⁵ The Report further states that “ultimately there is a possibility that in the long-term, AI could surpass human intellectual capacity”,Footnote ⁶⁶ and makes the much-commented suggestion to create “a specific legal status for robots in the long run, so that at least the most sophisticated autonomous robots could be established as having the status of electronic persons responsible for making good any damage they may cause”Footnote ⁶⁷ – a suggestion that was rejected by the European Commission. However, the Commission did follow up on the report’s proposal to establish a definition of AI and to draft ethics guidelines to guide use of the technology.

To this end, it set up a High-Level Expert Group on AI in June 2018 composed of fifty-two experts from various domains, tasked with the drafting of Ethics Guidelines for AI as well as Policy and Investment Recommendations.Footnote ⁶⁸ As part of its tasks, the Expert Group set up a working group to establish a definition of AI – an exercise that took just as long as agreeing on AI ethics principles – which resulted in the following formulation:

The following elements can already be distilled from this definition. First, as will be stressed in the next section, the Expert Group emphasised that systems are designed by human beings. This means that they do not ‘overcome’ us passively but that they are an active creation of persons who are therefore also responsible for their consequences.Footnote ⁷¹ Second, the definition takes into account that there is no ‘single’ AI system.Footnote ⁷² It indicates that AI systems can ‘use symbolic rules’, as in the case of knowledge-driven systems, thus not limiting the term AI only to data-driven systems. Accordingly, AI is considered an overarching concept covering a range of technologies that have certain properties in common, namely their ability to reason on or learn from the data provided to them, and to “act in the physical or digital dimension” based on such data. As regards data-driven systems, the definition indicates that AI systems can ‘adapt their behaviour’ over timeFootnote ⁷³ based on what they ‘learn’,Footnote ⁷⁴ which is in turn dependent on how they have been programmed to do so by a human being. Lastly, the definition also mentions hardware, thereby indicating that knowledge- and data-driven algorithms can also be incorporated into hardware to design machines that can carry out physical tasks, which is the aim of the field of robotics.

While the European Commission endorsed the Expert Group’s Ethics Guidelines, it did not retain the above definition in its subsequent proposal for a new AI regulation, published two years later in April 2021. Instead, it suggested a somewhat different definition of AI, encompassing “software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with”.Footnote ⁷⁵ Like the High-Level Expert Group, the Commission’s definition stressed that the objectives of AI are ‘human-defined’, also indicating a difference in style with the Delvaux report.Footnote ⁷⁶ At the same time, the Commission’s original proposal suggested to exhaustively list in annex I the specific techniques and approaches that should fall under the AI Act’s scope, so as to enhance legal certainty. These concerned:

1. (a) Machine learning approaches, including supervised, unsupervised and reinforcement learning, using a wide variety of methods including deep learning;

2. (b) Logic- and knowledge-based approaches, including knowledge representation, inductive (logic) programming, knowledge bases, inference and deductive engines, (symbolic) reasoning and expert systems;

3. (c) Statistical approaches, Bayesian estimation, search and optimisation methods.

Accordingly, the Commission originally opted for a broad general definition of AI, while simultaneously meticulously specifying the underlying techniques that would actually be caught. This implies a focus on the technical approach behind the systems and their specific research discipline classification rather than their properties or effects. The idea behind listing these techniques under an annex was to ensure flexibility in the future, in case new approaches come along that do not neatly fit under the existing ones – even if list-based definitional approaches inevitably risk being both over- and under-inclusive. Stakeholders have therefore proactively been seeking to influence the regulation’s definition of AI prior to its adoption.Footnote ⁷⁷

A number of actors have argued that the Commission’s original definition was too broad.Footnote ⁷⁸ Some argued that only data-driven approaches – and more specifically machine-learning applications – should be covered by the proposed AI Act,Footnote ⁷⁹ while others proposed that at least the approaches listed under point (c) should be excluded.Footnote ⁸⁰ In addition, it has been argued that not only AI systems but also other technologies that can be used for similar tasks should be included, to ensure a ‘level playing field’ between different technologies.Footnote ⁸¹ Evidently, a narrower definition also narrows down the scope of systems that are subjected to regulatory requirements and scrutiny. It is therefore no surprise that a regulatory battleground emerged around this definition, which continuously shifted in shape during the course of the negotiations on the AI Act between the European Parliament and Council.Footnote ⁸²

The two latter institutions decided to eliminate the Commission’s annex with AI approaches and techniques, and to solely include a (slightly altered) definition in the regulation’s main text. In the final version of the AI Act, which entered into force on 1 August 2024, Article 3(1) reads: “‘AI system’ means a machine-based system designed to operate with varying levels of autonomy, that may exhibit adaptiveness after deployment and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments”. This definition has the advantage of being closer to the one proposed by the OECDFootnote ⁸³ and is therefore seemingly more reflective of the slowly emerging global definitional consensus. It also lists the ‘actions’ that AI systems can be programmed to undertake based on the input they are provided, such as making recommendations and predictions, or taking decisions, and it explicitly includes the concept of ‘autonomy’.

I will come back to the AI Act’s definition of AI in Section 5.4 where I analyse the regulation in more detail. At this stage, suffice it to say that this definition may come across as rather broad, but that the actual regulatory scope of the AI Act is significantly narrowed down by the regulation’s Recital 12, which distinguishes it from ‘simpler traditional software systems or programming approaches’ and from ‘systems that are based on the rules defined solely by natural persons to automatically execute operations’, neither of which should fall under the AI Act’s scope. Furthermore, its scope is also narrowed down by the specific AI-applications listed under its substantive provisions, since not all systems that fall under the AI definition are also subjected to new regulatory requirements.

Importantly, this book is not interested in the definitional conundrums of AI, but rather in the adverse effects that the use of algorithmic regulation – by virtue of the scaled and automated decision-making it enables – may have on the rule of law and its principles, regardless of the underlying technique used for this purpose. Data-driven approaches are often considered to be more problematic than knowledge-driven approaches, given their reliance on potentially biased or erroneous datasets, their dynamic and evolving nature, their greater opacity, and their relative novelty – which also means less time has passed to develop appropriate mitigation measures for their risks. I could hence have opted to limit my investigation to the way in which public authorities rely on data-driven systems only, and leave aside the use of knowledge-driven systems. However, as noted above – the distinction between both is not firm, since systems can rely on both approaches, and on intermediary probabilistic approaches. Furthermore, as already hinted at, and as I will discuss further below, knowledge-driven approaches, too, can pose significant risks, even those that are only based on very crude rules. While they typically benefit from a higher degree of intelligibility and a greater level of control and predictability, their use can still cause substantial societal harm. Moreover, public authorities tend to rely both on knowledge- and data-driven approaches to automate administrative acts.

Consequently, in this book I will focus on algorithmic systems in general. Where the distinction between different approaches is relevant (for instance in light of a specific risk or drawback arising therefrom) I will mention whether the system is primarily knowledge- or data-driven. Rather than trying to set boundaries around which techniques are sufficiently ‘intelligent’ to fall under the AI umbrella, I will limit my use of the term AI altogether by reserving it only for my discussion of the AI Act in Section 5.4, given the Act’s heavy reliance on this term. In the remainder of the book, I will instead examine the deployment of algorithmic regulation by public authorities to inform or take administrative acts, regardless of the underlying approach or technique they rely on. Collectively, I will refer to the use of such systems as algorithmic regulation. With these technical aspects in mind, let me now consider the societal aspects of those systems.

2.2.1 Algorithmic Systems as Socio-technical Infrastructure

The intuition that the design, development and use of a technology is not a mere technical matter, but also a normative and political matter, is far from new. Already in 1964, Lewis Mumford juxtaposed democratic technics with authoritarian technics. He argued that certain technologies, and particularly systems used for automation, lend themselves more easily to authoritarian tendencies in society given their scale, their centralised and controlling nature, and their focus on efficiency and speed instead of human individuality.Footnote ⁸⁶ Over the past decades, scholarship on the relationship between technology and society has been burgeoning, producing theoretical frameworks that draw on a variety of disciplines.Footnote ⁸⁷

A chief insight of this scholarship, albeit still controversial for some,Footnote ⁸⁸ concerns the fact that technology is not neutralFootnote ⁸⁹ but that it embodies normative and political choices on the part of its designers, developers and users, which shape the technology’s affordances.Footnote ⁹⁰ These choices can not only be judged by the extent to which the technology is effective in reaching its goals, but also by how it can alter or entrench power structures and impact human relationships. By virtue of being part of a society, technical systems are inherently interwoven therein and influenced thereby. Yet, once created, they exert an influence on society in turn. This influence can have various modalities. It can be deliberate or accidental, known or obscure, significant or minor, positive or negative (and is often mixed), and it can manifest itself in the short or in the longer term. To expose this influence, both the social forces behind the technology and the technology’s particular characteristics should be assessed,Footnote ⁹¹ as each of these elements can contribute thereto.

A well-known example of a deliberate way in which artifacts have been designed to exert a particular influence on society, concerns Robert Moses’ low-hanging bridges on Long Island.Footnote ⁹² Their height prevented public buses – and the people of colour or lower-income passengers they typically carried – from accessing public beaches that were primarily intended for the recreation of white middle- and upper-class citizens, who could pass the low-hanging bridges with their cars.Footnote ⁹³ Another example, which is not deliberate yet nevertheless highly worrisome, concerns the way in which cars are designed by using crash-test dummies that reflect the average height and size of men, not taking into account the anatomy of women. This renders it 47 per cent more likely for a woman that is involved in a car crash to be seriously injured, and 71 per cent more likely for her to be moderately injured, than for a man.Footnote ⁹⁴

As I will elaborate in the next sections, this holds true not only for physical artefacts, but also for digital technologies – including algorithmic systems. An example of how the combined use of physical and digital technology can deliberately co-contribute to a societal impact, is found in Flanders. Since February 2021, Flemish municipalities have the competence to penalise excessive speed themselves by means of a fine, so as to increase the enforcement of speed limits. However, for smaller towns, the costs to set up automatised speed traps to measure cars’ driving speed are prohibitively high. Enter private companies. Through a contractual arrangement with several towns, a number of companies agreed to bear the installation costs of the automatised speed traps in exchange for a lumpsum of €24 for every fine the town issued. Of course, such business model implies that the more fines are issued, the more profit is made.Footnote ⁹⁵ The companies therefore contractually stipulated with the towns that physical infrastructures meant to discourage drivers from driving fast, such as speed bumps and chicanes, as well as priority arrangements for cyclists, are to be removed and not to be reintroduced for at least six years.Footnote ⁹⁶ After media reports and public outrage, the Home Office Minister annulled the contracts, citing the adverse impact on ‘the public interest’ and the importance of the town’s autonomy as a justification for the annulment.Footnote ⁹⁷

This example illustrates in a disconcertingly clear way how physical and digital infrastructure can be used to optimise one goal (e.g. private profits) over another (e.g. road safety in the public interest). In other words: the design, development, use, interplay, as well as the governance and ownership of infrastructure is not merely of technical but also of societal relevance. Moreover, it shows that the societal embeddedness of technology also translates into physical power by those who design, develop and deploy the system, over those who are subjected to the system. In sum, technological normativity shapes possibilities for action,Footnote ⁹⁸ both for those who actively make use of the system, and for those who are subjected to the system.

I described above how algorithms essentially consist of a given input, instructions and output. Moreover, they depend on the physical infrastructure that enables their functioning and, in particular as regards data-driven systems, the data they are fed. All of these individual elements, including the designers and owners behind it, therefore influence the system’s impact on those subjected to its use. Constitutive of these elements are human choices, such as the choice of what to optimise the system for, what data to use and how to categorise and label it, what elements to dismiss, how to translate knowledge to code, how to use and present the system’s output, and how to communicate the system’s capabilities. These choices also pertain to which programming language to use – bearing in mind that the affordance of language we code in also shapes how we think, for instance in terms of coding solutions – and at the meta-level, how to design and develop programming languages and the Application Programming Interfaces (APIs) through which they can be used.Footnote ⁹⁹

The implicit and explicit choices regarding all of these elements contribute to the system’s influence on society, rendering the human beings that make these choices – and that are positively or adversely affected thereby – part of the system’s broader socio-technical infrastructure.Footnote ¹⁰⁰ These choices are often invisible as they are not always rendered explicit, and tend to correspond to what Susan Star calls ‘the study of boring things’.Footnote ¹⁰¹ She gives the example of the physical infrastructure of our sewer system which may undoubtedly sound like a ‘boring thing’ to study and analyse, yet nevertheless constitutes an important element of modern cities’ utilities and reveals a lot of information about how we live and how society is organised. We barely notice this infrastructure’s existence, unless or until it breaks down and no longer functions as it should.Footnote ¹⁰² Star suggests that perhaps, “if we stopped thinking of computers as information highways and began to think of them more modestly as symbolic sewers”, the realm of the functional and value-laden choices behind them might become more visible.Footnote ¹⁰³

What Star notes about IT systems in general also applies to algorithmic systems. In fact, the term system already implies this infrastructural nature, typically referring to a set of things that work interdependently as parts of a broader mechanism or an interconnected network that forms a united whole.Footnote ¹⁰⁴ This notion hence emphasises that algorithmic systems are not isolated entities, but embedded in an overarching system, and part of a broader socio-technical infrastructure.Footnote ¹⁰⁵

To conclude, in order to understand the societal impact of algorithmic systems beyond the technical elements described above, it is useful to render visible at least part of the human choices (and hence the power structures) that underlie their design, development and use. Considered holistically, algorithmic systems are more than just the sum of their individual components, as the interplay of these components engenders societal effects.Footnote ¹⁰⁶ When analysing how algorithmic regulation is used in the public sector, I will therefore keep this broader dimension in mind. In the remainder of this section, I discuss some of the consequences of the societal embeddedness of algorithmic systems.

2.2.2 Risk of Human Error

When people apologise for a blunder by stating that they are ‘only human’, this seems to imply that non-human machines could be programmed without this ‘human’ tendency of making mistakes. Indeed, one of the ideas that drive the uptake of algorithmic systems in society is the fact that human errors and mistakes could be avoided or minimised, as a machine could carry out tasks without, for instance, becoming tired, drunk, ill or simply overlooking something. However, since the developers of algorithmic systems are ‘only human’ too, this is wishful thinking. Mistakes can occur throughout various instances of an algorithmic system’s lifecycle, for instance when gathering or selecting input, labelling or categorising data, translating knowledge and rules to code, designing the system’s optimisation function, interpreting the system’s output, or applying it in practice. Some mistakes will be easily noticeable and potentially also easily correctible. Others might be more difficult to spot and, especially if the system is used on a large scale, may cause significant damage before the mistake is rectified.

Consider the example of an algorithmic system to help allocate medical benefits, deployed by the Idaho Department of Health and Welfare. The system was set up to administer the state’s Development Disabilities Waiver programme, which establishes a personalised annual budget for individuals in need to contribute to their living costs. The personalised budget was calculated based on a number of datapoints, including a review of medical records, information gathered during a visit at the individual’s place, and an evaluation of the individual’s ‘Scale of Independent Behavior’.Footnote ¹⁰⁷ All of this information first had to be manually inputted into the system, which entails a first layer of risk for human error. The information can accidentally be inputted incorrectly, or it can be gathered in an erroneous manner (for instance in case the evaluator misinterprets someone’s ability to independently perform a task and hence attaches a wrong score to this criterion). Next, based on the inputted information, the system was programmed to make a budget decision that was supported by a data-driven model which aimed to predict the individual’s budget needs. The model in question was, however, deficient. In some instances, it decreased the budget of people whose medical needs had increased, indicating a structural flaw with serious consequences.Footnote ¹⁰⁸ Ultimately, the combination of various mistakes in the system led to budget reductions for a large number of individuals in need, without explanation, which forced those who were affected to start a class action to seek redress.Footnote ¹⁰⁹

Mistakes can also occur when data pulled from different databases loses its original contextual integrity and thereby no longer accurately reflects reality,Footnote ¹¹⁰ or when an organisation relies on data that reveals a spurious correlationFootnote ¹¹¹ yet erroneously interprets this as having a causal relationship. In the previous section, I gave the fictitious example of the fraud propensity of dog- versus cat-owners. Consider, however, the non-fictitious correlation that has been identified between US per capita cheese consumption and the number of people who die by becoming entangled in their bedsheets (correlation of 94.71 per cent between 2000 and 2009).Footnote ¹¹² One should hope that no public authority would believe that introducing cheese consumption restrictions will also lead to fewer deaths. More generally, mistakes can also arise when a reasoning process relies on the mistranslation of knowledge to code, or when the model or optimisation function of an algorithmic system is based on erroneous underlying assumptions about society that may not be easily verifiable – a point discussed further below.

Even basic algorithmic systems that do not rely on data-driven models or multiple datasets can contain errors which may lead to disastrous consequences. The example of the UK’s Post Office scandal, which involved reliance on a faulty software called Horizon to carry out transaction, accounting and stocktaking tasks, is a testimony to this.Footnote ¹¹³ Due to bugs in the software, the amounts reported sometimes indicated shortfalls of thousands of pounds, which the Post Office took at face value and – rather than considering that the system was flawed – used as a basis to prosecute postmasters, some of whom ended up in prison following convictions for false accounting and theft.Footnote ¹¹⁴ Accordingly, insufficient attention to errors in the software’s programming led to great human tragedy, which was only redressed two decades after the facts, when some of the falsely convicted postmasters already served many years in prison, lost their marriages or died.Footnote ¹¹⁵

In sum, just as mistakes can occur when human beings make decisions, so can mistakes occur when human beings design, develop and use algorithmic tools to inform or make decisions. Yet due to their systemic nature, the mistakes that occur in the context of algorithmic systems can lead to systemic types of harm, a point I will come back to in Chapter 4.

2.2.3 Risk of Bias

Another concern typically associated with human action and decision-making is ‘bias’. As human subjects, we are not isolated entities, but inherently influenced by our societal surroundings. Our education, upbringing, culture, social circles, language and societal roles shape our thoughts, opinions and values. While we can try to detach ourselves from these influences and seek an ‘objective’ point of view to consider facts and events, our inherent positionality in the world – including our motivations, ambitions and relationships – means that we inevitably look at the world from a certain angle. Our biases are not necessarily problematic, as they enable us to draw on our knowledge and experience to make sense of the world around us.Footnote ¹¹⁶ However, bias can also be fallacious and cloud our rational judgment, as well as having a discriminatory effect on the people we interact with or make decisions about.

One of the proclaimed benefits of algorithmic regulation is the fact that – unlike human beings – algorithmic systems do not suffer from human biases and achieve more objective decision-making, hence diminishing the risk of unjust discrimination.Footnote ¹¹⁷ After all, algorithmic systems lack moral agency and are hence not inherently motivated by their own self-interests or by prejudices in the way human beings are. Yet the myth that algorithmic systems generate purely objective outcomes has by now been busted, as it is widely acknowledged that they can reflect the prejudices and cognitive biases of their makers and users. Bias can also manifest itself through the data fed into the system.Footnote ¹¹⁸ If a facial recognition system is only trained on datasets showing pictures of white men, the system will not be able to recognise women and people of colour as accurately.Footnote ¹¹⁹ Accordingly, the outcomes of algorithmic systems, and the biases they reflect, hinge on the human decisions that lie at their origin. The fact that a broad range of societal domains, and hence also the knowledge about or data collected from these domains, are still plagued by inequalities and historic discriminations, renders the unchecked use of algorithmic systems liable to perpetuate and even exacerbate unjust bias – at scale.

Consider the by now well-known example of Amazon’s data-driven algorithmic system built to evaluate the CVs of incoming applicants. The choice of the system’s designers to rely on data from previous (successful) applicants, which were primarily white males, to train the algorithm’s model resulted in the algorithm assigning lower scores to incoming female CVs, and hence to discriminate applicants based on their gender.Footnote ¹²⁰ The outcome of this design choice was unlikely deliberate, yet had an adverse impact on numerous individuals, as the non-representative dataset led to unlawful discrimination.Footnote ¹²¹ One can also raise questions about the fact that most algorithmic voice assistants, such as Amazon’s Alexa, Apple’s Siri or Microsoft’s Cortana, typically have a female name and voice by default, thereby – consciously or not – risking to reinforce the societal paradigm of female subservience.Footnote ¹²²

Evidently, the risk of biased decision-making is also present without reliance on algorithmic regulation. It has therefore been argued by some that, since the decision-making process of algorithmic systems can at least be rendered transparent by ‘printing out and examining the program’ so as to ‘de-bias’ it (unlike the thought processes of biased public officials) algorithmic regulation remains the better option.Footnote ¹²³ Without taking a stance on which option is ‘best’, it must be pointed out that the prejudices embedded in algorithmic systems are not always visible or easily perceivable, for instance when they arise from reliance on proxies that are only indirectly discriminating, or from biased assumptions underpinning the system’s design choices or the selection of the data. Moreover, as I noted in Section 2.1, the decision-making process of algorithmic systems using deep-learning techniques are not intelligible, hence making it challenging to assess their potentially biased character. Finally, the significant difference in terms of the decision-making’s scale should be pointed out. Unlike a public official handling individual cases, in just a few seconds a biased algorithmic system can inform or adopt impactful administrative acts for thousands or even millions of citizens. Accordingly, those behind the proverbial wheel of the algorithmic system – its designers, developers and users – carry an important responsibility in this respect.

2.2.4 Opacity

One of the reasons it can be difficult to detect and mitigate the risk of errors or bias in the context of algorithmic systems is the fact that they can suffer from a transparency deficit. As pointed out in Section 2.1.3, certain data-driven systems are also denoted as ‘black-box’ systems, given the unexplainable nature of their inner decision-making processes.Footnote ¹²⁴ This problem is less manifest with knowledge-driven systems, which are typically more interpretable. The notion that algorithmic systems are opaque should thus be nuanced, especially when making general statements. Broadly speaking, a lack of transparency in the context of algorithmic systems can manifest itself in at least three non-exclusive ways, concerning, respectively: (1) the fact that an algorithmic system is used, (2) the way in which it is used and (3) the way in which it works.

First, public authorities that rely on algorithmic systems to inform or adopt administrative acts can omit to communicate, or choose to obscure, the fact that they deploy such systems. Given the system’s digital and hence potentially frictionless nature, individuals may not always be aware of the fact that they are being subjected to a (partially) automated decision-making process.Footnote ¹²⁵ This need not be problematic, especially if the decisions made by the system do not affect people in any meaningful way. If a Flemish-speaking public official uses an algorithmic translation tool to draft an email in French rather than asking a colleague for help, this is unlikely to affect anyone’s rights or interests, and unlikely needs to be rendered explicit. If, however, that public official outsources the decision of whether a citizen should receive social welfare benefits to an algorithmic system, the omission to communicate about this is a different matter, since this information is valuable to a citizen wanting to challenge the decision, particularly if she suspects error or bias. I will come back to this issue in subsequent chapters when analysing the impact of algorithmic regulation on the rule of law. Suffice it to state here that this type of opacity is human-chosen, and does not depend on the technical specificities of the system.Footnote ¹²⁶

Second, there can be opacity around the way in which an algorithmic system is used. This concerns, for instance, opacity around the type of data fed into the system, what the system was optimised for, which assumptions underly the system’s design, and how the outcomes of the system are used by the public authority that deploys it. These value-laden choices are rarely made transparent, which strengthens the idea that those choices do not exist. That idea is mistaken, yet it risks overshadowing the fact that the systems’ developers and deployers, who are usually already in a position of power, can retain power precisely through the non-contestable configuration of these systems.Footnote ¹²⁷

Third, there can be opacity around the algorithmic system’s inner workings. This is where the black-box problem truly comes in.Footnote ¹²⁸ For certain data-driven systems, it cannot be explained how their internal decision-making processes work, given the high level of complexity of the model’s functions. This renders it difficult to evaluate whether the processes followed by the system are based on robust and fair assumptions and comply with existing legislation.Footnote ¹²⁹ However, even when such inscrutable systems are used, transparency regarding the two other points above remains possible, and can already enhance the possibility to exercise oversight over the system. Information can in any case be provided about the way in which the system was designed and developed, what it was optimised for, which input was selected, which techniques were used, and how the system was tested for potential bias or inaccuracies. It should hence be ensured that this third type of opacity is not used as a pretence to also maintain human-chosen opacity to avoid public scrutiny.Footnote ¹³⁰ Moreover, to address the challenges of this third type of opacity, a research domain has developed itself around the notion of ‘explainable AI’, aimed at rendering such models more intelligible and at enabling developers and deployers of these systems to gain more insights into their internal processes.Footnote ¹³¹

Finally, it has been claimed that the choice between using a black box model and an interpretable model often comes down to making a trade-off between accuracy and interpretability since, despite their opaque nature, in some situations deep learning models can generate more accurate results.Footnote ¹³² However, this juxtaposition is not always accurate. Researchers have shown that, in some situations, a similar level of accuracy can also be reached with more interpretable models, and that this binary choice can hence be a false (and potentially misleading) dichotomy.Footnote ¹³³

2.2.5 Dependency on Data and Proxies

Algorithmic systems are highly dependent on data. For knowledge-driven systems, this dependency may be less pronounced, as the system’s functioning often relies on a pre-articulated model of reality that is codified into the system, yet this model ultimately also consists of machine-readable data based on which the system can reason. For data-driven systems this dependency is far more explicit, as they rely on (very large) datasets to derive a model in the first place. This also implies that the quality of the system’s output strongly hinges on the quality and accuracy of the data it is fed during the training, testing and use phase. Public authorities’ reliance on such systems in policy-making can be accompanied by the belief that, if only we have enough data about a certain phenomenon, we can use data-driven techniques to make normative decisions. This has also been referred to as dataism or “a belief in data as the enabler of a better, more effective, and objective society”.Footnote ¹³⁴ While the term dataism sounds almost spiritual, its definition actually corresponds rather well to so-called evidence-based approaches of public decision-making, grounded in ‘data’ and ‘science’,Footnote ¹³⁵ which often comes down to statistical analysis.

While this is of course a lofty goal, it is important to keep in mind that “data are not the facts themselves, but rather traces or marks of these facts”.Footnote ¹³⁶ Moreover, the facts that data represents “are about a phenomenon of interest, which is chosen by an observer from a number of different possibilities”,Footnote ¹³⁷ and hence cannot be said to reflect an Archimedean perspective. Likewise, the design and development of algorithmic models rely on “socially derived perceptions and understandings, not fixed universal, physical laws”.Footnote ¹³⁸ This implies the potential existence of a gap between the data and model on the one hand, and the reality that they represent on the other hand.Footnote ¹³⁹ Accordingly, when developing a model or drawing inferences from a dataset, this gap needs to be kept in mind. Moreover, even when this gap is minimal, and the dataset can be said to provide an adequate representation of reality, one must consider that a precise reproduction of reality may not always be desirable from a normative point of view. As explained above, in many societal domains, historic inequalities persist, which will also be reflected in the data collected about that domain, necessitating a cautious approach when using data about how things are to make predictions, recommendations or decisions about how things should be.

Unfortunately, the push for technocratic governance, which “assumes that complex societal problems can be deconstructed into neatly defined, structured and well-scoped problems that can be solved algorithmically and in which political realities play no role”,Footnote ¹⁴⁰ sometimes leads to a conflation of the positive and the normative. This can also be referred to as the is–ought fallacy.Footnote ¹⁴¹ Let me clarify with an example. Consider the use of an algorithmic system by a public authority to assist in hiring new officials at a Ministry.Footnote ¹⁴² It is one thing for this system to help assess whether a formal eligibility criterion for the job (for instance, having a specific degree) has been met. In this case, the system is not asked to help determine which grounds should render a job candidate eligible or right for the job, but merely to assess whether the already explicitly defined prerequisite is attained. It is, however, another thing for the system to help determine, for instance based on data of past hired public officials and how well they perform, which candidate should be interviewed.

The first algorithmic task can be said to belong to the positive realm. Someone already took a normative decision by deciding that one must have a specific degree to be eligible for the job, and the algorithm is merely deployed to peruse data to determine whether this is the case. The second task, however, belongs to the normative realm. The ministry here outsources the normative decision of what makes a public official ‘right for the job’ to the algorithmic system, which will in turn rely on the optimisation function and the data it was fed. The underlying normative grounds of the algorithmic recommendation may not be transparent here (perhaps the fact that they have much prior work experience or, to revisit the above example of Amazon,Footnote ¹⁴³ the fact that they are male), because it is not necessarily known which factors were flagged by the algorithmic system as relevant, yet it comes down to a normative decision nonetheless. As I discussed elsewhere,Footnote ¹⁴⁴ when humans seek to understand the best approach to deal with a problem, they already – often implicitly – have an idea of what the ideal outcome would be, based on their values and preferences. While algorithmic systems can help determine what the best course of action might be given a value X, they will never be able to determine the value that humans should strive for as such. Thinking otherwise is a naive approach at best, and leads to a dangerous discharge of responsibility at worst.

In addition, it should be borne in mind that not everything can easily or fully be captured by data. Complex social phenomena are not readily translatable to quantifiable metrics, and hence typically require the mediation of indicators, metrics and proxies.Footnote ¹⁴⁵ Since hardly any social phenomenon can be entirely reduced to metrics and indicators, something risks getting lost when translating information about such phenomena into a format that can be algorithmically computed and analysed.Footnote ¹⁴⁶ This intangible information deficit is also problematic in the context of algorithmic regulation, which is dependent on the quality of these indicators and proxies, and on the soundness of the assumptions underlying their use.Footnote ¹⁴⁷ Moreover, it places the official or authority responsible for the identification and collection of these indicators in a position of power (albeit typically a hidden one) since they can not only frame the problem that needs analysis, but also the norm or ideal that the system should seek to optimise, as well as indications that may constitute a deviation of the norm.

Consider the example of an algorithmic system used by a public authority to predict which individuals might commit fraud, and hence where it should focus its limited resources. While a person’s ‘propensity to commit fraud’ is difficult to quantify, there are some elements that will typically be used to provide an indication thereof, such as possible previous convictions of fraud, missing documents, unusual transactions or past complaints. These elements are more easily quantifiable and data-fiable and could thus be used as a proxy for ‘the propensity to commit fraud’, which is what the public authority is ultimately after. Crucially, however, the chosen proxies are not always reflective of the sought-after phenomenon: this will depend on the soundness of the assumptions made by the system’s developers.Footnote ¹⁴⁸ Furthermore, even if the indicators are well-chosen (by being relevant to analyse the phenomenon in question) they remain mere indicators: in practice, a citizen might very well have missing documents and make unusual transactions, yet nevertheless not in the slightest be prone to commit fraud. In short, the human condition cannot simply be reduced to numerical utility functions.Footnote ¹⁴⁹ It must therefore be ensured that such a reductionist approach to human beings – and to social phenomena more generally – does not ignore relevant and essential aspects of their humanity for the sake of speed and efficiency. This is especially important when the proxies and indicators are not based on actual behaviour or facts about the citizen in question, but on correlations that were identified with other individuals and groups.Footnote ¹⁵⁰

Finally, it must be noted that the data analysed by data-driven systems typically concerns information about individualisable human beings, in which case it is considered to be ‘personal data’.Footnote ¹⁵¹ The personalisation of administrative acts essentially relies on public authorities’ ability to collect data about citizens, and to use such data to profile them and draw inferences about their character, preferences or behaviour.Footnote ¹⁵² As the combination of different data-sets might yield new possibilities for analysis and insights, authorities are incentivised to both collect more data and keep it stored for a longer time, in case an opportunity arises to use it in another context.Footnote ¹⁵³ Clearly, this creates tensions with the fundamental rights to privacy and data protection,Footnote ¹⁵⁴ as the incentive to gather and analyse data from as many individuals as possible de facto leads to mass surveillance.Footnote ¹⁵⁵ As I discuss further below,Footnote ¹⁵⁶ the EU has legislation in place to protect individuals when their personal data is processed. However, this does not take away the fact that mass personal data-collection is taking place on a daily basis, which increases the asymmetry of information and hence also the asymmetry of power between government and citizen. It goes beyond this book to provide a thorough description of the impact of algorithmic regulation on the rights to privacy and data protection, yet it must be highlighted as an important factor when considering its societal aspects, since this impact considerably influences society’s shape and direction.Footnote ¹⁵⁷

2.2.6 Impact on Human Agency

Human agency denotes the capacity of human beings to act in a particular situation or environment, and is typically associated with the notion that they can do so intentionally and autonomously.Footnote ¹⁵⁸ Agency is traditionally also linked to responsibility since, generally speaking, one can only be deemed responsible for situations in which one has a certain level of agency.Footnote ¹⁵⁹ Since the very raison d’être of algorithmic systems is to take over tasks from human beings to make their lives ‘easier’ or ‘better’, their impact on human agency is a given, as humans essentially outsource their capacity of action to the system.Footnote ¹⁶⁰ As long as they have agency regarding the act of outsourcing, and regarding the consequences attached thereto, this need not be an issue. Conversely, the loss of such agency might also hamper their sense of responsibility.Footnote ¹⁶¹ The link between reliance on algorithmic systems and human responsibility – and in particular, the risk of a negative correlation between both – has extensively been discussed in scholarship.Footnote ¹⁶² I will not attempt to reproduce this discussion here, but will merely make some observations on how the deployment of algorithmic systems can impact the agency of the deployers of algorithmic systems in the public sector – an environment that is typically marked by hierarchical relationships.Footnote ¹⁶³

For this purpose, let me recall an experiment carried out by Stanley Milgram in the 1960s, to assess how obediently people act under ‘authority’, as this experiment likewise involved the mediation of a machine.Footnote ¹⁶⁴ Individuals were asked to administer increasingly high electric shocks to a volunteer, whenever that volunteer answered a question erroneously. While the shocks were fake, the grim results of his experiment indicated that there was a high rate of individuals who, sitting behind a machine and faced with the choice to obey to authority or refrain from hurting another person (even upon that person’s specific request to stop), all too often opted for the former.Footnote ¹⁶⁵ Milgram analysed the results of the – multiple variations of his – experiment and drew a number of conclusions that are also of relevance when considering the societal aspects of algorithmic systems.

First of all, Milgram observed that “distance, time and physical barriers neutralise the moral sense”.Footnote ¹⁶⁶ The further away the individual was from the volunteer subjected to the shock, the higher the obedience rate, despite the moral concern.Footnote ¹⁶⁷ In the context of algorithmic regulation, there is typically a physical distance between the person responsible for the system and the individual subjected thereto. Indeed, the system is meant to take over tasks from human beings, providing them with the possibility to monitor these tasks from a distance. However, as Milgram noted, this physical distance also facilitates an emotional distance from the individual subjected to the system, and hence from responsibility in case the individual is adversely impacted by the system’s outcomes.

Second, Milgram also explains that the individuals who participated in his experiment automatically adopted a number of internal mechanisms or ‘buffers’ to cope with the tension they faced in an ethically unclear situation, and to divest themselves of moral responsibility. One of those mechanisms concerns deference to a hierarchically higher authority that imposed the decision. In the context of algorithmic regulation, one can point to the fact that algorithmic systems often tend to have a sense of authority, due to their typically superior computational capabilities as well as their aura of objectivity, since they are based on technical rules and mathematical functions rather than on ‘biased’ human decisions.Footnote ¹⁶⁸ Reference can also be made to the known risk of automation bias, or the propensity for human beings to favour suggestions made by automated systems and to ignore contradictory information made without automation, even if correct.Footnote ¹⁶⁹ In a more banal sense, the fact that the public authority’s hierarchy decided that decisions should henceforth be informed or taken through algorithmic systems can also constitute an act of authority through which public officials can divest responsibility. It inevitably constraints their individual agency, thus potentially making them feel less responsible for the system’s problematic outcomes.

Third, Milgram describes “the tendency of the individual to become so absorbed in the narrow technical aspects of the task that he loses sight of its broader consequences”.Footnote ¹⁷⁰ The fact that the problematic act becomes fragmented (the official is no longer the sole person behind the act and is no longer directly faced with its consequences, but there is a chain of actions in between, mediated by technology) likewise facilitates the act’s execution.Footnote ¹⁷¹ Similarly, when it comes to algorithmic regulation, it can be noted that algorithmic systems are often composed of different components that interact with each other within a broader network or chain, which further alienates the system’s designer or deployer from its consequences and facilitates the evasion of responsibility. This has also been discussed in scholarship as the difficulty of the many hands problemFootnote ¹⁷² which, in the context of algorithmic systems, is only intensified by the opacity surrounding the different types of (interacting) conducts and systems.Footnote ¹⁷³

Finally, one can also point out Milgram’s warning that individuals tend to de-humanise the persons affected by their action by attributing impersonal qualities to those persons, and thereby making it easier to cope with the role of ‘hurter’. In the context of algorithmic regulation, the affected person is typically an unknown plurality of citizens, reduced to numeric abstractions and data-points in the system and hence also de-humanised in a more literal sense. All of these elements individually as well as collectively should be carefully taken into consideration when outsourcing impactful decisions about human beings to algorithmic systems. With this in mind, let us now explore how algorithmic regulation is relied upon by public authorities.

2.3.1 The Organisation of Public Authorities

The executive branch of power is responsible for the execution of laws, and its competences are often characterised as having a residual character, encompassing “anything that is neither legislative nor judicial”.Footnote ¹⁷⁵ Historically, the executive has been in charge of law enforcement, warfare and national security, as well as tax collection. Yet from the late nineteenth century onwards, with the rise of the welfare state, its competences significantly expanded, covering also the governance of welfare programmes, economic state interventions, and the regulation of societal risks – such as the risks raised by new technologies to public health or safety.

Increasingly, the general laws and policies adopted by the legislative branch of power required more sophisticated implementation techniques, whereby the executive branch became competent to adopt a growing range of administrative acts, both of individual and general application, often necessitating more specific knowledge or technical expertise.Footnote ¹⁷⁶ To exercise these more specialised functions, a large organisation of expert administrators was built, denoted as bureaucracy.Footnote ¹⁷⁷ As stated by Francesca Bignami, the nineteenth century can be described as one “of great optimism in the ability of public servants and the bureaucratic form of organisation to pursue the common good and advance the interests of society as a whole”.Footnote ¹⁷⁸ Public officials working at public authorities are (largely) unelected, but they in principle operate under the supervision and control of the government’s (largely) elected members.Footnote ¹⁷⁹

Bureaucracy has most famously been conceptualised by Max Weber in his work Economy and Society, as part of his broader reflections on modernity and the birth of the rational state.Footnote ¹⁸⁰ Weber’s conceptualisation of bureaucracy should be seen as an ‘ideal type’ or ‘analytical concept’ rather than as a factual description of how public authorities function.Footnote ¹⁸¹ He examined how the bureaucratic organisation of public administration was able to legitimise social control through legal-rational means, as opposed to more traditional or feudal forms of domination.Footnote ¹⁸² While not devoid of criticism,Footnote ¹⁸³ Weber’s work is still a cornerstone of public administration researchFootnote ¹⁸⁴ and proves helpful in discerning how public authorities operate today,Footnote ¹⁸⁵ since “bureaucracies continue to be pillars of public service provision”.Footnote ¹⁸⁶

Several features have been distinguished as characteristic of modern bureaucracy.Footnote ¹⁸⁷ While these features are grouped differently by various commentators,Footnote ¹⁸⁸ the most notable are:

• Efficiency: the optimisation of working methods and resources in order to achieve the desired goals in the most efficient manner;

• Legality: the legitimation of administrative action through formalised rules, standards and procedures;

• Rationality: the execution of tasks based on expert knowledge and reason, and the gathering of further knowledge to advance the organisation’s goals and create an institutionalised memory;Footnote ¹⁸⁹

• Objectivity and impartiality: the application of laws and procedures to all subjects equally, without privileging certain individuals or groups;

• Impersonality: the execution of tasks driven by impersonal official obligations rather than by personal interests or emotions – or sine ira et studio, as Weber put it;Footnote ¹⁹⁰

• Hierarchy: the establishment of relationships of super- and subordination, including the importance of loyalty and obedience to hierarchical authority;

• Specialisation: the organisation of tasks based on specific jurisdictional and competence areas, driven by a functional division of labour.

Note that, beyond this set of formal or procedural features, bureaucracy has also been described more substantively as “an expression of cultural values and a form of governing with intrinsic value”, whereby “administration is based on the rule of law, due process, codes of appropriate behavior, and a system of rationally debatable reasons”.Footnote ¹⁹¹ Public officials’ obedience to authority is therefore seen as the other side of the same rule of law-coin, as it legitimises their actions and ensures their accountability to society at large. Moreover, their duty to act in the public interestFootnote ¹⁹² renders them “guardians of constitutional principles, the law, and professional standards.Footnote ¹⁹³ Evidently, this duty also raises questions on the relationship between, on the one hand, the ‘public’ interests or values that public authorities should advance and, on the other hand, the ‘individual’ interests or values that private persons might hold dear – especially in a pluralistic society.Footnote ¹⁹⁴

Over the last century, the organisation of public authorities has known several waves of reform, inspired by neoclassical and private management ideas.Footnote ¹⁹⁵ This has led to the privatisation of services and an embrace of market competition, whereby citizens have become “a collection of customers with a commercial rather than a political relationship to government, and legitimacy is based on substantive performance and cost efficiency”.Footnote ¹⁹⁶ Despite these reforms, the core features of bureaucracy were retained, and most particularly the focus on procedural rationality and the efficient execution of tasks.Footnote ¹⁹⁷ Indeed, as noted by Galligan, “the premise of efficient administration in implementing the policies of government remains the dominant paradigm”.Footnote ¹⁹⁸ In what follows, I will therefore be using the notion of bureaucracy and public administration interchangeably.

2.3.2 The Pitfalls of Bureaucracy

A lot has been written about the downsides of bureaucracy, and it goes well beyond the scope of this book to reconstruct those rich debates here. I therefore limit myself here to a brief analysis of how some of the core features of public administration can run counter to the normative ideals of liberal democracy, as this will be relevant for my subsequent examination of the impact of algorithmic regulation on the rule of law. Such analysis also enables a better understanding of the challenges that are inherent to bureaucracy, as opposed to the challenges that are raised or exacerbated by algorithmic regulation in particular.

Despite the benefits that Weber identified with the bureaucratic form of organisation, he also acknowledged it might pose certain threats to substantive values such as individual freedom, and the risk of a concentration of power.Footnote ¹⁹⁹ For instance, in related work,Footnote ²⁰⁰ he noted that an overly instrumental conception of rationality can result in a stahlhartes Gehäuse or iron cage in which people are subjected to rules and procedures without ensuring that the substantive values these rules are supposed to serve are actually achieved.Footnote ²⁰¹

This echoes an oft-made distinction between two types of rationality, namely procedural and substantive rationality.Footnote ²⁰² As explained by Muellerleile and Robertson, substantive rationality is the value-laden framework that people draw on to determine their actions in a particular situation, typically based on their individual ethical perspective.Footnote ²⁰³ Conversely, procedural rationality draws on scientific and economic calculation, and is based on a depersonalised set of codified rules, laws and regulations, which is typical in bureaucratic organisations. When the execution of actions becomes a mere instrumental undertaking, without regard for their practical impact and the extent to which they advance the goals and values they should secure, procedural rationality can undermine substantive rationality, to the detriment of the values intrinsic thereto.

This risk has been raised by various scholars. In particular, it has been pointed out that the impersonal nature of bureaucracy (or the ‘rule of nobody’, as bureaucracy was called by Hannah ArendtFootnote ²⁰⁴) can lead to a ‘disinterestedness’ that erodes public officials’ sense of responsibility and morality, which can turn into amoralityFootnote ²⁰⁵ (or worse, immorality). One of the most fervent critiques of rational-legal bureaucracy has been formulated by Zygmunt Bauman in his Modernity and the Holocaust.Footnote ²⁰⁶ According to Bauman, the features of modern bureaucracy, particularly the emphasis on efficiency and procedural rationality, and the specialisation of tasks which risks obliterating the overall result of one’s actions, can undermine the possibility for moral action by individuals working for the state.Footnote ²⁰⁷ Bauman called attention to the ‘important role’ that bureaucratic culture played in the Holocaust (also commented on by Hannah Arendt in her characterisation of Eichmann as an amoral bureaucratFootnote ²⁰⁸), and characterised modern bureaucracy as ‘a moral sleeping pill’.Footnote ²⁰⁹ As summarised by du Gay, “for Bauman, the essence of bureaucratic structure and process’ is the dissociation of ‘instrumental rational criteria’ from ‘moral evaluations’ of the ends they serve”,Footnote ²¹⁰ leading to the threat of moral discharge and the dehumanisationFootnote ²¹¹ of those subjected to the impersonally applied procedures.Footnote ²¹²

It has also been argued that, precisely because of its emphasis on procedural rationality and rule-following, the organisation of public administration carries an inherent risk of authoritarianism, and a neglect of the rights of individuals.Footnote ²¹³ As, for instance, noted by Galligan, “administrative bureaucracies are naturally governed by procedural rigidity and a disregard for individualized differences; efficiency and self-interest prevail over fairness, and secrecy militates against explanation and justification”.Footnote ²¹⁴ Accordingly, public administration can be considered as more than a mere ‘neutral’ instrument to carry out public policies. Instead, by virtue of its inherent features, it can rather be seen as a mode of organisation that can aggravate potential authoritarian and illiberal elements already contained in the laws it implements.Footnote ²¹⁵

Others have taken up bureaucracy’s defence, arguing that bureaucratic objectivity “entails a trained capacity to treat people as ‘individual’ cases, i.e. apart from status and ascription, so that the partialities of patronage and the dangers of corruption might be avoided”.Footnote ²¹⁶ Moreover, one could also consider bureaucracy’s “instituted blindness to inherited differences in status and prestige” as “a source of democratic equalization”.Footnote ²¹⁷ Du Gay also nuances the above, by pointing out that the distinction between formal and substantive rationality has not been deployed by Weber and that it may be artificial since, in practice, a distinction between ‘means versus ends’ is not always straightforward.Footnote ²¹⁸ Rules can, after all, be designed with the specific aim of protecting and enhancing substantive rights and values, and can contribute to the institutionalisation of public accountability.

Furthermore, it has also been argued that an over-emphasis on substantive rationality might overlook the fact that there exists a plurality of (often conflicting) values.Footnote ²¹⁹ Bureaucracy can allow the state to remain ‘neutral’ and maintain its legitimacy in a pluralistic society, precisely because it focuses on procedural rationality and efficiency, and thereby enables “the expression and protection of a broader range of conflicting values held to be important by human beings”.Footnote ²²⁰ While the theme of state neutrality has been widely debated,Footnote ²²¹ it should be noted that Weber considered value conflict to be an inherent feature of modernity,Footnote ²²² and he was hopeful that democratic institutions, through an agonistic political process, could address these conflicts. Consequently, he believed in the power of democratic control to mitigate the pitfalls of the ‘iron cage’, and insisted on the need to make space for politics and democratic oversight amidst and alongside bureaucracy.Footnote ²²³

Interestingly, the two views I just outlined on bureaucracy’s merits and pitfalls foreshadow an important tension that is inherently part of society’s reliance on legal rules and procedures to exert social control, which will be closely examined in this book. On the one hand, we require abstract rules of general application, to ensure the law’s impartiality (‘procedural rationality’). On the other hand, we also require that the application of those general rules results in individual justice,Footnote ²²⁴ in light of the concrete particularities of each person and situation, and the way in which different values and interests matter to them (‘substantive rationality’). Based on the discussion above, it can be concluded that a combination of both is warranted, albeit not always evident, and that political oversight based on a democratic process is an important mechanism to foster this.

In addition to external oversight, there are other mechanisms that can curb the risks evoked above. One such mechanism is ensuring the ‘internal morality’ of public authorities. Notably, “Weber did not see bureaucrats, particularly civil servants, simply as mindless automatons but believed them perfectly capable of ethically principled conduct within their own proper sphere of action”.Footnote ²²⁵ He also emphasised their ethical duties and their need for ‘moral discipline’.Footnote ²²⁶ Indeed, normativity can also play a role inside public authorities, by incorporating substantive principles within the organisation of bureaucracy and requiring public officials to adhere thereto.Footnote ²²⁷ As I shall discuss extensively in Chapter 3, these substantive principles can be distilled from the more general commitment to the rule of law, and from other core tenets of liberal democracy.Footnote ²²⁸ Another mechanism concerns administrative discretion, which is prevalent in public authorities’ day-to-day organisation and can be used to soften the rigidity of procedural rules.Footnote ²²⁹ Since discretion will be an important theme in this book, especially when discussing its relationship with algorithmic regulation, let me dissect this concept more closely in the next section.

2.3.3 Administrative Acts in between Rules and Discretion

In essence, discretion provides public authorities with a certain level of autonomy when making a decision amongst a variety of options, based on their assessment and judgment.Footnote ²³⁰ It can be defined as “a power which leaves an administrative authority some degree of latitude as regards the decision to be taken, enabling it to choose from among several legally admissible decisions the one which it finds to be the most appropriate”.Footnote ²³¹ When exercising discretion, public authorities still do so within the confines of the law, which led Dworkin to state that “discretion, like the hole in a doughnut, does not exist except as an area left open by a surrounding belt of restriction. It is therefore a relative concept. It always makes sense to ask, ‘Discretion under which standards?””Footnote ²³² Unsurprisingly, the expansion of the modern state and the increase in government functions, including the increase of legislation that needs to be implemented and applied by the executive, also led to an expansion of discretionary power.

Discretion can arise in different circumstances. The most common one relates to the tension I indicated above, between the general nature of legal rules on the one hand, and the need for their individual application on the other hand. When determining a desirable policy outcome or goal, it is impossible for the legislator to determine in advance all the particular situations that may arise, and to provide precise instructions to the executive as to how it should act in each of these situations. Public authorities therefore typically have some discretion as to how precisely they will implement and enforce generally applicable rules and policies. In this sense, discretion is a side-effect of organising social control based on a system of legal rules.

However, discretion is more than a side-effect, since it can also be seen as “a positive way of conferring powers where it is important that officials have more freedom as to the way they are to be exercised than a detailed set of rules might allow”.Footnote ²³³ In some situations, applying a general rule to advance a just cause, without the possibility of tempering its concrete impact in potentially unforeseen or unanticipated circumstances, can actually lead to injustice. On the one hand, securing public services for millions of citizens naturally requires a certain level of organisation and systematisation. On the other hand, any system will also inevitably overlook the uniqueness of the individual cases that it systematises.

This dilemma was also discussed by Emmanuel Levinas, famous for his emphasis on the primacy of ethics in human relationships and the responsibility we have for the ‘other’.Footnote ²³⁴ He considered it inevitable that any system seeking to ensure justice for the many can become dehumanising precisely by approaching individuals in a general rather than an individualised manner.Footnote ²³⁵ Indeed, within such a system, “the other is no longer the unique person offering himself to the compassion of my responsibility, but an individual within a logical order or a citizen of a state in which institutions, general laws, and judges are both possible and necessary”.Footnote ²³⁶ This leads to the danger that the other is “extinguished in the system of universal laws”.Footnote ²³⁷ While this danger does not imply that all systematisation of public services should be rejected, it does mean that the system needs to be continually corrected and perfected against its own harshness.Footnote ²³⁸ This permanent correction of the system, however, cannot be expected to come from another system. Instead, Levinas believes it can be found in acts of ‘little goodness’, which he juxtaposes against a systematised ‘Goodness’.Footnote ²³⁹ The little goodness is “a goodness outside of every system, every religion, every social organization”.Footnote ²⁴⁰ It is “a correction of the impersonality of a system that both tries to realize justice but also disregards the invisible tears of people who, despite all their efforts, fall outside of this whole”.Footnote ²⁴¹

In the context of public authorities’ application of the law, it is this little goodness, this case-by-case correction to ensure justice in individual situations where the application of general laws might lead to injustice, that can be likened to the role of discretion. Indeed, discretion enables public officials to exercise a ‘little goodness’ that softens the hard edges of a generalised legal system. This is precisely why hardship clauses are sometimes incorporated in government policies, allowing public authorities to provide relief from the law’s application or to deviate from its implementation based on the specific individual circumstances.Footnote ²⁴² In this sense, discretion can counter the excesses of procedural rationality, by ensuring that rules are not applied overly rigidly, but with due regard to the ends they aim to serve, based on substantive rationality. It hence provides space to make trade-offs between different interests and values in a particular situation, without the need for the legislator to always anticipate those situations.

Discretion can arise due to other factors too. Certain risks that the legislator seeks to prevent or mitigate arise in situations of inherent complexity or uncertainty, requiring more comprehensive assessments by experts to identify the best course and timing of action. For instance, in the area of technology regulation, the legislator typically sets out a legal framework with general safety norms that should be met, but relies on public authorities and their risk analyses to develop more specific standards, procedures and guidelines. Furthermore, the fact that public authorities inevitably have limited resources at their disposal means they must optimise these resources in a way that allows them to reach their goals in the most efficient manner by setting certain priorities. Discretion has therefore been “long identified as necessary for administrative authorities to operate effectively within all modern legal systems”.Footnote ²⁴³

Note that the role of discretion applies not only to the implementation of legislation at the national level, but also to the implementation of European law. Indeed, when the EU adopts new legislation, certain legal provisions may explicitly or implicitly enable Member Sates’ public authorities to exercise some discretion as regards the ways in which this legislation will be implemented at the national level. In this context, discretion hence also serves as a tool that allows individual Member States to make trade-offs between different values and interests in line with their national traditions, as long as they remain within the confines set out by the EU legislator and by primary EU law.Footnote ²⁴⁴

Depending on the context in which public power is exercised, the executive’s discretion can be more or less extensive. Let me concretise this with an example. Consider the legal rule in the area of Belgian migration law, which grants a migrant in Belgium the possibility to apply for a residence permit “if exceptional circumstances justify the submission of this application in Belgium rather than abroad”.Footnote ²⁴⁵ In this case, discretion is rather vast: the law itself does not define a list of ‘exceptional’ circumstances, so it is up to the officials implementing the law to interpret this term, and to decide whether such circumstances are present. The relevant public authority (the Immigration Office, acting under the supervision of the ‘political’ executive) typically issues policy guidelines that set out which procedures will be followed in the implementation of this rule. On the government’s website, one can, for instance, read that “a long stay in Belgium, or integration into Belgian society, is not, in itself, an exceptional circumstance justifying an application for a residence permit in Belgium”.Footnote ²⁴⁶ This statement or policy reflects the executive’s discretionary choice of implementing the law in this way rather than in another, as it is not an explicit part of the text adopted by the legislator. Had a more migration-friendly government been elected, this policy might, for instance, have been different.

Contrast this with the Belgian legal rule that for every child, regardless of income level, parents receive a standard sum of childcare benefits.Footnote ²⁴⁷ In this situation, the general rule to be applied is rather clear, and discretion is near inexistent: if an individual has a child, it is clear that the public official in charge needs to allocate that person the standard sum. However, even in this context, discretion is not entirely absent. One can still conceive that the responsible public authority may adopt guidelines or procedures that set out which type of evidence is accepted to prove the existence of a child, and which process must be followed to apply for such benefits if they are not allocated automatically. Accordingly, even within the scope of a single administrative act – the allocation of childcare benefits – public authorities can have discretionary competences over some of the act’s aspects, and bound competences over other aspects.Footnote ²⁴⁸

As regards the adoption of administrative decisions, Galligan identifies three elements to the decision-making process, each of which allows for the exercise of discretion: (1) finding facts, (2) setting standards and (3) applying the standards to the facts.Footnote ²⁴⁹ Discretion is often associated with the second element (requiring the interpretation of vague or ambiguous legislation, or the creation of more specific standards based on broad legislation), and third element (requiring an element of judgment and assessment, even if the way in which the standard must be interpreted is clear). However, even the first element already implies some discretion (e.g. how will evidence regarding the facts be gathered and assessed?). Accordingly, drawing on the example above, discretion not only arises as regards the final decision (e.g. the allocation of childcare benefits) but also in various intermediary steps (e.g. how can the existence of a child be proven, how is the application process for benefits organised, how is the allocation calculated, and so on).

To summarise, public authorities and officials exercise their functions and take administrative acts based on a set of rules, yet they always have a varying level of discretion at their disposal which enables them to carry out their tasks in a way that is, ideally, both efficient and just. Of course, this does not mean that discretion is always exercised in a sound manner. The autonomy afforded by discretion also opens the door for deviations of the law and potential abuses. This is why, in line with the principles of the rule of law that I will discuss in Chapter 3, public authorities typically also adopt guidelines or procedures that set out how public officials should exercise discretion, thereby enhancing the predictability and consistency of the law’s application across various departments within the organisation. Moreover, democratic oversight and judicial review play an important role in ensuring that discretion – whether to apply, interpret or deviate from the general law – is used commensurably with the rule of law and other constitutional values and principles.

With this conception in mind of how public authorities carry out their functions and adopt administrative acts, in between rules and discretion, let me now examine the role that algorithmic systems can play in this context.

2.3.4 From Bureaucracy to Algocracy

The processing of information and the adoption of administrative acts based on such information is a core task of public authorities. As discussed above, the aspiration to do so efficiently, objectively and rationally already underpinned the nineteenth-century model of administrative bureaucracy and remained an important aim of public administrations ever since.Footnote ²⁵⁰ It is therefore no surprise that, as soon as affordable, algorithmic regulation became part and parcel of public authorities’ working methods, aimed at rendering their information processing activities more efficient.

In this regard, Zuurmond highlights that “bureaucracy and informatisation seem to go hand in hand”.Footnote ²⁵¹ Indeed, bureaucracy relies on the execution of administrative tasks based on expert knowledge and information, which in turn requires the collection of data – including data about legal subjects. According to Peeters and Widlak: “as state tasks expanded, especially in welfare states, so did the number of registrations and their importance. Knowing your citizens has never been more important as when you try to decide who is eligible to student grants, social security, health care, social housing, or pensions”.Footnote ²⁵² Besides the collection of information, the need for the speedy processing of such information also set in motion a broader process of digitalisation. When public authorities started to deploy computer systems in the twentieth century, they realised that the information they sought to process first had to be converted from an analogue format to computer-readable code (‘digitisation’). Only then, analytical and decision-making processes could be transformed to the digital realm too (‘digitalisation’).

Gradually, the image of bureaucrats sitting behind a stuffy pile of papers thus transformed into an image of bureaucrats sitting behind large computer screens. In early 2002, Bovens and Zouridis pointed out that “window clerks are being replaced by Web sites, and advanced information and expert systems are taking over the role of case managers and adjudicating officers. Instead of noisy, disorganized decision-making factories populated by fickle officials, many of these executive agencies are fast becoming quiet information refineries, in which nearly all decisions are pre-programmed by algorithms and digital decision trees”.Footnote ²⁵³ The uptake of algorithmic regulation in public administration is hence nothing new. Yet over time, public authorities “experienced several leaps of technological innovation”,Footnote ²⁵⁴ and the systems they relied upon became ever more sophisticated. Today, algorithmic regulation is not only based on basic decision trees, but also on more complex knowledge-driven systems and increasingly on data-driven systems too. Accordingly, a distinctive growth can be perceived both as regards the scale on which algorithmic regulation is used in the public sector, as well as the importance and impact of the acts that are being automated.Footnote ²⁵⁵

Evidently, the introduction of algorithmic regulation also impacted the way in which public officials take decisions. Instead of making judgment calls regarding the application of the law to concrete situations, their role became increasingly focused on filling in electronic forms and templates, based on which an algorithm can compute certain outcomes. Bovens and Zouridis conceptualised this as a turn from ‘street-level’ bureaucracy (a term coined by Michael Lipsky to denote public officials interacting with and taking decisions about citizens)Footnote ²⁵⁶ to ‘screen-level’ bureaucracy. As the uptake of algorithmic regulation increased, and the computers assigned to public officials became a networked digital infrastructure that connects databases and processes across public authorities, ‘screen-level’ bureaucracy developed further into ‘system-level’ bureaucracy.Footnote ²⁵⁷ The advent of the internet and the Internet of Things also played an important role in this regard, as it progressively enabled citizens to directly provide information to public authorities (knowingly or not), thereby facilitating the collection of their data.

Today, reliance on algorithmic regulation can hence be called systemic, as it underpins the functioning of public authorities at large. Algorithmic systems can be used for a myriad of functions within public authorities, from the automated translation of text and the filtering of incoming mail, to the formulation of replies to citizen questions and the adoption of administrative acts. This does not mean that the adoption of administrative acts is necessarily entirely automatised. As noted in Section 2.1.1, algorithmic systems can be used for the adoption of administrative acts (decision-making sensu stricto), yet in most public authorities these systems are still primarily used to inform or recommend administrative acts. This also implies that, today, the collection, processing, analysis and assessment of data by public authorities, which lies at the heart of their decision-making processes, is primarily carried out by algorithms.

Initially, it was thought that the embrace of algorithmic systems would alter the nature of bureaucracy, or even lead to its collapse by giving rise to a radically new ‘post-bureaucratic order’.Footnote ²⁵⁸ However, as already hinted at above and as noted by various scholars, “rather than less bureaucracy, we seem to experience its propagation and expansion at every turn”.Footnote ²⁵⁹ Instead of being dissipated, the ‘original’ bureaucracy merely turned into an ‘algorithmic’ or ‘digital’ bureaucracy. Muellerleile and Robertson note that “the digital bureaucracy is a world of data in motion, given direction and shape by new kinds of digital infrastructures – from codes to algorithms to platforms, whose digital footprint replaces the material archive, and whose experts are the new data scientists”.Footnote ²⁶⁰ Lorenz and others even speak of an ‘algocracy’, arguing that, “whereas the bureaucracy denotes the exercise of power through the office, the algocracy shows that power is exercised through algorithms”.Footnote ²⁶¹

The adoption of algorithmic systems by public authorities hence did not simply come down to the introduction of a new tool amongst others to rationalise public decision-making. It also fundamentally altered the organisation of public administrationsFootnote ²⁶² and brought along several side-effects. In Chapter 4, I will carry out an extensive analysis of the impact of those effects on one particular societal interest, namely the rule of law. In this section, already foreshadowing this analysis, I will only mention four general consequences of the uptake of algorithmic systems in the public sector.

First, one can observe that public authorities have increasingly formalised aspects of their decision-making processes, as such formalisation is necessary to express information and rules through computer-readable binary code.Footnote ²⁶³ As Bovens and Zouridis point out, “a conditionally programmed legal framework will lend itself much easier to ICT applications than a goal-oriented legal framework”.Footnote ²⁶⁴

Second, this algorithmisation also led to a reduction of discretion at the level of individual public officials. As previously described, algorithms rely on input and instructions to deliver certain outcomes, and are able to do so at scale. To benefit from these efficiencies of scale, public authorities hence seek to routinise and centralise processes.Footnote ²⁶⁵ Accordingly, “many decisions are no longer made at the street level by the worker handling the case; rather, they have been programmed into the computer in the design of the software”.Footnote ²⁶⁶ Indeed, decisions are increasingly guided by algorithmic systems and databases,Footnote ²⁶⁷ whereby public officials are often “no longer involved in handling individual cases, but direct their focus toward system development and maintenance, toward optimizing information processes, and toward creating links between systems in various organizations”.Footnote ²⁶⁸ This does not mean that administrative discretion has dissolved. Instead, the introduction of algorithmic regulation and the routinisation of decisions has pushed this discretion higher up the value chain, to the level of the designers and developers of the algorithmic systems. I will come back to this point in the following chapters, given the implications this has on the rule of law.Footnote ²⁶⁹

A third effect of the public sector’s algorithmisation is the extensive process of quantification and datafication this brought along, particularly given the more recent uptake of data-driven systems.Footnote ²⁷⁰ Today, public authorities have more data than ever at their disposal to inform their policy and decision-making processes. Furthermore, the computational abilities of algorithmic systems allow them to process and analyse such data at unprecedented speed.Footnote ²⁷¹ This also reinforces the bureaucratic tendency towards rationalisation, under the heading of ‘evidence-based’ decision-making. As Lorenz and others describe, the introduction of algorithmic systems “enables government organizations to quantify the uncertainty inherent in decision-making processes on the basis of data analysis by expressing it as probability and to thereby further rationalize this process: even though there is no full certainty about a situation, a more rational choice can be made based on probabilities”.Footnote ²⁷²

Finally, along with this rationalisation and datafication one can also discern a thoroughgoing formalisation of citizen registration and classification.Footnote ²⁷³ Algorithmic regulation typically requires that natural and legal persons are classified according to traits that are relevant for the regulation’s application, based on the various data-points that have been collected about them. This classification can be inserted into the system manually, but it can also be generated through a data-driven system programmed to identify patterns and, on that basis, classify citizens into various categories, and evaluate or score them.Footnote ²⁷⁴ Since these classifications are used to inform administrative acts, their contours are not without substantive consequences.

As noted by Peeters and Widlak, “classifications are by their very nature contested, because they are abstractions and simplifications of a complex social reality that highlight certain elements of that reality while ignoring others”.Footnote ²⁷⁵ However, while humans know that concepts and classifications are mere social constructs, and that they only represent a selective and partial aspect of that which is being classified, this cannot be said of algorithmic systems. For instance, my mother is not just a ‘mother’, but also a ‘wife’, a ‘daughter’, a ‘colleague’, a ‘friend’, a ‘consumer’, a ‘reader’, a ‘Belgian’, a ‘woman’ and more. Depending on the particular context, one or more aspects of an individual will be particularly focused on, for example to determine the applicability of certain legal rules, even if that aspect is but one part of a more comprehensive picture.Footnote ²⁷⁶ It is therefore possible, yet always partial, to place people and things into a category, even if we do so on a daily basis.Footnote ²⁷⁷

These conventional concepts, limited as they may be to reflect the richness of reality, are how we structure our world, and – by extension – our legal system. Yet as long as we find ourselves in an intersubjective environment, we can draw attention to the limitations of these concepts, contest them, explain why a certain categorisation is erroneous, provide nuance, or ask for additional options or categories given that we share the concept’s meaning.Footnote ²⁷⁸ Such explanation and contestation is not possible when classifications are made by an algorithmic system. Furthermore, the registration of information into databases also tends to reverse the burden of proof when seeking to correct such information. Indeed, “once something is registered, it is considered ‘true’ according to the principles of formal bureaucracy”.Footnote ²⁷⁹ And, as discussed above, the larger the database, the more chance that datapoints are erroneously registered or classified, which can evidently affect administrative acts that are based on such erroneous information.

This hints to the fact that algorithmic regulation not only generates benefits, but that it can also reinforce some of the pitfalls we already encountered with bureaucracy. I will revisit this issue expansively in further chapters. At this stage, suffice it to conclude that the logic of bureaucracy and the logic of algorithmic systems (and particularly the efficiency-oriented informatisation, standardisation, and rationalisation they bring along) seem rather aligned. While the uptake of algorithmic regulation does impact the way in which public officials perform their tasks, the core features of bureaucratic organisation that underpin those tasks appear to have remained in place.

Before concluding this chapter on algorithmic regulation, let me briefly make explicit what I have thus far mostly implied, namely the benefits that public authorities aspire to materialise when adopting algorithmic regulation.

2.3.5 Rationale of Algorithmic Regulation

“What if more poor families were to receive the state benefits they need? What if more patients’ lives can be saved? What if more children could be protected from situations of abuse? What if more people can be rescued from imminent flooding? What if more terrorist attacks could be prevented?”Footnote ²⁸⁰ The OECD does not beat around the bush regarding the opportunities it sees in the adoption of algorithmic systems. As of yet, evidence of the concrete benefits of algorithmic regulation is scarce, particularly as regards data-driven systems, given the relative novelty of their scaled use.Footnote ²⁸¹ As noted by van Noordt and Misuraca, “many of the benefits ascribed to AI for the public sector are not always based on empirical data, but often rely only on assumptions”, rendering it difficult to validate and assess the actual benefits of the adoption of algorithmic regulation in public authorities.Footnote ²⁸² In what follows, I discuss the aspired or expected benefits they generate, without taking the materialisation of such benefits for granted.

First, as already pointed out, the automated nature of algorithmic regulation means that its computations can be carried out on a vast scale (even population-wide), thereby enabling mass decision-making.Footnote ²⁸³ As Schartum notes,

Accordingly, by processing vast amounts of data in a short amount of time, algorithmic systems may increase the efficiency of processes, as the speed of their computations supersedes that of human decision-making. These efficiencies are also believed significantly to reduce costs given that “cognitive technologies could free up hundreds of millions of public sector worker hours”.Footnote ²⁸⁵

Second, the ability to peruse a large amount of data in a short amount of time also means algorithmic systems can help optimise decision-making processes.Footnote ²⁸⁶ Public authorities possess more data than ever before, yet this easily leads to an excess of information when resources to analyse such data are limited. As Michèle Finck puts it, “in an age of informational overload through the continuous generation of ever more data, computational learning may become the only means of making sense of data the quantity of which exceeds the capacities of human cognition”.Footnote ²⁸⁷ As discussed above,Footnote ²⁸⁸ data-driven systems are indeed able to identify patterns in data that public officials may not be able to see, and can on that basis provide recommendations that could in principle enable public authorities to reach their goals more effectively. Consider the example of an algorithmic system used by the French tax authorities to identify non-declared swimming pools based on Google maps images, thereby recuperating about 10 million euros of non-paid taxes in a short amount of time.Footnote ²⁸⁹

Third, algorithmic regulation is sometimes introduced with the explicit aim of reducing the risk of partial, erroneous, arbitrary, or biased decision-making, and to decrease the risk of corruption.Footnote ²⁹⁰ By codifying the application of rules designed to address these risks into algorithmic systems, an informational architecture is created that can prevent public officials to deviate from the law by design.Footnote ²⁹¹ Furthermore, the automated application of legal rules could also lead to their more consistent application, as the same rule will no longer be applied by different public officials who might have their own interpretation or judgment.Footnote ²⁹² It has been argued that, in this way, algorithmic regulation might not only enhance the law’s effectiveness, but also public authority’s legitimacy.Footnote ²⁹³

Fourth, algorithmic systems could take over tasks that are highly repetitive and intellectually unstimulating for public officials, or tasks that are dangerous and would unduly expose human beings to risks,Footnote ²⁹⁴ such as cleaning a nuclear site.Footnote ²⁹⁵ Reports that highlight the advantages of such systems typically emphasise that this frees up valuable time which public officials can spend on safer or more interesting tasks, or for better interactions with citizens in need, rather than replacing their jobs.Footnote ²⁹⁶

Fifth, algorithmic regulation, especially when based on data-driven systems, can also enable the personalisation of administrative acts at lower cost, thereby reconciling the massive scale of decisions that public authorities must take, with the need for individual tailoring.Footnote ²⁹⁷ More generally, the algorithmisation of the public sector and establishment of well-kept interoperable databases should avoid the need for citizens to provide the same data multiple times to different authorities,Footnote ²⁹⁸ and might enable public authorities to provide their services proactively based on the information they have, without the need for a citizen to request a service, let alone the need to physically visit the authority’s office.Footnote ²⁹⁹

Importantly, all of these benefits must be considered against a background of increasing pressure that public officials face to ‘do more with less’. Their responsibilities keep on increasing, while public spending is significantly being cut. As early as 2018, a survey conducted by McKinsey revealed that “43 percent of all public sector transformation efforts over the past five years have had cost reduction as a core goal”.Footnote ³⁰⁰ The global recession and the war in Ukraine arguably only magnified this financial pressure. At the same time, I must underline that the aim to cut back public spending by adopting technology is by no means a new development. Already in the 1980s, under the influence of the New Public Management movement (NPM) which espoused the integration of private sector management ideas into public administration,Footnote ³⁰¹ algorithmic systems were increasingly embedded in public processes. NPM reinforced the ideal of efficiency and imbued it with private sector tools such as market-based mechanisms, performance indicators, outsourcing and procurement, customer-service orientation, but also budget cuts and performance management of staff. The adoption of digital technology was seen as an important part thereof.Footnote ³⁰² While NPM has been subjected to criticism,Footnote ³⁰³ the push towards performance indicators and cost-savings remained, and similar ideas also permeated the subsequent ‘digital-era governance’ and ‘e-governance’ movements, which more explicitly focused on efficiency improvements through automated data analysis and electronic platforms.Footnote ³⁰⁴

Furthermore, a strong push for the uptake of algorithmic systems in the public sector also came from the European Commission. In 2009, EU Member States signed the Malmo Declaration on eGovernment,Footnote ³⁰⁵ which the Commission implemented through the European eGovernment Action Plan 2011–2015Footnote ³⁰⁶ and the eGovernment Action Plan 2016–2020,Footnote ³⁰⁷ each underlining the need to accelerate the digital transformation of governments. The Tallinn eGovernment Declaration in 2017 added an important impetus for Member States and the Commission to continue investing in the modernisation of the public sector, seen as indispensable to increase “the transparency, responsiveness, reliability, and integrity of public governance”.Footnote ³⁰⁸

Most recently, with its Coordinated Plan on AI in 2019Footnote ³⁰⁹ and 2021,Footnote ³¹⁰ the European Commission urged all EU Member States to adopt their own national AI strategy, including a plan for the technology’s adoption in the public sector.Footnote ³¹¹ The EU also explicitly finances the uptake of algorithmic regulation by Member States through the recovery and resilience facility that was established to meet the financial challenges raised by the Covid-19 pandemic,Footnote ³¹² and more indirectly through other EU budgets.Footnote ³¹³ In sum, Member States have reasons enough to adopt algorithmic regulation, and it can only be expected that this trend will be accelerated in the years to come.

A caveat should be made, as the benefits aspired by algorithmic systems are not automatically achieved.Footnote ³¹⁴ After all, the actual benefits of these systems entirely depend on how they are designed, developed and used, and whether they take into account the risks set out in previous sections. It is not because a system is developed or deployed with good intentions, that it is also developed and deployed in a good manner, with due attention to unintended consequences or problematic uses later on. Moreover, even when benefits are achieved, this does not mean they actually benefit all. Often, those who already find themselves in a beneficial position will be best placed to reap those benefits, whereas those who are in a vulnerable position may not necessarily be better off,Footnote ³¹⁵ a point that I will revisit later on.

In addition, the capacities of algorithmic systems, and particularly systems defined as ‘AI’, are sometimes oversold,Footnote ³¹⁶ and hyperbolic statements about their benefits, followed by disappointing results, are not uncommon.Footnote ³¹⁷ Yet once significant investments have been made to establish an algorithmic regulation project, sunken costs and path dependencies render it difficult to abandon it, even if the output is not up to standard. Finally, it is precisely the desire to reap the benefits of this technology, often with the explicit aim to increase individual and societal welfare, that can blind public authorities to its risks and that eventually gives cause to concerns.