5.2.1 Article 2 and 7 TEU

The rule of law is listed in Article 2 TEU as one of the foundational values of the EU, common to all Member States. It needs to be respected by states who aspire EU membership,Footnote ¹⁴ and it needs to be respected throughout a state’s EU membership.Footnote ¹⁵ At the same time, the Treaty does not detail what, precisely, it means by ‘to respect the value of the rule of law’. The drafters of the Treaty explained their selection for the values to be listed in Article 2 TEU based on the fact that these values “have a clear non-controversial legal basis so that the Member States can discern the obligations resulting therefrom which are subject to sanction”.Footnote ¹⁶ Given that I had to develop an analytical framework in Chapter 3 to concretise, based on an extensive examination of legal sources, the rule of law requirements that Member States must meet when exercising public power, I would beg to differ.Footnote ¹⁷

Be that as it may, Article 7 TEU provides a mechanism of protection in case the values listed in Article 2 TEU are threatened or infringed by a Member State. Although Article 7 TEU hence protects multiple values, it is typically referred to as the rule of law protection mechanism, since it not only protects the rule of law as one value amongst others, but it also consists of a legal provision that literally embodies the protective role of the law in a liberal democratic system. Article 7(1) TEU enables the Council, acting by a majority of four fifths of its members, to determine that there is a “clear risk of a serious breach by a Member State of the values referred to in Article 2”. This determination must be based on a reasoned proposal by one third of the Member States, by the European Parliament or by the European Commission, and it can only be taken after the European Parliament consented. The mechanism of Article 7(1) TEU requires only the clear risk of a serious breach,Footnote ¹⁸ and hence serves as a warning mechanism.Footnote ¹⁹ In that case, the Council will hear the Member State in question and can address recommendations to it.Footnote ²⁰ While I will not elaborate on this point here, it is worth highlighting that the analysis of Chapter 4 did indicate that the increasingly widespread use of algorithmic regulation, without the adoption of appropriate protection mechanisms to counter its risks, can entail a clear risk of a serious breach of the rule of law.

Article 7(2) TEU goes a step further, as it enables the determination of the ‘existence’ of a ‘serious and persistent’ breach by a Member State of the values referred to in Article 2. In this case, however, the determination can only be made through a unanimous decision of the European Council, based on a proposal by one third of the Member States or by the Commission and after obtaining the consent of the European Parliament.Footnote ²¹ This is because the stakes of this determination are higher. Once a decision has been taken, Article 7(3) TEU enables the Council, acting by a qualified majority, to decide to suspend certain rights deriving from the application of the Treaties to the Member State in question, including voting rights in the Council. Before the determination of a ‘clear risk’ of a serious breach, or of the ‘existence of a serious and persistent breach’ of EU values, the Member State concerned always has the right to be heard and to submit its observations.

Thus far, the European CommissionFootnote ²² and the European Parliament sought to trigger Article 7(1) against both Poland and Hungary,Footnote ²³ and the Council has organised several hearings to hear the countries’ positions. However, despite the calls by scholars, civil society organisations and even EU institutions to take further action,Footnote ²⁴ and despite the fact that the concerns remain unaddressed (especially in Hungary),Footnote ²⁵ the Council has come to the determination of neither a ‘clear risk’ nor the ‘existence of a serious and persistent breach’ of the rule of law. In addition, the Council’s unwillingness to ensure that the Commission’s recommendations to ameliorate the situation are implemented has led to much frustration.Footnote ²⁶ Given the role of the Council (consisting of representatives from the twenty-seven EU Member States), the procedure in question is primarily political in nature. Considering the extensiveness of the procedure’s potential consequences and the sheer symbolically weighty nature of its invocation, it is only considered a measure of last resort. Moreover, as long as there is at least one other Member State who vetoes the determination of the existence of a ‘serious and persistent breach’, the procedure of Article 7(2) stands no chance given the requirement for unanimity.Footnote ²⁷

The inability of Article 7 TEU to counter the ongoing rule of law threats in some EU Member States and, in particular, the reluctance to deploy its mechanism has been subjected to heavy criticism. It also pinpoints the dilemma that the EU and its Member States are faced with, in between maintaining the cooperation and goodwill of the Member States on other fronts, and ensuring adherence to the rule of (EU) law without further alienating infringing states. Either way, by the time Article 7 TEU finally comes into play, if ever, a lot of damage will already have been done.

One problem, which is also relevant to the risks identified in the context of algorithmic regulation, is that the erosion of the rule of law often occurs incrementally rather than suddenly.Footnote ²⁸ However, as previously noted, incremental changes will rarely trigger a sufficiently strong counter-reaction so as to lead to the adoption of a measure which is deemed as far-reaching as Article 7 TEU. This gives rise to a tragic observation: the mechanism designed to avoid systemic breaches of the rule of law is, precisely because of the systemic and incremental nature of those breaches, unable to carry out its proper function.

Setting the ineffectiveness of Article 7 TEU aside, the EU’s increased attention for the (dis)respect of the rule of law in Member States (perhaps precisely due to this ineffectiveness) did give rise to a number of soft-law initiatives that monitor the rule of law situation across the Union.Footnote ²⁹ Indeed, in recent years, the Commission’s rule of law toolbox has expanded, and now encompasses several evidence-gathering and documentation mechanisms based on rule of law-indicators. Examples are the ‘European Semester’,Footnote ³⁰ the ‘EU Justice Scoreboard’,Footnote ³¹ and the annual ‘Rule of Law Reports’Footnote ³² that essentially institutionalise a dialogue between the Commission and Member States. These mechanisms also provide useful information that could support the triggering of Article 7 TEU, the Conditionality Regulation or infringement actions, which I discuss in the following sections.Footnote ³³ While non-binding, and not providing any guarantee that they will contribute to a successful triggering of the Article 7 TEU procedure, unlike the former, these initiatives are able to play a preventative role rather than a mere reactive one.Footnote ³⁴ Furthermore, they enable the dissemination of information about the status of the rule of law in every EU Member State to the public at large, thereby contributing to transparency and potentially accountability at the national level too.

Regrettably, however, none of these initiatives currently have indicators about the potentially adverse impact of algorithmic regulation on the rule of law, or on the checks and balances that have been implemented to counter such impact. Quite to the contrary, if discussed at all, ‘digitalisation’ is presented as a desirable feature that Member States should ideally swiftly implement in light of the ‘efficiencies’ it can enable.Footnote ³⁵ The risks attached to it, particularly in the area of public administration, are left unspoken and unmonitored, despite their systemic nature and potential contribution to the systemic breaches of the rule of law. In other words, the effects of algorithmic regulation on the rule of law remains a blind spot. As it currently stands, neither Article 7 TEU nor the soft-law mechanisms aimed at monitoring the rule of law situation seem to offer safeguards against the threat conceptualised under Chapter 4.

5.2.2 The Conditionality Regulation

Besides soft-law initiatives, the European Union also sought to expand its rule of law toolbox with binding mechanisms. The adoption of the Conditionality Regulation, or Regulation 2020/2092 of the European Parliament and of the Council of 16 December 2020 on a general regime of conditionality for the protection of the Union budget, can be seen as a highlight in this regard.Footnote ³⁶ In addition to its legal relevance, I already noted in Chapter 3 that the Conditionality Regulation also has significant definitional relevance, as it provides “the first comprehensive all-encompassing internal-oriented definition of the rule of law adopted by the EU co-legislators”.Footnote ³⁷ While the European Commission had already proposed a conceptualisation of the rule of law and its six principles in 2019,Footnote ³⁸ by means of the Conditionality Regulation, this conceptualisation was also formally adopted by the European Parliament and the Council and enshrined in a piece of secondary legislation.Footnote ³⁹

The Regulation foresees a mechanism to suspend the transmission of EU funds to ‘rogue’Footnote ⁴⁰ Member States – or, more diplomatically, it conditions the transmission of EU funds to compliance with the rule of law. Its primary aim is hence to protect the EU budget and ensure the sound financial management thereof by Member States, while at the same time incentivising rule of law compliance through financial means. In essence, the regulation seeks to prevent that EU funds are being used for authoritarian or rule of law-infringing ends.Footnote ⁴¹ Some scholarsFootnote ⁴² noted that the European Union already had a similar mechanism at its disposal through Regulation 1303/2013, laying down common provisions regarding the governance of a number of EU funds, including the European Regional Development Fund, the European Social Fund, the Cohesion Fund, the European Agricultural Fund for Rural Development and the European Maritime and Fisheries Fund.Footnote ⁴³ Article 142 of that Regulation foresees a procedure for the suspension of payments by the Commission if, inter alia, “there is a serious deficiency in the effective functioning of the management and control system of the operational programme, which has put at risk the Union contribution to the operational programme and for which corrective measures have not been taken”.Footnote ⁴⁴ Based on the idea that “surely, a country without the rule of law cannot generate effective management and control systems”, Kelemen and Scheppele argue that this provision already enabled the suspension of EU funds to rule of law-infringing Member States, yet they “note with disappointment that the Commission has not yet had the will to use the power already in its hands”.Footnote ⁴⁵

Despite the seeming pre-existence of similar remedies, the adoption of the Conditionality Regulation, as a general regime of conditionality, was not without difficulty or controversy.Footnote ⁴⁶ As soon as it was adopted, both Hungary and Poland challenged the validity of the regulation’s legal basis before the Court.Footnote ⁴⁷ They claimed that the EU did not have the legal competence to adopt a regulation that defines the rule of law or determines criteria to establish breaches thereof.Footnote ⁴⁸ Moreover, they stated that the Regulation was not compatible with Article 7 TEU, which already, in their view, exhaustively, provides for a mechanism to protect the rule of law, thus precluding the EU to adopt another one.Footnote ⁴⁹ As noted in Chapter 3, the EU does not have a ‘general’ legal competence to enforce the rule of law, and hence had to resort to a domain-specific legal basis. Given the link of the Regulation’s suspension mechanism with the multi-annual financial framework (MFF), some have argued that the Regulation would need to be adopted under the MFF’s legal basis, Article 312 TFEU, which requires adoption through unanimity in the Council.Footnote ⁵⁰

Instead, the Commission resorted to Article 322(1)(a) TFEU as the Regulation’s legal basis, well aware that the unanimity requirement of Article 312 TFEU would prevent the Regulation to ever see the light of day, since Poland and Hungary would never agree to it. Article 322(1)(a) TFEU provides that the European Parliament and the Council, acting in accordance with the ordinary legislative procedure (i.e. without unanimity), can adopt regulations to set the financial rules determining “the procedure to be adopted for establishing and implementing the budget and for presenting and auditing accounts”. Accordingly, the Conditionality Regulation can be seen as an example of the EU’s use of legal competences in one particular field (e.g. the EU budget) to promote other aims in a more indirect way (e.g. the rule of law), thereby overcoming the hurdles of potentially limited competences.Footnote ⁵¹ This approach, while not uncriticised, is not new. Various legislative initiatives which are based on Article 114 TFEU, aimed at advancing the harmonisation of the EU’s internal market, also contribute (directly or indirectly) to the protection of fundamental rights, an area in which the EU’s competences are also not generalised.Footnote ⁵²

In the past, the Court has already frequently been called upon to ensure that the EU does not overstep its competences by adopting an erroneous legal basis, and did not shy away from invalidating legislation where it deemed this to be the case.Footnote ⁵³ However, in this case, the challenge raised by Poland and Hungary was unsuccessful, and the Court upheld the Regulation’s validity. It found that the Regulation does not create a general rule of law protection mechanism, and only limits itself to those rule of law-infringements that are directly linked to the EU budget, hence remaining within the confines of Article 322 TFEU.Footnote ⁵⁴ The Court also distinguished the Regulation’s mechanism from the procedure in Article 7 TEU, which hence did not preclude its adoption.

With the Court’s blessing, the Commission thus no longer faced an obstacle or excuse to launch the suspension mechanism.Footnote ⁵⁵ In April 2022, it therefore announced the initiation of the procedure against HungaryFootnote ⁵⁶ – the first step of a lengthy procedure.Footnote ⁵⁷ Indeed, before the suspension mechanism could actually be implemented, the procedure set out by the Regulation had to be followed, including a written notification to the Member State concerned, setting out the factual elements and specific grounds on which the findings are based; the Member State’s response and potential proposal of remedial measures; the Commission’s notification of its intention to propose an implementing decision in case it considers the remedial measures to be insufficient; yet another opportunity for the Member State to share its observations, particularly regarding the proportionality of the Commission’s envisaged measure; and, finally, an implementing decision by the Council acting by qualified majority, based on the Commission’s proposal.Footnote ⁵⁸ In March 2022, the Commission also adopted guidelines in which it set out how it will apply the regulation in more detail.Footnote ⁵⁹

Importantly, the suspension mechanism can only be invoked with regards to a limited set of rule of law-infringements that have a link with the EU budget, hence not providing a general suspension clause for all rule of law-infringements. After all, its purpose and its legal basis concern the protection of the EU budget rather than the rule of law in general. Article 3 of the Regulation sets out which Member State actions may be ‘indicative’ of breaches of principles of the rule of law for the purpose of the regulation, namely:

In addition to these ‘general’ indications, Article 4 sets out that action can be undertaken where it is established that “breaches of the principles of the rule of law in a Member State affect or seriously risk affecting the sound financial management of the Union budget or the protection of the financial interests of the Union in a sufficiently direct way”.Footnote ⁶⁰ To be deemed sufficiently direct, the breaches of the principles of the rule of law should fall under one of the following exhaustively listed categories:Footnote ⁶¹

1. (a) the proper functioning of the authorities implementing the Union budget, including loans and other instruments guaranteed by the Union budget, in particular in the context of public procurement or grant procedures;

2. (b) the proper functioning of the authorities carrying out financial control, monitoring and audit, and the proper functioning of effective and transparent financial management and accountability systems;

3. (c) the proper functioning of investigation and public prosecution services in relation to the investigation and prosecution of fraud, including tax fraud, corruption or other breaches of Union law relating to the implementation of the Union budget or to the protection of the financial interests of the Union;

4. (d) the effective judicial review by independent courts of actions or omissions by the authorities referred to in points (a), (b) and (c);

5. (e) the prevention and sanctioning of fraud, including tax fraud, corruption or other breaches of Union law relating to the implementation of the Union budget or to the protection of the financial interests of the Union, and the imposition of effective and dissuasive penalties on recipients by national courts or by administrative authorities;

6. (f) the recovery of funds unduly paid;

7. (g) effective and timely cooperation with OLAFFootnote ⁶² and, subject to the participation of the Member State concerned, with EPPOFootnote ⁶³ in their investigations or prosecutions pursuant to the applicable Union acts in accordance with the principle of sincere cooperation;

8. (h) other situations or conduct of authorities that are relevant to the sound financial management of the Union budget or the protection of the financial interests of the Union.

Following the Regulation’s application to Hungary, on 15 December 2022, the Council decided to freeze about €6.3 billion of budgetary commitments,Footnote ⁶⁴ citing rule of law breaches pertaining to public procurement procedures and prosecutorial action, as well as conflicts of interest and concerns around the fight against corruption (though not the concerns around judicial independence). At the same time, inspired by the Conditionality Regulation’s approach of financial incentivisation, the Commission also started relying on legal provisions in other funding mechanisms that tie a Member States’ receipt of funding to certain conditions (such as the Resilience and Recovery RegulationFootnote ⁶⁵ and the Common Provisions RegulationFootnote ⁶⁶), including the obligation to uphold the Charter of Fundamental Rights. On this basis, on 22 December 2022 the Commission decided to freeze about €21.7 billion of EU cohesion funds until Hungary adopted several measures regarding LGBTQI+ rights, academic freedom, asylum policies and judicial independence.Footnote ⁶⁷

These actions were initially applauded, yet the applause for the Commission was short-lived. A year later, on 13 December 2023, it decided to unblock €10.2 billion of the frozen EU cohesion funds, justifying its decision based on a ‘thorough assessment’ of Hungary’s newly adopted measures to strengthen the judiciary’s independence.Footnote ⁶⁸ While critics have characterised those measures as mere window dressing, others have pointed to the politically strategic move of the Commission, as its decision to unfreeze the funds came a day before European leaders needed Hungary’s cooperation to approve new aid to Ukraine.Footnote ⁶⁹ In March 2024, the European Parliament therefore decided to challenge the Commission’s decision before the Court of Justice, claiming that the Commission failed to fulfil its obligation to protect the EU budget and ensure that taxpayers’ money is not misused.Footnote ⁷⁰ If anything, this saga highlights that, despite the availability of several legal mechanisms to freeze EU funds, their application and effect are still highly dependent on the political willingness of EU actors and Member States to do so, and the bridging of political hurdles.

Let us, however, bracket this political aspect for a moment, and examine to what extent the Conditionality Regulation could at least theoretically serve as a mechanism to counter the threat of algorithmic rule by law. While it was certainly not conceived to offer protection in the specific context of algorithmic systems, it is worth asking whether it can nevertheless play a role in this context. Based on the contours set out above, a number of observations can be made.

First, given the Regulation’s focus on the governance and sound management of EU funds, it appears to be most suitable in countering actions relating to fraudulent behaviour by officials or corrupt public procurement practices. Conversely, many of the examples discussed under Chapter 4, for instance in the area of social welfare or criminal risk assessments, would hence not easily fall under its scope. As regards the examination of tax fraud (an area in which algorithmic regulation is already widely used) the Regulation requires a link with the EU budget, and hence would not apply to all fraud examinations (for instance under points (c) and (e)). That said, the Court’s judgment in Ackerberg FranssonFootnote ⁷¹ already clarified that, since the European Union’s own resources include revenue from Member States’ collection of VAT, the investigation and prosecution of tax fraud relating (even in part) to VAT affects the financial interests of the European Union.Footnote ⁷² It could hence reasonably be argued that, when public authorities deploy algorithmic systems to detect and prosecute tax fraud (including VAT), and they do so in a manner that does not comply with the six rule of law principles set out above, this would likewise fall under the Regulation’s scope.Footnote ⁷³

Yet one can also raise a more straightforward example of where algorithmic regulation might play a role. When the analysis of elements and risks based on which a procurement contract is allocated (or not) occurs with the help of algorithmic systems,Footnote ⁷⁴ one could argue that the potentially problematic design and use of these systems can adversely impact the rule of law in a way that is relevant for the purpose of the regulation. Procurement processes are increasingly supported by analyses carried out by algorithmic systems, and one can easily imagine that ‘objectivity’ might also be invoked to this end. And while such automation could make procurement processes more ‘efficient’,Footnote ⁷⁵ at the same time, one might also imagine the risk that, whether through negligence or deliberate design choices, the outcomes of the automated process favour some tenderers over others. This scenario would, at least in theory,Footnote ⁷⁶ fit rather neatly under Article 4(2)(a) of the Conditionality Regulation, and hence be susceptible to trigger (the first step of) the suspension mechanism. Accordingly, it appears that a number of applications of algorithmic regulation could relevantly fall under the scope of the Conditionality Regulation.

However, that fact in and of itself does not yet help us much further. Certainly, the Regulation could in theory disincentivise Member States to adopt algorithmic regulation in an irresponsible manner in the abovementioned areas. However, for this to be the case, Member States must first be aware of the risks posed thereby to the rule of law, which is not a given. Furthermore, these provisions will still not ensure that they set up appropriate transparency, control and oversight mechanisms over the algorithmic systems’ design and deployment process. And while, in theory, the Regulation could provide a means to penalise Member States who do not take the risks emanating from algorithmic regulation seriously, by the time such penalisation arrives, a lot of damage may already have occurred – damage that could potentially have been prevented or at least mitigated in case of ex ante oversight. Furthermore, as already noted previously, any ex post remedy will in any case be dependent upon the realisation that the algorithmic system was inappropriately designed or used, a realisation that is difficult to achieve if the use of the system is not transparent and open to systemic review rather than mere individual review.Footnote ⁷⁷

Consequently, while the Conditionality Regulation can at least help prevent that more EU funds are made available to Member States that, with or without the assistance of algorithmic regulation, infringe the rule of law’s principles, its relevance remains confined to areas that have an explicit link with the EU financial interests, and to Member States who disregard the rule of law to such extent that the Commission feels itself forced to launch the mechanism provided in the regulation. As a preventative tool for Member States who are negligent rather than deliberate in countering the adverse impact that their algorithmic systems can have on the rule of law, it will have little to no effect. It can thus be concluded that other safeguards, preferably of an ex ante character, are needed to address the threat of algorithmic rule by law.

5.2.3 Infringement Actions and Proceedings before National Courts

The two mechanisms I discussed above seek to counter, respectively, Member State breaches of EU values (including the rule of law) that are ‘serious and persistent’; and Member State breaches of the rule of law that directly affect the financial interests of the EU. Yet when it comes to the protection of the rule of EU law (namely Member States’ adherence to European Union law more generally), there are two other mechanisms that can be pointed out. The first concerns the infringement procedure enshrined in Articles 258–260 TFEU, which allows the European Commission or a Member State to seize the CJEU in case a fellow Member State failed to fulfil an obligation under the Treaties. The second concerns the ability to challenge a national act that breaches EU law before a national court, potentially through the preliminary reference procedure laid down in Article 267 TFEU. Both of these procedures enable the judicial review of the legality of Member States’ actions in light of their obligations under EU law.Footnote ⁷⁸ Indeed, Member States are obliged to take “any appropriate measure, general or particular, to ensure fulfilment of the obligations arising out of the Treaties or resulting from the acts of the institutions of the Union,”Footnote ⁷⁹ including obligations they might have under both primary and secondary EU legislation.

By virtue of the infringement procedure of Article 258 TFEU, the Commission may sue a Member State before the CJEU when it considers that it has failed to fulfil an obligation under the Treaties.Footnote ⁸⁰ It must first deliver a reasoned opinion on the matter and give the State concerned the opportunity to submit its observations. In the absence of the State’s compliance with the opinion within the period laid down by the Commission, it may take the matter to Court.Footnote ⁸¹ Article 259 TFEU also foresees that a Member State which considers that another State failed to fulfil an obligation under the Treaties can bring the matter to Court – after first passing by the Commission.Footnote ⁸² If the Court finds that the Member State has indeed failed to fulfil an obligation, the State shall be required to take the necessary measures to comply with the judgment of the Court.Footnote ⁸³

Importantly, if the Commission subsequently finds that the Member State did not comply with the Court’s judgment, it may take the matter before the Court again, this time with the request that the Member State be imposed a lump sum or penalty payment.Footnote ⁸⁴ Accordingly, the infringement procedure also provides the Commission with financial leverage to ensure that Member States comply.Footnote ⁸⁵

Note that the threshold for such a procedure does not consist of a ‘serious’, ‘persistent’ or ‘systemic’ breach of EU law, but simply of the failure to fulfil an obligation. This can also concern the mere obligation to transpose a directive into national law within the specified deadline.Footnote ⁸⁶ It should be noted that the European Commission has discretion as to whether or not it decides to bring a case to court, and it has no obligation to do so.Footnote ⁸⁷ That said, pursuant to Article 17(1) TEU, it is the Commission’s task to ensure that EU law is duly applied.Footnote ⁸⁸

Some have argued that the infringement procedure avenue constitutes an important tool for the Commission to counter rule of law infringements in the EU.Footnote ⁸⁹ Others have claimed, however, that: “despite ten years of EU attempts at reining in Rule of Law violations and even as backsliding Member States have lost cases at the Court of Justice, illiberal regimes inside the EU have become more consolidated: the EU has been losing through winning”.Footnote ⁹⁰ Admittedly, the infringement procedure is not designed to address systemic deficiencies of the rule of law, but rather provides a mechanism to ensure the enforcement of Member States’ specific EU law obligations. However, used tactically, it can serve to challenge Member States’ actions that adversely impact the rule of law, and the Commission is increasingly doing so. The Court is thereby enabled to carry out a judicial review of the conformity of a Member State’s actions with EU law, and to call out their illegality in case of non-conformity.Footnote ⁹¹

Another, more indirect, route to enable judicial review of Member States’ actions at EU level is provided by Article 267 TFEU, which establishes a procedure that can be initiated by natural and legal persons before their national courts, hence complementing public enforcement with private enforcement.Footnote ⁹² National courts play an essential role in the EU legal order, as they safeguard the application of EU law at the national level. Given the primacy of EU law and its direct effect in national legal orders,Footnote ⁹³ national courts must interpret national acts in line with EU law, and must even leave them aside when they breach EU law.Footnote ⁹⁴ When they are uncertain about the way in which they should interpret EU law to assess the validity of a national (administrative) act, a national court can initiate a preliminary reference procedure and seek guidance from the CJEU.Footnote ⁹⁵ In theory, the Court can only express itself about the interpretation of EU (primary and secondary) law rather than taking a stance about the validity of the national act. However, it should provide the national court with all the information it needs to undertake that assessment by itself.Footnote ⁹⁶ When based on such guidance, which is binding for all courts in the EU, the national court concludes that a legal act at national level breaches EU law, it needs to set it aside.

The disadvantage of this procedure is that a natural or legal person already needs to be involved in a legal challenge before a national court in order to request a preliminary reference procedure. Moreover, the invalidation of the national act does not necessarily imply an invalidation of the public authority’s working method more generally, as the judicial review is limited to the particular act in question rather than being systemic in nature, which showcases the limits of ex post judicial review rather than an ex ante preventative approach at the system level. But at least it offers natural and legal persons a legal avenue to protect the rights they derive from EU law, and to ensure that Member States fulfil their EU law obligations. In the context of Member States’ citizen surveillance, for instance, this procedure has already proven to be effective in invalidating national (and even EU) legislation that undermines the fundamental right to privacy and data protection.Footnote ⁹⁷

The question then is: which EU law obligations are relevant for the context of algorithmic regulation deployed by public authorities, and can pertinently be the object of an infringement procedure or of a legal challenge before a national court? Indeed, to trigger the use of these procedures, a specific provision of EU law must be breached – hence requiring the existence of a relevant EU law provision in the first place.

There is currently no general EU regulation or directive setting out the obligations that public authorities must fulfil to comply with the rule of law when taking administrative acts, regardless of whether they do so through reliance on algorithmic systems. The functioning of public administrations largely remained a matter of national competence, given its entwinement with the exercise of national powers. In situations where algorithmic regulation by public authorities leads to infringements of purely domestic law rather than EU law, the procedures mentioned above cannot be invoked and national remedies – to the extent available – need to be relied on instead. That said, in a wide array of public sector domains, a link to EU law can nevertheless be established given the increasing harmonisation of national law in fields that influence public administration, thus rendering purely domestic situations ever more rare. The EU has, for instance, adopted legislation in the area of migration law,Footnote ⁹⁸ social security,Footnote ⁹⁹ public procurement,Footnote ¹⁰⁰ environmental protection,Footnote ¹⁰¹ international transport and border control,Footnote ¹⁰² data (re)useFootnote ¹⁰³ and criminal justice.Footnote ¹⁰⁴

Whenever Member States implement EU law, they must respect fundamental rights as enshrined in the CharterFootnote ¹⁰⁵ (including, for instance, the right of defence, the presumption of innocence and the right to a good administration) as well as general principles of EU law.Footnote ¹⁰⁶ Therefore, if public authorities rely on algorithmic regulation while implementing EU law in a way that breaches individuals’ fundamental rights or general principles of EU law – for instance the general principle of equality, due to the discriminatory design or use of an algorithmic system – this constitutes the breach of a Member State’s obligation that can become the object of an infringement action or preliminary reference procedure. Yet when does a public authority implement EU law?

The most obvious case concerns the situation in which Member States’ public authorities act based on an EU regulation, directive or other legal act with binding force. To provide an example of an area of administrative law that has been (partially) harmonised, consider the domain of migration law, and more specifically the right to asylum. This right is enshrined in Article 18 CFREU, and has been further specified by secondary EU legislation that sets out which obligations Member States should respect vis-à-vis asylum applicants, and how they should evaluate an asylum application.Footnote ¹⁰⁷ Accordingly, when a mistranslation from law to code (whether deliberate or inadvertent) occurs in the context of algorithmic regulation that helps assess asylum applications, this can give rise to an EU law infringement, both in terms of the public authority’s failure to comply with the EU regulation or directive, and in terms of a potential breach of a fundamental right or general principle of EU law.

Besides harmonisation in vertical domains, there are also relevant pieces of legislation of horizontal nature. Consider, for instance, Council Directive 2000/43 that prevents discrimination based on racial or ethnic origin in a number of areas like social security and healthcare, thus setting out obligations that Member States must comply with.Footnote ¹⁰⁸ When algorithmic regulation affects the rights of individuals based on their ethnicity, this can hence constitute a breach of EU law when the administrative act pertains to social security, the provision of healthcare, or another area of administration listed in the directive. Moreover, beyond situations of implementing EU legislation, the case law of the CJEU has indicated that also acts “that constitute derogations from provisions of EU law, or acts adopted by the national authorities that only remotely are connected with EU law”,Footnote ¹⁰⁹ can fall under the heading of ‘EU law implementation’.Footnote ¹¹⁰

Finally, some authors have argued that Article 2 TEU could in and of itself be understood as the basis for an EU law obligation based on which an infringement action can be launched in case of non-adherence to the rule of law.Footnote ¹¹¹ For instance, Scheppele, Kochenov and Grabowska-Moroz have argued that Article 2 TEU could be relied upon to group isolated yet systemic infringements of EU law and, on that basis, to launch an infringement action, denoting this approach as a ‘tool of militant democracy’ through the launch of ‘systemic infringement procedures’.Footnote ¹¹² They believe that this approach could enable the Court, which has already shown itself an innovative protector of the principle of judicial independence by relying on Article 19(1) TEU, to play a more significant role in protecting other rule of law principles, which Article 7 TEU currently cannot due to the abovementioned political impasse. However, this understanding of Article 2 TEU is controversial, and as the authors themselves have noted, it is not widely supported.Footnote ¹¹³ Accordingly, the majority of scholars seem to consider that in order to be the object of an infringement procedure, the legal provision in question and the precise obligation it imposes on Member States needs to be more concrete, and cannot be brought under a collective Article 2 TEU umbrella.

That said, in December 2022, the Commission for the first time launched an infringement action against Hungary for rule of law-related breaches which not only cites violations of secondary EU legislation, but also the direct violation of Article 2 TEU itself.Footnote ¹¹⁴ Regardless of the outcome of this case, Bonelli and Claes point out that “the autonomous enforceability of Article 2 TEU remains a controversial legal construction, one that, if accepted by the Court, could put its legitimacy and authority under strain”.Footnote ¹¹⁵ They also rightfully question whether “the full judicialization of questions of ‘values’” is truly “the most promising and effective response to the challenges that constitutional backsliding processes create”.Footnote ¹¹⁶

Furthermore, notwithstanding the increasing harmonisation of national legislation, it should be stressed that a link with EU law cannot always be established, as there are still situations that are purely governed by domestic law. As I will discuss below,Footnote ¹¹⁷ this ups the game for any (new) EU legislative act that can provide safeguards for citizens whenever Member States deploy algorithmic regulation, as it can legally create a link with EU law (and hence with EU remedies) even in situations that are in principle considered domestic. Indeed, adopting EU legislation that governs the responsible use of algorithmic regulation by Member States in line with the rule of law would open up an avenue for the enforcement of these provisions both through the infringement procedure of Articles 258–260 TFEU, and through legal challenges brought by individuals before national courts (with the associated preliminary reference procedure pursuant to Article 267 TFEU).

At the same time, the utility of these remedies, even in situations where a concrete obligation under EU law exists, must not be overstated. While the judicial review they enable can certainly play a role in the protection of the rule of (EU) law at Member State level, this is woefully insufficient to tackle the adverse effects of algorithmic regulation as identified in Chapter 4.

First, the ex post nature of these procedures means that the damage has already been done. As noted previously, if the damage is irreversible, any ex post remedy will be of little consolation to those adversely affected. If the damage is not (entirely) irreversible, in any event, a lot of time will inevitably pass between the damage caused and the judicial action. In the case of an infringement action, it typically takes the European Commission (which can decide to launch an action at its sole discretion) months if not years to collect sufficient evidence and arrive at a decision to initiate proceedings, if it decides to act at all.Footnote ¹¹⁸ In the case of a legal challenge brought by an individual before a national court, the speed of the potential remedy will depend on how swift the administration of justice in a particular country is organised. Moreover, the count starts not from the moment that problematic algorithmic regulation is used, but from the moment that someone is aware of such use and decides to bring a case.Footnote ¹¹⁹ By the time a condemning judgment arrives, significant harm can have occurred, and it may be too late to meaningfully remedy the situation. In addition, if the national court decides to stay proceedings to submit a request for a preliminary reference by the CJEU, the waiting time is extended by on average at least another year.Footnote ¹²⁰ I invite the reader to reflect how much damage an infrastructure of algorithmic regulation, and the mass-decision-making it enables regarding millions of individuals, can cause in the time span of one year if left unaddressed.

Second, for an individual to bring a case before a court, she must not only be aware that algorithmic regulation is used (this is not straightforward given that many algorithmic applications are used in an untransparent mannerFootnote ¹²¹) but she must also have a sufficient incentive to start litigation.Footnote ¹²² If the damage at individual level is relatively small, the person may be unwilling to incur time and costs to do so, even if the damage at societal level may be significant. Furthermore, in many jurisdictions one needs to be able to demonstrate individual harm to have standing in court, which may not always be easy to prove when the harm primarily manifests itself at societal level or over the longer term, rather than at individual level and in the short term.Footnote ¹²³ And if individual harm can be proven (for instance when a right was erroneously denied) we have seen that courts may not always be well-equipped to deal with the systemic problems raised by the scaled use of algorithmic systems, as they are typically tasked with case-by-case reviews only.Footnote ¹²⁴ This means that the upstream design choices will remain untouched. The person may be re-allocated her benefits that were wrongfully denied, but this does not necessarily mean that choices at the upstream level will become transparent and contestable, and that harm to other interests will be avoided.

Finally, to invoke the procedures discussed above, a link with EU law must first be argued. As explained, while such a link can often be found, there are also situations that may not be governed by current EU law, hence precluding reliance on an existing EU remedy. Moreover, even if such a link is present, in certain Member States, the executive power is already exercising undue influence on national courts, and compromised their independence and impartiality.Footnote ¹²⁵ In those jurisdictions, which already showcase authoritarian tendencies, one can question the effectiveness of national judicial review as a means to prevent the exacerbation of those very tendencies.

For all these reasons, the mechanisms discussed above are inadequate to ensure that the implementation of algorithmic regulation by public authorities occurs in accordance with the rule of law, and that it does not exacerbate the threat of algorithmic rule by law. As stressed in Chapter 4, the extent and scale of the risks associated with algorithmic regulation require preventative action, inter alia through the organisation of ex ante and continuous oversight over the crucial decisions taken in the design and deployment process of algorithmic systems, rather than mere remedial action. Yet the currently available EU mechanisms that deal with the protection of the rule of (EU) law (Article 7 TEU, the Conditionality Regulation, or Articles 258–260 TFEU and 267 TFEU) are, in my view, not tailored to prevent the adverse impact of algorithmic regulation on the rule of law. This is not only so because they lack specific references to algorithmic systems as the potential cause of such adverse impact – and hence lack specific requirements as regards their use – but also because they only enable ex post solutions, which risk being too little too late. In sum, EU regulation pertaining to the protection of the rule of law does not seem to be sufficiently extended to tackle the risks of algorithmic regulation.

The question is now whether EU regulation that pertains to algorithmic systems can play a meaningful role in the protection of the rule of law. While there is, as of yet, no piece of EU legislation that deals specifically with public authorities’ rule of law obligations in the context of algorithmic regulation, there are several regulations that apply to public and private organisations alike when they inform or take their decisions with the assistance of algorithmic systems.Footnote ¹²⁶ For reasons of time and space, I will focus on the two most relevant ones for the purpose of my investigation: the General Data Protection Regulation (and neighbouring Law Enforcement Directive), which I discuss in Section 5.3, and the Artificial Intelligence Act, which I discuss in Section 5.4.

5.3.1 Need for a Legal Basis

To legitimise such data processing, Member States should in principle adopt a law that sets out the purpose of the processing, and that contains more specific provisions about the types of data that are processed; the data subjects concerned; the entities to, and the purposes for which, the personal data may be disclosed; the purpose limitation; storage periods; and any processing operations and procedures.Footnote ¹³⁵ This is also why the implementation of algorithmic systems by public authorities must have a lawful basis, which typically requires the adoption of a specific law that authorises the use of the system in line with the provisions of the GDPR (or the LED). The legal basis should indicate that the introduction of the algorithmic system, which can be invasive and impactful given the data processed and the scale at which it is deployed, is necessary and proportionate.

Importantly, the existence of a legal basis also renders it possible to subsequently challenge the system’s use if the basis on which it was adopted does not provide sufficient protection against potential adverse impacts of the system, or is not in accordance with human rights law. This is precisely what happened in the Dutch SyRI case, which centred around an algorithmic system aimed at identifying natural and legal persons which, based on a set of risk-indicators, ought to be further examined for social security or tax fraud.Footnote ¹³⁶ According to the government, the purpose of the system concerned “the prevention of and combating the unlawful use of government funds and government schemes in the area of social security and income-dependent schemes, preventing and combating taxes and social security fraud and non-compliance with labour laws”.Footnote ¹³⁷ The system allowed for the exchange of data amongst a variety of public authorities to facilitate the identification of fraud. Based on the system’s risk-indications, a risk report was made concerning an individual’s fraud potential. The SyRI-law defined a ‘risk report’ as

The system was already in use for years when, in 2014, a law was finally adopted to ensure a legal basis for its deployment, known as the SyRI-law. When the law was challenged in court, it was, however, found to be an insufficient legal basis to justify the system’s use. The court concluded that it did not provide sufficient safeguards to protect individuals against the impact on their right to privacy, after which the government halted the system’s use. For instance, the risk reports established by the system were put into a register.Footnote ¹³⁹ This left a clear trace for other public authorities which risked stigmatising those individuals, even if the flagging turned out to be erroneous. Moreover, the persons concerned were not informed of the fact that a risk report was made about them (unless they specifically asked for this information by themselves).Footnote ¹⁴⁰ And while the law also foresaw that public authorities had to justify the necessity and proportionality of the data exchange for the purpose of the risk analysis, no safeguards were included to ensure an adequate and comprehensive review of those justifications.

This case demonstrates that the GDPR is able to offer individuals protection by ensuring that the use of algorithmic systems needs to have an adequate legal basis. At the same time, it should be noted that the provisions of the GDPR, as such, do not necessarily provide insight (let alone participation) in any of the upstream decisions made by the coders, such as the assumptions that underlie the system, the interpretation and translation choices, or the model’s optimisation function. Moreover, in many cases, the legal basis can be overly vague or provide overly extensive processing powers to avoid that the law must be amended whenever new processing activities take place, thus also undermining the protection it offers towards those subjected thereto.Footnote ¹⁴¹ Note that the processing of special categories of personal data, such as data revealing racial or ethnic origin, political opinions, biometric data for the purpose of uniquely identifying a natural person, or data concerning a person’s sexual orientation, is in principle prohibited, yet exceptions exist,Footnote ¹⁴² and public authorities can typically rely on them to exercise their functions if such processing is authorised by law.Footnote ¹⁴³

5.3.2 Automated Decision-Making

The GDPR and LED allocate certain rights to persons whose personal data is processed, such as the right to information about the data processing, the right of access to their data, the right to rectify their data and the right to erasure.Footnote ¹⁴⁴ In addition to these general rights, natural persons also have specific rights in the context of automated decision-making. It should be pointed out that, for the purpose of the GDPR, automated decision-making is much broader than the adoption of an administrative act, but covers any type of decision taken through automated means, including profiling.Footnote ¹⁴⁵ Accordingly, all intermediate decisions that are taken by automated means to inform an administrative act (for instance the automated classification of individuals in one category or another) also count as such.Footnote ¹⁴⁶ Pursuant to Article 22 of the GDPR,Footnote ¹⁴⁷ whenever a decision is being taken about an individual based solely on automated processingFootnote ¹⁴⁸ which produces legal effects concerning her, or similarly significantly affects her, that individual has the right not to be subject to such decision.Footnote ¹⁴⁹ While inclusion of the word ‘solely’ seemingly excludes situations where a decision is merely recommended by an algorithmic system and subsequently reviewed and adopted by a human being, such review should in principle be meaningful in order to warrant the exclusion. Merely “fabricating” human involvement will not do.Footnote ¹⁵⁰

Moreover, under those circumstances, the data controllerFootnote ¹⁵¹ has the obligation to provide the individual concerned with information about the existence of automated decision-making and “meaningful information about the logic involved”, as well as the significance and the envisaged consequences of such processing for her.Footnote ¹⁵² Recital 71 of the GDPR also states that “such processing should be subject to suitable safeguards, which should include specific information to the data subject and the right to obtain human intervention, to express his or her point of view, to obtain an explanation of the decision reached after such assessment and to challenge the decision”. Yet given that recitals are non-binding, scholars disagree as to whether an actual individualised ‘right to explanation’ of an automated decision exists.Footnote ¹⁵³

These rights, along with other rights listed in the GDPR, can be restricted by Member State law as long as this restriction “respects the essence of the fundamental rights and freedoms and is a necessary and proportionate measure in a democratic society to safeguard” a range of interests, including national and public security, the prevention of criminal offences, and even – rather generally – “other important objectives of general public interest of a Member State”.Footnote ¹⁵⁴ In that case, the Member State law should however contain provisions that describe the data processing activity and its purpose, as well as the safeguards to prevent abuse.Footnote ¹⁵⁵

Also noteworthy is the fact that, whenever a type of processing (“in particular using new technologies”) is likely to result in a high risk to the rights and freedoms of natural persons, the controller shall, prior to the processing, carry out an assessment of the impact of the envisaged processing operations on the protection of personal data, referred to as a Data Protection Impact Assessment or DPIA).Footnote ¹⁵⁶ Pursuant to Article 35(3)(a) of the GDPR, a DPIA is particularly required in the case of “a systematic and extensive evaluation of personal aspects relating to natural persons which is based on automated processing, including profiling, and on which decisions are based that produce legal effects concerning the natural person or similarly significantly affect the natural person”.

Data controllers are, however, not obliged to make this assessment public, and input from the individuals concerned is only warranted “where appropriate” and “without prejudice to the protection of public interests or the security of processing operations”.Footnote ¹⁵⁷ Finally, to oversee compliance with these rights and obligations, the GDPR established national data protection authorities, acting “with complete independence in performing its tasks and exercising its powers”, which is especially important when supervising the data processing activities of public authorities.Footnote ¹⁵⁸

5.3.3 Evaluation: Necessary but Not Sufficient

With this in mind, to which extent can the safeguards afforded by the GDPR help counter the adverse impact of algorithmic regulation on the rule of law? Unfortunately, the conclusion is not overly optimistic. Certainly, the GDPR establishes a critical and necessary set of EU obligations that Member States should respect (which can also become the object of a procedure under Articles 258–260 TFEU or Article 267 TFEU) and provides individuals with important rights they can directly invoke in a national court against public authorities. And since algorithmic regulation very often implies personal data processing, those rights and obligations are unquestionably relevant in this context. That said, these safeguards cannot be called comprehensive.

As observed by Maja Brkan, the provision that aims to protect individuals against the adverse effects of automated decision-making, by containing “numerous limitations and exceptions, looks rather like a Swiss cheese with giant holes in it”.Footnote ¹⁵⁹ While individuals should be informed of the fact that decisions are being made about them in an automated way, recall that the right not to be subjected to such decisions is only present in case of a ‘solely’ automated decision with ‘legal effects’ or similar, that exceptions exist in the ‘public interest’ and that no transparency is foreseen about the algorithm’s functioning and the normative assumptions that underpin its design. In general, the GDPR contains no mechanisms that enable prior oversight over the upstream design of the algorithmic system (for instance to ensure its outcomes are accurate and non-biased); no obligatory mechanisms of public participation; no constitutional checks and balances as regards the translation from law to code; and no mechanisms that foster friction and internal critical reflection, for instance by mandating meaningful human oversight.

This is not to say that the GDPR and the LED, along with primary EU law protecting the rights to privacy and data protection, cannot play a role in ensuring that governments process the personal data of their citizens in a responsible manner, as previous legal challenges have demonstrated.Footnote ¹⁶⁰ For instance, in June 2022, the CJEU was seized by a preliminary reference procedure regarding the Passenger Name Record (PNR) Directive and the Belgian law implementing it.Footnote ¹⁶¹ The case concerned the automated processing of passenger data by public authorities in the context of border control, enabled by the establishment of large databases to search and identify passengers involved in a terrorist offence or serious crime. While the Court found that the Directive poses an “undeniably serious interference” with the rights to privacy and data protection,Footnote ¹⁶² it nevertheless concluded it was compatible with the Charter, as it required the predetermination of the criteria based on which the database could be searched, and required these criteria to be non-discriminatory.Footnote ¹⁶³ In the same breath, however, the Court noted that this “requirement precludes the use of artificial intelligence technology in self-learning systems (‘machine learning’), capable of modifying without human intervention or review of the assessment process and, in particular, the assessment criteria on which the result of the application of that process is based as well as the weighting of those criteria.”Footnote ¹⁶⁴ It added that

Accordingly, the PNR judgment affirms the important role that EU privacy legislation can play in the context of algorithmic regulation, and the importance of transparency about the way in which individuals’ data are being processed.Footnote ¹⁶⁶

At the same time, the lack of ex ante oversight mechanisms means that external accountability remains largely confined to an ex post stage, when the damage already occurred. Moreover, the focus lays primarily on harm to individual rather than to societal interests, such as the rule of law. It does not touch upon the intricacies of translating legal rules to code, nor does it provide more systemic remedies and oversight.Footnote ¹⁶⁷ In sum, these legal instruments do not provide sufficient safeguards to counter the threat of algorithmic rule by law, or even properly to counter the risks raised by algorithmic systems more generally. While this conclusion may be rather glum, it was not only reached by other scholars, but also by the European Commission itself, who in February 2020, when it published its White Paper on AI,Footnote ¹⁶⁸ observed that the current framework, including the GDPR, insufficiently protects people against the adverse impact of algorithmic systems.Footnote ¹⁶⁹ Consequently, in April 2021, it put forward a proposal for an AI Act in order to fill these legal gaps. I will therefore examine this Act next.

5.4.1 The AI Act’s Goals and Scope

5.4.2 The AI Act’s Regulatory Architecture

There are many different ways of regulating (human behaviour related to) AI systems.Footnote ²⁰⁰ The drafters of the AI Act have let themselves be inspired by product (safety) legislationFootnote ²⁰¹ instead of, for instance, legislation dealing with the protection of fundamental rights. The regulation hence treats AI as a product or service that must adhere to certain (primarily technical) requirements, meticulously set out in the regulation. The High-Level Expert Group’s recommendation to adopt a principle-based approach to AI’s regulation rather than an overly prescriptive oneFootnote ²⁰² has thus not been taken up. The legislator did take up the group’s suggestion for a risk-based approach, by distinguishing different categories of AI systems based on the extent of riskFootnote ²⁰³ they raise to health, safety and fundamental rights,Footnote ²⁰⁴ and imposes different obligations for each risk category. The regulation’s emphasis on obligations for AI providers and deployers, with only a very limited number of new rights for those subjected to the systems, further reflects its market-oriented vision.

The AI Act distinguishes fiveFootnote ²⁰⁵ categories:

1. (1) AI practices that pose an unacceptable level of risk and that are hence prohibited (with exceptions) (chapter II of the Act);

2. (2) AI systems that must comply with a set of requirements due to posing a high risk to health, safety or fundamental rights, and that must undergo a conformity assessment prior to their use or placement on the market (chapter III of the AI Act). These consist of two subcategories: a) AI systems that are (incorporated into products that are) already subjected to existing product safety legislation (Annex I of the AI Act); and (b) ‘stand-alone’ AI systems (Annex III of the AI Act);

3. (3) AI systems subjected to additional transparency obligations due to their risk of deceit or intrusiveness (chapter IV of the AI Act);

4. (4) General-purpose AI models, including a sub-category of models that pose a systemic risk due to their scale and capabilities (chapter V of the AI Act); and

5. (5) AI systems that are considered to pose only a minimal or no risk.

The last one is a residual category, including all systems and practices that are not explicitly listed under one of the other categories. These systems are not subjected to any new requirements, but can become the object of voluntary codes of conduct and guidelines (chapter X of the AI Act). AI systems that fall under the first four categories are described and listed either directly in the AI Act’s text or – in the case of high-risk systems – in annexes that can be updated over time.Footnote ²⁰⁶ The AI Act’s drafters hence coupled a risk-based approach with a list-based approach, to which I will come back later.

Categories and their requirements can overlap. Some systems can, for instance, be subjected to both the requirements imposed on high-risk systems and to the additional transparency obligations. AI providers and deployers must self-assess the category under which their AI system falls. For high-risk systems listed in Annex III, they can even self-assess whether their system – though listed in the annex – is nevertheless exempt from the high-risk requirements if it, in their view, “does not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making”.Footnote ²⁰⁷ Coupled with the fact that the conformity assessment of virtually all high-risk systems can be carried out by the systems’ providers themselves, this provides a high ‘margin of appreciation’ for the very actors the AI Act is supposed to regulate – a highly contentious point to which I will return.

Before discussing which applications of algorithmic regulation fall under the AI Act’s respective categories, let me also briefly describe how the AI Act’s requirements are enforced. The enforcement architecture of the AI Act is relatively complex, and involves several actors at different levels. The main action takes place at the national level. Member States must establish or designate an independent national competent authority (the role of which can also be undertaken by the data protection authority) to oversee the AI Act’s requirements.Footnote ²⁰⁸ These take on the role of notifying authoritiesFootnote ²⁰⁹ and market surveillance authorities,Footnote ²¹⁰ with the possibility for Member States to appoint different authorities for each taskFootnote ²¹¹ as long as there is a “single point of contact”.Footnote ²¹² The authorities’ oversight primarily takes place ex post, when an investigation reveals that an AI provider or deployer did not comply with the regulation. To coordinate the activities of the national authorities, the AI Act also establishes a European AI Board, composed of Member States’ representatives.Footnote ²¹³

While the Commission’s initial proposal was limited to the above, both the Council and the Parliament underlined the need for a stronger enforcement role at the level of the EU, especially for systems that affect the EU population at large or that cannot easily be monitored by individual states. This resulted in the establishment of an AI Office, housed at the European Commission.Footnote ²¹⁴ The AI Office is responsible for the enforcement of the requirements imposed on general-purpose AI models. Moreover, it provides the secretariat for the Board, convenes the Board’s meetings, and prepares its agenda.Footnote ²¹⁵ To further complicate the landscape of relevant actors, Articles 67 and 68 also respectively set up an Advisory Forum and a Scientific Panel of Independent Experts. The former consists of a group of stakeholders that provide expertise and advice to the Board and the Commission.Footnote ²¹⁶ The latter consists of experts that advise and support the AI Office in its enforcement tasks,Footnote ²¹⁷ for instance by alerting it of possible systemic risks posed by general-purpose AI models or by developing methodologies to evaluate their capabilities.

Finally, let me point out three more characteristics of the AI Act’s architecture. First, to expedite compliance monitoring of the Regulation’s obligations, the AI Act establishes an EU-wide database, managed by the European Commission,Footnote ²¹⁸ in which certain providers and deployers of AI systems need to register some basic information. Second, given the importance the EU attaches to AI-enabled innovation, the AI Act also provides measures ‘in support of innovation’, by setting up regulatory sandboxes in every Member State.Footnote ²¹⁹ Third, compliance with the AI Act’s requirements is facilitated by the establishment of harmonised standards (or common specifications).Footnote ²²⁰ Conformity therewith offers a presumption of conformity with the AI Act, which means that, in practice, the standards’ interpretation of the AI Act’s requirements can become the regulation’s de facto authority. This approach is not without criticism, as European Standardisation Organisations are primarily populated by industry actors and technical experts, with little participation from civil society organisations and experts with an ethical or legal background.Footnote ²²¹ More generally, one can also question whether requirements that are meant to ensure AI systems’ alignment with fundamental rights can ever be captured by a set of technical standards.Footnote ²²²

In what follows, let me now zoom in on the AI Act’s merits and pitfalls specifically in the context of algorithmic regulation, and the risks it poses to the rule of law.

5.4.3 Algorithmic Regulation in the AI Act

To what extent is algorithmic regulation as defined in this book – namely reliance on algorithmic systems to inform or take administrative acts – covered by the AI Act? At the outset, it can be noted that the AI Act does not fundamentally distinguish between systems used by the private and the public sector when it comes to the requirements it imposes on AI systems. In most cases, these requirements are the same, regardless of whether a system is deployed by a private or a public actor. There are two notable exceptions: the obligation for public authority deployers to register the high-risk systems they use in the EU database, and the obligation to carry out a fundamental rights impact assessment pursuant to Article 27 of the AI Act.Footnote ²²³ That said, applications of algorithmic regulation can be found in all of the AI Act’s categories, either explicitly (when a categorised system serves to inform or adopt administrative acts) or implicitly (when a categorised system can serve this purpose).

5.4.4 High-Risk Algorithmic Regulation

5.4.5 A Low Ceiling

Considering these shortcomings, it is worth asking whether Member States can still remedy these gaps by adopting stronger safeguards through national legislation. In essence, this question comes down to whether the AI Act, and its aim to harmonise rules on algorithmic systems across the EU to establish an internal AI market, aspires a form of minimum or maximum harmonisation. Minimum harmonisation “sets a common floor of regulation, which all Member States must respect, but it does not set a ceiling”, whereas maximum harmonisation “serves as both floor and ceiling”.Footnote ²⁸⁹ The AI Act is directly applicable in national legal orders and Member States will need to ensure that individuals can benefit from the protection it affords.

However, can Member States go further than what the regulation requires and provide stronger safeguards against AI’s adverse impact? If the AI Act seeks to ensure the maximum harmonisation of national rules, this would effectively prevent a Member State from addressing the AI Act’s shortcomings, as that would be contrary to the regulation’s objective and hence contrary to EU law.Footnote ²⁹⁰ As observed by Weatherill, this question thus expresses “a battle for the soul of the internal market”,Footnote ²⁹¹ as it essentially determines at which level regulatory power is held, and how much regulatory diversity the EU internal market can tolerate.

While the initial proposal of the AI Act still left the answer to this question somewhat ambiguous, the final version is more clear: “This Regulation ensures the free movement, cross-border, of AI-based goods and services, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation”.Footnote ²⁹² It hence appears that the AI Act is targeting the maximum harmonisation of national legislation, and that Member States cannot provide a higher level of protection by adopting stricter rules, as this may cause unwanted ‘market fragmentation’. There are only few exceptions such as the protection of workers, for which the regulation states that it “does not preclude the Union or Member States from maintaining or introducing laws, regulations or administrative provisions which are more favourable to workers in terms of protecting their rights in respect of the use of AI systems by employers”.Footnote ²⁹³ This explicit exception only seems to confirm the regulation’s ceiling-imposing nature.

In theory, a maximum harmonisation approach is meant to ensure an equal level of protection in all Member States. Yet in practice, given the AI Act’s deficiencies, this may in fact come down to an equally low level of protection of the rule of law. Certainly, the AI Act establishes EU law provisions that public authorities must comply with. It could thereby also serve as a basis to invoke EU remedies aimed at ensuring Member States’ compliance with their EU law obligations.Footnote ²⁹⁴ However, the fact that it does not impose stronger obligations on Member States to ensure their use of algorithmic regulation does not lead to algorithmic rule by law, inadvertently or intentionally, is a missed opportunity. And the fact that Member States may be legally unable to level this protection up through national legislation, unless they can argue it falls outside its scope, only makes this problem worse.

Admittedly, the AI Act does state that the harmonised rules it provides “should be without prejudice to existing Union law, in particular on data protection, consumer protection, fundamental rights, employment, and protection of workers, and product safety, to which this Regulation is complementary.”Footnote ²⁹⁵ It also mentions that a system’s classification as ‘high-risk’ should not be interpreted as indicating the lawfulness of its use under other Acts of Union law or national law implementing Union law, “such as on the protection of personal data, the use of polygraphs or the use of emotion recognition systems”.Footnote ²⁹⁶ However, the AI Act’s clear intention to comprehensively regulate systems that pose a risk to fundamental rights makes it difficult to argue that an application listed in Annex III is nevertheless unlawful. While not excluded in principle, the person claiming such unlawfulness (for instance due to an incompatibility with a fundamental right) would have to surmount a significant burden of proof, which may only be met in the case where a national law exists that explicitly sets out the system’s unlawfulness.Footnote ²⁹⁷ Moreover, in that case too, the risk still exists that an AI provider or deployer challenges the national law on internal market-based grounds, by claiming it constitutes an illegal market restriction for the very product the AI Act is meant to promote.Footnote ²⁹⁸

Finally, the maximum ceiling imposed by the AI Act also has repercussions for Member States’ participation in negotiations on AI regulation at the international level. For instance, during the Council of Europe’s negotiations of a new Convention on AI and human rights, democracy and the rule of law,Footnote ²⁹⁹ which largely took place in parallel with the negotiations of the AI Act,Footnote ³⁰⁰ the Commission requested and obtained a mandate to be the sole negotiator on behalf of EU Member States, rather than having Member States negotiating their own position.Footnote ³⁰¹ From its request, it was evident that the Commission considers that matters related to the Council of Europe’s Convention (meaning, matters related to AI’s impact on human rights, democracy and the rule of law) to already be comprehensively dealt with under the AI Act. This assumption is, however, fanciful considering the inadequate attention to the rule of law in the AI Act, as well as its underwhelming protection of fundamental rights and democratic participation in decisions on AI’s use. Nevertheless, the Commission wanted to ensure the Convention’s alignment with the AI Act, thereby also precluding Member States to advocate for stronger protection. This is despite the fact that, as already hinted at, one could legitimately wonder whether the EU can claim the competence to exhaustively regulate aspects pertaining to algorithmic regulation in public administration on an internal market legal basis.

It hence appears that the EU legislator not only left open important legal gaps in the AI Act, but that it is also preventing Member States to fill these gaps, in the name of countering market fragmentation. This begs the question whether the Union’s interest in being ‘the first’ AI legislator to reap the economic benefits of an internal market for AI are deemed more important than safeguarding its core values.

5.4.6 Evaluation: The Return of Techno-supremacy

Considering the above analysis, what should we now make of the AI Act in the context of the concerns identified in Chapter 4? Can it offer any help in countering the threat of algorithmic rule by law? Undoubtedly, the new regulation does introduce important legal safeguards to better protect individuals against the risks that some AI systems pose to their health, safety and fundamental rights. The establishment of a public enforcement mechanism; the introduction of prohibitions and requirements that must be met ex ante; as well as the inclusion of several rights for individuals that can fortify private enforcement channels are all to be welcomed. Furthermore, by imposing documentation and logging obligations that should facilitate ex post review, and by setting up an EU database to register the use of high-risk systems by public authorities, the AI Act also enhances transparency around algorithmic regulation.Footnote ³⁰²

However, overall, the protection the AI Act provides remains insufficient to meet the identified concerns. Its process is primarily based on self-assessments, and is not tailored to the specific risks arising in the context of algorithmic regulation. There are no proper mechanisms for public participation and input (for instance regarding the decision to deploy algorithmic regulation in the first place, or even just to help identify and assess the risks attached thereto); there are only limited transparency obligations towards those potentially affected by the systems; there is no obligation to make the extensive documentation of the system’s functioning accessible to researchers, civil society organisations or the public at large; there are no provisions around the procurement of systems, or limitations as regards who should be able to undertake translations from law to code, with which training, and with which constitutional checks and balances; there are no ex ante independent oversight mechanisms, nor mandatory periodic audits that can help ensure continuous oversight; and there are no provisions that enable systemic review.

These substantive shortcomings are coupled with concerns around the AI Act’s regulatory architecture. First, despite the role of the European AI Office, when it comes to public authorities’ use of AI, oversight is primarily organised at the national level, which means the extent to which people can enjoy the AI Act’s protection depends on the resources and skills of Member States. As the enforcement practice of the GDPR has shown, these vary significantly from one state to another.Footnote ³⁰³ And while national supervisory authorities ought to be independent, especially given their task to also review the actions of public authorities, experience with national data protection authorities has likewise indicated that such independence may not always be straightforward, even in countries that are not yet considered as undergoing ‘an autocratic turn’.Footnote ³⁰⁴

Second, the AI Act’s reliance on conformity assessment and technical standardisation, as part of the New Legislative Framework (NLF) approach, is inadequate to deal with the rule of law risks posed by algorithmic regulation. As briefly noted above, this is an approach the Commission typically uses in the context of EU product regulation and safety standards. As such, it makes sense to subject high-risk systems that are already part of NLF legislation (listed in Annex I, including machinery, medical devices and toys), to the same compliance mechanisms as before, with the addition of the new requirements of the AI Act. However, applying the same model of safety legislation to the high-risk systems listed in Annex III is problematic, as the risks they pose – not only to fundamental rights, but also to societal interests such as the rule of law – are not comparable. AI systems are treated as tangible products which simply need to conform to technical requirements and have a ‘CE’ marking affixed to them, rather than socio-technological systems. This approach might work reasonably well for fridges and dishwashers, yet “it is hardly appropriate for a digital technology which, on the Commission’s own account, may pose significant risk to the protection of nontangible values”.Footnote ³⁰⁵ As noted elsewhere, while the “text is infused with fundamental rights-language, it seems to take an overly technocratic approach to the protection of fundamental rights”,Footnote ³⁰⁶ by essentially translating those rights into a set of prescriptive rules, exhaustive lists, detailed technical safety standards, as well as handy templates and checklists. In addition, the reliance on harmonised standards that provide a ‘presumption of conformity’ in fact implies that the interpretation of the AI Act’s requirements is outsourced to the (primarily technical experts of) standardisation organisations,Footnote ³⁰⁷ despite their lack of representativeness and democratic accountability.Footnote ³⁰⁸

Interestingly, and sadly, parallels can hence be drawn with one of the risks pointed out in Chapter 4, namely that reliance on algorithmic regulation may exacerbate a techno-scientific approach to the law.Footnote ³⁰⁹ As noted previously, the danger associated with algorithmic rule by law is that the legal protection of individuals and the translation from open-ended legal rules to code is seen as a mere techno-scientific enterprise, to the detriment of human rights, the rule of law, and other essential values. When the regulated risk concerns the safety of machines, food or pharmaceuticals, a techno-scientific approach is neither new nor surprising. In those areas, it is common for the legislator to set out broad norms stating the objective of protecting the health and safety of individuals, which are subsequently elaborated and complemented with detailed technical requirements by domain experts who translate the objective of ‘safety’ into quantifiable, measurable and demonstrable safety standards that need to be complied with.Footnote ³¹⁰

Yet when it comes to protecting fundamental rights or the rule of law, this type of approach falls woefully short, as it is unable to do justice to the intricate contextual assessment and balance it requires between various interests. Nevertheless, this is precisely the AI Act’s approach, as it erroneously reduces “the careful balancing exercise between fundamental rights to a technocratic process, thus rendering the need for such balancing invisible”.Footnote ³¹¹ In this way, legal concepts such as the right to non-discrimination and the principle of equality are considered as elements that can be embodied by technical standards expressed in quantifiable and measurable specifications, checked by an internal ‘quality management process’, and algorithmically programmed, thereby maintaining the primacy of techno-rationality.Footnote ³¹² Worse still, the fact that this assessment and quality management occurs entirely in-house, without external oversight or ex ante accountability mechanisms, leaves these translation decisions entirely in the hands of technical experts, thus maintaining the supremacy of coders set out above.Footnote ³¹³

Lastly, the ceiling imposed by the AI Act’s maximum harmonisation means that Member States who want to offer a higher level of protection than the AI Act currently provides are practically unable to do so. Despite the regulation’s laudable intentions, and despite its introduction of valuable new safeguards, the overall picture that results from this analysis is therefore still rather bleak when it comes to the AI Act’s ability to counter the threat of algorithmic rule by law.