I. Introduction
The risks that Artificial Intelligence (AI) systems pose have prompted States to undertake regulatory measures. The EU adopted its Regulation on Artificial Intelligence (hereinafter AI Act),Footnote 1 which has been heralded as the first comprehensive framework intended to regulate AI.Footnote 2 The AI Act has already entered into force and while the implications from many of its provisions are yet to be better understood,Footnote 3 the AI Act has already been an object of useful commentaries and analysis.Footnote 4 Similarly to the EU, the Council of Europe (hereinafter CoE) has also very recently adopted the Framework Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law (hereinafter the CoE AI Convention).Footnote 5 At the time of writing the treaty has not entered into force.Footnote 6 It has been signed by eleven CoE Member States and by the EU, the United States of America, Canada, Japan, Uruguay and Israel. No State has ratified it yet.Footnote 7
The CoE AI Convention defines AI system as “machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations or decisions that may influence physical or virtual environments.” This definition aligns with the one enshrined in Article 3(1) of the AI Act. It has been considered relatively wide and meant to keep up with future technological developments.Footnote 8 With this definition in mind, this article provides an initial analysis of the CoE AI Convention. Four steps are followed. The first emphasises the necessity of understanding the CoE AI Convention within the context of its adoption as an international treaty negotiated within the CoE (Section II). This context has affected the personal scope of the treaty in terms of how it includes the regulation of the usage of AI systems by both public authorities and private actors. A detailed review of the available negotiation documents reveals that the concrete level of protection provided by the CoE AI Convention has been reduced, not least through the drafters’ final decision to regulate private actors only indirectly. Section III explains this personal scope of the Convention, including by describing the drafting negotiations concerning the regulation of public authorities and private actors. This regulation follows a risk-based approach, which shapes the obligations undertaken by States under the treaty. Section IV explains the risk-based approach and contrasts it with the approach under the EU AI Act. The argument put forward is that the treaty’s omission of explicit risk categories reflects its higher level of abstraction, which does not, however, entail weaker obligations. Section V explains the principles articulated in the Convention and intended to shape the obligations undertaken by States. It is argued that these principles, namely human dignity and individual autonomy, transparency and oversight, accountability, non-discrimination, data protection, reliability and risk management, can serve as interpretative tools for shaping the development of human rights standards.
A note on the methodology is due. As an international treaty, the CoE AI Convention is an object of treaty interpretation in accordance with the rules of interpretation articulated in Articles 31 and 32 of the Vienna Convention on the Law of Treaties (Vienna Convention).Footnote 9 Pursuant to Article 31(1) of the Vienna Convention, the treaty is to be interpreted “in accordance with the ordinary meaning to be given to the terms of the treaty in their context and in the light of its object and purpose.” Article 1(1) of the CoE AI Convention defines the object and purpose of the treaty as ensuring “that activities within the lifecycle of artificial intelligence systems are fully consistent with human rights, democracy and the rule of law.” Human rights law therefore forms an integral part for interpreting the treaty, which justifies the references to human rights in the forthcoming analysis. No attempt is undertaken to offer a comprehensive analysis as to how the provisions of the treaty could be interpreted in light of human rights law. In fact, human rights law standards are yet to develop with some higher level of precision in this area.Footnote 10 The principles expressed CoE AI Convention can help in the development of human rights law.Footnote 11 In this sense, a mutual interaction between the treaty and human rights standards can be expected.Footnote 12
According to Article 31(3)(c) of the Vienna Convention, “any relevant rules of international law applicable in the relations between the parties” can be also taken into account for interpreting the treaty. EU law and in particular the EU AI Act can be considered as a source of relevant rules. An integrated interpretative approach is therefore followed whereby not only human rights law is relevant, but more generally CoE and EU law need to be understood in light of each other.Footnote 13 There is indeed a plurality of sources that operate at European level,Footnote 14 and they need to be interpreted in an integrated way.Footnote 15 While no attempt for a comprehensive comparison is offered in this article, the EU AI Act does provide important points of reference for better understanding the CoE AI Convention. Section IV offers a more detailed comparison with the EU AI Act regarding the conditions for the classification of systems as high-risk under the Act. The reason is that this very classification can be considered as a key point of contrast at a more general conceptual level in terms of how AI systems have been regulated at European level (i.e., by the CoE and by the EU). This justifies a more in-depth comparative analysis, as offered by Section IV.
As it is the case with other CoE treaties,Footnote 16 the Explanatory Report published together with the treaty is an important source for better understanding how these obligations might be interpreted.Footnote 17 The Feasibility Study prepared by the CoE Ad Hoc Committee on AI that proposed the adoption of a new CoE treaty as an option, will serve as a benchmark for identifying the intended objectives and evaluating the extent to which they have been achieved in light of the obligations adopted in the final text. The Feasibility Study can be considered to form part of the historical background of the Convention. It can be also considered as part of the circumstances of the conclusion of treaty, which in accordance with Article 32 of the Vienna Convention on the Law of Treaties, may be used as a supplementary means of treaty interpretation.Footnote 18 The drafts of the treaty that were published during the negotiations are also included in the analysis. These drafts form part of the preparatory work of the treaty that is also a supplementary means of treaty interpretation. These means that are based on key documents such as Feasibility Study, drafts and the Explanatory Report, are used for interpreting and better understanding the principles articulated in the CoE AI Convention, namely human dignity and individual autonomy, transparency and oversight, accountability, non-discrimination, data protection, reliability and risk management.
II. The context and the drafting process
Similarly to the AI Act, the CoE AI Convention has been inspired by, one the one hand, “the unprecedented opportunities” that AI might offer,Footnote 19 including by improving the efficiency of administrative procedures,Footnote 20 and by the risks that systems might pose to human rights, on the other.Footnote 21 In contrast to the AI Act, however, the CoE AI Framework Convention is not a product safety regime.Footnote 22 The CoE AI Convention has to be understood in light of the broader mandate of the organisation within which it has been adopted. In particular, the mandate of the CoE is focused on protection of human rights, democracy and rule of law.Footnote 23 This is also clearly reflected in Article 1(1) that stipulates that the treaty aims “to ensure that activities within the lifecycle of artificial intelligence systems are fully consistent with human rights, democracy and the rule of law.”
The CoE’s mandate shaped the origins of the treaty and allowed the involvement of various actors. In particular, the origins of the treaty can be traced back to May 2019 when the CoE Committee of Ministers recognised the need for examining “the feasibility and potential elements on the basis of multi-stakeholder consultations, of a legal framework for the development, design and application of artificial intelligence, based on the Council of Europe’s standards on human rights, democracy and the rule of law.Footnote 24 In September 2019 the Committee of Ministers formally established the Ad Hoc Committee on Artificial Intelligence (CAHAI),Footnote 25 a body tasked to complete a feasibility study examining the possibility for the adoption of such a legal framework. According to the given mandate, this examination should be based on broad multi-stakeholder consultations, which could ensure the inclusion of diverse perspectives and concerns related to AI. In December 2020 CAHAI published its Feasibility Study,Footnote 26 where, as originally envisioned, multiple perspectives were presented and the foundations for a the new CoE treaty were formed. One option proposed in the Study was the adoption of a Framework Convention, as opposed to a Convention.Footnote 27 The latter imposes more concrete obligations and attributes rights to natural or legal persons.Footnote 28 A Framework Convention, on the other hand, is less prescriptive and detailed, it provides for “broad core principles and values” and leaves “a considerable margin of discretion for States as to how they implement the broader principles and values.”Footnote 29 The advantages from the latter were three: first, detailed legal obligations for AI systems can be premature; second, States would be more willing to become bound; third, rigid regulation might obstruct innovation.Footnote 30 The implications of choosing a Framework Convention will become clearer in the forthcoming analysis.
Following the completion of the CAHAI’s mandate, the Committee of Ministers established the Committee on Artificial Intelligence (CAI) with the task to “establish an international negotiation process and conduct work to finalise an appropriate legal framework on the development, design, use and decommissioning of artificial intelligence, based on the Council of Europe’s standards on human rights, democracy and the rule of law.”Footnote 31 It is important to understand the working process of CAI. This work was conducted in plenary sessions and drafting group sessions. As to the plenary sessions, there were ten such sessions.Footnote 32 The publicly available documents from these sessions offer no insights as to the positions of the different States involved.Footnote 33 The drafting group sessions were veiled with confidentiality.Footnote 34 Besides the four drafts that CAI decided to make public during the process, “the preparatory works [of the CoE AI Convention], including the positions of different states, are not publicly accessible.”Footnote 35
Even in relationship to the drafts made public during the process, there were reservations added as to the value of these drafts.Footnote 36 There were also strong criticism by CAI of any breaches of the confidentiality of the process.Footnote 37
As to the drafts made available during the drafting process, the Zero Draft was made public on 30 June 2022.Footnote 38 In February 2023 a Revised Zero Draft was published.Footnote 39 In July 2023 a Consolidated Working Draft of the Convention was published.Footnote 40 In December 2023 a draft before the final reading was published.Footnote 41 The final draft was adopted in March 2024 by the CAIFootnote 42 and later this was approved by the Committee of Ministers.Footnote 43
The Zero Draft and the Revised Zero Draft (hereafter, where relevant, referred to collectively as the Zero Drafts) were largely similar in content. Substantive changes were introduced in the Consolidated Working Draft, released in July 2023. These changes highlight the key points of contention, which will be addressed in the analysis below in relation to specific provisions. It is worth noting that, although the drafting process – as envisioned under the CoE’s mandate – involved a wide range of actors, including civil society stakeholders who provided input on the drafts,Footnote 44 the process was ultimately led and shaped by States. The EU was also involved.Footnote 45 Crucially, States that are not members of the CoE, such as USA, were also involved. Such a global reach was originally recommended by CAHAI.Footnote 46 Yet, it has been noted that the involvement of States, such as USA, diluted the safeguards and let to the exclusion of civil society.Footnote 47
The precise influence of individual States – including non-Member States of the CoE – on the drafting process is difficult to document. State positions regarding the meaning of specific treaty provisions have not been made public, which complicates interpretative efforts. It is important to emphasise, however, that these features of the drafting process – including its confidential nature – are not unique to the CoE AI Convention. Rather, they reflect broader practices within the CoE’s treaty-making framework. As such, the AI Convention should be understood within this institutional context.
For this understanding, it is relevant to observe that as early as the 1960s, a proposal to make the travaux préparatoires of CoE conventions – and, by extension, the negotiating positions of States – public was rejected.Footnote 48 Yet, at the same time, it was also agreed that “it would be appropriate to publish a report which, without revealing the attitudes of the various experts of governmental delegations during the proceedings, would be of a nature to facilitate the application of their provisions.”Footnote 49 This is how it was decided that the committee of experts (in relationship to the CoE AI Convention, this was CAI) that prepared the draft convention, would also issue an explanatory report.Footnote 50
At this point, an issue that arises concerns the role of the Explanatory Report in the interpretation of the treaty. The report does not constitute an authoritative interpretation of the text.Footnote 51 This is reflected in the second sentence of the CoE AI Convention Explanatory Report: “[t]he text of the explanatory report submitted to the Committee of Ministers of the Council of Europe” does not constitute an instrument providing an authoritative interpretation of the text of the Framework Convention although it may facilitate the understanding of it’s provisions.” Since the text of the treaty and the report are negotiated and adopted simultaneously, the report can be viewed as a supplementary means of treaty interpretation in the sense of Article 32 of the Vienna Convention on the Law of Treaties.Footnote 52 The Report can be also considered as part of the context in which the meaning of certain provisions is to be ascertained in the sense of Article 31(1) of the Vienna Convention. In the analysis that follows, the Explanatory Report, along with the available Drafts and the Feasibility Study – which together shed light on “the preparatory work of the treaty and the circumstances of its conclusion”,Footnote 53 will serve as important sources for interpreting the meaning treaty’s provisions. A crucial first step in this process is to clarify the treaty’s personal scope.
III. Scope: public authorities and private actors
1. Negotiating the scope
As the publicly available Drafts show, the personal scope of the Convention was an object of major changes. In particular, the Zero Drafts contained the concepts of “AI provider,” “AI user” and “AI subject.” These were removed with the Consolidated Working Draft. The references to “AI provider” and “AI user” allowed the Zero Drafts to directly regulate private actors, which was also reflected in the proposed personal scope of the Convention. In particular, the proposed scope contained the clarification that the Convention applies “regardless of whether these activities [design, development and application of AI systems] are undertaken by public or private actors.”Footnote 54 This direct regulation of private actors was removed with the Consolidated Working Draft. This removal is retained in the final version of the text.
The concept of “AI subject” allowed the Zero Drafts to contain multiple references to concrete rights of the “AI subjects.”Footnote 55 Relevant examples include: the right of access to the relevant records;Footnote 56 the right to human review;Footnote 57 and the right to know that one is interacting with an AI system.Footnote 58 Such concrete formulations of rights were removed with the Consolidated Working Draft.Footnote 59 This removal is retained in the final version of the text.
2. Understanding the scope
To better understand the scope of the treaty, the sections that follow clarity how the usage of AI systems by public authorities (Section III.2.a), by private actors whose conduct is attributable to the State (Section III.2.b.) and by private actors whose conduct is not attributable to the State (Section III.2.c.), is regulated.
a. Public authorities
CoE AI Convention regulates the conduct of States, not directly of private actors, which distinguishes it from the EU AI Act that regulates directly both private and public providers and deployers of AI systems.Footnote 60 As other CoE conventions that are international treaties by which States agree to adopt obligations under international law, the treaty applies to its State Parties. This implies that it applies to public authorities. Article 3(1)(a) of the treaty stipulates that “Each Party shall apply this Convention to the activities within the lifecycle of artificial intelligence systems undertaken by public authorities, […]”. This follows the rules from general international law. In particular, it corresponds to Article 4 of the International Law Commission (ILC) Articles on Responsibility of States for Internationally Wrongful Acts, which provides that “[t]he conduct of any State organ shall be considered an act of that State under international law, whether the organ exercises legislation, executive, judicial or any other functions, whatever position it holds in the organisation of the State, and whatever its character as an organ of the central Government or of a territorial unit of the State.”Footnote 61 The clarifications included in the Explanatory Report to the CoE AI Convention follow the same principle: “the Drafters’ shared understanding is that the term ‘public authority’ means any entity of public law of any kind or any level (including supranational, State, regional, provincial, municipal, and independent public entity) […].”Footnote 62
b. Private actors whose conduct is attributable to the State
Article 3(1)(a) of the treaty, however, adds that the Parties shall apply also the Convention to the activities undertaken by “private actors acting on their [public authorities’] behalf.” The Explanatory Report clarifies that this means that the Convention imposes obligations “in regard to activities for which public authorities delegate their responsibilities to private actors or direct them to act, such as activities by private actors operating pursuant to a contract with a public authority or other private provision of public service, as well as public procurement and contracting.”Footnote 63 This also follows the rules from general international law. In particular, it corresponds to Articles 5 and 8 of the ILC Articles on Responsibility of States for Internationally Wrongful Acts.
Articles 5 of the ILC Articles on Responsibility of States for Internationally Wrongful Acts provides that the activities of entities exercising elements of governmental authority are considered activities of the State, provided that these entities are empowered by the law of that State to exercise elements of the governmental authority and that they are acting “in that capacity in the particular instance.” While the ILC Articles, Article 3(1)(a) of the CoE AI Convention, and the Explanatory Report employ different terminology and concepts – creating some ambiguity – the key point is clear: when public authorities delegate tasks to private actors who then deploy AI systems, the Convention remains applicable. In such cases, it is the public authorities that bear ultimate responsibility, as the use of AI systems is attributed to the State.
As to Article 8 of the ILC Articles on Responsibility of States for Internationally Wrongful Acts, it provides that activities of private entities are considered activities of the State if these private entities act on the instructions of, or under the direction or control of the State in carrying out these activities. This aligns with the language in Article 3(1)(a) of the CoE AI Convention that refers to private entities acting on behalf of the State. It also aligns with the framing in the Explanatory Report, which refers to situations where public authorities direct private actors to undertake activities. If these activities include the usage of AI systems, the Convention applies.
In simple terms, all of the above means that there is a group of private entities whose conduct is attributable to the State because of delegation or control that implies the exercise of public powers. Any activities within the lifecycle of AI systems that these entities perform are therefore activities by the State Parties and all the obligations formulated in the CoE AI Convention apply. In this sense, the activities of these private entities are considered activities of the State. A possible contentious question here is the interpretation of the meaning of “acting on behalf” of the State authorities,Footnote 64 “prerogatives of official authority,”Footnote 65 and “delegation of their [public authorities’] responsibilities to private actors.”Footnote 66 The contention relates more generally to the meaning of public authority and of public powers.Footnote 67
c. Private actors whose conduct is not attributable to the State
Article 3(1)(b) of the CoE AI Convention addresses a second group of private entities. These are entities not covered by Article 3(1)(a) of the treaty and, in this sense, these are entities whose conduct is not attributable to the State since they do not exercise public powers. The conduct of these entities is not directly an object of regulation by the treaty. It is rather the State Parties that “shall address risks and impacts arising from” these private entities’ activities within the lifecycle of AI systems. It is the States that adopt the obligation under the treaty to “address the risks and impacts.”
States have two choices for how to fulfil this obligation. First, States can choose to apply the principles and obligations of the CoE AI Convention to private entities’ activities. This implies that States can choose to regulate at national level private actors by imposing requirements upon the latter similar to how the obligations in the treaty are framed. This could imply adopting legislation of, for example, private law nature that refers to concepts such as human dignity, individual autonomy, reliability, human rights etc., as used in the text of the treaty, to regulate private actors.
Second, States can choose to “take other appropriate measures to fulfil the obligation” to address risks and impacts.Footnote 68 This second choice seems to imply that States enjoy wider discretion in regulating risks and impacts at the national level.
Ultimately, though, the difference between the first and the second option is hard to clearly distinguish. The Explanatory Report says generally in respect to Article 3(1)(b) of the CoE AI Convention that it requires “the adoption or maintaining of appropriate legislative, administrative or other measures to give effect to this provision.” The Explanatory Report also adds that
[…], the obligation does not necessarily require additional legislation and Parties may make use of other appropriate measures, including administrative and voluntary measures. So while the obligation is binding and all Parties should comply with it, the nature of the measures taken by the Parties could vary.Footnote 69
Overall, then, private actors (that fall within the second group) are regulated indirectly “by virtue of the rights granted to, and obligations assumed by states” under the CoE AI Convention.Footnote 70 The treaty therefore can indirectly address private actors, which is very similar to how the European Convention on Human Rights (hereinafter the ECHR) can indirectly regulate private actors.Footnote 71 It can therefore be argued that the CoE AI Convention imposes positive obligations on States to regulate private actors.Footnote 72 Such positive obligations include preventive measures meant to ex ante protect individuals from harm or risk of harm.Footnote 73
IV. The risk-based approach
The obligations imposed by the treaty are shaped by a risk-based approach. According to Article 1(2) of the CoE AI Convention, the measures meant to regulate AI systems “shall be graduated and differentiated as may be necessary in view of the severity and probability of the occurrence of adverse impacts on human rights, democracy and the rule of law throughout the lifecycle of artificial intelligence systems.”Footnote 74 The phrase “adverse impacts on human rights” is noteworthy and raises the question about the difference between risks and “adverse impacts.”Footnote 75 To make things even more confusing, the Convention also refers to “potential to interfere with human rights.”Footnote 76 Potentiality seems to more closely align with the notion of risk. Ultimately, analytical confusion permeates the text of the Convention as to the role of risk in the conceptualisation of harm and the measures meant to be prevent and address harm (or risk of harm). As Section V.2. will show, this confusion existed from the very beginning of the drafting process and undermines the clarity of the obligations undertaken by States.
What is relevant at this juncture is that generally the regulatory model can be characterised as one based on risk. Therefore, similarly to the EU AI Act,Footnote 77 the CoE AI Convention has a risk-based approach: the measures “need to be tailored to the level of risk posed by an artificial intelligence system within the specific spheres, activities and contexts.”Footnote 78 The State Parties have discretion how to balance any competing interests in each sphere, activity or context.Footnote 79 The Explanatory Report notes, though, that the public sector sphere, might have some specificities that need to be taken into account.Footnote 80 Examples of these specificities are not provided; rather, the Explanatory Report gives examples of public sector areas (i.e., law enforcement, migration, border control, asylum and the judiciary), where the existence of specificities should be assumed. It also highlights that in certain spheres, there are “power asymmetries,” where certain groups within the society are in a weaker and vulnerable position, which needs to be taken into consideration.Footnote 81
Contrary to the EU AI Act, the CoE AI Convention does not create different levels of risks (e.g., unacceptable risks, high risks, minimum risks).Footnote 82 It does not therefore differentiate AI systems based on the risk that they might pose. The Zero Drafts did envision different levels of risk and measures tailored to the levels. Three levels were regulated in the Zero Draft: first, “significant levels of risk”;Footnote 83 second, “unacceptable levels of risk” that were intended to lead to full or partial ban only after the unacceptable risks have been identified and there were “no other measures available for mitigating” them;Footnote 84 third, unacceptable levels of risk that were intended to lead to direct bans of such systems.Footnote 85 These levels were removed very early in the drafting process.Footnote 86 Due to the confidentiality of the negotiating process, the precise reasons cannot be documented.
Since the CoE AI Convention does not differentiate levels of risk, the obligations imposed by the treaty are not tailored with reference to the level of risk a system might be accepted to pose. Neither is the regulation by the treaty tailored to specific sectors (such as asylum, immigration and border control).Footnote 87 Rather Chapter II of the Convention imposes general obligations that apply to all AI systems that fit within the Article 2 definition.Footnote 88 As already clarified, the addressees of these obligations are States.
The absence of defined risk levels in the CoE AI Convention is a key point of contrast with the EU AI Act and warrants a more elaborate consideration here. While a detailed examination of the AI Act’s differentiated regulation of risk levels lies beyond the scope of this analysis, it is important to acknowledge the significance of these levels – particularly given their inherent ambiguities and the potential they create for regulatory exclusions. This observation, in turn, provides a useful point of comparison with the CoE AI Convention, which also adopts a risk-based regulatory approach but does so without explicitly categorising risk levels and tailoring obligations accordingly.Footnote 89 One might initially assume that the Convention provides a lower level of protection, as it omits a high-risk category present in the EU AI Act. Nevertheless, as the subsequent analysis will illustrate, this assumption is overly simplistic given the avenues available for avoiding the high-risk designation under the EU AI Act.
Within the logic of the AI Act, risk levels play a central role. The Act has been described as aiming primarily to regulate those AI systems classified as posing high or unacceptable risk. To expose the ambiguities surrounding the risk classifications and the resulting regulatory uncertainties, particular attention can be given to the designation of “high-risk” AI systems. The AI Act does not define “high-risk” in general. Article 6 with its reference to Annex III of the Act indicates areas where the usage of AI systems can be assumed to be high-risk as a starting point.Footnote 90 Yet, a system listed in Annex III is not automatically assumed to be high-risk. More specifically, a system used, for example, in the examination of asylum claims is not assumed to be a high-risk by the mere fact that the system is used in a sensitive area, such as asylum, by public authorities that eventually take crucial decisions affecting human beings. Pursuant to Article 6(3) of the AI Act an additional threshold needs to be passed for such a system to be classified as a “high-risk.”Footnote 91
This provision stipulates that
an AI system referred to in Annex III shall not be considered to be a high-risk where it does not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making (emphasis added).Footnote 92
A contrario, an AI system assisting competent authorities in the examination of, for example, asylum claims, is classified as a high-risk system only if it poses “a significant risk of harm to the health, safety or fundamental rights of natural persons.” Here it is pertinent to clarify that it is the system itself that has to pose “a significant risk,” not the final decision taken by the public authority. The final decision (e.g., rejection of an asylum claim) might pose a risk of harm to the health, safety or fundamental rights. Yet, this is not what matters. What matters for the classification of the AI system as a “high-risk” is that the system itself when used to assist a decision poses this type of risk.
This raises difficult causation questions.Footnote 93 More specifically, it needs to be possible to demonstrate the causal link between the AI system and the risk of harm to health, safety and fundamental rights of persons. If the causal link between the AI system and the risk is shown, the system can be classified as a “high-risk” system. The question as to the relevant standard of causation is not at all addressed in the AI Act. The question as to how to measure the level of significance of the risk is also left open. Notably, Article 6(3) of the AI Act does not simply refer to risk, but only to “significant risk of harm.”
The usage of the concept of risk here is pertinent to highlight. It is not a causal link between the usage of the AI system and harm that needs to be shown, rather a causal link between the usage of the AI system and the risk of harm. This makes the causal inquiry easier since risk of harm is only a hypothetical. In then follows that since a causal link to risk is easier to show, the classification of a system as a “high-risk” system might be easier. Yet, as mentioned above, the risk needs to be “significant.” The severity threshold that is implied in the concept of “significant risk” might offset the above-mentioned relative easiness of demonstrating risk instead of demonstrating harm.
Even if only risk of harm (as opposed to harm) needs to be demonstrated, we need to have some understanding of the harm. The question then that arises is the following: Harm to what? Article 6(3) of the AI Act specifies “harm to the health, safety or fundamental rights of natural persons.” It might be difficult to imagine circumstances where health and safety are harmed, while fundamental rights are not harmed simultaneously. This makes the concurrent enumeration of these three concepts (i.e., health, safety or fundamental rights) difficult to comprehend.
It can nevertheless become comprehensible with the clarification that harm to safety and health, even if it infringes upon the important interests protected by human rights law (e.g., life, private and family life), does not necessary amount to violation of fundamental rights.Footnote 94 The reason is that even if the right to, for example, private life is infringed, this infringement can be proportionate and therefore not in violation of human rights law. This implies an inclusion of a proportionality analysis in the assessment whether the AI system poses a risk of harm. Such proportionality analysis implies highly complex and context-dependent reasoning.Footnote 95 At the end of the day therefore, the inclusion of fundamental rights in the threshold for classifying systems as high-risk, is not very helpful.
The inclusion of significant risk to fundamental rights in addition to significant risk to health and safety in Article 6(3) of the AI Act, can be interpreted to the effect that since fundamental rights do not cover only health and safety, the types of possible harms are wider. In other words, possible harms can include harms to privacy, family life, religious believes, procedural rights, etc. This expansion can be viewed in a positive light. Yet, as noted above, given that harms to interests such as privacy, family life etc. can be proportionate and thus not in violation of human rights, the expansion might not be very meaningful.
Article 6(3) of the AI Act stipulates that an AI system “does not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making.” The role of the addition, reproduced here in italics, requires closer examination. It could be interpreted as introducing the assumption that if an AI system does not materially influence the outcome – such as a decision – it should be presumed not to pose a significant risk, and therefore would not fall under the classification of a “high-risk” system.
When does an AI system “materially” influence a decision (i.e., the outcome of the decision-making)? Article 6(3) of the AI Act introduces assumptions when this is not the case. Under the following four conditions, an AI system is assumed not to materially influence the outcome and therefore not to be a high-risk system. These are when the system is “intended to perform a narrow procedural task,” “to improve the result of a previously completed human activity,” “to detect decision-making patterns or deviations from prior decision-making patterns and is not meant to replace or influence the previously completed human assessment, without proper human review,” and “to perform a preparatory task to an assessment relevant for the purposes of the use cases listed in Annex III.”Footnote 96 These four conditions are given as alternatives.
The framing of the four conditions, is open to interpretation. Whether a system performs a narrow procedural task can be a contentious question. The meaning of “narrow procedural task” is ambiguous. The same applies to the actual meaning of improving the result, not influencing and having only a preparatory task. This interpretational ambiguity can be addressed in the future via guidelines by the Commission.Footnote 97 The point here is that the conclusion that a system, for example, performs narrow procedural task, might not be that straightforward and might depend on the context.Footnote 98
To conclude, the AI Act introduces ambiguous exceptions to the classification of systems as high-risk, which ultimately appears to undermine its regulatory approach with categorisations of risk levels and the tailoring of the obligations to the levels. The drafters of the CoE AI Convention have avoided this, thereby sidestepping the above-mentioned difficulties (causation, proportionality review, contextualisation etc.) in the initial operation of classifying systems into risk levels. This can be also related to its nature as an international treaty whose provisions are more abstract and general.
Although not related to the classification of risk levels, the previously noted difficulties – regarding the role of human rights, the causal links between harm (or the risk of harm) to important interests and the use of AI systems, proportionality and contextualisation – also arise for interpreting the obligations imposed by the CoE AI Convention, as the next section will explore.
V. The obligations imposed upon states
This section explains the relevant obligations by proceeding in three steps. First, it addresses the treaty’s starting point: the express intention not to create new human rights obligations. Second, while the protection of human rights was initially identified as a gap that the CoE AI Convention was meant to fill, the lack of specification of the arguably existing human rights obligations raises doubts as to whether this gap has in fact been addressed. Finally, although the treaty does not concretise such obligations, it enshrines a number of principles specifically relevant to the operation of AI systems. These principles – human dignity and individual autonomy, transparency and oversight, accountability, non-discrimination, data protection, reliability, and risk management – are deliberately framed at a high level of generality but can nonetheless provide interpretative guidance for the future development of more concrete human rights standards and obligations.
1. Starting point and gaps meant to be addressed
To better understand the obligations imposed upon States by the CoE AI Convention, it is pertinent to underscore its starting point. The treaty does not aim to establish new human rights law obligations. This is made very clear in its Explanatory Report:
no provision of this Framework Convention is intended to create new human rights or human rights obligations or undermine the scope and content of the existing applicable protections, but rather, by setting out various legally binding obligations contained in its Chapters II to VI, to facilitate the effective implementation of the applicable human rights obligations of each Party in the context of the new challenges raised by artificial intelligence.Footnote 99
The text of Article 4 of the CoE AI Convention reflects this starting point. This provision imposes an obligation upon each State Party to “adopt and maintain measures to ensure that the activities within the lifecycle of AI systems are consistent with obligations to protect human rights, as enshrined in application international law and its domestic law.” This text does not add much specificity. It does not add much to the uncertainty regarding the answer to the question how human rights law limits and regulates AI systems.Footnote 100
To better understand the role of the CoE AI Convention and its possible added value, it is useful to go back to the Feasibility Study prepared prior to the adoption of the Convention. The Study highlights four “legal gaps”: (1) specification/concretisation of obligations; (2) identification of concrete principles specifically relevant to AI systems; (3) responding to systemic harms; and (4) responding to cross-border/transboundary nature of the harm. The following two subsections will concentrate on the first and second gaps,Footnote 101 given that most of the provisions adopted in the treaty pertain to them.Footnote 102
2. Protection of human rights
As to the first legal gap, the Feasibility Study notes that “the rights and obligations formulated in existing legal instruments tend to be articulated broadly or generally, which is not problematic as such, yet can in some instances raise interpretation difficulties in the context of AI.”Footnote 103 Concretisation of the obligations corresponding to the rights is therefore proposed. The Feasibility Study suggests that this can be attained by “specifying more concretely what falls under a broader human right and how it could be invoked by those subjected to AI systems.”Footnote 104
Here it is argued that CoE AI Convention does not really achieve such a concretisation. Article 4 of the treaty simply refers to the obligation upon States to protect human rights. The Explanatory Report to the Convention does little to clarify – and in some respects further obscures – the nature of Article 4. Namely, paragraph 38 from the Explanatory Report states that
[], parties are free to choose the ways and means of implementing their international legal obligations, provided that the result is in conformity with those obligations. This is an obligation of result and not an obligation of means. In this respect, the principle of subsidiarity is essential, putting upon the Parties the primary responsibility to ensure respect for human rights and to provide redress for violations of human rights.
This paragraph frames the obligations imposed by the treaty as obligations of result. Yet, if it is prevention of harm or risk of harm that is the result meant to be achieved, the obligations cannot be ones of result. Harm and risk of harm materialise all the time in relationship to various activities, and this harm and risk by themselves are never the sole basis for finding breach of obligations under human rights law.Footnote 105 The determination of breach is rather performed with reference to considerations of reasonableness, causation and State knowledge about the risk.Footnote 106 Reasonableness implies, for example, that even if the harm or the risk of harm was foreseeable (i.e., the State knew about it or ought to have known about it), and even if the State could have taken measures that causally could have prevented or mitigated the risk, the State might still not be in breach of its human rights law obligations. The reason is that it might not be reasonable to take measures,Footnote 107 since for example such measures might conflict with other legitimate and important interests.
If it is regulation of activities that is the result meant to be achieved, then indeed the obligation to regulate as such can be conceptualised as an obligation of result. That said, States still have discretion how to more specifically regulate, which makes the reference to a result (i.e., the result of regulation as such) not very helpful. Overall, then, not only does Article 4 of the CoE AI Convention fail to concretise, but the clarifications added to it by the Explanatory Report are confusing. As mentioned in Section II above, the Report is intended to provide some insights into the drafting process, without revealing the negotiating positions of the different States. Given the confusing formulation of the above quoted paragraph from the Explanatory Report, it can be inferred that the drafters were uncertain or could not agree how demanding and intrusive the obligations should be.Footnote 108
The overall confusion as to the role of human rights law is also evident from the various drafts of the treaty. Here Article 6(2) of the Zero Draft deserves to be quoted in full. It stipulated that the State Parties shall
ensure that the deployment and application of artificial intelligence systems in the public sector have an appropriate legal basis and that a careful preliminary consideration of the necessity and proportionality of the use of such system is carried out in view of the context of the deployment. (emphasis added).
This provision reflects the assumption that the use of AI systems in the public sector inherently implies interferences with human rights, which explains the imposition per se of the requirements for legal basis and for a necessity and proportionality assessment. It is the essence of human rights law to demand that any interference has a legal basis and that it is necessary and proportionate so that the interference does not constitute a violation.Footnote 109 In sum, the starting assumption in the Zero Draft was that the use of AI systems by public authorities has to comply with the standards of legality, necessity and proportionality, since the use per se interferes with human rights.
This starting assumption was diluted in the Revised Zero Draft to be eventually completely removed with the Consolidated Working Draft and in the final text of the treaty. To better understand the dilution, it is worthwhile to quote Article 5 of the Revised Zero Draft:
Each Party shall, within its respective jurisdiction, ensure that:
a. the application of an artificial intelligence system substantially informing decision-making by a public authority in the exercise of its function, or any private entity acting on its behalf, is fully compatible with its obligations to respect human rights and fundamental freedoms as guaranteed under its domestic law or under any relevant applicable international law;
b. any interference with human rights and fundamental freedoms by a public authority or any private entity acting on its behalf resulting from such application of an artificial intelligence system is compatible with core values of democratic societies, in accordance with the law and necessary in a democratic society in pursuit of a legitimate public interest. (emphasis added).
The addition of “substantially informing decision-making” in Article 5 of the Revised Zero Draft adds two thresholds: a causation threshold and a severity threshold. Both are reminiscent of the difficulties and ambiguities surrounding the “high risk” classification in the EU AI Act, already addressed in more detail above.
The addition of these thresholds means that the assumption is not anymore that any usage of AI systems by public authorities inherently causes harm to important interests protected by human rights law, and it is therefore inherently an interference that has to have a legal basis and comply with the standards of necessity and proportionality. There is rather a causation threshold – the decision has to be informed by the system. Such a causation threshold is also reflected in the expression “resulting from such an application.” Put it otherwise, the causal link between the usage of an AI system and any interference with human rights (i.e., harm to important interests) cannot be assumed. As to the severity threshold, it is reflected in the terms “substantially.” In other words, there is a causality standard that needs to pass a certain threshold.Footnote 110
The above quoted provisions were completely removed with the Consolidated Working Draft published on 7 July 2023,Footnote 111 a removal that, as suggested above, was related to the change and the limitation of the personal scope of the treaty (i.e., no need for specific provisions about public authorities since the Convention’s scope was limited to regulating the conduct of States). The new provision on the scope of the treaty refers to AI systems that have the potential to interfere with human rights.Footnote 112 The accepted text of the treaty also uses the same expression. In particular, Article 3(1) of the CoE AI Convention provides that “[t]he scope of this Convention covers the activities within the lifecycle of artificial intelligence systems that have the potential to interfere with human rights, democracy and the rule of law […].” The initial assumption that AI systems used by public authorities necessarily constitute an interference with human rights (and thus necessarily cause harm to important interests), was therefore replaced with the idea of potentiality to interfere. No causation or severity threshold are invoked in Article 3(1) of the Convention.
The resort to the causation and the severity threshold illustrates a general confusion and uncertainty about the role of human rights law. The Draft Framework Convention made public in December 2023,Footnote 113 whose text contains the various options available to the drafters to choose from, is very useful for showing this uncertainty. This Draft contained references to “violations of human rights,”Footnote 114 “adverse impacts on human rights,”Footnote 115 “unlawful harm or damage to the rights of individuals and legal persons,”Footnote 116 “the potential to interfere with human rights,” “significantly affecting human rights,”Footnote 117 and “impacting on human rights,”Footnote 118 All these references reveal the difficulties in articulating, first, the role of human rights, second, the causal links between AI systems used by public authorities and harm to important interests as protected by human rights law, and, third, the causal links between AI systems used by private actors and the role of the State protect these important interests.
3. Principles related specifically to the activities of AI systems
The second “legal gap” identified by the Feasibility Study is related the first one since it is also about specification. In particular, the second “legal gap” is intended to be addressed by means of explicitly legally enshrining certain principles that are specifically relevant to the operation of AI systems. These include human control and oversight, technical robustness, transparency and explainability.Footnote 119 Traceability via record-keeping and documentation of logs was also added,Footnote 120 which can enable the identification of any causal links between harms and the operation of the AI systems.Footnote 121 This in turn can enable the possibility of challenging the operation of the AI system under human rights law and the assessment whether there is any human rights law violation. Another principle that the Feasibility Study indicates as being specifically relevant to the operation of AI systems is ensuing that developers and deployers of AI systems have the necessary competence. The Feasibility Study observes that if these principles are not specifically regulated, this leads to uncertainty for both developers and deploys, which might hamper innovation.
The explicit legal enshrinement of certain principles that are specifically relevant to the operation of AI systems is intended to be achieved by Chapter III of the treaty. This Chapter enumerates principles “related to activities within the lifecycle of artificial intelligence systems.” Given their nature as principles, they are “purposefully drafted at a high level of generality” and are meant to have “very broad application to a diverse range of circumstances.”Footnote 122 As the Explanatory Report of the CoE AI Convention suggests, the drafters of the treaty assumed that the “detailed legal regime of human rights protection with its own set of rules”Footnote 123 is sufficient to further tailor the application of these general principles.
Yet, it is doubtful whether the current human rights regime can provide much specificity; it is still in the process of evolving to address the new challenges and harms to important interests arising from the use of AI systems.Footnote 124 Any obligations as corresponding to human rights are yet to be concretised. Such concretisation implies better understanding of the causal links between AI systems used by public authorities and harm to important interests as protected by human rights law, and, better understanding of the causal links between AI systems used by private actors and the role of the State protect these important interests. Such concretisation also implies proposals as to the measures (preventive or remedial) that might form the content of the human rights obligations. In the development of such proposals, the principles originating from the regulatory framework established specifically for AI systems is an important source of guidance. In this sense, it is the legal frameworks specifically regulating AI, such as the EU AI Act and the CoE AI Convention, that can help the development of human rights law.Footnote 125 Despite their high level of generality, the regulatory principles enumerated in Chapter III of the CoE AI Convention can be therefore helpful.
Being a source of guidance, these regulatory principles need to be better understood. This is the aim pursued in the following sections that consequently address the following principles: human dignity and individual autonomy, transparency and oversight, accountability, non-discrimination, data protection, reliability and finally, risk management. No comprehensive review is attempted. Rather some main contentious issues are identified in relationship to each principle.
a. Human dignity and individual autonomy
Article 7 of the CoE AI Convention enshrines the principle of respect for human dignity and individual autonomy. The Feasibility Study clarifies that “[t]o safeguard human dignity, it is essential that human beings are aware of the fact that they are interacting with an AI system and not misled in this regard.” The Study adds that “they should in principle be able to choose not to interact with it, and to not be subject to a decision informed or made by an AI system whenever this can significantly impact their lives, especially when this can violate rights related to their human dignity.”Footnote 126 As suggested in Section III.1. above, the Zero Drafts did contain specifically formulated rights to this effect.Footnote 127
The Explanatory Report does not go that far. It clarifies among other things that respect to human dignity implies that individuals are not reduced to “mere data points.”Footnote 128 According to the Explanatory Report therefore, individual autonomy does imply an ability to make choices and decisions and that individuals have “control over the use and impact of artificial intelligence technologies in their live.”Footnote 129 The possibility to refuse to be subjected to a decision-making process by a public authority that might use an AI system, is however not mentioned as part of the conceptualisation of “individual autonomy” in the Explanatory Report to the treaty.Footnote 130 Nevertheless, human dignity and individual autonomy may provide important guidance for the contextual interpretation of human rights obligations. In certain contexts, the content of these obligations could be understood to include the right not to engage with an AI system or not to be subject to decisions generated by such systems.
b. Transparency and oversight
The second principle enshrined in the CoE AI Convention is transparency and oversight. In particular, Article 8 of the treaty stipulates that
Each Party shall adopt or maintain measures to ensure that adequate transparency and oversight requirements tailored to the specific contexts and risks are in place in respect of activities within the lifecycle of artificial intelligence systems, including with regard to the identification of content generated by artificial intelligence systems.
A review of the different drafts related to transparency and oversight shows that there were no changes as to proposed formulations, which suggests that the final formulation of Article 8 of the treaty was not contentious. The only notable addition is the one made with the December 2023 Draft,Footnote 131 where identification of content generated by AI systems was added to the text. This was a positive addition, as also reflected in the actual text of Article 8 of the Convention, since it ensures that identification of such content is part of the transparency and oversight requirement. Although explainability is not mentioned in the text of the treaty,Footnote 132the Explanatory Report suggest that explainability is part of the principle of transparency. The Report clarifies that explainability
[.] refers to the capacity to provide, subject to technical feasibility and taking into account the generally acknowledged state of the art, sufficiently understandable explanations about why an artificial intelligence system provides information, produces predictions, content, recommendations or decisions, which is particularly crucial in sensitive domains such as healthcare, finance, immigration, border services and criminal justice, where understanding the reasoning behind decisions produced or assisted by an artificial intelligence system is essential. In such cases transparency could, for instance, take the form of a list of factors which the artificial intelligence system takes into consideration when informing or making a decision.Footnote 133
As noted above, the Explanatory Report lacks binding authority; nonetheless, the above paragraph presents meaningful clarifications that offer interpretative value. As the text of Article 8 and the Explanatory Report suggest, certain “sensitive domains” can be identified, where transparency and oversight measures have to be tailored. This can imply more robust requirements. As clarified in the Explanatory Report, this tailoring could involve, among other things, enhancing transparency regarding the inputs or criteria considered by the AI system.
c. Accountability/responsibility, remedies and oversight mechanisms
Transparency can enable the identification of harms and the identification of actors that might have caused this harm, which is turn is indispensable for ensuring responsibility.Footnote 134 The CoE AI Convention thus specifically addresses accountability and responsibility.Footnote 135 In particular, Article 9 of the treaty is generally framed in the following way:
Each Party shall adopt or maintain measures to ensure accountability and responsibility for adverse impacts on human rights, democracy and the rule of law resulting from activities within the lifecycle of artificial intelligence systems.
This might demand the adoption of new legal frameworks at national level or adaptation of the existing judicial, administrative, civil or other national liability regimes.Footnote 136 Such responsibility regimes are inherently linked with the right to effective remedy enshrined in Article 14 of the CoE AI Convention. Similarly to Article 8 of the treaty (transparency and oversight), Article 14 requires measures for documenting relevant information regarding AI system.Footnote 137
There are qualifiers, though: such documentation is required when the system has “the potential to significantly affect human rights.”Footnote 138 One can wonder what “significantly” means in this context and who decides whether the harm is significant or not. The same problem therefore emerges as the one discussed in Section IV above in relationship to Article 6(3) of the EU AI Act that excludes systems from the high-risk classification where they do not “pose a significant risk of harm to the health, safety or fundamental rights of natural persons.”
One difference in articulation is the reference to risk in the AI Act provision. As observed in Section IV, the inclusion of risk weakens the required causal link between the system and the harm. As the available drafts of the CoE AI Convention reveal, challenges emerged in the drafting process how to articulate the causal links between AI systems and harm (or potential harm) to important interests protected by human rights law.Footnote 139 These challenges inevitably affect the possibilities for remedies and establishment of responsibility.
Addition qualifier comes with Article 14(2)(b) of the CoE AI Convention: States have to adopt measures to ensure that the information about the AI system is “sufficient for the affected persons to contest the decision(s) made or substantially informed by the use of the system, and where relevant and appropriate, the use of the system itself.” The following questions are left open: Who decides when the information is sufficient? and Who decides whether the decision is substantially informed by the use of the system? Finally, no clear and straightforward obligation is imposed to inform about the use of the system itself. The commentary on Article 14 offered by the Explanatory Report seems to justify all these qualifications in the following way:
It is also important to recall that exceptions, limitations or derogations from such transparency obligations are possible in the interest of public order, security and other important public interests as provided for by applicable international human rights instruments and, where necessary, to meet these objectives.Footnote 140
Article 15 of the treaty strengthens the right to effective remedies by adding procedural safeguards. In particular, this provision stipulates that “where an artificial intelligence system significantly impacts upon the enjoyment of human rights, effective procedural guarantees, safeguards and rights, in accordance with the applicable international and domestic law” shall be ensured. Similarly to what was mentioned above, a threshold of significance (i.e., “significantly impact, significantly affect”) is imposed.
This leads to the creation of two tiers of human rights harms: (1) normal and (2) significant. Human rights law generally does not incorporate such tiers and the right to effective remedy applies irrespective of the significance of the harm. In particular, once the definitional threshold of a right is engaged (i.e., it is determined that the interests protected by the right are impacted/harmed/interfered with),Footnote 141 remedies with procedural guarantees need to be ensured.Footnote 142 Indeed, the significance of the harm in terms of severity might imply more robust procedural guarantees. This is relevant since, for example, the right to private life is interpreted very generously. It is also relevant since the remedial procedural obligations corresponding to, e.g., Article 8 of the ECHR (private life) can be less robust in comparison to those corresponding to Article 3 of the ECHR (torture, inhuman and degrading treatment), in light of the different severity thresholds.Footnote 143 Yet, the key point remains that remedies and procedural safeguards are required in cases of both minor interferences and significant interferences with the enjoyment of human rights. There might only be variations in the robustness of the procedural guarantees.
Article 15(2) of the CoE AI Convention has an additional weakness. Its texts says that “Each Party shall seek to ensure that, as appropriate for the context, persons interacting with artificial intelligence systems are notified that they are interacting with such systems rather than with a human.” The requirement to “seek to ensure” is weaker than “to ensure.” The robustness of the requirements for notification is further undermined by the qualification “as appropriate for the context.” It opens the possibility for an argument that notification in some contexts is not appropriate. Without notification, the affected person cannot know that an AI system has been used, which can undermine the possibility of subjecting the system to any scrutiny including via legal channels for establishing responsibility.
Finally, any discussion of responsibility must take into account the extent to which effective oversight mechanisms are available at domestic level in each State Party.Footnote 144 More generally, such mechanisms can include, national data protection authorities, equality bodies, human rights institutions with a general mandate or bodies specifically designated to oversee the application of AI systems. National legal systems differ as to the role of these mechanisms, including their standing and possible involvement in legal proceedings for establishment of responsibility.Footnote 145 And indeed, the CoE AI Convention does not link, on the one hand, Article 26(1) that obliges the State Parties to “establish or designate one or more effective mechanism to oversee compliance with the obligations of this Convention” with, on the other, Articles 14 (remedies) and 15 (procedural safeguards).Footnote 146 Although there is a clear obligation to have such mechanisms, States have discretion to determine their role.
Their role was weakened in the process of drafting the treaty. The Zero Draft contained the following proposed provision: “Parties shall establish or designate national supervisory authorities tasked, in particular, with overseeing and supervising compliance …”Footnote 147 As Article 18(2) of the Zero Draft suggested, supervising would imply a procedure that might lead to banning of systems. The possibility for banning of systems as part of the national authorities’ supervising procedures were removed with the Revised Zero Draft.Footnote 148 Further modifications were introduced with the Consolidated Working Draft, where the reference to “authorities” was removed. The more general framing of “oversight mechanisms” was adopted.Footnote 149 The Draft Framework Convention published on 18 December 2023 removed the reference to supervision.Footnote 150 In this way Article 26(1) of the treaty took its final shape.
Although the Parties have discretion as to the type of mechanisms and the cooperation between them (if there are multiple national mechanisms), Article 26(2) of the Convention imposes two important requirements: first, independence and impartiality of the mechanisms, and second, conferral of power, expertise and resources for effective fulfilment of their overseeing tasks. The Explanatory Report clarifies that the first requirement implies “a sufficient degree of distance from relevant actors within both executive and legislative branches.”Footnote 151 No distance from private actors is mentioned, which could be problematic given their possible influence and role in the setting of standards and technical specifications.Footnote 152
d. Non-discrimination
Article 10(1) of the CoE AI Convention imposes an obligation to adopt measures for ensuring non-discrimination “as provided under applicable international and domestic law.” Article 10 should be read together with Article 17 of the treaty that stipulates that “[t]he implementation of the provisions of this Convention by the Parties shall be secured without discrimination on any ground, in accordance with their international human rights obligations.” These non-discrimination provisions do not add further specificities. As the Explanatory Report clarifies, the Drafters’ intention was to “refer specifically to the body of the existing human rights law.”Footnote 153 The intention was not to create new human rights obligations.Footnote 154 Whether existing obligations sufficiently address discrimination by AI systems is a matter of ongoing debate. While limitations have been identified, possibilities for the development of non-discrimination law have been also acknowledged as available.Footnote 155 There is a growing body of research in this area.Footnote 156 Article 10 of the CoE AI Convention, though not detailed, can offer a foundation that could support the advancement of anti-discrimination norms in this domain.
An interesting feature of the CoE AI Convention is its implicit engagement with what may be conceptualised as structural discrimination, as reflected in Article 10(2). Initially, the Zero Drafts referred to “rights related to discriminated groups and people in vulnerable situations.”Footnote 157 The Consolidated Working Draft did not contain references to “people in vulnerable situations.” It rather included a new paragraph that “called upon [the Parties] to adopt special measures or policies aimed at eliminating inequalities and achieving fair, just and equal outcomes,” in line with domestic and international obligations.Footnote 158 The final version of Article 10(2) stipulates that “Each Party undertakes to adopt or maintain measures aimed at overcoming inequalities or achieve fair, just and equitable outcomes, in line with its applicable domestic and international human rights obligations.” The Explanatory Report clarifies that with the treaty the State Parties aim at “overcoming structural and historical inequalities.”Footnote 159 However, States preserve flexibility how to overcome these inequalities.
e. Data protection
Similarly to Article 10 (non-discrimination), Article 11 of the treaty that is about privacy and personal data protection, remains formulated at a relatively abstract level. This provision imposes two types of obligations. The first one is substantive – adoption or maintenance of measures to ensure that privacy and personal data are protected, “including through applicable domestic and international law, standards and frameworks.” The second one is procedural – adoption or maintenance of measures to ensure effective guarantees “in accordance with applicable domestic and international legal obligations.” The publicly available drafts suggest that the incorporation of these substantive and procedural obligations was widely accepted and remained uncontested during the negotiations.
While the Explanatory Report clarifies that States have discretion as to the substantive and the procedural measures,Footnote 160 it reveals that the key role of data protection laws was underscored during the drafting. Specific references are made to the CoE Convention on the protection of personal data (Convention 108+)Footnote 161 and the EU General Data Protection Regulation (GDPR).Footnote 162
In the existing literature, AI systems have been reviewed from the perspective of data protection. Data protection law has imposed certain standards and requirements.Footnote 163 Such a perspective is very relevant since the development or the use of AI systems implies the processing of personal data.Footnote 164 The regulation of data protection is important with its detailed rules for lawful data processing and the rights of data subjects.Footnote 165 The GDPR sets out a comprehensive framework of principles regarding data protection, including the principles of transparency, purpose limitation and data minimisation to be applied when processing personal data.Footnote 166 Article 22(1) of the GDPR prohibits individual decisions from being taken in a fully automated manner.Footnote 167 One innovation brought with Convention 108+ is the conferral of a right upon individuals “not to be subject to a decision significantly affecting him or her based solely on an automated processing of data without having his or her views taken into consideration.”Footnote 168
Despite the importance of data protection laws, it has also been acknowledged that AI systems raise problems that transcend data protection.Footnote 169 In addition, data protection law has its own limitations.Footnote 170 The detailed nature of these limitations has attracted growing academic attention.Footnote 171 While data protection law has certain limitations, its focus on the rights of individuals whose data is processed, along with the evolving judicial developments,Footnote 172 ensures that it will continue to play a key role. As formulated, Article 11 of the CoE AI Convention, is intended to capture these developments through its reference to both domestic and international legal standards.
f. Reliability
Article 12 of the CoE AI Convention imposes an obligation upon States to take measures “as appropriate” to promote reliability.Footnote 173 These measures are meant to ensure “adequate quality and security” of the AI systems. The Explanatory Report suggests that these measures include adoption of standards, technical specifications, assurance techniques and compliance schemes. The Explanatory Report adds that Article 12 of the treaty is “based on the assumption that the Parties are best placed to make relevant regulatory choices.”Footnote 174 This corresponds to the wide scope of discretion afforded to the State Parties under the treaty.
For ensuring reliability, there is a strong emphasis on standards and technical specifications. The advantage of standards is that they are concrete, in this way increasing clarity and certainty. At the same time standardisation as a method of regulation has been also criticised since it implies delegation of rule-making powers to non-representative or private bodies, which is not democratic and lacks legitimacy.Footnote 175 The Drafters of the CoE AI Convention have been sensitive to this problem. Paragraph 88 of the Explanatory Report reveals this sensitivity:
In some cases, it may not be enough to set out standards and rules about the activities within the lifecycle of AI systems. Measures to promote reliability may therefore include, depending on the context, providing relevant stakeholders with clear and reliable information about whether artificial intelligence actors have been following those requirements in practice. This means ensuring, as appropriate, end-to-end accountability through process transparency and documentation protocols. There is a clear connection between this principle and the principle of transparency and oversight in Article 8 and the principle of accountability and responsibility in Article 9 (emphasis added).
While conformity with established standards may establish a presumption of regulatory compliance,Footnote 176 the obligation imposed by Article 12 of the CoE AI Convention of ensuring adequate quality and security requires more than standard-setting. It demands possibilities for questioning and revoking the compliance presumption. For these possibilities to be available there should be transparency and access to information. As noted in Section V.3.c. above, transparency can enable the identification of harms and the identification of actors that might have caused this harm, which is turn is indispensable for ensuring responsibility.Footnote 177 There is therefore an interconnection between the obligations for reliability and remedies.
g. Risk management
Article 16 of the CoE AI Convention stipulates that the State Parties shall adopt measures for the “identification, assessment, prevention and mitigation of risks posed by artificial intelligence systems by considering actual and potential impacts to human rights, democracy and the rule of law.”Footnote 178 Such measures shall be “graduated and differentiated.” States are therefore afforded flexibility in deciding which measures to implement, both in general terms and in relation to the specific domains where AI systems are used.Footnote 179 As clarified above, in contrast to the EU AI Act, the CoE AI Convention does not identify certain domains where the application of AI systems is considered high-risk.
Article 16(2) of the treaty elaborates on the required measures by specifying what they should take into account in light of the principles of graduation and differentiation. Specifically, the measures should consider “severity and probability of potential impacts” and “the perspective of relevant stakeholders.” The provision also introduces a temporal dimension, stating that these measures should apply “iteratively throughout the activities within the lifecycle of the AI system.” In addition, Article 16(2) clarifies the nature of the required measures: “monitoring for risks and adverse impacts to human rights, democracy, and the rule of law,” “documentation of risks, actual and potential impacts, and the risk management approach,” and “where appropriate, testing of AI systems before making them available for first use and when they are significantly modified.”
Notably, the conceptual clarity of Article 16 is undermined by its inconsistent terminology. It refers variously to “risks to human rights,” “adverse impacts to human rights,” generic “risks,” and “actual and potential impacts” – not all of which are explicitly tied to human rights. This inconsistency renders the article conceptually unclear. Further compounding the issue, Article 16(3) introduces the phrase “incompatible with the respect for human rights,” without clarifying its relation to the terms used in the preceding paragraph in the same article.
Article 16(3) ensures discretion for the State Parties how to respond to the risks that might be identified. No obligation for banning system is imposed; rather, the State Parties adopt the procedural obligation to “assess the need for a moratorium or ban.”
VI. Conclusion
Overall, the provisions of the CoE AI Convention reflect a wide scope of flexibility and high level of abstraction, which leaves discretion to its State Parties. This aligns with the design of the treaty as a Framework Convention that does not confer concrete rights upon individuals. This is a feature that distinguishes it from, for example, the CoE Convention Convention 108+ for the Protection of Individuals with Regard to the Processing of Personal Data. The choice of a Framework Convention was intended to inter alia ensure the participation of wide range of States and more willingness by States to become bound.
The high level of abstraction can be also linked to the choice of not incorporating concrete risk levels and tailoring regulations to the levels. In this respect, the Convention differs from the EU AI Act. Yet, given the ambiguities in the formulations of the risk levels and the possibilities for exclusions in the AI Act, the more general approach in the Convention does not necessary imply less robust regulation.
The regulatory impact of the Convention should be assessed in light of its interaction with human rights law, particularly given the specific mandate of the CoE – the institutional context in which the treaty was proposed and adopted. The Convention seeks to facilitate the effective implementation of human rights obligations,Footnote 180 which are still evolving in relation to the use of AI systems. Although the link between human rights law and the harm caused by such systems is not clearly conceptualised in the treaty’s text, the principles set out in Chapter III offer interpretative guidance for the further development of human rights norms in this area. For instance, the principles of human dignity and individual autonomy may play a central role in shaping the contextual interpretation of human rights obligations. Article 10 of the Convention, though not detailed, can serve as a foundation for advancing anti-discrimination norms in the context of AI. Similarly, Article 11 reflects ongoing developments in the field of data protection and may inform the articulation of legal claims involving the use of AI systems. Overall, although the review of the available negotiation documents indicates a reduction in the level of protection and in the scope and specificity of the obligations formulated in the treaty, the principles it articulates – human dignity and individual autonomy, transparency and oversight, accountability, non-discrimination, data protection, reliability, and risk management – can nonetheless provide interpretative guidance for the future development of human rights standards in the field of AI.
Open access
