1 Introduction
Based on the case study of the profiling algorithm implemented in Public Employment Services (PES) in Poland, this article addresses the relationship between the law and its computer articulation understood as a conversion of legal norms into algorithms. Understanding how the law is articulated by computer means is important in the context of increasing use of algorithmic decision-making systems (ADMs) in public policies, including education (Broussard Reference Broussard2020), justice (Harcourt Reference Harcourt2005), criminal justice (Chiao Reference Chiao2019), tax, social security (Scarcella and Lexer Reference Scarcella and Lexer2019) or – as in our case – algorithmic profiling applied within labour market policy (LMP) (Desiere et al. Reference Desiere, Langenbucher and Struyven2019). Governments and public institutions employ ADMs to enforce the law and automate or inform decision-making processes regarding verification of entitlements, access to public services, fraud detection, risk scoring and need classification (Alston Reference Alston2019). More recently, there have also been experiments with artificial intelligence (AI)-based applications – for instance, in the Flemish PES (Desiere and Struyven Reference Desiere and Struyven2021) and the French National Family Benefits Fund (Dubois et al. Reference Dubois, Paris, Weill, Barrault-Stella and Weill2018).
The deployment of ADMs in the public sector is legitimised with promises of objective, cost-effective and efficient policies (e.g. Snellen Reference Snellen, Snellen and Van De Donk1998). As Bovens and Zouridis (Reference Bovens and Zouridis2002) and Hildebrandt (Reference Hildebrandt2018, p. 1; Reference Hildebrandt, Hildebrandt and Gaakeer2013) critically summarise this techno-optimistic way of thinking: ADMs are believed to constitute ‘the zenith of legal and rational authority’, in which technical means replace the imperfect control of the public administration, eliminate or at least heavily curtail discretionary decision-making of its staff, known as street-level bureaucrats (Lipsky Reference Lipsky2010), decrease costs of public agencies and render ‘legislation … self-executing’ thanks to its computer representation (see also Buffat Reference Buffat2015).
Many scholars criticise this narrative, arguing that the relationship between law and ADMs is not as straightforward as the concept of computer representation of law suggests. Bovens and colleagues (Reference Bovens and Zouridis2002), Zouridis et al. (Reference Zouridis, Bovens, Van Eck, Evans and Hupe2020), like Hildebrandt (Reference Hildebrandt, Hildebrandt and Gaakeer2013, Reference Hildebrandt2018) and others (Ustek-Spilda Reference Ustek-Spilda2020), draw attention to the discretion of statisticians, data analysts and programmers, who interpret and translate the enacted law into ADMs. These so-called ‘system-level bureaucrats’ (Bovens and Zouridis Reference Bovens and Zouridis2002) or ‘back-office policy-makers’ (Ustek-Spilda Reference Ustek-Spilda2020) gain power at the expense of the legislature, the executive and judiciary, but nevertheless escape political control and accountability. Hildebrandt (Reference Hildebrandt, Hildebrandt and Gaakeer2013, Reference Hildebrandt2018) additionally emphasises the incompatibility of the natural and computer languages as a source of possible discrepancies between the law and its translation in the form of algorithms. All these scholars call for research that will shed light on how law is translated into ADMs, and ponder the creation of a new system of checks and balances that will secure citizens against possible harms and risks resulting from the deployment of ADMs (Bovens and Zouridis Reference Bovens and Zouridis2002; Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013; Alkhatib and Bernstein Reference Alkhatib and Bernstein2019; DoCarmo et al. Reference DoCarmo, Rea, Conaway, Emery and Raval2021), which becomes a pressing issue in the light of high-profile scandals (e.g. Angwin et al. Reference Angwin, Larson, Mattu and Kirchner2016; van Bekkum and Zuiderveen Borgesius Reference van Bekkum and Zuiderveen Borgesius2021; Rachovitsa and Johann Reference Rachovitsa and Johann2022).
While questions of the harms and risks resulting from the deployment of ADMs in the private sector and the need for a new system of checks and balances have received considerable attention, little is known about how the law is articulated by computer means, what discrepancies between the law and ADMs are produced and how. This paper contributes to this debate by exploring the relationship between law and ADMs through an empirical case study. Our analysis is based on the controversial case of a profiling algorithm that was deployed between 2014 and 2019 as a decision support system for the staff of Polish PES. Its role was to advise case-workers on how to ‘sort people out’ (Bowker and Star Reference Bowker and Star1999; Garsten and Jacobsson Reference Garsten and Jacobsson2013) into three groups with differing obligations and rights in terms of access to active labour market programmes (ALMP). This profiling caused a great deal of controversy and was withdrawn by a subsequent government after the Constitutional Tribunal ruled that it violated data protection standards on procedural grounds (Kuba and Staszewska Reference Kuba and Staszewska2022). Our analysis includes a close comparison of legal and algorithmic frameworks and a reconstruction of the decision-making processes that led to discrepancies between the law and ADM.
The results of this research are twofold. First, we demonstrate discrepancies between the legal provisions that constituted the legal basis for the development of the profiling algorithm and the algorithm itself. We argue that these discrepancies – which concern the area of social and employment rights (Eubanks Reference Eubanks2017; Alston Reference Alston2019; Dencik and Kaun Reference Dencik and Kaun2020; Niklas and Dencik Reference Niklas and Dencik2021; Niklas Reference Niklas2022) – are indicative of backstage discretionary decision-making. They go far beyond what is assumed by the literature as a necessary by-product of representation or translation of law, which requires interpretation of the written legal text and its adaptation to the algorithmic system. Second, we reconstruct how these discrepancies came into being. We conclude that – surprisingly – discretion during the development of the profiling algorithm was to an important extent exercised by traditional policy-makers, namely the executive power represented by the Ministry of Labour and Social Policy (henceforth: the Ministry) and PES, rather than by ‘system-level bureaucrats’, as the literature would suggest.
To provide this empirical insight, this article consists of the following parts. Section 2 shows how our analysis is grounded in the existing scholarly literature on computer representation and translation of law. Section 3 provides background information concerning labour market policies (LMP) in Poland. Section 4 describes our data and methods. Section 5 presents discrepancies between the law and ADM, and the decision-making process that led to their creation. The last section concludes.
2 The relationship between law and algorithms as representation or translation
In current scholarship, there are two main strands of thought when it comes to the relationship between law and its enaction through algorithms. The first is enthusiastic about the articulation of law through code and discusses the relationship between the two in terms of representation. Its representatives (e.g. Kowalski and Sergot Reference Kowalski and Sergot1985; Bench-Capon and Coenen Reference Bench-Capon and Coenen1992) see decision-making by ADMs as a chance to implement law in an objective, efficient and economical way. Algorithmic decision-making is contrasted with arbitrary, biased and discriminatory decision-making by street-level bureaucrats (e.g. Snellen Reference Snellen, Snellen and Van De Donk1998), which is symptomatic of a general crisis of the legitimacy of public administration (Calo and Citron Reference Calo and Citron2021).
Typically, within the representation approach, legal knowledge-based systems are portrayed as superior to law, because they are ‘unambiguous’ – in the words of Bench-Capon and Coenen (Reference Bench-Capon and Coenen1992, p. 71) – ‘whereas the [legal] source may well contain ambiguities […] latent in almost any piece of legislation written in natural language’. Representing law with formal language equals getting rid of its ambiguities, while at the same time keeping track of choices made along the way (Reference Bench-Capon and Coenen1992, pp. 69 and 71; Allen and Saxon Reference Allen, Saxon and Nagel1991). Recent policy-making initiatives, like the case of digital-ready legislation in Denmark, propose to go one step further and reduce ambiguities during the legislative process by simplifying rules and specifying terminology, so that law becomes easily representable and enforceable by ADMs (for critical analysis, see Plesner and Justesen Reference Plesner and Justesen2022).
As pointed out already in 1985 by Kowalski and Sergot (Reference Kowalski and Sergot1985, p. 1270), computer articulation of legal rules raises a fundamental question: ‘How can we be sure that the computer representation of laws is accurate; and, therefore, that the conclusions they imply are correct?’ Since then, there have been several initiatives aimed at securing ‘faithful’ computer representation of law. For instance, Bench-Capon and Coenen (Reference Bench-Capon and Coenen1992) recommended ‘isomorphism’ as a method that enables the maintenance of a structural correspondence between the source documents (i.e. legislation) and their representation within the computer system. A major advantage of isomorphism is that it makes the system easier to adapt to the amendments of law or court verdicts that change its binding interpretation. Such adaptation is possible because the ‘precise links between source [i.e. legislation] and knowledge base [i.e. its computer representation]’ are kept and ‘the structure of the knowledge base is a faithful reflection of the structure of the sources’ (Bench-Capon and Coenen Reference Bench-Capon and Coenen1992, pp. 72–3).
In summary, not only the state of development of ADMs, but also their further adaptation is crucial for law to be appropriately represented. The stakes are high: losing the connection between the law and its computer representation results in breaking the principle of legality (Suksi Reference Suksi2021, p. 104), according to which actions of administrative bodies must be founded on the enacted laws. Constitutional legitimacy and the principle of legality are also the reason why some legal scholars warn against the use of AI: machine learning-based systems cannot be adapted to directly incorporate new legislation or jurisdiction (Suksi Reference Suksi2021). However, simple, rule-based algorithms – as the one analysed in our case study – seem less problematic in this respect, at least in theory.
The second strand of thought relevant for our study is sceptical towards the possibility of ‘faithful’ computer representation of law. Instead of representation, the relationship between law and code is conceptualised in terms of translation (Bovens and Zouridis Reference Bovens and Zouridis2002; Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013, pp. 5–6). This approach underlines three problematic aspects: the inevitable changes that are made during translation, the decision-making power that involves value-laden choices wielded by actors tasked with translation and the complex organisation of processes through which the code is created.
Translation is by definition an adaptation. To investigate it, Hildebrandt (Reference Hildebrandt, Hildebrandt and Gaakeer2013, p. 5) proposes to juxtapose the enacted, written rules with their computational articulation in order to identify ‘subtle or substantial transformations of the substance of rule’ and what is ‘lost in translation’. One of the reasons for discrepancies between written rules and code is their different form. As explained by Kitchin (Reference Kitchin2017, p. 17):
‘Coding […] consists of two key translation challenges centred on producing algorithms. The first is translating a task or problem into a structured formula with an appropriate rule set (pseudo-code). The second is translating this pseudo-code into source code that when compiled will perform the task or solve the problem. Both translations can be challenging, requiring the precise definition of what a task/problem is (logic), then breaking that down into a precise set of instructions, factoring in any contingencies such as how the algorithm should perform under different conditions (control).’
Since the law, as with any text, is prone to interpretation and not always structured according to the ‘if …, then …’ logic, translating means modifying rather than merely representing. Moreover, this modification implies redefinition of decision-making criteria. Hence, it raises concerns about whether criteria inherent in ADMs become legally or morally problematic as discriminatory, too simplistic or simply irrelevant (Barocas and Selbst Reference Barocas and Selbst2016; Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013; Oswald Reference Oswald2018; Yeung Reference Yeung, Yeung and Lodge2019).
Moreover, for Hildebrandt (Reference Hildebrandt, Hildebrandt and Gaakeer2013), Bovens and Zouridis (Reference Bovens and Zouridis2002) and Yeung (Reference Yeung, Yeung and Lodge2019) the processes of simplifying and reducing of ambiguities of a legal text are – in contrast to the representational approach – not necessarily positive. In their optics, imposition of one particular interpretation of a legal norm carries the risk of excessive rigidity, endangers due process by limiting the possibility of accounting for specific circumstances of the case and therefore reduces responsiveness of public administration (see also Alkhatib and Bernstein Reference Alkhatib and Bernstein2019).
Importantly, there is hardly any research showing how computer articulation modifies law, because access to the code is often restricted on the grounds of anti-fraud and commercial secrecy (Pasquale Reference Pasquale2015, p. 4; Busuioc Reference Busuioc2020), and cannot be deciphered without specific skills and knowledge (Kemper and Kolkman Reference Kemper and Kolkman2019; Bucher Reference Bucher2018, pp. 47–8). In this article, we use the rare opportunity of access to a simple algorithmic system to analyse how the code diverged from the legal rules it was supposed to embody.
Finally, a fixed interpretation within the computer system also invisibly modifies the established system of checks and balances founded on the separation of powers. Bovens and Zouridis (Reference Bovens and Zouridis2002) coined the term ‘system-level bureaucrats’ to draw attention to the growing power of new unaccountable actors translating law into algorithms: statisticians, programmers or data analysts. Rather than play a role similar to that of public administration and interpret legal rules in the light of individual cases, system-level bureaucrats act similarly to legislators by ‘drafting and composing the rules themselves’ (Bovens and Zouridis Reference Bovens and Zouridis2002, p. 181). With the implementation of both fully automated and decision support systems in welfare institutions, translation of law increasingly affects ‘who gets what, when and how’ – to cite a well-known definition of politics (Lasswell Reference Lasswell1936). Thus, interpretations of law inscribed in the computer systems are politically relevant and those who effectively shape the code can be regarded as new policy-makers and their decision-making as a form of politics.
For these reasons, Zouridis and colleagues (Reference Zouridis, Bovens, Van Eck, Evans and Hupe2020) call for more research on how the law is translated into ADMs and the role played by system-level bureaucrats. The rare existing studies in this respect show their growing political influence at the expense of the parliament and judiciary (Oskamp and Tragter Reference Oskamp and Tragter1997) and point to the complexity and fragmentation of decision-making processes that involve experts in law and mathematical modelling (Plesner and Justesen Reference Plesner and Justesen2022). Zouridis and colleagues (Reference Zouridis, Bovens, Van Eck, Evans and Hupe2020, p. 327) assume that the actual decision-making power of the executive also diminishes. This assumption should be, however, treated more as an empirical question, since technical means had been used in the welfare sector by the executive for its own gain (Pierson Reference Pierson2007; Eubanks Reference Eubanks2017). Our study fills in these gaps by comparing the algorithm with the law it supposedly enforces, identifying discrepancies between the two and analysing how and by whom they were produced. The results are presented after an overview of LMP in Poland and a description of the methods.
3 Reform of LMP in Poland
Algorithmic profiling was introduced in Poland in 2014 as part of a broader reform of LMP, initiated by a government formed by Civic Platform and the Polish People’s Party and more specifically the Ministry of Labour and Social Policy (Niklas et al. Reference Niklas, Sztandar-Sztanderska and Szymielewicz2015). The preparatory works on profiling started in 2012, after the economic crisis, when the registered unemployment rate in Poland reached 13.4 per cent and the unemployed population exceeded 2 million.
The neoliberal political narrative that legitimised this reform, as in other European countries at the time (Caswell et al. Reference Caswell, Marston and Larsen2010; Allhutter et al. Reference Allhutter, Cech, Fischer, Grill and Mager2020; Mozzana Reference Mozzana2019), questioned rights-based access to welfare, while focusing on the activation and responsibilisation of jobseekers, efficiency, rationalisation of spending, standardisation as well as tailoring ALMP to individual needs (see Garsten et al. Reference Garsten, Jacobsson, Sztandar-Sztanderska, Heidenreich and Rice2016 for the analysis of tensions between standardisation and tailoring). Despite these apparent discursive similarities with other European countries, it is important to note that the institutional structure of PES, as well as the overall design of LMP in Poland, differed significantly.
On the institutional level, the implementation of ALMP, together with other responsibilities such as registering the unemployed, verifying their entitlements and paying unemployment benefits, in Poland were the prerogative of local PES agencies (in Polish Powiatowe Urzędy Pracy), which were decentralised in 2000 (Grabowski Reference Grabowski2008; Mandes Reference Mandes, Heidenreich and Rice2016). Since then, there has been no direct relationship of subordination between the Ministry, as part of central government, and local PES agencies, as part of local self-governments. In this context, the Ministry had limited authority and could not directly influence local PES when it comes to distribution of ALMP. Having regulatory and supervisory prerogatives, the Ministry could exert such influence indirectly, by initiating changes in legislation or developing ICT systems for PES – and it did so in this case.
Additionally, after a previous series of neoliberal reforms that introduced cuts to benefits and conditionality in access to welfare in Poland, the unemployed already had very limited social rights compared to those in other EU countries (Spieser Reference Spieser2007; Portet and Sztandar-Sztanderska Reference Portet, Sztandar-Sztanderska and Lefresne2010). In 2012, when the works on profiling started, only 17 per cent of the unemployed received unemployment benefits. Restricted subgroups could also access additional low benefits from Social Insurance or Social Assistance, the legal criteria for eligibility to which were relatively stringent (Prusinowski Reference Prusinowski2014; Poławski Reference Poławski, Poławski and Zalewski2017; Kozek et al. Reference Kozek, Kubisa and Zieleńska2017) and were not subject to change during this reform. The only type of social protection accessible to all the unemployed was free public health insurance. The legal regulation linking public health care with the unemployed status had long been criticised by PES agencies and employers’ organisations for creating incentives to register with PES for persons who were neither willing nor able to take up work. PES complained that as a result their case-loads were higher and co-operation with employers hampered (Sztandar-Sztanderska Reference Sztandar-Sztanderska2016). The Ministry attempted to change this regulation but failed to secure the approval of the Ministry of Finance.
The political narratives legitimising the introduction of profiling left out the dramatically low level of spending on ALMP and staffing problems in PES, which resulted in low accessibility of ALMP. In 2011, fewer than thirteen per 100 registered unemployed received a form of ALMP (Sztandar-Sztanderska Reference Sztandar-Sztanderska2013, p. 4). At the same time the number of unemployed legally considered to be at greater risk in the labour market was high: on the eve of the reform, these priority target groups covered around 90 per cent of the unemployed, which left PES staff with discretion over whom to select for scarce ALMP (Sztandar-Sztanderska Reference Sztandar-Sztanderska2013, p. 24). Rather than secure increased funding for ALMP, which was politically and economically difficult, the Ministry introduced profiling, trying to redefine target groups and regain control over how PES distributes ALMP.
Importantly, profiling was implemented through two decision-making channels: the legislative process and the development of the algorithm that formed the basis of ADM. The legislative process was initiated by the Ministry, which proposed a draft legislation subject to public consultations and parliamentary works. New legislation included: the amendment of the Statute on Employment Promotion and Labour Market Institutions (2014; henceforth: Statute), enacted by the parliament; and the Regulation on Profiling Assistance for the Unemployed Person (2014; henceforth: Regulation), issued by the Minister. The Statute (Art. 33(2)(c)) distinguished three groups – called ‘profiles of assistance’ I, II and III – which differentiated the unemployed person’s access to ALMP as well as obligations (Table 1). Persons classified as profile II were given the possibility of applying for a wider range of services, including job placement, vocational counselling and programmes to support mobility, skill development and subsidised employment or self-employment (the provision of these services, however, depended on the available funding – Niklas et al. Reference Niklas, Sztandar-Sztanderska and Szymielewicz2015). Importantly, access to ALMP was now contingent on greater obligations, where non-co-operation resulted in sanctions, including the forfeiture of all entitlements. In other words, individuals assigned to profile II were subjected to a combination of so-called ‘enabling’ and ‘demanding’ activation (e.g. Eichhorst et al. Reference Eichhorst, Kaufmann and Konle-Seidl2008; van Berkel et al. Reference van Berkel, Caswell, Kupka and Larsen2017).
Table 1. Access to ALMP and obligations depending on a result of profiling
As for the profiles I and III, their access to ALMP was restricted. Profile I was allowed to use job placement and selected services available to profile II, if justified. Finally, persons classified as profile III were exempt from job search obligations and did not have the right (but also the obligation) to participate in any regular ALMP. They could access facultative programmes or outsourced services, but these were hardly available (Niklas et al. Reference Niklas, Sztandar-Sztanderska and Szymielewicz2015; Sztandar-Sztanderska et al. Reference Sztandar-Sztanderska, Kotnarowski and Zieleńska2021).
The development of the profiling algorithm was controlled by the Ministry. The algorithm was then integrated with the SyriuszStd software and used by all PES until the withdrawal of profiling in 2019. It was conceived as a recommender system, leaving PES staff with the final decision on whether to accept the automatic classification to a profile. However, PES staff rarely said ‘no’ to computer advice, as corrections of profiles were below 4 per cent (Sztandar-Sztanderska and Zieleńska Reference Sztandar-Sztanderska and Zieleńska2022), although case-workers sometimes used data input to influence results generated by ADM (Sztandar-Sztanderska and Zieleńska Reference Sztandar-Sztanderska and Zieleńska2018; Sztandar-Sztanderska and Zieleńska Reference Sztandar-Sztanderska and Zieleńska2022; on informal classifications and data input, see also Petersen et al. Reference Petersen, Christensen, Harper and Hildebrandt2021).
Importantly, the algorithm was kept secret despite numerous calls for transparency, leaving the general public and research community without solid knowledge of how the ADM worked. The Ministry presented the unemployed as ‘fraudsters’ who would try to manipulate profiling results if the algorithm were known (Niklas et al. Reference Niklas, Sztandar-Sztanderska and Szymielewicz2015), a typical narrative to justify black-boxing of the algorithm in the welfare sector (Eubanks Reference Eubanks2017). An example of the confusion as to how the ADM worked can be found in the article by Kuziemski and Misuraca (Reference Kuziemski and Misuraca2020), who mistook this simple rule-based system for AI-based technology (Sztandar-Sztanderska et al. Reference Sztandar-Sztanderska, Kotnarowski and Zieleńska2021).
4 Data and methods
In this article, we use data collected and analysed as part of a longer series of research projectsFootnote 1 (for details, see acknowledgements).
4.1 Comparison of algorithm and legal framework
To compare the algorithm with legal framework, we obtained usually inaccessible data (Pasquale Reference Pasquale2015; Mittelstadt et al. Reference Mittelstadt, Allo, Taddeo, Wachter and Floridi2016; Busuioc Reference Busuioc2020). Our analysis is based on internal ministerial documentation that was involuntarily disclosed to a watchdog organisation called the Panoptykon Foundation, when the administrative court confirmed that the algorithmic formula, affecting distribution of public services, should be treated as public information (Case II SAB/Wa 1012/15, 2016).
These materials consisted of, first, a written version of the profiling questionnaire, designed to collect data on the unemployed in order to process it using a mathematical formula and automatically generate classification into a profile; and, second, a ministerial handbook explaining the rationale for profiling and containing step-by-step instructions for PES (Ministerstwo Pracy i Polityki Społecznej 2014a). These documents gave us insight into the inner workings of the ADM at the level of a structured formula with an appropriate rule set (Kitchin Reference Kitchin2017). Despite its semi-analogue format, we refer to the questionnaire and the mathematical formula jointly as the algorithm, considering that the written version of the questionnaire specified data items and a set of instructions on how to process data and produce an output and therefore played an important role in the allocation of public services.
In the first step, we analysed the algorithm (Sztandar-Sztanderska and Zieleńska Reference Sztandar-Sztanderska and Zieleńska2020; Sztandar-Sztanderska et al. Reference Sztandar-Sztanderska, Kotnarowski and Zieleńska2021) and closely compared it to the Statute and Regulation and subsequent interpretations of these legal acts by courts (see also Godlewska-Bujok Reference Godlewska-Bujok2020). Following Hildebrandt’s comparative approach (Reference Hildebrandt, Hildebrandt and Gaakeer2013, p. 5), our aim was to find out whether computer articulation of the law diverged from its written articulation and, if so, how. More specifically, starting with each component of the algorithm (data items, specific elements of the mathematical formula), we searched for its legal basis in Statute and Regulation to see what was added or what was lost in translation. When reporting the results we focus on the most striking discrepancies that are indicative of backstage decision-making and consequential for social and employment rights.
4.2 Analysis of decision-making that led to the discrepancies between algorithm and legal framework
Next, we analysed the decision-making process to find out by whom and how the elements of the algorithm diverging from law were created. For this purpose, we studied publicly available documents from the legislative process and confirmed that the Ministry had refused access to ADM (Niklas et al. Reference Niklas, Sztandar-Sztanderska and Szymielewicz2015). The analysis of the ADM’s development process became possible only after the Ministry changed its initial position and decided to voluntarily provide us with additional materials: internal presentations documenting works on profiling and statistical reports that described statistical choices made during the algorithm’s development and maintenance (Ministerstwo Pracy i Polityki Społecznej 2013a, 2013b, 2013c, 2014b, 2014c, 2014d, 2014e, 2015, 2016a, 2016b). To put this information in context, our team participated in a discussion with persons responsible for the development (henceforth: DISCUSSION), conducted in-depth interviews with twelve selected participants of the development and legislative process (henceforth: PM1–12) and formulated additional questions to the Ministry to which we received written responses (henceforth: ENQUIRY).
All these data together provided us with information on dilemmas faced and decisions made during the algorithm’s development and its subsequent modifications: among others, different variants of the profiling questionnaire and the mathematical formula. We analysed them, paying attention to the timing of decisions, actors participating in decision-making and their decision-making power that resulted in what we identified as crucial elements of the algorithm diverging from law. More specifically, we wanted to verify whether legislation was the starting point and preceded the development of the algorithm (Oskamp and Tragter Reference Oskamp and Tragter1997) – as the concepts of representation and translation of law suggest – and to explore how and by whom these discrepancies were created, what was the role of the Ministry and PES and what was the role of system-level bureaucrats.
5 Scope and origins of discrepancies between the algorithm and legal framework
In this section, we present our findings as to what were the major discrepancies between the algorithm and law and how they were created. Juxtaposing legal acts with the algorithm highlighted differences when it comes to the degree of generality of regulation of the profiling process and criteria for profile determination. The Statute omitted any explicit guidance regarding these issues. Rather, it simply mandated that profiles be assigned in a manner that would ensure that the scope of ALMP is ‘appropriate’ from the point of view of individual ‘needs’. Notions of ‘appropriateness’ and ‘needs’ are illustrative of the ambiguous language of written law. The Statute did not elaborate on how this ‘appropriateness’ should be attained, the type of data required to assess the ‘needs’ of the unemployed or the tools to be employed.
As to the Regulation issued by the Minister, it enumerated factors to be considered during profiling in more detail but left plenty of space for interpretation. The fact the that types of data used for the purpose of profiling were designated in the Regulation, instead of the Statute, violated data protection rights on procedural grounds and was the reason why these provisions were deemed unconstitutional by the Polish Constitutional Tribunal in Case K 53/16 (2018). However, the discrepancies between the algorithm and legal framework would still persist even if these data – as defined in Regulation – were moved to the Statute and the procedural grounds for the unconstitutionality of the law ceased to exist.
To understand what this discrepancy consists of, we propose taking a closer look at the Regulation and comparing it to the algorithm. The Regulation clarified that a profile would be assigned as a result of ‘an analysis of the situation of an unemployed person’ and ‘his chances on labour market’ (male pronoun in the original). It also established two general aspects that were to be considered during this process: ‘distance from the labour market’ and ‘readiness to enter or return to the labour market’, and broke each of these two dimensions down into a closed catalogue of factors: fifteen in total. Importantly, some of these factors were also framed in equivocal terms, such as ‘readiness to adapt to labour market requirements’, ‘reasons to register with PES’ or ‘commitment to independent job search’ – to give only a few examples (Table 2). Note that the relevance of these factors for the assignment of a profile was not defined.
Table 2. Comparison of legal provisions implementing the profiling algorithm and the algorithm
The algorithm consisted of an electronic questionnaire integrated with the SyriuszStd software. The questionnaire was composed of twenty-four questions with standardised answer options. In eight questions, the software processed data about the unemployed (e.g. age, gender, education level) already recorded in the system. For the remaining sixteen questions, data were entered by a case-worker, who had to select one or multiple options from a list of pre-defined answer choices during the interview with an unemployed person. To assign a profile, each question had to be answered.
After the completion of the questionnaire, the software applied a hidden mathematical formula. First, it assigned a score of 0 to 8 points to an answer or a combination of answers (with the assumption that the higher the score, the lower the individual’s ‘employability’). Then, it connected answers with one of two variables, called, as in the Regulation, ‘distance from the labour market’ (D) and ‘readiness to enter or return to the labour market’ (R). Afterwards, the software calculated the sum of all individual weights counted as D or R and multiplied them by coefficients. It then compared this result with the fixed-point boundaries and recommended the categorisation of an unemployed person into one of three groups, leaving a case-worker with the final choice, which required justification only if the classification differed from the algorithmic advice.
Already, this preliminary analysis of the algorithm shows two significant discrepancies compared with the law, which we explore below. The first modification concerned the criteria according to which classificatory decisions were to be made, and the second, the imposition of a sequence of instructions that replaced the flexible assessment process.
5.1 Replacing broad legal criteria with specific data items
Profiling criteria articulated in law differed from those present in the algorithm. To include them in the algorithm, their meaning was fixed so that they could take the form of standardised answers. This departure from the initial legal definitions was criticised by the Constitutional Tribunal: ‘the questionnaire does not literally repeat the catalogue of factors to be taken into account during profiling, contained in […] regulation, but develops or specifies them, and therefore modifies them’ (Case K 53/16, 2018, p. 37). The Constitutional Tribunal found this modification problematic mostly because the questionnaire was ‘not an act of generally applicable law’, and processing of personal data must be regulated by the Statute; otherwise, the constitutional right to privacy is violated.
We argue that this modification was also problematic from the perspective of social and employment rights, as it redefined profiling criteria that affected the access of the unemployed to ALMP and the scope of their obligations. Consider these examples, the equivocal category ‘reasons for registration with PES’ present in the Regulation was articulated with the question: ‘What is the main reason you have registered with the employment agency?’ Note that the framing of the legal category in plural (‘reasons for registration with PES’) assumed the possibility of the co-existence of a variety of reasons without the necessity to order them as less or more important. The algorithm, however, required choosing one single answer among pre-defined response categories, making it impossible to represent such variety, as well as excluding reasons not present on the list. The necessity to oversimplify or to ignore specific circumstances recurred in other questions (e.g. number twelve, thirteen, eighteen, twenty-one). We treat them as an empirical illustration of excessive rigidity – a risk which legal scholars accurately predicted (Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013; Bovens and Zouridis Reference Bovens and Zouridis2002).
Source: Electronic questionnaire (for a full questionnaire, see Sztandar-Sztanderska Reference Sztandar-Sztanderska2024).
Particularly puzzling was the clear suggestion visible in the content of the cafeteria of answers that the authors of the questionnaire were critical towards legally binding social rights, including health care and benefits, and treated them as factors decreasing ‘employability’ (Sztandar-Sztanderska and Zieleńska Reference Sztandar-Sztanderska and Zieleńska2020, p. 7). The manual for PES articulated this rationale further, suggesting that not all unemployed persons have the right motivation to register with PES, namely ‘willingness to get help in finding a job’, represented by answers a and b (Ministerstwo Pracy i Polityki Społecznej 2014a, p. 12).
In a similar manner, the algorithm undermined the legitimacy of labour rights that offered protection for workers against illegal or coercive work, and defied safeguards regarding working conditions. This problem can be seen in the formulation of answers to the question: ‘What are you able to do in order to improve the chances of your finding work?’, which specified in an arbitrary manner the legal category ‘readiness to adapt to the requirements of the labour market’.
Source: Electronic questionnaire (for a full questionnaire, see Sztandar-Sztanderska Reference Sztandar-Sztanderska2024).
As the ministerial handbook for PES explained, all answers to this question except ‘I’m not prepared to do anything’ were interpreted as a sign of a desirable attitude:
‘Unemployed people are not always open to taking measures that can improve their situation on the labour market. […] They do not want to accept that in order to find employment they have to make serious efforts and sometimes even change a lot in their lives. The indication of various possibilities to adapt to the requirements of the labour market indicates greater openness, sometimes even determination to find a job’ (Ministerstwo Pracy i Polityki Społecznej 2014a, p. 13).
From the legal perspective, one is baffled to find the answer ‘undertake work without a contract’ as an interpretation of the legal criterion ‘readiness to enter or return to the labour market’, since it is in clear contradiction of the Labour Code, which states that there should always be an employment contract in writing (Labor Code 1974, Art. 29(2) – a legal provision that was introduced to protect security of workers, as a weaker party in the labour market, against illegal employment).Footnote 2
Answers b and i to the question above might also cause controversy, as they go well beyond the legal obligations of the unemployed. If we compare them with the legal definition of ‘suitable employment’ defined by Art. 2(1)(16) of the Statute – that is, employment that cannot be refused without sanctions – we realise that the authors of the questionnaire tacitly raised expectations towards the unemployed. The law did not oblige the unemployed to commute more than 1.5 hours to work (option b), to change their place of living (a) or to work abroad (j), to accept a contract for a task (i), which, as a civil law contract, does not fall under labour law and therefore does not provide the protections that labour law guarantees.
To summarise, the first type of modification to the substance of law involved a replacement of broad legal criteria with specific data items. These data items put into question binding social and labour rights and raised expectations towards individuals, distinguishing those who are willing to take up efforts exceeding their legal obligations, to the detriment of their own labour rights, from those who are not.
5.2 Replacing goal-orientated framework with ‘if …, then …’ logic
The second type of modification of the substance of law concerned the imposition of a sequence of instructions replacing the flexible assessment process. This sequence, composed of scoring, coefficients, linear function and cut-off points, significantly departed from legal provisions.
In contrast to the algorithm, which specified the score of each data item, the relevance of the fifteen factors listed in the Regulation was left to the PES agencies to be established on a case-by-case basis. There were no rigid criteria that a person had to meet to be included in a profile; the boundaries between the profiles were left blurred, so that the assigned ALMP would be the most ‘appropriate’ for the unemployed person’s situation and therefore the most effective. This is also how legal rules were subsequently interpreted by courts (Case II SA/Po 395/16, 2016).
In this sense, the Statute and the Regulation constituted an example of a ‘goal-oriented legal framework’, which as Bovens and Zouridis (Reference Bovens and Zouridis2002, p. 181) explain, contrary to a ‘conditionally programmed legal framework’ cannot be easily transformed into an algorithm, because it ‘only enumerates the interests that must be taken into account and weighed by’ public administration staff and leaves their relevance unspecified. The very idea of subjecting such law to computer articulation seems problematic, as it was not in a computer-representable form.
Moreover, the analysis of documentation revealed another crucial discrepancy that the legislature and the public were not made aware of: elements of the mathematical formula were calibrated with the aim of dividing the unemployed population into profiles according to pre-defined proportions, so that 60 per cent of the unemployed would be categorised as profile II and the rest equally divided between profiles I and III. The internal ministerial report explicitly stated this political goal, which guided the development of the algorithm:
‘The percentage shares […] are to be as close as possible to a 20–60–20 ratio […] Practical, economic and strategic considerations, as well as the experience of those involved in profiling assistance to the unemployed, argue in favour of such a division’ (Ministerstwo Pracy i Polityki Społecznej 2013c, p. 26).
As shown by Sztandar-Sztanderska and colleagues (2021), no statistical analysis was conducted to create profiles that would at least to some extent reflect the unemployed persons’ ‘needs’ – as prescribed by law – nor was there any attempt to estimate their labour market chances, as is usually done in cases of statistical profiling. Instead, different variants of a mathematical formula were tested based on pilot data, so that this division could be achieved and financial resources for ALMP divided accordingly. It means that the categorisations into profiles generated by the ADM did not so much reflect individual ‘employability’ as they were the product of prior political decisions concerning the distribution of funds on ALMP.
Furthermore, these proportions were afterwards quietly changed to 15–70–15 per cent, which again entailed the modification of the elements that made up the mathematical formula. Ministerial documentation estimated that the implemented changes were to influence categorisation of more than ten per cent of the unemployed population that constituted ‘a group of hundreds of thousands of people’ (Ministerstwo Pracy i Polityki Społecznej 2014e, p. 6). There could hardly be better evidence of how backstage decision-making during the development of ADM translated into tampering with the social rights and obligations of a large number of citizens.
5.3 Backstage decision-making process
In this section, we present our results as to when and by whom these discrepancies concerning the profiling process and its criteria were created.
Inspired by Oskamp and Tragter (Reference Oskamp and Tragter1997), we analysed the timing of decisions and reached the conclusion that the enacted law could not have been the starting point for the algorithm’s development. Most of the choices concerning the ADM were made before the Statute and the Regulation were enacted – that is, on 14 March 2014 and 14 May 2014 respectively. In fact, the works on the algorithm coordinated by the Ministry were already advanced before the legislative process officially started and the preliminary versions of Statute and Regulation were made public and put to consultation – that is, 31 July 2013 and 30 December 2013. As one of our interviewees participating in the development of the questionnaire put it: ‘We didn’t have the outlines of this law, because it was top secret, so we did some things a bit in the dark’ (PM3).
This statement was confirmed by other documents and information from an interview, during which another interviewee read aloud an e-mail exchange with the Ministry and verified the exact timeline (PM5). The works on profiling started in June 2012 (Ministerstwo Pracy i Polityki Społecznej 2013a). On 31 January 2013 – that is, more than a year before the enactment of both the Statute and the Regulation – the very first version of the algorithm with preliminary definition of data items and scoring of answers was ready (PM5). A draft questionnaire was then completed in March 2013 (PM5), put to pilot testing in twenty-four PES agencies between July and September 2013 after its integration with SyriuszStd software (Ministerstwo Pracy i Polityki Społecznej 2013c, p. 9) and then reworked again, so that the final version of the paper questionnaire defining all elements of the algorithmic formula was ready after 9–10 December 2013 (Ministerstwo Pracy i Polityki Społecznej 2013a, p. 22). This is when the algorithm was also calibrated to produce results according to 20–60–20 per cent proportions.
In contrast, the draft of the Regulation that specified factors which should be considered when assigning a profile was only made public by the Ministry and open for consultation on 30 December 2013 (Ministerstwo Pracy i Polityki Społecznej 2013d). Taking into account this timeline and what our informants said, it seems it was the process of the development of the algorithm that shaped the legal provisions which defined the profiling criteria rather than the other way round. Furthermore, we discovered that substantial changes to the algorithm, introduced to classify the unemployed into profiles according to new proportions (i.e. 15–70–15 per cent), were introduced only in 2015 – that is, long after the legislative process was over (Ministerstwo Rodziny Pracy i Polityki Społecznej 2016b). This means that, contrary to what legal scholarship implicitly assumes, substantial changes to ADM turned out to be caused by behind-the-scenes political decision-making rather than the need to adapt systems to new laws or legal jurisdictions.
As to why these backstage decisions were made and by whom, empirical evidence points to the executive, more specifically management of the Ministry and the Department of Labour Market (henceforth: Department) as playing a major role in the decision-making process, both the development of the algorithm and the preparation of the draft legislation (PM1–8, PM12). The second actor, of somewhat lesser importance, were selected representatives of PES, invited by the Department to participate in this process (PM3–5, 9).
Most likely for financial reasons, the Ministry decided to develop the written version of the profiling questionnaire and the mathematical formula in-house, opting out of outsourcing this task to external companies or experts (Ministerstwo Pracy i Polityki Społecznej 2013a, PM6, 7, 12, ENQUIRY, pp. 7–8). Instead, a ministerial team was created consisting of departmental staff as well as management and street-level bureaucrats from eight PES agencies. This team developed a preliminary version of the questionnaire, which included questions, standardised answers and initial scoring. As there was no statistical analysis conducted at this stage, decisions concerning specification of data items and initial scoring resulted from ‘brainstorming’, ‘disputes’ and ‘discussions’ (PM3, 4, 5, 8, 9, 10, 12) that were only loosely inspired by existing research on profiling, but mostly drew from experience of team members – that is, PES and Ministry staff and their understandings of the profiling goal. As one of our interviewees pointed out, even the scoring of answers was decided ‘in the course of discussions between the Ministry, the Labour Market Department, and the PES. That is, between the practitioners and the decision makers’ (PM12). The team also relied on feedback concerning phrasing of the questionnaire from twenty-four PES agencies, where the pilot was conducted.
Only at a later stage was a statistician involved. The role of this person was limited to fine-tuning the initial questionnaire, improving its reliability and calibrating the mathematical formula, so as to divide the unemployed according to pre-defined proportions. In other words, the statistician retained some degree of discretion, but their tasks were very narrowly defined, since key decisions had already been made. This finding leads us to the conclusion that the development of simple rule-based ADM creates an opportunity for backstage political decision-making for not only system-level bureaucrats, but also for the executive (in our case: the Ministry) and public administration, including street-level bureaucrats, if their representatives are consulted in the process.
The other salient finding is that the members of the team, both from the Ministry Department and PES, perceived the algorithm not as a representation of law, but rather as a pragmatic way of working around financial, political and legal difficulties that could not be easily solved otherwise (PM3, 9, 12). For the Ministry, the algorithm was a way of circumventing, if not hiding, the problem of insufficient resources for ALMP, as ADM was calibrated to invisibly reduce the share of priority target groups among the unemployed population: from around 90 per cent prior to the reform (Section 3) to first 60 and then 70 per cent (DISCUSSION, Ministerstwo Pracy i Polityki Społecznej 2013c). The Ministry also regarded the algorithm as a method to regain control over different criteria of distribution of ALMP applied by the decentralised PES, when there were no legal attempts to re-centralise them: ‘thanks to the prompting provided by the algorithm, that applied uniform criteria, the scale of variation in criteria and decision-making by different PES employees has been significantly reduced’ (Ministerstwo Rodziny Pracy i Polityki Społecznej 2015, p. 4).
From the perspective of the PES staff participating in the development, the algorithm seemed to some extent useful, because it helped to deal with insufficient resources and staff shortages in their local agencies by reducing the number of the unemployed who had access to ALMP. PES representatives shared a conviction, later made explicit in the handbook for PES published by the Ministry (Ministerstwo Pracy i Polityki Społecznej 2014a, p. 4), that refusing access to regular ALMP for profile III was justified, since the questionnaire would help to classify into this group persons who are ‘barely or not at all interested in support from labor offices in re-entering the labour market’, the majority of whom – in the opinion of our interviewees – registered in order to obtain health care insurance or other legally defined entitlements (PM3, PM9). Furthermore, the algorithm was even described during interviews as a way to circumvent ‘bad’ law, which was politically impossible to change at the moment of the reform:
‘We are actually trying to improve a little bit the functioning of something that has been functioning badly from the very beginning. And that is the effect of the Statute, which imposes an obligation on employment services to help. While this obligation is often exploited by the unemployed, who are interested in something completely different than getting a job […] And it would have to be a much broader project, with that kind of readiness for legislative change at the highest.’ (PM12)
Needless to say, such understanding of the role of the algorithm in terms of circumvention of law stands in direct opposition to its understanding in terms of representation. Instead of following and representing the law as faithfully as possible, interviewed members of the ministerial team critically evaluated the binding legal provisions regulating social rights to the point that some of them agreed that they should be algorithmically bypassed, if they cannot be changed. In the light of these findings, it becomes clear why the identified discrepancies between the law and the algorithm were created and how they were embedded in value judgments and interests of actors participating in ADM development.
6 Discussion and conclusions
By studying the case of the algorithm implemented in Poland to profile the unemployed and regulate their scope of obligations and access to ALMP, we contribute to the literature that problematises the connection between the law and its computer articulation.
First, we demonstrate that even in cases of simple ADMs implemented to supposedly enforce the law, significant discrepancies between the law and the algorithm might arise. In the profiling case, we observed how the algorithm became the source of new rules having nothing to do with the law it was supposed to embody. Discrepancies occurred in the content and relevance of criteria according to which ALMPs should be distributed and the scope of the unemployed persons’ obligations established. Such discrepancies between the law and the algorithm are problematic from the point of view of the rule of law and the principle of legality according to which actions of administrative bodies must be founded on laws enacted by authorised bodies through a legislative process (Macdonald, Correia and Watkin Reference Macdonald, Correia and Watkin2019; Bovens and Zouridis Reference Bovens and Zouridis2002; Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013). Therefore, this case study provides supporting evidence for the hypothesis that the introduction of ADMs modifies the separation of powers and decreases the role of legislature.
Second, our analysis shows that discrepancies between the algorithm and legal framework cannot be seen solely as a necessary by-product of the representation or translation process, which introduces modifications to law in order to adapt its written form to different requirements of the algorithmic system. Although different characteristics of law and code do play a role, conceptualising algorithm development as representation or translation of law might be misleading. We argue that the concepts of representation and translation of law as used by legal scholars (Bovens and Zouridis Reference Bovens and Zouridis2002; Hildebrandt Reference Hildebrandt, Hildebrandt and Gaakeer2013) and computer scientists (Kowalski and Sergot Reference Kowalski and Sergot1985; Allen and Saxon Reference Allen, Saxon and Nagel1991; Bench-Capon and Coenen Reference Bench-Capon and Coenen1992) carry several implicit assumptions that have proved difficult to sustain in the light of our findings. For instance, it is often assumed that the law precedes ADMs and that its articulation in the form of legal acts or its reinterpretation by courts constitutes a benchmark when the algorithmic system is developed or subsequently changed (Kowalski and Sergot Reference Kowalski and Sergot1985; Bench-Capon and Coenen Reference Bench-Capon and Coenen1992). Contrary to these assumptions, the algorithm analysed in this article was developed mostly before the start of the legislative process; there was no law yet to be translated or represented. Moreover, after the algorithmic system was deployed, the algorithm was substantially changed as a result of backstage political decision-making to change the distribution of resources without parliament and the public being either consulted or made aware of it.
Based on previous research, scholars also presuppose that there will be collective political effort to minimise discrepancies between the law and the algorithmic system, for instance, by applying a method of ‘isomorphism’ to render the algorithmic system more easily adaptable to subsequent changes of law and court verdicts (Bench-Capon and Coenen Reference Bench-Capon and Coenen1992), by enacting easily representable legislation (Plesner and Justesen Reference Plesner and Justesen2022) or, in the worst case scenario, by changing the law if adapting the system to the jurisdiction proves too difficult (Oskamp and Tragter Reference Oskamp and Tragter1997). In this case, such efforts to minimise discrepancies between the law and the algorithms did not take place. On the contrary, persons who participated in the creation of the profiling algorithm were critical towards the law regulating the scope of social rights to the extent that they wanted the algorithm to circumvent it rather than represent it faithfully.
Finally, researchers have drawn attention to the growing power of statisticians and programmers and emphasised the need for a new system of checks and balances that would subject decisions of these system-level bureaucrats to democratic control (Bovens and Zouridis Reference Bovens and Zouridis2002). What has escaped scholarly attention is the fact that the development of algorithmic systems also creates an opportunity for traditional policy-makers, namely the executive power and public administration, to avoid blame and obfuscate controversial decisions. Thus, we argue for the need to incorporate in a more systematic manner the political dimension into the analysis of algorithmic articulation of law.
Acknowledgements
In this article, we use a) the results of the project ‘Information technologies in public policy. Critical analysis of the profiling the unemployed in Poland’ financed by the National Science Centre, Poland) (grant no. 2016/23/B/HS5/00889), led by Karolina Sztandar-Sztanderska and conducted together with Alicja Palęcka, Michał Kotnarowski, Marianna Zieleńska, Barbara Godlewska-Bujok, Jędrzej Niklas and Joanna Mazur and b) literature review conducted within the project AUTO-WELF: Automating Welfare - Algorithmic Infrastructures for Human Flourishing in Europe. Project AUTO-WELF is supported by National Science Centre, Poland (grant no. 2021/03/Y/HS5/00263) under CHANSE ERA-NET Co-fund programme, which has received funding from the European Union’s Horizon 2020 Research and Innovation Programme, under Grant Agreement no 101004509. We would like to thank all above-mentioned team members: the results of other work packages they contributed to provided us with important contextual knowledge for the comparison of legal and algorithmic framework. We would also like to express our gratitude to Marianna Zieleńska and Michał Kotnarowski for conducting selected interviews that we analysed, to Jędrzej Niklas for his remarks on the earlier version of the paper and to Helena Teleżyńska for her help with the final version of the paper. We would also like to thank the organisers and the participants of Surveillance, Democracy, and the Rule of Law conference at the European University Institute for the opportunity to present our research there. We would also like to thank the University of Warsaw for funding Joanna Mazur’s participation in this conference within the IDUB programme. Finally, Joanna Mazur is supported by the Foundation for Polish Science (FNP).
Competing interests
All authors declare that they have no competing interests.
Data availability statement
Core data analysed for the purpose of this article are publicly available:
-
Legal documents cited in the text were derived from public domain resources.
-
Ministerial documents describing the profiling algorithm were derived from Panoptykon Foundation webpage (in Polish: Fundacja Panoptykon), which disclosed materials coming from leaks and obtained from the Ministry of Family, Labour and Social Policy after the court order. They consisted of: profiling questionnaire with scoring and Handbook on Profiling for Public Employment Services.
-
Sztandar-Sztanderska, Karolina, 2024, ‘Technologie informacyjne w polityce publicznej. Krytyczna analiza profilowania bezrobotnych w Polsce’, https://doi.org/10.18150/PRGRH1, RepOD, V1.
Other supplementary data (e.g. interviews) listed in the publication cannot be made available for the ethical reasons, such as protecting anonymity of research participants and minimising harm.
Open access
