13.1 Introduction

Comparative judicial behaviour ‘seeks to illuminate the choices judges make and the consequences of their choices for society’ (Epstein et al. Reference Epstein, Grendstad, Sadl, Weinshall, Epstein, Grendstad, Sadl and Weinshall2024). Indeed, the field of judicial behaviour has expanded beyond analyses of the American judiciary to attempt to characterise judges’ decision-making in many contexts around the globe. These accounts of judicial behaviour cross disciplinary boundaries and adopt a variety of theoretical and empirical approaches (see Epstein, Chapter 5 in this volume). This interdisciplinary field attempts to eschew the characterisation of judges as legal robots applying the law in a vacuum, and treats them as individuals that have preferences and are subject to the institutional constraints in which they operate. In other words, contrary to previous eras of academic belief, judges are humans too (e.g., Rachlinski and Wistrich Reference Rachlinski and Wistrich2017).

In this chapter, I apply this broad interdisciplinary range of research to the Court of Justice of European Union (CJEU) to assess the state-of-the-art in the scholarship. Importantly, I argue that empirical research is necessary to answer questions of judicial behaviour in EU law. I start first from a typology of this research adapted from Epstein and Weinshall (Reference Epstein and Weinshall2021) and Epstein et al. (Reference Epstein, Grendstad, Sadl, Weinshall, Epstein, Grendstad, Sadl and Weinshall2024) (see Table 13.1). Second, I assess the degree to which scholars of the CJEU have engaged with these approaches and provide suggestions for areas in which the CJEU scholarship can remedy its shortcomings and avenues for future research. Third, I ask whether research on the CJEU can simply follow these paths or if there are features unique to the CJEU’s international character that may generate innovative research that does not exist in other areas.

13.2 The Typology of Judicial Behaviour and the CJEU

Adapting the typology of Epstein and Weinshall (Reference Epstein and Weinshall2021) and Epstein et al. (Reference Epstein, Grendstad, Sadl, Weinshall, Epstein, Grendstad, Sadl and Weinshall2024), the study of judicial behaviour can be broadly separated into five approaches: (1) Legalism, (2) the Attitudinal Model, (3) Rational Choice Accounts, (4) Identity Accounts, and (5) ‘Thinking Fast’ Judging (see Table 13.1). Legalism broadly eschews political considerations and argues that judges primarily use neutral methods to determine the meaning of the law. In the civil law tradition, for example, the prevailing account is that only the legislature makes law with the judge applying the law as an ‘expert clerk. Presented with fact situations to which a ready legislative response will be readily found … the judge’s function is merely to find the right legislative provision, couple it with the fact situation, and bless the solution that is more or less automatically produced from the union’ (Merryman and Pérez-Perdomo Reference Merryman and Pérez-Perdomo2019: 36). Such approaches mainly discuss the principles and rules that guide the application of the law as opposed to considerations unique to judges or external to the judiciary (e.g., Conway Reference Conway2008).

Alternatively, the remaining accounts depart from simply contextualising judges’ decision-making as a function of legal constraints, and provide frameworks that emphasise individual, institutional, and external factors that influence the outcomes of cases. The Attitudinal Model, for example, argues that judges’ votes or opinion-writing on cases is a function of their policy preferences regarding case facts, with the underlying political preferences reflective of ideological or partisan concerns. Rational Choice accounts take the form of the Labour Market Model – in which judges are often trying to find the most efficient means to accomplish their goals on the bench (e.g., Epstein et al. Reference Epstein, Landes and Posner2013) – and more strategic analyses – in which they are weighing the preferences of institutional and external actors when charting the best course to achieve their goals (e.g., Lax and Cameron Reference Lax and Cameron2007). Identity Accounts emphasise judges’ personal and professional characteristics in influencing their choices (e.g., Boyd et al. Reference Boyd, Epstein and Martin2010; Glynn and Sen Reference Glynn and Sen2015). Lastly, ‘Thinking Fast’ Judging approaches emphasise judges’ use of heuristic and cognitive shortcuts to quickly solve cases (e.g., Posner and De Figueiredo Reference Posner and De Figueiredo2005).

Important to note is that the vast majority of this scholarship primarily concerns the United States and, even when outside of the United States (e.g., Helmke Reference Helmke2005; Ramseyer and Rasmusen Reference Ramseyer and Rasmusen2001), is focused primarily on domestic judiciaries (for notable exceptions, see Posner and De Figueiredo (Reference Posner and De Figueiredo2005); Stiansen (Reference Stiansen2022); Voeten (Reference Voeten2008)). In what follows, I divide the CJEU judicial behaviour scholarship into three categories: judge-level factors, internal institutional factors, and external factors that encompass various facets of the aforementioned typology. I survey and critique this scholarship, while providing areas for improvement. In sum, the CJEU scholarship has engaged well with rational choice accounts of judicial behaviour, but a considerable opportunity exists to apply attitudinal, identity, and ‘Thinking Fast’ approaches.

13.3 Judge-Level Factors

Judges have preferences over case outcomes (e.g., Segal and Spaeth Reference Segal and Spaeth2002), must often resolve a case in a collegial setting, (e.g., Edwards Reference Edwards1998), and have limited time to allocate to a large caseload (e.g., Epstein et al. Reference Epstein, Landes and Posner2013). While the vast majority of scholarship uses individual judges’ votes or written dissents (e.g., Garoupa et al. Reference Garoupa, Gili and Fernando2021; Voeten Reference Voeten2007), examining how these factors impact decision-making at the CJEU is an empirical challenge, as the CJEU is a per curiam court that does not publish individual judges’ votes on cases. Indeed, as former CJEU judge Josef Azizi (Reference Azizi2011: 55–56) explains, revealing judges’ positions on cases could put a judge

Despite the Court’s deliberate obfuscation of the decision-making process, scholars utilise available information to make inferences about how individual judges may affect case outcomes.

As Member States employ a variety of procedures to appoint their judges to the CJEU without requiring the consent of the other Member States (Dumbrovsky et al. Reference Dumbrovsky, Petkova and Van Der Sluis2014), resulting in a wide range of heterogeneity in judge background (Brekke et al. Reference Brekke, Fjelstul, Hermansen and Naurin2023), it is likely that judges at the CJEU also have differences in preferences. To assess these differences, Malecki (Reference Malecki2012) leveraged rotation among chambers to demonstrate that the judges’ preferences across are not uniform. Frankenreiter (Reference Frankenreiter2017, Reference Frankenreiter2018), additionally, provides evidence that the preferences of Advocates-General also vary across the European integration spectrum. Furthermore, Cheruvu (Reference Cheruvu2024) demonstrates empirically that when the judge-rapporteur (JR, opinion-writer) receives an observation (amici curiae brief) from their appointing Member State, the JR’s panel is likely to rule in favour of the appointing Member State.

The Court’s judges, nonetheless, make decisions collectively. CJEU President Koen Lenaerts (Reference Lenaerts2013: 1351) observes, ‘As consensus-building requires to bring on board as many opinions as possible, the argumentative discourse of the [CJEU] is limited to the very essential. In order to preserve consensus, the [CJEU] does not take “long jumps” when expounding the rationale underpinning the solution given to novel questions of constitutional importance.’ This consensus-building, however, may at times favour some judges over others.

For example, Cheruvu and Zeigler (n.d.) use the texts of judgments and convolutional neural networks to demonstrate how the content of judgments may systematically differ based on which judge is serving as the JR. Similarly, Ovádek et al. (n.d.) distinguish the contribution of non-rapporteur judges to the final text of a judgment. Wijtvliet and Dyevre (Reference Wijtvliet and Dyevre2021), furthermore, use expert interviews to place judges on the General Court on a pro-business latent scale and provide evidence that decisions on state aid and competition are correlated with the ideology of the median. Given the non-random assignment of cases to judges (e.g., Cheruvu and Krehbiel Reference Cheruvu and Krehbiel2022; Hermansen Reference Hermansen2020), however, a substantial inferential problem exists. That is, it is difficult to distinguish whether or not these differences among judges are a result of the cases the Court’s president assigns to them. Future scholarship can endeavour to formulate creative research designs that can promote causal identification (see Dyevre (Reference Dyevre2023) for a thorough discussion on this point).

Additionally, judges at the CJEU, in a similar vein to other courts around the world (e.g., Epstein et al. Reference Epstein, Landes and Posner2013; Roussey and Soubeyran Reference Roussey and Soubeyran2018), face strong workload pressures and have incentives to efficiently dispose of cases. The Court’s use of French as its working language also exacerbates this constraint, as Cheruvu (Reference Cheruvu2019) provides evidence that cases with a non-native French-speaking JR take more time to complete on average. To increase efficiency, the Court has increasingly relied on small chambers to decide cases (e.g., Kelemen Reference Kelemen2012). Running a series of computational simulations, Fjelstul et al. (Reference Fjelstul, Gabel and Carrubba2023) suggest that increasing the number of judges at the court as well as the use of the chamber system may lead to substantial efficiency gains for the court. The CJEU also now has new procedural tools to reduce the likelihood of hearing cases that deal with substantively similar issues, which Brekke et al. (Reference Brekke, Fjelstul, Hermansen and Naurin2023) argue increased efficiency at the CJEU. Future research can explore how workload pressures on judges may systematically affect the quality of decisions or affect the likelihood of certain judges having more influence over case outcomes relative to others.

Much of the aforementioned scholarship of the individual-level factors that affect the CJEU’s decision-making, however, has primarily leveraged ideological distinctions among judges on the pro-anti EU dimension (with the notable exception of Wijtvliet and Dyevre (Reference Wijtvliet and Dyevre2021)), or Member States’ overt signals to their judges through observations. Importantly, other salient cleavages, such as the left/right ideological divide (e.g., Cheruvu Reference Cheruvu2022; Larsson and Naurin Reference Larsson and Naurin2019) and social class, along with other identity-based factors, such as gender (e.g., Gill and Jensen Reference Gill and Jensen2020) and religion, are largely understudied (e.g., Shahshahani and Liu Reference Shahshahani and Liu2017). As such, the CJEU literature largely lacks a stream of research in the vein of the attitudinal model. Arguably, this dearth of research may largely be due to the lack of identifying votes, which functionally reduces the number of data points to analyse judges’ behaviour. Furthermore, given the variation in judges’ Member States and political parties that appointed them, placing the judges on a single ideological dimension without expert knowledge of them (e.g., Wijtvliet and Dyevre Reference Wijtvliet and Dyevre2021) is challenging.

To make headway in this regard, the scholarship requires models of CJEU decision-making. In other words, when a panel of judges hears a case and the members of the panel have different preferences on a given dimension, in expectation, what should be the outcome of the case? While models suggesting the median judge or the median of the majority is decisive may be applicable (e.g., Carrubba et al. Reference Carrubba, Friedman, Martin and Vanberg2012; Wijtvliet and Dyevre Reference Wijtvliet and Dyevre2021), the JR’s ability to draft the initial judgment may impact whether other judges are willing to expend costly effort to make a counteroffer (more details on this point in the following section). For example, judges with higher workloads, as the labour market model predicts, may find it more costly to exert effort to alter the JR’s judgment (Hermansen and Voeten Reference Hermansen and Voeten2024). Upon generating such predictions from a theoretical model of CJEU decision-making, scholars can then leverage various proxies of judicial ideology (e.g., Sen Reference Sen2015) to then examine the impact of ideology on outcomes. As such, while an individual judge’s behaviour in the form of a vote is not public information, scholars can generate and test predictions when they specify a model of decision-making.

13.4 Internal Institutional Factors

The internal organisation and rules of courts can substantially affect outcomes, as they fundamentally affect the bargaining process (e.g., Carrubba et al. Reference Carrubba, Friedman, Martin and Vanberg2012; Lax and Cameron Reference Lax and Cameron2007). At the United States Supreme Court in which the Chief Justice assigns opinion authorship when they are in the majority, for example, Lax and Rader (Reference Lax and Rader2015) provide evidence that opinion-authorship assignment affects the probability of a justice’s defection from the initial majority-coalition. At the CJEU and many other European courts, the Court’s president assigns a case to a JR when it arrives to the court. This allows the JR to have ‘a greater weight in the eyes of the other judges … [T]he rapporteur holds a near monopoly over knowledge of facts and other materials concerning the case, including the competing arguments, so the other judges may be left at an informational disadvantage’ (Kelemen Reference Kelemen2017: 43). After authoring a preliminary report, the JR presents it to the plenary session of all the judges, at which point a determination is made as to the size and composition of the panel that will hear the case. The First AG assigns the case’s AG when it arrives at the Court, with the plenary also determining whether the AG will issue an opinion.

The various steps of this process of case assignment to JRs and the composition of panels is consequential to outcomes at the CJEU. First, Hermansen (Reference Hermansen2020) provides evidence that the Court’s president (1) systematically assigns cases on similar topics to the same JRs and (2) assigns cases in which substantial disagreement exists among governments to judges whose governments are closer to the median ideologically among EU members (see Kelemen (Reference Kelemen2012) for a thorough theoretical justification of this point). While, on the one hand, the president encouraging judges’ specialisation in specific policy areas may increase consistency in the law, on the other hand, coupling this finding with research demonstrating JR’s ideological alignment with their appointing governments (e.g., Cheruvu Reference Cheruvu2024), may lead to systematic biases in case law favouring the positions of specific Member States. Future research can explore the potential counterfactual development of CJEU case law if the president were to assign cases quasi-randomly, as is the practice at US Circuit Courts (e.g., Hall Reference Hall2010).

Second, Cheruvu and Krehbiel (Reference Cheruvu and Krehbiel2022) provide evidence that when the CJEU faces a credible threat of legislative override, it is more likely to assign a case to a larger panel of judges. These larger panels, thus, are more likely to vote in favour of the position favoured by the Member States, complementing Hermansen’s (Reference Hermansen2020) finding regarding the president’s strategic JR-assignment. This discrepancy in decision-making between smaller chambers and larger formations, while strategic to accommodate Member State preferences, may also have other consequences. Although the Court’s delegation of cases to chambers provides efficiency gains (Fjelstul et al. Reference Fjelstul, Gabel and Carrubba2023), it leads to inconsistencies in the application and development of EU law (Fjelstul Reference Fjelstul2023), with these effects particularly pernicious as judges’ preferences become more heterogeneous. Research can further examine the discrepancies in case outcomes when different groupings of judges hear cases and connect such findings to the more general scholarship about institutional procedures and outcomes at courts.

In this area of research, substantial opportunity exists for synergies with the legal scholarship. For example, volume 19 issue 7 of the German Law Journal (2018) was entirely dedicated to the topic of judicial self-governance. Many of these articles, in particular, highlight the role of the president’s case assignment as potentially affecting the day-to-day operations of the court. These presidents ‘can either assign cases on a discretionary basis, or they determine (or at least heavily influence) the rules … of case assignment’ (Blisa and Kosař Reference Blisa and Kosař2018: 2045–46) in Ireland (O’Brien Reference O’Brien2018), Italy (Benvenuti and Paris Reference Benvenuti and Paris2018), France (Vauchez Reference Vauchez2018), the Netherlands (Mak Reference Mak2018), and the Czech Republic (Blisa and Kosař Reference Blisa and Kosař2018). Legal scholars and social scientists alike are aware of the potential implications of these institutional mechanisms on the development of law. Scholars of the CJEU, which as I previously mentioned also are increasingly interrogating the role of the president (Cheruvu n.d.a), can profit from engaging in this scholarship and can add another layer to theories of judicial behaviour in the form of institutional governance.

13.5 External Factors

Gibson et al. (Reference Gibson, Caldeira and Baird1998: 343) contest that ‘with limited institutional resources, courts are therefore uncommonly dependent upon the goodwill of their constituents for both support and compliance. Indeed, since judges often make decisions contrary to the preferences of political majorities, courts, more than other political institutions, require a deep reservoir of goodwill.’ Simply put, courts cannot always rely on other institutions to comply with or enforce their decisions. As such, a large comparative judicial politics scholarship explores the different external constraints courts face and the actions of courts to mitigate such constraints (e.g., Ferejohn and Weingast Reference Ferejohn and Weingast1992; Helmke Reference Helmke2005; Staton Reference Staton2010; Vanberg Reference Vanberg2015). Heeding the call of Staton and Moore (Reference Staton and Moore2011), scholars have increasingly analysed the CJEU similarly to its domestic counterparts as a court whose decisions take into account the potential enforcement problem, the possibility of legislative override, and the potential for retribution against individual judges through appointments.

First, scholars argue that courts can compel policy-makers to comply with their decisions if they have sufficient public support for their decision-making. As a member of the German Bundestag explained, ‘There is not a single deputy here who thinks it would be advisable to move against the court. A serious confrontation would just create a public discussion in which one could easily get a bloody nose’ (Vanberg Reference Vanberg2005: 121). As such, Gibson and Caldeira (Reference Gibson and Caldeira1995, Reference Gibson and Caldeira1998) and Caldeira and Gibson (Reference Caldeira and Gibson1995) examined public support for CJEU decision-making three decades ago and generally conclude that the CJEU does not have a large store of public support and that EU citizens would not accept a CJEU decision that they find objectionable. While Kelemen (Reference Kelemen2012) pushes back against these claims, the scholarship regarding the public’s general willingness to accept CJEU decisions requires updating.Footnote ¹ Krehbiel (Reference Krehbiel2021) provides evidence that the CJEU is less likely to issue a pro-integration ruling as public awareness of its decisions increases – suggesting that, unlike in other courts (e.g., Staton Reference Staton2010; Vanberg Reference Vanberg2005) – the CJEU believes that public awareness may be detrimental to its ability to obtain compliance and conditions its decisions on such awareness. Other scholars argue that politicisation of issues in the general public has systematically altered CJEU jurisprudence in citizenship rights (e.g., Blauberger et al. Reference Blauberger, Heindlmaier and Kramer2018; Blauberger and Martinsen Reference Blauberger and Martinsen2020).

Second, scholars argue that legislative and executive institutions may affect judicial decision-making. Broadly speaking, if courts care about their influence over policy and constraining governments, they may anticipate legislative reactions to their behaviour because they believe that doing so allows them to secure better long-term outcomes (e.g., Ferejohn and Weingast Reference Ferejohn and Weingast1992; Rogers Reference Rogers2001) or – returning to the first point – that open defiance may erode their public support and systematically undermine its ability to affect policy in the future (e.g., Carrubba Reference Carrubba2009; Gibson et al. Reference Gibson, Caldeira and Baird1998). Although substantial debate existed as to whether the CJEU was responsive to the preferences of Member States (e.g., Garrett Reference Garrett1995) or largely insulated from them (e.g., Burley and Mattli Reference Burley and Mattli1993; Stone Sweet and Brunell Reference Stone Sweet and Brunell1998) (see Naurin, Lindholm, and Schroeder, Chapter 2 in this volume, and Dyevre (Reference Dyevre2023) for a summary of this debate), recent scholarship provides systematic empirical evidence that CJEU is responsive to Member State preferences (see Krehbiel et al. Reference Krehbiel, Gabel, Carrubba, Randazzo and Howard2017) for a comprehensive review). Carrubba et al. (Reference Carrubba, Gabel and Hankla2008), Carrubba and Gabel (Reference Carrubba and Gabel2015), and Larsson and Naurin (Reference Larsson and Naurin2016) provide evidence that the CJEU is responsive to threats of legislative override as Member States express through their observations to the court. Larsson (Reference Larsson2021) further specifies this claim by distinguishing between political and constitutional overrides, suggesting that political overrides are the prevalent form of legislative action. Building on this scholarship, Cheruvu (n.d.b) demonstrates that when the Council President submits an observation in a case, the CJEU will wait until after the presidency is over to issue its decision in a case to reduce the probability of the President initiating override legislation. Larsson et al. (Reference Larsson, Naurin, Derlén and Lindholm2017) argue that as an additional tool to combat override, the CJEU will more likely embed its decisions in existing case law to increase the persuasiveness of its arguments to reluctant Member States. Schroeder (Reference Schroeder2023) demonstrates empirically that the CJEU is less likely to show deference to national courts as it receives more observations from Member States. In addition to these direct overtures through briefs, the CJEU may also face explicit domestic modification and non-adoption in Member States as a form of resistance to its decisions (e.g., Martinsen Reference Martinsen2015a, 2015b). Castro-Montero et al. (Reference Castro-Montero, Alblas, Dyevre and Lampach2018) also provide evidence that the CJEU is less likely to rule against Member States when infringement cases coincide with treaty negotiations. Although the CJEU has had a systemic preference towards supranationalism in its jurisprudence (e.g., Ovádek Reference Ovádek2021; Pollack Reference Pollack1997), its judges are responsive to these constraints when making decisions.

Third, scholars provide evidence that the process by which governments appoint, promote, and discipline judges affects judicial decision-making (e.g., Canes-Wrone et al. Reference Canes-Wrone, Clark and Kelly2014; Dunoff and Pollack Reference Dunoff and Pollack2017; Helmke Reference Helmke2005; Ramseyer and Rasmusen Reference Ramseyer and Rasmusen2003). Indeed, governments that want favourable decisions from courts will be prudent to appoint judges that are sympathetic to their policy proposals (e.g., Bonica and Sen Reference Bonica and Sen2017). Substantial variation exists across the EU for how judges are appointed (Dumbrovsky et al. Reference Dumbrovsky, Petkova and Van Der Sluis2014), with some processes more insulated from politics relative to others. The fact the CJEU does not publish judges’ votes may affect judges’ responsiveness to political pressure. For example, Cheruvu (Reference Cheruvu2024) finds that CJEU judges are responsive when their appointing government sends observations, but are not as responsive to observations when the government of their appointing Member State turns over. This finding suggests that judges have some ideological affinity with their appointing government, but feel protected from retaliation by the per curiam nature of CJEU rulings. Nonetheless, despite per curiam decisions, Hermansen and Naurin (n.d.) provide evidence that appointing governments decide (not) to (re)appoint judges based on ideology. The CJEU in 2010 introduced the Article 255 panel as a merit commission to evaluate judges on the court. Cheruvu et al. (n.d.) demonstrate that the presence of the panel systematically influences judges’ effort investments into their opinion-writing and affects their efficiency. Moving forward, scholars can further examine how these appointment pressures affect judicial decision-making and how Member States can influence the CJEU’s judgments on cases through appointments.

13.6 What Can Empirical Legal Studies of the CJEU Add to Judicial Behaviour?

The international character of the CJEU may raise a number of unique considerations that may affect judicial decision-making in ways that are largely unaddressed by previous scholarship. First, the CJEU brings together judges from a variety of legal cultures and traditions. Scholars provide substantial evidence across contexts regarding the relationship between legal traditions and judicial decision-making (e.g., Garoupa Reference Garoupa, Epstein, Grendstad, Šadl and Weinshall2024). At the CJEU, and other international courts, judges across legal traditions are making decisions together. Such differences among legal traditions may require judges to compromise in ways that are uncommon in their domestic legal systems. Furthermore, judges at the CJEU come from a variety of linguistic backgrounds that may lead to different understandings of EU law and create frictions among judges who must use French in writing their judgments, even if it is not their native language (e.g., Cheruvu Reference Cheruvu2019). Judges, for example, have to wait until all of the documents in a case are translated into French before they are able to proceed in their adjudication process. As McAuliffe (Reference McAuliffe2011: 98) states, ‘Together with the difficulties of manipulating a language that is not one’s own, the result is often a stilted and awkward text.’

Yet, despite these potential obstacles to effective judging, scholars also provide evidence of substantial socialisation among judges (e.g., Vauchez Reference Vauchez2012) resulting in a broadly pro-integrationist jurisprudence over time (e.g., Ovádek Reference Ovádek2021), and a ‘“CourtFrench” which necessarily shapes the case law produced and has implications for its development, particularly insofar as it inevitably leads to a type of precedent in that case law’ (McAuliffe Reference McAuliffe2011: 99). This insight about the diversity of judges’ backgrounds, but supposed coherence of the decision-making of the institution, may play an important role in theory development that may go beyond the existing reach of the judicial behaviour typology. Although extensive scholarship discusses the development of collegiality among judges (Edwards Reference Edwards2003; Nelson et al. Reference Nelson, Hazelton and Hinkle2022) and its effects on judicial decision-making, scholars can spend more time theorising about how judges adapting to an unfamiliar multilingual environment may potentially dampen the correlation between ideology and judicial decision-making. Alternatively, scholars may posit that as judges learn and are socialised into a new institution (Vauchez Reference Vauchez2012), they are able to leverage this capital over time to achieve their goals (Hermansen and Naurin, n.d.). Put differently, multilingual and diverse environments such as the CJEU may increase a judge’s costs to develop new doctrine or advance their new agenda, thus creating a unique obstacle relative to domestic courts in achieving one’s preferred outcome in a case.

14.1 Introduction

The reception of EU rules by national courts, the interplay between EU and domestic judicial actors, and the ensuing process of legal integration are central aspects of EU law as we know it today. Without the collaboration of domestic courts, the constitutional doctrines spelled out by the Court of Justice in foundational rulings would have remained dead letters. So too would the expanding body of EU directives and regulations churned out by EU institutions.

The judicial construction of Europe and the day-to-day operations of the EU multi-level court system have raised many intriguing questions. Nothing, initially at least, was self-evident about domestic courts embracing the agenda pushed by the Court of Justice, which was still a relatively obscure supranational body in the early 1970s. So what was the actual catalyst for judicial cooperation? Another puzzle relates to the depth and robustness of legal integration given the non-hierarchical structure of the EU multi-level judiciary. How far can integration go when domestic rulings cannot formally be appealed to the Court of Justice? How can the effectiveness of EU law be preserved when Court of Justice does not hold the power to strike down non-compliant domestic judgements?

Doctrinal scholars were the first to try and offer answers to these questions. Their perspective emphasised legal rules and their judicial exposition, typically within a normative perspective – for example, when the acte clair doctrine may legitimately be invoked to decline to submit a reference for a preliminary ruling. Much of this doctrinal work has come in the form of idiographic analyses of particular rulings or doctrines which do not aim to build, let alone test, theories, although some doctrinal constructions have been more ambitious, with theories of constitutional and legal pluralism an interesting case in point (Avbelj and Komárek Reference Avbelj and Komárek2012; Kumm Reference Kumm, Dickson and Eleftheriadis2012; MacCormick Reference MacCormick1999; Walker Reference Walker2002).

What initially spurred the development of empirical perspectives on the EU multi-level judiciary was the interest of political scientists and interdisciplinary-minded legal scholars in integration dynamics and the role the Court of Justice seemingly played in fostering the effectiveness of EU rule-making (Burley and Mattli Reference Burley and Mattli1993; Weiler Reference Weiler1994). This, in the 1990s, made the reception of EU law by domestic legal actors a popular testing ground for grand theories of integration, thereby turning it into the cradle of EU empirical legal studies. While empirical analyses have since expanded to other aspects of the EU’s legal architecture, a sizeable literature has accumulated over the years (some of the older literature is surveyed in Conant Reference Conant2007; Krehbiel et al. Reference Krehbiel, Gabel and Carrubba2017; Stone Sweet Reference Stone Sweet2010). Recent work has built off from previous efforts (Brekke et al. 2022; Dyevre and Lampach Reference Dyevre and Lampach2022) but also explored new avenues, including infra-national spatial variations in referral activity at provincial or regional level (Dyevre and Lampach Reference Dyevre and Lampach2021; Kelemen and Pavone Reference Kelemen and Pavone2018), the effect of workload (Dyevre et al. Reference Dyevre, Glavina and Atanasova2020) or the impact of the use of simplified judgments on the domestic judges’ willingness to submit references (Dyevre et al. Reference Dyevre, Lampach and Glavina2022).

This chapter takes stock of this scholarship through a critical lens. Empirical studies of the reception of EU law by domestic courts have undeniably delivered crucial insights, especially about the interlocutory procedure created by Article 267 TFEU. At the same time, though, the field suffers from substantial limitations at several levels and so there is plenty of room to do more and better deploying new empirical legal studies (ELS) methods and smart designs. One limitation arises from the tendency to operate with EU-specific theories instead of drawing from the broader, multidisciplinary litigation and judicial behaviour literature. A second limitation is more methodological. The methodological assumptions underpinning some of the most prominent empirical findings fail to address the so-called ‘credibility crisis’ in quantitative research (Angrist and Pischke Reference Angrist and Pischke2010). As we shall see, there has been far too little attention to the issues plaguing causal identification. Also, practical hurdles stemming from a combination of disparate national publication practices and the resources that parsing large collections of legal texts used to require means that we have at best a very patchy picture of what is happening outside the specific context of Article 267 TFEU.

To address these shortcomings, I suggest how the field may benefit from incorporating theoretical advances from research on judges in other contexts, a stronger emphasis on smart designs and experimental and quasi-experimental methods, and the deployment of data-crawling and Natural Language Processing (NLP) techniques.

14.2 Theories

An oft-cited article by the Czech-born American legal scholar Eric Stein published in 1981 paved the way for the subsequent development of behaviour-centred and, later, empirical approaches to EU law. Stein described the European judicial process as a ‘complex dialectic process’ involving multiple actors and groups, encompassing the judges of the Court of Justice, lawyers in the national ministries and the Commission Legal Service, judges on national courts, the attorneys appearing before them, as well as legal scholars and writers (Stein Reference Stein1981).

Stein hinted at the role of national courts as one of several aspects of legal integration left for future inquiry. The question was picked up a decade later by Joseph Weiler, when he undertook to explain the increasing willingness of national courts to cooperate with the Court of Justice, notably via the preliminary ruling mechanism. Weiler offered an explanation that stood in sharp contrast to the legalistic narrative then dominant in European legal discourse. ‘[N]ormative acceptance of the ECJ constitutional construct and practical utilization through the Article [267] process by national courts’, wrote Weiler, ‘may be rooted in plain and simple judicial empowerment. Has not power been the most intoxicating potion in human affairs?’ (Weiler Reference Weiler1994: 523). What lured domestic courts into acceptance was the prospect of acquiring prerogatives, such as the right to set aside parliamentary legislation, either inexistent in their legal system or which had remained the exclusive preserve of constitutional judges. Weiler’s empowerment theory posited that the incentive for cooperation would be strongest for the courts at the bottom of the judicial pecking order, since those were also the ones standing to gain most (Weiler Reference Weiler1991: 2426).

Karen Alter took Weiler’s insight about the differential institutional incentive of domestic judicial actors one step further. She saw rivalry among domestic courts as an additional factor in the interest calculus of domestic courts. This factor, she argued, gave lower courts the incentive to operate as the engine of legal integration:

Where Weiler seemed to view Community-law-driven empowerment primarily as a means to expand the courts’ power against the other branches of government, Alter’s court competition theory drew attention to the power dynamics inside national court systems.

The neo-functionalist theories articulated by Anne-Marie Slaughter and Walter Mattli (Burley and Mattli Reference Burley and Mattli1993; Mattli and Slaughter Reference Mattli and Slaughter1998) and, later, Alec Stone Sweet and Wayne Sandholtz (Stone Sweet and Sandholtz Reference Stone Sweet and Sandholtz1997) represented another prominent theorising effort. Borrowing insights from the neo-functionalist theory of regional integration advanced by Ernst Haas in the 1950s, legal neo-functionalists recast the infra-state actors as litigants, domestic judges, and, in some instances, even law professors. At the supranational level, the Court of Justice was viewed as the leading advocate of integration (Burley and Mattli Reference Burley and Mattli1993: 58). The Court’s constitutional doctrines conferred upon private individuals rights directly enforceable in domestic courts, effectively creating a pro-EU constituency with a direct stake in promulgation and enforcement of Community law (Burley and Mattli Reference Burley and Mattli1993: 60). According to Burley and Mattli, the preconditions for integration were met when domestic judges, actively courted by their supranational peers and under intensifying pressure from a small but committed group of Community law professors, eventually overcame their initial misgivings as they began to perceive the benefits of integration (Burley and Mattli Reference Burley and Mattli1993). The interplay of these actors kick-started a process of functional and political spillovers, resulting in the self-reinforcing expansion of the substantive domain of Community law and incremental shifts in the expectations of both Member State courts and governments (Burley and Mattli Reference Burley and Mattli1993: 67).

Though embracing most of Burley and Mattli’s theoretical assumptions, Wayne Sandholz and Alec Stone Sweet offered a more rigorous articulation of the hypothesised causal mechanisms, which also put greater emphasis on transnational exchange as the initial catalyst of integration (Stone Sweet and Sandholtz Reference Stone Sweet and Sandholtz1997: 306). Because of the Rome Treaty’s initial focus on the creation of a Common Market and the removal of trade barriers, those engaged in cross-border economic exchange would form the constituency with the strongest incentive to actively seek the enforcement of Community law. Sandholz and Stone Sweet specified the following basic causal mechanism: transnational exchange > litigation > dispute resolution. Assuming that dispute resolution through the Court of Justice would, by removing national regulatory barriers, in turn foster transnational exchange, they added the feedback loop: dispute resolution > transnational exchange. The assumption that the growth in transnational exchange would spur demands for supranational regulation, which in turn would serve as basis for more litigation, motivated the specification of an additional causal pathway: transnational exchange > legislation > litigation. The hypothesised set of interlocked pathways provided a clear, potentially testable account of how the self-reinforcing dynamics of cross-border exchange, litigation, dispute resolution, and supranational rule-making would work to propel and self-reinforce the integration process.

In their ambitious theorising efforts, Stone Sweet and Sandholtz, although more rigorous than most of what had preceded it, were surprisingly short on detail regarding the motives driving domestic judges to collaborate and supply cases to the Court of Justice. In a related contribution, though, Alec Stone Sweet and Thomas Brunell provided some elaborations of the motives which, they assumed, spur Member State courts to apply Community law and refer cases to the Court of Justice. De-emphasising the logic of empowerment, they argued that efficiency concerns had more explanatory power:

If anything, Stone Sweet and Brunell were even more sceptical vis-à-vis Alter’s court competition theory, which, they argued, overlooked the division of labour inherent in national court systems: because a core function of appellate judging is to resolve disputes involving legal interpretation and conflict of law, we would expect the appellate courts to be far more involved in the construction of the legal system than Alter imagines them to be (Stone Sweet and Brunell Reference Stone Sweet and Brunell1998b: 90).

Some scholars approached the EU legal regime from the perspective of international relations. They typically paid scant attention to domestic judicial dynamics (Carrubba Reference Carrubba2005; Carrubba and Gabel Reference Carrubba and Gabel2015; Garrett and Weingast Reference Garrett, Weingast, Goldstein and Keohane1993; Moravcsik Reference Moravcsik1998). A very different line of research drew on the work of French sociologist Bourdieu to investigate the emergence of an EU-law field comprising law professors, judges, and legal advocates as a crucial force in the consolidation of the Court of Justice of the European Union (CJEU)’s authority and power (Cohen and Vauchez Reference Cohen and Vauchez2011; Vauchez Reference Vauchez2016; see also the pioneering work of Schepel and Wesseling Reference Schepel and Wesseling1997). Until the 2010s, though, the literature exhibited a marked tendency to treat EU judicial dynamics as a sui generis phenomenon without meaningful equivalents in the rest of the world. Rare were the contributions drawing on or explicitly seeking to contribute to research on judicial behaviour conducted in other contexts.

The account of inter-judicial cooperation set forth by Stone Sweet and Brunell, incomplete as it may be, bears some affinity with the team model of adjudication elaborated by the American law and economics scholar Lewis Kornhauser (Kornhauser Reference Kornhauser2012). The team model emphasises the desire of resource-constrained judges to maximise the number of ‘correct’ case resolutions. Judicial work is presumed to consist of three basic tasks: fact-finding, law-finding, and law-creation. These tasks are typically divided among the higher and lower echelons of the judicial hierarchy. Lower courts specialise in fact-finding but rely on the guidance of higher courts for law-finding and law creation. Rather than as manifestations of ideological divergence, appeals and reversals are themselves viewed as expression of informational asymmetry as appellate courts apply their greater resources to correct the legal and factual errors of the lower level (Kastellec Reference Kastellec2017; Kornhauser 1999). Franciso Ramos Romeu was the first to apply this model to the reception of EU law by national courts in a little-noted contribution (Romeu Reference Romeu2006). Dyevre et al. (Reference Dyevre, Glavina and Atanasova2020) also draw on the team model of adjudication. They argue that, since requests for a preliminary ruling to the CJEU may only address points of law, higher courts will be more active interlocutors of the CJEU within the preliminary ruling mechanism. They further supplement this division-of-labour argument with a theoretical elaboration of the work–life balance incentives arising from variations in the judges’ workload, thereby drawing on the labour market theory of judging developed by Epstein et al. (Reference Epstein, Landes and Posner2013) in the US context. To the extent that writing a reference for a preliminary ruling typically adds to a judge’s workload, a larger caseload increases the opportunity cost of submitting a preliminary reference.

In parallel with greater reception of theoretical advances arising from research on domestic courts, there have been attempts to develop a more general understanding of compliance partnerships involving domestic and international adjudicators to allow meaningful comparisons between the European experience and other legal regimes such as the Andean Community and the Inter-American Convention of Human Rights (Alter and Helfer (Reference Alter and Helfer2010) and Huneeus (Reference Huneeus2011)).

Overall, the empirical literature on the domestic dimension of EU law still draws on a somewhat restricted set of theoretical perspectives, even compared to other aspects of the EU legal regimes, such as the relationship between the Court of Justice and other EU decision-makers. Aside from policy goals and strategic considerations, the broad, multidisciplinary field of judicial behaviour research is now considering a much wider spectrum of influences on judging. These include the cognitive heuristics and biases affecting how judges evaluate facts, evidence, and legal arguments (Guthrie et al. Reference Guthrie, Rachlinski and Wistrich2007; Spamann et al. Reference Spamann, Klöhn and Jamin2021; Wistrich et al. Reference Wistrich, Guthrie and Rachlinski2005). Deserving mention, too, is a growing literature attempting to determine the extent to which legal rules actually constrain judicial decision-making (Bartels and O’Geen Reference Bartels and O’Geen2015; Fischman Reference Fischman2021; Pang et al. Reference Pang, Friedman, Martin and Quinn2012; Richards and Kritzer Reference Richards and Kritzer2002).

These theoretical innovations have found little echo in EU ELS. The flipside is that, going forward, there is considerable room to enrich the field’s theoretical understanding of judicial conduct in the EU court system and to spell out new hypotheses for future research. Although controversial judicial policies occasionally originate in seemingly mundane cases, comprehensive content analyses of CJEU cases indicate that they mostly pertain to technical issues such as VAT, tariff nomenclatures, and the mutual recognition of driving licenses (Dyevre and Lampach Reference Dyevre and Lampach2021a). This suggests that the judges who pass on cases to the CJEU are not primarily driven by competition with other courts or the desire to escape their domestic political constraints but, more plausibly, by the desire to satisfy prevailing norms of ‘correct’ dispute resolution which require that judges minimise legal errors (Kastellec Reference Kastellec2017; Kornhauser 1999). On that account, the CJEU may be seen as having developed such a degree of expertise that national courts, especially higher courts specialising in law-finding and law-creation, see it as efficient to refer legal questions. While reflections along these lines can be found here and there in the literature (see Dyevre et al. Reference Dyevre, Glavina and Atanasova2020; Dyevre et al. Reference Dyevre, Lampach and Glavina2022; Romeu Reference Romeu2006; Stone Sweet and Brunell Reference Stone Sweet and Brunell1998b), a systematic theorisation of judicial decision-making in routine cases is still largely missing from its overall picture of judicial decision-making.

Even in salient cases, and despite efforts to theorise overt defiance (Dyevre Reference Dyevre2016a, Reference Dyevre2016b), more attention should be given to the principal–agent problems potentially stemming from the absence of an appeal procedure between domestic and EU courts. In appeal-structured judicial hierarchies, losing parties have the strongest incentive to appeal when they expect that the appeal court will come to a different determination. This litigant selection effect help detect instances of agency drift, that is, decisions by judicial subordinates deviating from the preferences of judicial superiors (Cameron and Kornhauser Reference Cameron and Kornhauser2005). By contrast, in a system structured around an interlocutory procedure like the preliminary ruling procedure, such litigant selection effect does not exist. On the contrary, since preliminary references are submitted at the behest of domestic courts, they have stronger incentives to refer legal questions when they expect to like the CJEU’s answer. This implies that judicial misalignment between EU and national courts might be substantially more widespread than if the EU judiciary were structured around an appeal mechanism.

Also, susceptibility to cognitive biases and their impact on responsiveness to EU law arguments in briefs and pleadings or willingness to submit references may differ widely depending on the workload and resources of EU and domestic judges. The cognitive strain experienced when encountering unfamiliar EU law arguments, for example, may spur domestic judges, particularly those with large caseloads, to prioritise domestic law arguments. Knowing this, litigants also have the incentive to deploy domestic law arguments to which judges will be more receptive, thereby favouring a domestic law equilibrium. Research along these lines would help better understand obstacles to the reception of EU law in domestic judiciaries.

To sum it up, these theoretical perspectives, which were born in different corners (economics, political science, psychology) of the social science community, offer promising ways to renew how we think about the reception of EU law in the domestic realm. Ultimately, though, they will only prove fruitful if accompanied by improvements in methodology and data collection.

14.3 Data: Existing Limitations and Potential Fixes

Until the late 1990s, researchers relied principally on qualitative evidence – semi-structured interviews, case studies, and small, non-random samples of CJEU and national court rulings – to back their narratives of a field where integration was still the central research question (Alter Reference Alter, Slaughter, Stone Sweet and Weiler1998; Burley and Mattli Reference Burley and Mattli1993; Weiler Reference Weiler1991; see also Naurin, Lindholm, and Schroeder, Chapter 2 in this volume).

While preceded by Golub (Reference Golub1996), the efforts of Stone Sweet and Brunell to collect and encode data on referral activity (Stone Sweet and Brunell Reference Stone Sweet and Brunell1998b, Reference Stone Sweet and Brunell1998a) marked a major quantitative turn. Subsequently, referral rates have served as the primary source of empirical evidence to claim or deny support for the neo-functionalist hypothesis (Hornuf and Voigt Reference Hornuf and Voigt2015; Lampach and Dyevre Reference Lampach and Dyevre2017; Stone Sweet and Brunell Reference Stone Sweet and Brunell1998a); to criticise Alter’s court competition hypothesis (Dyevre et al. Reference Dyevre, Glavina and Atanasova2020; Stone Sweet and Brunell Reference Stone Sweet and Brunell1998b); to document the effects of workload and hierarchy implied by team and labour market models of adjudication (Dyevre et al. Reference Dyevre, Glavina and Atanasova2020); and to assess the influence of monist and dualist legal traditions on the domestic reception of EU law (Carrubba and Murrah Reference Carrubba and Murrah2005).

Most studies using Article 267 referral data aggregate the data at the country-year level, which, given the level of heterogeneity among Member States and the strong temporal trend in the data, severely limits its usefulness for testing causal hypotheses (see discussion in Section 14.4). More recent studies, starting with Kelemen and Pavone (Reference Kelemen and Pavone2016), break down countries in subnational units, which does represent an improvement but still constrains the ability of researchers to isolate the motives of courts and individual judges from the influence of litigants and other factors. The GEOCOURT Dataset (Dyevre and Lampach 2021b) and the National Courts component of the CJEU IUROPA Database (Brekke et al. Reference Brekke, Fjelstul, Lyder Hermansen and Naurin2023) have data collected at court level but they do not compile information on the courts that have never interacted with the CJEU via the preliminary ruling procedure. Mayoral and Wind (Reference Mayoral and Wind2022) conduct a court-level analysis but restrict the scope of their study to a subset of sixty-eight high courts. Likewise, Dyevre et al. (Reference Dyevre, Lampach and Glavina2022) only consider the courts that have submitted at least one reference at some point in time.

The problem with studying only the courts that participate in the preliminary ruling mechanism is that it may hide a major selection effect: it is quite possible that the courts that do not send reference to the CJEU differ fundamentally from the courts that do. Unless resources are invested in the construction of a comprehensive database identifying all judicial bodies across the EU, from the most modest first instance tribunal up to appellate courts, this selection effect will continue to inhibit progress on the domestic dimension of EU judicial decision-making.

Going one step further in investigating domestic judicial dynamics would necessitate judge or case-level data. Even at supreme court level, there has been little effort to compile information on panel composition over time, although the datasets assembled by Hamann (Reference Hamann2019) and (Lindholm et al. Reference Lindholm, Derlén and Naurin2022) on, respectively, the German and Swedish supreme court judges represent important exceptions.Footnote ¹ Garnering systematic case-level data on EU law outside the preliminary ruling mechanism is made arduous by the sheer quantity of domestic rulings as well as by linguistic barriers and vast disparities in reporting practices and online availability across courts and Member States. There has been no comparable attempt to Chalmers (Reference Chalmers2000)’s manual annotation of all reported judgments in UK over the period 1971–1998. The Dec.Nat. database covers a large number of cases and courts, going back to 1959 (Hübner Reference Hübner2015) but data collection did not follow explicit, replicable criteria, making its exploitation for quantitative analysis questionable at best. Proxies for EU law use, such as citations to Court of Justice rulings or EU law acts (see, e.g., Romeu Reference Romeu2006), although easier to extract from the mass of case reports, are no real substitutes. Not only is their frequency likely to be distorted by norms governing opinion writing, but they do not capture the nature and direction of citations.

US researchers have collected information on millions of federal appeal court and state supreme court cases, allowing the investigation of behavioural hypotheses about judges’ cognitive biases and attitudes (Ash and MacLeod Reference Ash and Bentley MacLeod2015; Ash et al. Reference Ash, Chen and Lu2018, Reference Ash, Chen and Galletta2022). Getting anywhere close to this mark in the EU context will necessitate an unprecedented data-collection effort. However, advances in NLP and data-crawling methods along with increasing ability to access digitalised legal databases via Application Programming Interfaces (APIs) shall greatly facilitate this enterprise (Dyevre Reference Dyevre2021; see also Ovádek in this volume). NLP methods are increasingly able to operate across languages, and supervised classification models trained in one language show convincing performance on unseen documents written in another (Artetxe and Schwenk Reference Artetxe and Schwenk2019; Chalkidis et al. Reference Chalkidis, Fergadiotis and Androutsopoulos2021). Even with a large sample of human-labelled documents, though, training a model to detect EU law use in domestic rulings is not a trivial task. The baseline rate at which domestic rulings refer to EU law appear low. It is probably well below 5 per cent of all reported domestic decisions (and this rate is likely much lower even in some Member State judiciaries). This entails that a classifier with 95 per cent sensitivity (i.e., correctly labelling 95 per cent of the cases that do refer to EU law) and 90 per cent specificity (i.e., correctly labelling 90 per cent of the cases that do not refer to EU law) would still produce many false positives. In fact, assuming a 5 per cent baseline prevalence (which, in most Member States, is likely to be the upper bound of visible EU law use), a case classified as referring to EU would only have a 33 per cent probability of truly referring to EU law.Footnote ²

14.4 Methods: Promise and Pitfalls

EU empirical scholars have deployed a variety of methods to study dynamics on and around the reception of EU law by national courts.

Case studies constitute an important source of knowledge about the EU multi-level legal system, as shown by studies examining landmark rulings such as Van Gend en Loos, Cassis de Dijon and Costa v. ENEL (Arena Reference Arena2019; Alter and Meunier-Aitsahalia Reference Alter and Meunier-Aitsahalia1994; Dyevre Reference Dyevre2011; Ovádek Reference Ovádek2022; Rasmussen Reference Rasmussen2014). Much of the best and most insightful work on the domestic reception of the Court of Justice’s constitutional jurisprudence in Germany, France, and Italy relies on case studies and qualitative analysis, including semi-structured interviews (Alter Reference Alter2001; Davies Reference Davies2012; Pavone Reference Pavone2022). Some seminal work on the role of professional networks in the emergence of a CJEU-supporting EU law field (Cohen and Vauchez Reference Cohen and Vauchez2011; Vauchez Reference Vauchez2016) similarly relies on largely qualitative analysis. By applying qualitative interview methods EU empirical scholars have also generated important insights about the beliefs and attitudes of domestic judges in Poland, Germany, the Netherlands, Slovenia, Sweden, and Croatia towards EU law and engagement with EU courts (Glavina Reference Glavina2021; Jaremba Reference Jaremba2012; Leijon Reference Leijon2024; Nowak et al. Reference Nowak, Amtenbrink, Hertogh and Wissink2011).

Quantitative descriptive studies have also made significant contributions to our understanding of the domestic dimension of legal integration. While prominent theories, including Weiler’s empowerment and Karen Alter’s court competition theory emphasised the role of lower courts (Alter Reference Alter, Slaughter, Stone Sweet and Weiler1998, Reference Alter2001; Weiler Reference Weiler1991), descriptive analyses of referral activity have shown that higher courts do in fact account for the bulk of references (Gerring Reference Gerring2004). This is true in relative terms (higher courts are far less numerous) and, since the turn of century, in absolute terms too (Dyevre et al. Reference Dyevre, Glavina and Atanasova2020; Stone Sweet and Brunell Reference Stone Sweet and Brunell1998b). Likewise, Dan Kelemen and Tomasso Pavone singled out the geographic concentration of referrals in France by comparing French courts to Italian and German ones (Kelemen and Pavone Reference Kelemen and Pavone2018). Examining data for all Member States, Dyevre and Lampach, however, later showed that centralisation is the common pattern across the EU (Dyevre and Lampach 2021b). Large-scale descriptive accounts in the form of thematic attention in cases involving EU law can also be informative. Dyevre and Lampach (Reference Dyevre and Lampach2021a), for instance, apply topic modelling and other text-mining techniques to compare issue attention across preliminary ruling, infringement, and annulment proceedings.

Empirical methods come with their strengths and weaknesses. Case studies and qualitative methods are well suited for exploration (especially when there are many unknown unknowns) to map the institutional environment or to illustrate a process or a mechanism. Their limitations are also well discussed. One is external validity. A case study describes what is in essence a single data point and the danger exists that too much is inferred from an overfitting description. Nor do case studies provide the strongest basis to test theories, unless the theory solely proffers existential propositions (Gerring Reference Gerring2004; King et al. Reference King, Keohane and Verba2021). Selection effects and even the aspects that the researcher picks out for description may induce a confirmation bias – particularly if the theory’s development was itself motivated by the case under study.Footnote ³

This said, making valid causal inferences with quantitative methods is no easy feat either, even when the data is complete and accurately measured (see the recent discussion in empirical constitutional studies (Chilton and Versteeg Reference Chilton and Versteeg2021; Petersen and Chatziathanasiou Reference Petersen and Chatziathanasiou2021). Looking back, early quantitative studies seem methodologically naïve. Stone Sweet and Brunell (Reference Stone Sweet and Brunell1998a), for instance, claimed to have found evidence for neo-functionalism in the strong correlation between intra-EU trade and referral activity. Their paper reported regression results with a cross-section of eleven Member States (taking average annual trade and references) and a cross-pooled analysis with only one control (with N = 246 and staggering adjusted explained variance R² = 0.77!). Their analysis ignored concerns of reversed causality and omitted variable bias as well as dependencies between observations (the same Member States are observed over time, not multiple random samples). CJEU rulings may stimulate trade, thus reversing the direction of causality (a point made by Pitarakis and Tridimas Reference Pitarakis and Tridimas2003), while Member State courts and judges differ in many observed and non-observed ways which might plausibly affect the rate at which courts use the preliminary ruling procedure.

Models reported in subsequent publications adopted assumptions better suited to the data structure (e.g., panel data analysis with negative binomial distribution for referral activity per country over time) and often included many more controls for potential confounders (see, e.g., Carrubba and Murrah Reference Carrubba and Murrah2005; Kelemen and Pavone Reference Kelemen and Pavone2016). Some cross-country studies report models with country and year fixed effects (e.g., Kelemen and Pavone Reference Kelemen and Pavone2018) which go some ways towards capturing unobserved country-specific time-invariant effects as well as unobserved short-term events and shocks affecting entire cross-sections of countries. However, while this mitigates the problem of causal identification, it does not definitely solve it. Referral rates, for example, exhibit a strong upward trend without pronounced peaks and troughs. This means that any variable exhibiting a temporal trend is bound to be (either positively or negatively) correlated with referral activity. For this reason we should take claims about the effect of EU-law specialised firms or the intensity of cross-border trade (as measured by EU trademark registrations) on referral rates (Kelemen and Pavone Reference Kelemen and Pavone2018; Lampach et al. Reference Lampach, Wijtvliet and Dyevre2020) with a grain of salt.

Problems of this sort have precipitated the credibility crisis in economic research (Angrist and Pischke Reference Angrist and Pischke2010). Because randomisation ensures that treatment and control groups share similar observed and non-observed characteristics, making causal identification comparatively neat and straightforward, the upshot of the methodological debate has been to elevate the randomised controlled experiment, whether in the field or in the lab, to the gold standard for causal identification (Rubin Reference Rubin2007).

The credibility crisis did not sound the death knell of observational studies. Instead, it has spurred efforts to develop smart research designs. Researchers have increasingly turned to natural and quasi-natural experiments (DiNardo Reference DiNardo, Durlauf and Blume2010), instrumental variable estimation (Angrist et al. Reference Angrist, Imbens and Rubin1996) and methods that mimic experimental settings by constructing control groups and synthetic counterfactuals (Abadie Reference Abadie2021; Abadie et al. Reference Abadie, Diamond and Hainmueller2015; Sekhon Reference Sekhon2009; Stuart Reference Stuart2010). Dyevre et al. (Reference Dyevre, Glavina, Lampach, Ovádek and Wijtvliet2018), for instance, seek to estimate the impact of the Brexit referendum on British courts by creating a synthetic UK judiciary to serve as counterfactual.

However, such methods are still far from being mainstream. Inevitably, as methodological standards for causal identification improve, many past findings will be revisited and questioned. Ultimately, this shall help place our understanding of EU judicial behaviour on stronger empirical foundations.

14.5 Conclusion

The literature surveyed in this chapter has substantially advanced our understanding of how domestic courts engage with EU law, particularly through the lens of the preliminary ruling mechanism under Article 267 TFEU. Empirical research has mapped out patterns of interaction between national judiciaries and the CJEU, revealing valuable insights into the dynamics of judicial cooperation and conflict within the EU’s multi-level legal system. By analysing referral activity, scholars have illuminated the roles that institutional incentives, court hierarchies, and legal traditions play in shaping the reception of EU law across Member States.

Yet the empirical literature remains constrained by EU-specific theoretical frameworks, often overlooking the broader disciplinary advances in judicial behaviour and institutional analysis. This insularity limits the field’s ability to generate comparative insights and test hypotheses that might apply beyond the EU context. Methodologically, the literature has not always lived up to the highest standards, notably with respect to causal inference, casting doubt on some prominent findings.

While the study of legal integration can also be viewed as the field’s specific contribution to judicial behaviour research, there is considerable room to enrich the theoretical debate with insights from research on judicial hierarchies, jurisprudential regimes, or the influence of cognitive biases. Research shall also benefit from a greater emphasis on randomisation, natural experiments, and synthetic control methods. Data is still a challenge. But as this chapter has tried to show, there are solutions there too. NLP, machine learning, and web-crawling methods promise to improve data collection and facilitate research on the domestic dimension of the EU court system.

15.1 EU Law Litigation and Empirical Legal Studies

An outsider perusing studies of litigation in the EU might be struck by how they ‘have a distinctly American air about [them]’ (Kelemen Reference Kelemen2011: 5). And for good reason. Just as empirical legal studies (ELS) in its most influential contemporary form developed in the interwar US before making its transatlantic crossing (Pavone and Mayoral Reference Pavone, Mayoral, Bartl and Lawrence2022), so too was the post-World War II US a progenitor of empirical research on litigation (Trubek Reference Trubek1990) before Europeanists took up this agenda and analysts began worrying about adversarial legalism spreading to the European Union (EU) (Kagan Reference Kagan1997). A telltale sign of this crossover is that most of the pioneers of EU legal mobilisation and litigation studies were trained in the US (ex. Alter Reference Alter2000, Reference Alter2001; Alter and Vargas Reference Alter and Vargas2000; Conant Reference Conant, Risse, Caporaso and Green Cowles2001, Reference Conant2002; Cichowski Reference Cichowski1998, Reference Cichowski2004, Reference Cichowski2007) and built on scholarship on the rule of law and judicialisation spearheaded by US-based scholars (Burley and Mattli Reference Burley and Mattli1993; ex. Scheingold Reference Scheingold1965; Stone Sweet Reference Stone Sweet1999; Weiler Reference Weiler1994).

Today, empirical studies of EU law litigation are more home-grown than transplants. Thanks to a new generation of scholars, we know a lot more about which interest groups litigate, and whether litigation substitutes or complements other advocacy strategies (Hofmann and Naurin Reference Hofmann and Naurin2020; Lejeune and Ringleheim Reference Lejeune and Ringelheim2023; van der Pas Reference van der Pas2024; Vanhala Reference Vanhala2018); the role that activist lawyers, scholars, and judges have played in mobilising European law and soliciting national court referrals to the Court of Justice of the EU (CJEU) (Avril Reference Avril2019; Byberg Reference Byberg2017a, Reference Byberg2017b, Reference Byberg2017c; Mayoral and Perez Reference Mayoral and Pérez2020; Pavone Reference Pavone2019, Reference Pavone2022; Vauchez Reference Vauchez2015); how varying levels of litigation have shaped EU policies toward the environment, consumer protection, disability rights, migration, labour, and free movement (Hofmann Reference Hofmann2025; Krommendijk and van der Pas Reference Krommendijk and Van der Pas2022; Lejeune Reference Lejeune2017; Passalacqua Reference Passalacqua2021; Pavone Reference Pavone2019; Vanhala Reference Vanhala2009; Zglinski Reference Zglinski2024); and why EU law remains unmobilised in particular places or among classes of prospective litigants (van der Pas Reference van der Pas2024; Passalacqua Reference Passalacqua2022; Pavone Reference Pavone2020a, Reference Pavone2020b, Reference Pavone2022).

We unpack some of the foregoing insights in this chapter, but we also aim to resuscitate the fertility of the US–EU comparison that sparked this field of study in the first place. By comparing whether well-established patterns of legal mobilisation and strategic litigation in the US also arise in the EU, we can ponder the relative distinctiveness of the European context. We focus on one phase of legal mobilisation – litigation – while recognising that it constitutes ‘just one potential dimension or phase of a larger, complex, dynamic, multistage process of disputing’ (McCann Reference McCann, Caldeira, Kelemen and Whittington2008: 525). While research in both the US and EU suggests that the ‘haves’ (businesses and interest groups clustered in cities) disproportionately litigate, we will show that EU-centered studies disagree about whether litigation capacity begets judicial influence and downstream effects. To advance this debate, we borrow from Galanter (Reference Galanter1983a) and ‘read the landscape of disputes’ by drawing on an original dataset of parties involved in all preliminary references to the CJEU from 1958 to 2018 (Hermansen et al. Reference Hermansen and Voeten2024). We query (1) who litigates, (2) who influences judicial decisions, (3) with what downstream ‘radiating effects?’ (Galanter Reference Galanter, Boyum and Mather1983b). As we will demonstrate, although the corporate ‘haves’ are ubiquitous participants in EU law litigation, they are neither its sole nor arguably most effective agents: individual ‘have nots’ punch above their weight.

15.2 Who Goes to Court?

15.2.3 Our Data: Individuals, Businesses, and Organised Interests in Preliminary References

Where case studies have examined the strategic choices of interest groups and surveys have measured their litigation tendencies, we have lacked a comprehensive picture of who litigates before the CJEU and their broader impact, such as shaping legal integration or attracting attention. To address this, we analyse a dataset of 7,353 national court referrals (preliminary references) to the Court from 1957 to 2018, starting with the key question: who litigates?

Figure 15.1 shows the types of litigants involved in preliminary references to the CJEU. The procedure enables private actors to challenge national policies with EU legal claims. The Court is often depicted as a business-oriented forum where corporations and trade associations push economic liberalisation (Kahraman Reference Kahraman2023; Scharpf Reference Scharpf2010). Consistent with this view, businesses make up a large share of the caseload (62 per cent), alongside public institutions and different social welfare organisations (70 per cent). Yet two patterns stand out: a substantial number of cases involve individual litigants, while advocacy groups are strikingly underrepresented.

Figure 15.1

Types of litigants involved in preliminary references to the CJEU (1957–2018).

Figure 15.1 Long description

The X-axis represents the number of cases, ranging from 0 to 5000. The graph plots bars for public institution, company, individual, N G O, other, and E U institution. The values are 5100, 4550, 3450, 700, 200, and 25, respectively. Note, all values are approximated.

The patterns in Figure 15.1 are even more striking when considering that EU law grants relatively few justiciable rights to individuals compared to businesses. Surprisingly, some 47 per cent of the preliminary references involve individual litigants – a significantly higher share than individuals’ presence on the US Supreme Court docket before the civil rights era (Epp Reference Epp1998). Why, then, are individuals such active participants in EU law litigation? And what does this reveal about the Court’s role in European integration?

A closer look at the policy areas (‘subject matter’) where individuals litigate sheds light on the first question (Figure 15.2). Their involvement with EU law is often tied to cross-border economic activities, which seems to align with the CJEU’s reputation as an economic court. Individuals sometimes litigate on similar issues to – or even alongside – businesses, particularly in disputes concerning the products they sell and the taxes they pay (‘agriculture and fisheries,’ ‘free movement of goods,’ ‘competition,’ ‘intellectual property,’ ‘taxation’).

Figure 15.2

CJEU cases involving individual litigants (1957–2018), classified by their subject matter.

Figure 15.2 Long description

The X-axis represents the number of court cases, ranging from 0 to 400. The Y-axis lists the topics on which individuals litigate. Social policy, 410. Area of freedom security and justice, 320. Social security for migrant workers, 320. Freedom to provide services, 300. Agriculture and fisheries, 250. Free movement of workers, 170. Taxation, 160. Free movement of goods, 99. Consumer protection, 60. Transport, 50. Environment, 45. Citizenship of the Union, 43. Free movement of capital, 40. Fundamental rights, 25. External relations, 20. Competition, 20. Principles, objectives, and tasks of the treaties, 20. Anti-discrimination, 20. Intellectual, industrial, and commercial property, 20. Staff regulations and employment conditions, 15. Provisions governing the Institutions, 5. Note, all values are approximated.

Yet, a significant portion of cases involve relatively resourceless individual litigants in disputes over social rights (‘social policy’, ‘social security’, and ‘staff regulations and employment conditions’), often overlapping with cross-border activities, such as litigation on the free movement of workers and service providers (e.g., self-employed migrants). The Court’s longstanding case law grants rights to migrant workers and their families, adding a crucial social and human element to what might otherwise be seen as a purely (neo-)liberal economic project (Caporaso and Tarrow Reference Caporaso and Tarrow2009). Figure 15.2 also illustrates the Court’s central role in EU-level social policies (Martinsen Reference Martinsen2015b). Deprived of the fiscal capacity to pursue redistributive policies, the EU instead acts as a litigation-centered regulatory state, creating rights that citizens can – and have – claimed (Kelemen Reference Kelemen2011).

Lastly, many individuals are involved in litigation on relatively new or increasingly salient EU policy areas. These include consumer rights, antidiscrimination, and fundamental rights – fields where EU legislation has become central and increasingly attracts lawyers’ and scholars’ attention (Dyevre et al. Reference Dyevre, Glavina and Ovádek2021). The second-most frequent topic involves individuals’ interactions with the justice system, particularly within the Union’s area of freedom, security, and justice. This reflects the EU’s broader goal of harmonising justice standards across borders. For example, the Schengen Agreement and the European Arrest Warrant (EAW) ensure mutual recognition in criminal matters and the handling of asylum seekers.

From an EU scholar’s perspective, these topics are neither new nor surprising. However, from a legal mobilisation standpoint, the implications are puzzling: Individuals play a far greater role in EU law litigation than in the American context, and many CJEU cases rely on the participation of relatively dispossessed groups – migrants, workers deprived of income, asylum-seekers, and (suspected) criminals. What explains their presence? Answering this question opens fertile avenues for future research. To what extent have organised interests empowered individuals and/or business interests – in line with Epp (Reference Epp1998)’s argument about ‘litigation support structures’? Or building on Conant et al. (Reference Conant, Hofmann, Soennecken and Vanhala2016), what are the EU-level factors – the stock of EU law and rights, the receptivity of the Court itself, and the support provided by Euro-lawyers – that may explain the difference with our American example? As evidenced by the distribution of policy areas, clearly the European legal stock – the legal protections given by the EU legislator – structures the litigation we see. New policy areas such as judicial cooperation and social policies, as well as the EU’s stricter standards on data protection may catalyse substantial changes in the role of individuals also in the future.

The legal mobilisation literature also highlights the pivotal role of interest groups in shaping judicial agendas. However, unlike the US Supreme Court, which accepts input through amicus curiae briefs, the CJEU only admits observations by Member States and EU institutions, whereas standing is determined by national regulations (Case and Givens Reference Case and Givens2010). Consequently, many organisations operate behind the scenes, remaining absent in the Court’s official documents. No comprehensive overview of interest organisations’ involvement in CJEU litigation exists to date. Yet in our data, we find that only about 10 per cent of preliminary references during the period involved such organisations. Their limited presence contrasts sharply with the legislative arena, where the Commission often forms proactive alliances with diverse interest groups (Hix and Høyland Reference Hix and Høyland2022: 234–39).

Figure 15.3 shows significant cross-national differences in cases involving professional organisations and cause groups. Litigation by professional organisations is most common in northern and southern Europe. Denmark, known for its strict political control over preliminary references (Rytter and Wind Reference Rytter and Wind2011; Wind Reference Wind2010), has the highest share of cases involving industry/employer organisations and trade unions. Labour unions are also highly active in Denmark, followed by Finland, Sweden, France, and Spain. Trade associations feature prominently in references from Italy, France, Spain, Belgium, the UK, Denmark, Estonia, and Finland.

Figure 15.3

National differences in interest group involvement in preliminary reference cases.

Figure 15.3 Long description

The top left map presents the share of labor unions. The color gradient scale ranges from light to dark shades, representing the share of references from 0 to 0.10. The map indicates that countries such as Germany, the U K, and certain Scandinavian countries have a higher proportion of references, while countries in Eastern Europe and parts of Southern Europe have a lower proportion. The top right map presents the share of professional associations. The color gradient scale ranges from light to dark shades, representing the share of references from 0 to 0.10. This map indicates a slightly different pattern, with a higher share of references in countries like Germany, Switzerland, and some parts of Scandinavia. The bottom left map presents the share of trade associations. The color gradient scale ranges from light to dark shades, representing the share of references from 0 to 0.10. This map highlights a higher share of references in countries like Germany, Italy, and Spain. The bottom right map illustrates the share of cause groups. The color gradient scale ranges from light to dark shades, representing the share of references from 0 to 0.6. This map shows a higher share of references in countries like Germany, Austria, and some parts of Scandinavia. In all four maps, the countries, including Ukraine, Belarus, and Norway, have the lowest value.

Figure 15.4 highlights the dominance of corporate actors among organised interests. Trade associations outnumber labour unions and professional associations by a ratio of 3 to 1. This aligns with Olson’s (Reference Olson1965) observation that diffuse interest organisations face greater coordination challenges than business groups. Labour unions and professional associations often view labour mobility as a threat to their members’ economic activities, making EU law litigation for migrant rights less appealing. Egan (Reference Egan2014) notes a similar tradeoff for professional organisations in her comparative study of interstate labour mobility in the US and EU. These groups set standards that shield members from outside competition. Listing hundreds of state-regulated professions on both sides of the Atlantic, Egan underscores that integration remains incomplete even in federal systems.

Figure 15.4

Types of interest groups involved in preliminary reference cases.

Figure 15.4 Long description

The X-axis represents the number of cases, ranging from 0 to 300. The Y-axis represents 9 organized interests in E C J preliminary reference cases. Trade association, 297. Professional association, 95. Labor union, 95. Cause group, 70. Environment, 60. Consumer, 50. Other, 47. Leisure, 35. Humanitarian, 20. Note, all values are approximated.

Unlike labour unions and professional associations, trade associations are generally assumed to support European-level regulation, as a single framework facilitates cross-border trade. Sandholtz and Zysman (Reference Sandholtz and Zysman1989) illustrate this dynamic, showing how business interests lobbied the Commission and national governments to advance the single market. Stone Sweet and Sandholtz (Reference Stone Sweet and Sandholtz1998) expand on this logic by conceptualising ‘transactors’ as businesses engaged in cross-border trade: by bringing lawsuits that trigger preliminary references, transactors help dismantle trade barriers, fostering further trade and demand for the Court’s services. This dynamic, however, does not seem to generalise to business organisations. In our data, only five of 137 trade associations are international; most are national producer organisations. Can we really expect these organisations to push for dismantling protectionist policies? To preview the answer we unpack in the next section: no.

15.3 Who Influences Judges?

15.3.3 Our Data: Who Are the Vehicles of European Legal Integration?

The Court’s decisions often align with the observations of the Commission and Member State governments. However, neither actor sets the agenda in preliminary references, which are mainly initiated by private parties, including individuals, businesses, and interest groups. To what extent can these parties influence the CJEU’s decisions, and what are the implications for European integration?

To address this question, we draw on data from the IUROPA project, which tracks litigants’ and observers’ positions in 3,246 legal questions from 1,748 preliminary references adjudicated between 1995 and 2011 (Lindholm et al. Reference Lindholm, Naurin and Šadl2023). We focus on which private parties are most effective at advancing European integration by limiting national autonomy. The logit model in Table 15.1 and Figure 15.5 estimates the likelihood of a restriction in national autonomy, based on the involvement of various actors: professional organisations, cause groups, individuals, public institutions, and others. Cases involving two businesses serve as the baseline, and the model controls for the net number of governments supporting restrictions.

Table 15.1Court rulings to restrict national autonomy in preliminary references in the 1995–2011 period (logistic regression, reference level for all the predictors is businesses)

	Dependent variable:
	Restrict national autonomy
Labour union involved (ref. business)	0.882***
	(0.291)
Trade association involved (national)	−0.402**
	(0.173)
Trade association involved (international)	0.795
	(0.597)
Professional association involved	−0.205
	(0.292)
Cause group involved	0.369*
	(0.216)
Individual involved	0.375***
	(0.076)
Public institution involved	−0.125
	(0.098)
Other actor type involved	−0.080
	(0.308)
Net government support for restriction of national autonomy	0.242***
	(0.021)
Constant	0.332***
	(0.101)
Observations	3,246
Log Likelihood	−2,278.336
Akaike Information Criterion	4,576.671

Note: ^*p<0.1; ^**p<0.05; ^***p<0.01

Figure 15.5

What are the vehicles of European integration? Types of actors linked to restrictions in national autonomy (Illustration of the results presented in Table 15.1).

Figure 15.5 Long description

The X-axis represents log-odds, ranging from negative 1 to 3. The Y-axis represents the effect of organized involvement on the restriction of national autonomy, reference level is businesses. The minimum, median, and maximum values for the following are. Labor union involved, 0.3, 0.9, and 1.45. Trade association involved, international, negative 0.4, 0.8, and 1.95. Individual involved, 0.3, 0.4, and 0.5. Cause group involved, negative 0.05, 0.4, and 0.8. Other actor type involved, negative 0.7, negative 1, and 0.5. Public institution involved, 0.2, 0.4, and 0.5. Professional association involved, negative 0.75, negative 0.2, and 0.4. Trade association involved, negative 0.7, negative 4, and negative 0.1, respectively. Note, all values are approximated.

We find little evidence that businesses are effective liberalisers driving European integration. Instead, individuals, labour unions, and cause groups are significantly associated with decisions restricting national autonomy. Of these, individuals exert the greatest influence. Their involvement implies a 46 per cent higher likelihood of a restriction compared to cases involving only businesses. Although rare, labour union cases are also impactful, being 2.4 times more likely to lead to such restrictions. Similarly, cases involving cause groups, such as those advocating for environmental or consumer protection, are associated with a 45 per cent higher likelihood of restricting national autonomy. In contrast, national trade associations are linked to fewer restrictions, with a 33 per cent reduction in likelihood compared to cases involving only businesses. EU-wide trade associations show the opposite trend, but this finding is not statistically significant. Professional associations also appear linked to preserving national autonomy, though this effect is likewise not statistically significant.

The findings challenge influential studies of EU law litigation. Where Stone Sweet and Sandholtz (Reference Stone Sweet and Sandholtz1998) identified businesses as ‘transactors’ driving European integration, we find that individuals, more than businesses, fulfill this role: their cross-border activities have significantly shaped the CJEU’s integrationist case law (Zglinski Reference Zglinski2024). Furthermore, the predominance of national, rather than EU-wide, trade associations tends to produce decisions that uphold protectionist policies. Rather than dismantling borders, business associations often reinforce them.

Our findings raise pressing questions for future research. How much of individuals’ influence stems from agenda-setting – bringing transformative cases to the Court – or from their legal capability, such as strong arguments or skilled lawyers? Hermansen et al. (Reference Hermansen, Tommaso and Boulaziz2025) show that individuals generally enjoy lower-quality legal representation than businesses, so unlike decades of research in the American context, it seems that resource inequalities do not explain individuals’ high litigation success. Contrary to assumptions of pro-business bias, could the CJEU have a pro-individual bias?

15.4 What Are the Downstream Effects of Litigation?

15.4.3 Our Data: Unequal Academic Attention

For CJEU rulings to spur follow-up litigation and bottom-up compliance pressures, the legal community must be aware of emerging legal opportunities (Hermansen et al. Reference Hermansen, Tommaso and Boulaziz2025; Weiler Reference Weiler1994). Law journals play a crucial role in dissemination, serving practitioners scouting litigation strategies, national judges seeking precedents, and future lawyers in training. This heightened awareness of CJEU cases contributes to the radiating effects of litigation, extending beyond direct policy compliance to knowledge diffusion.

Our analysis examines the number of law journal commentaries on preliminary rulings (1957–2018) as a function of the types of parties involved. The first model (Table 15.2) compares the mean number of commentaries across national and international law journals, while the second focuses on the Common Market Law Review (CMLR). Established in the 1950s as part of the CJEU’s outreach efforts (Byberg Reference Byberg2017c), the CMLR publishes commentaries on only about 10 per cent of cases. To account for national variation in references and publication practices, the first model includes random intercepts for the Member States where cases originate. Does the type of litigants involved draw varying levels of attention from legal scholars?

Table 15.2Academic attention to CJEU cases involving different types of litigants (results from two linear models; reference level is businesses)

	Dependent variable
	All journals	CMLR
	linear	logistic
	mixed-effects
	(1)	(2)
Labour union	2.770***	−0.524
	(0.875)	(0.468)
Trade association	0.027	0.190
	(0.505)	(0.216)
Professional association	1.817**	0.329
	(0.841)	(0.314)
Cause group	3.496***	0.963***
	(0.615)	(0.202)
Other NGOs	1.652***	0.111
	(0.560)	(0.242)
Individual	0.705***	0.441***
	(0.204)	(0.092)
Public institution	−3.921***	−0.345***
	(0.221)	(0.093)
Other organisation	0.472	0.107
	(0.707)	(0.297)
Constant	7.647***	−2.558***
	(0.309)	(0.097)
Observations	7,345	7,353
Log Likelihood	−26,086.530	−1,935.700
Akaike Information Criterion	52,195.060	3,889.401
Bayesian Information Criterion	52,270.980

Note: ^*p<0.1; ^**p<0.05; ^***p<0.01

Unsurprisingly, given our review of EU legal mobilisation scholarship, both models indicate that cases involving cause groups attract significant academic attention. Such cases generate 3.5 more articles on average than cases involving only businesses and are 2.6 times more likely to be discussed in at least one CMLR commentary. Though rare, these cases often disrupt national autonomy, reflecting our earlier finding that cause groups and individuals are more likely to spur integrationist case law. A similar pattern appears for cases involving individuals. On average, these cases generate twice as many commentaries across all venues and are 55 per cent more likely to attract commentaries in the CMLR than businesses (see Figure 15.6). This aligns with Dyevre et al. (Reference Dyevre, Glavina and Ovádek2021)’s observation that legal scholars disproportionately focus on rulings involving fundamental rights raised by individuals rather than cases involving competition and taxation typically brought by corporate litigants.

Figure 15.6

The probability of a case commentary in the Common Market Law Review (CMLR) as a function of the type of litigants involved (comparisons are done with cases involving businesses only).

Figure 15.6 Long description

The X-axis represents log-odds, ranging from negative 2 to 2. The Y-axis represents academic attention to cases where interest groups are involved, results from a linear model, reference level is businesses. The following are the minimum, median, and maximum values. Cause group involved, 0.6, 0.9, and 1.3. Individual involved, 0.3, 0.4, and 0.6. Professional association involved, negative 0.3, 0.4, and 0.9. Trade association involved, negative 0.4, 0.1, and 0.6. Other N G O involved, negative 0.4, 0.1, and 0.6. Other involved, negative 0.5, 0.1, and 0.6. Public institution involved, negative 0.5, negative 0.3, and negative 0.2. Labor union involved, negative 1.1, negative 0.5, and 0.3, respectively. Note, all values are approximated.

Comparing across models, it seems national law journals are more sensitive to challenges to national autonomy than the European-wide focus of the CMLR. For instance, cases involving labour unions and trade associations receive significantly more attention than those involving businesses in the broader academic community, but this effect is absent in the CMLR. This highlights how radiating effects, such as knowledge diffusion through academic journals, are influenced by the actors and stakes in litigation, deepening our understanding of strategic litigation’s downstream impacts. It also shows that the economic advantages enjoyed by businesses and trade associations do not necessarily translate into radiating effects. As US-based studies suggest, being a sympathetic litigant often garners more attention than being a resourceful one.

15.5 Conclusion

EU law litigation is an excellent propellant for understanding legal mobilisation and strategic litigation more broadly. Our goals in this chapter were threefold: (1) to compare litigation in the EU to well-established litigation patterns in the US; (2) to surface who litigates EU law, who influences CJEU decisions, with what downstream effects; and (3) to spotlight striking empirical patterns that can serve as springboards for future research.

Does EU law litigation match litigation patterns in the US? Yes: in some respects – but more importantly, no in others. Yes: when we compare litigation of US federal and constitutional law with litigation of EU law, resourceful litigants capable of lawyering-up and absorbing the costs of litigation tend to dominate the scene. The corporate ‘haves’, often clustered in economically prosperous and trade-prone cities, are protagonists in both contexts. Our results also support the inference that labour unions are reluctant litigators in both contexts, particularly compared to trade associations.

But beyond these similarities, what is most striking are the empirical differences that emerge when comparing the EU and American contexts, alongside the surprising patterns of EU law litigation that beg for further analysis. Individuals, for instance, comprise a majority of litigants involved in referrals to the CJEU across 14 Member States, and these individuals cannot be dismissed as ‘haves’ in disguise: they tend to be migratory workers, asylum seekers, and retirees. Yet despite their relative dispossession, it is individuals (alongside labour unions and cause groups, although the latter are infrequent litigators) who exert the highest influence over CJEU decisions. And this judicial influence cuts against existing presumptions: they – not businesses and trade associations – are the primary drivers of European integration, because it is they who tend to catalyse liberalising CJEU rulings that impose restrictions on national autonomy. Contra some portrayals of EU law litigation as a neoliberal corporate tool, businesses and trade associations are less effective in influencing CJEU decisions, and when they do it is usually to safeguard protectionist policies. Furthermore, the ‘haves’ appear to have little advantage in catalysing a key radiating effect of litigation: attracting attention and spurring debates within the legal community.

On balance, then, it is individuals who appear to punch above their weight in the EU context: individuals litigate EU law more than we might expect, are surprisingly effective at influencing CJEU decisions in ways that advance European integration, and are magnets of issue attention, particularly among legal professionals who are well-positioned to organise follow-up litigation and pro-compliance campaigns. To invoke a musical analogy, it is as if businesses and trade associations are the nominal protagonists in an opera – they have more lines and nearly constant stage presence – yet the most striking and memorable arias are delivered by a supporting cast of individuals, cause groups, and to a lesser extent labour unions. What makes an effective protagonist in an opera? Or, what makes an effective litigant in the EU legal system? There are many answers to this question, because like an opera, EU law litigation is a complex, evolving, disharmonious, multistage process. But if we had to venture an answer, it is this: the ‘haves’ are certainly protagonists of EU law litigation, but they are neither the sole nor arguably the most effective protagonists.

16.1 Introduction

The founders of the European Union set up a system to resolve collective action problems that was unique in international law. In order to secure compliance and the credibility of joint commitments, the drafters of the Treaties tasked one of the EU’s institutions, the European Commission, with monitoring compliance and conferred upon it the ability to initiate administrative and judicial proceedings ‘for failure to fulfil legal obligations’ (Article 258 TFEU). Over the years, the Commission has developed a broad toolkit to bolster its monitoring and enforcement powers (Smith Reference Smith, Drake and Smith2016). Where conflicts over compliance cannot be resolved in dialogue with the offending Member State, the Commission can call on the Court of Justice of the European Union (CJEU) to adjudicate. Member State governments – as High Contracting Parties (Article 1 TEU) – have strengthened the judicial enforcement procedure by allowing for substantial fines against Member States that have been found to have failed obligations. In a less anticipated fashion, the CJEU created a parallel channel for – de-centralised – enforcement of EU law. In declaring EU law supreme and directly effective, from the mid 1960s onwards, citizens, groups, and companies could challenge public authorities before national courts for alleged infringements of EU law (Pavone Reference Pavone, Gallo and Cecchetti2024). This combination of centralised and de-centralised enforcement ensures that the European Union remains a uniquely effective supranational organisation.

While the legal outlines of this enforcement framework have not changed significantly since the introduction of financial penalties at Maastricht,Footnote ¹ the channels by which EU law has been enforced in practice over these three decades have shifted dramatically. We require real-world data to trace these shifts – legal texts and policy documents often diverge starkly from the empirics. Research on Member State compliance with EU law has long relied on empirical data, but an empirical focus on enforcement has lagged. The reflexive reference to the Commission as the ‘Guardian of the Treaties’, for example, ignores that the number of infringement procedures it pursues has precipitously declined since the Barroso II Commission. As Figure 16.1 shows, numbers of CJEU judgments in infringement procedures are back to levels not seen since the 1980s, when the Union was much smaller in size and scope.Footnote ² By contrast, the number of de-centralised proceedings over the same period has strongly increased.

Figure 16.1

CJEU judgments by procedure.

Figure 16.1 Long description

The y-axis represents the number of C J E U judgments, ranging from 0 to 400. The x-axis represents the years, starting from 1960 and extending to 2020. The graph plots two categories of judgments, namely, preliminary reference and infringement procedure. Preliminary reference follows a general upward trend with mild fluctuations from 1 in 1960 to 400 in 2019, and 370 in 2024. The infringement increases from 0 and reaches its highest at 150 in 2024, which decreases with fluctuation. Note, all values are approximated.

This development has not gone unnoticed by academic commentators. In previous work, I have outlined how the Commission actively drives this process by facilitating conditions for private, de-centralised enforcement (Hofmann Reference Hofmann2018, Reference Hofmann2019). Gerda Falkner has argued that the development should be ‘read in the context of mounting doubts if, or how much and at what price, infringements can at all be stopped against the will of the relevant government’ (Falkner Reference Falkner2018: 770). More recently, Daniel Kelemen and Tommaso Pavone have conducted interviews with Commission officials that pointed to political pressures as an impulse for the observed shift (Kelemen and Pavone Reference Kelemen and Pavone2023). They suggest that the Commission has actively retreated from centralised enforcement in order to garner goodwill from Member State governments for newer policy projects. Other contributions highlight a greater willingness by the Commission to tolerate non-compliance where compliance would significantly impact national legal orders (Zhang Reference Zhang2022), and an increased emphasis on informal compliance-promoting tools (Smith Reference Smith, Drake and Smith2016).

The outlines of the development are now well established. Less attention, however, has been paid to the substance of Commission enforcement action. As this chapter will show, not only is the Commission doing less, its substantive and geographic focus of activity has also changed markedly over the last decade. From a previous emphasis on internal market rules, transport, taxation, and the environment, only the latter remains a substantive priority. A small but increasing percentage of cases pursued by the Commission today concern justice and migration, including rule of law backsliding in Poland and Hungary, ‘golden passport’ schemes in Malta, Cyprus, and Bulgaria, and rules concerning police and judicial cooperation throughout the Union. The geographic focus of enforcement action has concomitantly shifted from the previous ‘usual suspects’ – Spain, Italy, Greece and Portugal – to more recently joined Member States.

This contribution investigates this substantive and geographic shift. I will use data on EU legislative activity and data on the infringement procedure to draw a profile of Commission enforcement activity over the years. In doing so, I demonstrate how empirical research based on diverse data sources can be employed to answer questions about the real-world effect of EU ‘law on the books’. The chapter proceeds as follows. The following section summarises previous findings on three factors that drive Commission enforcement activity: the degree of legislative production in the EU, Member State implementation record, and the Commission’s own enforcement policy. The next section compares this to empirical findings on the current state of centralised enforcement. First, I look at how the Commission describes its own priorities and challenges in enforcement. I then present the data sources employed in the empirical overview. This is followed by, first, an overview of the decline in centralised enforcement measured against legislative production, and second, a look at geographic and substantive shifts in Commission enforcement action. A final section concludes and reflects on the trajectory of empirical research in this field.

16.2 Centralised Enforcement

In the present context, the term enforcement describes the process by which a Member State is brought to comply with EU law by means of (a threat of) sanctions following a legally prescribed procedure (Tallberg Reference Tallberg2002: 611). Compliance, in turn, describes ‘a state of conformity or identity between an actor’s behavior and a specified rule’ (Raustiala and Slaughter Reference Raustiala, Anne-Marie, Carlsnaes, Risse and Simmons2002: 539). While de-centralised enforcement of EU law relies on private actors to bring cases in national courts, centralised enforcement is in the hands of the European Commission. As the EU’s oft-heralded ‘Guardian of the Treaties’, the Commission monitors the implementation and application of EU law by the Member States, and it works towards securing Member State fulfilment of their obligations (Article 17 TEU). Existing research has highlighted that the amount of activity the Commission undertakes to do this broadly rests on three conditions: first, the number of legislative acts that create legal obligations, coupled with the number of Member States that are required to apply them, second, the degree of Member State compliance, and third, the Commission’s own ability and proclivity to allocate resources to enforcement. Together, these three factors should explain the variance in aggregate infringement cases over any given period of time. However, for the external observer, all three come with measurement problems. I will elaborate one by one.

16.3 Commission Enforcement Action

The following section first looks at how the Commission itself describes its enforcement policy and then compares this to available data on how it carries it out in practice. These data will focus on the ‘visible layer’ of Commission enforcement activity (Smith Reference Smith, Drake and Smith2016: 49), and specifically on the final stage of the formal infringement procedure – the referral of a case to the CJEU. It is a conscious choice that omits the less visible (or invisible) layers of centralised enforcement, which potentially take up more of the Commissions time and resources than legal action, but which are difficult to systematise. Referrals to the CJEU are a specific subsection of centralised enforcement and we have no means of ascertaining whether it is representative of enforcement activity as a whole. As Melanie Smith writes, ‘reasons for not proceeding with a case can be complex, resource driven, strategy driven, administrative, and often relate to the particular culture of the DG (and each Commissioner) and the surrounding political circumstances’ (Smith Reference Smith, Drake and Smith2016: 66). Nonetheless, referrals cover those areas of conflict where Member States are not willing to concede to Commission demands and where the Commission is willing to cease control over the process by calling upon a third party to adjudicate.

16.3.3 Trends in Commission Enforcement Action

What exactly has happened to centralised enforcement? Data on the Commission’s use of the infringement procedure paints a rather clear picture. The Commission has drastically scaled down this layer of its enforcement activity. The historical peak of judicial enforcement can be located with the Prodi and Barroso I Commissions. The enforcement activity profiles of the Juncker and von der Leyen I Commissions resemble more the Commissions under presidents Gaston Thorn in the early 1980s and Jaques Delors in the early 1990s, a time when Commission terms were shorter, the EU had less than half the number of Member States, and was much narrower in policy scope.

Figure 16.2 shows the total number of court referrals from 1978 to November 2024.Footnote ¹⁰ It separates the underlying cases into two types: one type based on the failure by Member States to notify transposition measures to the Commission, and one type capturing all other kinds of infringements. Since the first type gets triggered more or less automatically when a Member State misses a deadline for communicating transposition measures, the second type is sometimes called ‘discretionary’ to underline the fact that these are cases the Commission actively chooses to pursue. Non-notification cases made up the majority of Commission enforcement actions from the mid 1990s to the end of the first decade of the 2000s, but they have only played a minor role since the Barroso II Commission. I am not aware of any arguments in the literature that the Commission might be strategic about the pursuit of non-notification cases. The introduction of the expedited sanction procedure in Article 260(3) TFEU in cases of non-notification may have contributed to increased Member State discipline as they anticipate costs (Smith Reference Smith, Chalmers and Arnull2015: 366). The decline in centralised enforcement has been less stark but still very pronounced with regard to discretionary proceedings. In these cases, too, directives are the focus of Commission enforcement action.

Figure 16.2

Number of referrals 1978–2024.

Figure 16.2 Long description

The Y-axis represents the number of referrals, ranging from 0 to 600. The X-axis represents C J E U judgments by procedure. The graph plots bars for no and yes of non-notification cases. Jenkins from 1977 to 1981, 5 and 2. Thorn from 1981 to 1985, 140 and 30. Delors 1 from 1985 to 1989, 200 and 190. Delors 2 from 1989 to 1993, 190 and 110. Delors 3 from 1993 to 1994, 60 and 110. Santer from 1994 to 1999, 210 and 380. Prodi from 1999 to 2004, 500 and 600. Barroso 1 from 2004 to 2009, 570 and 630. Barroso 2 from 2009 to 2014, 280 and 140. Juncker from 2014 to 2019, 190 and 70. Von der Leyen 1 from 2019 to 2024, 140 and 90, respectively. Note, all values are approximated.

Figure 16.3 shows the types of legal acts cited in discretionary cases that the Commission has decided to refer to the CJEU.Footnote ¹¹ The Commission sometimes combines the enforcement of secondary law with a reference to primary law, such as in a case against Germany, initiated in 2015, in which the Commission held that German minimum and maximum tariffs for architects and engineers infringed both the Services Directive and Article 49 TFEU on the free movement of services.Footnote ¹² Other cases rest solely on primary law, such as a 2020 case against a Maltese investor citizenship scheme which the Commission found to infringe on the principle of sincere cooperation (Article 4(3) TEU) and Union citizenship (Article 20 TFEU).Footnote ¹³ Both categories are not rare but have been eclipsed by the enforcement of directives since the Santer Commission. Regulations and decisions are very rarely subject to infringement proceedings. The data contain only thirty-six cases based on decisions and 219 cases based on regulations that the Commission has decided to refer to the CJEU since 1978. The Commission’s admission to deficits in the enforcement of regulations is clearly borne out in the data.

Figure 16.3

Type of act infringed.

Figure 16.3 Long description

The Y-axis represents the number of discretionary referrals, ranging from 0 to 300. The X-axis plots for decision, directive, regulation, treaty, and treaty plus secondary. Jenkins from 1977 to 1981, 1, 2, 3, 3, and 2. Thorn from 1981 to 1985, 5, 70, 10, 70, and 15. Delors 1 from 1985 to 1989, 5, 60, 30, 50, and 40. Delors 2 from 1989 to 1993, 3, 70, 15, 40, and 50. Delors 3 from 1993 to 1994, 1, 20, 5, 10, and 20. Santer from 1994 to 1999, 2, 130, 20, 30, and 35. Prodi from 1999 to 2004, 2, 330, 40, 80, and 45. Barroso 1 from 2004 to 2009, 5, 320, 65, 120, and 50. Barroso 2 from 2009 to 2014, 3, 180, 20, 50, and 25. Juncker from 2014 to 2019, 5, 120, 20, 30, and 20. Von der Leyen 1 from 2019 to 2024, 2, 180, 20, 20, and 20, respectively. Note, all values are approximated.

Given the centrality of directives, one possible explanation for the decline in centralised enforcement is a concomitant decline in the production of new directives by the EU legislature. Figure 16.4 depicts aggregate statistical data on EU legislative output available on EUR-Lex for every month since 1990.Footnote ¹⁴ While these data do not suggest a decline in legislative activity overall, there has been a clear shift away from directives and regulations and an increase in the number of decisions. This reflects restrained policy activism that began with the Barroso II Commission (Kassim et al. Reference Kassim, Connolly, Dehousse, Rozenberg and Bendjaballah2016: 666). Fewer legislative directives mean fewer transposition deadlines that Member States can miss, and hence fewer automatic non-notification cases. A simple bivariate regression of the variables underlying Figures 16.2 and 16.4 shows that the number of legislative directives produced by the legislature during the term of a Commission explains about 75 per cent of the variance in the number of CJEU referrals.Footnote ¹⁵ Even for discretionary cases, the percentage of the variance in referrals explained by the number of legislative directives produced per Commission term of office is also rather high (65 per cent). While Kelemen and Pavone (Reference Kelemen and Pavone2023: 793) found no correlation between Commission enforcement and the number of new legislation combined with the number of Member States that need to implement it (what Börzel calls ‘violative opportunities’, Börzel Reference Börzel2021b), ignoring the number of Member States and focusing on new legislative directives alone yields a strong relationship. The Commission clearly appears to focus much of its enforcement work on new legislative directives, but the accession of new Member States has not resulted in increased activity.

Figure 16.4

New legislative acts.

Figure 16.4 Long description

The Y-axis represents the number of legislative acts, ranging from 0 to 2000. The graph plots directives, regulations, and decisions. Delors 2 from 1989 to 1993, 0 to 250, 250 to 1400, and 1400 to 1750. Delors 3 from 1993 to 1994, 0 to 100, 100 to 800, and 800 to 1050. Santer from 1994 to 1999, 0 to 200, 200 to 1350, and 1350 to 2200. Prodi from 1999 to 2004, 0 to 300, 300 to 1300, and 1300 to 2500. Barroso 1 from 2004 to 2009, 0 to 300, 300 to 250, and 250 to 2600. Barroso 2 from 2009 to 2014, 0 to 100, 100 to 700, and 700 to 2200. Juncker from 2014 to 2019, 0 to 150, 150 to 600, and 600 to 2400. Von der Leyen 1 from 2019 to 2024, 0 to 100, 100 to 600, and 600 to 2700, respectively. Note, all values are approximated.

Given the discretion the Commission enjoys in its enforcement policy, such a focus reflects a decision on the part of the Commission on how to allocate scarce resources. Regulations, too, require an adaptation of national behaviour, but as described, Commission documents confirm that this is not a focus (European Commission 2023: 16–17). Moreover, as the Commission’s own implementation reports demonstrate (European Commission 2020, 2022b), older pieces of legislation also give frequent rise to conflicts over compliance. The Commission’s enforcement policy does not seem to give much room to this. Finally, one area of decline in enforcement that cannot be explained by legislative production is primary law. Previous Commissions had a strong focus on primary law in bringing centralised enforcement actions, while the Juncker and von der Leyen I Commissions have hardly pursued this. This can be read as a strategic decision that gives support to the impression of a declining importance of negative integration in the EU, which some authors have recently highlighted (van den Brink et al. Reference van den Brink2023; Zglinski Reference Zglinski2024). At the same time, it casts doubt on the Commission’ assertion that its enforcement actions emphasise ‘fundamental rights and fundamental freedoms’ (European Commission 2017: 9).

16.3.4 Geographic and Substantive Shifts in Centralised Enforcement

The overview so far has focused on overall numbers of Commission decisions to refer infringement cases to the CJEU. In the following, I will look at geographic and substantive shifts in Commission enforcement action. Since my interest is in shifts in Commission enforcement policy, that is in the way the Commission strategically allocates resources to enforcement, I focus here on discretionary cases and exclude cases pursued for non-notification of transposition measures.

Figure 16.5 illustrates the geographic focus of Commission action in discretionary cases. The picture here is quite striking. Almost the entirety of the decline in discretionary enforcement action pertains to the fifteen ‘old’ Member States. The overall number of cases concerning the ‘new’ twelve Member States has not substantially changed since they became Member States. The von der Leyen I Commission referred an almost equal number of cases to the CJEU relating to both groups of Member States, despite there being fewer ‘new’ Member States and their smaller population size. The emphasis of Commission enforcement action has evidently shifted eastward, and it is expending less effort towards the old ‘usual suspects’.

Figure 16.5

Old and new Member States.

Figure 16.5 Long description

The X-axis represents the number of discretionary referrals, ranging from 0 to 500. The graph plots bars for the member states of the old and new. Barroso 1 from 2004 to 2009, 500 and 350. Barroso 2 from 2009 to 2014, 200 and 70. Juncker from 2014 to 2019, 120 and 70. Von der Leyen 1 from 2019 to 2024, 70 and 60, respectively. Note, all values are approximated.

Figure 16.6 further differentiates by the most prominent ‘targets’ of Commission enforcement action at the referral stage. What is striking here is the ‘disappearance’ of Italy and Spain as the foci of Commission enforcement action. This disappearance began with the Barroso II Commission and found its completion with von der Leyen I. The Barroso I Commission made 92 (discretionary) decisions to refer Italy to the CJEU, while the von der Leyen I Commission did so only 10 times, the same number as against Slovakia. The coincidence of this development with the unfolding Eurocrisis seems noteworthy, and Kelemen and Pavone allude to this in their research (Kelemen and Pavone Reference Kelemen and Pavone2023: 802). As a result, there are no longer any ‘usual suspects’ in Commission enforcement action. Italy and Spain are now ‘normal’ Member States, and, if anything, ‘newer’ Member States are now overrepresented in centralised enforcement action. Contrary to what might be expected, Commission action against democratic backsliding in Hungary and Poland is not driving this pattern – Poland in particular had already been a focus of attention during the Barroso I Commission, and cases against Poland have since declined. Nonetheless, the Juncker and von der Leyen I Commissions have pursued enforcement action against democratic backsliding, and this represents an entirely new and challenging focus of centralised enforcement (Anders and Priebus Reference Anders, Priebus, Lorenz and Anders2021). However, research has shown that the Commission did not pioneer a creative use of legal sources to counter such backsliding (Mandujano Manriquez and Pavone Reference Mandujano Manriquez and Pavone2024), but followed legal innovations developed by the CJEU in response to claims by sophisticated domestic litigants (Ovádek Reference Ovádek2023).

Figure 16.6

Individual Member States.

Figure 16.6 Long description

The X-axis represents the number of discretionary referrals, ranging from 0 to 75. The graph plots bars for the member states, namely Italy, Greece, Spain, Portugal, France, Hungary, Malta, and Slovakia. Barroso 1 from 2004 to 2009, 13, 55, 60, 45, 23, 37, 1, 13, and 3. Barroso 2 from 2009 to 2014, 20, 24, 23, 15, 25, 23, 10, 2, and 3. Juncker from 2014 to 2019, 17, 15, 17, 15, 17, 10, 17, 2, and 1. Von der Leyen 1 from 2019 to 2024, 10, 15, 7, 10, 13, 3, 10, 7, and 10, respectively. Note, all values are approximated.

Figure 16.7 depicts shifts in the substantive focus of Commission enforcement action over time. The data contain information on the DG in charge of the file, from which I infer the policy area that the file pertains to. Figure 16.7 focuses on four policy areas that have traditionally been prominent foci of Commission action (the environment, the internal market, taxation and customs union, and mobility and transport) as well as ‘justice and migration’Footnote ¹⁶ as a newer field for centralised enforcement. The internal market had historically been the focus of Commission enforcement activity until it lost importance during the Barroso II Commission. This decline is much more pronounced than for other policy areas. I am not aware of a specific explanation for this development, but it coincides with the rise to prominence of the SOLVIT administrative network, which handles internal market conflicts. Writing in 2022, Dorte Sindbjerg Martinsen, Ellen Mastenbroek, and Reini Schrama described this network as issuing ‘more problem-solving cases than the European Commission sends out opening letters as part of its infringement procedure’ (Martinsen et al. Reference Martinsen, Mastenbroek and Schrama2022: 1532). Taxation eclipsed the internal market during Barroso II and the Juncker Commission, but virtually disappeared during von der Leyen I. Cases relating to mobility and transport increased up to and including the Juncker Commission. The von der Leyen I Commission, however, had only one clear focus: environmental cases. Almost half of all cases that the von der Leyen I Commission decided to refer to the Commission pertain to the environment. The environment first became the most frequently enforced policy area in the late 1990s during the Santer Commission and it has remained the Commission’s (now sole) focus since. It is in fact the only policy area that has had a more or less steady number of cases since the big drop from the Barroso I to the Barroso II Commission. Finally, the area of justice and migration has seen a slow but steady increase in focus since it first emerged during the Santer Commission, but overall case numbers are still low.

Figure 16.7

Policy areas.

Figure 16.7 Long description

The X-axis represents the number of discretionary referrals, ranging from 0 to 150. The graph plots bars for the policy area, namely environment, internal market, taxation and customs union, mobility, and justice and migration. Delors 1 from 1985 to 1989, 15, 50, 47, 3, and 0. Delors 2 from 1989 to 1993, 40, 50, 35, 2, and 0. Delors 3 from 1993 to 1994, 1, 20, 3, 2, and 0. Santer from 1994 to 1999, 70, 45, 27, 24, and 4. Prodi from 1999 to 2004, 185, 120, 45, 26, and 10. Barroso 1 from 2004 to 2009, 153, 165, 75, 35, and 10. Barroso 2 from 2009 to 2014, 69, 35, 73, 26, and 10. Juncker from 2014 to 2019, 48, 17, 30, 47, and 17. Von der Leyen 1 from 2019 to 2024, 60, 23, 10, 10, and 20, respectively. Note, all values are approximated.

It is difficult to say that this substantive focus, including drastic changes over time, could be predicted based on the Commission’s stated enforcement priorities. None of the general policy documents make specific mention of the environment as a prime priority. Only the increase in cases brought in the field of justice and migration can be tied to the Commission’s stated emphasis on fundamental rights issues, but their comparatively small number makes it hard to speak of a genuine priority. Unstated sectoral strategies seem better candidates for an explanation, as would be a closer examination of the organisation and resources of enforcement units within Commission DGs (European Commission 2023: 9; European Court of Auditors 2018: 26–28).

Figure 16.7 also replicates patterns of litigation that Urška Šadl, Lucía López Zurita, and Sebastiano Piccolo found in citation networks between CJEU judgments more broadly (Šadl et al. Reference Šadl, López Zurita and Piccolo2023). They demonstrate that the CJEU is moving away from case-law that relies on market freedoms to liberalise domestic markets, towards case-law that shapes (harmonises) European markets by way of secondary legislation. Liberalising judgments decline in numbers, whereas harmonising judgments increase (Šadl et al. Reference Šadl, López Zurita and Piccolo2023: 849). I find similar trends in Commission enforcement action, which might be an indication that this development is driven by litigants as much as the CJEU itself. This fits with recent work by Martijn van den Brink, Jan Zglinski, and Mark Dawson, who also find a shift in emphasis from negative to positive integration in the internal market (van den Brink et al. Reference van den Brink2023). Similar to Šadl, López Zurita, and Piccolo (Šadl et al. Reference Šadl, López Zurita and Piccolo2023: 840), I also find a slow increase of cases in the field of justice and migration. Commission enforcement in this sense is aligned with broader legal developments that are driven by an interplay of litigants, national courts, and the CJEU.

16.4 Conclusions

This chapter has explored empirical data on recent developments in the centralised enforcement of EU law. It summarised previous literature on Commission enforcement practices and contrasted this with data on the actual use of the infringement procedure, focusing on referrals to the CJEU – the last stage of the procedure that is still under control of the Commission. I explored three factors that influence real-world behaviour. The first is the EU’s legislative output and the number of Member States that need to implement it. The data show a strong relationship between the number of referrals and the number of new legislative directives produced by the EU legislature, but no effect of the number of regulations and decisions, which are rarely subject to enforcement action. This is acknowledged by the Commission, which only recently highlighted the need to develop instruments for the monitoring of the application of regulations. The number of Member States that are required to implement EU law also does not seem to influence Commission activity – the 2004 enlargement did not lead to increased enforcement. In fact, data on the geographical ‘targets’ of infringement action show that the Commission has drastically reduced its activities vis-à-vis the fifteen ‘old’ Member States, whereas the amount of activity directed towards the ‘new’ twelve Member States has not changed significantly since 2004. Previously prominent targets of enforcement action, first and foremost the Italian government, no longer stick out. Commission enforcement no longer has any ‘usual suspects’.

The second factor discussed was Member State implementation performance, or, broadly speaking, Member State ‘compliance’. Literature has highlighted the Commission’s complete discretion in pursuing infringements and its strategic selection of cases, which gives rise to the impression that Commission action does not necessarily reflect the ‘true’ state of compliance. Independently assessing compliance is a daunting task for research. Moreover, the concept of compliance presupposes an agreement on what behaviour is mandated by rules, and a broad literature has shown that in the past, the Commission used its powers of enforcement to pursue political objectives, such as the breaking up of public monopolies, employing interpretations of legal obligations that were not shared by the original signatories of the European Treaties. However, the data give little indication that the Commission still pursues such objectives today.

The third factor is the Commission’s own enforcement policy. Part of the literature on this issue is devoted to a normative critique of the opacity of Commission action in enforcement. My data do not allow me to address this. What I did offer, though, is a comparison between the Commission’s own stated priorities and its activity profile. One take-away here is that unpublished sectoral priorities seem to be driving Commission activity more than the general objectives stated in central policy documents. The only discernible substantive focus of Commission enforcement action is environmental law, which the general policy documents hardly mention. Empirical research can highlight such discrepancies between text and the real world. Commission documents, however, also stress efforts to increase access to effective domestic remedies for EU law infringements. This does correspond to empirical findings on the Commission’s efforts to ‘outsource’ enforcement to private actors. More recently, the Commission’s focus on domestic remedies has expanded to broader efforts to protect the rule of law, and indeed the data show growing activity of the Commission in this field, albeit with low overall numbers. The Commission’s stated emphasis on ‘fundamental freedoms’ does not translate into a continued focus on market integration. The ‘four freedoms’ (free movement of goods, services, capital, and workers) have largely disappeared from the Commission’s agenda, as have internal market rules more broadly, which have gone from prime priority to afterthought.

I will conclude with three observations. First, empirical data on Commission enforcement action is comparatively easy to come by and lends itself to fruitful analysis of broader developments in law and policy. Rather than use such data to infer the state of Member State compliance, data produced by the Commission can be used much more readily to learn about the Commission’s own political priorities. Second, since enforcement is central to the real-world impact of EU law on the ground, doctrinal research on procedural administrative law, remedies, and access to courts can very fruitfully be combined with empirical data on the actual use of enforcement procedures. Third, more can be done to connect empirical data on centralised enforcement with data on de-centralised enforcement. The Commission’s ongoing retreat from centralised enforcement leaves gaps in the effectiveness of EU law that can only partially be compensated by private legal action. Such private legal action is comparatively easy to trace if it includes a preliminary reference to the CJEU, but the life of EU law on the ground can only be traced with comprehensive data on national proceedings, which are harder to come by. This should be a focus of future research.

17.1 Introduction

The Court of Justice of the European Union (the CJEU or the Court) has been central to the development of the European project, and its role in fostering European integration is widely acknowledged. From early on, the Court was presented as the motor of EU integration, capable of issuing key legal decisions with a high transformative power and then relying on its authoritative force to have them implemented by national courts.Footnote ¹ This role has, however, not gone unnoticed or uncriticised. The Court has frequently been accused of activism, favouring EU integration over Member State preferences and economic goals over social ones. Ultimately, the main complaint relates to the Court’s alleged capacity to set the direction of EU integration: legal integration impacts on political integration; domestic policies and political preferences are overturned by the Court and the supremacy of EU law.

Whereas judicial power is often said to mark EU integration, the actual impact of the CJEU and its ability to generate policy change remains disputed. What happens after a ruling is issued by the Court is, to an important extent, still a black box and occupies a space of stark disagreements in EU law and politics. In this chapter, we examine how the rulings of the CJEU might impact public policy at the EU (horizontal) and national (vertical) levels, that is, the extent to which the Court’s decisions shape the policy output of other processes.Footnote ² Consequently, our understanding of public policy is broad and includes any choice by political, executive, and judicial actors.Footnote ³ We further consider different decision types (from contained compliance to over-compliance), including instances in which the decision of the political actors is to not act (negative or non-decisions).Footnote ⁴

While there is no agreed framework for directly assessing the impact of the Court in the design, modification, and implementation of policy, most accounts focus on the extent to which the Court is capable of driving integration. These theories explore the extent to which the CJEU can push integration against the political preferences of other actors, or by taking those preferences into account. The divide between intergovernmentalism and neofunctionalism remains the central point of debate. Most of the discussion revolves around how other actors influence and react to the rulings of the Court, and how the Court takes this input into account. Thus, both theories provide a good entry point to think about the Court’s impact on policy. However, we will argue in this chapter that we need to move beyond this divide to properly address impact as an autonomous area of study.

Intergovernmentalist approaches highlight the influence of Member States’ preferences on the decisions taken by the CJEU.Footnote ⁵ Intergovernmental logic argues that politics drives integration, and that national policy preferences define judicial outcomes.Footnote ⁶ The Court is likely to make a decision in the sense of preserving national interests if several Member States have argued for such an outcome in their interventions.Footnote ⁷ The reason for this is that the Court has a limited impact when its rulings contradict the preferences of the Member States, who could override that decision with new or amended legislation or lack of implementation. Overturning or defying a ruling means refusing its general impact and isolating the design of public policy at both the EU and domestic level from the influence of the Court.

For neofunctionalists, the Court is relatively insulated from the influence of the Member StatesFootnote ⁸ and can take decisions independently of the preferences of Member States or, more generally, of other political constraints. The possibility of override is remote and does not significantly influence the Court. Moreover, the Treaties are ‘incomplete contracts’ and require specification in legislation and judicial decisions.Footnote ⁹ In terms of impact, this implies that the Court can define policy at the EU level and in the national legal orders, as the possibility of overturning its decisions is, at best, slim.

While these theories remain central in understanding the political impact of the Court, they are less specific about the concrete impact of its rulings in designing, modifying, and implementing policy at the EU and national levels. Yet, a lot of research in the last two decades has tried to investigate precisely that, building on different theories, methodologies, and disciplines, and frequently focusing on concrete case studies in specific policy areas. While these different literatures shed light on several aspects, they do not speak to each other and, consequently, provide only a patchy or fragmented knowledge of the impact of the Court on policy. It is here where empirical legal studies (ELS) can provide a research agenda aiming to achieve a global view of this phenomenon and bridge the distance between the different disciplines and methodologies. Our premise is clear: without empirical research, any estimation of the real impact of the Court’s case law is necessarily incomplete, and the knowledge thus remains to a large extent disconnected from the reality on the ground.

The chapter is structured as follows: Section 17.2 presents the state of the art. As the literature cuts across the fields of both law and political science, our aim is to connect the different approaches. We first summarise the existing research on how the CJEU’s decisions impact on EU policies (horizontal impact) and then examine how the decisions of the Court impact on three policy dimensions at the domestic level (vertical impact): (1) national legislation, (2) administrative practices and enforcement, and (3) national judicial decisions. Section 17.3 reflects on where research gaps remain and how ELS can contribute to filling them.

17.2 State of the Art in EU Law

This section maps out the literature on the policy impact of the CJEU’s decisions. In order to provide a complete picture of the impact of the decisions in policy,Footnote ¹⁰ when analysing the potential influence of the Court’s rulings we take into account the three components of public policy as defined by Hall:Footnote ¹¹ (a) the abstract or general policy goals pursued, (b) the concrete policy instruments to implement them, and (c) the operational settings or calibrations used when the instruments are deployed.Footnote ¹² Therefore, we look not only at the possible influence of the Court in setting a legislative agenda, but also on how it might affect the concrete formulation of the policy and/or its implementation at the street level.

This section does not study the policy process as such, but it does consider its different stages (agenda setting, policy formulation, decision-making, policy implementation, and policy evaluation).Footnote ¹³ It does so along two axes: horizontal and vertical. The former refers to the impact of the Court on policy at the EU level and thus focuses on how other EU actors are constrained, or not, by the case law of the Court. The latter explores the impact of the Court’s rulings at the domestic level. This distinction is useful to systematise the literature, but these two axes are not completely disconnected: for instance, Member States might be impacted by the judgments not only in the production of their national policies, but also as members of the Council.

17.3 Conclusion

In this chapter, we explored the policy impact of the judgments of the Court of Justice. We aimed at conceptualising this impact drawing on theories of integration and systematising the many examples in the literature. In so doing, we proposed to disentangle the study of policy impact from the more frequent, and broader, study of the political impact of the Court. We posit that a clearer, more exhaustive view can be gained from keeping the focus on the way in which the rulings of the CJEU shape public policy at the EU and national levels. To do so, we argued that a new methodological approach is needed, and the first step is to differentiate the impact of the Court’s rulings horizontally (on EU public policy) and vertically (on national public policy). We examined the literature on policy impact along these two lines.

Our review highlighted that research in this area is rich. Research on the horizontal level is more prominent, and a lot of the discussion has centred on conceptual divergences and a strong focus on constitutionalisation. This literature is also not definitive as per the extent of the impact of the Court’s rulings in the legislative output of the Union. While some authors see the impact of the Court considerably limited by the EU legislator, others go as far as to consider the EU legislator as ‘an agent of the Court’. This disagreement seems largely a disciplinary divide: political scientists see more limitations to the influence of the Court than legal scholars, who focus on the effects and consequences of the over-constitutionalisation of EU law and legal integration.

At the vertical level, most scholars look at individual case studies, focusing on concrete policy areas (most frequently social law) in a given country or set of countries and within a temporal framework. The literature review carried out in this chapter suggested that the impact of the Court on national policies varies greatly across Member States and policy fields. Indeed, even with similar conditions, the degree to which Member States are able to shape their own policies differs greatly, and any study of the influence in national policies needs to take the domestic conditions into account.Footnote ⁹⁵ As a result, we have only limited knowledge on how the Court’s judgments impact policy at the domestic level.

Research in the area might appear patchy, which in our view is due to the varied methodologies, disciplines, and theories on which it relies. Indeed, the studies reviewed are grounded on different traditions, from political science to sociology and economy or law, and are thus interested in different aspects and provide diverse findings. Even if much is gained from delving into this research, what is missing is a connection between the different strings, leaving knowledge in this domain, to a large extent, fragmented. Consequently, we lack a comprehensive understanding of the policy impact of the Court and our knowledge remains unavoidably limited.

We posit that for more exhaustive and generalisable knowledge we need a more comprehensive framework and better empirical strategies. This is where ELS can make a significant contribution, one which could expand our understanding of the Court. We propose to start from what seems the most obvious, peremptory need: a research agenda aimed at building bridges and connections between the different strands of the literature in order to provide a more comprehensive view of the policy impact of the Court. For instance, a new and comprehensive empirical strategy could overcome the seemingly strict division between the dynamic and constrained views on the Court, which would also provide more nuance on the extent and effect of negative integration.

ELS is ideally situated for connecting strands in the literature and developing new strategies for the study of the Court given its strong methodological grounds and its intrinsic interdisciplinary nature. In this sense, the chapter highlights many aspects in which more comprehensive research could greatly improve our understanding of the Court. For instance, we still know little on the implementation of judgments of the Court at the national level, particularly in fields outside of social policy. Similarly, little is known about the administrative application of Court’s judgments, even though ultimately national administrations are in charge of EU law on the ground. We believe that a growing awareness on the need of transparency will provide ample opportunities for gathering new data, hence allowing access to the material needed for developing a solid empirical strategy for the study of impact.

Notwithstanding the promise of ELS for the study of policy impact of the Court, this chapter has not concealed the myriad difficulties in measuring that impact. These difficulties are logical for several reasons: First, compared to legislation, judgments have a certain degree of ambiguity, which necessarily affects their influence.Footnote ⁹⁶ Second, and perhaps more importantly, measuring the impact or influence of the Court’s judgments is complex, insofar as it involves a multiplicity of factors, not always apparent, and access to data, which is oftentimes not public. Thus, even if ELS open new and promising ways for studying impact by the Court, any empirical legal scholar should take the difficulties and limitations of this approach into consideration to be successful. Some of these difficulties are intrinsic to the project and common to any interdisciplinary research: how to define policy impact? How to measure it? Other problems are of a more pragmatic nature. Perhaps most pressingly, the poor availability of data hampers large quantitative studies data: decisions of national courts are not always published and administrative decisions are rarely public. Establishing what is realistic and implementable will simultaneously show the potential and the limitations of ELS. However, we have recent examples of how limitations in data access might be overcome by interdisciplinary approaches and coordinated work between different disciplines.Footnote ⁹⁷ We suggest this as a new inroad for improving the study of the policy impact of the Court.

18.1 Introduction

Systematic Content Analysis (SCA) of legal text offers a methodological and replicable tool to analyse a large body of legal text. It typically consists of three steps: (1) selecting cases based on a clearly defined selection criteria; (2) coding cases according to a pre-determined coding protocol; and (3) analysing the coded cases, in particular by using statistics. While most legal (doctrinal)Footnote ¹ and social science researchFootnote ² involves a careful reading of selected written materials, SCA stands out as a research methodology that provides an objective, systematic, and generalisable manner to analyse large bodies of legal text.Footnote ³ The objectivity requirement necessitates that each research step adheres to explicitly formulated processes. These rules aim to minimise, if not entirely eliminate, subjective judgments and biases of the researcher. Similarly, the systematic requirement dictates that selection criteria and coding of the text follow consistently applied rules. The generality requirement indicates that findings should hold theoretical significance rather than merely offering descriptive information about the content. The research design must correspond to a theory and be interpreted in that context. Embodying fundamental principles of social science inquiry, these three requirements aim to apply scientific methods to analyse a body of judgments, administrative decision-making, legislation, and other forms of legal text.Footnote ⁴

As explored in detail elsewhere,Footnote ⁵ content analysis emerged as a modern research technique amidst the financial crisis of the 1930s in the US. Initially utilised by sociologists, it aimed to examine how media reflected and influenced public opinion,Footnote ⁶ as well as to delineate trends in communication content.Footnote ⁷ Its development accelerated during the Second World War, to secure gathering political and military intelligence by inspecting radio and press communications to predict Nazi propaganda strategies.Footnote ⁸ Those techniques spread to other areas of social science and humanities after the war, and began to be conceptualised and to converge across disciplines.

In 1952, Berelson’s Content Analysis in Communication Research marked a milestone as the first comprehensive textbook on content analysis.Footnote ⁹ Early proponents positioned content analysis primarily as a quantitative method, emphasising the exploration of frequencies derived through counting.Footnote ¹⁰ Moreover, these studies predominantly focused on manifest content – that which is readily observable on the surface.Footnote ¹¹ Over time, content analysis evolved beyond its quantitative roots. It was also employed to guide more qualitative approaches, and to explore the latent content of the underlying meaning of the text.Footnote ¹²

Since the 1980s, SCA has experienced rapid growth both in usage and range of disciplines, largely driven by technological advancements. As ‘people cannot count, at least not very high’,Footnote ¹³ the rise of computer-aided statistical and text analysis software, the digitalisation and accessibility of online resources, and the emergence of automated content analysis have all contributed to the method’s expansion.Footnote ¹⁴ Technological development advanced the size and scope of digitalised data (potential databases) and enabled faster and more robust analysis of vast amounts of information.Footnote ¹⁵

Despite its widespread adoption in the social sciences, content analysis was not readily applied to the study of law for many years. This reluctance may have stemmed from various factors. Some blame the prevailing belief among judges, policy-makers, and legal scholars that law is an objective and apolitical concept, a timeless and sacrosanct ideal unaffected by those responsible for its administration.Footnote ¹⁶ According to this notion, the study of the law should focus on the black-letter law and leading cases and legal precedence, emphasising doctrinal analysis rather than the application of law in practice. Others proposed more practical obstacles hindering the systematic study of law, such as law students’ legal educationFootnote ¹⁷ or that the access to judicial and other decision-making data, which is crucial for content analysis, was often (and still sometimes is) tightly controlled by courts and other institutions and is made available only to select entities and researchers.Footnote ¹⁸ This limited access poses significant challenges to conducting comprehensive analyses of legal texts.

Empirical legal research in general, and the embrace of content analysis as a useful methodology to study law in particular, were deeply influenced by the American Legal Realism movement. Notable figures such as Judge Holmes and his successors advocated for law to embrace scientific methods to uncover the underlying political, economic, and societal conflicts that shape legal rules. They argued that hypotheses and assumptions that lay on the basis of law should be tested against empirical evidence, and case law should be subject to scientific scrutiny and analysis.Footnote ¹⁹ While many of the Legal Realists advocated for the application of scientific methods in jurisprudence, empirical legal analysis did not become a focal point of their engagement. Unlike other interdisciplinary approaches stemming from Legal Realism, moreover, SCA did not evolve into an independent school of thought championing specific intellectual traditions or normative policy-relevant concerns.Footnote ²⁰

SCA of legal text was first described as a unique legal methodology as late as 2008 in a seminal article by Hall and Wright.Footnote ²¹ Their contribution not only identified and conceptualised the method but also surveyed its application in analysing legal decisions by courts or administrative adjudicative bodies, outlining best practices. As of the cut-off-date of June 2006, Hall and Wright had reported 134 projects utilising SCA, indicating a notable increase in its usage over time. Prior to 1990, less than one project per year employed SCA, but this figure rose to an average of 5.7 projects per year in the 1990s, and eight projects in the early 2000s.Footnote ²² Although not explicitly stated in their article, most of these contributions focused on US law and were published in American journals, highlighting the prominence of SCA within the context of US legal scholarship.Footnote ²³ In the time that has passed since Hall and Wright conducted their study, the SCA of legal analysis has undergone significant advancements, largely due to the emergence of judicial analytics tools. These tools leverage advanced technological techniques, including machine learning, network analysis, and natural language processing, to code and analyse patterns in legal decision-making processes.Footnote ²⁴

By contrast, the use of SCA for the study of EU law is still underdeveloped.Footnote ²⁵ This chapter points to the limited popularity of this methodology in Europe, as well as to its vast potential. It focuses on the field of EU internal market law and governance. The remainder of this chapter is organised in three sections. Section 18.2 surveys the use of SCA in legal scholarship on the internal market (1957–2023). This section builds on and corresponds to the 2008 survey undertaken by Hall and Wright on the application of SCA in legal scholarship. It uses some of the same variables for comparative purposes, while incorporating additional variables relating specifically to the EU internal market. This section does not attempt to provide a comprehensive account of the use of SCA in this area. That would be an almost impossible exercise, as much of the scholarship employing some form of the method does not identify it by name.Footnote ²⁶ The survey, therefore, seeks to shed light on the growing adoption of SCA, identify the areas of research, the type of research questions and data coded, and the academic actors and publishers involved.

Next, Section 18.3 submits that despite the scarcity of scholarship in Europe, SCA is a natural choice for studying EU internal market law and governance. More specifically, it highlights three distinctive features of EU law: (1) its inherent ambiguity, uncertainty, and vagueness, driven by the generality of the Treaties provisions and secondary laws, which often provide frameworks rather than precise norms and the multicultural nature of European legislators; (2) the intricate multi-level governance system, where rule-making and application are shared across regional, national, and supranational levels; and (3) limited publications and weak open judicial data practices concerning significant body of EU legal text and its application, both at the EU and Member States level.

Finally, the chapter concludes by suggesting that the European legal community has a significant opportunity to advance the study of EU internal market law by using SCA to complement doctrinal studies of law. Openly acknowledging the use of SCA and adhering to its established best practices during the research design, writing, and review could foster an SCA culture tailored to the European context.

18.2 Survey of SCA Internal Market Scholarship

18.2.2 Findings

Figure 18.1 summarises the development of the use of SCA scholarship for the study of EU internal market law and governance, recording the number of SCA publications per year included in the database. It demonstrates the delay in adopting SCA in this area, particularly in comparison with the US experience.Footnote ³⁵ The SCA scholarship on the internal market has only emerged during the 2010s and gained some momentum in the past decade.

Figure 18.1

SCA scholarship per year.

The limited adoption of SCA in Europe is particularly notable when keeping in mind that Hall and Wright’s survey was limited to the study of legal decisions by courts or administrative adjudicative bodies. The studies included in this database and reported in Figure 18.1, by comparison, refer to applying the SCA technique to a broader variety of legal sources.Footnote ³⁶ As demonstrated in Figure 18.2, in addition to judgments of EU (11), national courts (6), decisions of the EU Commission (7), other EU (2), or national (6) and administrative adjudicative bodies, the SCA studies included in this database explore the body of EU legislation (8), parliamentary debates and resolutions (5), and policy documents and consultations (9). Keeping this in mind, therefore, Figure 18.2 points to the limited study of judicial behaviour and politics in the EU using SCA, especially in comparison to the US.Footnote ³⁷ As elaborated in Section 18.3 below, this might be tied to a host of technical barriers, as well as a more principled opposition to this line of research in various European legal systems and traditions.

Figure 18.2

Type of coded legal text in the database.

Figure 18.2 Long description

The values are 3 for A G opinions, 7 for the commission’s decisions, 11 for E U courts judgments, 8 for E U law, 5 for E U parliamentary debates and resolutions including committees, 9 for E U policy documents and consultations, 2 for E U’s independent authorities decisions and opinions, 6 for national authorities’ decisions and opinions, 6 for national courts judgements, 1 for national laws, and 3 for other. Note, all values are approximated.

The slower adoption of SCA in the field of EU internal market law and governance is also reflected by the number of cases coded in each study, reported in Figure 18.3.Footnote ³⁸ According to Hall and Wright’s database, between 2000 to June 2006, the median number of cases coded per project amounted to 399 cases (total of 52 projects), with one project coding as many as 22,000 judgments. The median number of coded cases concerning EU internal market law and governance, by comparison, is merely 61. As illustrated by Figure 18.3, 42 per cent of the European studies involved the coding of 100 or fewer cases (26 per cent of less than fifty cases), 10 per cent of between 100–200 cases, and only 18 per cent 500 cases or higher.

Figure 18.3

Number of coded cases per study.

The limited number of coded cases might be partially explained by the very limited use of automated coding for SCA projects in Europe.Footnote ³⁹ Out of the 38 papers in the database, only two involved a mix of automated and manual coding.Footnote ⁴⁰ Hence, and despite the potential technological advancements since the conclusion of Hall and Wright’s study in 2006, not only the number – but also the scale – of SCA studies concerning EU internal market law and governance has remained modest. Moreover, while Hall and Wright reported a consistently increasing median number of cases coded from the 1990s, there is no trend of an increase in the number of cases recorded in the European studies since the early 2010s.

One may question the added value which SCA can bring to traditional doctrinal legal research when the former is based on a relatively modest number of cases. Like SCA, (high-quality) doctrinal research also involves carefully reading and analysing a body of jurisprudence or other legal texts and making inferences about emerging trends and meanings. Hence, it could be argued that smaller-scale SCA studies do not offer great benefits beyond perhaps a more organised way to read cases. The potential advantages of SCA in such scenarios, nevertheless, may be context-specific. They are shaped, in particular, based on the research question, sampling criteria, and variables coded. For example, an SCA study of a small sample of cases from the total population, which is not grounded in probability or non-probability sampling criteria, is likely to have both limited validity and added value compared to doctrinal research. By contrast, a comprehensive account of all judgments rendered during a particular timeframe across a variety of variables might help expose trends and patterns that are not visible when relying on traditional legal methodologies, even if the number of coded cases is relatively modest.

Next, the survey of SCA-based EU internal market law and governance scholarship reveals that commercial aspects of EU law have received comparably greater attention. As summarised in Figure 18.4, SCA was most often used to explore matters related to EU competition law, Intellectual Property Rights (IPRs), taxation and fiscal policy, and sector regulation. Social aspects of EU law and the general operation of the EU justice system have remained largely untouched by this methodology.Footnote ⁴¹

Figure 18.4

Areas of EU law.

Figure 18.4 Long description

The values are 1 for Covid recovery, E U courts operation, health, E U membership accession, looping, sustainability, refugee crisis, technology and innovation, and European social charter. 2 for criminal law, theory of empirical analysis, and E U aims and values. 4 for sector regulation. 5 for taxation and fiscal policy. 7 for I P Rs and competition law.

These findings can partially be explained by some authors making multiple SCA contributions to their fields (e.g., in competition lawFootnote ⁴² and IPRsFootnote ⁴³), rather than differences related to the background or inherent interdisciplinarity of scholars involved in certain areas of EU law. The uneven acceptance of SCA scholarship among certain areas of EU law also does not appear to be linked to publication venues. The studies included in the database were evenly spread among journals (for articles) and publishers (for books and book chapters), where all of the recorded journals published only a single SCA paper.

The survey points to an interesting observation about the type of scholars involved in the SCA studies of EU internal market law and governance. As Figure 18.5 suggests, half of the recorded SCA studies were conducted by non-legal scholars.Footnote ⁴⁴ In other words, much of the insights about EU internal market law and governance using this technique are made by political scientists, economists, and business administration, as well as other social science researchers. This finding is in line with previously reported trends about empirical research in Europe in general, whereby a growing share of the cutting-edge research, which used to be the preserve of legal scholars, is now conducted by researchers outside the legal discipline.Footnote ⁴⁵

Figure 18.5:

Authors’ affiliation.*

*Note: The author’s affiliation was coded according to the school, department, or factually listed in the first comment of each publication. Hence, law, economics, and political science, for example, are distinguished from social science.

This observation may indicate another missed opportunity within the realm of the SCA of EU internal market law and governance. Given their doctrinal training, background, and knowledge, legal scholars possess unique advantages in identifying legally significant research questions and empirically confirming or challenging some of the conventional wisdom and assumptions in their areas of expertise. Moreover, the integration of doctrinal legal insights can significantly enhance the quality and validity of SCA-based research. Achieving conclusions of substantial legal and practical significance often requires a nuanced legal understanding of the relevant legal rules and enforcement settings to effectively select cases, define variables, and analyse data. Thus, ‘traditional’ doctrinal legal analysis and SCA should not be seen as opposing methodologies but rather as complementary approaches. While social scientists and teams of interdisciplinary researchers undoubtedly offer important insights to the study of EU internal market law and governance, there remains considerable untapped potential for legal scholars to apply SCA effectively in their research.

Finally, a closer examination of the body of SCA research focusing on internal market law and governance reveals a relatively limited range of research questions being explored through this methodology. As commentators have previously noted, SCA of legal text can address various research questions, which can be roughly categorised into three groups: (1) descriptive research questions, which involve ‘gathering, organising, tabulating, and depicting data’ (research projects in this category typically aim to uncover trends in the case law, the success rates of certain arguments, and the language used in the decision-making process); (2) predictive research questions, which seek to forecast future courses of action based on past experiences and trends identified through SCA; and (3) perspective research questions, which aim to recommend evidence-based law and policy reforms.Footnote ⁴⁶ Despite this potentially wide scope of application, EU scholarship has predominantly focused on the first category of descriptive research questions. Specifically, while many American SCA studies have explored judges’ personal characteristics (e.g., gender, age, education, and politics) as predictors or explanatory factors in decision-making, such scholarship has not gained traction in the EU context.Footnote ⁴⁷

18.3 The Potential for SCA-based Research of EU Internal Market Law and Governance

SCA of EU internal market law and governance is still in its early days. As demonstrated in the previous section, there are few studies, typically coding a limited number of cases, concerning limited areas of EU law, and focusing on a narrow type of research questions. Nevertheless, this section suggests that the exploration of EU internal market law and governance may benefit from greater use of this methodology, particularly given (1) the nature of EU law, (2) its multi-governance character, and (3) by creating new databases and tools to explore the EU internal market in face of the limited publications and weak open judicial data practices concerning a significant body of EU legal text. This section discusses each of those features of EU law in turn, referring to examples of studies using SCA to explore those gaps based on the database presented in the previous section.

First, SCA has the potential to serve as a powerful tool for deciphering the ambiguous, uncertain, and vague provisions of EU law and governance in the internal market and beyond. Unlike many national legislations, EU laws often only offer general legal frameworks rather than defined norms.Footnote ⁴⁸ The EU Treaties, secondary, and other forms of regulation typically provide no or very limited explanations of the underlying values of the EU legal order and the aims of the specific legal provisions.Footnote ⁴⁹ The ambiguity surrounding EU law is ascribed to a combination of factors, including the need to reach a political compromise between representatives of the Member States, negotiations and amendments over legal text conducted mostly by non-native speakers and later translated into the various Member States’ languages, and the interpretation and application of those rules by judges and lawyers coming from vastly different legal traditions.Footnote ⁵⁰ The scope, meaning, and boundaries of EU law and governance, therefore, are affected by the multicultural nature of European legislators and multilingualism.Footnote ⁵¹ Further uncertainties pertain to the interaction between EU and national law and how far the EU competencies reach.Footnote ⁵² The division of competencies between the Member States and the EU is politically charged and contested, and EU law offers only incomplete, open-textured, and imprecise rules.Footnote ⁵³ For example, while the EU Treaties introduce the principles of conferral, subsidiarity, proportionality, and sincere cooperation to guide this division, they do not contain a clear resolution tool in case the application of those principles results in conflicts.Footnote ⁵⁴

Against this background, SCA has already assisted in identifying how EU internal market law and governance is being interpreted and applied in practice, whether it follows a coherent and consistent interpretation,Footnote ⁵⁵ how the interpretation of national actors differs across subject areas or borders,Footnote ⁵⁶ what laws are being enforced and which are not, and how the governance mechanism operates.Footnote ⁵⁷ The method was used to investigate how the division of competencies between the EU and the Member States operates across different fields of law,Footnote ⁵⁸ and how the bodies involved in applying EU internal market and governance make use of their discretion. For example, SCA research exposed if and how the decision-making process was biased against certain national citizens and companies.Footnote ⁵⁹ From a policy perspective, SCA may also be used to limit the scope of discretion in the administration of EU law. As Bugarič notes, ‘[t]he most effective means of limiting the scope of discretion in public administration lies not in more detailed legislation but in … bringing public administration operations into the public domain’.Footnote ⁶⁰

SCA has the potential to complement doctrinal studies in the quest to clarify the meaning of various provisions of EU law and governance precisely because it shifts the focus away from leading cases, which are often the subject of doctrinal research, to the day-to-day interpretation and application of EU norms by various EU and national actors. This shift allows for a more comprehensive understanding of how EU laws are practically implemented and interpreted across different contexts, offering valuable insights that doctrinal analysis alone might miss.

Second, SCA is particularly well-suited to explore matters related to multi-level governance within the EU. European integration is an ongoing policy-creation process. Its authority and policy-making are influenced and shared across regional, national, and super-national levels.Footnote ⁶¹ Various national political institutions and actors play roles in both rule-making and rule-implementation, and these institutions and actors often exhibit considerable procedural and institutional diversity.Footnote ⁶² European and transnational integration necessitates cooperation among distinct governmental levels (local, subnational/regional, national, European, and transnational) and between them and non-public actors.Footnote ⁶³ As a result, EU law gains effect not only by the implementation of the 27 national legal systems (which can diverge in particular when EU law sets only minimum harmonisationFootnote ⁶⁴), but also by its application by national courts, national administrative agencies, national traders, consumer, and environmental organisations, service providers, employers and employees, interest groups,Footnote ⁶⁵ and European citizens. Despite the critical importance of the enforcement of EU law, however, there is a limited understanding of the extent to which EU law is applied and by whom.Footnote ⁶⁶

SCA offers a tool to explore how different regional, national, super-national, and EU actors comprehend and implement the provisions of EU law and governance.Footnote ⁶⁷ It was used to investigate how national systems implemented EU law and to uncover divergences and inconsistencies in interpretation that may not be readily apparent using doctrinal or case-study-specific methodologies,Footnote ⁶⁸ and how the operation of the EU- and national-level legislative and enforcement systems co-exist.Footnote ⁶⁹ Moreover, SCA was used to trace the integration agenda of EU legislation,Footnote ⁷⁰ as well as resistance to EU integration by national actors.Footnote ⁷¹

SCA studies aiming to explore the features of EU multi-level governance and decentralised enforcement face unique challenges regarding research design and the research teams’ composition. Language barriers, diverse European legal traditions, and the varying contexts in which EU law and governance are being applied at the national level affect the selection of cases and bodies subject to exploration, as well as the definition of variables. Consistent with the EU principle of procedural and institutional autonomy, this national diversity often necessitates an EU-wide team of project leaders and large teams of coders, each an expert in their respective legal systems. Consequently, coordinating such extensive and specialised teams is often essential for effectively addressing the complexities inherent in SCA studies within the EU context.Footnote ⁷²

Finally, SCA projects can play a critical role in addressing the scarcity of empirical data on the operation and application of EU law and governance by generating new databases and tools. As elaborated below, the systematic gathering and analysis of EU legal texts face various challenges, resulting from limited publications and weak open judicial data practices. Overcoming these obstacles through SCA can significantly enhance the empirical foundation of EU legal studies, providing deeper insights into how EU law is implemented and enforced across different jurisdictions.

A significant body of EU legal text is not published or publicly available. This may come as a surprise, considering the EU’s apparent commitment to a relatively high degree of openness and transparency. Article 1 TEU, for example, emphasises that in the process of creating an ever closer union among the people of Europe, ‘decisions are taken as openly as possible to the citizen’, and Article 15(1) TFEU adds that ‘to promote good governance and ensure the participation of civil society, the Union’s institutions, bodies, offices and agencies shall conduct their work as openly as possible’. Regulation 1049/2001 further expands on those general principles, establishing the right to access EU documents, subject to some restrictions.Footnote ⁷³ Similarly, the judgments of the EU courts are mostly published and reasoned,Footnote ⁷⁴ and the EU courts acknowledged that publishing the Commission’s decision contributes to ensuring the observance of EU law, serves the public interest of knowing as fully as possible the reason behind the Commission’s action, increases compliance, and invites those who have suffered damages to launch private claims or seek judicial review.Footnote ⁷⁵

Nevertheless, access to a significant body of EU legal text and the decision-making process remains limited.Footnote ⁷⁶ For instance, the EU Courts held that the obligation of Article 15 TFEU applies only to the exercise of the ‘administrative tasks’ of their operation, and not to their judicial powers.Footnote ⁷⁷ Similarly, the right to access the court file and publicity of trial is garnered by the EU Courts to the parties to the judicial proceedings. The general public enjoys only a lower and undefined standard of openness.Footnote ⁷⁸

Moreover, the EU’s obligations regarding openness and transparency do not uniformly extend to the national actors responsible for applying EU law.Footnote ⁷⁹ Rather, based on the EU principle of procedural and institutional autonomy, each Member State adopts its own national rules and practices on publications, even with respect to national legal text and decisions applying or implementing EU law. Significant divergences persist among the Member States in this regard. While some national rules require the publication of all or some judgments (often those of higher-instance bodies), others have no publication obligations or leave the decision to the relevant court or presiding judge.Footnote ⁸⁰ In some cases, formal national publication rules are not respected in practice, and some national courts place logistical hurdles that significantly limit access to judgments, even for research purposes.Footnote ⁸¹ Furthermore, even when national judgments are published, in some Member States they are heavily redacted or anonymised, making it difficult to follow the trail of process and understanding its outcomes.Footnote ⁸²

Another distinctive challenge in systematically studying EU legal texts arises from the manner in which judicial and administrative decisions are made. Unlike the US, where court judgments often include dissenting opinions, decisions by EU Courts and administrative bodies are typically rendered collegially, without the expression of divergent views.Footnote ⁸³ The absence of dissenting opinions is often attributed to the EU’s policy objective of garnering broader support for its legal framework. It is argued that presenting minority views could potentially undermine acceptance of EU Court rulings within the legal-political systems and among the general public of the Member States, particularly on contentious or sensitive issues.Footnote ⁸⁴ This lack of diverging opinions also characterises many EU Member States, particularly those with civil law traditions.Footnote ⁸⁵ This decision-making format complicates the profiling of judges in relevant chambers and hinders the observation of trends in their rulings. Consequently, as previously noted, this type of research, which is widespread across the Atlantic, is constrained in Europe.

Even when published and accessible, a significant portion of the body of EU law is not structured or integrated into databases that facilitate automatic downloading and processing, essential for conducting large-scale SCA projects.Footnote ⁸⁶ Despite the growing momentum of Open Data initiatives over recent decades, the advancement in opening up judicial decision-making and creating accessible datasets has been notably slow in many jurisdictions across Europe.Footnote ⁸⁷

The scarcity of accessible and readily processable data concerning the implementation of EU internal market law and governance is exacerbated by the limited adoption of SCA within both academic circles and policy evaluations. In addition to the sparse SCA academic literature highlighted in the previous section, impact assessments and policy evaluations of EU law and governance often concentrate on substantive EU rules. The European Commission, in particular, allocates little resources for gaining an understanding of the diverse institutional settings of the actors applying EU law and the interplay between national authorities and courts.Footnote ⁸⁸ This situation perpetuates a vicious cycle: As European scholarship and policy-making predominantly rely on doctrinal legal research focused on black-letter legal rules rather than their practical application and enforcement, fewer information and datasets conducive to advancing SCA and empirical legal research are generated.

Some of the challenges associated with the limited access to legal text and their analysis, particularly those necessary for understanding judicial and administrative decision-making processes, are not unique to EU law.Footnote ⁸⁹ Yet, while studies in the US and Canada, for instance, have made strides in overcoming those hurdles, ‘[t]he huge black box called enforcement has never really been opened during the long history of EU law’.Footnote ⁹⁰ Generating a robust body of SCA studies holds the promise of unravelling the ‘black box’ of EU enforcement. These studies are indispensable not only for addressing fundamental legal research questions by using new empirical–systematic methodologies, but also for creating new databases and toolsFootnote ⁹¹ that bolster further SCA research and skills among legal researchers of the EU internal market law and governance.

18.4 Final Remarks

SCA has begun to gain footing in Europe over the past decade. While this methodology is regularly used for studying law in the US, especially for exploring various facets of judicial decision-making, the survey of EU scholarship presented in this contribution illustrates that SCA studies of EU internal market law and governance remain limited. They are not only scarce in number but also constrained in terms of the areas of EU law investigated, the types of research questions pursued, the number of cases coded in each study, the use of automated coding, and the involvement of legal scholars. This contribution aimed to underscore the untapped potential of SCA in examining EU internal market law and governance, addressing both the challenges and common hurdles encountered in such studies.

Following the initial experience with SCA, the European legal community stands at a crucial juncture to advance this methodology. The development of a robust body of SCA research on EU internal market law and governance could benefit from increased attention to methodological rigour and the development of common best practices tailored to the European legal traditions and the EU’s structural characteristics throughout the research process – from initial design to study write-up and review by editors and peers. This is particularly true not only for the studies that explicitly recognise the use of SCA (which were the focus of the survey presented in this contribution), but also to many other projects that implicitly employ similar analytical techniques. Similarly to what was highlighted by Hall and Wright in their critique of earlier American-based SCA studies, although many studies have been published in leading law journals and sparked much academic debate, ‘[i]n project after project, legal researchers reinvent this methodological wheel on their own’, and have to learn ‘how to do content analysis on the fly, feeling at first as if we each discovered something new until we learned that we had each done the same thing independently’.Footnote ⁹²

The European legal community stands to gain significantly from openly embracing SCA to engage in a more methodologically sound study of EU law. Recognising the use of SCA and adhering to the best practices developed by legal researchers and social scientists over the past 70 years has the potential of leading to more robust sample selection, variable definition, coding interpretation, and validity checks, as well as to identifying and discussing the limitations of studies. This, in turn, will not only improve methodological quality, but could cultivate a SCA culture within the European legal community. It could foster the development of skills, formation of peer groups, shared conventions tailored to the distinctive aspects of EU law and governance, and generate new databases supporting future empirical research and policy evaluations. By standardising and refining the application of SCA, legal scholars can effectively contribute to more insightful analyses and interpretations of EU legal frameworks.

Reflecting its political salience, European migration law (EML) has grown significantly as a research field. So far, however, the field has been dominated by doctrinal approaches and methodological debates centred on discussions about more teleological versus more statist orientations.Footnote ¹ At its outset, however, EML presents certain methodological challenges for traditional legal approaches. EML is a composite field spanning multiple different regimes, most notably international refugee law, international and European human rights law, EU law, but also ancillary bodies of law such as the law of the sea, aviation law, and transnational labour law. Moreover, EU law in this area is dependent on legislative harmonisation between Member States over immigration. The latter has proven difficult to secure with respect to EU external borders and policies for third-country nationals, given that these are areas where states have traditionally sought to preserve sovereign discretion.Footnote ² EML has thus moved from earlier hopes for EU supranationalism to face ‘reluctant harmonisation’,Footnote ³ with a range of issues simultaneously governed by EU, national, and international law. Whether due to attempts by Member States to recoup national sovereignty through national measures, or the relative ineffectiveness of regional rules in some areas, EML could almost be characterised as ‘organised hypocrisy’, with long-standing norms or competence decisions that are frequently undermined, as an enduring attribute of the EU legal order.Footnote ⁴

Whilst extending from a regional framework of EU law, the lack of effective harmonisation and interaction with other international and regional legal regimes have created spill-over effects, whereby EML come to be characterised by indeterminacy, which undermines its systemic qualities, and makes it difficult to disentangle.Footnote ⁵ On the one hand, EML is horizontally fragmented as the legal protection of migrants in the European space now depends, for a large part, not on EU law but on the rights afforded by the European Court of Human Rights (ECtHR) and national courts.Footnote ⁶ On the other hand, EML is vertically fragmented, as despite decades of attempts to encourage trans-national judicial dialogue, judges have difficulty aligning on similar questions of EML.Footnote ⁷ It can thus be difficult to adequately capture the inter-operations and machinations of EML by focusing on one regime alone, particularly as EU courts promote a discourse that champions the autonomy of the legal order.

Whilst EML is hardly the only area of European law subject to this type of multi-level implementation problem, the issue is particularly acute considering the politicisation of migration and asylum law across virtually all European countries. Asylum law is perhaps the most salient example in this regard. The EU legal framework presumes judicial harmonisation between national judges and decision-makers of EU Member States, but this has never materialised in practice.Footnote ⁸ This reflects not only the residual sovereignty left to Member States in terms of how to design and populate national asylum institutions, but also the fact that legal decision-making in this area hinges on somewhat ambiguous level provisions and incomplete evidence.Footnote ⁹ Research has thus shown that legal decision-making on asylum is subject to inter-subjective assessments and bias, which enable ‘local yardsticks’ to determine who is entitled to the protections of EU law.Footnote ¹⁰ This has resulted in a high level of variability between Member States which contributes to gross outcome variations, such that ‘a person can have 90 per cent chance being accepted as a refugee in one EU country, whilst her chances are virtually nil next door’.Footnote ¹¹

EML scholarship has long recognised and lamented this situation, yet dominant doctrinal methods rarely, if ever, engage these issues head on. On the one hand, approaches that focus on EML as deriving from the broader operational structure of EU lawFootnote ¹² or the EU legislative framework tend to neglect or downplay normative developments that take place under adjacent regimes of law.Footnote ¹³ On the other hand, research that adopts an inter-regime focus easily loses sight of the diversity or incongruence of approaches at the national level.Footnote ¹⁴ Whilst each of these approaches has merits, they assume that that legal decision-making will take place in accordance with the rule of law, but legal methods themselves cannot address the behaviour of the norm user, particularly when decision-making inconsistency is endemic to the law.

At a more general level, EU legal method has been criticised for its myopic focus because it derives its standards as ‘mimesis of judicial discourse’, and thus tends to reproduce certain narratives.Footnote ¹⁵ This is, in a sense, a classic criticism of doctrinal legal method as a system of research set with certain epistemological limitations as it focuses principally on finding what is the valid law.Footnote ¹⁶ As de Witte notes, ‘writing about EU law means exposing which norms of EU law exist on a given subject, how they interact, and how they are put into practice by the legal system.’Footnote ¹⁷ To properly heed that call in an area such as EML may ultimately require one to move beyond doctrinal research. To this end, a recent ‘empirical turn’ is slowly taking foothold and opens up new ways of researching EML law, creating different opportunities and epistemological commitments to the mainstream of EU law scholarship.Footnote ¹⁸ Scholars have employed socio-legal,Footnote ¹⁹ ethnographic,Footnote ²⁰ and political science methodsFootnote ²¹ to open up space for thinking on what can be known about EML, how it is applied, and individuals that are subjected to it. Yet, for the most part, this literature still adopts an external approach for analysing EML, which does not offer means for disentangling the broader normative framework, nor its inherent legal problems.

In this contribution, we thus seek to outline an empirical research agenda for EML that puts the law first in its analysis but enhances analytical capacity through the power of computation.Footnote ²² To do so, we build from the existing computational turn in legal research, which in an EU law context has been employed to analyse large corpora of EU case law for well over a decade.Footnote ²³ Pioneering such an approach in EML, however, requires further methodological considerations.Footnote ²⁴ Reflecting the particularities of the field, it needs to be attuned to both multi-sited case law at the international level, and the vertical production of case law across international, regional, and national levels. Moreover, reflecting the existing state of the art, we seek to develop a computational legal research agenda that can carry novel insights for a largely doctrinally oriented audience. That is, our ambition is both normative and empirical and has policy implications by helping to enhance protections for individual migrants and asylum seekers and foster more equitable asylum systems.Footnote ²⁵ The stakes are certainly high, and we remain wary to promote computational legal methods as a panacea, and thus in Section 19.4 we articulate some of the limits of computational legal research when it is projected as a stand-alone method absent of cross-methodological and interdisciplinary exchange.

The chapter is structured as follows. Section 19.1 introduces the computational turn in the legal discipline before outlining its developing applications in EU law studies. Section 19.2 investigates the use of case citation networks to analyse the lack of judicial dialogue between national judges on questions of migration law. Section 19.3 explores the potential for forms of machine learning to open the black box of asylum law judging. Section 19.4 concludes by noting some of the benefits and limitations of our approach and canvasses some further possible opportunities in going forward.

19.1 Computational Methods for European Migration Law

Computational analysis when applied to law treats law as data. Computational law forms part of the field of text analytics, ‘the discovery of knowledge that can be found in text archives’, or more specifically, legal analytics, the process of ‘deriving of substantively meaningful insight from some sort of legal data’.Footnote ²⁶ The predominant mode of computational analysis of legal materials now focuses on ‘translation of legal texts into machine-readable data capable of quantitative analysis to determine the content, causes, or consequences of legal decisions’.Footnote ²⁷ Legal scholars have been increasingly drawn to new opportunities arising from computational methods by enabling tasks that would simply not be possible without the processing power of computation, and using machine learning to develop the scale and scope of legal research.

Computational law aims to identify recursive patterns in legal materials by coding legal texts as data and analysing these texts with algorithms. In principle, no forms of ‘text’ are off limits for this line of research.Footnote ²⁸ The majority of this work has thus far taken the form of coding written legal texts, such as judicial decisions, statutes, and international treaties, into a machine readable form and then analysing the texts on the basis of texts which are contained in the documents.Footnote ²⁹ Computational law thereby allows researchers to develop efficiencies, synergies, and more in-depth insights than traditional methods, with a focus that can be extended from the practice of law and its concern for the normative application of legal principles, to the study of the nature and operation of the legal system itself.Footnote ³⁰ In this way, a computational approach can offer fresh insights on traditional legal questions but also questions which have traditionally been approached through philosophical or social scientific forms of analysis.

The computational turn as emerging in the legal discipline draws from rich traditions of empirical method in political science, the digital humanities, and empirical legal research.Footnote ³¹ In reflection of a wider turn to interdisciplinary methods, both EU law and international law scholarship is warming up to treating law as data. In one respect, this represents an extension of the ‘empirical turn’ in legal scholarship’,Footnote ³² which has sought to ‘provide scholars with tools to gain new facts, see existing ideas through a different lens, and engage in a more nuanced analysis’ of law and its practices.Footnote ³³ At the same time, it speaks to more pragmatic objectives of helping lawyers to find ‘what works’, as an essential motivation for the turn to more realist agendas.Footnote ³⁴ In this sense, the computational turn in law is ultimately a product of, and responsive to, profound changes in the way we receive and process legal information, as law has itself become ‘a “big data” phenomenon’.Footnote ³⁵ However, computational legal analysis is not necessarily at odds with more traditional and practice-oriented approaches. It can also operate in tandem with doctrinal methods, serving to enhance ‘the validity, reliability, and transparency’ of doctrine, whilst preserving its capacity for conceptual analysis and attention to legal detail.Footnote ³⁶

What computational law adds to existing doctrinal and empirical legal research is then the ability to broaden analysis in a way that permits greater ability to ‘test, validate, and refine existing frameworks’.Footnote ³⁷ It enables researchers to ‘provide empirical evidence to validate hunches and prove legal intuitions correct’,Footnote ³⁸ or to ‘ask new and more ambitious questions about how international law works’.Footnote ³⁹ Computational law further renders legal research scalable and empowers research to study law with unprecedented depth and breadth’,Footnote ⁴⁰ and gain ‘a more stable and comprehensive quantitative basis’ for incisive analysis of a research question.Footnote ⁴¹ Computational law also further promotes efficiency in research, as computers replace assistants for repetitive tasks thus enabling ‘faster, methodical, more efficient leafing through case law’.Footnote ⁴²

These gains are being rapidly realised in EU law scholarship, in part, because EU law documents come in machine readable form and are highly structured, and are thus an ideal testing ground for these technologies. Computational text analysis requires text in digital form, and the large corpora of EU law is freely available online. These documents can be downloaded or ‘web scraped’ using Python and R. With coding, computational methods can thereafter be employed to investigate simple features such as the length of texts, to more advanced analyses of discursive elements that can be identified through instruction or more autonomous forms of machine learning.Footnote ⁴³ The methods develop with such pace that a literature review by time of publication will be out of date, and in particular now with the increasing utility of Large Language Models (LLMs).Footnote ⁴⁴

Existing computationally driven research on EU, however, has overwhelmingly focused on the role and production of the Court of Justice of the European Union (CJEU). Consider, for example, the pioneering analysis of precedent information by Mattias Derlén and Johan Lindholm,Footnote ⁴⁵ Urška Šadl’s use of network analytics to measure how precedents change and evolve,Footnote ⁴⁶ or Artur Dyvre’s explorations of the role of institutional actors in shaping EU law.Footnote ⁴⁷ This body of scholarship has foundationally shown how the use of computational legal methods can challenge conventional wisdoms in EU law, not least, on how EU legal institutions build their legal authority, and how the legal system grows.Footnote ⁴⁸ A new generation is joining this fold by using computational methods to explore fundamental issues of EU constitutional law and in particular, relations arising from the role of national courts.Footnote ⁴⁹ National courts are of course not a new field of research for empirical legal studies, though work in this area often remains similarly siloed as it tends to focus exclusively on legal issues of EU law. There remains unrealised potential to leverage the power of computational methods to amplify research scale and scope, thereby integrating multiple different sites or sources of law at a level that would previously have been thought impossible.Footnote ⁵⁰ Further inroads for exploring the dynamic nature of EU law could open up by using the same methods to analyse the practices of administrative and quasi-judical bodies that apply EU law as the frontline of engagement with national legal orders. Building on recent advances in this regard, we seek to demonstrate the purchase of computational methodologies specifically addressing the traits of EML law and practice.

19.2 Citation Networks and the Lack of Judicial Dialogue of Migration Law Judges

EML is composed of a patchwork of laws arising at regional and national levels, which together comprise a loosely knit normative framework pertaining to a legal subject. It is not a legal system in the traditional sense of the word, nor can it be properly conceived as its own legal order. Rather, it operates as a transnational field of practice that combines norms from a number of different regimes, in particular, with the significant overlap of migration, asylum, and human rights normative bases. It is perhaps better thought of as a ‘legal space’ characterised by ‘topologies’ where multiple legalities co-exist, as they are multiple overlapping norms which may be relevant to any legal problem.Footnote ⁵¹ But it is also a ‘topographical’ space because its terrain is not flat, nor is it necessarily neutral subject to networks and hierarchies that informally comprises varied corpora of EU law, regional human rights, and national immigration law – which in any given case may move to ascendance, to name but a only few core elements.Footnote ⁵²

But this can also make EML difficult to navigate for scholars and practitioners alike. There is no central court for interpreting and implementing EML, simply because there is no court which holds sole jurisdiction over such a complicated legal arrangement. Despite early hopes that the CJEU could become a ‘new refugee court’Footnote ⁵³ following foundational cases like Y and Z,Footnote ⁵⁴ and X, Y, and Z,Footnote ⁵⁵ the court has tended to adopt a minimalist approach to fundamental issues, such as the right to seek asylum enshrined in Article 18 of the Charter of European Union.Footnote ⁵⁶ In addition, on several key juridical issues the court has simply shied away from establishing jurisdiction, such as the conclusion of migration agreements with third countries,Footnote ⁵⁷ or the responsibility of the Union’s own border agency in the context of migration control.Footnote ⁵⁸

The lack of a centralised mechanism for authoritative judicial interpretation over commonly worded legal protections has not meant an absence of international and legal case law production, however. Instead, EML has largely developed through processes of ‘surrogate judicialisation’ that functions like a ‘work around’, whereby, for example, the UN treaty bodies at international levelFootnote ⁵⁹ and the ECtHR at regional levelFootnote ⁶⁰ have taken up a large portion of rights complaints from immigrants in the EU regional area – both from the vantage points of regimes originally not intended to focus on migrant rights. Migration law scholars have tended to celebrate and embrace this development, insofar that is has extended a new terrain of international fora for migrants and refugees, who traditionally have lacked access to redress beyond national avenues.Footnote ⁶¹ Others have warned that the judicialisation of EML through human rights risks becoming a cause célèbre, pointing to how the expansion of human rights case law for migrants coincide with increasing political backlash.Footnote ⁶²

From a more theoretical vantage point, these developments show how EML has become constitutively entangled – to the point where it is practically difficult to isolate its constituent parts because the proliferation of cross-cutting case law connections has significantly reshaped interpretation in the underlying regimes.Footnote ⁶³ The epistemological challenge of cognising a ‘European’ migration law as a such has been recognised by other scholars, insofar as it privileges a certain way of looking at migration law at the expense of other legible paradigms.Footnote ⁶⁴ However, the broader epistemological challenge also translates to a methodological problem. If we tend to isolate only parts of EML through a singular regime optic, we may fail to properly undergird its normative potential. In practical terms, it may also be unduly time consuming for migration lawyers to keep abreast with normative developments across multiple different regimes, and to comprehensively scope the connection points most conducive to a given case.

One way in which computational methods can assist EML is thus by helping create ‘coherence out of chaos’Footnote ⁶⁵ with less selective or other types of bias than traditional doctrinal legal methods. In our previous research, we have used computational methods to map the human rights turn in international migration law using case citation network analysis. Network analysis is a tool for representing connections between ‘nodes’ that capture the relationship between types of information in a network and has been applied to analyse large bodies of case law by tracing connections through citations between legal decisions.Footnote ⁶⁶ Whereas doctrinal analysis typically focuses on ‘leading cases’ and controversial decisions, network analysis enables researchers to reveal less obvious links, and to examine their significance as well as the network’s overall characteristics.

For our analysis we identified the penetration of migration issues in the ECtHR’s jurisprudence by scraping the data for a set of keywords: Refugee; Asylum; Refouler; [^-n]refoulement; Non-refoulement; Nonrefoulement; Immigration; Deport; Migrant. This gave us a set of 3,273 cases (nodes). The cases were then connected through case-to-case citations (edges), but many of these were never cited and are thus peripheral to the network (unless they are very recent cases). The main body of cases that made up the citation network consisted of 1,174 cases. The network had a total of 4,613 edges (connections). We then calculated the most important network features of each node in the network, and through a number of different metrices calculated high network scores. High network scores in this instance are driven by the frequency of court citations; the more the case is cited, the higher the network score. For this purpose, we then used the ‘Betweenness’ and ‘Eigenvector’ methods to detect network centrality as the measure of influence of a node (i.e., a case) in the network. We then used this data to calculate the ECtHR’s most cited decisions in network of migration decisions and represent them graphically.

Some of the cases in Figure 19.1 may be more familiar to EML researchers than others. Saadi v. Italy is likely commonly cited by the ECtHR for its well-known findings on the absolute nature of non-refoulement. Čonka v. Belgium might also be familiar for the proposition that effective remedies on deportation must hold a suspensive effect under the terms of the Convention. M.S.S. v. Belgium is a cardinal case of EML for its examination of the compatibility of transfers under the Dublin Regulation, whilst Salah Sheekh was an important case on internal protection alternatives, whilst Üner v. The Netherlands affirmed importance principles for cases concerning the expulsion of long-term immigrants who had been convicted of criminal offences.

Figure 19.1

Case citation network of the ECtHR’s migration law jurisprudence.

Figure 19.1 Long description

Each node represents a specific legal case, with the case name and involved parties labeled. The size of the nodes varies. The cases involve various countries, including Italy, the United Kingdom, Russia, Belgium, Austria, the Czech Republic, the Netherlands, Switzerland, Latvia, Turkey, Georgia, and Bulgaria. The cases include Saadi versus Italy, El-Masri versus The Former Yugoslav Republic of Macedonia, Othman Abu Qatada versus The United Kingdom, Maslov versus Austria, Muminov versus Russia, D H and Others versus The Czech Republic, Uner versus The Netherlands, Na. versus The United Kingdom, Shamayev and Others versus Georgia and Russia, Al-Skeini and Others versus The United Kingdom, M S S versus Belgium and Greece, Abdolkhani and Karimnia versus Turkey, Nasrulloyev versus Russia, Hirsi Jamaa and Others versus Italy, Nada versus Switzerland, A. and Others versus The United Kingdom, Conka versus Belgium, Saadi versus The United Kingdom, and Stanev versus Bulgaria.

Identifying case law on the basis of citations can serve theoretical and practical aims. In our study, we further showed how this type of research can be used to assess how much weight the court ascribes to a migration-specific case but also to identify case law which might be relevant for a legal argument. By constructing networks for the Court’s Article 1, 3, and 8 case law, we further showed how migration-related decisions are amongst the most cited of the Court’s case law on these articles, thus evincing their role in building wider jurisprudence. Other scholars have analysed cross-citation on migration issues between the ECtHR and CJEU.Footnote ⁶⁷ Further research could explore the interaction of EML with regimes outside the European spaceFootnote ⁶⁸ as well as emerging normative issues associated with climate change.Footnote ⁶⁹ In this way, network analysis can tackle horizontal problems of entanglement by helping to identify the extent of normative spread, coherence, and conflict beyond single regime analysis.

First, network analysis provides empirical evidence of authoritative judicial practice. Networks can make visible the relative importance of precedent by identifying centrality in the branches of the web in a network,Footnote ⁷⁰ or its density across similar legal issue areas.Footnote ⁷¹ Network analysis can also identify arbitrary authority as ‘the rich get richer’ – citation accrues subsequent citations as a highly cited precedent.Footnote ⁷² In this way, the technology can be used to ascertain if citation has increased or decreased and the ‘tipping points’ where precedent becomes ‘good law’ in a network.Footnote ⁷³ Mapping the web of case law thus also provides greater ability to deride outlier interpretations. For instance, do restrictive decisions on extra-territorial jurisdiction, such as the ECtHR’s decision in Bankovic v. Belgium, reflect widespread judicial practice across jurisdictions, or is Al-Skeini v. the UK now the leading precedent? EML scholars and lawyers gain a stable empirical basis from which cases can be argued across jurisdictions and a more encompassing overview of areas of law that are ripe for further doctrinal analysis.

Second, network analysis can be used to map the evolution of legal principles over time and thus help to identify the factors or events that seem to drive the process of judicial cross-citation.Footnote ⁷⁴ It can be used to identify the incremental reach of law over specific factual issue areas, such as the extension of legal protection over specific classes of specially protected persons.Footnote ⁷⁵ For example, network analysis could reveal the scope of the ‘particular social group’ legal criteria for asylum protection across European countries. Network analysis could also be coupled with techniques of natural language processing to extract accompanying text surrounding citation of a precedent in order to assess the factors that might constrain a court in following a precedent. Such an analysis could help to identify erroneous restraints that inhibit the transposition of precedents due to perceived differences in national legal traditions.Footnote ⁷⁶

Finally, network analysis can be used to detect whether the legal precedents that are cited in legal scholarship reflect their actual importance in a given area of jurisprudence. For instance, Olsen and Frese have used network analysis to identify the cases most frequently cited in the ECtHR’s Article 14 discrimination cases, and compared the findings to British, French, and German textbooks.Footnote ⁷⁷ The authors found significant discrepancies in what is cited in the textbooks with what is actually cited by the Court, thus reflecting an incomplete or outmoded picture of current law. This problem is arguably accentuated in EML as the discourse is heavily dependent on a small number of seminal textbooks and key case law data bases sponsored by EU institutions.Footnote ⁷⁸

Network analysis thus offers significant potential to build a more cohesive EML through doctrinal analysis or to highlight cross-jurisdictional differences and patterns in interpretation in order to narrow targets for litigation. In a field where national courts remain the principal drivers of interpretative developments in international law this is particularly important.Footnote ⁷⁹ Language and procedural differences, as well as the lack of trans-judicial dialogue, means that national judges rarely talk to each other on many core aspects of the law, largely because there are often inherent cultural – and constitutional – differences on issues that broach questions of national sovereignty.Footnote ⁸⁰ Enhancing scale through computational methods will of course not solve this but could make a significant step in helping to promote a better understanding of EML. Other methods, moreover, can also be invoked to examine decision-making heuristics amongst and across different national law judges, which we now turn to in Section 19.3 below.

19.3 Data Driven Analysis of Adjudicator Bias in the Common European Asylum System

A large part of doctrinal research on EML is framed towards elaborating the legal norms which are impugned in migration and asylum cases, such as the cardinal principle of non-refoulement, which constitutes a quintessential focus of EML. However, the design of asylum and expulsion procedures are largely unregulated by international law,Footnote ⁸¹ and even in states with similar legal traditions asylum procedures can be very different.Footnote ⁸² The Asylum Procedures Directive has proven difficult to implement, reflected in a widening gap between EU acquis prescriptions and national laws.Footnote ⁸³ Implementing effective asylum procedures is furthermore constrained by epistemic uncertainties, which enable high degrees of discretion in determining who is entitled to protection.Footnote ⁸⁴

Thus, despite decades of moves towards regional harmonisation, the chance of receiving asylum in Europe often appears so arbitrary that it is an ‘asylum lottery’.Footnote ⁸⁵ The UN Refugee Agency (UNHCR) states that within the EU ‘recognition rates for the same asylum-seeking populations can vary from between 1 per cent to over 50 per cent’,Footnote ⁸⁶ with studies suggesting that the EU has had only a limited impact on asylum outcomes,Footnote ⁸⁷ and it is one of the principal causes of ‘asylum shopping’, which tends to exacerbate the problem by encouraging a ‘race to the bottom on asylum policy’.Footnote ⁸⁸ What makes refugee status determination (RSD) as the last great enigma of EML scholarship so recalcitrant? Refugee lawyers have been troubled by outcome variation for quite some time, and despite a rich literature investigating its origins, no clear solutions have emerged for the problem.Footnote ⁸⁹ Legal scholars suggest inconsistency may arise through differences in institutional design, procedures, and legal interpretations across or within national asylum systems.Footnote ⁹⁰ Others point to differences in standards of evidenceFootnote ⁹¹ or problematic use of fact-finding reports.Footnote ⁹² Some propose that variation can arise from a lack of financial resourcesFootnote ⁹³ or interference to judges.Footnote ⁹⁴ The issue is thus not easily resolved within the internally focused confines of doctrinal legal scholarship.

Social scientific research suggests that the law is just but one factor in outcome variation. Some scholars argue that rules and standards can be applied in an ad hoc fashionFootnote ⁹⁵ or that the RSD is innately intersubjective.Footnote ⁹⁶ Asylum procedures are thus prone to forms of explicit and implicit bias,Footnote ⁹⁷ gendering dynamics,Footnote ⁹⁸ performative scripts,Footnote ⁹⁹ and cultural misrecognitions between applicants and decision-makers.Footnote ¹⁰⁰ Political science research suggests that asylum decisions might be impacted by legal extraneous factors such judge’s party affiliation and political events,Footnote ¹⁰¹ whilst medical studies emphasise the impact of stress amongst decision-makers and asylum seekers.Footnote ¹⁰² In sum, as Gorlick rightly notes, ‘perhaps there is no standard answer for differential recognition rates among like-minded states’.Footnote ¹⁰³ However, existing disciplinary perspectives from social sciences and law on this problem are rarely integrated, which limits the findings and prevents wider theorisation of the issues at stake.Footnote ¹⁰⁴

Computational research on outcome variation in RSD, however, holds significant potential to broaden our understanding of the problem on a more stable empirical basis. Early steps have been taken in this direction by empirical legal researchers working in a number of jurisdictions. In a seminal study, Ramji-Nogales et al. found significant variance in asylum decision-making by US judges even within the same sub-national courts, with evidence that ‘one judge is 1,820 per cent more likely to grant an application for important relief than another judge in the same courthouse’.Footnote ¹⁰⁵ Similarly, Rehaag’s study of Canadian asylum found one judge 50 times more likely than other judges in the same court to grant in favour of asylum applicants,Footnote ¹⁰⁶ whilst in Australia, Ghezelbash, Dorostkar, and Walsh found that one judge had a ‘success relative to the median’ that was 185 per cent higher than judges at the mean level (0), that is, a success rate of 23 per cent compared to the success rate of 7 per cent average.Footnote ¹⁰⁷ However, the research is currently empirically divided across national contexts, which makes it difficult to explain outcome variation in asylum decision-making between countries.

In our previous research, we have undertaken a comparative analysis of some of the factors which may be driving outcome variations through a dataset of 15,535 asylum cases tried by the Danish Refugee Appeals Board in the period 1995–2020 (RAB).Footnote ¹⁰⁸ We analysed these case files using Natural Language Processing (NLP) models to extract information from the text through regular expressions (a technique to define and match textual patterns that we were required to use because Danish language NLP is underdeveloped.) The case files are in the form of a judicial decision and stipulate not only the legal reasoning underpinning the decision, but also the procedural history, relevant facts and documents, as well as interview transcripts with the applicant. This is a rich dataset, which allows us to ask complex and sophisticated research questions concerning recurrent argumentation grounds, tracing the process and steps of asylum adjudication from the time when the applicant enters Denmark until the final decision, understanding why the RAB overturns first instance decisions, and inspecting the presence of systemic biases.

The RAB was established in 1983 as a quasi-judicial body and is composed of a chairperson who is a judge and two other members representing the Danish Immigration Ministry and the Danish Bar Association.Footnote ¹⁰⁹ First-instance decisions are made by the Immigration Service and Decisions rejected at first instance are automatically appealed to the RAB. The RAB as a quasi-judicial body has full legal competence to assess questions of fact and law, which means that there is no possibility to appeal its decisions to the ordinary courts. The RAB’s decisions are based on Danish immigration law, but relevant legal criteria remain largely based on international and EU law. Although the Danish opt out means that Denmark is not bound by certain EU law regulations in the asylum domain, it remains part of the Common European Asylum System (CEAS) and is also bound by certain directives due to its participation in the Schengen legal framework, and RAB decision-makers routinely follow developments in the EU legal sphere.Footnote ¹¹⁰

The information extracted first allowed us to analyse some factors that extant literature has deemed important to explain differences in asylum recognition rates, such as the marital status of the applicant, the sex and the age of the applicant, and the sex of the judge. In our analysis, we found no statistical difference for these factors. This suggests that previous studies on outcome variations might not be generalisable across all jurisdictions.Footnote ¹¹¹ In our case, the fact that we did not observe a difference in recognition rates based on the sex of the person presiding over the asylum decision might reflect both differences in national legal systems, training, and appointments, but it could also be due to the fact that in the Danish system appeal decisions are taken by a panel rather than by a single judge. This underscores the importance of taking into account differences in procedural and institutional architecture when moving to analyse and compare administrative decision-making at the national level, as these factors may significantly impact how EU law is understood and implemented.

We further used this data as a basis for comparing outcome variation between countries where data was available from previous research. We did this by employing statistical models. The results of our analysis of relative rates of dispersion are displayed in Figure 19.2.

Figure 19.2

Comparative analysis of recognition rates of asylum decision-makers.

Figure 19.2 Long description

Top left. The bar graph titled, Denmark, plots recognition rate ranging from 0.1 to 0.4 versus rank ranging from 0 to 30. The graph follows an increasing trend from (0, 0) to (0.42, 38). Top right. The bar graph titled, Australia plots recognition rate ranging from 0 to 0.20 versus rank, ranging from 0 to 30. The graph increases from (0, 0) to (30, 0.22). Middle left. The bar graph titled, Canada from 2008 to 2011. It plots recognition rate ranging from 0 to 0.30 versus ranks from 0 to 40. The graph increases from (0, 0) to (39, 0.35). Middle right. The bar graph titled, Canada from 2013 to 2016. It plots recognition rate, ranging from 0.05 to 0.15, versus rank, ranging from 0 to 40. The graph increases from (0, 0) to (46, 0.23). Bottom left. The bar graph titled, U S A, plots recognition rate ranging from 0 to 0.8 versus rank ranging from 0 to 500. The graph increases from (0, 0) to (499, 1). Bottom right. The multi-line graph titled, Comparison plots density ranging from 0 to 8 versus recognition rate from 0 to 1. It plots curves for Denmark, Australia, Canada from 2008 to 2011, Canada from 2013 to 2016, and U S A. They peak between the recognition rate of 0 to 0.4. Note, all data are approximated.

We can now broaden our analysis to include a comparison with other jurisdictions outside of the EU jurisdictional area. Here, we conducted a similar analysis on our data as well as data for other jurisdictions: the rates of the US judges for the period 2016–2021,Footnote ¹¹² the rates of the Australian judges,Footnote ¹¹³ and the overall rates of the Canadian leave judges in the judicial review stage for the periods 2008–2011 and 2013–2016.Footnote ¹¹⁴ Both EU law and European human rights law introduce more specific procedural standards and additional bases for protection compared to Australia, Canada, and the United States, who at the international level are primarily orientated towards the 1951 Convention Relating to the Status of Refugees. Whilst it would thus make little sense to compare outcomes or overturn rates directly, it is still relevant to compare internal coherence and variation amongst individual decision-makers.

Figure 19.2 shows the density plot of individual judges/panel chairs within each jurisdiction. Turning to the normalised comparison in the bottom-right panel, it immediately becomes clear that each jurisdiction (as expected) exhibits different overturn rates – the peak of each line representing the mode, that is, the overturn rate occurring most often across the individual judges. Looking at the distribution patterns, it is further clear that most jurisdictions exhibit a clear peak. Looking at both of the Canadian curves, the higher peak for the early period suggests a somewhat closer distribution of overturn rates amongst the majority of judges prior to the reform; yet what the reform achieved was an overall more unimodal outcome distribution, without the previous second peak representing a smaller cluster of judges with a higher turnover rate. The Danish plot is similarly unimodal, but with a wider and more symmetrical bell-shaped curve around the peak – suggesting slightly more individual but normal-distributed variation around the mean. The Australian curve exhibits several peaks, indicating different clusters of individual judges with similar overturn rates. Last, but not least, for the US data, the distribution does not have a well-defined mode or peaks, such that the US data exhibits the highest degree of dispersion.

The dispersion of the distribution tells us something about consistency amongst decision-makers. If cases are randomly assigned to judges, the variation observed in countries that exhibit a bell-shaped distribution are less likely to result from individual bias. Adjudicators that exhibit overly large or overly low recognition rates can be understood as outliers not descriptive of the overall system, although they impact the overall system. Under this understanding of consistency, the US system appears to be less consistent because of the variation pattern. However, two caveats apply. First, if the allocation of cases to judges is non-random, this type of variation may be explained on other grounds. Second, a certain degree of variation is to be expected given the specific circumstances of individual cases, and some variation will be inevitable, as legal scholars have noted, because it reflects ‘degrees of specialization, complexity, dynamism, emotional or ideological content, and spectrum of choice’.Footnote ¹¹⁵

Thus, on the one hand, the analysis seems to show that not all apparently random legal processes are equally random, on the contrary, some appear to exhibit a greater deal of decision-making consistency than others. However, it is important to not draw overly strong conclusions without considering all of the other variables which may shape the results. Domestic legal cultures can strongly affect national peculiarities, and it is relevant to consider that Danish society functions with a strong pull towards consensus.Footnote ¹¹⁶ The Danish experience is further instructive as the results are likely skewed by what in US empirical legal studies is known as ‘panel effects’, the impact judges have on each other with typical regard to such factors as race, gender, and ideology.Footnote ¹¹⁷ As previously mentioned, Danish panels are constituted with mixed expertise (i.e., judges, civil society, and lawyers)Footnote ¹¹⁸ which may mute the role of preference, whereas in the US the decisions of politically appointed judges often show a strong correlation with certain outcomes – that is, more ‘progressive’ judges are more likely to grant relief whilst more ‘conservative’ judges are more likely to deny applications.Footnote ¹¹⁹ Institutional design can also affect variation, and the Canadian data is interesting in this regard because the Canadian distribution in 2013–2016 is less skewed than 2008–2011, which indicates that inequality was reduced. This was precisely the result of Rehaag’s first major study finding inconsistency amongst Canadian asylum law judges and its recommendations, as judicial reform measures were subsequently undertaken by the state to ameliorate inequalities.Footnote ¹²⁰ Due to the confidential nature of this data, the data richness also comes with other challenges related to the need to anonymise data sources, which can further inhibit effective comparison on asylum data.

19.4 Potentials and Pitfalls of Staging Computational Interventions in EML

To briefly conclude, it is necessary to circle back to some critical points which have emerged from our discussion. First, the analysis has clearly shown some of the problems of large-scale quantitative analysis, and ultimately, what it can teach us. Quantification is a representation, and at best, relies on a high degree of interpretation. At worst, it reifies particular aspects of the world, selected because they happen to lend themselves more to statistical analysis. Computational research is inherently biased towards written materials, and will never be able to tell us what happened, for example, in a court room, which is the domain of qualitative forms of inquiry.Footnote ¹²¹ Whilst this is hardly a new problem, existing literature often underestimates the extent to which these dynamics are accentuated by algorithmic analysis, which may introduce additional ‘black box’ challenges of interpretation and in that sense entail a double removal of human subjectivities within the research. This underscores the importance of approaching computational research with proper domain legal expertise, but also being open to mixed-method approaches that draw in more critical perspectives as a means for interrogating and questioning research findings and data practices.Footnote ¹²²

This leads to another important reflection for researchers undertaking computational analysis of EML in a field such as EU law, which is overwhelmingly doctrinally orientated. Computational research often receives pushback from doctrinal scholars who claim that it devalues the normative aspect of law.Footnote ¹²³ The experience of researchers working on the Supra Legem Project (2017) in France is instructive here. Using a simply predictive algorithm, the project found significant disparities in asylum decision-making associated with the identity of asylum judges. Yet, the project also faced significant backlash from judges and the legislature, who subsequently introduced a general ban on research on individual judicial behaviour.Footnote ¹²⁴ Policy audiences may not always welcome hearing evidence that is unsettling, and interventions must be crafted with due regard for practices for implementing policy reforms.Footnote ¹²⁵ It must always be recalled that in contrast to computational discourse, law is a ‘slow hermeneutic’ that requires reasons, deliberation, and dialogue.Footnote ¹²⁶ In other words, connecting the empirical to the normative through computational research on EML is no easy task, but we have hoped in this contribution to lay some possible ways to bring this about.