Policy Process Analysis (PPA)

The main method we rely on for studying the political dynamics during the crisis and the variety of policy responses across our selected EU and country episodes is policy process analysis (PPA) (Bojar et al. Reference Bojar, Kyriazi, Oana and Truchlewski2021a). PPA intends to be a comprehensive method for the data collection and analysis of policymaking debates. As such, PPA aims at capturing the public face of policymaking, that is, the subset of actions in a policymaking process that are presented to the general public through the mass media. The method relies on analyzing media data based on systematic hand-coding of indicators related to the actors involved in the policy debate, the forms of action they engage in, the arena where the actions take place, the issues addressed, and the frames used to address these issues. The resulting dataset allows for the construction of more aggregate indicators at different levels of analysis (at the episode level, at the actor level, at different time units) for studying the policymaking debate and the political dynamics surrounding it from multiple angles, both statically and over time.

In its design, PPA is a specific form of political claims analysis (PCA) (see Koopmans and Statham Reference Koopmans and Statham1999) and also incorporates elements from other methods previously employed to study protest events (protest event analysis [PEA]) (see Hutter Reference Hutter and Grande2014; Kriesi et al. Reference Kriesi2020) or contentious politics (contentious episodes analysis [CEA]) (see Kriesi, Hutter, and Bojar Reference Bojar2019; Bojar et al. Reference Bojar, Kyriazi, Oana and Truchlewski2023) by making use of the systematic coding of media data. At its core, PPA is also an event-based methodology that focuses on identifying distinct actions undertaken by a variety of actors addressing particular issues and how they unfold over time. However, while PEA and CEA are usually limited to identifying either actions in the form of protest events or actions initiated by a limited set of actors (government versus challengers) to study mostly contentious politics, PPA enlarges the empirical scope to the study of entire policy debates.

Its encompassing scope and event-based focus make PPA a specific form of political claims analysis (PCA) (Koopmans and Statham Reference Koopmans and Statham1999). Starting from a critique of protest event and political discourse newspaper analysis as being too “protest-centric” and focused primarily on nonroutine protest actions, PCA argues for the need to include events that occur outside the context of reported protest but that are important for understanding conflict. As such, PCA extends event coding to including actions that take on institutional forms, such as legal actions, and including public and institutional actors beyond social movements. Our PPA methodology takes this critique seriously by also enlarging the empirical scope of the analysis to include both institutional and noninstitutional actions and actors. Where PPA departs from PCA is in its focus. PCA originated as a method primarily focused on studying the demand side of politics by taking as its starting point claims making (“strategic demands made by collective actors within a specific contested issue field”) (Koopmans and Statham Reference Koopmans and Statham1999: 206) and attempting to enlarge the study of contentious politics and placing it into its wider context. By contrast, our PPA methodology is essentially supply-side focused by having as a starting point policymaking processes and specific policy debates while attempting to place these in their wider political context. It is this supply-side focus that drives our strategy of analyzing selected empirically delimited policy episodes rather than general contested issue fields and studying the policy debate surrounding them in a systematic fashion.

In its supply-side focus, PPA also comes close to another approach to the study of policymaking processes – the comparative policy agendas (CPA) project (Baumgartner, Green-Pedersen, and Jones Reference Jones and Baumgartner2006). However, rather than focusing particularly on the agenda-setting phase of policymaking as the CPA does, PPA systematically incorporates into a single framework information about all the major components of a policy debate. Therefore, as further detailed in the section below, PPA documents actions ranging from formal steps in the policymaking process to administrative and nonstate actions but also protest events and even single verbal claims. Due to its goal of studying both the politics and the policymaking surrounding a particular episode, the actors documented in PPA are not restricted to solely governments; rather, they include all actors involved in the debate – political parties, civil society actors, supranational actors, and third-country actors.

Similar to CEA, PPA attempts to capture the middle ground between a qualitative narrative approach and a quantitative approach to describing the chronology of policy processes (see Kriesi et al. Reference Kriesi2019). By including extensive string descriptions of each action, PPA provides a rich body of qualitative information on the politics and policymaking surrounding policy debates. At the same time, by coding specific action characteristics, it allows for the construction of systematic, comparative indicators at various levels: countries, episodes, and actor types. In its qualitative inclination, one could think of this approach as being related to process tracing in that we seek to document all the various chains of actions involved in a policy debate as they unfold over time in a systematic fashion. However, process-tracing is aimed mainly at single-case inferences about the intervening causal process, that is, on the causal mechanisms that link a given cause to an outcome in a single case (George and Bennett Reference Georgiou and Zaborowski2005: 206–207; Beach and Pedersen Reference Beach and Pedersen2016: 4–5). In contrast, our method is aimed at combining such single-case inferences with cross-case inferences, making it essentially a comparative method. We therefore analyze the refugee crisis by comparing the variety of countries and episodes based on a combination of both qualitative evidence on the sequences of events in the form of descriptive narratives and systematic, quantitative indicators measuring relevant characteristics of these episodes (e.g., politicization and conflict intensity, as further detailed in the next section). In doing so, we aim to study within-country policy processes and how they evolve from problem pressure via domestic actor constellations and conflicts all the way to policy outcomes but also to compare such policy processes in a systematic fashion.

To sum up, PPA is well suited for our research goals in studying the refugee crisis for two main reasons. First, its broad empirical scope allows us to focus on a wide variety of actions and actors in systematically reconstructing the various policy debates both at the EU level and at the level of the member states. We can therefore use PPA for identifying the policymaking repertoires employed at these different levels, as well as for systematically studying the wide variety of actors involved in these debates, their configurations, and their discursive strategies. Second, by aiming at the middle ground between quantitative and qualitative approaches, PPA allows us to combine systematic, comparative indicators of the various aspects of politics and policymaking surrounding policy debates with the reconstruction of the narrative chronology of these policy debates by the use of a rich body of qualitative evidence.

Having set up our empirical universe as bounded segments of the policy debate that take the form of distinct policy episodes embedded in the broader context of the 2015–16 refugee crisis, the first step in constructing our PPA dataset was defining and gathering the media corpus to be analyzed. Therefore, the first decision we were confronted with was source selection, that is, the selection of news media to be studied. Depending on the level of policymaking, we selected either international news sources (for the EU level) or national news sources (for the level of the member states). We used the news aggregator platform Factiva for document retrieval, as it provided us access to a large number of media outlets, which allowed for systematic multicountry comparison together with transparent and replicable selection criteria on the source.

Following good practice standards in working with media data from methods such as protest event analysis (Hutter Reference Hutter and Grande2014; Kriesi et al. Reference Kriesi2019), we also tried to engage with issues of selection bias (e.g., Earl et al. Reference Earl, Martin, McCarthy and Soule2004; Ortiz et al. Reference Pardos-Prado2006), that is, with the biases associated with news source selection and their coverage of debates, actions, or events. In order to mitigate such biases, we adopted several strategies. First, we relied on a wide variety of media sources, rather than a single source, in order to be able to capture as many aspects of the policy debates as possible. Second, as just mentioned, in order to mitigate biases related to newsworthiness and proximity, we selected news sources that are proximate to the level of analysis: For EU debates, we focused on large news agencies (Agence France Presse, Associated Press, Reuters, Financial Times, Euronews, ANSA, BBC, MTI), while for national debates, we relied on national media. Third, in order to mitigate biases related to the political motives of the various sources and their potential impact on news coverage, we selected news sources on different sides of the political spectrum. Consequently, for each of the eight selected sources, we selected one major newspaper left of center and one right of center in terms of ideological leaning (with some minor exceptions related to data availability).

After selecting the news sources, the second decision related to corpus construction consisted of the identification of the keywords used for the retrieval of articles related to a particular episode. One of the main considerations at this step was achieving a balanced relevance ratio – the ratio between false positives (irrelevant articles that the keyword combination retrieved as positive hits) and false negatives (relevant articles that the keyword combination filtered out as negative hits). Since our data is manually coded, we aimed for a relatively slim but robust corpus. That is, our corpus needed to be manageable in terms of the number of articles identified so as to not make the coding process too cumbersome and resource intensive, but it still allowed us to capture the full range of actions in a given policy episode without missing relevant articles filtered out by a too restrictive keyword combination.

In practical terms, the selection of keywords related to each of the EU and country episodes was performed by the authors of the book in close collaboration with a team of native-language-speaking coders (mostly comprised of political science PhD students who were also knowledgeable about the subject at hand – the refugee crisis). At this stage, we took advantage of the capabilities of the news aggregator Factiva, which allowed us to construct complex search strings using Boolean algebra and its standard logical operators. For each episode, we chose an initial set of episode-specific keywords based on secondary sources (policy reports, secondary scientific literature, etc.) and initial search queries in the national press. We then further refined this initial keyword selection through an iterative process of going back and forth between the selection and the corpus obtained. We selected those keyword combinations that passed the initial reading of the selected articles and achieved a satisfactory balance between the size of the corpus and the number of events filtered out.

After having constructed the corpus, the last step in the PPA coding process consisted of action coding. As already mentioned, PPA is an event-based methodology and hence the unit of observation at the level at which the data is collected is an action. An action in our framework is defined as “an act, or a claim by an actor with a prominent role in the political world that has a direct or indirect relevance for the policy debate” (Bojar et al. Reference Bojar, Kyriazi, Oana and Truchlewski2023). Therefore, within our framework, actions can be steps in the policymaking process, verbal claims, episode-related protest events, and other types of actions that we outline in the coding scheme below. This definition is rather open-ended because the relevance of an action is contingent on the specificities of the actual policy debate at hand.

Note that while the lowest level of observation is an action, the unit of analysis at which we draw conclusions can be pitched at any level of aggregation (actor types, issue categories, entire episodes, types of countries, etc.) depending on the research question, as will become apparent in the following chapters. In order to measure the various features of actions, action coding is based on a common core of variables that are coded for each of the actions in each episode: the arena where the action takes place, its (procedural) form, its (substantive) type of engagement with the policy, its overall direction vis-à-vis the policy, its direction vis-à-vis target actors, the organizational characteristics of the actor undertaking them, the organizational characteristics of the target actor, the issues it engages with, and the normative frames used by actors to present their positions to the public (Bojar et al. Reference Bojar, Kyriazi, Oana and Truchlewski2023).

Based on initial trial rounds of action coding, we refined these major characteristics of an action with specific categories relevant for the refugee crisis. This resulted in a detailed codebook with hierarchically organized categories at various levels of specificity. The codebook was complemented by a dedicated coding spreadsheet that was provided to the coders to make the data collection process as systematic and comparable throughout country episodes as possible. At the end of the coding process, at the national level, our team identified 6,338 codable actions for the 40 episodes, yielding 157 actions per episode on average. However, there is considerable variation in how eventful the individual episodes are, ranging from 48 actions in the Residence Permits episode in Sweden to no less than 363 actions during the quota referendum in Hungary. In fact, Hungary has proven to be the most eventful of our eight countries with 1,204 actions, followed by Greece with a total of 1,086 actions. At the EU level, we have coded 1,257 actions in the six episodes, with the EU–Turkey Deal being the most eventful one (437 actions), while the EU–Libya episode had the lowest number of actions (62). These two datasets are complemented by the EU–Turkey Deal episode and the debate surrounding it in four member states containing an additional 1,138 actions. In the following text, we describe how each action characteristic was implemented in our data collection effort.

The first set of characteristics for each action that we have identified is related to the arena where it takes place. Arena choice is an important aspect of the policy debate because it can shape the type of actors that gain access to policymaking, the size and type of audiences that participants can address, and the type of policy options on the table as a function of the gate-keeping role of agenda setters (Timmermans Reference Timmermans2001; Lowi and Nicholson Reference Lowi and Nicholson2009; Princen Reference Princen2011). Arenas are also important because procedural forms of action depend on where they take place. We identify nine types of arenas in our codebook (see Figure 3.1) varying from decision-making institutional arenas such as the national governments to less institutionalized arena types such as protest or society more generally. Furthermore, for each of these nine arenas, we also identified specific forms of the action. For some of the arenas, the set of action forms was based upon long-standing traditions in the pertinent literature, such as the set of action repertoires in the protest arena (Traugott Reference Trauner1995; Della Porta Reference Della Porta, Snow, della Porta, Klandermans and McAdam2013), while for others, such as the media arena, it was decided inductively based on our trial coding.

Figure 3.1 Policy action arenas at the national and EU levels

Figure 3.1 shows that most of the actions in our dataset at both the EU level and the national level take place in the media area (these usually come in the form of statements, press conferences, interviews, etc.). Unsurprisingly, the next most prominent arenas in our dataset at the national level are national governments and parliaments, while European institutions and the cross-national arena prevail at the EU level. Beyond these arenas, our data collection effort also captured actions taking place in the electoral arena, in the protest arena, and at the level of society more generally, thereby providing us with a multifaceted picture of the policy debate not only in venues mostly dedicated to supply-side actors but also in venues where demand-side actors such as civil society organizations most often operate.

After settling the “where” of the action, the next set of characteristics refers to the type of action that actors undertake with reference to the policy proposal and to other actors involved in the policymaking process. In this respect, we included a wide action repertoire, distinguishing between policymaking steps, policy claims, administrative state actions, and nonstate actions. It is at this level that our PPA methodology is distinguished from other methods dedicated to analyzing policymaking processes such as the comparative policy agendas (CPA) project (Baumgartner et al. Reference Baumgartner, Green-Pedersen and Jones2006). Rather than only studying formal steps in the policymaking process, our dataset also includes verbal claims and statements made by a variety of actors in the policy process. In fact, as Figure 3.2 reveals, the most prominent policy action forms at both the national and EU level are precisely policy claims (these usually include actions such as full verbal support/opposition of the policy, clarifications, apologies, and verbal commitments to further action).

Figure 3.2 Policy action types at the national and EU levels

Distinguishing between policy claims and formal policy steps provides us with a more nuanced picture of how the policy debates unfolded, as these substantive types of action most often also indicate whether the action implies a broad level of agreement or disagreement with the underlying policy on the table. In addition to the substantive action types, we also use a general policy direction code (positive, negative, or neutral) as an indicator of the actor’s position regarding the issue at stake. Finally, since in most of the episodes we follow up on the implementation of the policy in question and also include actions undertaken by nonstate actors (such as policy evaluations, NGOs involved in the implementation of a particular policy), we also consider administrative actions performed by state and nonstate actions.

Beyond characteristics of the action itself, the actors involved in a particular policy debate are of particular interest to us, as is shown in Part II of our book. By studying actors and the actions they undertake, we are able to analyze conflict structures and dynamics of coalition formation at both the national and EU levels, which is crucial in the negotiation stages of these policy episodes. Note that at this stage, we try to identify not only which actors undertake a particular action but also whether that particular action is targeted at other actor(s) in the policy debate. We therefore take into account two types of actions: monadic actions, which only have an initiator actor who addresses an issue, and dyadic actions, which have initiator actors who address not only an issue but also a target actor. For the dyadic actions, similar to the policy direction code, we introduce an actor direction code (negative, positive, or neutral) that captures the actor’s relational position vis-à-vis the target actor regardless of how they relate to the policy as such.

The actor characteristics that we identify in our PPA data collection are organized hierarchically at four levels: their nationalities; their broad institutional affiliations (such as the national government); their narrow institutional affiliation (a particular ministry); and, in the case of individual actors, their position within the institution’s hierarchy (executive, subexecutive, or lower rank). This hierarchical organization allows us to study actor configurations at various levels of specificity, identify both domestic coalitions and cross-national coalitions, as well as capture dominant decision-making modes such as executive decision-making or partisan contestation.

In Figure 3.3, we present the share of actors involved in policy debates at the EU and national levels according to their broad institutional affiliations. National governments are the central actors in our domestic policy debates, with more than 30 percent of the actions being initiated by them. In contrast, inter- and supranational actors are the initiators of most actions (more than 80 percent) at the EU level. Despite these two categories unsurprisingly taking center stage at their respective levels, we can see that other national institutions (e.g., regional authorities), political parties both in government and in opposition, as well as interest groups and civil society actors have nontrivial shares of actions, especially at the domestic level.

Figure 3.3 Initiator actor types at the national and EU levels

Although in all the episodes we select actions relating to a particular policy proposal, most of the time the debates tend to revolve around more than one issue. Many of these proposals are in fact policy packages containing multiple issues that collectively make up the policy debate. Moreover, many actions do not directly relate to the policy but are important nevertheless because they have the potential to influence the future course of the debate. Differentiating between specific issues allows us to capture the more fine-grained thematic crisis responses, which are discussed in more depth in Chapter 5. We therefore introduce a set of issue codes organized in such a way as to capture a broad categorization of migration-related policy areas as reflected in the organization of asylum and migration policies in the EU member states.

Figure 3.4 presents the broad categorization of issues our episodes involve.Footnote ² We can see that at both the EU and the national level, asylum issues and border control issues dominate the agenda. As some actions are directed toward a whole policy package (i.e., an episode), we introduce this as a specific issue. As we are interested also in some actions that do not directly relate to migration policy but are relevant for the policy debate at hand, we have complemented this with an “others” category to capture impactful actions and/or events in our episodes, such as issues pertaining to diplomatic relations between countries or humanitarian tragedies.

Figure 3.4 Issues at the national and EU levels

Finally, an important characteristic we include in our study relates to the discursive frames actors use. This essentially refers to the ways in which actors justify their action or interpret the political problem at hand. Such discursive framing is important because it can shape other actors’ attitudes and behavior (Koopmans and Statham Reference Koopmans and Statham1999; Rucht and Neidhardt Reference Rutz1999). The frames employed give us an overview of the communication strategies employed by the various actors involved in the policy debate and can be used to identify discursive coalitions in the political process.

Figure 3.5 presents the main frames identified in the refugee crisis grouped into four major categories. These broad categories were constructed inductively based on several rounds of trial coding and were further adapted though the data collection process. We observe that while humanitarian and democratic frames appear to be important at both the EU and national levels, there is still a wide discrepancy between the discursive strategies that actors employ at the two levels regarding other framing categories. At the national level, sovereignty, security, and identity frames dominate the discourse, but at the EU level, international solidarity arguments take a more central place. Chapter 9 will further delve into the issue of framing, looking at the role of discursive coalitions within the refugee crisis.

Figure 3.5 Frames at the national and EU levels

If above we have presented general descriptions of the major characteristics of actions captured by our PPA dataset, these characteristics also stand behind the formation of systematic, comparative indicators used across our country episodes. One example of such an indicator that is used extensively in the following chapters is politicization (De Wilde Reference De Wilde2011; Hooghe and Marks Reference Hooghe, Marks, Jones, Menon and Weatherill2012; Hutter and Kriesi Reference Bremer, Schulte-Cloos, Kriesi and Hutter2019b). Politicization allows us to capture the expansion of the scope of conflict within the political system (Hutter and Grande Reference Hutter and Kriesi2014: 1003). We conceive of politicization as a multifaceted concept involving a dimension of salience (the number of actions occurring in a particular episode in a particular time frame) and a dimension of polarization (the share of positive and negative actions in that timeframe), both of which are captured by our PPA dataset.Footnote ³ Another indicator based on our PPA dataset that we use in Chapter 6 is conflict intensity, which is designed to capture the conflictual nature of the policy actions undertaken by actors. We define conflict intensity as a combination of the type of policy action that the actor undertakes and the direction of their actions vis-à-vis their target actors. While politicization allows us to capture the expansion of the scope of conflict, conflict intensity allows us to capture its nature, as some policy actions in our dataset are more conflictual than others (e.g., threats and denigrating opponents are more conflictual than simply proposing a new policy or negotiating) and as actions can be negative, positive, or neutral toward target actors. Accordingly, each action in our dataset is assigned an ordinal conflict intensity score on a 5-point scale based on a classification of policy actions and direction toward the target actor.

Complementary Data Collection Methods

While PPA constitutes the core data collection method used in our study, its empirical reach is not all-encompassing. First, our PPA data can unveil party competition dynamics related to the particular episode at hand but not the wider spectrum of such dynamics in the immigration field in our particular countries. Second, the PPA data described above are not suitable for capturing public opinion dynamics in the refugee crisis, such as the salience of immigration issues in the public or the public legitimacy of the policy outputs. Third, our PPA data allow us to capture the rhetorical devices employed by different actors in the refugee crises only to a limited extent. For these reasons, we complement the PPA data with various other datasets throughout the following chapters. While some of these datasets are widely known and available (e.g., major surveys such as the European Social Survey and the Eurobarometer), some others have been originally collected for the purpose of this book. We briefly describe these latter types of data collection strategies in the following sections.