Hostname: page-component-7d684dbfc8-dh8xm Total loading time: 0 Render date: 2023-09-27T20:04:06.590Z Has data issue: false Feature Flags: { "corePageComponentGetUserInfoFromSharedSession": true, "coreDisableEcommerce": false, "coreDisableSocialShare": false, "coreDisableEcommerceForArticlePurchase": false, "coreDisableEcommerceForBookPurchase": false, "coreDisableEcommerceForElementPurchase": false, "coreUseNewShare": true, "useRatesEcommerce": true } hasContentIssue false

Debating Europe, transforming identities: assessing the impact of deliberative poll treatment on identity

Published online by Cambridge University Press:  16 January 2017

Danilo Di Mauro*
Università Unitelma Sapienza, Viale Regina Elena, 295, 00161 Rome, Italy Badia Fiesolana - Via dei Roccettini 9, I-50014 San Domenico di Fiesole (FI), Italy
Irena Fiket
Centre for the Study of Political Change (CIRCaP), Faculty of Political Sciences, Via Mattioli 10, I 53100 Siena, Italy


Although there is a considerable amount of talk about transformative power of deliberation on identity, the debate in literature remains highly theoretical in underlying the benefits of deliberative model for EU Integration. So far, little empirical evidence is available on the actual impact of deliberation. Can deliberation enhance European identity? We specifically address this question by using deliberative polling quasi-experiment that involved random sample of 348 European citizens in 2 days deliberation on issues of European concern. The comparison of citizens’ sense of belonging to both EU and nation states before and after deliberation, allowed us to explore the effects of deliberation on identity and further test it against the control group. The analyses show that when European citizens are enabled to deliberate on European issues beyond national borders their exclusive national identity decreases and they become more community minded. The observed transformation of identities is further analyzed in order to explore the relationship between European and national identity. The analyses indicate that even after deliberative treatment in which European identity has been activated the relationship between multiple identities remains compatible.

Research Article
© Società Italiana di Scienza Politica 2017 


European identity, as other collective identities, is not a given unchangeable construct, but rather a context-dependent one (Castano and Yzerbyt, Reference Castano1997; Risse, Reference Risse2004), that could be intensified when exposed to certain stimuli. Within this article, we aim to test empirically this assumption by verifying whether deliberative mini publics may represent contexts that stimulate the formation and the development of the European identity. The deliberative polling held in 2009 in Brussels provided data for this research, enabling us to fill in an empirical gap in literature about deliberation and identity formation. The assumption that deliberation has a ‘community-generating power’ (Cooke, Reference Cooke2000) remains indeed strongly normative due to the lack of empirical findings. This lack is even more acute in European studies where deliberation is mostly conceived in theoretical terms. At the same time, many deliberative mini publics have already been promoted by the EU in order to develop European Identity. Therefore, on one hand, there is a strong normative claim that deliberation could promote European identity and a belief of EU institutions that citizens’ involvement in deliberation plays an important role in defining the future of the EU. On the other hand, there is a complete lack of empirical research on the effects of deliberation on European identity.

In fact, our knowledge on formation and development of citizens’ European identity is mainly derived from studies based on Eurobarometer (EB) or similar surveys (Duchesne and Frognier, Reference Duchesne and Frognier1995; Green, Reference Hansen and Andersen2000; Citrin and Sides, Reference Citrin and Sides2004). Those studies confirmed that the levels of European identity are rather stable over time,Footnote 1 although the types of data used show only a partial and static picture of the phenomenon and present many limits in the explanation of its origins. On the contrary, the data produced by deliberative mini publics that simulate a public sphere (Fung, Reference Gastil, Bacci and Dollinger2003), as a place where identities could be formed, may give us an insight on how the process of identity formation and development works.

Studies that explored European identity also showed that national and European identity are nested (Medrano and Gutierrez, Reference Duchesne and Frognier2010), that national identity affect attitudes toward EU and European integration (McLaren, Reference McDermott2006), and that most of the EU citizens feel ‘national’ and European at the same time. Following these findings, we will explore whether deliberative mini publics promoted by the EU can really have a significant impact on different types of identity. We will analyze the effects of deliberation on both, national and European identity and the association between them.

In the first part of the article we briefly review the studies at the origin of our research question. The second part focuses on the description (and discussion) of the EuroPolis, a European-wide deliberative poll (DP) quasi-experiment which provided the data for the empirical part of our study. Empirical analyses of the effects of deliberation on both, the European and national identity are presented in the third part. Finally, the conclusion remark and discuss our findings.

Deliberation and European identity

With the deliberative turn (Neyer, Reference Neyer2006), the relationship between collective European identity and deliberation assumed a central place in EU studies. Although there is a considerable amount of talk about European integration through deliberation, about potential of deliberation for overcoming the democratic deficit of Europe (Eriksen and Fossum, Reference Eriksen2000; Eriksen, Reference Farrar, Fishkin, Green, List and Luskin2006), and about transformative power of deliberation on preferences and identity (Checkel, Reference Checkel2000), the debate in literature remains highly theoretical in underlying the benefits of deliberative model for EU Integration. So far, little empirical evidence is available on the impact of deliberation on EU integration. When available, it is focused on describing the deliberation within institutional bodies (Joerges and Neyer, Reference Karpowitz and Mendelberg1997) or discussing the models of policy-making of the EU (De la Porte and Nanz, Reference De la Porte and Nanz2004).

Empirical studies that specifically address citizens’ deliberation within EU are rare and usually do not approach explicitly the question of identity. Focusing on deliberative mini publics promoted by EU institutions, Abels (Reference Abels2009) and Hüller (Reference Isernia and Fishkin2010) explored their potential to improve EU democratic performance and their (limited) capacity to influence democratic processes. While recognizing positive short-term effects on participants’ political skills and competences, they also suggest that the deliberative devices could, eventually, foster a sense of European identity, given that one of the main goals of mini publics examined was to develop a sense of European identity. However, the question of European identity was not the main object of their studies also due to the lack of data measuring identity.

Up till now, the only empirical evidences related to the relationship between European identity and deliberation are provided by Fiket et al. (Reference Fishkin2014), that analyzed the contents of discussions on immigration of four EuroPolis groups. Their study highlights that European mini publics may foster identity change developing a self-awareness of citizens of a shared European polity. Other studies using EuroPolis dataFootnote 2 did not specifically investigate the issue of European identity. Those studies showed that is possible for European citizens, to deliberate across barriers of language and nationality, to become more informed, to consider different arguments (Fishkin et al., Reference Fligstein, Polyakova and Sandholtz2014), to change their vote choice (Bernhagen and Schmitt, Reference Bernhagen and Schmitt2014) and formulate ‘judgments based on arguments offering reasons’ (Gerber et al., Reference Green2014).

Along with the scientific community, EU institutions also consider the citizens’ participation and deliberation as a possible solution for low levels of European identity and for the democratic deficit of the EU (Olsen, Reference Olsen2003). European elites, in fact, since the very beginning of the political Union perceived European identity as a response to various political crises. Following the idea that without developed sense of common identity, there is no progress of the project of EU integration. European identity has also been placed at the top of the research agenda of the European Commission since the 1990s and the 5th Framework Program. The interest of EU institutions in deliberative ‘experimentalism’ came along with the interest in European identity and its development. Deliberative mini publics promoted by the European Commission under the Plan D, the European Citizens Consultations and the AGORA projects promoted by the European Parliament as well as deliberative projects funded under the Framework Programs represents in fact, only a ‘new generation’ of remedies proposed by the EU institutions as an answer to political crisis of the EU.

Therefore, we believe that there is a need to explore empirically the real potential of deliberative mini publics.

From a theoretical point of view, a valuable perspective on what could be the impact of deliberation on European identity is offered by constructivist understanding of European identity. The constructivist approach is inspired by a variety of theories such as social identification, social constructivism, and group reference theories (Herrmann et al., Reference Hüller2004; Checkel and Katzenstein, Reference Checkel and Katzenstein2009). This wave of studies rediscovered European identity, differently than previous studies that used identity to explain other social phenomenon’s related to European integration, such as support for European integration, as it focused on European Identity as a dependent variable. According to constructivist literature, the identity is a part of individual self-concept that derives from the consciousness of being a member of a social or political group. The awareness of belonging to a group is seen rather as a process than as a given unchangeable fact. More precisely, the constructivist approach to European identity, conceptualizes European identity as a process of becoming, through the practices of interaction, socialization, dialogue, and discussion (Checkel, Reference Checkel2001; Herrmann and Brewer, Reference Herrmann, Risse and Brewer2004; Checkel and Katzenstein, Reference Checkel and Katzenstein2009; Risse, Reference Risse2010). The most important factor that influences the formation of European identity is the psychological existence of Europe in peoples’ minds (Risse, Reference Risse2010). The concept of entitativity (Castano, Reference Castano and Yzerbyt2004) can help to understand the formation of identity. Entitativity refers to the extent to which a community is perceived to be a real entity. Empirical findings speak clearly in support of the entitativity thesis: those who are in close and frequent contact with EU institutions, policies, and symbols are those who show higher level of European identity (Castano, Reference Castano and Yzerbyt2004; Risse, Reference Risse2010; Sigalas, Reference Sigalas2010; Fligstein et al., Reference Fung2012). So, the main assumption that follows from entitativity thesis is the more citizens are in ‘contact’ with the EU, the more EU will become real in their lives and therefore the more they will feel as a part of the EU community.

Another concept provided by the literature that is also beneficial for understanding the formation of the European identity of the citizens is European public sphere(s) (Trenz, Reference Trenz2005, Reference Trenz2009; Risse, Reference Risse2010) defined as a social construct which ‘emerges in the process through which Europeans engage one another and debate issues of common European concerns across borders’ (Risse, Reference Risse2010: 11). At the same time the European public sphere is seen as an arena where the European identity is constructed. Even here, the empirical findings speak clearly: the more people participate in EU public sphere, like for instance, during electoral campaigns and salient moments such as enlargement (see Risse, Reference Risse2010), the more they will feel as Europeans.

All these considerations on formation/construction/development of European identity provided by the above-mentioned studies, even if differently named and defined, underline the same assumption that stands behind the deliberative model of identity development: contact with other co-citizens and discussion on common EU problems will make the existence of the EU become ‘real’ in people’s minds, increase the awareness of common problems and common faith and therefore they will develop a sense of belonging to the same community. In our paper we try to test this assumption by using the data produced by DP quasi-experiment in which a random sample of 348 European citizens participated. More precisely, we hypothesize that the process of deliberation that happens within the deliberative setting will lead to intensified European identity, similar to the process of socialization and learning that take place within a European public sphere(s). With this we are not stating that our consideration related to the deliberation in DP and other deliberative mini-publics can be directly translated to the discussion on European public sphere. The real practical value of our research is to be found in the fact that we are testing the potential of, already very numerous, deliberative tools promoted and implemented by the EU institutions with the aim to foster dialog between, and identity of, its citizens.

In the next part of the article, we will describe the deliberative setting that we analyzed: EuroPolis–European-wide DP quasi-experiment. Besides, we will briefly expose the limits and advantages of DP as a setting that aims to operationalize deliberation.

Research design: EuroPolis–European deliberative poll quasi-experiment

DP is a deliberative setting created by Fishkin in 1988 for studying processes of deliberation and opinion formation, having as a main aim to show that people could become ‘better citizens’ when they are given the opportunity to engage in meaningful deliberation on public issues (Fishkin, Reference Fishkin1997; Luskin et al., Reference Macedo2002; Hansen and Andersen Reference Herrmann and Brewer2004). On one hand, DP is a deliberative practice that aims to improve democratic performance and it was already used as a democratic instrument with direct influence on the policy-making process (Fishkin, Reference Fishkin, Luskin and Siu2009). On the other, DP is a social science quasi-experiment designed to test deliberative democracy assumptions. Many of the empirical studies that have explored the effects of deliberation using the DP confirmed that participation in DP produces different ‘democratic’ effects: political sophistication, political interest, internal political efficacy, political trust, political ‘respect’, political empathy, ‘sociotropism’, and more positive attitudes toward the political system (Hansen and Andersen, Reference Herrmann and Brewer2004; Fishkin, Reference Fishkin, Luskin and Siu2009; Mansbridge, Reference Medrano and Gutierrez2010). Although the main idea on which deliberative mini-publics such as DP are based is that discussion and deliberation have a positive effect both on the health of democracy and the citizens, the main focus of empirical research remained primarily concerned with understanding the effects of deliberation on citizens’ opinions about the issue at hand. The assessment of the ‘community-generating power’ of deliberation does not represent the main aim of DP and similar designs in general, but it does represent the specific goal of our research.

Operationalization of deliberation in EuroPolis deliberative poll

Process of deliberation in DP is operationalized and implemented trough five key steps. First, a cross-sectional random survey is conducted on a representative sample of the population. Second, a random sub-sample of citizens is selected and is invited to participate at the deliberative event. Third, balanced background materials (briefing materials) are sent to those who agreed to participate in order to inform them about the issues. Fourth, participants come together for some days (usually from 1 to 3), and are randomly assigned to moderated small groups where they discuss the issues. As part of their small group discussions, they develop questions to ask to a balanced panel of experts and politicians during the plenary sessions. Fifth, at the end of the event they fill in a questionnaire. The effects of DP are assessed through the comparison of the data collected by questionnaires. Most of the deliberative mini publics promoted and implemented by EU are structured in similar way.

Note that even if a DP usually includes some elements of experiments such as treatment, pre and post test, and sometimes control groups and random assignment, it seems that it does not fully meet the standards of controlled experiment (Farrar et al., Reference Fiket, Olsen and Trenz2010). In fact, we are rather dealing with a quasi-experimental design in the case of DP. This is mainly because it lacks a high-level control of the variables compared with control implemented in experimental design (Campbell and Stanley, Reference Campbell and Stanley1963). Scientific experiments, differently than deliberative quasi-experiment such as DP, are internally valid – the researcher is sure that there are no other variables, except those manipulated, that influence the observed phenomena so the variation observed can only be attributed to the manipulation of controlled independent variables. The ‘deliberation’ treatment in DP consists of exposure to anticipation of the event at the moment of recruitment, briefing materials, discussion in small groups, moderators’ interventions, interactions with experts and politicians and informal discussions outside of the structured sections. In this way, it is not possible to precisely distinguish which aspect of deliberative ‘one grand treatment’ (Luskin et al., Reference Macedo2002) is responsible for variation on the dependent variable(s) – the outcome of deliberation (Karpowitz and Mendelberg, 2011).

Besides, the self-selection of individuals that, at minor or major extent, is taking place in deliberative quasi-experiments,Footnote 3 represents another possible threat to internal validity of DP since those individuals who self-select, in systematic way differ from randomly selected individuals (McDermott, Reference Mansbridge, Bohman, Chambers, Christiano, Fung, Parkinson, Thompson and Warren2002).

Another problem that precludes the possibility of DP, and other deliberative quasi-experiments, to qualify as true experiments is that they are affected by Duhem–Quine problem. It posits that it is not possible to test one hypothesis in isolation because an empirical test necessarily includes a set of auxiliary hypotheses a part from the main one. Those auxiliary hypotheses are background assumptions which are not proven by previous empirical tests (Bardsley et al., Reference Bardsley, Cubitt, Loomes, Moffat, Starmer and Sugden2010). In deliberative quasi-experiments we test the main hypothesis – that the process of deliberation will bring certain benefits – assuming that the process of deliberation will take place. The main auxiliary hypothesis depends on other hypotheses dealing with inclusiveness, equality of participation, exposure to different opinions, the reason giving requirement just to name those that are central to all deliberative theories. The latter hypotheses again, depend on other hypotheses which could be: moderators really facilitate exchange of arguments and do not push the discussions in direction they consider appropriate; participants have enough time to reflect on the arguments, etc. The consequences of this problem are highly significant when it comes to the interpretation of the results. In DP the extent to which inclusiveness, equality of participation, exposure to different opinions, and reasoned opinion expression take place is not assessed. Basically, what deliberation should actually look like is inherent in the DP setting (Siu, Reference Siu2008) but not measured afterwards, differently than its effects. Using the pre and post questionnaires as only methods of measurement does not tell us anything about the way in which deliberation in DP is actually performed.Footnote 4

However, even though the way in which deliberation is operationalized and implemented in DP is far from ideal, it still represents one of the best ways to implement deliberation (Mansbridge, Reference Medrano and Gutierrez2010) and what is more important, most of deliberative experimentation of the EU institution is based on implementation of similar deliberative tools.

Two topics were discussed during the EuroPolis DP: climate change and immigration at the European level.Footnote 5 The exercise itself proceeded following the standard design: small group discussions, formulation of the questions, and plenary sessions.Footnote 6 Participants were assigned into 25 small groups consisting of two or three languages. In order to allow all participants who spoke 21 different languages to communicate in their mother tongue, simultaneous translation was provided for each group.

On the first day, participants attended the welcome plenary after which they met in small groups and filled in the questionnaire (Time2 questionnaire).Footnote 7 When this task was completed, the discussion on the first topic (immigration) started and it was concluded with the formulation of questions for the plenary session with experts, scheduled for the next day. The second day started with the plenary session in which three experts participated in the discussion on immigration. After the plenary, participants returned to their small groups and discussed the climate change issue. After discussion and formulation of questions, citizens participated in a plenary session that involved two experts on climate change. The day concluded with a social dinner. The third day instead started with a small group session and concluded with a final plenary session.Footnote 8 After the plenary, participants filled in the Time3 questionnaire.

EuroPolis followed the DP standard design adding two important elements of inquiry: the survey was repeated more times and control groups were included as a part of the research design. Inclusion of a control groups allowed the comparison of those who participated at the DP (test group), those who declined to participate at the DP (i.e. non-participants), and those who never were informed about the DP (i.e. control group).Footnote 9 Overall, the data have been collected four times: before, during and after the DP, and one more time after the 2009 EU Parliamentary elections held in June. More precisely, the first survey of about 4300 EU randomly selected citizens started 1 month before the quasi-experiment (Time1).Footnote 10 In total, 3000 randomly selected individualsFootnote 11 out of 4300 interviewed were invited to take part and a random sample of around 400 individuals was drawn from all those accepting the invitation. The latter group became the test group and 348 of them attended the DP event. They filled in the questionnaire both at the beginning of the DP (Time2) and at the end of it (Time3). In parallel, 1300 randomly selected individuals out of the initial 4300 interviewed were not invited to the DP event, but they were interviewed both in the initial survey (Time1) and in the final survey conducted in June after the 2009-EU Parliamentary elections (Time4). This latter group was the control group. The Time4 post-elections survey was also submitted to the test groupFootnote 12 (for a graphical representation see figure 1 in Isernia and Fishkin, Reference Isernia, Fishkin, Steiner and Mauro2014: 315). The questionnaires contained items that measured the following information: policy preferences and levels of knowledge on two policy issues discussed, views on European integration process and perceptions of EU institutions and decision-making processes. It also collected the data about political participation, interest in politics, political knowledge, trust in others, involvement in community work, and sense of belonging to the EU. Finally, socio-demographic data of respondents have been also collected at Time1.


In order to study the possible effect of deliberation treatment on European identity, we rely on three questions. In the EuroPolis questionnaire, there were actually two questions that explicitly measured European identity and one that measured national identity. Even though our research focus on European identity, we decided to explore the relationship between European and national identity because literature showed that the relationship between those two identities are indissolubly linked.

The first question measuring European identity (similar to the so called ‘Moreno Question’) asks the respondent to prioritize between the nation and Europe, putting the two identities in a head-on comparison. It reads as follows:

And if you had to choose just one of the following alternatives, what would you say you see yourself as…? 1) [NATIONALITY] only, 2) [NATIONALITY] and European, 3) European and [NATIONALITY], 4) European only, 5) None of the above (Spontaneous).

The second question that we used attempts to measure European identity on a 0–10 point scale:

On a scale from 0 to 10, where ‘0’ is ‘not at all’, ‘10’ is ‘completely’, and ‘5’ is ‘exactly in the middle’, how much would you say you think of yourself as being European?

The same question wording is used for the measurement of national identity:

On a scale from 0 to 10, where ‘0’ is ‘not at all’, ‘10’ is ‘completely’, and ‘5’ is ‘exactly in the middle’, how much would you say you think of yourself as just being from [COUNTRY]?

Before analyzing the effects of deliberation treatment and testing our hypothesis, it is necessary to demonstrate that the sub-sample of participants is not substantially different from the population that is aiming to represent.

Although the self-selection process during the recruitment of participants (see Luskin et al., Reference Macedo2002) may cause some biases in the characteristics of this sub-sample (distorting therefore the results of the quasi-experiment) previous analyses demonstrated that ‘there is no dramatic over-representation of people of any particular political leaning among the participants in our experiment’ (Isernia and Fishkin, Reference Isernia, Fishkin, Steiner and Mauro2014: 321).

In Table 1, we show the results of the tests conducted to assess if the sample of participants substantially differ from non-participants (quasi-control group) and the control group on socio-demographic variables,Footnote 13 the level of knowledge and the questions about European identity. Only three of eight socio-demographic variables show a statistically significant difference at the 0.05 level. The sample of participants is slightly better educated than the group of non-participants but is not more educated than the control group. It also has more females than males, but the differences of the means between this group and non-participants are really small (0.06 and 0.08); and it is a slightly higher upper class than the control groups.Footnote 14

Table 1 Paired sample tests: participants, not participants, and control group

P1=participants in all the waves (N=329); P2=not participants in all the waves (N=2715); P3=participants in Time1 (N=348); P4=control group in Time1 (N=1305).

*P<0.05; ** P<0.005.

Variables measuring European identity show similar results. European identity as measured on a 0–10 point scale shows significant differences of means only between participants and non-participants, but this value is limited to approximately half a point (0.47). Still, this difference does not hold when participants are compared with the control group. Overall, the participants of EuroPolis feel only slightly more European than non-participants but the differences with the control group are either not significant (think of yourself as being European) or really small (European vs. national identity). Other variables concerning European integration confirm this picture, while the level of knowledge is the same in all compared groups.

Since the data confirmed that, in statistical terms, the sample of participants tends to be highly representative, it is now possible to address our main research question: does deliberation on European issues (within the arena composed by European citizens) affects European identity?

Empirical evidences: effects of deliberation on identity

Given the availability of the data for separate phases of deliberation we compare all the four waves (Time1 to Time4) collected for participants and the two waves that surveyed the control group. In this way we are able to explore the effects of all the phases of the DP and show which one had stronger influence on identity. At the Time1, about 3000 interviewed individuals were informed about EuroPolis and asked to participate. Only a sub-sample (348 individuals) of those interviewed participated to the quasi-experiment (see Isernia and Fishkin, Reference Isernia, Fishkin, Steiner and Mauro2014: 316). At Time2 participants arrived in Bruxelles and about half of the sample already read briefing materials (that were sent to them previously),Footnote 15 while at Time3 they participated in all the phases of the deliberative exercise. Time4 questionnaire, instead, was administrated to both participants and the control group after European Parliamentary elections (that were held few weeks after the DP).

Figure 1 shows the levels of European and national identity in the first question we used. The percentage of exclusive nationalist (participants choosing ‘Nationality only’) decreased from the 15% (Time1) to, respectively, 8, 7, and 7% in the successive waves (Time2 to Time4). The number of those who indicated first nationality and then Europe increased from 60% (Time1) to 65% (Time2), 70% (Time3) and 67% (Time4). The number of participants choosing European and Nationality increased only 1% point in the successive waves, while choice European only shows equal percentages (around 2.5%) in all, except for the Time2 wave (3.7%). In sum, the effect of deliberative quasi-experiment was strongest for the category of exclusive nationalist and between Time1 and Time2. The input that participants to the DP received between those two phases consisted on invitation to participate and the briefing materials.

Figure 1 National vs. European identity: participants in all the waves.

Moving to the second question, we grouped respondents in three categories according to their level of European identity.Footnote 16 Coherently with our first findings, the percentage of participants ranking at lower levels of European identity at Time1 decreased in all the other waves, while the number of those with higher sense of European identity grew after the deliberative quasi-experiment (Figure 2). The percentage of people in category high increased at Time3 (3% points) and Time4 (8% points). The opposite occurred with the percentage of the national identity identifiers. In particular, the percentage of participants that chose the highest levels of national identity decreased from 70% (Time1) to 56% (Time2), 54% (Time3), and 61% (Time4) while medium and low levels of national identity grew. Medium level of national identity grew about 9% points at Time2 and Time3 (Figure 3).

Figure 2 Percentages of European identity levels: participants in all the waves.

Figure 3 Percentages of National identity levels: participants in all the waves.

In a nutshell, a first descriptive analysis shows an increase of European identity of the participants, and even more strikingly, a decrease of national identity also vis-à-vis the European one. First of all, the quasi-experiment seems to affect the priority between national and European identity of the respondents (Figure 1), but the strongest effect is on the national identity, that decrease since Time2. Another interesting fact is that participants have quite consistent and similar attitudes in all purely deliberative phases of the quasi-experiment (Time2 and Time3). Participants seem to maintain the changes they made also at Time4 confirming that deliberation treatment, as operationalized in DP, can also have ‘medium-term’ effects.

We made statistical tests of participants’ attitudes, before, during, and after the event, by using paired comparison tests of the means in the four waves (Time1 through Time4). For each question we compared means at Time1 with the successive waves, for a total of three tests (Table 2). Moreover we run the same test between control group at Time1 and Time4.Footnote 17

Table 2 Paired tests of the means: participants at Time1 vs. following waves

*P<0.05; **P<0.005; ***P<0.001.

In the case of the first question, we observed a slight degree of change from Time1 through all the phases of the DP. Differences between the means are showed in Table 2. They are statistically significant only for Time1−Time2, showing that, in this case, the effect of deliberation is not significant.

The second question, on the contrary, suggest an effect of the deliberation process with a significant (P<0.05) increase in the mean values from Time1 to Time3 and Time4. According to these results, the increase in participants’ European identity is related to the deliberation process. At the same time, national identity shows a strong and significant decrease from Time1 to the other waves. National identity decreased on average of 0.9, 1.06, and 0.7, respectively, at Time2, Time3, and Time4 showing a higher effect after the discussion phase (Time3) and, on the whole, an influence of the quasi-experiment in all its phases.

Our analysis showed that there is a low and significant increase in the European identity and a stronger simultaneous decrease of the sense of national identity resulting from the DP quasi-experiment.

Our findings are confirmed further by the results of the analyses of the attitudes of the control group. As already mentioned they were surveyed at Time1 and Time4: that is, before the quasi-experiment and few weeks after participants took part in the DP. Therefore, the comparison in the case of control group refers to Time1 and Time4. Paired comparison tests show that respondents of the control group minimally change their sense of European identity. None of those changes are statistically significant for P<0.05 (Table 3), therefore confirming that changes we observed in the test group are to be ascribed to the quasi-experiment treatment, since the same changes did not occur within the control group.

Table 3 Paired tests of the means: control group at Time1 and Time4.

Differences of the means are not statistically significant (two-tailed).

The analyses of individual level changes offer a further insight about the effects of the quasi-experiment. Rather than considering the mean values, it shows the changes of sides at the individual level.Footnote 18 The changes related to the priority of identities reveal that 37, 33, and 34% of people, respectively, at Time2, Time3, and Time4 answered in a different way than at the beginning of the event, but only about 20% moved from one side to the opposite one (Table 4). As far as the other two questions (European and national identity) are concerned, a high percentage of respondents changed their positions. These changes refer to more than 60% of participants in the three comparisons showed (Table 4). More than 20% moved from a neutral position (value 5 ‘exactly in the middle’) to one of the two opposite directions. The main difference between the results of these two questions is that, accordingly with our previous findings, the strongest changes (i.e. the complete changes of side) are observed in the case of national identity. More precisely, a large majority of participants (above 64%) change their positions toward stronger European identity during the DP, but only 7% of them completely change sides, moving from the lowest sense of identity to the highest one and vice versa. The percentage of those who change sides is much higher in the case of national identity (14–16%), and we know, from the comparison at the aggregate level, that most of them moved toward a lower sense of national identity (Table 3). Once again the comparisons between participants’ attitudes at Time1 and the other waves of the quasi-experiment confirm that a major change occurred for national identity and between the Time1 and the other phases of the DP.

Table 4 Individual changes of participants T1 through T4

T1=Time1; T2=Time2; T3=Time3; T4=Time4.

A first look to our findings may lead to the conclusion that a competition between national and European identity exists and that the deliberative process exacerbates it. At the same time, many recent contributions showed the absence of a conflicting relationship between national and European identity (among other see Citrin and Sides, Reference Citrin and Sides2004; Risse, Reference Risse2004, Reference Risse2010; Bruter, Reference Bruter2005; Duchesne and Frognier, 2008). Aiming to understand better this relationship and to test if the national identity increases when the European one decreases and vice versa, we decided to analyze the possible existence of a negative relationship between the two forms of identities. Table 5 reports the odds of ordinal logit models where the degree of national identity (0–10) is the predictor and the European identity (‘national only; national and European; European and national, European only’) is the dependent variable. Models have been estimated for the four waves. Socio-demographic variables as gender, age, education, economic class self-positioning, and ideology were inserted as control variables.

Table 5 Ordinal logistic regression: national identity (predictor) – European identity

*P<0.1; **P<0.05; ***P<0.01; ****P<0.001.

The analyses show that national identity is significantly and negatively related to higher levels of European identity through the four waves. Figure 4 shows how the probability to choose one of the four categories of the dependent variable (‘national only; national and European; European and national, European only’) changes from Time1 to Time3.Footnote 19 The effect of the DP quasi-experiment therefore does not modify substantially the relationships observed before the event. At the same time, it is worth mentioning that the significant effect of weak national identity disappears at Time3 for ‘Nationality only’: that is, those who mentioned a value below 5 at Time3 have not significant different probabilities to choose Nationality only (Figure 4).Footnote 20 Once again an effect of the DP quasi-experiment on national identity is confirmed.

Figure 4 Probability changes for national identity (0–10) on national vs. European identity categories (categories: nationality only, nationality and European, European and nationality, European only) both at Time1 and Time3 (EuroPolis data set). Only significant values are showed.


The analyses presented in this paper give credibility to two major findings.

Before all, we showed that participation in deliberation process held between European citizens could strengthen citizens’ sense of European identity and simultaneously weaken their sense of national identity.

However, our first finding should be read with our second major finding in mind since it shows that the role of purely discussion phases of deliberative treatment (the activities between Time2 and Time3) is less important than expected. The most evident changes in identity (both national and European) occurred at the very beginning of the quasi-experiment, when structured discussion has not already taken place.

Before the discussion phases started, all participants were ‘plunged’ into the European environment. They became more aware of Europe and its policies, through the briefing material and informal discussion with other participants, they find themselves in the center of EU institutions, they freely interacted with people coming from all the EU countries and they surely felt privileged to be invited to discuss with and to be listen by EU representatives and stakeholders while being substantially hosted by the EU Commission (that financed the project).

Europe with its people, institutions, and policies became salient in minds of participants creating some sort of priming effect (Zaller, Reference Zaller1992) motivating therefore participants to adopt a more ‘European’ point of view. The priming effect could have been also more pronounced given the tendency of DP to promote cosmopolitan worldviews (Gastil et al., Reference Gerber, Bächtiger, Fiket, Steenbergen and Steiner2010). The validity of priming hypotheses seems to be confirmed by Fiket et al. (Reference Fishkin2014) in their study. They reported that participants often discussed about the purpose of EuroPolis and were aware of expectation that they should develop some kind of common identity (Fiket et al., Reference Fishkin2014: 67). And while we cannot test if the priming may be one of the mechanisms, integrated into ‘deliberative grand treatment’, responsible for the changes in identity, we certainly can say that during DP Europe became the real entity for the citizens involved. In that sense, the entitativity thesis (Castano, Reference Castano and Yzerbyt2004) holds truth.

These considerations speak to two audiences. First, to the scholars of European integration, our findings show that identity is not a given unchangeable construct and thus they have advanced the possibility of development of the European identity. Second, to the scholars of deliberative democracy, these findings underline the existence of the ‘community generating’ power of deliberation. Deliberative mini publics could be seen as useful tools for fostering European identity. Still, while our findings are quite optimistic, at least two limits should be considered: the scale problem and problem related to the persistence of the effects of deliberative treatment. If the deliberative mini publics remain isolated moments, without any strong link to the wider social system that hosts them, then their effects will remain limited only to the population that is involved in the specific deliberative practice. The integration of such innovative practices within a public sphere and political system – the scale problem – emerges in the core of the research in the field (Parkinson, Reference Parkinson2003; Mansbridge et al., Reference McLaren2013). The possible solutions cannot prescind from other limits related to the temporal dimension. There is a risk that the effects of deliberation on identity will be short term if the continuity of deliberative moments is not guaranteed. If identity is a social construct that is context dependent and therefore could be developed, when the stimulus that develops it is missing the identity could decrease again.

Both limits of deliberative exercises could be overcome by guaranteeing continuity and high visibility through information and dissemination tools. In this sense, the EU authorities are certainly on the right way: from the 2001 White Paper of governance to the Lisbon Treaty, EU institutions progressively developed and promoted institutional devices based on the deliberative model.


This paper originates from our work at the 7th Framework Program project EuroPolis–European deliberative polity-making project lead by the University of Siena. The authors are grateful to Center for the Studies of Political Change (CIRCaP) of the University of Siena and particularly to the project coordinator Pierangelo Isernia.

Financial Support

The research received no grants from public, commercial, or non-profit funding agency.


The replication dataset is available at


1 The most comprehensive study in terms of time period covered, conducted by Isernia et al. (Reference Isernia, Fiket, Serrichio and Westle2012) analyzes all publicly available survey indicators of citizens’ identification with Europe for the period between 1971 and 2007 and shows that European identity did not change significantly during the explored period, thereby confirming its stability. The study included 49 data sets: European Community Studies (1971 and 1973), standard EB (1976–2007), World Value Surveys (1981, 1990, 1999), and International Social Survey Programme (2003).

2 See the Special issue edited by Isernia and Fishkin (Reference Isernia, Fishkin, Steiner and Mauro2014).

3 We reported the results of the tests conducted to assess if the sample of participants in EuroPolis substantially differ from non-participants (quasi-control group) and the control group below.

4 Although this surely represents the problem of DP, it should be noted that it affects all mini publics since they need to operationalize idealized normative theory (Karpowitz and Mendelberg, 2011) transforming it in testable hypotheses. What heavily complicate the process of operationalization is that there is no definition of deliberation accepted by all theorists (Macedo, Reference Mansbridge1999). For this reason, when scholars talk about deliberation, they often refer to different phenomena. This problem does not occur only when we compare theoretical and empirical research of deliberation; it even occurs within these two areas. A variety of studies using empirical research have applied different research designs, different measurements, and different definitions of deliberation (Neblo, Reference Neblo2007).

5 The team of experts and stakeholders prepared a draft of briefing materials that aimed to provide participants with basic and balanced information about these two issues. It included alternative purposed policies about the issues, arguments about social costs and benefits, position of interest groups, and party families of the EU Parliament. Additionally, the first part of briefing materials described the institutional design of the European Union and its policy-making processes. The inclusiveness and accuracy of the briefing materials were assessed by two stakeholders’ committees (one for each issue). Experts that contributed to draft the briefing materials were also invited to attend EuroPolis event by participating in the plenary sessions.

6 Detailed description of EuroPolis can be found in Isernia and Fishkin (Reference Isernia, Fishkin, Steiner and Mauro2014) and Isernia et al. (Reference Joerges and Neyer2013).

7 First questionnaire (Time1 questionnaire) was filled in, by future participants to the DP, at the moment of recruitment.

8 The speakers at the final plenary session were the Estonian President Toomas Hendrik Ilves, the former Italian Prime Minister Giuliano Amato (in video conference), the former Danish MEP Jens Peter Bonde, and the former Belgian Deputy Prime Minister Isabelle Durant. Participants had the possibility to raise their questions on immigration, climate change, and EU decision-making to all speakers.

9 The presence of quasi-control and control groups represents the added value of EuroPolis DP because it allows for the major control of its internal validity.

10 The recruitment and first survey of European citizens started on 15 April 2009. It was conducted by TNS Opinion.

11 The sample has been stratified proportionally, according to the number of seats allocated to each Member State in the European Parliament.

12 See Isernia and Fishkin (Reference Isernia, Fishkin, Steiner and Mauro2014) for the description of both the sample selection and interview methods.

13 The number of cases is different since slightly changed during the waves of interview. All in all we considered 348 participants at Time1 and 329 answering all the waves. For a detailed discussion about the different types of people interviewed see Isernia et al. (Reference Joerges and Neyer2013).

14 Similar socio-demographic differences were also observed in other socio-political surveys.

15 ‘43% of the participants declared they had read it all before their arrival, and another 20% had read more than half of it, while only 6% said they had not read it at all’, Isernia and Fishkin (Reference Isernia, Fishkin, Steiner and Mauro2014: 316).

16 For both questions (on European and national identity) individuals were grouped into the three categories: low-level identifiers (answers from 0 to 3), medium-level identifiers (from 4 to 6), and high-level identifiers (from 7 to 10). Therefore, the original 11 points scale was recoded in three categories: 0–3; 4–6; 7–10.

17 This analysis considers the 329 (out of the 348) participants answering to all the waves of the survey (see Isernia et al., Reference Joerges and Neyer2013).

18 For an explanation of the difference between aggregate and individual changes see Luskin et al. (Reference Macedo2002).

19 Probability changes refer to the models without any other independent variables (i.e. only National identity from 0 to 10 and each dummy). The models are not aimed to describe the best predictors explaining the variations in the percentages of people mentioning each category, but rather to show how the relationships change when National Identity varies. Portions of lines not showed at Time3 are not significant. Scales are different for the first (Nationality only) and the last categories (European only) in order to demonstrate the shape of the curves. We show results only for Time1 and Time3 for synthesis: no notable differences are present for the other models.

20 Accordingly also values >5 for National identity do not show a significant relationship with ‘European only’ category.


Abels, G. (2009), ‘Citizens deliberation and EU democratic deficit: is there a model for participatory democracy?’. Tubinger Arbaitspapiere zur Integrationsforschung (TAIF), No. 1/2009, Tübingen, pp. 1–38.Google Scholar
Bardsley, N., Cubitt, R., Loomes, G., Moffat, P., Starmer, C. and Sugden, R. (2010), Experimental Economics. Rethinking the Rules, Princeton, NJ and Oxford: Princeton University Press.Google Scholar
Bernhagen, P. and Schmitt, H. (2014), ‘Deliberation, political knowledge and vote choice: results from an experiment with second-order elections’, European Union Politics 15(3): 352371.CrossRefGoogle Scholar
Bruter, M. (2005), Citizens of Europe? The Emergence of Mass European Identity, Basingstoke: Palgrave Macmillan.CrossRefGoogle Scholar
Campbell, D.T. and Stanley, J.C. (1963), Experimental and Quasi-Experimental Designs for Research, Chicago, IL: Rand McNally.Google Scholar
Castano, E. (2004), ‘European identity: a social psychological perspective’, in R.K. Herrmann, T. Risse and M.B. Brewer (eds), Transnational Identities. Becoming European in the EU, Lanham, MD: Rowman and Littlefield, pp. 4058.Google Scholar
Castano, E. and Yzerbyt, V. (1997), ‘Building a European identity’. Paper presented at the European Union Studies Association EUSA Biennial Conference, May 29–June 1, Seattle, WA.Google Scholar
Checkel, J.T. (2000), ‘Building new identities? Debating fundamental rights in European institutions’. ARENA WP 12/2000, Oslo.Google Scholar
Checkel, J.T. (2001), ‘Why comply? Social learning and European identity change’, International Organization 55(3): 553588.CrossRefGoogle Scholar
Checkel, J.T. and Katzenstein, P.J. (2009), European Identity, Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Citrin, J. and Sides, J. (2004), ‘More than nationals: how identity choice matters in the new Europe’, in R.K. Herrmann, T. Risse and M.B. Brewer (eds), Transnational Identities. Becoming European in the EU, Lanham, MD: Rowman and Littlefield, pp. 161185.Google Scholar
Cooke, M. (2000), ‘Five arguments for deliberative democracy’, Political Studies 48(5): 947969.CrossRefGoogle Scholar
De la Porte, C. and Nanz, P. (2004), ‘OMC – a deliberative-democratic mode of governance? The cases of employment and pensions’, Journal of European Public Policy 11(2): 267288.Google Scholar
Duchesne, S. and Frognier, A.-P. (1995), ‘Is there a European identity?’, in O. Niedermayer and R. Sinnott (eds), Public Opinion and Internationalized Governance, Oxford: Oxford University Press, pp. 193226.Google Scholar
Duchesne, S. and Frognier, A.-P. (2008), ‘National and European identifications: a dual relationship’, Comparative European Politics 6(2): 143168.CrossRefGoogle Scholar
Eriksen, E.O. (2006), ‘The EU – a cosmopolitan polity?’, Journal of European Public Policy 13(2): 252269.CrossRefGoogle Scholar
Eriksen, E.O. and Fossum, J.E. (2000), Democracy in the European Union – Integration Through Deliberation?, London: Routledge.Google Scholar
Farrar, C., Fishkin, J.S., Green, D.P., List, C., Luskin, R.C. and E.L. Paluck (2010), ‘Disaggregating deliberation’s effects: an experiment within a deliberative poll’, British Journal of Political Science 40(2): 333347.CrossRefGoogle Scholar
Fiket, I., Olsen, E.D.H. and Trenz, H.-J. (2014), ‘Confronting European diversity: deliberation in a transnational and pluri-lingual setting’, Javnost – The Public 21(2): 5773.CrossRefGoogle Scholar
Fishkin, J.S. (1997), The Voice of the People: Public Opinion and Democracy, New Haven, CT: Yale University Press.Google Scholar
Fishkin, J.S. (2009), When the People Speak: Deliberative Democracy and Public Consultation, New York: Oxford University Press.Google Scholar
Fishkin, J.S., Luskin, R.C. and Siu, A. (2014), ‘Europolis and the European public sphere: empirical explorations of a counterfactual ideal’, European Union Politics 15(3): 328351.CrossRefGoogle Scholar
Fligstein, N., Polyakova, A. and Sandholtz, W. (2012), ‘European integration, nationalism and European identity’, Journal of Common Market Studies 50(Suppl 1): 106122.CrossRefGoogle Scholar
Fung, A. (2003), ‘Recipes for public spheres: eight institutional design choices and their consequences’, The Journal of Political Philosophy 11(3): 338367.CrossRefGoogle Scholar
Gastil, J., Bacci, C. and Dollinger, M. (2010), ‘Is deliberation neutral? Patterns of attitude change during “the deliberative polls™”’, Journal of Public Deliberation 6(2): 133.Google Scholar
Gerber, M., Bächtiger, A., Fiket, I., Steenbergen, M. and Steiner, J. (2014), ‘Deliberative and non-deliberative persuasion: mechanisms of opinion formation in EuroPolis’, European Union Politics 15(3): 410429.CrossRefGoogle Scholar
Green, D.M. (2000), ‘On being European: the character and consequences of European identity’, in M.G. Cowles and M. Smith (eds), The State of the European Union, Risks, Reform, Resistance, and Revival , Oxford: Oxford University Press, pp. 292322.CrossRefGoogle Scholar
Hansen, K.M. and Andersen, V.N. (2004), ‘Deliberative democracy and the deliberative poll on the euro’, Scandinavian Political Studies 27(3): 261286.CrossRefGoogle Scholar
Herrmann, R.K. and Brewer, M.B. (2004), ‘Identities and institutions: becoming European in the EU’, in R.K. Herrmann, T. Risse and M.B. Brewer (eds), Transnational Identities. Becoming European in the EU, Lanham, MD: Rowman and Littlefield, pp. 122.Google Scholar
Herrmann, R.K., Risse, T. and Brewer, M.B. (2004), Transnational Identities. Becoming European in the EU, Lanham, MD: Rowman & Littlefield.Google Scholar
Hüller, T. (2010), ‘Playground or democratisation? New participatory procedures at the European Commission’, Swiss Political Science Review 16(1): 77107.CrossRefGoogle Scholar
Isernia, P., Fiket, I., Serrichio, F. and Westle, B. (2012), ‘But still it does not move. Functional vs identity based determinants of European identity’, in D. Sanders, P. Magalhães and G. Tóka (eds), Citizens and the European Polity: Mass Attitudes Towards the European and National Polities, Oxford: Oxford University Press, pp. 110139.CrossRefGoogle Scholar
Isernia, P. and Fishkin, J.S. (eds) (2014), ‘Special issue: can European elections be deliberative? The 2009 EuroPolis deliberative poll’, European Union Politics 15(3): 311327.Google Scholar
Isernia, P., Fishkin, J.S., Steiner, J. and Mauro, D.D. (2013), ‘Toward a European public sphere – the EuroPolis project’, in R. Kies and P. Nanz (eds), Is Europe Listening to Us? Successes and Failures of EU Citizen Consultations, Farnham: Ashgate, pp. 79124.Google Scholar
Joerges, C. and Neyer, J. (1997), ‘Transforming strategic interaction into deliberative problem-solving: European comitology in the foodstuffs sector’, Journal of European Public Policy 4(4): 609625.CrossRefGoogle Scholar
Karpowitz, C.F. and Mendelberg, T. (2011), ‘An experimental approach to citizens deliberation’, in J.N. Druckman, D.P. Green, J.H. Kuklinski and A. Lupia (eds), Handbook of Experimental Political Science, Cambridge, MA: Cambridge University Press, pp. 258272.CrossRefGoogle Scholar
Luskin, R.C., Fishkin, J.S. and Jowell, R. (2002), ‘Considered opinions: deliberative polling in Britain’, British Journal of Political Science 32: 455487.CrossRefGoogle Scholar
Macedo, S. (ed.) (1999), Deliberative Politics: Essays on Democracy and Disagreement, New York: Oxford University Press.Google Scholar
Mansbridge, J. (2010), ‘Deliberative polling as the gold standard’, The Good Society 19(1): 5562.CrossRefGoogle Scholar
Mansbridge, J., Bohman, J., Chambers, S., Christiano, T., Fung, A., Parkinson, J., Thompson, D.F. and Warren, M.E. (2013), ‘A systemic approach to deliberative democracy’, in J. Parkinson and J. Mansbridge (eds), Deliberative Systems: Deliberative Democracy at the Large Scale, Cambridge: Cambridge University Press, pp. 126.Google Scholar
McDermott, R. (2002), ‘Experimental methods in political science’, Annual Review of Political Science 5: 3161.CrossRefGoogle Scholar
McLaren, L.M. (2006), Identity, Interests and Attitudes to European Integration, Basingstoke: Palgrave Macmillan.CrossRefGoogle Scholar
Medrano, J.D. and Gutierrez, P. (2010), ‘Nested identities: national and European identities in Spain’, Ethnic and Racial Studies 24(5): 753778.CrossRefGoogle Scholar
Neblo, M. (2007), ‘Family disputes: diversity in defining and measuring deliberation’, Swiss Political Science Review 13(4): 527557.CrossRefGoogle Scholar
Neyer, J. (2006), ‘The deliberative turn in integration theory’, Journal of European Public Policy 13(5): 779791.CrossRefGoogle Scholar
Olsen, J.P. (2003), ‘What is a legitimate role for Euro-citizens?’, Comparative European Politics 1(1): 91110.CrossRefGoogle Scholar
Parkinson, J. (2003), ‘Legitimacy problems in deliberative democracy’, Political Studies 51(1): 180196.CrossRefGoogle Scholar
Risse, T. (2004), ‘European institutions and identity change: what have we learned?’, in R.K. Herrmann, T. Risse and M.B. Brewer (eds), Transnational Identities. Becoming European in the EU, Lanham, MD: Rowman and Littlefield, pp. 247271.Google Scholar
Risse, T. (2010), A Community of Europeans? Transnational Identities and Public Spheres, Ithaca, NY: Cornell University Press.Google Scholar
Sigalas, E. (2010), ‘Cross-border mobility and European identity: the effectiveness of intergroup contact during the ERASMUS year abroad’, European Union Politics 11(2): 241265.CrossRefGoogle Scholar
Siu, A. (2008), Look who’s talking: examining social influence, opinion change and argument quality in deliberation. Unpublished PhD thesis. Stanford, CA: Stanford University.Google Scholar
Trenz, H.-J. (2005), ‘The European public sphere: contradictory findings in a diverse research field’, European Political Science 4: 407420.CrossRefGoogle Scholar
Trenz, H.-J. (2009), ‘In search of European public sphere: between normative overstretch and empirical disenchantment’. RECON Online Working Paper 2008/07, Oslo.Google Scholar
Zaller, J.R. (1992), The Nature and Origins of Mass Opinion, Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Figure 0

Table 1 Paired sample tests: participants, not participants, and control group

Figure 1

Figure 1 National vs. European identity: participants in all the waves.

Figure 2

Figure 2 Percentages of European identity levels: participants in all the waves.

Figure 3

Figure 3 Percentages of National identity levels: participants in all the waves.

Figure 4

Table 2 Paired tests of the means: participants at Time1 vs. following waves

Figure 5

Table 3 Paired tests of the means: control group at Time1 and Time4.

Figure 6

Table 4 Individual changes of participants T1 through T4

Figure 7

Table 5 Ordinal logistic regression: national identity (predictor) – European identity

Figure 8

Figure 4 Probability changes for national identity (0–10) on national vs. European identity categories (categories: nationality only, nationality and European, European and nationality, European only) both at Time1 and Time3 (EuroPolis data set). Only significant values are showed.

Supplementary material: Link

Di Mauro Dataset