2.1 Majority versus minority participants and group status

Intergroup contact programs require participation of members from at least two adversarial groups. The program we worked with in Israel brings together youth from a majority (Jewish-Israelis) and a minority (Arab-Palestinians residing within Israel) group. Ideally, both groups should benefit from the program, although evidence suggests that minorities tend to benefit less, and are less likely to change their outgroup attitudes as a function of intergroup contact (Tropp and Pettigrew, Reference Tropp and Pettigrew2005). Creating a program that serves both groups equally in the field is complicated, as evidence suggests that members of different status groups have different priorities for intergroup encounters. Many civil-society interventions with a focus on commonalities (e.g., common humanity) reflect the preference of majority groups to be liked and accepted by the minority group (Dovidio et al., Reference Dovidio, Gaertner and Saguy2009). Meanwhile, members of the minority group tend to prefer for societal inequalities and injustices to be more directly addressed (Hässler et al., Reference Hässler, Ullrich, Sebben, Shnabel, Bernardino, Valdenegro, Van Laar, González, Visintin and Tropp2022). This dynamic leads us to expect the program to be less successful in changing the prejudice and ingroup-regulation behavior of Arab-Palestinian participants compared to Jewish-Israeli participants.

2.2 Intergroup contact outcomes

We measure prejudice through several measures of outgroup regard, but also assess ingroup regulation outcomes. In conflict-affected societies where everyday opportunities for intergroup contact are rare (Enos, Reference Enos2017), civil-society programs are often among the very few modes of intergroup contact. Despite their potential, these programs typically do not reach a significant share of a country's population. In addition to reducing prejudice, it is thus desirable that intergroup contact programs increase participants’ motivation to influence their non-participant ingroup peers (Ditlmann et al., Reference Ditlmann, Samii and Zeitzoff2017).

Influence strategies may include ingroup censuring, ingroup policing, persuasion of outgroup positive intent, and perspective sharing. Ingroup censuring (Ditlmann and Samii, Reference Ditlmann and Samii2016) and policing (Fearon and Laitin, Reference Fearon and Laitin1996) entail privately or publicly condemning an ingroup members’ aggression toward the outgroup. In terms of persuasion, people can convince their peers that outgroup members have benevolent instead of hostile intentions, thus disrupting hostile attribution biases (McGlothlin and Killen, Reference McGlothlin and Killen2006; van Dijk et al., Reference van Dijk, Thomaes, Poorthuis and Orobio de Castro2019), or share the perspective of the outgroup with their ingroup members (Bruneau and Saxe, Reference Bruneau and Saxe2012).

5.1 RCT design

For the RCT, our implementing partner recruited participants across grades four to six (typically ages 10–13) within three research sites (locations A, B, and C within Figure 1). The three sites for the RCT were not randomly selected. For this reason, in the analysis that fuses the RCT and survey data, we control for covariates that might be out of balance across the sites.

In a manner that is consistent with the way that the program generally operates, teachers at schools and community centers in the RCT sites publicized the program and collected applications. Our partner leads with sports in recruiting which reduces concerns over how selection into our study population affects generalizability. Even if there was outcome-relevant selection into our sample (e.g., participants’ parents being particularly open minded) an average effect of the treatment on the treated is still a reasonable estimand. Were this or a similar extracurricular program to be scaled up, youths would still have to opt into it, and hence the effects we estimate are for the subpopulation of youths who would plausibly choose to participate in the program. Once recruited into the RCT, applicants were randomly assigned to the program (treatment group) or put on a waiting list that would permit access in a subsequent season (control group). Youth in the treatment group were placed on the sports teams that then went through the season-long curriculum described earlier.

The teams formed in the RCT sites included youth who were granted access through the RCT random assignment, as well as youth outside the RCT participant pool. The latter came through channels that could not be randomized and were not included in the research (e.g., school programs, younger age cohorts from the same organization that advanced to the next level). As a result, each year only a few slots in the teams were available for randomization. In each location, we filled these slots over five seasons (2014–2018, see Figure 1), with new cohorts of RCT participants. Control group participants were often admitted into the program after one season for ethical reasons, which is why we could not track RCT cohorts beyond one season. Nonetheless, our non-experimental survey data, which we incorporate using a method discussed below, allow us to make predictions about the effects of multiyear exposure under three assumptions described in the next section.

In locations A and B, treatment assigned was at the individual level. In location C, assignment occurred at the 12-person group level, because the available participants were already grouped into teams by a cooperating sports club that could not be split up. Ensuring consistency with this randomization protocol, our analysis below accounts for potential clustering among the group-assigned participants in location C (Gerber and Green, Reference Gerber and Green2012). Table 1 shows compliance rates and attrition. Out of the youth assigned to the control condition, nine did not comply. This happened because coaches or teachers had a strong preference that certain children should be on a team, either because the child would benefit the team a lot (typically due to strong athletic skills) or the child would benefit a lot. For ethical reasons, we did not intervene to prevent such crossovers. We treat those who did not comply with the treatment as non-compliers, and those for whom we were not able to obtain outcome data as attrition. We conducted endline surveys with RCT-treated and control youth at the end of the season, typically about seven months after assignment.

Table 1.

Treatment assignment, compliance, and attrition

Table 2 shows the demographic composition of our RCT sample. While the distribution of basic demographic attributes is balanced between those assigned to treatment and control groups, the overall sample exhibits important idiosyncrasies (Appendix, Table 1). All Arab-Palestinian participants in our sample are female, largely a consequence of organization's deliberate efforts to improve Arab-Palestinian female engagement in sports. In our analyses of the RCT, as per our pre-analysis plan, we control for gender and location, which is perfectly predictive of ethnicity.

Table 2.

RCT sample descriptive statistics

5.2 General survey and fusion design

Each year we also carried out a general cross-sectional survey (Table 3) with participants from the broader population of program sites from which the RCT sites were selected. The general survey functions as a census of participants at sites with programming comparable to the RCT. The aim of the general survey was to assess how program exposure relates to outcomes beyond the first year, and it uses largely the same measures as the RCT surveys. The larger sample size also gave us information on heterogeneity across Arab-Palestinian and Jewish-Israeli youth.

Table 3.

Survey sample descriptive statistics

We “fuse” the RCT with observational data to estimate longer-term treatment effects, using a machine-learning method (random forest, Breiman, Reference Breiman2001). Our approach is most similar to that of Athey et al. (Reference Athey, Chetty and Imbens2020) and Imbens et al. (Reference Imbens, Kallus, Mao and Wang2022), who show how to combine RCT and observational samples to estimate effects that extend beyond the timeline of the RCT. The basic idea is to model the long-term outcomes in the observational data and then use that model to impute long-term outcomes for the RCT subjects. Athey et al. (Reference Athey, Chetty and Imbens2020) and Imbens et al. (Reference Imbens, Kallus, Mao and Wang2022) assume that the observational data have a panel structure, such that both short- and long-term outcomes are measured for each subject. Our situation is more restrictive in that our observational dataset consists of repeated cross sections. Thus, we cannot impute long-term outcomes for our RCT subjects using a model that relates outcome variables to each other over time. Rather, our imputations rely only on covariates. This is the key difference between our approach and that of Athey et al. (Reference Athey, Chetty and Imbens2020) and Imbens et al. (Reference Imbens, Kallus, Mao and Wang2022). We use a flexible random forest model as the basis of the imputations. As we describe below, we use a conservative bounding approach to address possibly endogenous attrition in the observational data. Our approach could be helpful for civil-society programs that lack the resources to collect longitudinal data, although we note that the conditions for causal identification are stronger than Athey et al. (Reference Athey, Chetty and Imbens2020) and Imbens et al. (Reference Imbens, Kallus, Mao and Wang2022) since our observational data do not contain direct information on how individuals’ outcomes evolve over time.

The longer-term effect that we target is the effect of continued participation in the program on individual's outcome trajectories as they progress through 10, 11, and 12 years of age. If youths remain in the program, then for each year they get older, they also accumulate another year in the program. In the RCT, we have youths of mixed ages that are either exposed to the program for one year or not at all; in the general survey, we have youths of mixed ages that are all exposed to the program, some for one year, others for two or three years. The fusion combines the information from the general survey and the RCT to make predictions about what would happen to RCT participants if they stayed in the program from the time they are 10 to the time they are 12, as compared to how their outcomes would evolve during those years without being in the program. We focus attention on 10–12 year olds because they are the majority participants in our RCT sample.Footnote ³ We use the covariate profile of the ten year olds to define a reference population. The fusion analysis was also specified in our pre-analysis plan.

More formally, let Y _p(t) denote an individual's outcome p periods since the onset of treatment eligibility given assignment to treatment condition t. The difference in outcome trends while under the program as compared to never being in the program is given by the following sequence:

$$\eqalign{& E[ Y_1( 1) \mid {\rm Age} = 10] - E[ Y_1( 0) \mid {\rm Age} = 10] \cr & E[ Y_2( 1) \mid {\rm Age} = 11] - E[ Y_2( 0) \mid {\rm Age} = 11] \cr & E[ Y_3( 1) \mid {\rm Age} = 12] - E[ Y_3( 0) \mid {\rm Age} = 12] ,\; }$$

where the expectations (E[ ⋅ ]) are with respect to our reference population (the ten-year-old RCT participants). Age and time in the program are perfectly collinear here, and so time-in-program and age-specific program effects cannot be disentangled. In the Appendix (Section 5), we show that there is no appreciable interaction effect with age in the RCT, which provides indirect evidence that the fusion analysis is picking up on effects from accumulated exposure. In any case, the analysis is meaningful in characterizing how trends would differ depending on whether a person participated in the program from age 10 to 12.

A model of the data-generating process (DGP) and identifying assumptions are needed to bring all of this information together to identify the trend effect. Figure 2 illustrates the DGP that justifies our approach. The variables X and U denote observed and unobserved, respectively, background characteristics of individuals. The variable T is for whether a person is assigned to treatment or control. The variables Y ₁ and Y ₂ are outcomes in a first period and a subsequent period, respectively, and S is an indicator for whether an individual stays in the program in the subsequent period. We use Y ₂ and S to characterize potential attrition biases, as described below.

Figure 2.

Directed acyclic graph showing the DGP for treatment assignment, treatment effects in year one, selection into programming in year two, and treatment effects in year two.

The first assumption that is implicit under this graph is that the treatment operates the same way regardless of whether the site is an RCT site or a general program site. That is, we assume that an RCT participant's potential outcome under treatment would have been the same if, counterfactually, their site was just part of the general program. All of our control observations come from RCT sites. The second assumption from the graph is that selection into an RCT site depends, systematically, only on observed characteristics (X). We justify these assumptions from the fact that RCT sites were selected for incidental reasons: they were sites where new cohorts were already planned on the basis of considerations that had been used to initiate programming more generally. At the same time, given the clustered nature of the selection into an RCT site (even though treatment assignment within RCT sites was mostly at the individual level), we expect some incidental imbalances in background characteristics across RCT versus general program sites. Hence, we control for background characteristics, which we chose based on past research and the suggestions of the implementing organization. These covariates are the program year and individuals’ gender, ethnicity, region of residence, religiosity, parents’ location of birth, and parents’ occupations.

Another key assumption is that, conditional on X, individuals in the different RCT control group cohorts can be used to construct the control trend (E[Y ₁(0)|Age = 10],  E[Y ₂(0)|Age = 11],  E[Y ₃(0)|Age = 12]). Consider the ten year olds that are in the control group in year one. Ideally we would observe their outcomes under control in years two and three to construct their cohort-specific control trend. However, we agreed with our implementation partner that individuals in the control group would have the opportunity to participate in subsequent years. Thus, we assume that, conditional on covariates, we can estimate this cohort-specific control trend by using outcomes from the 11-year-old control group members in year two, and 12-year-old control group members in year three.

Given these assumptions, for year one outcomes, we have that (Y ₁(1),  Y ₁(0))⊥T| X, in which case,

$$\eqalign{& E[ Y_1( 1) \mid {\rm Age} = 10] - E[ Y_1( 0) \mid {\rm Age} = 10] \cr & \quad = \int E[ Y_1 \mid T = 1,\; {\rm Age} = 10,\; X = x] dP( x \mid {\rm Age} = 10) \cr & \qquad- \int E[ Y_1 \mid T = 0,\; {\rm Age} = 10,\; X = x] dP( x \mid {\rm Age} = 10) ,\; }$$

where E[Y ₁| T = 1,  Age = 10,  X = x] can be estimated from the combined RCT treatment group and general survey data, E[Y ₁| T = 0,  Age = 10,  X = x] can be estimated from the RCT control group data, and the covariate distribution P(x|Age = 10) is derived from the RCT treatment and control group data.

Estimating effects beyond one year are complicated by the fact that some individuals drop out of the program each year, and this could occur for reasons that also affect outcomes in subsequent years. Figure 2 illustrates this possibility for outcomes in year two. (The same logic would apply for outcomes in year three.) We observe second-year-treated outcomes conditional on S, and so if we apply the method that we used for year one outcomes for year two outcomes, we would be estimating:

$$E[ Y_2( 1) \mid {\rm Age} = 11,\; S = 1] - E[ Y_2( 0) \mid {\rm Age} = 11] ,\;$$

where the outcomes for the “no drop-out” types E[Y ₂(1)|Age = 11,  S = 1] could differ from the overall mean E[Y ₂(1)|Age = 11] because S is a “collider” variable (Elwert and Winship, Reference Elwert and Winship2014). Now, if drop-out is endogenous only to T and X, then the approach that we used for year one outcomes would also work for year two (and year three) results, because the conditioning on T and X would remove any confounding. In Figure 2, this would imply that the gray-dashed arrows are not active, and our point estimates would be valid. However, if drop-out is endogenous to Y ₁ or U as well as T and X (i.e., the gray-dashed arrows are active), then our approach would not identify the relevant effects. Because our general survey data are repeated cross sections (administrative constraints and privacy concerns made a panel survey impossible), we cannot control for any Y ₁ variables. Even if we could, there may be unmeasured outcomes or background characteristics that confound the analysis.

From our discussions with program administrators, most of the year-to-year dropout was attributed to exogenous logistical factors, rather than participants deciding to leave because of a bad experience. Nonetheless, to be fully transparent, we use the trimming bounds of Lee (Reference Lee2009) to characterize “worst-case” consequences of endogenous attrition. We focus on an effect that pertains to “no drop-out” types in each period. Ideally, to estimate an effect for such “no drop-out” types, we would use an estimate of E[Y ₂(0)|Age = 10,  S = 1]. However, what we can estimate with our RCT control group data is

$$\eqalign{E[ Y_2( 0) \mid {\rm Age} = 11] & = E[ Y_2( 0) \mid {\rm Age} = 11,\; S = 1] P( S = 1 \mid {\rm Age} = 11) \cr & \quad + E[ Y_2( 0) \mid {\rm Age} = 11,\; S = 0] P( S = 0 \mid {\rm Age} = 11) ,\; }$$

which we can see is based on a mixture of individuals for which S would be one or zero were it that they, counterfactually, were in the general program. Considering that we do not have outcome data for survey participants that dropped out of the program after year one, what we can do is to estimate Lee bounds on E[Y ₂(0)|Age = 11,  S = 1] by trimming the upper and lower tails of the control group distribution by the retention rate P(S = 1|Age = 11). We estimate this rate using the general survey data. We also follow Lee (Reference Lee2009) in using covariates to tighten the bounds. The details are explained in the Appendix (Section 12).

5.3 Outcome measures

Corresponding to our analytical framework, we measure the effects of the intervention on outgroup regard and ingroup regulation. We operationalize them with several indicators, each consisting of batteries of items (Table 4). We use youth-appropriate and validated measures of outgroup regard, as well as create new ingroup regulation measures building on the existing literature: tendency for ingroup censuring, ingroup policing, peer persuasion, and perspective sharing. Cross-sectional survey questions are largely identical to RCT questions, although they also included descriptive measures capturing program experience and additional operationalizations of our outcomes. Survey measures were identical for all participants irrespective of their ethnic membership with the exception of the measure of perspective sharing tendencies, since the perspective each group might share differs.

Table 4.

RCT and annual surveys measures

Given the contentiousness of the political situation and the fact that all participants in our sample are minors, measuring actual behavior on our outcomes of interest is particularly challenging. Our measures thus capture self-reported behavioral intentions as proxies for the actual behavior. We form the outgroup regard index using principal component analysis because we consider the seven different indicators to be different operationalizations of the same underlying construct. Our approach for the ingroup regulation index is different, because we are interested in a series of qualitatively different strategies that participants can employ. As such, the index of Ingroup Regulation is formed as a sum of Z-scores, treating all indicators equally.