2.1 The problem

Consider the graph in Figure 1. For concreteness, and previewing one of our applications, let D be the treatment assignment for a community policing program, M be some measure of program implementation and Y be an outcome of interest, such as crime rates. If we were interested in the effect of the program on the outcome, it would be natural to compare the outcomes in the treated group ( $D=1$ ) with the outcomes in the control group ( $D = 0$ ). However, the treated group potentially includes communities that did not actually implement the program, likely attenuating the treatment effect estimate.

Figure 1

Causal structure of concern. A treatment (D), which is unconfounded with the outcome of interest (Y) may affect M in some sub-group. While some confounders of M and Y may be measured (X), we cannot rule out the presence of unobserved common causes of M and Y (U).

A natural temptation is to look at only the group that implemented the program effectively, that is, conditioning on M. This is problematic; first, since M is post-treatment, conditioning on it blocks part of the effect of D on Y. Second, it introduces a bias. We cannot typically rule out unobserved confounders of M and Y (shown as U). Because M is a consequence of both D and U, it is a collider (Pearl Reference Pearl2009), and conditioning on it can create an association between D and U, leading D and Y to become associated (via $D \leftarrow U \rightarrow Y$ ) for reasons other than the causal effect of D.

To avoid confusion when comparing this approach to IV below, note that the estimand of interest here is the effect of the treatment (D) on the outcome (Y) within a sub-group related to M. While the causal structure resembles that assumed in the IV framework, in that setting the effect of interest is that of M on Y, while D plays the role of instrument. That is, the TE in our setting corresponds to the intent-to-treat or reduced-form effect in the analogous IV setting.

2.2 Examples

This arrangement appears in a wide variety of settings we organize into four broad types, all sharing the causal structure shown in Figure 1.

Type 1. Manipulation, attention and implementation checks: In these cases, M represents a variable that, while not necessary for an effect of D on Y, is thought to be of central importance for this effect. This encompasses our motivating example, where realization of the program on the ground (M) is thought to be important for its effect on Y. Other examples include cases where M measures participants’ compliance with a protocol, or responses to manipulation checks or attention screens performed after the treatment.Footnote ²

Type 2. Mechanism of interest: Here, M may not be the primary or only way for D to affect Y, but it is of theoretical interest as part of a mechanism under investigation. For example, in our analysis of Broockman and Kalla (Reference Broockman and Kalla2016) below, we wish to examine the effect of a perspective-taking intervention on support for transgender non-discrimination policies among participants who react to the intervention with increased warmth toward transgender individuals. This setting resembles questions in the mediation literature regarding direct versus indirect effects, but as we describe in Section 3.6, the TRACE engages a different causal question. Our approach also avoids problematic assumptions regarding the absence of mediator–outcome confounders, albeit at the cost of partial rather than point identification.

Type 3. “Necessary condition” mediators: In this setting, M is a necessary condition for Y to possibly occur (equal 1). An example can be found in studies of the effect of a driver’s race (D), as perceived by police, on the potential for police violence (Y) during a traffic stop (M). The chance of a traffic stop occurring ( $M=1$ ) may be itself a function of police-perceived race. Y can be defined specifically as “police violence during a traffic stop,” ensuring that no police violence of this kind can occur without a stop, that is, $Y=0$ if $M=0$ . Although this is merely an extreme case of M being a mediator, we give it a separate category because the necessity of $M=1$ for $Pr(Y=1)>0$ can lead to point identification, as detailed below.

Type 4. Treatment-responsive measurement of M: In another setting, we may wish to know the effect of D on Y among those with some value of M, where measurement of M is affected by D. For example, suppose we wish to know the effect of cancer screening on survival, among those who would have received a positive result for cancer in that screening (ruling out always-takers). The result here depends on our ability to reason about the effect of screening on survival for patients who, if screened, would have screened negative, on its own or in comparison to that in the group who would screen positive. This engages a complicated set of concerns about the effects of screening on behaviors, the false negative rate, potential harms due to false positives and other challenges that we examine elsewhere.

3.1 Setup and notation

Continuing with the notation defined above, let $Y_i(d, m)$ denote the potential value of Y for unit i with treatment status $D_i=d$ and mediator value $M_i=m$ . Let $M_i(d)$ denote the “potential mediator,” the value of M for unit i under treatment status d (Imai et al. Reference Imai, Keele, Tingley and Yamamoto2014; Imai, Keele, and Yamamoto Reference Imai, Keele and Yamamoto2010). By writing potential outcomes, we assume consistency for both the mediator and the outcome: when $D_i=d$ and $M_i=m$ are observed, the observed outcome satisfies $Y_i = Y_i(d, m)$ and the observed mediator value satisfies $M_i = M_i(d)$ .

In the following analysis, we need only consider two potential outcomes for Y: $Y_i(0, M_i(0))$ and $Y_i(1, M_i(1))$ . Accordingly, we define $Y_i(d) \equiv Y_i(d, M_i(d))$ , so that the joint counterfactual need not be written explicitly when the mediator takes its natural value under treatment d. Thus, we can simply write the TE as

(1) $$ \begin{align} TE \equiv \mathbb{E}[Y_i(1) - Y_i(0)]. \end{align} $$

When it is necessary to consider mediator values other than $M_i(d)$ , we explicitly write both arguments as $Y_i(d, m)$ . This is needed only where we describe mediation-related estimands to contrast them with our approach (see Section 3.6, Table 1).

3.2 Defining the TRACE

Our target quantity of interest is a TE of D on Y, averaged over only units that, if treated, would have $M=1$ Footnote ³ :

(2) $$ \begin{align} \text{TRACE} &\equiv \mathbb{E}[Y_i(1) - Y_i(0) \mid M_i(1)=1] \end{align} $$

(3) $$ \begin{align} &= TE[M_i(1)=1].\qquad \end{align} $$

We also define the TE among the units that, if treated, would have $M=0$ , which we call TRACE(0):

(4) $$ \begin{align} \text{TRACE(0)} &\equiv \mathbb{E}[Y_i(1) - Y_i(0) \mid M_i(1)=0] \end{align} $$

(5) $$ \begin{align} &= TE[M_i(1)=0].\quad \end{align} $$

3.3 Partial identification of the TRACE

Identification of the TRACE is complicated by M’s status as a post-treatment variable, combined with possible unobserved common cause confounders of M and Y. If investigators believed all common causes of M and Y were observed (i.e., there was no U on Figure 1), it would be possible to identify the effect of D on Y conditioning on M, because all the paths opened by this conditioning could be closed again.Footnote ⁴ However, we consider this unrealistic in many settings, and of no interest for the problems on which we focus. For example, if D is a community-driven development program, and M is an indicator of whether a project chosen by the community was actually built, one expects many unobservable features to be common causes both of project implementation and the outcomes of interest Y. We therefore focus on the case in which such $M-Y$ confounding cannot be ruled out.

Table 1

Comparison of different estimands for settings in which post-treatment variables are relevant

Table 1 Long description

The table has four columns labeled Estimand, Definition, Among whom, and Identification assumptions. The first row under Estimand is LATE (I V) with definition double struck E left bracket Y open parenthesis 1 close parenthesis minus Y open parenthesis 0 close parenthesis vertical bar M open parenthesis 0 close parenthesis equals 0 comma M open parenthesis 1 close parenthesis equals 1 close bracket, applies to Compliers, and requires relevance, exogeneity, and exclusion restriction. The second row is SACE with definition double struck E left bracket Y open parenthesis 1 close parenthesis minus Y open parenthesis 0 close parenthesis vertical bar M open parenthesis 0 close parenthesis equals M open parenthesis 1 close parenthesis equals 1 close bracket, applies to Always-takers, and requires monotonicity, M open parenthesis 1 close parenthesis up tack up tack M open parenthesis 0 close parenthesis, double struck E left bracket Y open parenthesis 1 close parenthesis vertical bar AT close bracket equals double struck E left bracket Y open parenthesis 1 close parenthesis vertical bar C close bracket, double struck E left bracket Y open parenthesis 0 close parenthesis vertical bar AT close bracket equals double struck E left bracket Y open parenthesis 0 close parenthesis vertical bar Def close bracket, given X. The third row is CDE open parenthesis m close parenthesis with definition double struck E left bracket Y open parenthesis D equals 1 comma M equals m close parenthesis minus Y open parenthesis D equals 0 comma M equals m close parenthesis close bracket, applies to All units, and requires unconfoundedness of M minus Y. The fourth row is TRACE with definition double struck E left bracket Y open parenthesis 1 close parenthesis minus Y open parenthesis 0 close parenthesis vertical bar M open parenthesis 1 close parenthesis equals 1 close bracket, applies to Compliers and always-takers, and is identified by postulated TRACE open parenthesis 0 close parenthesis. The table highlights differences in target populations and identification assumptions across estimands. Table footnote defines AT as Always-taker, C as Complier, and Def as Defier.

Note: AT: Always-taker; C: Complier; Def: Defier.

3.3.1 No-assumption trimming bounds

We begin by establishing bounds on the TRACE that rely only on assumptions standard in a randomized experimental setting. The first component of the TRACE, $\mathbb {E}[Y(1) \mid M(1)=1]$ , is directly identifiable. Among treated units, the observed outcome equals $Y(1)$ and those with $M=1$ are precisely those with $M(1)=1$ , both by consistency. Hence, we can identify $\mathbb {E}[Y(1) \mid M(1)=1, D=1]$ , which under ignorability of D yields $\mathbb {E}[Y(1) \mid M(1)=1]$ . We denote the corresponding sample estimator as $\overline {Y(1)}_{M(1)=1}$ .

The second component, $\mathbb {E}[Y(0) \mid M(1)=1]$ , presents the difficulty. To observe $Y(0),$ we must examine units with $D=0$ , but among these units we do not observe $M(1)$ and therefore cannot identify units with $M(1)=1$ . Nevertheless, we can identify and estimate $\Pr (M(1)=1)$ as $\widehat {\Pr }(M=1 \mid D=1)$ . This allows us to place sharp bounds on $\mathbb {E}[Y(0) \mid M(1)=1]$ . The lowest (highest) possible value of this mean, $\overline {Y(0)}_{\text {low}}$ ( $\overline {Y(0)}_{\text {high}}$ ), is the average over the lowest (highest) $\widehat {\Pr }(M=1 \mid D=1)$ fraction of the control outcome distribution. These bounds imply corresponding bounds on the TRACE estimate in a sample:

(6) $$ \begin{align} \widehat{\text{TRACE}}_{\text{low}} &= \overline{Y(1)}_{M(1)=1}-\overline{Y(0)}_{\text{high}}, \end{align} $$

(7) $$ \begin{align} \!\!\!\!\widehat{\text{TRACE}}_{\text{high}} &= \overline{Y(1)}_{M(1)=1} - \overline{Y(0)}_{\text{low}}. \end{align} $$

This bound is sharp: values as low as $\text {TRACE}_{\text {low}}$ and as high as $\text {TRACE}_{\text {high}}$ (inclusive) can be obtained with the observed data.Footnote ⁵

3.3.2 Leveraging assumptions on TRACE(0)

While the no-assumption bounds provide a valuable starting point, they often will not be independently informative. The core of our approach focuses on an additional source of leverage investigators may have: defensible assumptions about how large the treatment effect could be for those who would have $M=0$ if treated. By the law of iterated expectations,

(8) $$ \begin{align} TE &= \text{TRACE} \cdot Pr(M(1)=1) + \text{TRACE}(0) \cdot Pr(M(1)=0) \end{align} $$

(9) $$ \begin{align} \text{TRACE} &= \frac{TE - \text{TRACE(0)}\cdot Pr(M(1)=0)}{Pr(M(1)=1)},\quad\qquad\qquad \end{align} $$

where we suppress the i index for quantities representing group proportions (e.g., $Pr(M(1)=1)$ ) to simplify notation. Given randomization of D, the TE is identifiable, as are $Pr(M(1)=1)$ and ${Pr(M(1)=0)}$ :

(10) $$ \begin{align} Pr(M(1)=m) &= Pr(M(1)=m \mid D=1) \quad \text{randomization of } D \end{align} $$

(11) $$ \begin{align} &\ \ \,= Pr(M=m \mid D=1) \quad \text{consistency of } M(d). \end{align} $$

The only unknown on the right-hand side of Expression (9) is TRACE(0). Thus, for any postulated range of TRACE(0) values, we can recover the implied range of TRACE values. The resulting range will be narrower when $Pr(M(1)=0)$ is smaller, that is, when the fraction of “implementers” is higher. Additionally, the entire procedure can be done conditioning on values of X.

3.4 Sharpness of combined bounds

The approach we recommend and illustrate below presents both the no-assumption trimming bounds and the partial identification results obtained for any postulated range of TRACE(0), highlighting the region consistent with both. This ensures that assumptions about TRACE(0) can provide identification leverage where possible, while results remain sharp by simultaneously enforcing the no-assumption trimming bounds.Footnote ⁶

To see that the TRACE(0)-based bounds are sharp when they do not violate the trimming bounds, note first that TRACE values are algebraically determined by (i) the postulated value of TRACE(0) and (ii) two quantities identified from the data ( $TE$ and $\Pr (M(1)=1)$ ). TRACE values generated in this way are thus compatible with the data, as long as the assumed TRACE(0) is not itself inconsistent with the data. This condition could be guaranteed by imposing trimming bounds on TRACE(0), analogous to those derived for the TRACE above. However, this is unnecessary because these TRACE(0) trimming bounds are precisely those that imply the TRACE trimming bounds. This follows directly from the algebraic relationship linking the TRACE and TRACE(0), a result which we verify in Appendix D. Consequently, imposing the no-assumption trimming bounds on the TRACE and combining them with bounds obtained under assumptions on TRACE(0) yields a sharp partial identification region.

3.5 Related estimation approaches

Several common estimation strategies are worth noting for their widespread use, regardless of whether they target a well-defined causal estimand in this setting. As already noted, one of these is post-treatment conditional estimation: comparing mean outcomes for treated and untreated units with observed $M = 1$ . We compare this analytically to the TRACE in Section 3.6 below. However, several other practices merit comment.

• • The intention-to-treat (ITT) approach evaluates causal effects based on the assigned treatment when not all participants adhere to treatment assignment. Our proposal essentially formalizes the intuition that the ITT may be “diluted,” then explores the “un-diluted” values implied by varying assumptions on TRACE(0).

• • “As-treated” analysis estimates the effect of the realized treatment rather than the assigned treatment, reclassifying units according to what they actually received regardless of assignment. This estimate will suffer bias from confounding if characteristics that moved a participant to treatment implementation are associated with characteristics that affect a participant’s outcome.

• • “Per-protocol” analyses instead retain the original treatment arms but restrict each to participants who adhered to their assigned protocol—retaining those with $M = 1$ in the treated group and ${M = 0}$ in the control group. This conditions on a post-treatment variable in both arms, breaking the randomization.

Finally, a related practical approach is to model $M(1)$ for all units based on available (pre-treatment) covariates. One can then condition on the estimated $\widehat {M}(1)$ (see, e.g., Vanacore et al. Reference Vanacore, Gurung, Sales and Heffernan2024). This characterizes the TE across groups defined by their pre-treatment covariates. The predicted $\widehat {M}(1)$ is at best an imperfect proxy for $M(1)$ ; nevertheless, this approach may prove valuable when it is less important to precisely learn the TRACE but desirable to have some directional estimate of how it changes as a function of relevant pre-treatment covariates.

3.6 Comparison to related estimands

Several existing estimands address queries related to post-treatment variables, but differ from the TRACE in the groups they include and the identification opportunities they present. In making these comparisons, it is helpful to follow the structure of the principal stratification framework (Frangakis and Rubin Reference Frangakis and Rubin2002). Specifically, imagine that all units fall into one of four possible strata: compliers ( $M(1) = 1, M(0) = 0$ ), defiers ( $M(1) = 0, M(0) = 1$ ), always-takers ( $M(1) = 1, M(0) = 1$ ) and never-takers ( $M(1) = 0, M(0) = 0$ ). The TRACE is the average treatment effect for always-takers and compliers, while TRACE(0) is the average effect over never-takers and (if present) defiers.

Table 1 compares related estimands. Though ours is unique (excepting equivalences in special cases), the primary emphasis of our contribution lies in the relatively straightforward partial identification strategy it allows.

3.6.1 Comparison to instrumental variables

The LATE, also called the complier average causal effect (Baker and Lindeman Reference Baker and Lindeman1994; Imbens and Angrist Reference Imbens and Angrist1994), is

(12) $$ \begin{align} \mathbb{E}[Y_i(1) - Y_i(0) \mid M_i(0) = 0, M_i(1) = 1]. \end{align} $$

This estimand differs from the TRACE in two primary ways.Footnote ⁷ First, they differ definitionally in that the LATE considers the average effect among compliers, while the TRACE considers the average effect among compliers and always-takers. These estimands become equivalent when we can rule out always-takers (through one-way non-compliance), or when effects for compliers and always-takers are the same. Second, the LATE is the target estimand for the IV approach, which for identification requires that D only affects the outcome through M, that is, there is no direct effect of D on Y. The $D \to Y$ path violates this core IV assumption and is an important contributor to the TE we are seeking to learn (see Figure 1).Footnote ⁸

Consider, for example, cases where the investigator argues that TRACE(0) = 0. This may occur in “Type 3” settings where it is impossible for D to affect Y when $M = 0$ , and we can further argue either for monotonicity (no defiers) or for no effect among defiers. The assumption of “no effect of D on Y when $M = 0$ ” may sound similar to the exclusion restriction, but they differ in a key way: the exclusion restriction requires the total absence of a direct $D \to Y$ effect, while TRACE(0) = 0 requires no direct effect only for those with $M(1) = 0$ (never-takers and, if present, defiers). In some cases, a direct $D \to Y$ effect in the “reactive” subgroup is not only expected, but is precisely our quantity of interest. In the police violence setting discussed below, for example, we wish to study—not rule out—a possible effect of police-perceived race (D) on the use of violence (Y) when a stop occurs ( $M = 1$ ).

3.6.2 Comparison to survivor average causal effect

The SACE arises in settings where the outcome is censored due to a post-treatment factor, such as survival (Egleston, Scharfstein, and MacKenzie Reference Egleston, Scharfstein and MacKenzie2009; Hayden, Pauler, and Schoenfeld Reference Hayden, Pauler and Schoenfeld2005; Imai Reference Imai2008; Robins Reference Robins1986; Rubin Reference Rubin2000; Tchetgen Tchetgen Reference Tchetgen Tchetgen2014; Zhang and Rubin Reference Zhang and Rubin2003) or sample selection/attrition (Lee Reference Lee2009; Semenova Reference Semenova2025). The SACE is defined as

(13) $$ \begin{align} \text{SACE} = \mathbb{E}[Y_i(1) - Y_i(0) \mid M_i(0) = 1, M_i(1) = 1]. \end{align} $$

It considers only the effect among always-takers, that is, “always-survivors” when $M = 1$ is survival to endline. Because we cannot observe which units are always-takers, point identification strategies rely on assumptions that ensure mean values of $Y_i(1)$ or $Y_i(0)$ do not differ between certain principal strata.

The TRACE differs from the SACE only by including compliers. The survival settings in which the SACE originates can introduce complications for our approach by rendering the TE unidentified. As discussed in Section 4.1, this can sometimes be addressed, particularly in settings where M is a “necessary condition mediator” and D is assumed to be monotonic in M. Outside of the survival setting, the SACE is relevant when reinterpreted as the average effect among always-takers, that is, where M does not affect loss to follow-up (see, e.g., Hudgens and Halloran Reference Hudgens and Elizabeth Halloran2006). Existing work on partial identification and trimming bounds for the SACE has proposed similar bounds for $\mathbb {E}[Y(d) \mid AT]$ (Horowitz and Manski Reference Horowitz and Manski1995; Imai Reference Imai2008; Lee Reference Lee2009; Samii et al. Reference Samii, Wang and Zhou2025; Semenova Reference Semenova2025; Zhang and Rubin Reference Zhang and Rubin2003). We discuss similar bounds for the TRACE in Section 3.7.

3.6.4 Comparison to post-treatment conditional estimation

While the problems with estimating effects conditional on $M = 1$ are well known (see, e.g., Montgomery et al. Reference Montgomery, Nyhan and Torres2018), comparing this approach to the TRACE can be instructive. Defining $\text {DIM}_{M=1} \equiv \mathbb {E}[Y \mid M = 1, D = 1] - \mathbb {E}[Y \mid M = 1, D = 0]$ , we can show that under monotonicity of M in D,Footnote ⁹

(14) $$ \begin{align} \text{TRACE} - \text{DIM}_{M=1} = \left(\mathbb{E}[Y_i(0) \mid AT] - \mathbb{E}[Y_i(0) \mid C]\right) \left(\frac{\Pr(C)}{\Pr(AT) + \Pr(C)}\right). \end{align} $$

In “Type 3” cases (where $M = 0$ guarantees $Y = 0$ ), $Y(0) = 0$ among compliers, reducing this to

(15) $$ \begin{align} \text{TRACE} - \text{DIM}_{M=1} = \mathbb{E}[Y_i(0) \mid AT] \left(\frac{\Pr(C)}{\Pr(AT) + \Pr(C)}\right), \end{align} $$

which for non-negative outcomes implies TRACE $\geq $ DIM $_{M=1}$ .

3.7 Monotonicity-based trimming bounds

A potentially tighter trimming bound is possible when (i) M is observed for all units and (ii) the user is willing to assume monotonicity ( $M(1) \geq M(0)$ , i.e., no defiers). The problematic quantity for identification, $\mathbb {E}[Y(0) \mid M(1)=1]$ , then consists of only always-takers (AT) and compliers (C). Specifically, letting $\alpha \equiv \Pr (AT \mid M(1)=1)$ ,

(16) $$ \begin{align} \mathbb{E}[Y_i(0)\mid M_i(1)=1] &= \mathbb{E}[Y_i(0)\mid AT] \alpha + \mathbb{E}[Y_i(0)\mid C] (1-\alpha). \end{align} $$

Under monotonicity, untreated units with $M=1$ are necessarily always-takers, so $\mathbb {E}[Y(0)\mid AT]$ is identified as $\mathbb {E}[Y \mid D=0, M=1]$ . Further, $\alpha $ is identified as $\frac {\Pr (M=1 \mid D=0)}{\Pr (M=1 \mid D=1)}$ , with sample estimator $\hat {\alpha }$ . The unidentified portion of $\mathbb {E}[Y(0)\mid M(1)=1]$ is solely due to $\mathbb {E}[Y(0)\mid C]$ , which we can bound based on the distribution of outcomes for units with $D=0$ and $M=0$ . This group includes compliers and never-takers (NT), with an identified share $\pi $ being compliers:

$$\begin{align*}\pi \equiv \frac{\Pr(C)}{\Pr(C)+\Pr(NT)} = \frac{\Pr(M=1\mid D=1)-\Pr(M=1\mid D=0)}{\Pr(M=1\mid D=1)-\Pr(M=1\mid D=0) + \Pr(M=0\mid D=1) }. \end{align*}$$

We can then construct trimming bounds $\widehat {\mathbb {E}}[Y(0)\mid C]_{\text {low}}$ and $\widehat {\mathbb {E}}[Y(0)\mid C]_{\text {high}}$ as the means of the lowest and highest $\hat {\pi }$ fractions of outcomes in this group. These yield monotonicity-based trimming (MT) bounds on $\widehat {\mathbb {E}}[Y(0) \mid M(1)=1]$ of

(17) $$ \begin{align} \widehat{\mathbb{E}}[Y_i(0) \mid M_i(1)=1]_{\text{low}}^{\text{MT}} &= \widehat{\mathbb{E}}[Y_i(0)\mid AT]\hat{\alpha} + \widehat{\mathbb{E}}[Y_i(0)\mid C]_{\text{low}}(1-\hat{\alpha}), \end{align} $$

(18) $$ \begin{align} \widehat{\mathbb{E}}[Y_i(0) \mid M_i(1)=1]_{\text{high}}^{\text{MT}} &= \widehat{\mathbb{E}}[Y_i(0)\mid AT]\hat{\alpha} + \widehat{\mathbb{E}}[Y_i(0)\mid C]_{\text{high}}(1-\hat{\alpha}), \end{align} $$

which finally yields the MT bounds on TRACE:

(19) $$ \begin{align} \widehat{\text{TRACE}}_{\text{low}}^{\text{MT}} &= \overline{Y(1)}_{M(1)=1} - \widehat{\mathbb{E}}[Y_i(0) \mid M_i(1)=1]_{\text{high}}^{\text{MT}}, \end{align} $$

(20) $$ \begin{align} \widehat{\text{TRACE}}_{\text{high}}^{\text{MT}} &= \overline{Y(1)}_{M(1)=1} - \widehat{\mathbb{E}}[Y_i(0) \mid M_i(1)=1]_{\text{low}}^{\text{MT}}. \end{align} $$

4.1 Hypothetical demonstration: Perceived race, police stops and violence

Here, we demonstrate the use of this approach for understanding the influence of race on police behavior, a setting in which selection into administrative records data has made causal inference challenging. Consider a scenario in which police officers observe drivers and have a perception (accurate or not) of each driver’s race. Let $D=1$ indicate that the police perceived the driver to be from a minority group. The police may then choose to stop the vehicle (M). During that stop, police violence may or may not occur (Y). Only encounters that result in stops appear in police administrative data. Thus, analyses using only these data automatically condition on $M=1$ , leading to post-treatment bias when comparing rates of police violence by driver race among recorded stops (Knox et al. Reference Knox, Lowe and Mummolo2020).Footnote ¹⁰

Because Y is defined specifically as “violence during a police stop,” $Y=0$ when no stop occurs ( $M=0$ ) by construction, making this an example of a “necessary condition mediator” (Type 3). Recall that TRACE(0) is the TE among never-takers (units with $\{M(1)=0,M(0)=0\})$ and defiers (units with $\{M(1)=0, M(0)=1 \}$ ). Suppose we can rule out defiers, arguing that if a person is not stopped when perceived to be a minority, they would not be stopped when perceived to be non-minority. Consequently, the $M(1)=0$ group would include only never-takers. Because this group is never stopped regardless of perceived race, it follows that $Y(1)=Y(0)=0$ . Thus, the TE among never-takers is deterministically zero, implying $TRACE(0)=0$ , and point identifying the TRACE.Footnote ¹¹

Estimation requires only the sample moments needed to estimate (i) the TE and (ii) the proportion with $M(1)=1$ . The former requires knowing the share of encounters with perceived-minority drivers that result in violence ( $Pr(Y=1 \mid D=1)$ ), as well as the corresponding share for perceived-non-minority drivers ( $Pr(Y=1 \mid D=0)$ ). This may not be directly available, but could be backed out from alternative combinations of information, such as the fraction of cases with violence involving perceived minority drivers ( $Pr(D=d \mid Y=1)$ ), the overall proportion of police-perceived minority drivers ( $Pr(D=1)$ ) in the relevant population and the overall proportion of encounters resulting in violence ( $Pr(Y=1)$ ). However, estimating the second quantity, $Pr(M(1) = 1)$ , requires data on the total number of encounters with perceived minority drivers, in addition to data on the number of minority stops. Knox et al. (Reference Knox, Lowe and Mummolo2020) similarly conclude that the TE can be point identified in this setting given data on the total number of perceived minority and non-minority encounters.

Finally, investigators can directly estimate $\text {DIM}_{\text {M=1}}=\mathbb {E}[Y \mid D = 1, M = 1]-\mathbb {E}[Y \mid D = 0, M = 1]$ , using only data on encounters that result in stops. As noted in Section 3.6, under monotonicity and with a “necessary condition mediator,” the TRACE will exceed this quantity by $\mathbb {E}[Y(0) \mid AT]\left (\frac {Pr(\text {C})}{Pr(\text {C})+Pr(\text {AT})}\right )$ . Note also that $\mathbb {E}[Y(0)\mid AT] = \mathbb {E}[Y \mid D=0, M=1]$ , which is also identifiable using only data from stops. Since this term is non-negative, $\text {TRACE} \geq \text {DIM}_{\text {M=1}}$ . Further, when $\frac {Pr(\text {C})}{Pr(\text {C})+Pr(\text {AT})}$ is unknown, the TRACE remains bounded between $\text {DIM}_{\text {M=1}}$ and $\text {DIM}_{\text {M=1}} + \mathbb {E}[Y(0) \mid AT]$ .Footnote ¹²

4.2 Effects of community policing on mob violence in Liberia

4.2.2 Results

Figure 2 illustrates what can be claimed about the effect among implementing types (TRACE) given any assumption about the effect among non-implementing types (TRACE(0)).Footnote ¹⁵ The vertical axis represents postulated values of TRACE(0), while the horizontal axis represents corresponding TRACE estimates. The no-assumption trimming bounds are shown in green and correspond to point estimates of $-$ 0.438 and 0.397 for the lower and upper bounds on TRACE.

Figure 2

Estimated TRACE of community policing intervention on reported mob violence, among communities for which the intervention would have produced a successful Community Watch Forum (implementing types), across postulated values of the treatment effect among non-implementing types (TRACE(0)). Line segments represent 95% confidence intervals. The number on the far right represents the “naive” ITT estimate, assuming constant treatment effects across strata of M. The gray shaded region represents the intersection of the no-assumption bounds and the range of estimates assuming a value for TRACE(0) less than or equal to the value of the TRACE, in the same direction. The green dots show the point estimates of the no assumption bounds, and corresponding 95% confidence intervals obtained by bootstrap. The pink bounds represent the MT bounds described in Section 3.7, and the pink crosshatch shows the intersection of the MT bounds with the assumptions on TRACE(0). All models include police zone (block) fixed effects and baseline levels of reported mob violence, averaged at the community-level, as a covariate.

Figure 2 Long description

The plot has Estimated TRACE on the horizontal axis ranging from about -2.5 to 2.5, and Postulated TRACE(0) on the vertical axis from approximately -1.5 to 1.5. Black dots with horizontal line segments represent point estimates and 95 percent confidence intervals for different postulated values of TRACE(0), arranged vertically from top to bottom. A blue value, negative 0.229, is marked at the intersection of the dashed horizontal line at zero and the vertical line at the corresponding estimated TRACE, indicating the naive I T T estimate. The gray shaded region, located near the origin, marks the intersection of the no-assumption bounds and the range of estimates for TRACE(0) less than or equal to the TRACE value in the same direction. Green dots with horizontal lines at the bottom show point estimates and bootstrap confidence intervals for the no-assumption bounds. Pink horizontal bounds and a pink crosshatched region highlight the M T bounds and their intersection with the assumptions on TRACE(0), as described in Section 3.7. All models include police zone fixed effects and baseline community-level mob violence as covariates.

To illustrate interpretation, consider three possible assumptions. First, suppose we assume that the effect of the community policing intervention is the same among implementing and non-implementing types (i.e., that $\text {TRACE} = \text {TRACE}(0)$ ). Under this assumption, our point estimate of the TRACE is equivalent to that of the ITT, represented by the intersection of the dashed lines at an estimate of $-$ 0.23.Footnote ¹⁶ HC2-based confidence intervals for the ITT are shown using the (blue) vertical line segments around the point estimate. Note that confidence intervals for the TRACE are larger than for the ITT even where the two point estimates are equivalent. This is because the former represents an inference about a sub-group (those with $M(1)=1$ ).

Second, if we are willing to posit that the intervention had no effect on mob violence in non-implementing type communities (i.e., that $\text {TRACE}(0) = 0$ ), we recover an effect size among implementing types of $-$ 0.59, more than twice the magnitude of the ITT, in the hypothesized direction (a reduction in mob violence).

Third, and more practically, we would likely be unwilling to endorse either of the assumptions above, especially using an arbitrary cutoff for community awareness as our implementation indicator (M). However, we are more willing to entertain the assumption that the intervention produced a larger benefit (a more negative effect on mob violence) among implementing types than among non-implementing types. Researchers may argue this directly based on the necessity of M to achieving a meaningful effect.Footnote ¹⁷ If true, this implies estimates for the TRACE that fall to the left of the vertical dotted line.Footnote ¹⁸ If we wish to further assume that TRACE and $\text {TRACE}(0)$ have the same sign, we bound the TRACE on the left at the point where $\text {TRACE}(0)=0$ . This range corresponds to point estimates of TRACE between $-$ 0.23 and $-$ 0.59. By combining the confidence intervals around these estimates with the confidence intervals around the no-assumption bounds, we obtain the valid region for TRACE (Figure 2, shaded region).

The results also imply that any argument for a TRACE (point) estimate larger than $-$ 0.59 would require us to believe that the intervention produced an average increase in mob violence in non-implementing type communities. This could be possible, for example, if police lacked the capacity to respond to increased demand for law enforcement resulting from other aspects of the intervention and—in the absence of a viable lawful community-based alternative—communities (would have) resorted to vigilante justice. Investigators and subject experts can reason about the plausibility of such opposite-directional effects among non-implementing types in this or any given setting.

Figure 3

Estimated TRACE of community policing intervention on reported incidents of crime among implementing type communities, across postulated values of the treatment effect among non-implementing type communities (TRACE(0)), using two different implementation measures (M).

Figure 3 Long description

The top panel is titled Liberia, M equals C W F aware 20 percent, outcome equals Number of Crimes. The y-axis is labeled Postulated TRACE zero, ranging from negative 5 to 5. The x-axis is labeled Estimated TRACE, ranging from negative 15 to 10. Each horizontal line represents a different postulated value of TRACE zero, with a central dot and error bars. Most points cluster near zero, with a few extending left and right. A blue label at y equals zero marks negative 0.0152. The bottom panel is titled Liberia, M equals Comm. meeting aware 20 percent, outcome equals Number of Crimes. The y-axis is Postulated TRACE zero, negative 5 to 5. The x-axis is Estimated TRACE, negative 4 to 4. Horizontal lines with central dots and error bars are plotted for each y value, with most points near zero. A blue label at y equals zero marks negative 0.0152. Both panels use different mediators to estimate the effect of community policing on reported crime, showing similar distributions and central estimates.

4.3 Effects of in-person canvassing on support for transgender rights

4.3.1 Setting

Our second empirical application is a field experiment by Broockman and Kalla (Reference Broockman and Kalla2016), examining the effects of a door-to-door perspective taking intervention on attitudes toward transgender people and support for a transgender non-discrimination law. The intervention involved a 10-minute conversation between canvassers and voters in Miami, Florida, during which canvassers asked voters to talk about a time when they were judged for being different and encouraged them to see connections between their own experiences and those of transgender people. Registered voters who completed a baseline survey (n = 1,825) were randomly assigned to receive either the perspective taking intervention or a placebo intervention. Among the 501 voters who answered the door in either condition, outcomes were measured in follow-up online surveys three days, six weeks, and three months after the intervention.

Broockman and Kalla (Reference Broockman and Kalla2016) find evidence that the intervention both reduced prejudice against transgender people and increased support for policies benefitting them: six weeks and at three months post-intervention, treated individuals were significantly more tolerant of transgender people and more supportive of an ordinance protecting them from discrimination in housing employment and public accommodations.Footnote ²¹

Investigators may then wish to know what the effect of the intervention on policy attitudes would be if they could look just at individuals for whom the intervention (would have, if treated) produced a positive effect on attitudes toward transgender people. For this application, we term this subset “reactive types,” noting that this shorthand is imperfect. We consider this an example of a “mechanism of interest” (type 2) question described above. Our primary outcome Y is support for the transgender non-discrimination law six weeks after treatment (wave 3), measured on a seven-point scale. Our mediator (M) captures the change in subjective feelings toward transgender people from baseline. The authors measure attitudes toward transgender people during all waves using a standard 0–100 feeling thermometer, where a higher number represents “warmer” feelings. We code $M=1$ if an individual’s thermometer score increased between baseline and wave 3, and $M=0$ otherwise.Footnote ²²

Figure 4

Estimated TRACE of in-person canvassing intervention on support for transgender non-discrimination law, among those for whom the intervention would have led to an improvement in feelings toward transgender people from baseline (reactive types), across postulated values of the treatment effect among non-reactive types (TRACE(0)). Line segments represent 95% confidence intervals. The number on the far right represents the “naive” ITT estimate, assuming constant treatment effects across strata of M. The gray shaded region represents the intersection of the no-assumption bounds and the range of estimates assuming a value for TRACE(0) less than or equal to the value of the TRACE, in the same direction. The green dots show the point estimates of the no assumption bounds, and corresponding 95% confidence intervals obtained by bootstrap. All models include control variables used in Broockman and Kalla (Reference Broockman and Kalla2016), including baseline levels of both transgender attitudes and policy support.

Figure 4 Long description

The x-axis is labeled Estimated T R A C E, ranging from approximately minus 2 to plus 4. The y-axis is labeled Postulated T R A C E zero, spanning minus 2 to plus 2. At the center, a gray shaded region intersects the axes near zero, representing the overlap of no-assumption bounds and estimates for T R A C E zero less than or equal to T R A C E. Blue points with horizontal confidence intervals are located within the shaded region, corresponding to bootstrap estimates. Green dots with horizontal confidence intervals appear at the bottom left and right, marking the no-assumption bounds. Black dots with horizontal line segments extend vertically from top to bottom, each representing a confidence interval for estimated T R A C E at different postulated T R A C E zero values. The far right displays the naive I T T estimate in blue, labeled 0.269. All models include control variables as specified in Broockman and Kalla, including baseline transgender attitudes and policy support.