2.1 Setting

Suppose that there are $N (\geq 2)$ pairs and each pair is composed of two units. Let $Y_{ij}$ , $R_{ij}$ , and $X_{ij}$ denote the realized outcome, realized response, and treatment status, respectively, for unit i of pair j where $i =1,2$ and $j = 1,2,\ldots ,N$ . $R_{ij}$ is equal to one (zero) when $Y_{ij}$ is observed (missing). $X_{ij}$ is equal to one (zero) when unit i of pair j is assigned to the treated (control) group. Since this is a pairwise randomized experiment, for every pair j, either $X_{1j}=1, X_{2j}=0$ or $X_{1j}=0, X_{2j}=1$ holds.

I make the stable unit treatment value assumption (Imbens and Rubin Reference Imbens and Rubin2015). $Y_{ij}(1)$ and $R_{ij}(1)$ denote the potential outcome and response for unit i of pair j if treatment is assigned to the unit ( $X_{ij} = 1$ ), respectively. $R_{ij}(1)$ is equal to one (zero) when $Y_{ij} = Y_{ij}(1)$ is observed (missing) in the case of treatment. $Y_{ij}(0)$ and $R_{ij}(0)$ are defined similarly for the case of control.

The main estimand in this section, the finite sample ATE, is denoted by $\tau \equiv E \{ Y_{ij}(1) - Y_{ij}(0) \} \equiv \frac {1}{2N} \sum _{j=1}^{N} \sum _{i=1}^{2} \{ Y_{ij}(1) - Y_{ij}(0) \}$ . The full sample estimator is denoted by $\hat {\tau }_{F} \equiv E(Y_{ij} \mid X_{ij} = 1) - E(Y_{ij} \mid X_{ij} = 0)$ . This estimator is not available in the case of attrition. Rather, analysis of $\hat {\tau }_{F}$ provides reference benchmarks against which this study compares the properties of the UDE and PDE.Footnote ¹ The UDE is denoted by $\hat {\tau }_{U} \equiv E(Y_{ij} \mid X_{ij} = 1, R_{ij} = 1) - E(Y_{ij} \mid X_{ij} = 0, R_{ij} = 1)$ . Obviously, it can be defined only when $N_{t} \equiv \sum _{j=1}^{N} \sum _{i=1}^{2} X_{ij} R_{ij} \geq 1$ and $N_{c} \equiv \sum _{j=1}^{N} \sum _{i=1}^{2} (1 - X_{ij}) R_{ij} \geq 1$ . The PDE is denoted by $\hat {\tau }_{P} \equiv E(Y_{ij} \mid X_{ij} = 1, R_{1j} = R_{2j} = 1) - E(Y_{ij} \mid X_{ij} = 0, R_{1j} = R_{2j} = 1)$ . It can be defined only when $N_{tc} \equiv \sum _{j=1}^{N} R_{1j} R_{2j} \geq 1$ .

In order to clarify the properties of estimators in the propositions below, for $x=0,1$ , this study denotes the between-pair and within-pair deviations of potential outcome by $\beta _{ij}(x) \equiv \{ Y_{1j}(x) + Y_{2j}(x) \}/2 - E \{ Y_{i^{\prime }j^{\prime }}(x) \}$ and $\omega _{ij}(x) \equiv Y_{ij}(x) - \{ Y_{1j}(x) + Y_{2j}(x) \}/2$ .

This study assumes pairwise randomization of treatment assignment. That is, each $X_{1j}$ is ignorable and independent, and $\Pr (X_{1j} = 1) = 1/2$ . In addition, the following three assumptions of potential responses are optional; I will invoke one of them in each of the propositions below. “FS” stands for the “finite sample.” First, I specify the assumption under which no outcomes are missing and thus $\hat {\tau }_{F}$ is available.

Second, we assume a perfect match in the sense that the potential responses are the same between the units in every pair. For instance, in the cases of (monozygotic) twins and littermates of the same sex, matched-on variables (e.g., [part of] DNA) may completely explain the missingness pattern. Under this assumption, $N_{t}$ and $N_{c}$ are constant regardless of treatment assignment, and the properties of $\hat {\tau }_{U}$ can be expressed simply enough to understand its essence.

Finally, we consider the assumption under which $N_{tc}$ is constant regardless of treatment assignment, when the properties of $\hat {\tau }_{P}$ can be presented concisely. This assumption holds if (but not only if) either (or both) of the following two scenarios is true in every pair j. The first scenario is when attrition is unitwise matched ( $R_{1j}(1) = R_{2j}(1), R_{1j}(0) = R_{2j}(0)$ ). (Therefore, Assumption 2 always leads to Assumption 3 but not vice versa.) The second scenario is when each unit is either an “always-reporter” ( $R_{ij}(1) = R_{ij}(0) = 1$ ) or a “never-reporter” ( $R_{ij}(1) = R_{ij}(0) = 0$ ) (Gerber and Green Reference Gerber and Green2012, 225). For instance, in the cases of blind tests, subliminal stimuli, and administrative records, it is likely that $R_{ij}(1) = R_{ij}(0)$ .

2.2 Bias

In this section, the operator $\mathbb {E}(\cdot )$ takes expectation over the random assignment of the treatment. Now, I present the bias of each ATE estimator.

A few remarks are in order. First, unless we assume ignorable attrition, not only $\hat {\tau }_{U}$ but also $\hat {\tau }_{P}$ are biased for $\tau $ . Second, $\hat {\tau }_{P}$ has a causal interpretation under a weaker assumption than $\hat {\tau }_{U}$ . Under Assumption 3 and $N_{tc} \geq 1$ , I define a kind of local average treatment effect (LATE) of “always-reporting pairs” by $\tau _{P} \equiv E \{ Y_{ij}(1) - Y_{ij}(0) \mid R_{1j}(1) = R_{2j}(1) = R_{1j}(0) = R_{2j}(0) = 1 \}$ .Footnote ² This is a causal effect because it is a principal effect (Frangakis and Rubin Reference Frangakis and Rubin2002) where the corresponding principal stratum is a set of such pairs that $R_{1j}(1) = R_{2j}(1) = R_{1j}(0) = R_{2j}(0) = 1$ . It follows that $\hat {\tau }_{P}$ is unbiased for $\tau _{P}$ : $\mathbb {E}(\hat {\tau }_{P} ) - \tau _{P} = 0$ . This argument may correctly remind readers of instrumental variable estimation for noncompliance cases. By contrast, even under Assumption 2 and $N_{t}, {N_{c} \geq 1}$ , it is difficult to interpret $\hat {\tau }_{U}$ as a causal effect unless $R_{ij}(1) = R_{ij}(0)$ for all i and j, in which case $\hat {\tau }_{U}$ is reduced to $\hat {\tau }_{P}$ .

2.3 Variance

In this section, $\mathrm {Var} (\cdot )$ and $\mathrm {Cov} (\cdot , \cdot )$ denote the finite sample variance and covariance, respectively, and the operator $\mathbb {V}^2 (\cdot )$ takes variance over the random assignment of the treatment. Here are the variances of the three ATE estimators.

It is worth while mentioning that even though the number of units used for estimation is not smaller for $\hat {\tau }_{U}$ than for $\hat {\tau }_{P}$ (i.e., $N_{t}, N_{c} \geq N_{tc}$ ), $\hat {\tau }_{U}$ can be less efficient than $\hat {\tau }_{P}$ (i.e., $\mathbb {V}^2 (\hat {\tau }_{U})> \mathbb {V}^2(\hat {\tau }_{P})$ ) if $\mathrm {Var} \{ \omega _{ij}(x) \mid R_{1j}(x) = R_{2j}(x) = 1, R_{1j}(1 - x) = R_{2j}(1 - x) = 0 \}$ is sufficiently larger than $ \mathrm {Var} \{ \omega _{ij}(x) \mid R_{1j}(1) = R_{2j}(1) = R_{1j}(0) = R_{2j}(0) = 1 \}$ for $x=0,1$ .

2.4 Variance Estimator

How should we estimate the variances of these three ATE estimators? For certain situations, some scholars advocate “breaking the matches” (Lynn and McCulloch Reference Lynn and McCulloch1992), namely, analyzing data from pairwise randomized experiments as if they are completely randomized experiments. Thus, I begin with the Neyman variance estimator. Suppose that Assumption 1 holds. For $x=0,1$ , if $X_{ij} = x$ , a natural estimator of $\omega _{ij}(x)$ is $\hat {\omega }_{ij}(x) \equiv Y_{ij} - E(Y_{ij} \mid X_{ij} = x)$ . Accordingly, we may estimate $\mathrm {Var} \{ \omega _{ij}(x) \}$ in the first and second terms of the equation in Proposition 2 (1) by

$$ \begin{align*} \frac{N}{N - 1}\mathrm{Var} \{ \hat{\omega}_{ij}(x) \mid X_{ij} = x \} =\frac{N}{N - 1}\mathrm{Var} ( Y_{ij} \mid X_{ij} = x ). \end{align*} $$

The third term of the equation in Proposition 2 (1) “is generally impossible to estimate empirically because we never observe both $Y_{ij}(1)$ and $Y_{ij}(0)$ for the same unit” (Imbens and Rubin Reference Imbens and Rubin2015, 92). Thus, if we dismiss the third term, we derive the Neyman variance estimator of $\hat {\tau }_{F}$ as

$$ \begin{align*} \widehat{\mathbb{V}}^{\text{Neyman}}(\hat{\tau}_{F}) \equiv \frac{1}{N - 1} \{ \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1 ) + \mathrm{Var} ( Y_{ij} \mid X_{ij} = 0 ) \}. \end{align*} $$

Similarly, without Assumption 1, if $N_{tc} \geq 2$ or $N_{t}, N_{c} \geq 2$ , we can derive the Neyman variance estimators of $\hat {\tau }_{P}$ or $\hat {\tau }_{U}$ as

$$ \begin{align*} \begin{split} \widehat{\mathbb{V}}^{\text{Neyman}}(\hat{\tau}_{P}) & \equiv \frac{1}{N_{tc} - 1} \{ \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1, R_{1j} = R_{2j} = 1 ) + \mathrm{Var} ( Y_{ij} \mid X_{ij} = 0, R_{1j} = R_{2j} = 1 ) \} \\ \widehat{\mathbb{V}}^{\text{Neyman}}(\hat{\tau}_{U}) & \equiv \frac{1}{N_{t} - 1} \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1, R_{ij} = 1 ) + \frac{1}{N_{c} - 1}\mathrm{Var} ( Y_{ij} \mid X_{ij} = 0, R_{ij} = 1 ), \end{split} \end{align*} $$

respectively.

Nonetheless, researchers do not have to give up estimating the third terms in the equations in Proposition 2. A merit of pairwise randomized experiments is that even if $X_{1j}=1, R_{2j}=1$ , analysts may estimate $\omega _{1j}(0)$ by $\hat {\omega }_{1j}(0) \equiv -\hat {\omega }_{2j}(0)$ because $\omega _{1j}(0) = - \omega _{2j}(0)$ . This finding is the most important contribution of this study. Therefore, when $N_{tc} \geq 2$ , we may estimate $\mathrm {Cov} \{ \omega _{ij}(1), \omega _{ij}(0) \mid R_{1j}(1) = R_{2j}(1) = R_{1j}(0) = R_{2j}(0) = 1 \}$ in the third terms by

$$ \begin{align*} \frac{N_{tc}}{N_{tc} - 1}\mathrm{Cov} \{ \hat{\omega}_{ij}(1), \hat{\omega}_{ij}(0) \mid R_{1j} = R_{2j} = 1\} = - \frac{N_{tc}}{N_{tc} - 1}\mathrm{Cov} ( Y_{tj}, Y_{cj} \mid R_{1j} = R_{2j} = 1), \end{align*} $$

where $Y_{tj} = \sum ^{2}_{i=1} X_{ij} Y_{ij}$ and $Y_{cj} = \sum ^{2}_{i=1} (1 - X_{ij}) Y_{ij}$ . Accordingly, when $N_{tc} \geq 2$ , I propose the adjusted Neyman variance estimators as

$$ \begin{align*} \begin{split} \widehat{\mathbb{V}}^{\text{Adj-Neyman}}(\hat{\tau}_{F}) & \equiv \frac{1}{N - 1} \{ \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1 ) + \mathrm{Var} ( Y_{ij} \mid X_{ij} = 0 ) - 2 \mathrm{Cov} ( Y_{tj}, Y_{cj}) \} \\ & = \frac{1}{N - 1} \mathrm{Var} ( Y_{tj} - Y_{cj})\\ \widehat{\mathbb{V}}^{\text{Adj-Neyman}}(\hat{\tau}_{P}) & \equiv \frac{1}{N_{tc} - 1} \{ \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1, R_{1j} = R_{2j} = 1 ) + \mathrm{Var} ( Y_{ij} \mid X_{ij} = 0, R_{1j} = R_{2j} = 1 )\\ &\quad - 2 \mathrm{Cov} ( Y_{tj}, Y_{cj} \mid R_{1j} = R_{2j} = 1) \} \\ & = \frac{1}{N_{tc} - 1} \mathrm{Var} ( Y_{tj} - Y_{cj} \mid R_{1j} = R_{2j} = 1)\\[20pt] \widehat{\mathbb{V}}^{\text{Adj-Neyman}}(\hat{\tau}_{U}) & \equiv \frac{1}{N_{t} - 1} \mathrm{Var} ( Y_{ij} \mid X_{ij} = 1, R_{ij} = 1 ) + \frac{1}{N_{c} - 1}\mathrm{Var} ( Y_{ij} \mid X_{ij} = 0, R_{ij} = 1 ) \\ &\quad - \frac{2 N^2_{tc}}{N_{t} N_{c} (N_{tc} - 1)}\mathrm{Cov} ( Y_{tj}, Y_{cj} \mid R_{1j} = R_{2j} = 1). \end{split} \end{align*} $$

It turns out that the adjusted Neyman variance estimator of $\hat {\tau }_{F}$ is reduced to what scholars recommend (Gerber and Green Reference Gerber and Green2012, 77; Imai Reference Imai2008, 4861; Imbens and Rubin Reference Imbens and Rubin2015, 227). This paper extends it to $\hat {\tau }_{P}$ and, in particular, $\hat {\tau }_{U}$ . I now derive the properties of these variance estimators.

A merit of $\widehat{\mathbb{V}}^{\text {Adj-Neyman}}(\hat {\tau }_{F})$ and $\widehat{\mathbb{V}}^{\text {Adj-Neyman}}(\hat {\tau }_{P})$ is that they cannot have negative bias (and thus they are conservative). By contrast, the other variance estimators can be downwardly biased.

3.1 Setting

Following Imai (Reference Imai2008) and Imbens and Rubin (Reference Imbens and Rubin2015, chs. 6 and 10, esp. pp. 109 and 229), this paper supposes that the above N pairs of a finite sample are drawn from a super-population that is composed of $N^{*} (> N)$ pairs, with $N^{*}$ large, but countable. I define super-population variables and operators in the same way as the corresponding finite sample variables and operators, and denote them by adding superscript * to each term. In particular, the main estimand of this section, the super-population ATE, is denoted by $\tau ^{*} \equiv E^{*} \{ Y^{*}_{i^{*}j^{*}}(1) - Y^{*}_{i^{*}j^{*}}(0) \} \equiv \frac {1}{2N^{*}} \sum _{j^{*}=1}^{N^{*}} \sum _{i^{*}=1}^{2} \{ Y^{*}_{i^{*}j^{*}}(1) - Y^{*}_{i^{*}j^{*}}(0) \}$ .

I assume random sampling of pairs. In addition, the following three assumptions of potential responses are optional in the same spirit as the finite sample case. SP stands for the “super-population.”

Note that unlike the case of a finite sample, in the super-population perspective, $N_{t}$ , $N_{c}$ , and $N_{tc}$ are not constant across sampling of pairs even under Assumption 2^* or 3^* .

3.2 Bias

In this section, the operator $\mathbb {E}^{*}(\cdot )$ takes expectation not only over the random assignment of the treatment but also over the random sampling of the pairs. Define

$$ \begin{align*} \bar{\beta}^{*}(x \mid r_{1}, r_{0}) \equiv E^{*} \{ \beta^{*}_{i^{*}j^{*}}(x) \mid R^{*}_{1j^{*}}(1) = R^{*}_{2j^{*}}(1) = r_{1}, R^{*}_{1j^{*}}(0) = R^{*}_{2j^{*}}(0) = r_{0} \}. \end{align*} $$

The bias of each ATE estimator is as follows.

As in the case of a finite sample, unless we assume ignorable attrition, not only $\hat {\tau }_{U}$ but also $\hat {\tau }_{P}$ is biased for $\tau ^{*}$ . It also holds that $\mathbb {E}^{*}(\hat {\tau }_{P} ) - \tau ^{*}_{P} = 0$ . Furthermore, note that in general,

$$ \begin{align*} \mathbb{E}^{*} ( \hat{\tau}_{U} \mid N_{t}, N_{c} \geq 1)- \tau^{*} \neq E^{*} \{ \beta^{*}_{i^{*}j^{*}}(1) \mid R^{*}_{i^{*}j^{*}}(1) = 1 \} - E^{*} \{ \beta^{*}_{i^{*}j^{*}}(0) \mid R^{*}_{i^{*}j^{*}}(0) = 1 \} \end{align*} $$

because

$$ \begin{align*} \mathbb{E}^{*} \Big( \frac{ N_{tc} }{ N_{t}} \Big| N_{t}, N_{c} \geq 1 \Big) \geq \frac{ N^{*}_{tc} }{ N^{*}_{t}}, \quad \mathbb{E}^{*} \Big( \frac{ N_{tc} }{ N_{c}} \Big| N_{t}, N_{c} \geq 1 \Big) \geq \frac{ N^{*}_{tc} }{ N^{*}_{c}}. \end{align*} $$

3.3 Variance

In this section, $\mathrm {Var}^{*} (\cdot )$ and $\mathrm {Cov}^{*} (\cdot , \cdot )$ denote the super-population variance and covariance, respectively, and the operators $\mathbb {V}^{2*} (\cdot )$ and $\mathbb {V}^{*} (\cdot , \cdot )$ take variance and covariance, respectively, not only over the random assignment of the treatment but also over the random sampling of the pairs. In addition, I assume that under Assumption 3^* , either $\lim _{N^{*} \rightarrow \infty } N^{*}_{tc} = \infty $ or $\lim _{N^{*} \rightarrow \infty } N^{*}_{tc} < 2$ holds. I also assume that under Assumption 2^* , the same conditions hold not only for $N^{*}_{tc}$ but also for $N^{*}_{t} - N^{*}_{tc}$ and $N^{*}_{c} - N^{*}_{tc}$ .

Below, I derive the super-population variance of the three ATE estimators in the limit. Note that I increase $N^{*}$ , not N.

Like the case of a finite sample, $\mathbb {V}^{2*}( \hat {\tau }_{U} \mid N_{tc} \geq 1)$ can be larger than $\mathbb {V}^{2*}( \hat {\tau }_{P}\mid N_{tc} \geq 1)$ .

3.4 Variance Estimator

Finally, I show the super-population biases of the two variance estimators in the limit.

Proposition 4^* (2) is new to my knowledge and surprising. Even if we do not assume ignorable attrition, the adjusted Neyman variance estimator is unbiased for the super-population variance of the PDE. Here is an intuitive explanation: If we regard always-reporting pairs as an alternative super-population and apply Proposition 4^* (1) to it, we obtain Proposition 4^* (2). In a nutshell, $\widehat{\mathbb{V}}^{\text {Adj-Neyman}}(\hat {\tau }_{P})$ corresponds to the design of pairwise randomization. On the other hand, the bias directions of the other variance estimators are unknown.