2.1 Assumptions

Assumption 2 no unmeasured confounding variables

The potential outcomes are independent of treatment assignment conditional on study membership: ${Y}_{ij}(0),{Y}_{ij}(1)\perp {A}_{ij}\mid {X}_{ij}$ .

Assumption 2 implies that the study indicator ${X}_{ij}$ is the sole confounding variable relevant to the treatment assignment in the pooled analysis. This assumption is reasonable in the context of meta-analysis of randomized clinical trials. While randomization within each trial ensures that pretreatment covariates are balanced between treatment arms, differences across studies (captured by ${X}_{ij}$ ) may induce confounding in the pooled analysis when data from multiple studies are combined. Thus, conditioning on study membership is essential to account for such between-study heterogeneity in the pooled analysis.

As demonstrated in Figure 1, heterogeneity in study design can induce confounding in pooled analyses of randomized trials. Specifically, when different studies employ varying randomization ratios, the study indicator ${X}_{ij}$ may be associated with treatment assignment ${A}_{ij}$ (i.e., ${X}_{ij}\to {A}_{ij}$ ). In addition, heterogeneity in baseline event rates across studies—for example, due to differences in patient characteristics or clinical settings—may induce an association between ${X}_{ij}$ and the observed outcome ${Y}_{ij}$ (i.e., ${X}_{ij}\to {Y}_{ij}$ ). Thus, ${X}_{ij}$ serves as a between-study confounder that must be accounted for to obtain unbiased causal effect estimates. Importantly, within each individual study, randomization ensures that no other unmeasured pretreatment confounders exist. Therefore, we posit that conditioning on ${X}_{ij}$ is sufficient to control for all confounding between treatment assignment and outcome in the combined analysis.Reference Rubin³⁴ When different studies have different randomization ratios, ${X}_{ij}$ , the study indicator is associated with the treatment assignment. ${X}_{ij}$ may also be related to the observed outcome ${Y}_{ij}$ . This is because the prevalence of ${Y}_{ij}$ might be associated with the heterogeneous baseline event rates across studies. Thus, ${X}_{ij}$ serves as a between-study confounding variable. However, within each study, all other pretreatment confounders are controlled because of the randomization procedure. Thus, we can assume there is only a covariate ${X}_{ij}$ serves as a confounder for the potential outcomes.

Figure 1

The directed acyclic graph (DAG) illustrating the assumed causal relationships between the treatment (A), outcome (Y), and a study indicator (X). In this structure, X confounds the relationship between A and Y, as it influences both variables.

2.2 Proposed Causal Meta-Analysis Framework Under Assumptions

To adjust the confounder effect of ${X}_{ij}$ , propensity score weighting is a commonly used approach in the causal inference literature.Reference Rosenbaum and Rubin^35, Reference Austin³⁶ It creates a pseudo-weighted population where the covariate distribution in the treated subjects is the same as the covariate distribution in the not-treated subjects. The causal estimand ${\tau}_w$ for the weighted population is displayed as:

$$\begin{align*}{\tau}_w=\frac{E\left(h(X)\left({\mu}_1(X)-{\mu}_0(X)\right)\right)}{E\left(h(X)\right)},\end{align*}$$

where ${\mu}_1(X)=E\left(Y(1)|X\right),{\mu}_0(X)=E\left(Y(0)|X\right),$ and $h(X)$ is the weighting function of the propensity score $e(X)$ to shift the population from its original covariate distribution $f(x)$ . With different forms of $h(X)$ , the WATE can be equivalent to different target estimands, such as the ATE when $h(X)=1$ .

The individual estimated weight for the $i$ th subject within the $j$ th study, denoted by ${m}_{ij}$ , is a function of the estimated propensity score ${\widehat{e}}_{ij}=\widehat{P}\left({A}_{ij}=1|{X}_{ij}\right)$ from logistic regression. When the form of $h(X)$ is a function of the propensity score, we have  $\widehat{h}\left({X}_{ij}\right)=h\left({\widehat{e}}_{ij}\right)$ . In the treatment arm, ${m}_{ij}$ is equal to  $\frac{\widehat{h}\left({X}_{ij}\right)}{{\widehat{e}}_{ij}}$ . In the control arm, it is equal to $\frac{\widehat{h}\left({X}_{ij}\right)}{1-{\widehat{e}}_{ij}}$ . With the weights ${m}_{ij}$ , the following estimator estimates the WATE, that is,

(1) $$\begin{align}{\widehat{\tau}}_w=\frac{\sum_{j=1}^K{\sum}_{i=1}^{n_j}{A}_{ij}{Y}_{ij}{m}_{ij}}{\sum_{j=1}^K{\sum}_{i=1}^{n_j}{A}_{ij}{m}_{ij}}-\frac{\sum_{j=1}^K{\sum}_{i=1}^{n_j}\left(1-{A}_{ij}\right){Y}_{ij}{m}_{ij}}{\sum_{j=1}^K{\sum}_{i=1}^{n_j}\left(1-{A}_{ij}\right){m}_{ij}}.\kern3.839998em\end{align}$$

Under the assumption for the between-study confounders and no within-study confounders, the estimated propensity scores for all subjects within the $j$ th study are the same because ${X}_{ij}$ is a study indicator. Moreover, the estimate of the propensity score is thus reduced to the estimated randomization ratio for the $j$ th study. In other words, for the $i$ th subject,

(2) $$\begin{align}{\widehat{e}}_{ij}={\widehat{\lambda}}_jI\left({X}_{ij}=j\right)\kern1.2em j=1,..~,K,\kern4.919997em\end{align}$$

where ${\widehat{\lambda}}_j=\frac{n_{1j}}{n_j}$ is the proportion of treatments in the $j$ th study and $I\left(\cdot \right)$ is an indicator function. Thus, $\widehat{h}\left({X}_{ij}\right)={\widehat{h}}_jI\left({X}_{ij}=j\right)$ with ${\widehat{h}}_j=h\left({\widehat{\lambda}}_j\right)$ when $j=1,\dots, K$ . Based on the aggregated data by using Equation (2) and the corresponding ${\widehat{h}}_j$ , we propose the fixed-effect CMA estimator:

(3) $$\begin{align}{\widehat{\tau}}_{CMA}={\sum}_{j=1}^K{w}_j\left({\widehat{p}}_{1j}-{\widehat{p}}_{0j}\right),\kern7.679995em\end{align}$$

where ${w}_j={\left({\sum}_{j=1}^K{n}_j{\widehat{h}}_j\right)}^{-1}{n}_j{\widehat{h}}_j$ . In Supplementary Materials, we show how the CMA estimator shown in Equation (3) can be obtained from Equation (1) by only using the aggregated data ${\widehat{h}}_j$ .

The variance of the ${\widehat{\tau}}_{CMA}$ can be calculated by finding the influence function of ${\tau}_w$ . The variance estimator can thus be obtained by ${\left({n}^2\;{\widehat{\theta}}^2\right)}^{-1}{\sum}_{j=1}^K{\widehat{V}}_j$ , where $\widehat{\theta}=\widehat{E}\left(h(X)\right)$ , and the empirical of mean for $h(X)$ and ${V}_j$ can be calculated by the aggregated data shown in Supplementary Materials.

With the different forms of ${w}_j$ , the CMA is designed to target different causal estimands tailored to specific populations. In the following subsections, we define the target population for each target estimand—ATE, ATT, ATC, and ATO. Subsequently, we present the corresponding forms of ${w}_j$ for CMA in the context of ATE, ATT, ATC, and ATO. In addition, we establish the relationship between CMA and traditional meta-analysis. Finally, we present the connections between ATE, ATT, ATC, and ATO.

2.3 ATE, inverse probability weighting, and weighting by study size

The ATE is $E\left(Y(1)-Y(0)\right)$ , which is equal to WATE if $h(X)=1$ . The target population of ATE is interpreted as the patients from both the treated and control groups across all studies. Under such case, ${m}_{ij}$ in the control arm is $\frac{1}{1-{\widehat{e}}_{ij}}$ and ${m}_{ij}$ in the treatment arm is $\frac{1}{{\widehat{e}}_{ij}}$ , also known as the inverse probability weighting. With Equation (3) under the constant of the function, $h=1$ , the proposed CMA estimator will be in the following formula:

(4) $$\begin{align}{\sum}_{j=1}^K\frac{{\widehat{p}}_{1j}{n}_j}{n}-{\sum}_{j=1}^K\frac{{\widehat{p}}_{0j}{n}_j}{n}.\kern7.199995em\end{align}$$

It has the causal interpretation, that is, the estimated pooled event rate for the treatment arm versus the control arm, whereas the subjects counterfactually from both comparison groups are given the treatment or control. Interestingly, we find that it is equivalent to the traditional meta-analysis estimator weighted by the study-specific sample sizes, ${w}_j^{ATE}=\frac{n_j}{n}$ . The derived closed-form variance for CMA-targeted ATE is

$$\begin{align*}{V}^{ATE}&={n}^{-2}{\sum}_{j=1}^K{n}_{1j}{\widehat{p}}_{1j}{\left[{e}_j^{-1}\left(1-{\widehat{\tau}}_1\right)+\left(1-{e}_j\right){M}_j\right]}^2+{n}_{1j}\left(1-{\widehat{p}}_{1j}\right){\left[{e}_j^{-1}{\widehat{\tau}}_1+\left(1-{e}_j\right){M}_j\right]}^2\\&\quad+{n}_{0j}{\widehat{p}}_{0j}{\left[{\left(1-{e}_j\right)}^{-1}\left(1-{\widehat{\tau}}_0\right)+{e}_j{M}_j\right]}^2+{n}_{0j}\left(1-{\widehat{p}}_{0j}\right){\left[{\left(1-{e}_j\right)}^{-1}{\widehat{\tau}}_0-{e}_j{M}_j\right]}^2,\end{align*}$$

where ${M}_j=-{e}_j^{-1}\left({\widehat{p}}_{1j}-{\widehat{\tau}}_1\right)-{\left(1-{e}_j\right)}^{-1}\left({\widehat{p}}_{0j}-{\widehat{\tau}}_0\right)$ , ${\widehat{\tau}}_1={\sum}_{j=1}^K\frac{{\widehat{p}}_{1j}{n}_j}{n}$ , and ${\widehat{\tau}}_0={\sum}_{j=1}^K\frac{{\widehat{p}}_{0j}{n}_j}{n}$ .

2.4 ATT and ATC

If our target estimand is ATT, that is, ${\tau}_w=E\left(Y(1)-Y(0)|A=1\right)$ with $h(X)=e(X)$ . The target population for ATT is the patients from the treated groups across all studies. Under such case, when substituting $h(X)=e(X)$ into Equation (3), the formula of the CMA estimator will be as follows:

(5) $$\begin{align}{\sum}_{j=1}^K\frac{n_{1j}}{n_T}\left({\widehat{p}}_{1j}-{\widehat{p}}_{0j}\right),\kern7.919994em\end{align}$$

where ${n}_T={\sum}_{j=1}^K{n}_{1j}$ denotes the number of subjects in the treated groups across studies. The weights are ${w}_j^{ATT}=\frac{n_{1j}}{n_T}$ . The variance of CMA for ATT is

$$\begin{align*}{V}^{ATT}&={n}_T^{-2}{\sum}_{j=1}^K{n}_{1j}{\widehat{p}}_{1j}{\left[\left(1-{\widehat{\tau}}_1\right)+\left(1-{e}_j\right){M}_j\right]}^2+{n}_{1j}\left(1-{\widehat{p}}_{1j}\right){\left[-{\widehat{\tau}}_1+\left(1-{e}_j\right){M}_j\right]}^2\\&\quad+{n}_{0j}{\widehat{p}}_{0j}{\left[{\left(1-{e}_j\right)}^{-1}\left(1-{\widehat{\tau}}_0\right){e}_j+{e}_j{M}_j\right]}^2+{n}_{0j}\left(1-{\widehat{p}}_{0j}\right){\left[{\left(1-{e}_j\right)}^{-1}{\widehat{\tau}}_0{e}_j-{e}_j{M}_j\right]}^2,\end{align*}$$

where the ${M}_j=-{\left(1-{e}_j\right)}^{-1}\left({\widehat{p}}_{0j}-{\widehat{\tau}}_0\right)$ ,  ${\widehat{\tau}}_1={\sum}_{j=1}^K\frac{n_{1j}}{n_T}\;{\widehat{p}}_{1j}$ , and ${\widehat{\tau}}_0={\sum}_{j=1}^K\frac{n_{1j}}{n_T}\;{\widehat{p}}_{0j}$ .

Similarly, if our target estimand is ATC, that is, ${\tau}_w=E\left(Y(1)-Y(0)|A=0\right)$ with $h(X)=1-e(X)$ . The target population of ATC is the patients from the control groups across all studies. Under such case, the proposed CMA estimator is

(6) $$\begin{align}{\sum}_{j=1}^K\frac{n_{0j}}{n_C}\left({\widehat{p}}_{1j}-{\widehat{p}}_{0j}\right),\kern7.919994em\end{align}$$

where ${n}_C={\sum}_{j=1}^K{n}_{0j}$ denotes the number of subjects in the control across studies. The weights are ${w}_j^{ATC}=\frac{n_{0j}}{n_C}$ . The variance of CMA for ATC is

$$\begin{align*}{V}^{ATC}&={n}_C^{-2}{\sum}_{j=1}^K{n}_{1j}{\widehat{p}}_{1j}{\left[{e}_j^{-1}\left(1-{\widehat{\tau}}_1\right)\left(1-{e}_j\right)+\left(1-{e}_j\right){M}_j\right]}^2\\&\quad +{n}_{1j}\left(1-{\widehat{p}}_{1j}\right){\left[-{e}_j^{-1}{\widehat{\tau}}_1\left(1-{e}_j\right)+\left(1-{e}_j\right){M}_j\right]}^2+{n}_{0j}{\widehat{p}}_{0j}{\left[-\left(1-{\widehat{\tau}}_0\right)-{e}_j{M}_j\right]}^2\\&\quad +{n}_{0j}\left(1-{\widehat{p}}_{0j}\right){\left[{\widehat{\tau}}_0-{e}_j{M}_j\right]}^2,\end{align*}$$

where the ${M}_j=-{e}_j^{-1}\left({\widehat{p}}_{1j}-{\widehat{\tau}}_1\right),{\widehat{\tau}}_1={\sum}_{j=1}^K\frac{n_{0j}}{n_C}\;{\widehat{p}}_{1j},$ and ${\widehat{\tau}}_0={\sum}_{j=1}^K\frac{n_{0j}}{n_C}\;{\widehat{p}}_{0j}$ .

2.5 ATO, overlap weighting, and Mantel–Haenszel estimators

For the overlap weighting, the function for the targeted overlap population is $h(X)=e(X)\left(1-e(X)\right)$ . Thus, we used the ${m}_{ij}=1-{\widehat{e}}_{ij}$ if the $i$ th subject is in the treated arm and ${m}_{ij}={\widehat{e}}_{ij}$ for the $i$ th subject in the control arm. The target population of ATO is the population of patients who exhibit the greatest clinical equipoise or highest uncertainty regarding both comparison groups. By using the overlap weights correspondingly, the proposed CMA estimator is reduced to

(7) $$\begin{align}{\left({\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\right)}^{-1}{\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\left({\widehat{p}}_{1j}-{\widehat{p}}_{0j}\right).\kern6.119996em\end{align}$$

From Equation (5), it can be seen as the weighted average estimator with the weight being the Mantel–Haenszel weights. Thus, the target estimand of the Mantel–Haenszel estimators is the ATO. The weights are ${w}_j^{ATO}={\left({\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\right)}^{-1}\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}$ . The variance of CMA under ATO is

$$\begin{align*}{V}^{ATO}&={\left({\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\right)}^{-2}{\sum}_{j=1}^K{n}_{1j}{\widehat{p}}_{1j}{\left(1-{e}_j\right)}^2{\left[\left(1-{\widehat{\tau}}_1\right)+{M}_j\right]}^2\\&\quad+{n}_{1j}\left(1-{\widehat{p}}_{1j}\right){\left(1-{e}_j\right)}^2{\left[-{\widehat{\tau}}_1+{M}_j\right]}^2+{n}_{0j}{\widehat{p}}_{0j}{e}_j^2{\left[\left(1-{\widehat{\tau}}_0\right)+{M}_j\right]}^2\\&\quad+{n}_{0j}\left(1-{\widehat{p}}_{0j}\right){e}_j^2{\left[{\widehat{\tau}}_0-{M}_j\right]}^2,\end{align*}$$

where ${M}_j=-{e}_j\left({\widehat{p}}_{1j}-{\widehat{\tau}}_1\right)-\left(1-{e}_j\right)\left({\widehat{p}}_{0j}-{\widehat{\tau}}_0\right),{\widehat{\tau}}_1={\left({\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\right)}^{-1}{\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}{\widehat{p}}_{1j},$ and ${\widehat{\tau}}_0={\left({\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}\right)}^{-1}{\sum}_{j=1}^K\frac{n_{0j}{n}_{1j}}{n_{0j}+{n}_{1j}}{\widehat{p}}_{0j}$ . The target population size can be different for different target estimands when the randomized ratios are different across studies, and an example of target estimands and population sizes is illustrated in Supplementary Appendix S1.

2.6 Connection between ATE, ATT, ATC, and ATO

When the allocation ratios across all studies are consistent, such that for the $j$ th study ( $j=1,\dots, K$ ), the ratio $\frac{n_{0j}}{n_{1j}}=c$ , where $c$ is a constant, we find ourselves in an ideal scenario where treatment $A$ and study indicator $X$ are uncorrelated. In this case, the weights derived from CMA for ATE, ATT, ATC, and ATO are shown in Equations (4) to (7) and are identical. As detailed in the Supplementary Material, the weights from CMA are uniformly $\frac{n_j}{\sum_{j=1}^K{n}_j}$ . Consequently, the estimates for ATE, ATT, ATC, and ATO are identical when the randomization ratios across studies are identical. It is worth noting that the sample size of the different target populations is not assumed to be the same under such a scenario.