Marginal Structural Models and Inverse Probability of Censoring Weights

It is simplest to think of the MSM with IPCW method as an extension of the simple censoring approach, whereby the bias associated with censoring is removed through the use of a weighting mechanism and the creation of a “pseudo population” (a population adjusted for the distortions that arise from the prognostic differences between switchers and nonswitchers) (Reference Hernan, Brumback and Robins14). The MSM with IPCW approach involves three key steps.

1. (1) First, switchers are censored at the point of switch. In Supplementary Figure 1a, we illustrate survival times observed for ten patients (five control group patients, and five experimental group patients) from a fictitious RCT, in which switching from the control group onto the experimental treatment was permitted after disease progression. In this example, three of the five control group patients switched, and are censored at the point of switch.

2. (2) Second, baseline and time-dependent information on patient characteristics that are prognostic factors for mortality and that influence the probability of switching are identified. These might include factors such as age and/or tumor burden and should be prespecified, and/or based on evidence or expert opinion. A dataset is then created such that each patient has a full set of values for these prognostic characteristics from randomization to the time of last contact. Models are fitted to this dataset to predict the probability of not switching in control group patients, conditional on the identified prognostic factors (Supplementary Figure 1b). These probabilities are used to determine the size of the weight applied to each patient.Control group patients who did not switch but who have similar patient characteristics to patients who did switch are assigned higher weights, allowing them to account for themselves and for similar patients who switched. The weights may change for each patient over time, as the characteristics displayed by patients change. In our example, Patient 2, a control group patient who did not switch treatments, must have displayed dissimilar prognostic characteristics to patients who switched, because he or she received weights of less than 1.0 throughout his or her survival experience. In contrast, Patient 5 must have displayed similar prognostic characteristics to switchers, because he or she received a weight of greater than 1.0 after disease progression. All experimental group patients receive a weight of 1.0 throughout the MSM/IPCW analysis, because we do not wish to adjust for any switching in the experimental group (in this example). This step creates the (weighted) pseudo population.

3. (3) Finally, an analysis to estimate the adjusted treatment effect is undertaken (typically, a standard Cox regression model [Reference Cox15] is used to estimate a hazard ratio) (Supplementary Figure 1c). The Kaplan-Meier graph of survival probability against time presented in Supplementary Figure 1c presents the observed control group and experimental group Kaplan-Meier curves, and the Kaplan-Meier curve of the weighted control group (i.e., the pseudo population).

The main challenge in using the MSM/IPCW approach is to ensure that the weightings compensate appropriately for the bias created by censoring switchers. For this to be fully achieved, data must be available at baseline and over time on all prognostic factors for mortality that also influence switching; this is known as the assumption of “no unmeasured confounding” (Reference Hernan, Brumback and Robins14–Reference Robins and Finkelstein16). We can never be sure that this assumption holds, because we do not have perfect knowledge on which characteristics are prognostic. However, a clear statement of which prognostic characteristics have been considered and why (e.g., based upon data from other trials and/or clinical judgement), and comprehensive data collection on these may help to re-assure decision makers that the result of an MSM/IPCW analysis is credible.

A further problem is that, although it may be desirable to include as many potentially prognostic covariates as possible within the MSM/IPCW analysis, in relatively small RCTs this may lead to convergence issues, that is, the method does not produce a result. Also, specific patients may be assigned very high weights, particularly if there are very few control group patients who did not switch treatments. In this case, the method becomes prone to substantial error, because there is a higher likelihood that these remaining patients may not be representative of the wider trial population.

Two-Stage Adjustment

The Two-Stage adjustment method to address the treatment switching problem was developed for the situation that often pertains in RCTs of oncology treatments, where switching only occurs after disease progression (Reference Latimer, Abrams and Lambert2;Reference Latimer, Abrams and Lambert12;Reference Latimer, Abrams, Lambert, Morden and Crowther13). The method involves three steps, which allow survival times that would have been observed in the absence of switching to be estimated (these are also known as “counterfactual” survival times).

1. (1) First, a “secondary baseline” is identified for patients in the control group (assuming that we wish to adjust for switching only in the control group). Switching must only occur after this secondary baseline. Supplementary Figure 2 provides an illustration of a Two-Stage adjustment analysis applied to the fictitious RCT previously introduced. In this example, the point of disease progression is used as the secondary baseline for each patient (Supplementary Figure 2a).

2. (2) Second, postsecondary baseline (in this case post-disease progression) survival is analyzed within the control group (Supplementary Figure 2b). A treatment effect (in the form of an “time ratio”) associated with the experimental treatment is then estimated specifically for control group switching patients, compared with control group nonswitchers, using a model that is adjusted for patient characteristics measured at the time of the secondary baseline. Patients are assumed to be at a similar stage of disease at the secondary baseline, and if prognostic characteristics are measured at this time-point we can adjust for any differences between switchers and nonswitchers, and thus produce an unbiased estimate of the treatment effect in switchers. The prognostic characteristics considered here are similar to those considered in the MSM/IPCW method previously described, but for the Two-Stage method, data are only required on these at the time of disease progression, not beyond.

3. (3) Third, the treatment effect associated with switching is used to “shrink” the survival times observed in switchers, to provide estimates of the survival times that would have been observed in the absence of switching (Supplementary Figure 2c). The adjusted control group survival times are then compared with observed experimental group survival times to obtain an estimate of the adjusted treatment effect.

This approach is reliant on switching occurring after a specific disease-related time-point, so that a secondary baseline can be defined. At this time-point, the assumption of “no unmeasured confounders” must hold (information must be available on all patient characteristics that are associated with survival), creating similar challenges to those discussed for this assumption in the previous section. A further limitation of the Two-Stage method is that it only adjusts for differences between switchers and nonswitchers at the time of the secondary baseline: it does not capture any changes in patient characteristics that may occur between the secondary baseline and the time of switch. Hence, switching must occur soon after the secondary baseline, otherwise differences between switchers and nonswitchers may not be adequately adjusted for.

The Two-Stage adjustment method is prone to convergence issues if trial sample sizes are small, particularly if there are several important prognostic characteristics. Also, estimates of the treatment effect in switchers will be prone to error if there are very few control group patients who do not switch. However, research has suggested that the Two-Stage adjustment method is less sensitive to these issues than the MSM/IPCW (Reference Latimer, Abrams and Lambert12;Reference Latimer, Abrams, Lambert, Morden and Crowther13).

A further potential issue with the Two-Stage method arises around something called “re-censoring.” It is not discussed here, due to its complexity, but it is discussed at length elsewhere (Reference Latimer, Abrams and Lambert2;Reference White, Babiker, Walker and Darbyshire17), and is summarized in the Supplementary Material for this study. Importantly, the Two-Stage method can be applied with or without re-censoring and either approach could be prone to bias, depending upon the characteristics of the trial and the treatment in question.

Rank Preserving Structural Failure Time Models (RPSFTM)

Like the Two-Stage method, the RPSFTM estimates survival times that would have been observed had treatment switching not occurred (Reference Robins and Tsiatis18). The method involves three steps.

1. (1) First, for each patient, survival is split into two segments: time spent “on” the experimental treatment (referred to as T _{On i} ), and time spent “off” the experimental treatment (referred to as T _{Off i} ), defined according to treatment initiation and discontinuation times for each patient. In Supplementary Figure 3a, this is illustrated for the fictitious RCT previously introduced.

2. (2) The second step is reliant upon two key assumptions. It is assumed that if no patients in either trial group had received the experimental treatment, the average survival time in the two groups would have been equal because the two groups were created through randomization. It is further assumed that all patients, whether in the control or experimental group, receive the same degree of benefit (treatment effect, or what is referred to as e ^{− ψ₀} in the formula presented in Supplementary Figure 3b) from their time on experimental treatment, known as the “common treatment effect” assumption. Survival is, therefore, a simple function of time on and off the treatment and the treatment effect (Supplementary Figure 3b).Given these assumptions, a process called g-estimation is used to identify the treatment effect. G-estimation essentially amounts to a grid search of possible values for the treatment effect. Using the formula presented in Supplementary Figure 3b, for each patient the inverse of the treatment effect (i.e., e ^ψ₀ ) is applied as a factor to the time spent on treatment, to nullify the effect of having received the treatment, hence providing untreated survival times (referred to as U_i in Supplementary Figure 3b). Once this has been done for all patients, untreated survival times are compared between randomized groups; the “true” value of the treatment effect is the one that results in equal untreated survival times between the randomized groups.

3. (3) Once the “true” value of the treatment effect has been identified, adjusted survival times can be calculated for the control group and compared with the observed experimental group survival times to obtain an estimate of the treatment effect adjusted for treatment switching (Supplementary Figure 3c).

The assumption that randomization has “worked” should not be problematic in an RCT, and is also required by standard ITT analyses. The common treatment effect assumption may be more problematic, if the capacity to benefit from a treatment is dictated by when it is received. In oncology trials, treatment switching often occurs after some defined point of disease progression, and it may not be plausible to assume that switchers will attain the same benefit from treatment (relative to the time it is taken for) as those who received the treatment immediately upon randomization. More complex versions of the RPSFTM, which allow the common treatment effect assumption to be relaxed, have proven unsuccessful (Reference White, Babiker, Walker and Darbyshire17;Reference Robins and Greenland19;Reference Yamaguchi and Ohashi20).

The structure of the RPSFTM model suggests that patients must be either “on” or “off” treatment, if the control treatment is active this may be problematic because it may be possible to be on treatment x, on treatment y, or off treatment. The RPSFTM can still be applied in these circumstances, but requires additional assumptions (Reference Latimer, Abrams and Lambert2).

Simulations have shown that the RPSFTM performs extremely well when the common treatment effect assumption holds (Reference Morden, Lambert, Latimer, Abrams and Wailoo11–Reference Latimer, Abrams, Lambert, Morden and Crowther13). Also, RPSFTMs are less sensitive to issues arising from small patient numbers than the Two-Stage adjustment and IPCW methods (Reference Latimer, Abrams and Lambert12;Reference Latimer, Abrams, Lambert, Morden and Crowther13). However, problems can occur and the g-estimation procedure may not work well with very small patient and event numbers. Like the Two-Stage method described in the previous section, the RPSFTM can be applied with or without re-censoring (see the Supplementary Materials for information on re-censoring).

Using Adjustment Analyses and Other Relevant Information in Decisions

Although decision makers appear to be reluctant to place reliance on methods that they and/or their stakeholders do not fully understand, they appear to accept that ignoring the likely effects of switching in trials may lead to underestimation of the extent of benefits and possible harmful effects and, in some cases, decisions that deny patients access to effective products. There is, therefore, an interest in using adjustment analyses as an indicator of the likely extent of confounding that switching gives rise to, and to at least consider such analyses in an overall judgement of the likely comparative effectiveness and cost-effectiveness of a new treatment. Those making such judgements may also find it helpful to see data from sources outside the pivotal trial.

Sources of relevant data from outside the main trial include registry data (retrospective or prospective alongside the trial) or data from other trials that may inform estimates of long-term survival unaffected by switching (Reference Latimer, Abrams and Lambert2;Reference Drummond, Evans and LeLorier22). External data from relevant RCTs unaffected by treatment switching that included the same endpoints as the trial under investigation are likely to be particularly useful.

It was recognized in discussions that using external or observational data is likely to give rise to methodological issues, such as population and protocol differences and the age of the data. But such analyses may still represent useful supporting evidence, particularly if patient-level data can be obtained such that adjustments can be made for differences in patient characteristics. Again, it was clear from discussions at the Workshop that the acceptability of combining data from multiple sources varied between stakeholders, reflecting concerns regarding validity and relevance.