Treatment switching in a trial occurs when patients in one group switch from the treatment specified for that group to the treatment specified for another group in the trial (most commonly from the control group treatment to the experimental group treatment), or to another treatment that is not part of the original trial protocol. Treatment switching as defined here is sometimes referred to as “crossover,” but we use the term “treatment switching” in this study to avoid confusion with trials that have a crossover design (whereby each patient is intended to receive both treatments and act as their own “control”) (1).
Treatment switching may help to ensure that patients in the trial are not denied access to new treatments when they and/or their doctors believe that those treatments may offer them benefit. But it can lead to challenges in the interpretation of the findings of the trial for the patients and doctors taking part in the trial, for future patients and the doctors who will treat them, for regulators, for payers (by which we mean those in health systems or health insurance plans responsible for deciding whether, and if so how much, the system or plan will pay or reimburse for a drug), and hence for companies developing and marketing new drugs (2).
Treatment switching appears to be increasingly common in trials of some types of cancer drugs. A 2005 study reported that crossover occurred in around 25 percent of all published cancer trials (3), and a recent study reports that crossover was present in trials of over half of the cancer drug appraisals conducted by NICE in England (4). It is giving rise to a range of concerns about how trials of cancer drugs can best be designed, managed, analyzed, and interpreted. Some of these concerns have been identified in scientific papers discussing statistical and/or policy issues around treatment switching, but there has been little formal discussion of the topic within and between the stakeholder groups. The 2014 Adelaide International Workshop sought to address this by bringing together senior representatives of key stakeholders from around the world to discuss the issues, identify where there is agreement on appropriate approaches to address treatment switching in the design and analysis of cancer trials, and identify where further discussion or work is needed.
METHODS
The Adelaide International Workshop was convened by Bellberry, a not for profit organization that exists to promote and improve the welfare of research participants and the quality, efficiency, and effectiveness of research. The Workshop was conducted over 1.5 days in October 2014. Forty-one people attended from patient organizations, regulators, health technology assessment (HTA)/payer bodies, universities (ethicists, statisticians and health economists), and life sciences companies from Australasia, Europe, and North America (the Supplementary Table lists all participants and their affiliations). The program included a range of presentations, with most of the time dedicated to plenary and break-out group discussions. All attendees received in advance a detailed agenda for the meeting, a background paper (5), details of several case studies for discussion, and copies of guidance on treatment switching published or in development by bodies around the world. The development of these materials was overseen by a Scientific Committee (see Supplementary Table for a list of members).
All participants agreed that those attending the Workshop should be free to report anything that was said in the discussions, but not which participants said it. They also agreed that the background paper should be publicly available (see Supplementary Material) but that the detailed case studies, the report for participants of points made in the various sessions, and the slides presented should be available only to participants. These arrangements were designed to protect potentially sensitive information and to promote free and open discussion by ensuring the confidentiality of individuals’ contributions.
This study represents the authors’ attempt to capture what emerged from extended discussions between multiple stakeholders at the Workshop. It has benefited greatly from comments from attendees on drafts of an earlier study but it is not intended to comprise a consensus statement from those present at the Workshop or the organizations they are associated with, or a statement of the policies of any of those organizations. It is offered as a contribution to the debate, with the aim of stimulating further discussions (and hopefully agreement and action) within and, importantly, between the various stakeholder groups, at national and international levels.
RESULTS
Context
It was clear from discussions at the Workshop that a number of wider contextual factors need to be understood when considering treatment switching in cancer trials.
Drug Development Paradigms
There is increasing pressure to achieve faster testing and approval of promising drugs so that patients can get earlier access (6). This is leading to increased use of “fast track” paradigms for regulatory approval (7) and is influencing the design of many clinical development programs. This is often associated with a greater reliance on intermediate end points (such as progression-free survival, PFS) in trials for licensing (8–10). There are also concerns about the possible impact on innovation of any further increases in the already high cost of bringing drugs to market, and this is causing increased scrutiny of the evidence required by regulators and by those responsible for reimbursement. At the same time, there is an increase in targeted therapies with associated challenges in recruiting the numbers of patients required to allow clear answers to be obtained from trials.
Cancer Care
Trial participation is increasingly seen as part of routine care for cancer patients, with many health systems seeking to increase participation and with some using it as a measure of the quality of care (11). While there are schemes in some health systems to allow patients special access (e.g., Therapeutic Goods Administration, 2015) (12), patients may see enrolment in a trial as a way to gain early access to new treatments. Patients living with cancer, particularly those facing a terminal diagnosis with no effective therapeutic options, naturally place high hopes on new treatments. But, while significant improvements in survival have been achieved in a small number of cancers, there are concerns that for many cancers new drugs offer relatively modest improvements in PFS and/or overall survival (OS). Furthermore, the extension in life is often associated with exposure to significant drug toxicity and high financial cost (e.g., Fogo et al.) (13). Patients and their doctors therefore need information on health related quality of life (HRQoL) and outcomes such as tolerability, symptoms, and functioning, as well as PFS and OS. Given this context, treatment switching is usually found in trials involving patients with incurable cancer rather than in studies where long term cure is the goal.
Regulatory Evidence Requirements
Regulators are tasked with assessing the balance of benefit and risk for a drug and granting market access for those where the benefits outweigh the risks. While they regard OS as a key outcome, in some clinical settings, they are prepared to approve a drug on the basis of improvements in PFS, symptoms or HRQoL if those improvements are clinically significant and outweigh any potential harms or adverse effects, and there are grounds to believe that the drug will improve OS (or at least that there will be no detriment in OS) (6;8;9;14).
Payer Evidence Requirements
Payers are tasked with considering the benefit that a drug offers (as assessed by its effects on patient outcomes and the quality of care provided) in relation to its cost, to determine either whether a drug is appropriate for their healthcare system or plan at the price at which the manufacturer offers it, or to determine the price the system will offer the manufacturer given the extent of benefit demonstrated.
Most systems or plans have defined resources to meet a wide and growing range of demands for care. In many systems, the payer is required to consider value for money to promote equity of access to benefit for patients across the system. Many payers are therefore interested in the overall benefit a drug delivers in relation to its cost. In most systems, overall benefit is judged in relation to the length and quality of a patient's life (reflecting both the benefits and adverse effects of a treatment) following treatment (15–18). This means that where a drug is able to demonstrate a benefit in PFS but cannot demonstrate benefit in OS and HRQoL, the manufacturer may be expected to propose or accept a lower price (often substantially lower) than would be agreed for an OS benefit. If the manufacturer does not feel able to accept this, the result may be that the drug is not covered or reimbursed in the system, or the manufacturer decides not to make the drug available at the price the system is prepared to pay. In either case, some or all patients in the system will then be denied access to the drug.
Randomized Controlled Trials and the Implications of Treatment Switching
The Workshop noted that randomized controlled trials (RCTs) are generally accepted as the best way to establish differences between treatment alternatives for the patient populations from which the trial participants are selected. Observational studies can play an important part in interpreting the relevance of trial findings to wider populations and monitoring outcomes in the real world following approval, but they are generally not considered to provide as reliable an indication of treatment effects as RCTs. Thus, RCTs remain key to the assessment of new treatments by regulators, payers, clinicians and patients.
Treatment switching introduces bias into a trial from the point of switching onward. A range of methods is available for analyzing and interpreting the results of trials where treatment switching has occurred. These are discussed in more detail in the companion paper to this study (19). As noted there, key stakeholders express differing degrees of confidence in such methods to adjust for switching. When the assumptions involved in these methods are met and the rates of treatment switching are not too high, published research has demonstrated that some of the methods can provide reliable estimates of “true” treatment effects (i.e., the treatment effect that would have been demonstrated if no treatment switching had occurred). Often, however, the assumptions are not fully met, and/or it is difficult to ascertain whether or not they are met, and/or the proportion of patients switching treatments is high. In practice, decision makers are often reluctant to place confidence in estimates of treatment effects derived from the analysis of trials with treatment switching, particularly when the analytic methods are not well explained or justified, or are based on assumptions that have not been (or cannot be) justified.
The impact of treatment switching depends on when and why it occurs, and on the outcomes of interest. If treatment switching occurs at disease progression, this should generally not affect the ability of a trial to establish differences in PFS between the control and experimental groups (though the knowledge that a patient can switch may introduce bias in not fully blinded trials). Other subsequent end points, in particular OS, adverse effects, potential harms, and other measures relating to HRQoL post-progression will, however, be affected. If treatment switching occurs because of a perceived loss of equipoise resulting from an interim analysis or from information or events outside the trial (e.g. regulatory approval being granted for use in later lines of treatment; results becoming available from other trials of the treatment or of similar agents), the trial's ability to establish differences in all outcomes, beneficial and harmful, will be affected.
Given the impact of switching on the validity of trials and their utility for decision makers, there was extensive discussion at the Workshop of the implications of switching for the ethics and the management of trials.
It was noted that, in general, the primary purpose of conducting medical research is to generate knowledge that will be of benefit to the future treatment of patients—whether those in the research or others with the same condition. Reliable knowledge generation is an important factor in ensuring that clinical research is ethical (20). While this goal can never take precedence over the rights and interests of individual research subjects (21), it was believed that patients consenting to take part in trials of any treatment should understand that the primary purpose of medical research is to generate new knowledge.
Cancer trials raise some particular ethical issues. In conditions with terminal diagnoses, patients may assume that the benefits of an unproven new drug may outweigh any adverse effects or risks and that it is worth trying and/or risking. As noted previously, patients may enroll in a trial of a new cancer drug in the hope of gaining early access to the drug, particularly if there are no valid alternatives available. While there should in theory be equipoise about the relative benefits of treatments being offered to patients in a trial, many patients participating in cancer trials appear to be doing so in the hope of personal benefit during the trial.
Trials need to be conducted so as to provide reliable information to inform the future treatment of all patients with the condition (possibly including the trial participants themselves), while ensuring that those participating in the trial are not disadvantaged and are given the best chance of optimal outcomes at all stages. In trials in general, this balance is achieved by defining in the trial protocol arrangements for interim analysis to establish whether or not equipoise still exists and, if it does not and one treatment has been shown to be superior, to allow all patients to switch to that treatment. But patients in cancer trials may also switch on disease progression and/or when information from outside the trial leads patients and/or clinicians to believe (on whatever grounds) that the experimental treatment is superior. Switching in cancer trials is particularly likely when there are no further active treatments available for patients after progression, or when the drug being used in the experimental arm of the trial has already been granted regulatory approval for use in later lines of treatment (even if the trial in question is investigating earlier lines).
As noted earlier, if the primary outcome of the trial is PFS, then switching at progression will not affect the trial's ability to detect an effect on PFS. It will, however, affect its ability to detect other subsequent benefits and harms that may be of relevance to the patients in the trial as well as future patients. Furthermore, it may not always be in the interests of patients in the trial to switch at disease progression or on demonstration of a benefit of the experimental drug on PFS. Patients vary in their responses to treatments, and a drug that improves PFS will not necessarily improve postprogression survival (PPS) or quality of life (22;23), so that a patient in the control group who is doing well on the control treatment may sometimes be better advised to stay on that treatment rather than switch.
The arguments for and against allowing treatment switching in a trial therefore depend upon the nature and stage of the disease, the nature of the benefits and any harms that are expected from the treatment, the nature of the primary and secondary end points being studied, and the nature of the treatments currently available for patients in the trial and at later stages in their disease. There is no single optimal approach.
At a practical level, treatment switching has become an established part of trials in some areas of cancer. Patients choose whether or not to enroll in trials and are free to withdraw from a trial at any point, and in some situations (particularly where a drug is already approved for other indications), clinicians may be able to prescribe the treatment in the experimental arm of a trial outside the trial if they believe it is in the patient's best interest. Recruiting and retaining patients in trials is becoming increasingly problematic if treatment switching is not permitted. It is therefore important that agreement is reached between patients, clinicians, regulators, payers, and manufacturers on how switching will be addressed in the design and analysis of trials.
Statistical Methods
Attendees at the Workshop discussed various statistical methods for handling treatment switching in the analysis of a trial and, in some cases, combining data from outside the trial in the analysis of, and/or decision making based upon, the trial findings. These issues are developed in detail in Latimer et al. (19). As noted there, there is no single optimal approach to analyzing and interpreting the data from trials where treatment switching has occurred. The choice of methods will depend upon the nature of the trial, the extent of treatment switching and the point(s) at which it has occurred, the extent to which the assumptions of different statistical approaches are met and can be shown to be met within the setting of the trial, and the availability of data to support the application of those approaches.
Proposals for Good Practice in Relation to Treatment Switching
Despite the complexity of the issues involved and the range of perspectives represented at the Workshop, the discussions pointed up a number of ways in which the handling of treatment switching could be better addressed. We bring these together here in some proposals for good practice which we hope will be helpful—in themselves and as a stimulus to further debate that may ultimately lead to more formal guidelines—to those planning new trials and clinical development programs, and to those seeking to manage and analyze existing trials where treatment switching has occurred or is likely to occur. Table 1 provides a summary of recommendations.
Summary of recommendations
Planning Clinical Trials and Clinical Development Programs
a. Companies designing clinical development programs for cancer drugs should discuss with patient advocates, clinicians, regulators, and payers the key outcomes that the program needs to demonstrate to support informed decisions on market approval, reimbursement/coverage, and use of the drug. They should consider carefully the importance to decision makers of evidence on OS, adverse effects and other aspects of HRQoL, and PFS, and ways in which this evidence can best be developed through trials and other studies to meet each stakeholder's requirements.
b. To maximize the chances to address treatment switching at the time of analysis, and hence the value of the trial, trial protocols should define clearly and justify if, when and on what basis treatment switching will be allowed within the trial, in particular whether it will be at some predefined stage of the progression of the patient's disease (or when a patient no longer benefits from treatment, for example, due to toxicity or if maximum benefit has been achieved) or at loss of equipoise following a predefined interim analysis of all the trial data. The protocol should also state how requests from patients to switch treatments in other circumstances will be handled. The approach adopted should reflect the nature and stage of disease of the patients being recruited into the trial, the expected course of the disease within and beyond the trial, the known effectiveness of the control treatment within the trial, and the availability and effectiveness of further treatments beyond the trial. The approach adopted should reflect discussions with patient advocates and with payers as well as with clinicians and regulators.
c. To increase the likelihood of analysis of the trial data being valid and convincing, when a trial protocol allows for or envisages the possibility of treatment switching, it should specify the methods by which it is anticipated the results will be analyzed to take appropriate account of the likely effects of treatment switching, and these methods should be discussed in advance with key decision makers such as regulators, HTA/payer bodies and patient representatives. The methods should reflect the nature and timing of the treatment switching to be allowed and the expected extent of switching. The trial protocol should define how data that are required from within the trial to apply the chosen methods of analysis will be collected. The protocol should also define how all patients will be followed up after they stop their randomized treatment to collect information required to analyze the trial outcomes. An analysis plan should describe what analyses will be undertaken to provide information on the plausibility of the key assumptions associated with the applied adjustment methods. Where switching is to be allowed, and producing statistically significant OS results is the objective, trial sponsors should consider basing sample size calculations on anticipated applications of adjustment methods. See Latimer et al. for a fuller discussion of these issues (19)
d. Clinical development programs should consider studies in addition to new trials that can be used to provide more information on: (i) the likely effect of treatment on key trial outcomes, for comparison or combination with data from the trial to allow an estimate of the “true” treatment effects to be made with more confidence; and/or (ii) outcomes not addressed (or inadequately addressed) as a result of omission from the trial protocol, of the powering of the trial, or confounding caused by treatment switching; and/or (iii) the relevance of the trial results for wider patient populations.
Such studies might include the post-trial follow-up of trial patients, the analysis of patient data from other trials not impacted by treatment switching, and observational studies of cohorts of relevant patients. These external sources of information may be useful in supplementing analyses of the trial data themselves to determine what the effect of the new treatment would have been had treatment switching not occurred.
e. Research ethics committees and others responsible for the review and quality assurance of trial protocols should satisfy themselves that: (i) the definition of trial end points and the approach to treatment switching and analysis of the trial results are clear and appropriate; (ii) the approach reflects discussion with patient advocates, protects the interests of patients in the trial, and will deliver information to improve the treatment of patients in the future; (iii) the arrangements and rationale for treatment switching are made clear in the information patients will be given about the trial when their consent is sought; and (iv) proposals for the collection and analysis of data from within and beyond the trial will allow conclusions to be drawn from the findings of the trial.
Managing Clinical Trials
a. The approach to treatment switching and the way in which this protects the interests of patients in the trial and will deliver information to improve the treatment of patients in the future should be explained and discussed with patients when they are invited to join a trial and asked to give their informed consent.
b. The trial sponsor should maintain regular communication with all clinicians participating in a trial, and clinicians should maintain regular communication with patients in their local trial centers. When information arises from within or outside the trial that participating patients and/or clinicians may see as relevant to their views on the effectiveness for them of the treatments in the different arms of the trial, this should be communicated to and discussed with trial participants. Where possible, this should be accomplished ahead of disclosure to the media or others, to ensure that trial participants receive balanced information that can help them make informed decisions about their best interests and their ongoing participation in the trial, with or without switching treatment, or withdrawal from the trial. In this regard, wherever possible and appropriate, industry and other trial sponsors should have discussions with patient advocates and patient associations. Careful contingency planning may be required, because information, accurate and inaccurate, can spread quickly in some situations.
Analyzing Clinical Trials
a. In the presence of treatment switching that is deemed to materially affect the results of a trial, a standard ITT analysis may be supplemented with analyses that attempt to adjust for switching and which may help to address the particular decision problems faced by different stakeholders.
b. Where adjustment methods are used, trial characteristics and the treatment switching mechanism should be considered alongside the pivotal assumptions associated with each of the statistical adjustment methods to identify which of the analyses (ITT or any of the adjusted analyses) are likely to provide the least amount of bias in each case. Concerns over the validity of methodological assumptions should be addressed as far as possible using the types of analyses described in Latimer et al. (19). Detailed justifications should be provided for chosen methods, and consideration should be given to the direction of bias if methodological assumptions are not satisfied. The sensitivity of results should be tested through application of all methods deemed to be potentially appropriate for each case (if a range of methods can be shown to be potentially appropriate for a particular case, and each provides similar estimates of the treatment effect, decision makers may have more confidence in the results).
c. Decision makers vary as to whether and when they are prepared to consider the results of statistical adjustment methods. Therefore, trial designers and sponsors should wherever possible discuss their proposed approach to analysis with key decision makers to agree upon a preferred approach in the light of the nature of the trial and the switching that has actually occurred, building upon pre-trial discussions and the analysis proposals included in the trial protocol (see above).
d. In circumstances where the impact of treatment switching is likely to be important, statistical adjustment methods are not the only avenue that may be explored when attempting to address treatment switching. Other options exist (see Latimer et al. [19]). These should be discussed with key decision makers as views on their validity and relevance vary.
DISCUSSION AND RECOMMENDATIONS FOR FURTHER ACTION
The Adelaide Workshop was a valuable first step toward identifying the wide range of issues raised by treatment switching in cancer trials, the views of different stakeholders on these, and priorities for further work. There was good agreement on what the challenges are and on the need to address them in the design and analysis of individual trials and through development of more general guidance. There was also a good degree of agreement on the specific strengths and weaknesses of different approaches to analyzing trials where switching has occurred. While all recognized that these methods have important limitations, the level of understanding of these varied and views differed on the extent to which such analyses could and should be used to inform decisions. The Workshop did not reach specific conclusions about when treatment switching should be recommended or discouraged, though there did appear to be scope to reach agreement on situations where it would be more and less appropriate.
On the basis of the Workshop discussions, we suggest the following priorities for further action.
Further Discussion and Agreement on General Approaches and Policies to Address Treatment Switching in Cancer Trials
There is a need for wider discussions to test the analysis of the issues and the proposals for good practice set out in this study through further multi-stakeholder discussions; different stakeholders have different perceptions, expectations, decision rules and requirements, and proposals need to take account of all these considerations if they are to be adopted and be of value in practice. It is particularly important to involve patients and patient advocates in discussions because they are the people for whom and with whom trials and associated studies are done. It is also important to involve payers in the discussions because they are the group for whom treatment switching at present appears to present the greatest challenges. There is a need for payers as a group to discuss the issues and possible solutions to them, including the use of complex statistical methods to analyze the results of trials with treatment switching. There is also a need for payers to be brought into the wider discussions so that their perspective and needs can be better understood by the other stakeholders and so that payers can better understand the patient perspective and the constraints that drug developers and trial designers face.
Development of Guidance on “Good Practice” in the Design and Analysis of Trials and Wider Clinical Development Programs for Cancer Drugs
We recommend work by multi-stakeholder groups to develop more detailed guidance on good practice for: (a) Methods and mechanism for achieving ongoing engagement with patients, clinicians, payers, and other stakeholders in discussions on treatment switching. (b) Approaches to treatment switching for different disease stages, trial end points, and treatment landscapes. (c) Matching the statistical approach to correct for treatment switching to the trial design, and adapting the design of trials and wider clinical development programs to support the analytic approach. (d) Methods for integrating evidence from trials with treatment switching, statistical adjustments to correct for that switching, and data from observational and other studies from outside the trial. (e) Approaches to using information from these sources and techniques in regulatory and coverage decisions.
CONCLUSIONS
The management of treatment switching in trials is of concern to all stakeholders. The perspectives and some of the specific concerns of different stakeholders vary according to the nature of the decisions they make and the systems they work within. Nonetheless, on the basis of the discussions at the Adelaide International Workshop, there appears to be a considerable common ground on approaches to addressing treatment switching in cancer trials.
We hope that the proposals for good practice offered in this study will be helpful to those designing, managing and analyzing trials and wider clinical development programs for cancer drugs. We also hope that they will provide a platform for the development of more formal guidance to inform the work of regulators, payers, industry, trial designers, and other stakeholders.
The Adelaide meeting focused on treatment switching in trials of cancer drugs. Some of the issues considered are specific to cancer and cancer trials, but others would appear to be relevant to trials of treatments for other diseases. It is hoped that the discussion in this study will also be of value to those considering the design and analysis of trials in other areas where treatment switching may be an issue.
SUPPLEMENTARY MATERIAL
Supplementary Table 1 http://dx.doi.org/10.1017/S026646231600009X
CONFLICTS OF INTEREST
C.H. received an honorarium from Bellberry for acting as facilitator of the Adelaide Workshop and for work on drafting a Review of the discussions at the Workshop for attendees, and has received various payments for consultancy services on health technology assessment to governments, health systems and life sciences companies around the world. The University of Sheffield received funding from Bellberry for acting as the scientific secretariat for the Adelaide Workshop. N.L. has provided training courses and consultancy support on the topic of treatment switching for pharmaceutical companies, for which payment has been received. L.S. is a Director of Bellberry Ltd and receives a stipend for director fees. R.W. is a member of the Pharmaceutical Benefits Advisory Committee and Medical Services Advisory Committee for which she receives payment from the Australian Government, but attended the meeting and authored this study in a personal capacity.
Open access
