Background and Objectives
Over the past decade, the continuous market entry of new therapies, which are either high volume—for treating many patients (i.e., antidiabetic agents) or high cost—for a single treatment course (i.e., oncology therapies), has escalated pharmaceutical spending (Reference Permanand and Bak Pedersen1). It was recently reported that pharmaceutical spending accounts for almost 20 percent of the total health expenditure in OECD countries, and because funding from governments and social insurance schemes plays the largest role in pharmaceutical purchasing, this rise bears significant implications for the budget of health systems (2).
The growing healthcare expenditure poses pressures for pharmaceutical manufacturers to demonstrate real-world value for money beyond that of safety and efficacy and simultaneously for national healthcare payers to engage in strategic pricing and reimbursement policies that ensure patients’ access to new therapies while optimizing budget impact (Reference McCabe, Bergmann, Bosanquet, Ellis, Enzmann and von Euler3;Reference Taylor, Drummond, Salkeld and Sullivan4). Although most new products are assessed as part of Health Technology Assessment (HTA) processes in many countries, the data available on the cost-effectiveness of high-cost therapies, particularly in oncology, are severely lacking at the time of product launch (Reference Garrison, Towse, Briggs, De Pouvourville, Grueger and Mohr5). Uncertainties arise due to the often immature evidence available from controlled trials on the real-world clinical outcomes of newly launched pharmaceuticals, meaning that the benefits of a new product cannot be fully estimated at drug launch; uncertainties may be present around treatment eligibility of patient subgroups, generalizability of trial results to clinical practice, the use of surrogate outcome measures instead of “hard” end points and subsequent transferability of surrogate outcomes used in trials to real-world studies (Reference Ferrario and Kanavos6). As these challenges can lead to delayed reimbursement decisions and patient access, manufacturers and payers are seeking ways to collaboratively manage the market entry of new pharmaceutical products and mitigate risks related to premature evidence (Reference Wilsdon and Barron7;Reference Carlson, Sullivan, Garrison, Neumann and Veenstra8); one way to achieve this has been through the introduction of contractual arrangements between the two parties, referred to as Managed Entry Agreements (MEAs) or Risk Sharing Agreements (RSAs).
MEAs are used in many countries worldwide and primarily in Europe, in accordance with country-specific governance and preferences around evidence requirements and evaluation for new medicines (Reference Pritchett, Wiesinger, Faria-Billinton, Brown, Murray and Stoor9). Cross-country differences in HTA assessment requirements have led to a well-documented disparity in the respective risk-sharing practices followed by countries (Reference Ferrario and Kanavos6). A review of MEAs in the EU showcased that only for two drug-indication pairs in the whole sample, an MEA was applied to all six study countries, and even between these, there was variation in the type of MEAs applied (Reference Pauwels, Huys, Vogler, Casteels and Simoens10).
Despite the growing interest of healthcare systems and manufacturers in the use of MEAs over the past decade, there is still a knowledge gap in the drivers of this variation, partially due to a lack of transparency in the negotiation from both parties (Reference Piatkiewicz, Traulsen and Holm-Larsen11).
Even though the literature has concluded that countries indeed differ in their MEA implementation practices, with MEAs being highly specific to the HTA decision-making context in which they operate, the current body of relevant literature arises mainly from secondary evidence and remains largely descriptive in nature (Reference Carlson, Sullivan, Garrison, Neumann and Veenstra8;Reference Ferrario and Kanavos12–Reference Adamski, Godman, Ofierska-Sujkowska, Osińska, Herholz and Wendykowska14). Therefore, we aimed to (i) make a methodological contribution to existing research on determining whether the uptake of MEAs differs between countries and (ii) if so, to understand whether specific HTA decision-making uncertainties and considerations play some role in determining such differences.
Methods
Sample Selection
A retrospective analysis of HTA appraisals on all oncology drugs (i.e., all L01 molecules, based on the Anatomical Therapeutic Chemical classification) which obtained regulatory approval by the European Medicines Agency (EMA) and by the Therapeutic Goods Administration (TGA) in Australia between 1 January 2009 and 15 June 2018 (at the drug-indication pair level) in Australia (AUS), England (ENG), Scotland (SCOT), and Sweden (SE). Oncology was selected as our study therapeutic class because it has been documented to be the therapeutic class with the largest proportion (38 percent) of implemented MEAs; it is also the therapeutic class where MEAs continue to be increasingly implemented (Reference Ferrario and Kanavos6). Study countries were selected because they all implement MEAs, they all have long-established HTA policies and processes to guide their coverage decisions, they have both a publicly available list of MEAs and publicly available HTA reports that provide a sufficient level of information for the purposes of this analysis, and they use similar criteria in their decision-making processes (i.e., clinical and/or cost-effectiveness), allowing for comparability across agencies (Reference Nicod and Kanavos15).
The sample used for this analysis comprises a small part of a larger sample of drugs studied for a different, broader project on HTA. Nevertheless, the aim/scope of that project is not related to that of our study and neither is the methodology we used for data analysis and management. The only common aspects between the two studies relate to the overarching framework used for data collection, as well as the classification and validation of dimensions studied, as described below.
Methodological Framework for Data Collection
The methodology underpinning the data collection process was based on the literature (Reference Nicod and Kanavos15;Reference Cerri, Knapp and Fernández16) where it is suggested that the final outcome of an HTA appraisal is driven by (i) the clinical and cost-effectiveness evidence submitted (i.e., clinical trial design and end points, safety, economic models, and comparators) and (ii) the interpretation of this evidence, influenced by the perception of uncertainty around this evidence, by setting-specific preferences on risk and other social value considerations.
For the purposes of our analysis, a simplified methodological framework was adopted based on the assumption that the impact of the clinical and economic evidence submitted (Stage 1) on the final decision outcome (Stage 4) is captured through the respective uncertainties that this evidence has raised or not raised (Stage 2). Therefore, the final HTA outcome (Stage 4) is a function of the uncertainties raised (Stage 2) and other social value and system-specific considerations (Stage 3) (Figure 1).
Methodological framework on the analysis of the HTA process and variables included therein. Note: ICER, Incremental Cost-Effectiveness Ratio; SVJs, Social Value Judgments. Source: The authors based on the literature (Reference Nicod and Kanavos15;Reference Cerri, Knapp and Fernández16).
The HTA process was divided in four different stages corresponding to: (1) the evidence submitted and appraised (e.g., trial type, clinically meaningful end points; response rate/disease progression/safety end points, comparators, Incremental Cost-Effectiveness Ratio (ICER) range and economic model used), (2) the interpretation of this evidence/uncertainties raised (i.e., clinical and economic evidence-related uncertainties around clinical benefit and study/research design and those around economic modeling and cost-effectiveness, respectively), (3) Social Value Judgments (SVJs) and HTA system-specific considerations (i.e., additional dimensions of societal value that a new technology adds, beyond its clinical evidence/benefit and cost-effectiveness such as the innovation and administration advantage it offers, value dimensions specific to the disease area the technology addresses, such as its severity, rarity, and unmet need, or whether it is a condition toward the end of life, where the benefit of a treatment is valued more highly, and/or system-specific considerations such as the use of a single or multiple technology appraisal (MTA) in England that shape decision-making processes for each study country), and (4) the final decision outcome, classified as (i) L = List (i.e., positive HTA recommendation), (ii) LWC = List with conditions, where the technology has been accepted with restrictions but which are not classified as MEAs (e.g., a product should be used in a subpopulation of its licensed indication, and/or it should be used in a second line or higher line of therapy, and/or it should be used in a specific dose only, and/or it requires monitoring, and/or it requires prescription by a specialist), (iii) LWCMEA = Listed with conditions/restrictions including, among others (if any), at least one classified as an MEA (e.g., simple discount, free stock, rebate, patient access scheme, commercial access agreement coverage with evidence development and/or additional data collection), and (iv) DNL = Do not list (i.e., negative HTA recommendation). We preferred a four-category outcome variable over the three-outcome variable traditionally used in the HTA literature (i.e., listed, listed with conditions, rejected) as it better reflects the multiple coverage options available when studying conditional/restricted HTA decisions. As L and DNL decisions would, by definition, not lead to some kind of a condition/restriction, for the purposes of this analysis, we studied only those drug-indication pairs with a conditional/restricted recommendation decision (i.e., LWC or LWCMEA).
Data Collection
SVJs and uncertainties were classified and defined based on the literature (Supplementary Appendix, Table 1) (Reference Ferrario and Kanavos12;Reference Nicod and Kanavos15;Reference Angelis and Kanavos17;Reference Nicod and Kanavos18), and the classification was also discussed and validated between the authors and external referees. Data on the above stages per drug-indication pair in all study countries were extracted only from the official, publicly available HTA appraisals, which were published in the Web sites of the respective HTA bodies (i.e., PBAC (AUS), NICE (ENG), SMC (SCOT)), TLV (SE)) (Supplementary Appendix, Table 2); other relevant sources of data, such as the county councils’ group on new drug therapies (NLT) in Sweden were not searched. Where needed, searches were conducted in local languages (English and Swedish) to improve the accuracy and comprehensiveness of the extraction. The data extracted were fed into a database stratified by an HTA agency to describe and classify MEAs across the respective HTA bodies and ultimately facilitate data analyses. Data collection was undertaken between June and November 2018 and only the final HTA recommendation reports available for the drug-indication pairs studied were used for data collection (Supplementary Appendix, Table 3).
Data Analysis
For the first part of our hypothesis (i.e., determining whether the uptake of MEAs differs between countries), Pearson's chi-squared (χ 2) test of independence was used to test for differences in the restricted outcome (LWC or LWCMEA) between agencies. Cohen's kappa scores (κ) of cross-country agreement levels were also measured as an additional robustness check.
Agreement between agencies was measured based on whether conditional/restricted HTA outcomes across agencies included a form of MEA as a condition (or not); this allowed a comparison of the observed inter-rater agreement with the agreement expected by chance, ranging from poor (κ = 0) to perfect (κ = 1), with negative values of κ corresponding to cases where the inter-rater agreement was even less than that expected by chance (Reference Nicod19).
Finally, for the second part of our research question (i.e., understand whether specific HTA decision-making uncertainties and considerations play some role in driving divergent LWCMEA decisions between countries), variables under the HTA appraisal (Figure 1) were initially analyzed by means of descriptive statistics, including percentages and crosstabulations (Excel® 2013). Assuming that all categories of uncertainties and SVJs were applicable to all drug-indication pairs studied, these were treated and coded as binary variables based on whether they have been raised and considered (or not), respectively, in the HTA report for each drug-indication pair. As such, bivariate analyses were also performed, using Pearson's chi-squared (χ 2) test of independence, to identify which of these variables drive differences between the LWC and LWCMEA sample overall, and across agencies. For the former comparison, where uncertainty and SVJ dimensions had small sample sizes (i.e., five or less observations), the Fisher's exact test was also performed as a robustness check. The SPSS® (v.24.0) was used to perform statistical tests and measure inter-rater agreement.
Results
Sample Characteristics
A total of 74 molecules were studied across the four countries, corresponding to a total sample of n = 296 drug-indication pairs. Of these, 7 percent (n = 21) were Listed (L), 9.5 percent (n = 28) were Not Listed (DNL), and 63 percent (n = 186) were Listed with at least one type of condition/restriction (LWC or LWCMEA). Other outcomes included drug-indication pairs that were “Not Assessed” (10 percent, n = 29), “Not submitted” (4 percent, n = 12), or formed only an “Advice or Health Economic Report” (6.5 percent, n = 20) (Table 1).
Differences in HTA variables studied between conditional/restricted recommendation decisions (LWC and LWCMEA) and their respective P values
Note: LWC, Listed With Criteria; LWCMEA, Listed With Criteria, including an MEA; NICE, National Institute of Clinical Excellence; SMC, Scottish Medicines Consortium; PBAC, Pharmaceutical Benefits Advisory Committee; TLV, The Dental and Pharmaceutical Benefits Agency.
Of the conditional/restricted HTA outcomes (n = 186), 88 percent (n = 163) were LWCMEA overall, but only 17 percent (n = 32) were LWCMEA across all countries for the same molecules. This was further emphasized by the Cohen's kappa scores measuring the level of inter-rater agreement between agencies across their LWCMEA outcomes. The scores ranged from −.29 to .33 (Table 2), demonstrating a poor-to-moderate agreement. More precisely, it was shown that only SCOT and SE had a moderate agreement, whereas the rest of the countries had a poor or even negative agreement between them. Cross-country differences in MEA utilization were further strengthened by results of the χ 2 test, which showed statistically significant differences between the study countries in terms of their LWCMEA recommendation decisions (P < .05).
Κ scores (k, [95 percent CI]) of inter-rater agreement in the commonly assessed and conditional/restricted HTA outcomes across countries
Note: NICE, National Institute of Clinical Excellence; SMC, Scottish Medicines Consortium; PBAC, Pharmaceutical Benefits Advisory Committee; TLV, The Dental and Pharmaceutical Benefits Agency.
a Scores were generated only with the mutual drug-indication pairs that were both commonly assessed and listed with conditions (LWC or LWCMEA) among all study countries (i.e., 15 molecules commonly assessed between England, Scotland, Australia, and Sweden and 34 molecules commonly assessed and listed with conditions between England, Scotland, and Australia.
Clinical and Economic Uncertainties
Among all clinical uncertainties raised, those that seemed to differ distinctly in proportion between LWC and LWCMEA were population generalizability (13% vs. 31%, respectively), followed by relevance to clinical practice (22% vs. 37%) and study design (35% vs. 51%) (Table 1). Uncertainties around all other clinical evidence aspects were raised at a nearly equal proportion between the LWC and LWCMEA sample, for example, clinical evidence submitted (48% vs. 47%, respectively), clinical benefit (65% vs. 72%), and clinical comparator (35% vs. 29%). Overall, it was shown by χ 2 tests, and where applicable, by the Fisher's exact test, that there were no statistically significant differences between LWC and LWCMEA groups in terms of clinical uncertainties (Table 1).
Clinical uncertainties did not drive any statistically significant differences between countries within the LWC sample. Nevertheless, in the LWCMEA sample, agencies differed significantly in raising clinical uncertainties around study design (P < .05), clinical comparator (P < .05), population generalizability (P < .001), and relevance to clinical practice (P < .0001) (Figure 2).
Cross-country variation in clinical/economic uncertainties and SVJs raised and considered respectively by HTA agencies for drug-indication pairs approved with MEAs. Note: ENG, England; SCOT, Scotland; AUS, Australia; SE, Sweden; QoL, Quality of Life. Source: The authors.
Differences between the LWC and LWCMEA groups were more prominent when studying economic uncertainties. Those that underpinned significant differences included utilities (4% vs. 42%, P < .0001; Fisher's exact significance), followed by economic comparator (0% vs. 22%, P < .01; Fisher's exact significance) and cost-effectiveness (39% vs. 72%, P < .01) (Table 1).
Furthermore, uncertainties around utilities (P < .0001), economic comparator (P < .01), cost-effectiveness (P < .01), modeling (P < .0001), model type (P < .05), and costs included (P < .0001) also drove differences between agencies within the LWCMEA group (Figure 2). Finally, only cost-effectiveness (P < .05) generated statistically significant differences among the agencies within the LWC sample.
Social Value Judgments
There was no statistical difference in the likelihood that most categories of SVJs were considered for drugs that were listed on the basis of LWC versus LWCMEA. Exceptions to this were innovation (9% vs. 53%, P < .0001; Fisher's exact significance), administration advantage (9% vs. 33%, P < .05; Fisher's exact significance), and special considerations (30% vs. 56%, P < .05) (Table 1). In contrast, within the LWCMEA sample, statistically significant differences were observed across countries in the likelihood of considering most SVJs, including unmet need (P < .0001), special considerations (P < .0001), impact on society (P < .01), impact on Quality of Life (QoL) (P < .0001), impact on emotional and functional burden (P < .0001, respectively), severity of disease (P < .0001), innovation (P < .0001), and administration advantage (P < .0001) (Figure 2). Finally, only the last three SVJs seemed to drive statistically significant (P < .05) cross-country differences in the LWC sample too.
Discussion
We demonstrated significant disparities in the conditional/restricted recommendations across all cancer drugs appraised by four HTA agencies between 2009 and 2018. More precisely, we demonstrated a poor level of agreement in MEA implementation across countries as indicated by the kappa scores. Our results suggest that the countries followed different strategies in dealing with the risk/uncertainty arising from the respective evidence submitted by manufacturers on new oncology therapies.
Diverging MEA outcomes between countries were influenced heavily by economic evidence uncertainties including those around cost-effectiveness, utilities, and costs included in the economic model, highlighting agency-specific preferences on cost-effectiveness thresholds and evidentiary requirements for economic modeling. Similar findings around the importance of economic considerations, and notably, the criterion of cost-effectiveness in determining the final HTA recommendation, have been described elsewhere (Reference Nicod19–Reference Maynou-Pujolras and Cairns21).
Clinical evidence uncertainties were less influential than economic toward listing a drug with an MEA; this was not surprising because in many cases it has been demonstrated that uncertainties around the strength of clinical evidence and benefit often lead to rejections commonly across agencies, without allowing any flexibility for conditions or funding negotiations (Reference Nicod and Kanavos15;Reference Nicod19). However, clinical uncertainties related to setting-specific characteristics (i.e., relevance of the technology in question and of the clinical comparator to the country/region-specific clinical practice, and/or the generalizability of trial population to the setting-specific population) were found to play a role in cross-country variation within the LWCMEA sample. This finding confirms that some agencies might place a greater emphasis on evidence related to clinical practice and trial population compared with other agencies (Reference Nicod and Kanavos15). It follows that uncertainties around these factors may also play a role in the uptake of risk-sharing negotiations across specific countries.
In terms of SVJs, our findings suggest that most social value considerations determined cross-country differences within the LWCMEA group only, with the exception of innovation, administration advantage, and severity of disease, which underpinned variation within the LWC group too. This observation highlights that considerations around innovation and burden of disease might be crucial in allowing greater flexibility toward funding with conditions/restrictions in general, whereas considerations around impact on QoL, societal impact, emotional/functional burden, as well as other special considerations (i.e., end-of-life criteria) could be influential specifically toward funding with an MEA as a restriction.
Indeed, the HTA literature has recognized that factors such as the burden of disease the treatment addresses, aspects of the treatment's innovation level, but also the wider socioeconomic implications of the treatment largely affect the perceived value of new medical technologies (Reference Kanavos and Angelis22). Similarly, a number of setting-specific decision modulators (e.g., the SMC modifiers, the NICE's end-of-life criteria, and the human dignity principle for TLV) can contribute to a greater flexibility toward the acceptance of uncertainty or higher and uncertain ICERs (Reference Nicod and Kanavos18). Nevertheless, as shown in Figure 2, the extent to which the above factors are considered across countries in their LWCMEA recommendation decisions can fluctuate significantly, whereas even in countries where these factors are taken into account, they are not necessarily reported in their assessment processes.
Ultimately, this links to discussions about the role of MEAs in the implementation of Value-Based Pricing (VBP) policies through negotiations that enable weight-adjustment of cost-effectiveness thresholds for new medicines that tackle diseases with a higher burden, demonstrate greater therapeutic innovation, and/or have wider societal benefits, such that they reflect any of these additional elements. For example, experience from TLV has shown that in Sweden, risk-sharing agreements indeed complement the VBP system for out-patient drugs and enable stakeholders to mitigate different types of uncertainty (Reference Andersson, Svensson, Persson and Lindgren23). Of course, greater clarity on the long-term outcomes of MEAs is also key to understanding the feasibility of MEA negotiations as a tool for the efficient enactment of VBP policies.
To the best of our knowledge, this is the first systematic analysis that confirms cross-country differences in the uptake of MEAs and provides an understanding of the HTA decision-making variables that might influence such differences. Similar, largely descriptive studies have also identified differences in the design and implementation of MEAs across countries but still lack an in-depth analysis and transparency around the HTA determinants of MEAs (Reference Ferrario and Kanavos6;Reference Carlson, Sullivan, Garrison, Neumann and Veenstra8;Reference Piatkiewicz, Traulsen and Holm-Larsen11;Reference Morel, Arickx, Befrits, Siviero, van der Meijden and Xoxi13;Reference Gonçalves, Santos, Silva and Sousa24). To date, there have been no best-practice guidelines in MEA implementation, and only a few scientific papers suggest some related principles, such as the proposal by KCE in Belgium for good practice in (performance-based) MEAs (Reference Vogler, Paris and Panteli25). As such, results from this analysis contribute to shedding light on the rationale/strategies behind the implementation of MEAs across countries and, therefore, facilitate policy relevant research on the creation of implementation guidelines and/or regulations on “risk-sharing” policies. Finally, providing a transparent, evidence-based description of the HTA decision-making variables that can typically influence an approval with MEAs could be applied in practice by policy makers to facilitate/accelerate HTA decision making and, therefore, allow for more timely reimbursement decisions and consequently more timely access to new, high-cost medicines.
Limitations
This research contributes to an improved understanding of the potential factors that drive conditional/restricted HTA recommendations with versus without an MEA and why these two outcomes might differ significantly between countries for the same drug, presenting with similar clinical evidence across countries. Nevertheless, our findings should be interpreted with caution, because there are certain limitations in our analysis that have hindered the accuracy and robustness of our results.
Firstly, variables under Stage 1 of the HTA process (Figure 1), such as the type of clinical evidence submitted or the ICER submitted, have not been included in this analysis based on the assumption that their influence on determining an LWC or LWCMEA outcome will be captured through their respective uncertainties and whether these were raised or not in the decision-making process. Because variables under both stages have been found to have an impact on the final HTA recommendation (Reference Nicod and Kanavos15;Reference Andersson, Svensson, Persson and Lindgren23), future analyses could include these variables as a robustness check.
Additionally, as budget impact is taken into account only by the Pharmaceutical Benefits Advisory Committee (PBAC), a budget impact-specific uncertainty was not considered due to the lack of comparability across countries. Nevertheless, any budget impact-related concerns raised by the PBAC would have already been reflected in the “cost”- and “modeling”-related uncertainties.
Furthermore, as the calculation of the κ scores required the assumption of paired observations to be met, we performed the calculation only on those drug-indication pairs that were assessed by all four agencies, thus reducing the available sample size significantly. As these analyses could have been more robust if the sample size was increased, it is suggested that a future replication of this study augments the sample size through the inclusion of molecules from additional therapeutic areas. Similarly, as our analysis covered medicines reimbursed only at the national level, future analyses could also account for technologies negotiated at the hospital level.
Finally, advanced statistical modeling was not used at this stage, as our aim was to understand which of the variables play at least some role in driving key differences in the conditional/restricted HTA recommendation decisions between countries. However, in subsequent analyses, the key variables identified herein can be included in a multinomial logistic regression model to also understand their level of impact on divergencies between the conditional/restricted HTA recommendation decisions and the types of MEAs in place across countries.
Overall, it is important that the findings reported here are interpreted with caution, given that the data collected and analyzed were sourced from publicly accessible reports that may, in some instances, represent amended versions of the assessment process to preserve manufacturers’ commercial sensitivities and as such, may not present an absolute reflection of the committee's considerations.
Conclusions and the Way Forward
MEAs are implemented globally and particularly in oncology, to address uncertainties arising from the high cost and simultaneous immature clinical evidence of new, innovative pharmaceuticals. We showed that the uptake of MEAs across countries for the same drugs might differ substantially, and it is subject to setting-specific evidentiary requirements on economic modeling, the comparators, costs, and utilities included therein but primarily also subject to preferences on social value considerations, such as the socioeconomic and QoL impact of the treatment appraised, as well as setting the specific burden of disease. A better understanding of the criteria that determine MEA utilization across countries is fundamental for future research aimed at informing country-specific, “best-practice” guidelines for successful MEA negotiations.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0266462321000039.
Acknowledgments
We are grateful to the following experts for their invaluable collaboration in the data collection and validation process: Anna-Maria Fontrier, Mackenzie Mills, and Erica Visintin.
Funding
The authors received no specific funding for this work.
Open access
