Hostname: page-component-848d4c4894-75dct Total loading time: 0 Render date: 2024-06-01T23:43:44.324Z Has data issue: false hasContentIssue false

The role of quantitative bias analysis for nonrandomized comparisons in health technology assessment: recommendations from an expert workshop

Published online by Cambridge University Press:  20 November 2023

Thomas P. Leahy*
Affiliation:
Putnam Associates LLC, Westport, Ireland
Isabelle Durand-Zaleski
Affiliation:
AP-HP, Health Economics Research Unit, Department of Public Health, Henri Mondor Hospital, Paris, France Methods, UMRS 1153, French National Institute of Health and Medical Research, Paris, France Faculty of Medicine, Université Paris Est Creteil, Creteil, France
Laura Sampietro-Colom
Affiliation:
Health Technology Assessment (HTA) Unit, Hospital Clinic of Barcelona, Barcelona, Spain
Seamus Kent
Affiliation:
Flatiron Health UK, St Albans, UK
York Zöllner
Affiliation:
Department of Health Sciences, HAW Hamburg, Hamburg, Germany
Doug Coyle
Affiliation:
School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada
Gianluigi Casadei
Affiliation:
Medical Biotechnology and Translational Medicine, University of Milan, Milan, Italy
*
Corresponding author: Thomas P. Leahy; Email: thomas.leahy@putassoc.com

Abstract

The use of treatment effects derived from nonrandomized studies (NRS) in health technology assessment (HTA) is growing. NRS carry an inherently greater risk of bias than randomized controlled trials (RCTs). Although bias can be mitigated to some extent through appropriate approaches to study design and analysis, concerns around data availability and quality and the absence of randomization mean residual biases typically render the interpretation of NRS challenging. Quantitative bias analysis (QBA) methods are a range of methods that use additional, typically external, data to understand the potential impact that unmeasured confounding and other biases including selection bias and time biases can have on the results (i.e., treatment effects) from an NRS. QBA has the potential to support HTA bodies in using NRS to support decision-making by quantifying the magnitude, direction, and uncertainty of biases. However, there are a number of key aspects of the use of QBA in HTA which have received limited discussion. This paper presents recommendations for the use of QBA in HTA developed using a multi-stakeholder workshop of experts in HTA with a focus on QBA for unmeasured confounding.

Type
Method
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

All authors are contributed equally.

References

Frampton, GM, Fichtenholtz, A, Otto, GA, et al. Development and validation of a clinical cancer genomic profiling test based on massively parallel DNA sequencing. Nat Biotech. 2013;31(11):10231031.CrossRefGoogle ScholarPubMed
Qin, B-D, Jiao, XD, Liu, K, et al. Basket trials for intractable cancer. Front Oncol. 2019;9:229.CrossRefGoogle ScholarPubMed
Ramagopalan, S, Leahy, TP, Ray, J, et al. The value of innovation: association between improvements in survival of advanced and metastatic non-small cell lung cancer and targeted and immunotherapy. BMC Med. 2021;19(1):17.CrossRefGoogle ScholarPubMed
Patel, D, Grimson, F, Mihaylova, E, et al. Use of external comparators for health technology assessment submissions based on single-arm trials. Value Health 2021;24(8):11181125.CrossRefGoogle ScholarPubMed
Concato, J, Corrigan-Curay, J. Real-world evidence-Where are we now? New Engl J Med. 2022;386(18):16801682.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence. Sotorasib for previously treated KRAS G12C mutation-positive advanced non-small-cell lung cancer [TA781]. 2022.Google Scholar
National Institute for Health and Care Excellence. Andexanet alfa for reversing anticoagulation from apixaban or rivaroxaban [TA697]. 2021. Available from: https://www.nice.org.uk/guidance/ta697/evidence.Google Scholar
Canadian Agency for Drugs and Technologies in Health. CADTH Reimbursement Recommendation: Tafasitamab (Minjuvi). 2022.Google Scholar
Vanier, A, Fernandez, J, Kelley, S, et al., Rapid access to innovative medicinal products while ensuring relevant health technology assessment. Position of the French National Authority for Health. BMJ EBM; 2023. Published Online First: 14 February 2023. doi: 10.1136/bmjebm-2022-112091CrossRefGoogle ScholarPubMed
Wieseler, B, Neyt, M, Kaiser, T, Hulstaert, F, Windeler, J. Replacing RCTs with real world data for regulatory decision making: A self-fulfilling prophecy? BMJ. 2023;380:e073100. doi:10.1136/bmj-2022-073100.CrossRefGoogle ScholarPubMed
Lash, TL, Fox, MP, Cooney, D, Lu, Y, Forshee, RA. Quantitative bias analysis in regulatory settings. Am J Public Health 2016;106(7):12271230.CrossRefGoogle ScholarPubMed
Leahy, TP, Kent, S, Sammon, C, et al. Unmeasured confounding in nonrandomized studies: Quantitative bias analysis in health technology assessment. J Comp Eff Res. 2022;11(12):851859.CrossRefGoogle ScholarPubMed
Sammon, CJ, Leahy, TP, Gsteiger, S, Ramagopalan, S. Real-world evidence and nonrandomized data in health technology assessment: using existing methods to address unmeasured confounding? J Comp Eff Res. 2020;9:969972.CrossRefGoogle ScholarPubMed
Gray, CM, Grimson, F, Layton, D, Pocock, S, Kim, J. A framework for methodological choice and evidence assessment for studies using external comparators from real-world data. Drug Saf. 2020;43:623633.CrossRefGoogle ScholarPubMed
VanderWeele, TJ, Ding, P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med. 2017;167(4):268274.CrossRefGoogle ScholarPubMed
Rosenbaum, PR. Sensitivity analysis for certain permutation inferences in matched observational studies. Biometrika 1987;74(1):1326.CrossRefGoogle Scholar
Groenwold, RH, Sterne, JA, Lawlor, DA et al. Sensitivity analysis for the effects of multiple unmeasured confounders. Ann Epidemiol. 2016;26(9):605611.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence. NICE health technology evaluations: The manual (Process and methods [PMG36]). 2022. Available from: https://www.nice.org.uk/process/pmg36/chapter/evidence.Google Scholar
Canadian Agency for Drugs and Technologies in Health. Procedures for CADTH Reimbursement Reviews. 2022.Google Scholar
Akobeng, AK. Understanding randomised controlled trials. Arch Dis Child. 2005;90(8):840844.CrossRefGoogle ScholarPubMed
Kent, S, Salcher-Konrad, M, Boccia, S et al. The use of nonrandomized evidence to estimate treatment effects in health technology assessment. J Comp Eff Res. 2021;10(14):10351043.CrossRefGoogle ScholarPubMed
European Commission. Regulation (EU) 2021/2282 of the European Parliament and of the Council of 15 December 2021 on health technology assessment and amending Directive 2011/24/EU. 2021.Google Scholar
Schneeweiss, S. Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf. 2006;15(5):291303.CrossRefGoogle ScholarPubMed
Leahy, TP, Duffield, S, Kent, S et al. Application of quantitative bias analysis for unmeasured confounding in cost–effectiveness modelling. J Comp Eff Res. 2022;11:861870.CrossRefGoogle ScholarPubMed
Wilkinson, S, Gupta, A, Scheuer, N et al. Assessment of alectinib vs ceritinib in ALK-positive non–small cell lung cancer in phase 2 trials and in real-world data. JAMA Netw Open. 2021;4(10):e2126306.CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence. Corporate document [ECD9]: NICE real-world evidence framework. 2022.Google Scholar
Supplementary material: File

Leahy et al. supplementary material
Download undefined(File)
File 24.9 KB