Skip to main content
Search
Search
Advanced Search
Skip main navigation
Open access
Research Article
4 April 2024

Augmenting external control arms using Bayesian borrowing: a case study in first-line non-small cell lung cancer

Authors: Alessandria Struebing https://orcid.org/0009-0007-1894-3372 [email protected], Chelsea McKibbon, Haoyao Ruan, Emma Mackay, Natalie Dennis, Russanthy Velummailum, Philip He, Yoko Tanaka, Yan Xiong, Aaron Springford, and Mats RosenlundAuthor Info & Affiliations
Volume 13, Number 5
https://doi.org/10.57264/cer-2023-0175
PDFFormats

Abstract

Aim: This study aimed to improve comparative effectiveness estimates and discuss challenges encountered through the application of Bayesian borrowing (BB) methods to augment an external control arm (ECA) constructed from real-world data (RWD) using historical clinical trial data in first-line non-small-cell lung cancer (NSCLC). Materials & methods: An ECA for a randomized controlled trial (RCT) in first-line NSCLC was constructed using ConcertAI Patient360™ to assess chemotherapy with or without cetuximab, in the bevacizumab-inappropriate subpopulation. Cardinality matching was used to match patient characteristics between the treatment arm (cetuximab + chemotherapy) and ECA. Overall survival (OS) was assessed as the primary outcome using Cox proportional hazards (PH). BB was conducted using a static power prior under a Weibull PH parameterization with borrowing weights from 0.0 to 1.0 and augmentation of the ECA from a historical control trial. Results: The constructed ECA yielded a higher overall survival (OS) hazard ratio (HR) (HR = 1.53; 95% CI: 1.21–1.93) than observed in the matched population of the RCT (HR = 0.91; 95% CI: 0.73–1.13). The OS HR decreased through the incorporation of BB (HR = 1.30; 95% CI: 1.08–1.54, borrowing weight = 1.0). BB was applied to augment the RCT control arm via a historical control which improved the precision of the observed HR estimate (1.03; 95% CI: 0.86–1.22, borrowing weight = 1.0), in comparison to the matched population of the RCT alone. Conclusion: In this study, the RWD ECA was unable to successfully replicate the OS estimates from the matched population of the selected RCT. The inability to replicate could be due to unmeasured confounding and variations in time-periods, follow-up and subsequent therapy. Despite these findings, we demonstrate how BB can improve precision of comparative effectiveness estimates, potentially aid as a bias assessment tool and mitigate challenges of traditional methods when appropriate external data sources are available.

Shareable abstract

In #CancerResearch, #RealWorldData can provide external control arms (ECAs) for single arm trials. But are ECAs fit-for-purpose when comparing treatments? We show how historical trials and #BayesianBorrowing might be used to increase confidence in ECAs.

Plain language summary

Researchers and health agencies need accurate and reliable estimates of the relative effectiveness of treatments. In some cases, an ‘external control group’ can be created from pre-existing healthcare records, eliminating the need for a concurrent control and the associated patient burden. However, it can be challenging to find comparable patients with similar characteristics, and comparable information on health outcomes for those patients. Often, there are just not enough patients available in existing records to make a meaningful comparison. Bayesian borrowing can be used to include control arm information from historical studies, even if these studies differed somewhat from the present study.
In this article, we investigate the ability of external controls and Bayesian borrowing to replicate findings from a randomized controlled trial (RCT – the gold standard). We begin with an RCT and then remove the control group before constructing an external control group using real-world data from electronic health record data. We then use Bayesian borrowing to add information from another historical trial in the same indication to see whether the original RCT results can be replicated, and under what conditions. Our example illustrates common challenges of working with real-world data and provides practical insights for the incorporation of additional data sources when comparing the effectiveness of treatments that have not been compared directly in an RCT.

Supplementary Material

File (supplementary figure 1a.jpeg)
File (supplementary figure 1b.jpeg)
File (supplementary figure 2a.jpeg)
File (supplementary figure 2b.jpeg)
File (supplementary figure 3.jpg)
File (supplementary table 1.xlsx)
File (supplementary table 2.xlsx)
File (supplementary table 3.xlsx)
File (supplementary table 4.xlsx)
File (supplementary table 5.xlsx)
File (supplementary table 6.xlsx)

References

Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Hariton E, Locascio JJ. Randomised controlled trials - the gold standard for effectiveness research: study design: randomised controlled trials. BJOG 125(13), 1716 (2018).
PubMed
Google Scholar
2.
Kempf L, Goldsmith JC, Temple R. Challenges of developing and conducting clinical trials in rare disorders. Am. J. Med. Genet. A 176(4), 773–783 (2018).
PubMed
Google Scholar
3.
Agrawal S, Arora S, Amiri-Kordestani L et al. Use of single-arm trials for US Food and Frug Administration drug approval in oncology, 2002–2021. JAMA Oncol. 9(2), 266–272 (2023).
PubMed
Google Scholar
4.
Mulder J, Teerenstra S, Van Hennik PB et al. Single-arm trials supporting the approval of anticancer medicinal products in the European Union: contextualization of trial results and observed clinical benefit. ESMO Open 8(2), 101209 (2023).
PubMed
Google Scholar
5.
Mack C, Christian J, Brinkley E et al. When context is hard to come by: external comparators and how to use them. Therap. Innovat. Regul. Sci. 54, 2168479019878672 (2019).
Google Scholar
6.
Gagne JJ, Thompson L, O'keefe K, Kesselheim AS. Innovative research methods for studying treatments for rare diseases: methodological review. BMJ 349, g6802 (2014).
PubMed
Google Scholar
7.
Ghadessi M, Tang R, Zhou J et al. A roadmap to using historical controls in clinical trials – by drug information association adaptive design scientific working group (dia-adswg). Orphanet J. Rare Dis. 15(1), 69 (2020).
PubMed
Google Scholar
8.
Thorlund K, Dron L, Park JJH, Mills EJ. Synthetic and external controls in clinical trials - a primer for researchers. Clin. Epidemiol. 12, 457–467 (2020).
PubMed
Google Scholar
•• This primer highlights the current landscape of the use of SCAs and provides an appraisal framework to assess future publications regarding the use of SCAs.
9.
Velummailum RR, Mckibbon C, Brenner DR et al. Data challenges for externally controlled trials: viewpoint. J. Med. Internet Res. 25, e43484 (2023).
PubMed
Google Scholar
10.
Gray CM, Grimson F, Layton D, Pocock S, Kim J. A framework for methodological choice and evidence assessment for studies using external comparators from real-world data. Drug Saf. 43(7), 623–633 (2020).
PubMed
Google Scholar
•• Provides an overview and evaluation framework for the appropriate selection of external data and design for use in regulatory settings.
11.
Food and Drug Administration. Considerations for the design and conduct of externally controlled trials for drug and biological products (2023). https://www.fda.gov/media/164960/download
Google Scholar
12.
Food and Drug Administration. Considerations for the use of real-world data and real-world evidence to support regulatory decision-making for drug and biological products, draft guidance for industry (2021). https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-use-real-world-data-and-real-world-evidence-support-regulatory-decision-making-drug
Google Scholar
13.
National Institute for Health and Care Excellence. NICE real-world evidence framework (2022). https://www.nice.org.uk/corporate/ecd9/chapter/overview
Google Scholar
14.
Franklin JM, Patorno E, Desai RJ et al. Emulating randomized clinical trials with nonrandomized real-world evidence studies: first results from the RCT duplicate initiative. Circulation 143(10), 1002–1013 (2021).
PubMed
Google Scholar
15.
Ramagopalan SV, Hernán MA, Pinilla P, Thorlund K, Kent S. in ISPOR Europe 2022. (2022).
Google Scholar
16.
Jaksa A, Louder A, Maksymiuk C et al. A comparison of seven oncology external control arm case studies: critiques from regulatory and health technology assessment agencies. Value Health 25, 1967–1976 (2022).
PubMed
Google Scholar
• This research article highlights the common critiques from regulatory and health technology assessment agencies from oncology submissions which have incorporated the use of external control arms.
17.
Seeger JD, Davis KJ, Iannacone MR et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiol. Drug Saf. 29(11), 1382–1392 (2020).
PubMed
Google Scholar
18.
Viele K, Berry S, Neuenschwander B et al. Use of historical control data for assessing treatment effects in clinical trials. Pharm. Stat. 13(1), 41–54 (2014).
PubMed
Google Scholar
19.
Ming-Hui C, Joseph GI. Power prior distributions for regression models. Statis. Sci. 15(1), 46–60 (2000).
Google Scholar
20.
Best N, Price RG, Pouliquen IJ, Keene ON. Assessing efficacy in important subgroups in confirmatory trials: an example using Bayesian dynamic borrowing. Pharm. Stat. 20(3), 551–562 (2021).
PubMed
Google Scholar
21.
Dron L, Golchi S, Hsu G, Thorlund K. Minimizing control group allocation in randomized trials using dynamic borrowing of external control data – an application to second line therapy for non-small-cell lung cancer. Contemp. Clin. Trials Commun. 16, 100446 (2019).
PubMed
Google Scholar
22.
Richeldi L, Azuma A, Cottin V et al. Trial of a preferential phosphodiesterase 4b inhibitor for idiopathic pulmonary fibrosis. N. Engl. J. Med. 386(23), 2178–2187 (2022).
PubMed
Google Scholar
23.
Food and Drug Administration. Guidance for the use of Bayesian statistics in medical device clinical trials (2010). https://www.fda.gov/media/71512/download
Google Scholar
• This guidance document highlights the early precedent for the use of advanced methods such as Bayesian borrowing in medical device trials.
24.
Lim J, Walley R, Yuan J et al. Minimizing patient burden through the use of historical subject-level data in innovative confirmatory clinical trials: review of methods and opportunities. Therap. Innov. Regul. Sci. 52(5), 546–559 (2018).
PubMed
Google Scholar
25.
European Medicines Agencies. Concept paper on extrapolation of efficacy and safety in medicine development (2013). https://www.ema.europa.eu/en/documents/scientific-guideline/concept-paper-extrapolation-efficacy-and-safety-medicine-development_en.pdf
Google Scholar
26.
European Medicines Agencies. Guideline on clinical trials in small populations (2006). https://www.ema.europa.eu/en/documents/scientific-guideline/guideline-clinical-trials-small-populations_en.pdf
Google Scholar
27.
Brunner HI, Abud-Mendoza C, Viola DO et al. Safety and efficacy of intravenous belimumab in children with systemic lupus erythematosus: results from a randomised, placebo-controlled trial. Ann. Rheum. Dis. 79(10), 1340–1348 (2020).
PubMed
Google Scholar
28.
Food and Drug Administration. Bla 125370/s-064 and bla 761043/s-007 multi-disciplinary review and evalaution Benlysta®(belimumab) for intravenous infusion in children 5 to 17 years of age with sle (2021). https://www.fda.gov/media/127912/download
Google Scholar
• This FDA acceptance demonstrates the acceptability and use of Bayesian borrowing in a regulatory setting.
29.
Food and Drug Administration. Pediatric postmarketing pharmacovigilance review - Belimumab (2022). https://www.fda.gov/media/161020/download
Google Scholar
31.
Food and Drug Administration. Interacting with the FDA on complex innovative trial designs for drugs and biological products (2020). https://www.fda.gov/media/130897/download
Google Scholar
32.
Singh N, Temin S, Baker S et al. Therapy for stage IV non–small-cell lung cancer with driver alterations: ASCO living guideline. J. Clin. Oncol. 40(28), 3310–3322 (2022).
PubMed
Google Scholar
33.
Daly ME, Singh N, Ismaila N et al. Management of stage III non–small-cell lung cancer: ASCO guideline. J. Clin. Oncol. 40(12), 1356–1384 (2021).
PubMed
Google Scholar
34.
R Core Team. R Foundation for Statistical Computing (2022). https://www.R-project.org
Google Scholar
35.
Herbst RS, Redman MW, Kim ES et al. Cetuximab plus carboplatin and paclitaxel with or without bevacizumab versus carboplatin and paclitaxel with or without bevacizumab in advanced nsclc (SWOG S0819): a randomised, Phase III study. Lancet Oncol. 19(1), 101–114 (2018).
PubMed
Google Scholar
36.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183(8), 758–764 (2016).
PubMed
Google Scholar
• Highlights the best practices for emulating a target trial using real-world data to perform SCA analyses.
37.
Cuyún Carter G, Barrett AM, Kaye JA et al. A comprehensive review of nongenetic prognostic and predictive factors influencing the heterogeneity of outcomes in advanced non-small-cell lung cancer. Cancer Manag. Res. 6, 437–449 (2014).
PubMed
Google Scholar
38.
Berghmans T, Paesmans M, Sculier JP. Prognostic factors in stage III non-small-cell lung cancer: a review of conventional, metabolic and new biological variables. Ther. Adv. Med. Oncol. 3(3), 127–138 (2011).
PubMed
Google Scholar
39.
Williamson E, Morley R, Lucas A, Carpenter J. Propensity scores: from naive enthusiasm to intuitive understanding. Stat. Methods Med. Res. 21(3), 273–293 (2012).
PubMed
Google Scholar
40.
Zubizarreta JR, Paredes RD, Rosenbaum PR. Matching for balance, pairing for heterogeneity in an observational study of the effectiveness of for-profit and not-for-profit high schools in Chile. Annals Appl. Stat. 8(1), 204–231 (2014).
Google Scholar
• Details the cardinality-matching method used in the current study to construct the SCA from real-world data.
41.
Fortin SP, Johnston SS, Schuemie MJ. Applied comparison of large-scale propensity score matching and cardinality matching for causal inference in observational research. BMC Med. Res. Methodol. 21(1), 109 (2021).
PubMed
Google Scholar
42.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat. Med. 28(25), 3083–3107 (2009).
PubMed
Google Scholar
43.
Socinski MA, Bondarenko I, Karaseva NA et al. Weekly nab-paclitaxel in combination with carboplatin versus solvent-based paclitaxel plus carboplatin as first-line therapy in patients with advanced non-small-cell lung cancer: final results of a Phase III trial. J. Clin. Oncol. 30(17), 2055–2062 (2012).
PubMed
Google Scholar
44.
Guyot P, Ades AE, Ouwens MJ, Welton NJ. Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan–Meier survival curves. BMC Med. Res. Methodol. 12, 9 (2012).
PubMed
Google Scholar
45.
Carpenter B, Gelman A, Hoffman MD et al. Stan: a probabilistic programming language. J. Statis. Softw. 76(1), 1–32 (2017).
PubMed
Google Scholar
46.
Stuart EA, Lee BK, Leacy FP. Prognostic score-based balance measures can be a useful diagnostic for propensity score methods in comparative effectiveness research. J. Clin. Epidemiol. 66(Suppl. 8), S84–S90.e81 (2013).
PubMed
Google Scholar
47.
Food and Drug Administration. Cid case study: external control in diffuse b-cell lymphoma. https://www.fda.gov/media/155405/download
Google Scholar
48.
Ibrahim JG, Chen MH, Gwon Y, Chen F. The power prior: theory and applications. Stat. Med. 34(28), 3724–3749 (2015).
PubMed
Google Scholar
49.
Popat S, Liu SV, Scheuer N et al. Addressing challenges with real-world synthetic control arms to demonstrate the comparative effectiveness of Pralsetinib in non-small-cell lung cancer. Nat. Commun. 13(1), 3500 (2022).
PubMed
Google Scholar
50.
Hsu GG, Mackay E, Scheuer N, Ramagopalan SV. Keeping it real: implications of real-world treatment outcomes for first-line immunotherapy in metastatic non-small-cell lung cancer. Immunotherapy 13(18), 1453–1456 (2021).
PubMed
Google Scholar
51.
Sengupta S, Ntambwe I, Tan K et al. Emulating randomized controlled trials with hybrid control arms in oncology: a case study. Clin. Pharmacol. Ther. 113(4), 867–877 (2023).
PubMed
Google Scholar
•• Demonstrates the application of Bayesian borrowing methods in a hybrid control arm setting.
52.
Ali MS, Prieto-Alhambra D, Lopes LC et al. Propensity score methods in health technology assessment: principles, extended applications, and recent advances. Front. Pharmacol. 10, 973 (2019).
PubMed
Google Scholar
53.
Majumdar A, Davi R, Bexon M et al. Building an external control arm for development of a new molecular entity: an application in a recurrent glioblastoma trial for mdna55. Stat. Biosci. 14(2), 285–303 (2022).
Google Scholar
54.
Neuenschwander B, Capkun-Niggli G, Branson M, Spiegelhalter DJ. Summarizing historical information on controls in clinical trials. Clin. Trials 7(1), 5–18 (2010).
PubMed
Google Scholar
55.
Schmidli H, Gsteiger S, Roychoudhury S et al. Robust meta-analytic-predictive priors in clinical trials with historical control information. Biometrics 70(4), 1023–1032 (2014).
PubMed
Google Scholar
|