Skip to main content
Open access
Methodology
13 June 2024

Evaluating the robustness of an AI pathfinder application on eligibility criteria in multiple myeloma trials using real-world data and historical trials

Abstract

Background: Eligibility criteria are pivotal in achieving clinical trial success, enabling targeted patient enrollment while ensuring the trial safety. However, overly restrictive criteria hinder enrollment and study result generalizability. Broadening eligibility criteria enhances the trial inclusivity, diversity and enrollment pace. Liu et al. proposed an AI pathfinder method leveraging real-world data to broaden criteria without compromising efficacy and safety outcomes, demonstrating promise in non-small cell lung cancer trials. Aim: To assess the robustness of the methodology, considering diverse qualities of real-world data and to promote its application. Materials/Methods: We revised the AI pathfinder method, applied it to relapsed and refractory multiple myeloma trials and compared it using two real-world data sources. We modified the assessment and considered a bootstrap confidence interval of the AI pathfinder to enhance the decision robustness. Results & conclusion: Our findings confirmed the AI pathfinder's potential in identifying certain eligibility criteria, in other words, prior complications and laboratory tests for relaxation or removal. However, a robust quantitative assessment, accounting for trial variability and real-world data quality, is crucial for confident decision-making and prioritizing safety alongside efficacy.

Supplementary Material

File (supplementary material.docx)

References

Papers of special note have been highlighted as: • of interest
1.
Van Spall HGC, Toren A, Kiss A et al. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 297(11), 1233–1240 (2007).
2.
Scoggins JF, Ramsey SD. A national cancer clinical trials system for the 21st century: reinvigorating the NCI Cooperative Group Program. J. Natl Cancer Inst. 102(17), 1371–1371 (2010).
3.
Kim ES, Uldrick TS, Schenkel C et al. Continuing to broaden eligibility criteria to make clinical trials more representative and inclusive: ASCO-friends of cancer research joint research statement. Clin. Cancer Res. 27(9), 2394–2399 (2021).
4.
Duggal M, Sacks L, Vasisht KP. Eligibility criteria and clinical trials: An FDA perspective. Contemp. Clin. Trials 109, 106515 (2021).
• Offers a valuable insight from FDA perspective and guidance on eligibility criteria for optimizing trial design and ensuring patient inclusion, safety and regulatory compliance.
5.
FDA services US department of Health and Human. Enhancing the diversity of clinical trial populations-eligibility criteria, enrollment, practices, and trial design guidance for industry. Available at: https://www.federalregister.gov/documents/2020/11/10/2020-24881/enhancing-the-diversity-of-clinical-trial-populations-eligibility-criteria-enrollment-practices-and# (2020).
• Offers the FDA's perspective on the imperative to enhance diversity within clinical trial populations, providing comprehensive guidance on eligibility criteria, enrollment practices and trial design to foster inclusivity and improve the generalizability of study outcomes.
6.
Liu R, Rizzo S, Whipple S et al. Evaluating eligibility criteria of oncology trials using real-world data and AI. Nature 592(7855), 629–633 (2021).
• Provides a deeper understand on how real-world data and artificial intelligence are used to assess the significance of eligibility criteria in oncology trials.
7.
Rogers JR, Pavisic J, Ta CN et al. Leveraging electronic health record data for clinical trial planning by assessing eligibility criteria's impact on patient count and safety. J. Biomed. Inform. 127, 104032 (2022).
• Presents a machine learning approach employed to assess the impact of eligibility criteria on patient enrollment rates and safety using electronic health record data, offering valuable insights into optimizing clinical trial design.
8.
Dimopoulos M, Quach H, Mateos MV et al. Carfilzomib, dexamethasone, and daratumumab versus carfilzomib and dexamethasone for patients with relapsed or refractory multiple myeloma (CANDOR): results from a randomised, multicentre, open-label, phase 3 study. Lancet 396(10245), 186–197 (2020).
9.
Dimopoulos MA, Oriol A, Nahi H et al. Daratumumab, lenalidomide, and dexamethasone for multiple myeloma. N. Engl. J. Med. 375(14), 1319–1331 (2016).
10.
Palumbo A, Chanan-Khan A, Weisel et al. Daratumumab, bortezomib, and dexamethasone for multiple myeloma. N. Engl. J. Med. 375(8), 754–766 (2016).
11.
Dimopoulos MA, Terpos E, Boccadoro M et al. Daratumumab plus pomalidomide and dexamethasone versus pomalidomide and dexamethasone alone in treated multiple myeloma (APOLLO): an open-label, randomised, phase 3 trial. Lancet Oncol. 22(6), 801–812 (2021).
12.
Stewart AK, Rajkumar SV, Dimopoulos MA et al. Carfilzomib, lenalidomide, and dexamethasone for relapsed multiple myeloma. N. Engl. J. Med. 372(2), 142–152 (2015).
13.
Moreau P, Masszi T, Grzasko N et al. Oral ixazomib, lenalidomide, and dexamethasone for multiple myeloma. N. Engl. J. Med. 374(17), 1621–1634 (2016).
14.
Dimopoulos MA, Moreau P, Palumbo A et al. Carfilzomib and dexamethasone versus bortezomib and dexamethasone for patients with relapsed or refractory multiple myeloma (ENDEAVOR): a randomised, phase 3, open-label, multicentre study. Lancet Oncol. 17(1), 27–38 (2016).
15.
Lonial S, Dimopoulos M, Palumbo A et al. Elotuzumab therapy for relapsed or refractory multiple myeloma. N. Engl. J. Med. 373(7), 621–631 (2015).
16.
Moreau P, Weisel KC, Song KW et al. Relationship of response and survival in patients with relapsed and refractory multiple myeloma treated with pomalidomide plus low-dose dexamethasone in the MM-003 trial randomized phase III trial (NIMBUS). Leuk. Lymphoma 57(12), 2839–2847 (2016).
17.
Birnbaum B, Nussbaum N, Seidl-Rathkopf K et al. Model-assisted cohort selection with bias analysis for generating large-scale cohorts from the EHR for oncology research. Available at: https://arxiv.org/abs/2001.09765
18.
Ma X, Long L, Moon S et al. Comparison of population characteristics in real-world clinical oncology databases in the US: Flatiron Health, SEER, and NPCR. Available at: https://www.medrxiv.org/content/10.1101/2020.03.16.20037143v3
• Provides a deeper understanding of Flatiron Health data heterogeneity and its implications for research and clinical practice.
19.
Optum® de-identified Electronic Health Record dataset (2007–2021).
• Provides a deeper understanding of OPTUM electronic Health record offering a valuable resource for researchers to analyze long-term trends, outcomes and patterns in healthcare delivery and patient care.
20.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183(8), 758–764 (2016).
• Presents a comprehensive framework and decision guide aimed at facilitating the selection of optimal strategies for effectively emulating a clinical trial using real-world data.
21.
Rajkumar SV, Dimopoulos MA, Palumbo A et al. International Myeloma Working Group updated criteria for the diagnosis of multiple myeloma. Lancet Oncol. 15(1), e538–e548 (2014).
22.
Xu S, Ross C, Raebel MA et al. Use of stabilized inverse propensity scores as weights to directly estimate relative risk and its confidence intervals. Value Health 13(2), 273–277 (2010).
• Introduces a statistical approach aimed at reducing confounding in the estimation of relative risk when analyzing real-world data.