Skip to main content
Open access
Research Article
4 January 2024

Justifying the source of external comparators in single-arm oncology health technology submissions: a review of NICE and PBAC assessments

Abstract

Background: The drive to expedite patient access for diseases with high unmet treatment needs has come with an increasing use of single-arm trials (SATs), especially in oncology. However, the lack of control arms in such trials creates challenges to assess and demonstrate comparative efficacy. External control (EC) arms can be used to bridge this gap, with various types of sources available to obtain relevant data. Objective: To examine the source of ECs in single-arm oncology health technology assessment (HTA) submissions to the National Institute for Health and Care Excellence (NICE) and the Pharmaceutical Benefits Advisory Committee (PBAC) and how this selection was justified by manufacturers and assessed by the respective HTA body. Methods: Single-arm oncology HTA submission reports published by NICE (England) and PBAC (Australia) from January 2011 to August 2021 were reviewed, with data qualitatively synthesized to identify themes. Results: Forty-eight oncology submissions using EC arms between 2011 and 2021 were identified, with most submissions encompassing blood and bone marrow cancers (52%). In HTA submissions to NICE and PBAC, the EC arm was typically constructed from a combination of data sources, with the company's justification in data source selection infrequently provided (PBAC [2 out of 19]; NICE [6 out of 29]), although this lack of justification was not heavily criticized by either HTA body. Conclusion: Although HTA bodies such as NICE and PBAC encourage that EC source justification should be provided in submissions, this review found that this is not typically implemented in practice. Guidance is needed to establish best practices as to how EC selection should be documented in HTA submissions.

Plain language summary

What is this article about?

Diseases with high unmet needs are often assessed in trials with no comparator arm. As new treatments need to be compared with what is used in current clinical practice, data from other sources, called an external control arm, can be used to bridge the gap. The type of external control arm used and why it has been chosen should be explained when making submissions to a health technology agency.

What were the results?

Submissions made to two health technology agencies were reviewed to identify whether justification was provided for the type of external control arm used. This review demonstrated that this justification was rarely provided.

What do the results of the study mean?

These findings highlight the need for guidance to be established regarding the documentation of external control arms in submissions made to health technology agencies.

Supplementary Material

File (supplementary materials.docx)

References

Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
ICH. E10: choice of control group and related issues in clinical trials (2000). https://database.ich.org/sites/default/files/E10_Guideline.pdf
• A guideline outlining control groups in clinical trials.
2.
US FDA. Draft Guidance: Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products Guidance for Industry (2019). https://www.fda.gov/media/133660/download
3.
Tenhunen O, Lasch F, Schiel A, Turpeinen M. Single-arm clinical trials as pivotal evidence for cancer drug approval: a retrospective cohort study of centralized european marketing authorizations between 2010 and 2019. Clin. Pharmacol. Ther. 108(3), 653–660 (2020).
4.
Goring S, Taylor A, Müller K et al. Characteristics of non-randomised studies using comparisons with external controls submitted for regulatory approval in the USA and Europe: a systematic review. BMJ Open 9(2), e024895 (2019).
5.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am. J. Epidemiol. 183(8), 758–764 (2016).
6.
Franklin JM, Pawar A, Martin D et al. Nonrandomized real-world evidence to support regulatory decision making: process for a randomized trial replication project. Clin. Pharmacol. Ther. 107(4), 817–826 (2020).
• A review in which real-world evidence was used to emulate data from randomized trials.
7.
Burcu M, Dreyer NA, Franklin JM et al. Real-world evidence to support regulatory decision-making for medicines: considerations for external control arms. Pharmacoepidemiol. Drug Saf. 29(10), 1228–1235 (2020).
•• Provides definitions of different types of external control arms.
8.
Mishra-Kalyani PS, Amiri Kordestani L, Rivera DR et al. External control arms in oncology: current use and future directions. Ann. Oncol. 33(4), 376–383 (2022).
• Provides an introduction to the concepts of external control arm use in oncology.
9.
Patel D, Grimson F, Mihaylova E et al. Use of external comparators for health technology assessment submissions based on single-arm trials. Value Health 24(8), 1118–1125 (2021).
•• A previously published health technology assessment (HTA) review on the use of external control arms for single-arm trial based submissions.
10.
Thorlund K, Dron L, Park JJH, Mills EJ. Synthetic and external controls in clinical trials – a primer for researchers. Clin. Epidemiol. 12, 457–467 (2020).
11.
US FDA. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products: Guidance for Industry (2023). https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-design-and-conduct-externally-controlled-trials-drug-and-biological-products
12.
National Institute for Health and Care Excellence (NICE). ECD9: NICE real-world evidence framework (2022). https://www.nice.org.uk/corporate/ecd9/chapter/overview
•• A framework which outlines key considerations and the implementation of real-world evidence within NICE guidance.
13.
Canada's Drug and Health Technology Agency (CADTH). Guidance for Reporting Real-World Evidence (2023). https://www.cadth.ca/sites/default/files/RWE/MG0020/MG0020-RWE-Guidance-Report-Secured.pdf
14.
Jaksa A, Louder A, Maksymiuk C et al. A comparison of seven oncology external control arm case studies: critiques from regulatory and health technology assessment agencies. Value Health 25(12), 1967–1976 (2022).
•• A previously published review of the use of external control arms in FDA submissions.
15.
Jahanshahi M, Gregg K, Davis G et al. The use of external controls in FDA regulatory decision making. Ther. Innov. Reg. Sci. 55(5), 1019–1035 (2021).
16.
Hashmi M, Rassen J, Schneeweiss S. Single-arm oncology trials and the nature of external controls arms. J. Comp. Eff. Res. 10(12), 1053–1066 (2021).
17.
Seeger JD, Davis KJ, Iannacone MR et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiol. Drug Saf. 29(11), 1382–1392 (2020).
18.
HTAi. Health Technology Assessment International (HTAi) (2021). https://htai.org/
19.
Simon R, Blumenthal GM, Rothenberg ML et al. The role of nonrandomized trials in the evaluation of oncology drugs. Clin. Pharmacol. Ther. 97(5), 502–507 (2015).
20.
NHS England Cancer Drugs Fund. Cancer Drugs Fund (2019). https://www.england.nhs.uk/cancer/cdf/
21.
National Institute for Health and Care Excellence (NICE). TA401: bosutinib for previously treated chronic myeloid leukaemia (2016). https://www.nice.org.uk/guidance/ta401
22.
National Institute for Health and Care Excellence (NICE). TA451: ponatinib for treating chronic myeloid leukaemia and acute lymphoblastic leukaemia (2017). https://www.nice.org.uk/guidance/ta451
23.
National Institute for Health and Care Excellence (NICE). TA462: nivolumab for treating relapsed or refractory classical Hodgkin lymphoma (2017). https://www.nice.org.uk/guidance/ta462
24.
National Institute for Health and Care Excellence (NICE). TA478: brentuximab vedotin for treating relapsed or refractory systemic anaplastic large cell lymphoma (2017). https://www.nice.org.uk/guidance/ta478
25.
National Institute for Health and Care Excellence (NICE). TA487: venetoclax for treating chronic lymphocytic leukaemia (2017). https://www.nice.org.uk/guidance/ta487
26.
National Institute for Health and Care Excellence (NICE). TA491: ibrutinib for treating Waldenstrom's macroglobulinaemia (2017). https://www.nice.org.uk/guidance/ta491
27.
National Institute for Health and Care Excellence (NICE). TA510: daratumumab monotherapy for treating relapsed and refractory multiple myeloma (2018). https://www.nice.org.uk/guidance/ta510
28.
National Institute for Health and Care Excellence (NICE). TA524: brentuximab vedotin for treating CD30-positive Hodgkin lymphoma (2018). https://www.nice.org.uk/guidance/ta524
29.
National Institute for Health and Care Excellence (NICE). TA540: pembrolizumab for treating relapsed or refractory classical Hodgkin lymphoma (2018). https://www.nice.org.uk/guidance/ta540
30.
National Institute for Health and Care Excellence (NICE). TA554: tisagenlecleucel for treating relapsed or refractory B-cell acute lymphoblastic leukaemia in people aged up to 25 years (2018). https://www.nice.org.uk/guidance/ta554
31.
National Institute for Health and Care Excellence (NICE). TA559: axicabtagene ciloleucel for treating diffuse large B-cell lymphoma and primary mediastinal large B-cell lymphoma after 2 or more systemic therapies (2019). https://www.nice.org.uk/guidance/ta559
32.
National Institute for Health and Care Excellence (NICE). TA567: tisagenlecleucel for treating relapsed or refractory diffuse large B-cell lymphoma after 2 or more systemic therapies (2019). https://www.nice.org.uk/guidance/ta567
33.
National Institute for Health and Care Excellence (NICE). TA589: blinatumomab for treating acute lymphoblastic leukaemia in remission with minimal residual disease activity (2019). https://www.nice.org.uk/guidance/ta589
34.
National Institute for Health and Care Excellence (NICE). TA604: idelalisib for treating refractory follicular lymphoma (2019). https://www.nice.org.uk/guidance/ta604
35.
National Institute for Health and Care Excellence (NICE). TA677: autologous anti-CD19-transduced CD3+ cells for treating relapsed or refractory mantle cell lymphoma (2021). https://www.nice.org.uk/guidance/ta677
36.
PBAC. Public Summary Document (PSD) November 2014 PBAC Meeting: ponatinib (2014). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2014-11/ponatinib-psd-11-2014
37.
PBAC. Public Summary Document (PSD) March 2014 PBAC Meeting: Brentuximab Vedotin (2014). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2014-03/brentuximab-vedotin-psd-03-2014
38.
39.
PBAC. Public Summary Document (PSD) March 2015 PBAC Meeting: Brentuximab Vedotin (2015). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2015-03/brentuximab-vedotin-adcetris-psd-03-2015
41.
PBAC. Public Summary Document (PSD) November 2015 PBAC Meeting: pralatrexate (2015). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2015-11/pralatrexate-folotyn-psd-11-2015
42.
PBAC. Public Summary Document (PSD) November 2016 PBAC Meeting: brentuximab vedotin (ASCT naïve) (2016). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2016-11/brentuximab-asct-naive-psd-november-2016
43.
PBAC. Public Summary Document (PSD) July 2017 PBAC Meeting: pembrolizumab (2017). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2017-07/pembrolizumab-psd-july-2017
44.
PBAC. Public Summary Document (PSD) November 2018 PBAC Meeting: pembrolizumab (PMBCL) (2018). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2018-11/Pembrolizumab-PMBCL
45.
PBAC. Public Summary Document (PSD) July 2018 PBAC Meeting: blinatumomab (2018). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2018-07/Blinatumomab-psd-july-2018
46.
National Institute for Health and Care Excellence (NICE). TA395: ceritinib for previously treated anaplastic lymphoma kinase positive non-small-cell lung cancer (2016). https://www.nice.org.uk/guidance/ta395
47.
National Institute for Health and Care Excellence (NICE). TA416: Osimertinib for treating EGFR T790M mutation-positive advanced non-small-cell lung cancer (2016). https://www.nice.org.uk/guidance/ta653
48.
National Institute for Health and Care Excellence (NICE). TA571: brigatinib for treating ALK-positive advanced non-small-cell lung cancer after crizotinib (2019). https://www.nice.org.uk/guidance/ta571
49.
National Institute for Health and Care Excellence (NICE). TA628: lorlatinib for previously treated ALK-positive advanced non-small-cell lung cancer (2020). https://www.nice.org.uk/guidance/ta628
50.
National Institute for Health and Care Excellence (NICE). TA643: entrectinib for treating ROS1-positive advanced non-small-cell lung cancer (2020). https://www.nice.org.uk/guidance/ta643
51.
National Institute for Health and Care Excellence (NICE). TA489: vismodegib for treating basal cell carcinoma (2017). https://www.nice.org.uk/guidance/ta489
52.
National Institute for Health and Care Excellence (NICE). TA517: avelumab for treating metastatic Merkel cell carcinoma (2018). https://www.nice.org.uk/guidance/ta517
53.
National Institute for Health and Care Excellence (NICE). TA592: cemiplimab for treating metastatic or locally advanced cutaneous squamous cell carcinoma (2019). https://www.nice.org.uk/guidance/ta592
54.
National Institute for Health and Care Excellence (NICE). TA492: atezolizumab for untreated PD-L1-positive locally advanced or metastatic urothelial cancer when cisplatin is unsuitable (2017). https://www.nice.org.uk/guidance/ta492
55.
National Institute for Health and Care Excellence (NICE). TA530: nivolumab for treating locally advanced unresectable or metastatic urothelial cancer after platinum-containing chemotherapy (2018). https://www.nice.org.uk/guidance/ta530
56.
National Institute for Health and Care Excellence (NICE). TA630: larotrectinib for treating NTRK fusion-positive solid tumours (2020). https://www.nice.org.uk/guidance/ta630
57.
National Institute for Health and Care Excellence (NICE). TA644: entrectinib for treating NTRK fusion-positive solid tumours (2020). https://www.nice.org.uk/guidance/ta644
58.
National Institute for Health and Care Excellence (NICE). TA722: pemigatinib for treating relapsed or refractory advanced cholangiocarcinoma with FGFR2 fusion or rearrangement (2021). https://www.nice.org.uk/guidance/ta722
59.
National Institute for Health and Care Excellence (NICE). TA704: trastuzumab deruxtecan for treating HER2-positive unresectable or metastatic breast cancer after 2 or more anti-HER2 therapies (2021). https://www.nice.org.uk/guidance/ta704
63.
PBAC. Public Summary Document (PSD) November 2017 PBAC Meeting: crizotinib (2017). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2017-11/crizotinib-psd-november-2017
65.
PBAC. Public Summary Document (PSD) July 2018 PBAC Meeting: avelumab (2018). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2018-07/Avelumab-psd-july-2018
66.
69.
PBAC. Public Summary Document (PSD) July 2015 PBAC Meeting: ponatinib (2015). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2015-07/ponatinib-psd-july-2015
70.
PBAC. Public Summary Document (PSD) November 2015 PBAC Meeting: idelalisib (follicular lymphoma) (2015). https://www.pbs.gov.au/pbs/industry/listing/elements/pbac-meetings/psd/2015-11/idelalisib-zydelig-follicular-lymphoma-psd-11-2015
72.
Bullement A, Podkonjak T, Robinson MJ et al. Real-world evidence use in assessments of cancer drugs by NICE. Int. J. Technol. Assess. Health Care. 36(4), 388–394 (2020).
73.
Hernán MA, Sauer BC, Hernández-Díaz S, Platt R, Shrier I. Specifying a target trial prevents immortal time bias and other self-inflicted injuries in observational analyses. J. Clin. Epidemiol. 79, 70–75 (2016).
74.
Lou J, Kc S, Toh KY et al. Real-world data for health technology assessment for reimbursement decisions in Asia: current landscape and a way forward. Int. J. Tech. Assess. Health Care 36(5), 474–480 (2020).
75.
Berger ML, Sox H, Willke RJ et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: recommendations from the joint ISPOR–ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol. Drug Saf. 26(9), 1033–1039 (2017).