Open access

Methodology

13 November 2026

Real-world data to support clinical evidence generation in oncology: considerations for biomarker-based cohort selection, end point assessment and other methodological issues

Authors: Mehmet Burcu https://orcid.org/0000-0003-4572-0987 [email protected], Le Su https://orcid.org/0000-0003-2810-1703, Andrew N Freedman https://orcid.org/0000-0003-1451-4208, Wei Zhou https://orcid.org/0000-0003-0224-2559, and Geoffrey Liu https://orcid.org/0000-0002-2603-7296Author Info & Affiliations

Publication: Journal of Comparative Effectiveness Research

https://doi.org/10.57264/cer-2026-0157

PDF

Abstract

Randomized clinical trials remain the evidentiary gold standard for oncology, but they are often insufficient to fully characterize benefit–risk in rare cancers, biomarker-defined subgroups, and other settings where trial evidence is limited. As precision oncology advances, real-world data and real-world evidence (RWE) offer an increasingly important opportunity to address evidentiary gaps outside traditional trial populations. Yet the use of RWE in oncology requires more than access to data; it requires fit-for-purpose methods that account for the distinctive challenges of biomarker-based cohort selection, end point definition and bias mitigation. This paper reviews methodological considerations for oncology RWE generation and synthesizes selected use cases to illustrate their application in practice. We highlight unique issues arising in oncology RWE research, including incomplete or delayed biomarker ascertainment, heterogeneity in testing platforms and thresholds, misclassification of molecularly defined cohorts, variability in line-of-therapy assignment and time-related biases that can compromise interpretability. We also examine real-world end point ascertainment and discuss considerations for assessing end point validity, completeness and translatability. In addition, we outline how common oncology clinical trial end points can be operationalized as approximate real-world constructs, recognizing that differences in assessment schedule, measurement method and available data may affect comparability and performance. By integrating these methodological challenges with practical design considerations, this paper provides timely guidance to support generation of credible RWE for clinical development in oncology.

Plain language summary

What is this article about?

As precision oncology continues to advance, the evidence base increasingly depends on fit-for-purpose real-world data sources and methods to address gaps left by randomized clinical trials, especially in rare cancers, biomarker-defined subgroups and other settings where trial evidence is limited. This article synthesizes these challenges and provides practical considerations for the design and interpretation of credible oncology real-world evidence (RWE) studies.

What methodology is described?

This manuscript reviews key methodological considerations for the use of real-world data and RWE in oncology. Specifically, this paper reviews issues and considerations, with particular emphasis on biomarker-based cohort selection, cohort-defining variables and biomarker data availability, cohort representativeness, longitudinal considerations, oncology real-world end points (evaluation, validation and performance) and considerations for avoidable biases including immortal time and other biases, with oncology specific examples. The article also provides various distinct examples of RWE used in regulatory decision making in Oncology.

Why is this important?

As the oncology real-world data landscape continues to evolve, close attention to these methodological considerations will be essential to ensure that RWE studies generate credible, interpretable and decision-relevant evidence for regulators, clinicians and other stakeholders.

Randomized controlled trials (RCTs) remain the evidentiary gold standard for medical product approvals; however, they are not always sufficient to support benefit–risk evaluations or clinical and regulatory decisions in certain situations, such as rare diseases, specific cancer types and molecular subgroups of cancer [1–6]. Real-world data – broadly defined as observational data derived from electronic health records (EHRs), claims and billing activities, product and disease registries, cohort studies and patient generated data – when analyzed, generates RWE that can complement RCT information and fill critical gaps in our knowledge for clinical practice and regulatory decision making [7]. While RWD has a well-established role in postmarket safety surveillance, its application and acceptance for evaluating effectiveness of medical products, particularly in oncology, has been more limited [3,8].

In recent years, this reality has spurred interest by regulators and health authorities to explore the use of real-world data (RWD) and real-world evidence (RWE) to support regulatory decision making. Table 1 summarizes selected regulatory agencies and key frameworks or guidance documents that outline methodological, data quality and submission considerations for the appropriate use of RWE and RWD, including noninterventional studies, registry-based studies, externally controlled trials and pragmatic trial designs, to support regulatory decision-making across the product lifecycle [9–31]. These selections are targeted to represent key, diverse regulatory geographies and do not constitute an exhaustive systematic review. Beyond the documents summarized in the table 1, regulatory authorities across North America, Europe and Asia (e.g., China, Japan and Singapore) have also issued other related guidance, and professional organizations such as the European Society for Medical Oncology (ESMO), the American Society of Clinical Oncology (ASCO) and Friends of Cancer Research have published complementary oncology-specific documents [32]. Multi-stakeholder projects and focused workshops have identified key opportunities to strengthen the RWD landscape, including efforts to develop real-world end points to evaluate the performance of medical products in routine cancer care [33–39] and efforts to establish RWD-based external comparators to single arm trials or as part of a hybrid approach where RWD augments the control group of an RCT [40–44].

Table 1. Selected regulatory agencies frameworks/guidance for the use of real-world data/evidence (RWD/E).

Organization	Year	Title	Description	Ref.
The FDA	2025	Use of RWE to support regulatory decision-making for medical devices	Outlined criteria for assessing the relevance and reliability of RWD to generate RWE supporting regulatory decision-making for medical devices	[31]
	2024	RWE: considerations regarding noninterventional studies for drug and biological products	Discussed attributes regarding the design and analysis of a noninterventional study to support a drug’s effectiveness or safety for FDA review	[9]
	2023	Considerations for the use of real-world data and RWE to support regulatory decision-making for drug and biological products	Consolidated FDA’s considerations to study with RWD, including transparency regarding data collection and analysis, access to RWD, study monitoring, safety reporting and other responsibilities	[17]
	2022	Submitting documents utilizing real-world data and RWE to FDA for drugs and biologics	Provided practical recommendations on information to be included with submissions containing RWD/RWE, including the general information, purposes, study design and RWD sources	[11]
	2023	Considerations for the design and conduct of externally controlled trials for drug and biological products	Addressed methodological considerations for using external controls, including RWD-based comparators, to support evidence generation	[18]
	2024	Integrating randomized controlled trials for drug and biological products into routine clinical practice	Discussed clinical trials with streamlined protocols, pragmatic and hybrid trial designs, essential data collection and leveraging RWD sources	[14]
	2023	Real-world data: assessing registries to support regulatory decision-making for drug and biological products	Provided guidance on evaluating registries' fitness-for-use, linking registries with other RWD, supporting FDA review of submission with registry data	[12]
	2024	Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making for drug and biological products	Described considerations for assessing fitness-for-purpose of EHR and claims data, focusing on selection of appropriate data sources, development and validation of study design elements and data traceability and quality	[13]
	2023	Data standards for drug and biological product submissions containing real-world data	Outlined challenges in standardizing RWD, processes for documenting and managing RWD, and considerations for conforming RWD to FDA-supported study data standards, mapping RWD to submission standards, and performing data transformations	[16]
	2018	Use of electronic health records in clinical investigations	Provided recommendations on the use of EHR data in clinical investigations, EHR–EDC interoperability, Ensuring EHR data quality and integrity, Meeting FDA inspection and recordkeeping requirements	[10]
	2018	Framework for FDA's RWE program	Established FDA’s overarching framework for evaluating the use of RWE to support regulatory decision-making	[15]
The EMA	2025	Reflection paper on the use of real-world data in noninterventional studies to generate RWE	Focused on methodological principles that are considered important for noninterventional studies	[22]
	2024	RWE provided by EMA: support for regulatory decision-making	Outlined how RWE from RWD supports regulatory decisions, the study types available, EMA’s role in selecting data sources, and the process for requesting RWD studies	[23]
	2021	Guideline on registry-based studies	Focused on the methodological, regulatory and operational aspects involved in using registry-based studies	[25]
	2024	Data quality framework for EU medicines regulation: Application to real-world data	Provides specific recommendations as derived from the Data Quality Framework for EU Medicines regulation	[28]
	2025	Good practice guide for the use of the HMA-EMA catalogue of real-world data sources and studies. Version 2.0	Best-practice guidance for describing, cataloguing and using RWD sources transparently	[26]
	2025	RWE framework to support EU regulatory decision-making	Overview of learnings on the experience acquired with RWE generation	[24]
	2020	EMA regulatory science to 2025	Sets EMA’s long term regulatory science priorities and recommended promoting use of high quality RWD in decision-making	[27]
	2020	European Medicines Agencies network strategy to 2025	Network-wide strategy highlighting coordinated use of RWE, big data and analytics across EU regulators	[30]
	2020	Priority recommendations of the HMA-EMA joint Big Data Task Force	Established 10 priorities	[29]
The MHRA, UK	2021	Guidance on the use of real-world data in clinical studies to support regulatory decisions	Pointed to consider when evaluating whether a RWD source is of sufficient quality for the intended use (updated in 2025)	[19]
The MHRA, UK	2021	Guidance on randomized controlled trials using real-world data to support regulatory decisions	Provides considerations for planning prospective randomized trials using RWD to support regulatory decisions, including UK trial authorization requirements and key elements of trial design such as end point selection and safety data needs (updated in 2025)	[20]
Health Canada	2019	Elements of real-world data/evidence quality throughout the prescription drug product life cycle	Provides overarching principles to guide the generation of RWE that would be consistent with the regulatory standard of evidence in place in Canada and internationally	[21]

EHR: Electronic health record; EMA: European Medicines Agency; FDA: US Food and Drug Administration; MHRA: Medicines and Healthcare products Regulatory Agency; RWD: Real-world data; RWE: Real-world evidence.

Oncology is one of the foremost disease areas that may benefit from fit-for-purpose use of RWD, driven by high unmet medical need, rapid pace of product development, and often premature survival data available at the time of initial regulatory submission [45,46]. In addition, approximately 25% of all cancers are rare [47], when classified by histology, and an even larger percentage when the molecular subgroup of a relatively common tumor is included in the definition of rare cancers. For these cancers, large RCTs are often neither feasible or ethical, and RWE can provide critical complementary evidence when trial populations are small or follow-up is limited. The Food and Drug Administration (FDA)’s recent guidance on targeted therapies in low-frequency molecular subgroups underscores this need [48].

If conducted appropriately, RWE from fit-for-purpose RWD sources and designs may support regulatory decision-making across the product lifecycle, including postauthorization safety and long-term effectiveness monitoring, benchmarking and external control comparisons for single-arm trials, and, in select settings, provision of substantial or supportive evidence of efficacy for original approvals, new indications and label expansions. Table 2 illustrates selected regulatory use cases demonstrating how different real-world evidence approaches have been applied to support oncology-related product approvals [49–53]. For example, in the setting of a new indication, one product received regulatory approval for the prophylaxis of acute graft-versus-host disease, where substantial evidence was derived from a clinical trial with a registry-based real-world comparator analysis in patients undergoing unrelated-donor hematopoietic stem cell transplantation [51]. In another case supporting an original marketing application, RWE played a decisive role in approval for a rare overgrowth syndrome, with substantial evidence of efficacy derived from a single-arm retrospective chart review of medical records [50]. These examples are not intended to represent all typical oncology RWE uses; rather, they illustrate the range of regulatory contexts in which RWD sources have been used.

Table 2. Examples of use cases of real-world evidence (RWE) in oncology to support regulatory submissions.

Regulatory purpose	Role of RWE	Real-World Study Design	Product information	Indication	RWD Source	RWD Category
Original marketing application approval	Supportive evidence to benchmark results of the single-arm clinical trial	Real-world comparator analyses	Blinatumomab (Blincyto) Regulatory agency: FDA 2014/EMA 2015	CD19-positive Philadelphia chromosome-negative B-cell precursor acute lymphoblastic leukemia	Chart reviews	Medical records
Postauthorization safety	Mandated requirement to monitor long-term response and overall survival	Prospective long-term follow-up study	Axicabtagene ciloleucel (Yescarta) Regulatory agency: FDA 2017/EMA 2018	Relapsed/refractory large B-cell lymphoma or PMBCL after ≥2 lines of therapy	CIBMTR transplant outcomes registry	Registry
New indication	Substantial evidence of efficacy	Retrospective study with concurrent comparator	Orencia (Abatacept) Regulatory agency: FDA 2021	Prophylaxis of acute GVHD in unrelated-donor HSCT	CIBMTR transplant outcomes registry	Registry
Original marketing application approval	Substantial evidence of efficacy	Single-arm retrospective chart review	Vijoice (Alpelisib) Regulatory agency: FDA 2022	PIK3CA-related overgrowth spectrum (PROS) in adults and ≥2 years	Chart reviews	Medical records
Original marketing application approval	Supportive evidence to benchmark results of the single-arm clinical trial	Real-world comparator analyses	Carvykti (Ciltacabtagene autoleucel) Regulatory agency: FDA 2022/EMA 2022	Relapsed/refractory multiple myeloma after ≥4 prior lines	Multicenter EMR + research databases	Medical records

This table presents a nonsystematic, targeted selection of key use cases chosen to illustrate diverse use cases of real-world evidence in oncology to support regulatory submissions.

CIBMTR: Center for International Blood and Marrow Transplant Research; EHR: Electronic health record; EMA: European Medicines Agency; EMR: Electronic medical record; FDA: US Food and Drug Administration; GVHD: Graft-vs-host disease; HSCT: Hematopoietic stem cell transplant; PMBCL: Primary mediastinal B-cell lymphoma; PROS: PIK3CA-related overgrowth spectrum.

Besides regulatory applications, RWD is also increasingly applied across the entire pharmaceutical research and development continuum, from early drug discovery and translational research to clinical development. In particular, large clinico-genomic datasets have emerged as powerful resources for hypothesis-generating research. Linking patient-level genomic profiling with longitudinal clinical outcomes can help identify novel, clinically actionable biomarkers that predict treatment response in routine care [54] and investigate drug resistance mechanisms for novel target identification [55].

The evidentiary bar for regulatory decision-making is high, and many RWD applications are more appropriately framed as supporting clinical evidence generation, contextualization of trial results, treatment-pattern assessment or hypothesis generation. Considering the rapidly evolving landscape and increasing interest in the use of RWD/RWE in oncology, important methodological and operational gaps remain. In particular, there is a paucity of practical, consolidated guidance on the detailed methodological considerations required to design and implement robust oncology-specific RWE studies. This paper aims to address this gap by providing a methodological framework tailored to oncology research, with emphasis on the technical decisions involved in biomarker-based cohort selection, end point ascertainment and bias mitigation.

Methodological challenges & design considerations

While Table 2 describes several use cases in which real-world evidence provided supportive or substantial evidence, there are also use cases that faced limitations and critiques. For instance, in the evaluation of capmatinib (TABRECTA), regulators found that a retrospective real-world dataset intended as an external control exhibited inconsistencies and methodological deficiencies, such as small sample sizes, post hoc cohort assignment and noncontemporaneous treatments, which precluded its acceptance. Similarly, for remestemcel-L-rknd (RYONCIL), the FDA concluded that real-world evidence derived from registries was inconclusive due to selection bias, unmeasured confounding, and missing data (e.g., lack of concomitant medication records) [56]. In addition, recent regulatory reviews and other published reports have pointed out methodological and operational issues important for the appropriate design and interpretation of studies that incorporate RWD [8,40,44,57–64]. Some of these key learnings are summarized below.

Cohort selection

Cohort-defining variables & biomarker data availability

Cohort selection is context- and disease-dependent. The initial step is to determine the most salient variables necessary to accurately identify the population of interest; determine the homogeneity, representativeness, generalizability and size of the cohort required to answer the research question and establish the feasibility of capturing required inclusion/exclusion variables consistently and accurately within available RWD sources. While not every oncology RWD cohort is biomarker-defined, biomarkers are among the most common and consequential cohort-defining variables in precision oncology, alongside disease stage, histology, prior treatment history, performance status and comorbidities.

In oncology, genetic alterations including driver mutations are an increasingly important component of cancer subclassification. For example, a contemporary efficacy study of a new antineoplastic therapy in chronic lymphocytic leukemia may require identifying patients who have a 17p deletion, which is known to render cancer cells resistant to chemotherapy and confer a poor prognosis [65]. In the absence of explicit information on biomarker testing and results, other data sources including prescriptions, claims or alternative data sources could be used to infer biomarker status (depending on the clinical context); however, the accuracy of using a proxy for biomarker status must be characterized, with a threshold prespecified. For another example, patient selection in non-small-cell lung cancer depends on PD-L1 tumor proportion score measured by a companion diagnostic (CDx) assay, with prespecified cut-offs (e.g., ≥1%, ≥50%) that directly drive treatment eligibility. In routine clinical practice, however, PD-L1 testing is performed across multiple assay platforms, including FDA approved CDx assays and laboratory-developed assays, which are not fully interchangeable. When building an RWD-based cohort, the assay platform and corresponding threshold must therefore be explicitly accounted for. Thus, a robust approach for direct or indirect capture of all cohort-defining variables (e.g., biomarker results, stage at diagnosis or histology) from RWD is a minimum requirement for designing a cohort within an RWD study.

Biomarker data raises concerns about comparison group sampling and clinically relevant denominators. For example, ALK rearrangements are uncommon; therefore, assessment of the mutation in routine care, availability of tumor tissue samples for re-assessment, and overall prevalence of the mutation are relevant considerations in the feasibility evaluation. In instances of rare molecular subtypes, new study designs may improve the capability of collecting RWD. One such novel method is a ‘basket’ or ‘umbrella’ multi-institutional observational study which involves either multiple molecular subtypes that are studied as part of a single study or a single molecular subtype that is observed across many baskets of disease sites (tumor-agnostic). Such study designs, which are observational equivalent of a multi-cohort basket or umbrella clinical trials, are more efficient than multiple single-site or single-marker studies, with sharing of resources, protocols and ethics applications across multiple subtypes. CARMA-BROS is one such example of these new study designs [66].

Furthermore, as biomarker-based clinical research becomes commonplace, additional technical and methodological considerations arise when developing a biomarker-defined cohort using RWD. First, there needs to be adequate clinical annotation (or linkage) to necessary biomarker and genomic information to address the use case. Next, unlike clinical trials where genomic testing occurs on or prior to study entry, in the real world, genomic testing may occur at any point in the patient's disease course and the timing of the testing needs to be proactively addressed. In terms of specific genetic alterations of interest, the tissue source (primary or metastatic site), exposure to antineoplastic therapy, details of testing methods and relevant thresholds, and type of genetic mutation (somatic vs germline) should all be acknowledged. These details are often incomplete or missing in many RWD sources and may require supplemental chart review, data linkage or sensitivity analyses. The extent to which each element is required should be determined by the research question and intended use. Moreover, for diseases amid evolving genomic testing patterns, the potential for phenotypic drift within the cohort as well as changes in the standard of care over time should be considered. Restricting the cohort of interest to those patients tested within a specific window of time – for instance, contemporaneous to the enrollment of a trial of interest – may be important, depending on the use case.

Cohort representativeness

In addition to demographic, clinical, tumor, socioeconomic and geographic characteristics, there are several other variables to consider when determining cohort representativeness in oncology research using RWD. Care can occur across various settings including inpatient or outpatient clinics, community, academic or hospital-based practices, and each site of care will have distinct patient populations, drug formularies and treatment patterns. In addition to the care setting, the timing and type of treatment needs to be representative of the cohort of interest. For example, when comparing an RWD-derived cohort to a clinical trial population, it is preferred to have a real-world cohort contemporaneous to the trial time period to mitigate biases due to temporal changes in clinical practice and advances in standard of care or supportive care [40,67].

Longitudinal considerations

Other key aspects to consider when designing effectiveness studies using RWD include the timing of entry into the cohort, measurement of exposure and follow-up time to establish the longitudinal clinical journey. A prespecified index date defines when a patient enters the study cohort and often serves as the start date for time-to-event analyses. In oncology, depending on the clinical context and research question, examples of index dates include the date of treatment initiation (first-line therapy) or date of initial or metastatic diagnosis. It is also necessary to accurately measure treatment exposure which can be time-varying depending on the disease and regimen of interest. The optimal index date depends on tumor type, disease setting, treatment sequence and study objective. Accurate line-of-therapy algorithms are particularly important because treatment regimens may include induction, maintenance, rechallenge, add-on therapy, dose modifications and combination therapies. In hematologic malignancies such as multiple myeloma, patients may receive multiple lines of combination therapy over a chronic disease course, making regimen construction, allowable gaps, drug substitutions and transitions between lines especially important to prespecify. Treatment receipt can be delayed by the presence of toxicity, adverse events, care coordination or patient preference. The calculation of a regimen should be defined to include the specific drugs, acceptable drug substitutes and allowable clinical gap between doses to be applied as a standard to the dataset. An important consideration in observational data is the potential introduction of immortal time bias (discussed in further detail below) which can be defined as including time periods in an outcome analysis before the exposure starts when the outcome could not have possibly occurred [68,69].

End points in oncology

Outcomes & end points

End points are critical in oncology research when assessing the safety and effectiveness of therapy as well as understanding the impact of treatment patterns on how a patient feels, functions or survives [70]. Encapsulated in the definition of an end point is not only information on the measurement itself, but also the methodology and timing of measurement assessment [71,72].

In oncology, common end points include overall survival (OS), changes in tumor burden (e.g., objective response rate [ORR]), treatment-based end points (e.g., time to next treatment [TTNT] or time to treatment discontinuation [TTD]), lab-based end points (e.g., LDH or tumor markers) and composite end points (e.g., progression-free survival) [70,73]. Table 3 provides an illustrative mapping of selected common oncology end points to their real-world constructs. Another relevant end point in oncology is drug tolerability, which requires capture of toxicity as well as symptom burden via both clinician and patient reporting or can sometimes be inferred through measurement of treatment-based end points such as TTD. Recent evidence of survival benefit of routine assessment of patient symptoms and toxicities has also led some institutions to collect routinely such information, which provide another potentially available dimension of RWD [74,75]. Inferences may be based upon healthcare utilization as measured through administrative data or direct data collection from patient surveys or through algorithmic proxy related to a treatment change such as TTNT or TTD. As the capacity to collect patient data continues to grow, data from novel sources such as wearables may be incorporated into newer end points to measure drug efficacy and tolerability [76]. The selection of an appropriate end point(s) depends on population, disease, data source and use case. In oncology, increasing OS remains the ultimate goal; however, in some cancers, OS does not provide the most direct measure of therapeutic benefit due to factors such as a long natural history of disease, subsequent lines of therapy, high levels of synchronous comorbidities or cost among other factors [77]. In these cases, other end points that more directly measure the benefit of treatment, such as tumor burden, tolerability, patient functioning, quality-of-life or treatment-based end points, may be more appropriate. As an example, measuring rates of tumor control would provide a more direct measure of the benefit of a new surgical procedure for localized prostate cancer in older patients compared with OS, given the high rates of comorbid illness and noncancer mortality in this population [78]. Researchers should consider measuring multiple end points in tandem to understand the full clinical context to support robust conclusions.

Table 3. Illustrative mapping of selected common oncology clinical trial end points to real-world end point constructs.

Clinical trial end point	Approximate real-world construct	Key issues and considerations in RWD
Overall survival	Real-world overall survival using composite mortality sources	Completeness of death capture, linkage to mortality sources, informative loss to follow-up
Progression-free survival	Real-world clinician/radiology-anchored progression	Irregular assessment timing, missing imaging, lack of centralized and RECIST-based review
Objective response rate	Real-world tumor response or real-world objective response rate based on radiology reports (protocol-driven strict centralized and independent review not feasible in most cases)	Radiology text quality, abstraction algorithm, response confirmation and assessment window, lack of centralized and RECIST-based review
Duration of response	Duration from real-world tumor response to first documented progression/death based on date of progression or death information from composite mortality sources or progression data from clinician notes or radiology reports (protocol-driven strict centralized and independent review not feasible in most cases)	Requires reliable response and progression dates; may be sensitive to scan frequency, lack of centralized and RECIST-based review
Treatment discontinuation or duration of therapy	Time to treatment discontinuation, time to next treatment, time on treatment	Discontinuation may reflect progression, toxicity, preference, access or planned treatment duration
Safety and tolerability	Diagnosis codes, laboratory values, dose modifications, discontinuation, patient-reported outcomes, clinician notes	Under-capture and differential documentation are common; chart review or linkage may be needed

RECIST: Response Evaluation Criteria in Solid Tumors; RWD: Real-world data.

Data sources & considerations for ascertaining real-world end points

There are several factors to consider in ascertaining real-world outcomes, including feasibility of capture, translatability to clinical trial end points and validity of the measure. For example, mortality information, used to determine the OS end point, has the benefits of a consistent definition across RWD sources, importance to patients and direct objective measurement. However, death may not be always recorded completely in routine clinical care. Missing death data can lead to biased estimates of OS [79,80]. It is critical to assess completeness of a dataset’s mortality data and investigate the potential bias introduced by missing death data [35,81]. It is often necessary to combine multiple mortality data sources, including EHR death data (structured and unstructured), government mortality records/databases, commercial sources and health plan disenrollment to form a composite mortality variable. The cause of death is often unavailable or require special data request (cancer registries) in observational datasets. It is important to note that the cause of death information might be particularly relevant in indolent cancers or in older or highly comorbid populations where competing risks of death may be increased.

In an oncology clinical trial setting, end points based on changes in tumor burden commonly rely on Response Evaluation Criteria in Solid Tumors (RECIST) [82], which are based on predetermined schedule of specific radiographic assessments, technical radiographic quality, and an objective scoring system that incorporates percent change in prespecified lesions from baseline [82]. There are several challenges that arise when using scans obtained through routine care to measure tumor burden using RECIST, including the quality, modality, timing and lack of a central radiological interpretation of the imaging. In routine clinical care, tumor burden is typically not assessed by clinicians using RECIST [37]. Opportunities exist to evaluate the associations between radiologic interpretation of images obtained through clinical trials and routine care, but further methodological work is necessary to determine the impact of inconsistent assessment timing and various radiographic techniques. For example, when using an outcome that is not time-based, such as best overall response, the impact of assessment frequency may be less severe.

Methodology to evaluate the full clinical interpretation of radiology reports obtained in routine practice could provide another opportunity for assessing disease status over time. Information on tumor burden can also be available via other sources in EHRs, such as clinician notes, radiology reports, pathology reports and labs. These complementary types of clinical data can be utilized to identify clinical progression and response in the real world, either through curation by expert abstractors [36], or via natural language processing and machine learning methods [33]. While these real-world end points are not necessarily interchangeable with RECIST-based end points, they provide summative information about the totality of clinical assessment over time and may be operationalized as real-world tumor response or real-world ORR when the data source and algorithm support that definition.

An alternative method of assessing the benefit of a therapy in RWD may be to measure treatment-based end points, including duration of treatment, TTNT and TTD using structured treatment information including administrative claims for prescription fill (for oral antineoplastics), or medication orders or administrative records in an EHR [38]. However, similar to real world clinical end points, treatment-based end points have their own limitations. It is important to prespecify the expected schedule of therapy, as certain therapies are given indefinitely while others may include a planned number of cycles, and not all schedules may be appropriate for the use of treatment-based end points [73]. There may be uncertainty around whether patients actually received a treatment that was ordered, particularly for oral medications to be taken at home. In addition, discontinuing or changing treatments may be due to toxicity or financial burden rather than disease progression or gaps in data that can occur as a result of both medical (e.g., hospitalizations) and nonmedical (e.g., changing health insurance) events. Such additional information could be available in some datasets and might add additional clarity to the treatment exposure history; however, it may require linkage of inpatient and outpatient health datasets to acquire a complete assessment.

Other oncology end points including both measurement of clinical benefit as well as drug safety can also be sourced from RWD, depending on data availability from individual or linked data sources. For example, patient-reported outcome (PRO)-based end points could provide important insights into the patient perspective on medication tolerability but are currently not widely captured and available for observational research. As the use of electronic PROs during routine clinical care in oncology increases [83], PRO end points using RWD may become another viable real-world end point of specific interest to patient focused drug development. In addition, nonquantitative end points, such as clinical narratives, can add qualitative depth to quantitative results and may be particularly useful in small cohorts describing rare subpopulations [74,84].

Evaluating real-world end point performance

Due to their novel nature, a framework is necessary to evaluate the performance characteristics of real-world end points and broadly consists of addressing the following issues. First, it is important that an end point reflects a clinically relevant outcome; this can be ascertained by eliciting input from clinical experts regarding the accuracy and relevance of the end point definition and approach. Second, the ability to reliably capture the end point from real world data should be determined. This entails analysis of the completeness, reliability and reproducibility of the proposed approach. Third, assessment of the calibration of real-world end points to other outcomes should be completed. Several oncology studies and cross-sector efforts including Friends of Cancer Research pilot projects have evaluated associations among real-world end points, including rwOS, rwTTD, and rwTTNT and have provided methodological recommendations for end point construction and interpretation [38,85]. However, no single real-world end point should be considered universally optimal. The most appropriate end point depends on the disease context, treatment setting, data source and decision context. Not every RWD study needs to benchmark or replicate a clinical trial end point; some studies are designed to describe routine care, evaluate utilization, assess safety or characterize long-term outcomes that are not measured in trials. When comparison with clinical trial evidence is intended, limitations related to end point translatability, assessment frequency and measurement differences should be explicitly evaluated. The process for assessing this association may differ depending on the availability of a gold standard for comparison. For example, a real-world mortality end point can be compared with a national death registry, such as the National Death Index in the USA [81]. A gold standard allows for a more straightforward assessment of sensitivity and specificity and provides opportunity to investigate the impact of missing or inaccurate data. Other real-world end points in oncology may not have an obvious gold standard for comparison, such as real-world assessment of change in tumor burden, as discussed above. Assessing the characteristics and quality of real-world end points will require an alternative methodologic approach that may include incorporation of clinical feasibility along with triangulation among other data variables and data sources, which may require prospective study design and data capture [86,87].

Immortal time & other time-related biases

As oncology research often relies on time-to-event analyses, biases involving the index date and follow-up time are of importance. Immortal time bias occurs when, because of the index date definition, there is a time period when the outcome could not possibly be observed (Figure 1) [68,69]. This can inflate survival estimates and lead to biased results [88], but the impact of this bias varies depending on the clinical context. For example, several weeks of immortal time bias would hypothetically have more impact in advanced non-small-cell lung cancer with an expected survival that is measured in months, compared with metastatic hormone sensitive breast cancer, where survival is often measured in years. Similarly, left truncation occurs when the entry criteria for the cohort occurs after the index date [89]. Oncology studies involving patients who have received next-generation sequencing (NGS) testing present this potential issue. In those cases, if the index date is set to be the start of treatment and the criteria for cohort entry is NGS testing, which may occur after the index date, patients with longer survival after the index date have a higher probability of receiving NGS testing and thereby entering the cohort. This issue is compounded if the RWD involves any prospective data collection (which would require patient consenting), as happens with some registries and prospective observational studies; in those settings, unless the data are collected as part of routine practice, there can be significant biases in which patients participate (healthier, without language barrier). Finally, bias may originate from issues when patients are lost to follow-up and censored from the analysis. Although time-to-event analyses include standard censoring methods to account for patients lost to follow-up, if the censoring process is informative and associated with the outcome or patient characteristics, biased results may follow [90]. For example, in RWD survival analyses, it is possible that patients seen at certain sites of care may have different rates of loss to follow-up or documentation of outcomes [35,41], and that the site of care is associated with the true outcome.

Illustration of immortal time in treated and untreated patients with metastatic cancer. — Figure 1. Illustration of immortal time in a treated and in an untreated patient with a metastatic cancer.

Analytic considerations

Characteristics of the underlying data or cohort selection process may introduce a variety of biases that must be considered when designing and interpreting studies that use RWD [91]. A more comprehensive list of biases is also further discussed in Table 4 with Oncology-focused examples and methods to mitigate these biases [68,85,92–97]. There are also other analytical considerations to mitigate bias. For comparative effectiveness or safety analyses with a causal objective, a target trial emulation framework can be used to explicitly specify eligibility criteria, treatment strategies, treatment -assignment procedures, time zero, follow-up, outcomes, and the analysis plan [98]. Despite efforts to ensure data completeness, some level of missing data may be present and missing data strategies, such as multiple imputation, should be carefully considered. To address the presence of confounders and resultant biases, there are a variety of analytic adjustment techniques, including multivariable outcome regression models, propensity score adjustment, inverse probability of treatment weighting and inverse odds weighting, among others. The most appropriate technique will vary depending on the specific case and goals of the analysis; in some cases, simple techniques such as alignment of cohorts may be adequate, while in other cases more complex techniques are required [41]. In addition to confounders that are captured in the dataset, there may also be concern of unmeasured confounders that could not be ascertained using RWD. In these cases, sensitivity analyses can determine whether the results are sensitive to unmeasured confounders of a plausible, hypothesized magnitude [99]. Negative controls may be incorporated as a sensitivity analysis to assess the potential impact of residual confounding and other sources of bias in RWE studies. For example, consider a hypothetical comparative effectiveness study of a newly approved adjuvant oncology treatment versus standard-of-care therapy. Early recipients of the newly approved treatment may differ from patients receiving standard care; they may have higher-risk disease, greater frailty, or a different comorbidity burden. If these characteristics are not fully captured in the available RWD, residual confounding may remain after adjustment. In this setting, an appropriately selected negative control outcome may be considered to assess whether underlying differences between the treatment groups remain after adjustment. However, the selection and interpretation of negative controls are context-dependent and require careful consideration [100,101]. In time-to-event analyses, techniques may be used to accommodate specific types of censoring and truncation. For example, in oncology, disease progression can only be observed at assessment times, which can have varying frequencies in RWD. In some cases, interval censoring methods may be employed to account for the discrepancy between the true event time and the time in which it can be observed [102]. Lastly, blinding data analysts to treatment status, or other important factors, can mitigate unconscious bias that may be introduced at the analysis stage [61].

Table 4. List of potential biases and methods to mitigate these biases^†.

Bias type	Description	Oncology example	Methods to address bias	Ref.
Selection	Observed real-world cohort differs from the target population in a systematic manner. Challenges the external validity of the study	Patients who are volunteer to be part of clinical trials are different than nonparticipants across various demographic and clinical characteristics Patients on clinical trials are by definition expected to live a minimum amount of time but it is difficult to implement life-expectancy criteria in the real-world data	Sampling design, rebalancing methods	[85,92,94,95]
Confounding bias (measured confounders)	An observed factor is associated with both the exposure and outcome, but is not in the causal pathway. Challenges the internal validity of the study	The association between smoking history and mortality in NSCLC is confounded by age; older patients are more likely to be smokers and also have higher mortality	Restricting, matching, stratification, multivariable regression, propensity score analysis
Confounding bias (unmeasured confounders)	Same as above, but the factor is not measured in the available data	Education level and household income are important determinants of where cancer care is received but not routinely available socio demographic factors	Instrumental variable analysis, sensitivity analysis, difference-in-differences analysis, tipping point analysis	[97]
Channeling	A type of confounding bias (also known as confounding by indication) where patients are preferentially assigned treatments based on characteristics which are also associated with their outcome	In oncology, sicker patients are often treated with less aggressive, and potentially less effective, therapy than healthier patients. Therefore, the estimated treatment effect may be confounded by underlying comorbidity	Propensity score methods, inverse probability of treatment weighting, and inverse odds weighting	[93]
Misclassification	Patients are assigned to the incorrect exposure or set of patient characteristics	Patients without brain imaging are considered as not having brain metastases rather than unknown brain metastases	Sensitivity analysis
Ascertainment	Differential assessment of outcomes among patients particularly relevant in oncology RWD, where tumor burden and other outcomes are not assessed on a specific schedule, as is the case with an RCT	Differential scan timing between real world patients and clinical trial patients can impact time-to-event analysis	Standardization of outcome assessment at design stage, stratification, multivariable regression analysis, sensitivity analysis	[96]
Immortal time bias	A period of time is included in the analysis for one of the groups when the outcome could not possibly be observed	Analyzing the effectiveness of postmastectomy radiation therapy in breast cancer is subject to immortal time bias due to variation in the timing of the initiation of radiation therapy	Careful choice of index date and study design, time-dependent analyses, matched case control, landmark analysis	[68]
Performance bias	When one group of patients gets more attention or are treated differently than the other group which may impact comparison of the outcome of the study. Challenges the internal validity of the study	Patients with end-stage cancer (with limited options for systemic therapy) may be continued on immunotherapy through progression due to lack of alternative options and perceived minimal risk for toxicity which would be different than how a patient would be treated on a clinical trial	Prespecify key aspects of care delivery, standardize among groups, and control for known differences when able

†

List of methods to address bias contains a selection of strategies and is nonexhaustive. In addition to the methods listed here, randomization mitigates many potential biases when available to implement; however, in many real-world data studies the option is not available.

NSCLC: Non-small cell lung cancer; RCT: Randomized controlled trial; RWD: Real-world data.

Conclusion

RWD/RWE can play an important complementary role in oncology clinical evidence generation, particularly in settings where randomized trials are infeasible, limited or insufficient to address additional clinically meaningful questions. However, the value of RWD depends on fit-for-purpose cohort selection, reliable biomarker ascertainment, appropriate end point definition and validation and rigorous methods to address bias, missing data and time-related confounding. As the oncology RWD landscape continues to evolve, close attention to these methodological considerations will be essential to ensure that studies utilizing RWD sources generate credible, interpretable and decision-relevant evidence for regulators, clinicians and other stakeholders.

Executive summary

•

As precision oncology continues to advance, the evidence base increasingly depends on fit-for-purpose real-world sources and methods to address gaps left by randomized clinical trials, especially in rare cancers, biomarker-defined subgroups and other settings where trial evidence is limited.

•

This paper reviews issues and considerations, with particular emphasis on biomarker-based cohort selection, cohort-defining variables and biomarker data availability, cohort representativeness, longitudinal considerations, oncology real-world end points and considerations for avoidable biases with oncology specific examples.

•

Cohort selection should be driven by the research question and the variables needed to define the target population. For most real-world evidence (RWE) studies in oncology, disease stage, histology, prior treatment, performance status, comorbidities and timing of diagnosis or treatment should be considered and utilized in the study design, rather than relying on diagnosis codes alone.

•

For biomarker-defined cohorts, assay-level detail should be captured and biomarker timing should be prespecified. Testing platform, cut-off thresholds, tissue source, somatic versus germline status, and primary versus metastatic tissue should be considered, since assay noninterchangeability and timing of testing can distort cohort definition.

•

End point evaluation should be guided by clinical relevance, capture reliability, and comparability with trial end points. Overall survival is an objective end point but may be incomplete when death data are missing, while response and progression end points are more difficult to capture in routine care because standardized Response Evaluation Criteria in Solid Tumors assessment and central review are generally not available.

•

End point performance should be validated in light of assessment frequency and measurement method. Real-world progression free survival (rwPFS) and objective response rate (rwORR) can be affected by irregular scan timing, nonstandardized interpretation and incomplete documentation, so rwPFS and rwORR should not always be assumed to be interchangeable with RECIST-based trial end points.

•

Several common biases in oncology RWE may be mitigated through careful design and analysis. Depending on the study question and clinical context, immortal time bias, left truncation, informative censoring, missing data and confounding should be anticipated, with mitigation supported by prespecified index dates, appropriate cohort alignment, multiple imputation, propensity-based adjustment, sensitivity analyses, and where needed, interval-censoring methods.

•

As the oncology real-world data (RWD) landscape continues to evolve, close attention to these methodological considerations will be essential to ensure that studies utilizing RWD sources generate credible, interpretable and decision-relevant evidence for regulators, clinicians and other stakeholders.

Acknowledgments

The authors thank Nicole Mahoney for her review and comments.

Disclaimers

The views and opinions expressed in this article are those of the authors and do not necessarily represent the views and opinions of the National Cancer Institute, the National Institutes of Health, the US Department of Health and Human Services or any other organization, employer, or affiliated group.

Financial disclosure

The authors received no financial and/or material support for this research or the creation of this work.

Competing interests disclosure

M Burcu and L Su report current employment by Merck Sharp & Dohme LLC, a subsidiary of Merck & Co., Inc., (NJ, USA). M Burcu reports stock ownership in Merck & Co., Inc., (NJ, USA). W Zhou reports stock ownership in Eli Lilly and Company, (IN, USA) and in Merck & Co., Inc., (NJ, USA). The authors have no other competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript apart from those disclosed.

Writing disclosure

No funded writing assistance was utilized in the production of this manuscript.

Open access

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/

References

Papers of special note have been highlighted as: • of interest

Baumfeld Andre E, Reynolds R, Caubel P, Azoulay L, Dreyer NA. Trial designs using real-world data: the changing landscape of the regulatory approval process. Pharmacoepidemiol. Drug Saf. 29(10), 1201–1212 (2020).