R WE ready for reimbursement? A round-up of developments in real-world evidence relating to health technology assessment: part 21

The US FDA’s new leadership has placed target trial emulation (TTE) at the center of its regulatory modernization strategy, signaling a transformative shift in how real-world evidence (RWE) will shape drug approval processes [1]. The TTE framework for the analysis of real-world data (RWD), which as noted in previous parts of this series [2,3], provides a structured approach for designing observational studies that mirror the design principles of randomized trials, thereby minimizing biases inherent in traditional observational research. The FDA notes that the TTE framework applied to RWD offers the potential to generate reliable causal evidence, often at reduced costs compared with traditional trials. As such the FDA have suggested TTE may support a regulatory shift from requiring two pivotal clinical trials – currently standard for many FDA approvals – to accepting a single well-designed study. Additionally, post-market surveillance using TTE applied to large real-world datasets will enable the FDA to detect safety signals in real-time while assessing real-world effectiveness. This capability will be especially valuable for treatments targeting rare diseases, where pre-market randomized trials may be impractical due to small patient populations, and suggests that the FDA may be more willing to approve treatments on the basis of single-arm trials for these patients with long-term effectiveness data coming from the analysis of post-launch RWD.

There is increasing endorsement of the TTE approach by regulators and health technology assessment (HTA) agencies [4]. This is important as the data submitted to regulators generally forms the core of submissions to HTA bodies and payers. However, the use of TTE in formal regulatory and HTA submissions is limited to date [5,6]. The FDA’s vocal endorsement of TTE presents a significant opportunity for pharmaceutical companies to generate robust RWE to support their submissions, potentially allowing for earlier regulatory approval and reimbursement.

While the FDA’s embrace of TTE frameworks offers promise, the practical reality of how regulatory and HTA agencies currently evaluate RWE reveals significant gaps and inconsistencies. The Framework for Real-World Evidence Assessment to Mitigate Evidence Uncertainties for Efficacy/Effectiveness (FRAME) methodology was developed to systematically examine these evaluation processes and identify opportunities for improvement [7]. The FRAME researchers analyzed 15 medicinal products across 68 submissions to authorities in North America, Europe and Australia between January 2017 and June 2024, including five regulatory agencies and six HTA bodies. The data collection process involved extracting information on 74 variables from publicly available assessment reports. These variables included 43 describing submission characteristics and RWE type, plus 30 variables potentially influencing RWE’s role in decisions. The latter were grouped into three main areas: clinical context factors such as disease severity and unmet need; strength of evidence factors covering data sources, study design and treatment effect size; and process factors like protocol planning and authority interactions. The study covered three main RWE study types, with single-arm trials using RWD external controls being most common, followed by observational studies and one randomized controlled trial linked to RWD.

The study revealed four main findings that provide important insights into how authorities evaluate RWE. First, there was notably low granularity in publicly available assessment reports, with authorities commenting on limited numbers of the 30 variables that could influence decision making. While the FDA and Australia’s Pharmaceutical Benefits Advisory Committee (PBAC) commented on at least 50% of the variables, other authorities like Health Canada and Gemeinsamer Bundesausschuss (G-BA) commented on less than a third. Clinical context variables were most consistently discussed, though even in this area, no authority addressed more than two-thirds of the variables on average. Second, the research found variability in how different authorities assessed the same RWE studies. Some alignment was observed between some regulators, for example, the EMA and FDA, where nine out of 13 RWE studies were assessed by both agencies as playing similar roles in decision making. For HTA agencies, for Haute Autorité de Santé (HAS) and G-BA six out of seven RWE studies had comparable roles in decision making; similarities in decisions were also observed among England’s National Institute for Health and Care Excellence (NICE), Canada’s Drug Agency (CDA-AMC) and PBAC decisions; however, there was divergence in assessment between these archetypal HTA agencies (HAS, G-BA [clinical effectiveness driven HTA archetype] vs NICE, PBAC and CDA-AMC [cost-effectiveness driven HTA archetype]).

RWE played a primary role in 20% of regulatory assessments and 9% of HTA body evaluations, while serving a supportive role in 46% and 57%, respectively. Crucially, effect size emerged as the key determining factor, with large effect sizes consistently noted in submissions where RWE was considered primary evidence. Finally, the study found limited use of advanced RWE study designs across submissions, and that HTA agencies had broader use of RWD considering factors such as health equity and mode of administration.

Based on these findings, the researchers proposed five key recommendations to enhance the evaluation and use of RWE in regulatory and HTA decision-making. The first three recommendations address challenges encountered in learning from publicly available assessment reports. They suggest establishing structured assessment reports using templates or checklists informed by the variables identified in their research, introducing standardized sections that describe submitted RWE in detail, and creating an ongoing public repository to track and characterize future RWE submissions. The fourth recommendation focuses on strengthening collaboration between authorities to develop consistent terminology and common principles for RWE assessment. Such initiatives could help promote better understanding and awareness of RWE’s role in decision-making by highlighting areas of inconsistency and offering insights into the underlying rationale for key divergences between authorities. The fifth recommendation involves developing or enhancing decision-support tools that could help sponsors evaluate the feasibility of using RWE for their specific clinical contexts. These tools could assist companies in deciding whether to invest resources in developing RWE approaches by considering the characteristics of both their evidence and clinical development programs. While the analysis from FRAME perhaps reflects the findings of previous similar investigations [6,8,9], it shows a critical disconnect between the theoretical potential of RWE and its practical application in decision-making, highlighting that improvements in both methodology and transparency are needed before RWE can fulfill its promise as a reliable basis for regulatory and reimbursement decisions. The FRAME analysis however does not assess the impact of recently published RWE guidance from regulatory and HTA bodies, which advocate for better reporting of RWE using the HARPER template for example [10]. It remains to be seen if reporting on reasons for decisions made on submitted RWE will improve in the coming years. An important item to note is that it seems that only RWD effectiveness findings that have a large treatment effect are ‘believed’, which suggests that education on how methods such as TTE can minimize bias may be important for decision makers. For manufacturers, as noted above, when preparing for regulatory and/or HTA submissions, it is important to follow best practice highlighted in guidance documents and apply rigorous approaches to RWD study design, analysis and reporting.

Addressing the methodological and procedural challenges identified by FRAME, the Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration offers a concrete example of how stakeholders can work together to create structured, actionable frameworks for RWE implementation in healthcare decision-making [11]. The collaboration aimed to develop a framework that would facilitate the use of RWE by Canadian decision-makers for reassessment of cancer drugs and refinement of funding decisions and drug price negotiations.

The CanREValue framework takes a four-phase approach developed using extensive stakeholder engagement across Canada. Phase I involves identification, selection and prioritization of RWE questions, recognizing that RWE analyses cannot be completed for all funded drugs or all identified uncertainties. This phase incorporates a multicriteria decision analysis rating tool developed specifically for prioritizing RWE questions, with seven criteria assessing both the importance of the RWE question and the feasibility of conducting an RWE study. It is suggested that the multicriteria decision analysis results are publicly shared for transparency on what RWE studies were prioritized and why. Phase II provides guidance on planning and initiating RWE studies through a standardized implementation plan that serves as both a checklist and process guide. The implementation plan consists of three sequential components developed collaboratively with key stakeholders: Part A outlines critical study information including project details, rationale and stakeholder roles; Part B develops a tailored study protocol informed by existing templates such as HARPER [10]; and Part C creates a project summary. The final implementation plan, including study protocol and project summary, must be agreed upon by all relevant stakeholders before undertaking the actual RWE study to ensure alignment on both approach and application of results. The third phase of the CanREValue framework focuses on executing RWE studies through robust data collection and analysis methods to build confidence in reassessment recommendations. The CanREValue Collaboration established partnerships with data experts and organizations across Canada, conducting a mapping exercise to identify relevant cancer data elements and assess provincial capabilities for RWE analysis. Through a demonstration project, they explored using administrative health databases for cancer RWE studies and developed key recommendations including: coordinating data access across multiple sites, synchronizing analysis timelines, employing chart reviews for missing data, understanding data limitations during planning and sharing analysis plans and code between provinces. Phase IV involves conducting reassessment, where results from RWE studies are formatted into reassessment submission templates and evaluated by HTA agencies for potential changes to funding recommendations.

The CanREValue collaboration found that robust RWE studies could still be conducted despite RWD challenges such as variation in data availability between provinces, data content limitations of administrative datasets and lengthy timelines for data access. The framework’s emphasis on stakeholder engagement ensures RWE generated reflects the needs and perspectives of diverse stakeholders directly involved in cancer drug funding decisions. For manufacturers, the CanREValue framework provides a structured approach for RWE generation for cancer drugs, suggesting that early collaboration with payers, clinicians and patients can improve the relevance and impact of RWE studies.

The emergence of structured frameworks like FRAME and CanREValue reflects a growing sophistication in how RWE is generated, evaluated and applied in decision-making contexts. For manufacturers, these advances present both opportunities and challenges, requiring investment in methodological expertise while offering new avenues for demonstrating product value. The increasing standardization and acceptance of RWE approaches across regulatory agencies and HTA bodies suggest that high-quality RWE will become increasingly essential for successful market access strategies.