R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 14

The terms ‘real-world data’ (RWD) and ‘real-world evidence’ (RWE) are often used inconsistently or interchangeably, causing confusion. As such, authors from the FDA have published a commentary clarifying their definitions [1]. The publication highlights how it is not only non-interventional (observational) studies that produce RWE, but in reality, various study designs combining different data sources can generate RWE, for example externally controlled trials with RWD comparator arms. Conversely, other data, such as using aggregated summary data from the literature about disease prevalence is not classed as RWE by the FDA.

The authors state that consistent RWD/RWE terminology can help the FDA track approvals relying on RWE, and it can also help manufacturers more accurately specify study designs and data sources used to generate RWE in FDA submissions. The authors discuss how commentators reviewing FDA appraisals of RWE studies have been inconsistent with how the FDA defines RWE, leading to confusion which they want to avoid [2]. In summary, the article clarifies FDA terminology around RWD and RWE. This clarity in definitions will be helpful for manufacturers submitting RWE to them, but the challenge may arise when submitting the same RWE to different regulators and HTA agencies across the globe if definitions are not consistent. Of note, the EMA has a different definition of RWD as compared with the FDA [3]. Hopefully this publication spurs an effort toward global agreement on the definitions of RWD and RWE in order to allow consistency in submissions and tracking of use and acceptance of RWE.

Target trial emulation (TTE) refers to the concept of designing observational studies to emulate hypothetical randomized controlled trials (RCTs) [4]. The goal is to apply the key principles of RCTs to help reduce bias in observational studies addressing causal questions such as comparative effectiveness, and the approach has been called out as best practice by HTA agencies in England, Canada and France, among others [5–9]. Despite this endorsement, there have not been any clear examples of RWD applying the target trial framework being submitted to HTA bodies. One of the reasons for this may be because of the content-rich RWD needed to truly adopt the TTE approach. Moler-Zapata et al. attempt to implement TTE in a RWD study to evaluate the cost-effectiveness of emergency surgery versus non-emergency surgery for two acute gastrointestinal conditions [10]. The example was chosen because there is limited RCT data to address this question and therefore likely reflects a situation for HTA agencies where RWD will be used to assess treatment effectiveness. Through this exercise, the authors found four main challenges when applying the target trial framework: defining the study population; defining the treatment strategies; establishing time zero (baseline); and adjusting for unmeasured confounding. Based on their experience they offer helpful guidance to address these challenges including using expert judgement to adapt trial eligibility criteria to RWD (when the RWD source lacks the variables to truly apply inclusion and exclusion criteria) and appropriate comparators. The authors also stress the importance of carefully considering immortal time bias when establishing time zero; and using appropriate statistical methods to handle confounding. After tackling these issues, the paper concludes that applying target trial principles produces robust evidence to inform HTA decisions where RCT data is lacking. While this was the case in the example used by Moler-Zapata et al., it remains to be seen if this is true in other settings where RWD may be missing more variables to appropriately apply TTE and/or in the case of RWD used as an external control arm (and whether RWD can match the trial arm protocol). While the target trial framework was conceptualized a number of years ago, it has only been endorsed by HTA agencies in the last year, which is perhaps one of the reasons why it has not been well used by industry to date in regulatory and HTA submissions. Another reason may be because of the implementation challenges as observed by Moler-Zapata and colleagues; hopefully the recommendations from this paper will be helpful for those applying the TTE approach to causal RWD studies to be submitted for HTA.

The National Institute for Health and Care Excellence's (NICE) framework for using RWE was launched in July 2022 [5]. As noted in earlier parts of this series, this framework is perhaps the most comprehensive RWE guidance released by any healthcare decision maker (HTA agency or regulator) to date [11]. Duffield and Jonsson from NICE have recently published their thoughts in this journal on the impact of the framework a year after launch [12]. The authors start by highlighting how NICE believe RWE can add value in their decision making – in particular, for proportionate HTA, dynamic (living) guidances, monitor the uptake of guidances and to address uncertainty post evaluation of technologies to allow earlier patient access. The NICE framework aims to provide guidance on ‘what good looks like’, encouraging both the more rigorous generation of RWE and also greater consistency in the appraisal of it. In the year since publication, the authors reflect that while a lot of the framework was focused on comparative effectiveness studies, RWE is the preferred source of evidence for a number of other use cases, and the framework has been influential in NICE for these purposes. These use cases include comparing trial populations to UK patient characteristics, estimating baseline event rates for modelling, supporting extrapolation of trials and to identify real-world dosages of medicines used and associated costs. The authors state that RWE to provide comparative effectiveness estimates of medicines remains in early stages but demonstration projects are ongoing. NICE takes a ‘living’ approach to updating the framework so that it remains up to date. Small updates have been made since the initial publication such as referencing the HARPER protocol template [13]. Areas for future consideration include data needs for rare diseases (exploring federated data networks for example) and the use of RWE to evaluate medical technologies. 2023 has seen the publication of a number of RWE guidances, and as one of the first bodies to produce one, it is good to see NICE reflecting on the impact their framework is making and identifying areas to update. Hopefully the NICE team will continue to do this – in an ever-evolving field, checking what RWE is being submitted and how it is being appraised allows us all to learn for the ultimate goal of patient access to medicines. Ideally, other HTA agencies will follow NICE's suit and also provide reflections on the impact of their guidances.

It is well known that patient access to medicines in the US is much quicker than in Europe. As such, researchers from NICE and Flatiron Health were interested in whether these differential timelines could be capitalized upon in terms of evidence generation [14]. In particular, they wanted to assess whether RWD on the new medicine arising from post FDA approval real-world use in the US could be useful for reimbursement decision making ex-US (in this case in England for NICE). The authors reviewed 60 NICE oncology technology appraisals associated with 39 cancer therapies. For 59 out of the 60 appraisals, the technology was approved in the US before NICE published final recommendations. The median time from US FDA approval to NICE submission and final guidance publication was approximately 6 months and 19 months, respectively. The available US RWD for these medicines between FDA approval and NICE milestones was investigated- however generally there was not enough patients and/or follow-up of patients to address questions of real-world overall survival in time for NICE decision making. This was true even for re-appraisals of medicines that were part of a managed access agreement. There may be other types of evidence, such as profiling patients receiving treatment in clinical practice and understanding their treatment duration which could be valuable and be generated in time for initial NICE decision making. For real-world effectiveness ultimately these findings show that the use of US RWD will be more appropriate for later reassessments by ex-US HTA agencies, perhaps as we move toward more living/lifecycle HTA in the future [15].