R WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 4

The potential for real-world data (RWD) to be used to estimate comparative treatment effects has increasingly been recognized by health technology assessment (HTA) agencies and other reimbursement authorities. Notably, this recognition has extended beyond the traditional post-launch setting to include the potential use of real-world external control arms in the prelaunch setting, particularly when randomized estimates of the treatment effect of interest are not available. Despite the increasing recognition of the potential of real-world external control arms, work published by Jaksa et al. earlier this year provides compelling evidence that comprehensive guidelines describing best practices for such studies are lacking [1]. In the study the authors defined a set of criteria that guidelines for real-world evidence should meet, and classified 41 published methodological guidelines from regulators, HTA agencies, RWE initiatives and professional societies against them. The output of the exercise clearly demonstrated the gaps in existing guidelines and led the authors to call for collaborative efforts to support the development of comprehensive consensus guidelines for real-world studies in regulatory and reimbursement settings.

In the absence of such guidance, those carrying out real-world studies have been left to determine best practices from a fragmented set of related guidelines published by regulators, HTA bodies and academic groups. In this regard, the recent publication in the Journal of Comparative Effectiveness Research by colleagues from the EU Horizon 2020 IMPACT-HTA program provides welcome insight for those generating and reviewing such evidence [2]. In the study the authors developed a set of best practice recommendations for those generating non-randomized comparisons based on a review of existing guidelines, engagement with HTA bodies and a workshop with 16 subject matter experts from eight countries. The broad recommendations cover best practices in study planning, design, analysis and reporting and, while many of the recommendations may be seen as not being terribly new and echo those covered in existing guidelines, a number of previously neglected approaches such as quantitative bias analysis are covered. While insightful, the paper stops short of providing detailed methodological guidelines, instead echoing the calls of Jaksa et al. for collaborative work to develop such guidelines. The paper also provides recommendations for additional HTA system developments that might better facilitate the greater use and acceptance of non-randomized treatment effects in HTA, including the further development of early engagement and conditional access processes, the training of HTA staff in the relevant methodologies and the greater support of initiatives to improve data access and develop novel methodologies.

Additionally of note, the Haute Autorité de Santé (HAS) have recently sought to address the need for such guidelines with the publication of their methodological guide for the use of real-world studies in the assessment of medicinal products and medical devices in France [3]. The HAS guide provides insights on a number of areas including considerations around situations in which the use of RWE may be beneficial and approaches to address bias and confounding. Despite this, recommendations in the guide are relatively broad in nature and those involved in conducting real world studies for the purpose of submission to HAS are directed to a set of referenced international methodological guidelines for more detailed guidance on best practices. While this is perhaps an understandably efficient approach, it means the guide does not address a number of the areas highlighted by both Jaksa et al. [1] and the HTA-IMPACT team [2] and leaves those conducting studies including real world control arms with the aforementioned challenge of piecing together a best practice approach from the relatively fragmented set of existing non-comprehensive guidelines.

The need for clear guidance on what RWD and methods of analysis would be accepted by HTA bodies is crucial. The alternative is that patient data is used ‘blindly’, raising ethical concerns, and the potential for patients to be denied access to potentially beneficial medicines as a result of avoidable concerns about bias and overall data quality. There is an opportunity for all stakeholders to collaborate to ensure that new guidelines are clearer and more specific, so that RWD can be used effectively to help establish clinical benefit in the absence of other data. While we wait for the collaborative development of more concrete guidelines, future work using synthetic control arms could take note of the recent additions to the guidance by undertaking analyses, such as quantitative bias analysis to address confounding, with a hope that it will take us a step closer to the acceptance of RWE by HTA agencies for the assessment of clinical benefit, ultimately allowing patient access to beneficial medicines.