Deriving more value from RWE to ensure timely access of medicines by patients
First draft submitted: 27 April 2017; Accepted for publication: 28 April 2017; Published online: 12 July 2017
Real world evidence (RWE) is defined as a form of evidence generated to answer a question or test a hypothesis using real world data (RWD) and using appropriate design and statistical analyses [1]. Others define RWE as the intersection between the randomized controlled trials (RCTs) and effectiveness of treatment in usual clinical practice [2]. To those with public health expertise, RWE is not totally a new phenomenon. Population-based studies using RWD are often conducted to evaluate the effectiveness and value of health programs and policies. Thus, RWE is an old wine in a new bottle. RWD characterized as patient registries have been a long tradition in the Nordic countries due to publicly financed healthcare and a unique person identifier, which enables linkage of data from different registries [3]. However, the impact of RWD/RWE on health technology assessment (HTA) decision-making process regarding new treatments is not well assessed and the explanatory factors are not well known.
Why RWE now?
RWE has recently gained attention from patients, providers, payers, regulators and pharma industry. People are living longer in almost every part of the world with increasing demand for a healthy living. Technological advancement enables quick flow of data and information, thus increasing level of knowledge on health and on what works and what does not [4]. Patients have high expectations on health systems to offer safe and effective treatments.
Parallel to this, global public health trends show increasing burden of noncommunicable diseases due to unhealthy lifestyle [5], implying a large proportion of people living a long unhealthy life about 30–40 years.
Healthcare costs are rising at a much faster rate than gross domestic product (GDP), creating budget deficiencies [5]. This reinforces payers to ensure that they put money in treatments that are cost effective, add value to patients and improve population health.
Thus, regulatory agencies begin to realize the benefit of RWE for regulatory decision-making for medicines and medical devices.
Thus, in order for the industry to sell their products, they need RWE studies to address effectiveness, efficiency, value and safety of their products. Some companies have developed strategies to generate RWE, others are using ad hoc solutions, while the rest are waiting for the RWE wave to pass. Developing a new treatment is costly, yet a large proportion will never make it to the market. RWE is a potential cost-saver in various phases of the drug development lifecycle [6].
RCTs are regarded as the gold standard for providing the scientific evidence of clinical efficacy of treatments. However, evidence from the RCTs does not address real life conditions due to strict selection criteria with a limited study population, questionable external validity and generalizability [7]. RWE studies, on the other hand, demonstrate effectiveness in real world clinical practice, are generalizable, broader, more representative of the general patient population and demonstrate value for patients and benefits within health systems [1,7–8]. RWE can therefore bridge the gap between efficacy and effectiveness.
Realizing the real value of RWE
RWE can complement RCTs and can play a big role in ensuring the practice of value-based medicine (VBM) in healthcare. VBM is the practice of medicine that incorporates the highest level of evidence-based data with the patient-perceived value conferred by healthcare interventions for the resources expended [9,10].
Although RWE has quickly gained momentum, however, there is a gap between enthusiasm level and how it is actually put in practice. We discuss below four key factors that can contribute to realizing the real value of RWE and bridge this gap.
Understanding the framework for generating RWE
RWD is sometimes interchangeably used to mean RWE. Thus, understanding the scientific framework to generate RWE is a key factor in getting the real value from RWE. De Lusignan et al. have previously demonstrated a four-step framework for generating RWE [1]. In this paper, we suggest a six-step framework starting with addressing evidence gaps (Figure 1). Appropriate identification of evidence gaps need to include various perspectives from patients, providers, healthcare professionals, payers and regulators. Failure to do this may result in formulating incorrect research questions, which may impact the rest of the steps toward RWE generation. Thus, the evidence generated could be of little use.

Figure 1. Framework for developing real world evidence.
RWD: Real world data; RWE: Real world evidence.
Another crucial step is the identification and assessment of RWD. The quality of the available RWD sources vary significantly and there is limited research to understand this aspect. A structured search and assessment of quality and detailed information on available RWD sources, including disease/intervention registries, is essential to enable qualitative outcomes. Additionally, periodic assessments of data quality are needed to ensure accuracy and quality of data [11].
In Figure 1, we additionally suggest that the generated RWE should be disseminated. Disseminating RWE would contribute to determining standards for clinical effectiveness and identifying new evidence gaps and RWD sources, ultimately increasing efficiency in the industry and health systems and benefiting patients.
Increasing reputation of RWE studies
RCTs can minimize bias. RWE studies, on the other hand, are often associated with bias due to confounding, selection bias and follow-up of patients [12], leading to under- or over-estimation of the true effects.
A large proportion of RWD is underutilized due to the quality of data, limited skills to evaluate fragmented data with sophisticated methodology. Assessment of 103 Swedish healthcare quality registries demonstrated difficulties related to generalizability about their quality [3]. Explanations included: completeness, variations in coverage and validity, and information on timeliness. However, there are no agreed standards on scientific methods for measuring data quality, nor what type of RWD is acceptable to HTA's assessments.
Synergus has assessed the quality RWD resources related to breast cancer, stroke, diabetes and heart failure in Europe. They demonstrated, for example that Sweden, Norway, Italy and Spain showed the best use of heart failure registries as reflected by impact scores of the journals where the data were published [13]. The quality of data seem to play a big role in publication acceptance, especially in high impact score journals.
There are advanced statistical methods that can appropriately address bias in RWE studies [14] and statistical packages to perform. Stel et al. evaluated statistical methods that can minimize bias, including multivariable analysis, propensity score risk adjustment and the instrumental variable method [15]. Instrumental variable analysis was demonstrated as the closest to RCTs regarding minimizing selection bias.
Capabilities for generating RWE solutions
RWE capabilities include financial resources, RWD sources, skills and expertise and structure (organization). We do not think that the big challenge is related to financial resources or organizational structure for RWE. In fact, many pharma companies have created an internal structure for RWE and allocated resources [6].
We highlight challenges related to RWD, skills to analyze RWD and expertise generate relevant RWE within the industry. Within the HTA agencies, the challenges are related to the quality of RWD and interpretation of the evidence.
Appropriate RWD research practices in relation to data collection (quality, variables, time frames, periodic monitoring), planning, sample sizes, consent, privacy, data linkages and data analysis are essential.
Traditional design of research has limitations in understanding the real world environment and health systems. There is often limited engagement of patients and relevant stakeholders in research design. Additionally, there are limited skills in epidemiologic methods [17] to analyze complex data while appropriately handling biases. The industry often has an inward perspective [18], creating a missed opportunity to take advantage of relevant skills that are needed in the identification of evidence gaps, analyses of RWD, generation of evidence and efficient communication of RWE solutions to regulators, payers, providers and patients.
Public–private partnerships & creating a critical mass of RWE scientists
Partnerships between public and private sector have long been impended by thick walls dominated by suspicion [19]. Such partnership can solve challenges related to identifying relevant evidence gaps in real life, RWD, evidence generation and dissemination, determining standards for clinical effectiveness and finally supporting younger scientists. It is the basis of such partnerships that we can realize the real value of RWE.
Creating a critical mass of RWE scientists is important and we support the idea by Gill et al. of initiating a scientific association of RWE [20]. The RWE association could act as a strategic step toward bringing together RWE scientists, relevant partners to create a multiplier effect related to focused leadership, further development of RWE research methods, easier access and better use of RWD, attracting younger RWE scientists and improving the scientific reputation of RWE.
Is RWE here to stay or is it a trendy phenomenon that will go way?
Patients and providers want evidence proving that treatments are safe and effective. Payers are pressured with budgets and want evidence proving that treatments are cost effective. Epstein et al. suggest approaches to effective process of value evidence generation both in and outside the clinical drug development process [21].
There is a growing trend toward conditional approval by the regulators. This is where a new treatment or device is approved to treat a subpopulation while companies are asked to generate more evidence based on real world conditions [22].
HTAs are adapting a similar approach to recommend on condition that companies can provide RWE to address effectiveness in real world patients in actual healthcare setting [23]. However, there are no common policies for which type of RWD is needed to generate evidence in the HTA process [24] and how RWE is used [25].
There is an ongoing discussion on adaptive licensing in Europe, USA and Japan and its implementation will have a huge implication on volume of RWE studies [26].
Way forward with RWE
RWE is here to stay as it enables VBM's approach. The demand for high-quality RWD sources and RWE will continue to grow.
The industry, payers and regulators need to embrace and implement RWE in a transparent and consistent manner and scientists need to increase the reputation of RWE to ensure value for patients.
Success factors include doing things differently, access high-quality RWD, use of sophisticated analyses, generation and dissemination of relevant RWE solutions and creating a critical mass of RWE scientists with multidisciplinary expertise.
Additionally, the HTAs particularly in the European context need to adapt a similar approach on which type of RWE solutions is acceptable to bridge the gap between efficacy and effectiveness.
Acknowledgements
Some of the issues covered in this article have been the subject of presentations given by SW Andersson at the Evidence 2017 conference (22–23 February 2017, London, UK).
Financial & competing interests disclosure
The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.
References
1.
De Lusignan S, Crawford L, Munro N. Creating and using real-world evidence to answer questions about clinical effectiveness. J. Innov. Health Inform. 22(3), 368–373 (2015).
2.
Looney W. Real world evidence – from volume to value. Pharmaceutical Executive 36, 10 (2016).
3.
Emilsson L, Lindahl B, Köster M et al. Review of 103 Swedish healthcare registries. J. Intern. Med. 277, 94–136 (2014).
4.
Upbin B. PatientsLikeMe is building a self-learning healthcare system. Forbes (2013). www.forbes.com/sites/bruceupbin/2013/03/01/building-a-self-learning-healthcare-system-paul-wicks-of-patientslikeme/#7e10eed6e30e.
5.
Muka T, Imo D, Jaspers L et al. The global impact of non-communicable diseases on healthcare spending and national income: a systematic review. Eur. J. Epidemiol. 30(4), 251–277 (2015).
6.
Hughes B, Kessler M. The $1 billion real world evidence. IMS Health (2014). www.imshealth.com/files/web/Global/Services/Services%20TL/rwes_breaking_new_ground_d10.pdf.
7.
Czirakym M, Pollock M. Real world evidence studies. Applied Clinical Trials (2015). www.appliedclinicaltrialsonline.com/real-world-evidence-studies?pageID=2.
8.
Sherman RE, Anderson SA, Dal Pan GJ et al. Real-world evidence – what is it and what can it tell us? N. Engl. J. Med. 375(23), 2293–2297 (2016).
9.
Petrova M, Dale J, Fulford BK. Values-based practice in primary care: easing the tensions between individual values, ethical principles and best evidence. Br. J. Gen. Pract. 56(530), 703–709 (2006).
10.
Brown MM, Brown GC. Update on value-based medicine. Curr. Opin. Ophthalmol. 24, 183–189 (2013).
11.
Ludvigsson JF, Andersson E, Ekbom A et al. External review and validation of the Swedish national inpatient register. BMC Public Health 11, 450 (2011).
12.
Trojano T, Tintore M, Montalban X et al. Treatment decisions in multiple sclerosis – insights from real-world observational studies. Nat. Rev. Neurol. 13(2), 105–118 (2017).
13.
Synergus. Where is the best quality real world data regarding heart failure in Europe. https://synergus.com/blog/where-best-quality-real-world-data-regarding-heart-failure-europe.
14.
Anderson SR, Auquier A, Hauck WW et al. Statistical Methods for Comparative Studies: Techniques for Bias Reduction. Wiley & Sons, NJ, USA (2009).
15.
Stel VS, Dekker FW, Zoccali C, Jager KJ. Instrumental variable analysis. Nephrol. Dial. Transplant. 28(7), 1694–1699 (2013).
16.
Garavan T. Global talent management in science-based firms: an exploratory investigation of the pharmaceutical industry during the global downturn. Int. J. Human Res. Management 23, 12 (2012).
17.
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342(25), 1887–1892 (2000).
18.
Thygesen LC, Ersboll AK. When the entire population is the sample: strengths and limitations in register-based epidemiology When the entire population is the sample: strengths and limitations in register-based epidemiology. Eur. J. Epidemiol. 29, 551–558 (2014).
19.
Hall D. Why public–private partnerships don't work. Public Services International (2015). www.world-psi.org/sites/default/files/rapport_eng_56pages_a4_lr.pdf.
20.
Gill JL, Avouac B, Duncombe R et al. The use of real world evidence in the European context: an analysis of key expert opinion. London School of Economics (2016). http://blogs.lse.ac.uk/lseenterprise/files/2016/12/RWE_in_Europe_Paper1.pdf.
21.
Epstein RS, Sidorov J, Lehner JP, Salimi T. Integrating scientific and real-world evidence within and beyond the drug development process. J. Comp. Eff. Res. 1(Suppl. 1), 9–13 (2012).
22.
Garrison L, Neumann P, Erickson P et al. Using real world data for coverage and payment decisions: the ISPOR real-world data task force report. Value Health 10(5), 326–335 (2007).
23.
Pietri G, Masoura P. Market access and reimbursement: the increasing role of real-world evidence. Value Health 17(7), A450–A451 (2014).
24.
Makady M, Ten Ham R, De Boer A et al. Policies for use of real-world data in health technology assessment (HTA): a comparative study of six HTA agencies. Value Health 20(4), 520–532 (2017).
25.
White R. Building trust in real-world evidence and comparative effectiveness research: the need for transparency. J. Comp. Eff. Res. 6(1), 5–7 (2017).
26.
Eichler HG, Baird HG, Barker R et al. From adaptive licensing to adaptive pathways: delivering a flexible life-span approach to bring new drugs to patients. Clin. Pharmacol. Ther. 97(3), 234–246 (2015).
Information & Authors
Information
Published In
Copyright
© Future Medicine Ltd.
History
Published online: 12 July 2017
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Deriving more value from RWE to ensure timely access of medicines by patients. (2017) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2017-0030
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Lisa Masucci, Diedron Lewis, Jiahao Zhao, Caitlin Carter, Kelvin K.W. Chan, William W.L. Wong, The use of real-world evidence among healthcare payers: a scoping review, International Journal of Technology Assessment in Health Care, 10.1017/S0266462325100445, 41, 1, (2025).
- Kelvin Chan, Seungree Nam, Bill Evans, Claire de Oliveira, Alexandra Chambers, Scott Gavura, Jeffrey Hoch, Rebecca E Mercer, Wei Fang Dai, Jaclyn Beca, Mina Tadrous, Wanrudee Isaranuwatchai, Developing a framework to incorporate real-world evidence in cancer drug funding decisions: the Canadian Real-world Evidence for Value of Cancer Drugs (CanREValue) collaboration, BMJ Open, 10.1136/bmjopen-2019-032884, 10, 1, (e032884), (2020).
- Sajan Khosla, Robert White, Jesús Medina, Mario Ouwens, Cathy Emmas, Tim Koder, Gary Male, Sandra Leonard, Real world evidence (RWE) – a disruptive innovation or the quiet evolution of medical evidence generation?, F1000Research, 10.12688/f1000research.13585.2, 7, (111), (2018).
