Developing an integrated strategy for evidence generation
It is common for pharmaceutical companies to consider evidence generation as the responsibility of individual departments (e.g., clinical development, medical affairs and health economics and outcomes research). This typically means that evidence is generated in a sequential fashion; for example, waiting for regulatory approval before initiating an outcomes-based study. This is a relatively risk-averse strategy that has served the industry well in generating evidence to satisfy regulatory and reimbursement decisions, and until recently, there has been little need to improve it.
An integrated approach to evidence planning that involves the bringing together of randomized clinical trial (RCT)- and real-world evidence (RWE)-based approaches across all departments offers an innovative operating model. It benefits industry by generating the RWE required to meet the increasing demands of decision makers, and benefits patients by generating value-based outcomes that translate into clinically meaningful effects in the real world. There are many components that can be informed by RWE when establishing an integrated evidence strategy and optimizing evidence generation. Here, we focus on end point strategy, Phase III population identification, RCT recruitment, patient-reported outcomes (PROs), outcome agreements and effectiveness prediction.
End point strategy
Clinical research should ultimately improve patient care. For this to be possible, trials must evaluate outcomes that reflect real-world scenarios and address the concerns of patients, physicians and payers [1]. Pharmaceutical companies should adopt approaches to identify end points that translate into valued outcomes that can be measured in the real world. One way is to ensure that trials are conceived and designed with greater input from patients, caregivers, payers and physicians, by establishing ‘priority-setting partnerships’ that enable these groups to work together to identify and prioritize uncertainties about the effects of treatments that could be answered by research [2]. Mobile solutions, such as patient- or disease-relevant applications, can enhance patients’ experiences and improve their outcomes, thereby capturing patient insights to better inform clinical end point selection [3].
Phase III population identification
The use of real-world data (RWD) to optimize trial design, particularly in assessing trial feasibility (i.e., likely patient attrition for the suggested inclusion and exclusion criteria), is widely recognized [4,5]. More recently, sophisticated methods to assess the impact of each trial criteria on recruitment, efficacy end points and risk have been developed, which could permit use of a patient population that is more representative of the real world in clinical trials without compromising efficacy or increasing the potential risks [6,7].
RCT recruitment
RWE can be used to identify patients for potential recruitment into trials by analyzing data patterns on diagnosed patients and applying these through predictive analytics to identify patients who are at high risk for, and could therefore have the condition. One sponsor conducting a clinical trial in patients with a rare disease in Europe overcame plateauing numbers of enrolled patients by collaborating with their contract research organization in a data- and analytics-driven approach to identify target patient pools and accelerate recruitment [8].
Patient-reported outcomes
Selection and utilization of appropriate PRO measures across a development program will continue to be important. However, an effective PRO strategy must go beyond ‘traditional’ questionnaires to assess the impact of an intervention on key measures, particularly adherence and quality of life. It is imperative to ensure that the voice of the patient is heard and incorporated into many aspects of evidence generation. Digital health communities present an opportunity to tap into real-world, unguided self-reported patient narratives that can be used to evaluate whether PRO measures truly capture the patient voice and impact of disease on health-related quality of life. One innovative method that can help ensure this information is captured is social media listening. With the correct structure and context, social media listening can provide insights into patient characteristics, treatment satisfaction, treatment switching, reasons for switching and much more [9]. Studies capturing this patient-reported data can be extended by linking data from claims and electronic medical record data systems to obtain a more complete picture [10]. Social media listening is only one option for tuning into the patient voice and validating the relevance of PROs from a patient perspective. Information held within other health communities (such as patient networking sites, disease-specific forums, general health forums and doctor Q&As) can be harnessed and processed using natural language processing and machine learning tools (such as RLytics) and analytical models (such as SPEC-R), which categorize and quantify reports regarding symptomatology and functioning [11,12].
Outcome agreements
Outcome agreements, where reimbursement of a manufacturer's product is based at least partly on measured patient/health system outcomes, are becoming increasingly prevalent. For example, the shift from volume- to value-based reimbursement put in place by the Medicare Access and CHIP Reauthorization Act of 2015 replaces traditional fee-for-service payments with a financial incentive framework that rewards for improved quality, outcomes and costs. This has provided strong incentives for other payers, such as those in the UK, to move in the same direction.
There are several points to consider with such agreements – which outcomes are measured and how, who measures the outcomes and where (i.e., which data source), and how is success defined? Integration of RWE early in this process can help to address some of these questions before forming agreements, and in establishing RWE platforms that enable patient outcomes to be measured as part of a negotiated outcome agreement.
Effectiveness prediction & optimization
Advanced analytics from RWD can help identify barriers to diagnosis, improve understanding of care pathways, and examine patients who are receiving suboptimal care and have great needs for outcome improvement. Advanced analytics can also support the development of predictive algorithms that identify high-value segments of a patient population, for whom a treatment demonstrates its greatest effectiveness, or provides a strong likelihood of response to a treatment compared with alternative treatment options. Analytics identifying such high-value subgroups of patients can lead to screening algorithms or clinical decision support tools to help physicians identify patients on inappropriate treatments and reduce negative outcomes. Last, bridging studies – which address questions relating to inherent differences between clinical trial and local populations – generate RWE to demonstrate whether outcomes observed in clinical trials can be extrapolated to the local population. This evidence helps to meet the demands of payers and providers for local RWE around the time a product is launched [13].
Benefits & risks of an integrated approach
A new approach will always have a perceived element of risk, and there are aspects of an integrated approach to evidence generation that may be of concern to those accustomed to the ‘traditional’ industry development model. For example, defining a target/optimum patient population more precisely with an integrated RWE plan can drive external pressures (e.g., payers limiting reimbursement to the ‘highest-value’ subgroups) and internal pressures (e.g. higher price to mitigate a smaller patient population, difficulty in achieving the required net present value for go/no-go decisions). Compared with traditional regulatory end points, some patient-relevant disease outcomes (e.g. PROs) will be impacted differently by treatment interventions, with potential variation between patient cohorts – practically, this may mean a more complex and potentially less clear value proposition. There are also challenges in balancing the different evidence requirements of payers within different countries.
However, these concerns are far outweighed by the potential benefits of an integrated approach. The early planning and generation of evidence required by payers means the evidence required for reimbursement submission is available without delay, and more effective clinical trial design in terms of recruiting, site feasibility and end points will bring efficiencies in terms of time and cost. In addition, a cross-functional approach ensures the required expertise is available at all stages of the product life cycle, and external collaborations (e.g. with digital agencies or academia) permit access to data sources and technical, analytical and clinical expertise.
The changing external environment as a driver of an integrated approach
We have already discussed the merit of an integrated approach as a way for industry to be proactive in generating meaningful, patient-relevant evidence. However, there are several external factors that put pressure on the function-based approach to evidence generation, including regulatory requirements regarding RWD, payer- and patient-relevant end points playing an increasing role in reimbursement decisions, and increasing constraints on healthcare budgets.
Regulatory requirements
The signing of the 21st Century Cures Act mandates the US Food and Drug Administration (FDA) to create guidance for the use of RWE to support regulatory decision-making [14]. In addition, the FDA recently issued guidance on how it evaluates RWE in support of medical devices [15], and has also stated its plan to consider RWE more broadly [16]. RWE is already routinely used in the EU, particularly for products already on the market and for safety monitoring and drug utilization [17]. However, the European Medicines Agency also recognizes that RWE has great potential to support lifecycle product development and monitoring and to improve decision-making for regulation and health technology assessment (HTA) [17]. Real-world studies may also be mandated by regulators as a condition of market authorization.
Payer- & patient-relevant end points
HTA agencies worldwide are exploring whether RWD provide insights not captured by RCTs. Such insights can make the difference between success and failure in patient access. For example, in June 2013, the Scottish Medicines Consortium (SMC) rejected a new oncology product because of the lack of a sufficiently robust economic analysis, despite one Phase II RCT demonstrating significantly longer overall survival compared with standard of care [18]. Following this, the SMC gave a positive reimbursement decision based on a revised economic model that used RWE to demonstrate a substantial improvement in quality of life [18].
Increasing constraints on healthcare budgets
To control ever increasing healthcare budgets, there is a need to identify the value that a new medicine offers the healthcare system. Hence, payers demand contextual evidence on the disease and its management to ensure value in the local healthcare setting. Outcome agreements are one way of managing the risk associated with adoption of a drug. For example, Novartis has agreed to value-based pricing for Entresto, which is influenced by the rate of heart failure-associated hospitalization [19].
Growing evidence opportunities
The data and technology needed to address this changing external environment have previously either not been available or have been inaccessible or difficult to work with. As rich, diverse sources of RWD become available and accessible, and analytical tools continue to grow in power and sophistication, the opportunity to efficiently generate the RWE needed to support improved decision-making exists [20].
Conclusion
An integrated approach to evidence generation is not a tick-box exercise; it represents a complete change in mindset and way of working from the traditional models of evidence generation, and typifies a new way of doing business. Establishing a cross-functional method of working and bringing together different skill sets from across the organization is just the first step; ensuring tangible success of the integrated model requires pharmaceutical companies to invest in RWE activities early in the product life cycle, at risk. A successful integrated approach to evidence planning and generation will provide decision-makers with evidence that is delivered earlier, through more efficient and representative studies, and better tailored to the payer, provider and patient. Ultimately, it is patients who will benefit.
Financial & competing interests disclosure
Development of this editorial was funded by Novartis Pharma AG. M Olson is an employee of Novartis Pharma AG. The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Medical writing support was provided by Carly Sellick of PharmaGenesis London, London, UK with funding from Novartis Pharma AG. The manuscript was developed with valuable input from Kris Kahler and Hui Cao of Novartis Pharma AG, Basel, Switzerland.
References
Papers of special note have been highlighted as: • of interest
1.
Heneghan C, Goldacre B, Mahtani KR. Why clinical trial outcomes fail to translate into benefits for patients. Trials 18(1), 122 (2017).
2.
James Lind Alliance. About priority setting partnerships (2017). www.jla.nihr.ac.uk/about-the-james-lind-alliance/about-psps.htm.
3.
Reiman EM, Langbaum JB, Tariot PN. Endpoints in preclinical Alzheimer's disease trials. J. Clin. Psych. 75(6), 661–662 (2014).
4.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int. J. Nurs. Stud. 50(5), 587–592 (2013).
5.
Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med. Res. Methodol. 10, 67 (2010).
6.
QuintilesIMS. Insights: research & development; impact of genomic data on building RWE for R&D (2016).
7.
Karcher H, Meng J, Fu S, Loefroth E, Cao H, Peress E. Optimal design of pre-authorization trials for effectiveness evaluation in severe asthma. Poster Presented at: 19th Annual European Congress for the International Society for Pharmacoeconomics and Outcomes Research. Vienna, Austria, 29 October–2 November 2016.
8.
QuintilesIMS. A new wave of innovation for real-world evidence. Access Point 7(13), 1–57 (2017).
• Access Point is a publication with its sights set squarely on ‘what's next?’ for real-world evidence (RWE). Through articles and case studies from some of the world's leading RWE experts, the authors aim to evaluate trends, introduce new applications and push those involved in the field to continually rethink ways of generating RWE.
9.
Risson V, Saini D, Bonzani I, Huisman A, Olson M. Patterns of treatment switching in multiple sclerosis therapies in US patients active on social media: application of social media content analysis to health outcomes research. J. Med. Internet Res. 18(3), e62 (2016).
• A good example of how the applicability of social media analysis to outcomes research can be tested using a comprehensive automated listening combined with filtering and analysis of data by specialists.
10.
Risson V, Ghodge B, Bonzani IC et al. Linked patient-reported outcomes data from patients with multiple sclerosis recruited on an open internet platform to health care claims databases identifies a representative population for real-life data analysis in multiple sclerosis. J. Med. Internet Res. 18(9), e249 (2016).
• An example of how patient-reported outcome data captured through online communities can be linked with data from healthcare claims databases in order to identify a representative population for real-world data analysis.
11.
Doogan S, Heid J, Benosman S et al. Using self-reported patient experiences to evaluate patient reported outcomes (PRO) instruments: learnings from digital patient communities in psoriatic arthritis. Am. J. Cardiol. A1423 (2016).
12.
Sunkureddi P, Gibson D, Doogan S et al. Using self-reported patient experiences to understand patient burden: learnings from digital patient communities in ankylosing spondylitis. Am. J. Cardiol. A1426 (2016).
13.
Liu JP, Chow SC. Bridging studies in clinical development. J. Biopharm. Stat. 12(3), 359–367 (2002).
14.
21st Century Cures Act, HR 34, 114th Cong (2015). www.congress.gov/bill/114th-congress/house-bill/34/.
15.
US FDA. Use of real-world evidence to support regulatory decision-making for medical devices (2016). www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm513027.pdf.
• Guidance document for industry and US FDA staff which communicates the FDA's perceptions on the use of RWE to support regulatory decision-making for medical devices.
16.
Regulatory Affairs Professionals Society. Califf: leveraging real world evidence is ‘top programmatic priority’ for FDA (2017). www.raps.org/Regulatory-Focus/News/2016/05/11/24929/Califf-Leveraging-Real-World-Evidence-is-Top-Programmatic-Priority-for-FDA/.
17.
European Medicines Agency. Paper for STAMP meeting 10 March 2016: update on real world evidence data collection (2016). http://ec.europa.eu/health//sites/health/files/files/committee/stamp/2016–03_stamp4/4_real_world_evidence_background_paper.pdf.
18.
Scottish Medicines Consortium. Aflibercept (Zaltrap) (2014). www.scottishmedicines.org.uk/SMC_Advice/Advice/878_13_aflibercept_Zaltrap/aflibercept_Zaltrap_Resubmission.
19.
Managed Healthcare Executive. Novartis signs on to value-based pricing for Entresto (2016).http://managedhealthcareexecutive.modernmedicine.com/managed-healthcare-executive/news/novartis-signs-value-based-pricing-entresto.
20.
Califf RM, Robb MA, Bindman AB et al. Transforming evidence generation to support health and health care decisions. N. Engl. J. Med. 375(24), 2395–2400 (2016).
• Overview of how evidence generation can be transformed to make better use of RWE to support health and healthcare decision-making.
Information & Authors
Information
Published In
Copyright
© 2018 Future Medicine Ltd.
History
Received: 20 September 2017
Accepted: 22 September 2017
Published online: 20 October 2017
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Developing an integrated strategy for evidence generation. (2017) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2017-0073
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Suguru Okami, Kako Shimotsumagari, Yasunari Sadatsuki, Evolving Real‐World Data and Evidence Use for New Drugs and Regenerative Medical Products Approvals in Japan—An Analysis of the 6‐Year Trend, Clinical Pharmacology & Therapeutics, 10.1002/cpt.70081, 118, 6, (1405-1421), (2025).
- Won Chan Lee, Chris Blanchette, Shibani Pokras, Javed Shaikh, Jared Miller, The evolution and future of integrated evidence planning, Expert Review of Pharmacoeconomics & Outcomes Research, 10.1080/14737167.2025.2497876, 25, 6, (855-862), (2025).
- Joseph A. DiMasi, Melvin Skip Olson, Zachary Smith, Kenneth A. Getz, Gorana Capkun, Assessing the Value of Integrated Evidence Approaches in Drug Development, Therapeutic Innovation & Regulatory Science, 10.1007/s43441-025-00778-y, 59, 4, (808-816), (2025).
- Nikolaus G. Oberprieler, Manel Pladevall-Vila, Catherine Johannes, J. Bradley Layton, Asieh Golozar, Martin Lavallee, Fangfang Liu, Maria Kubin, David Vizcaya, FOUNTAIN: a modular research platform for integrated real-world evidence generation, BMC Medical Research Methodology, 10.1186/s12874-024-02344-w, 24, 1, (2024).
- Melvin (Skip) Olson, Gorana Capkun, The value of evidence and its role in driving product strategy, Journal of Comparative Effectiveness Research, 10.57264/cer-2024-0074, 13, 8, (2024).
- Daniel Furtner, Gabor Hutas, Bryan Jie Wen Tan, Roland Meier, Journey from an Enabler to a Strategic Leader: Integration of the Medical Affairs Function in ESG Initiatives and Values, Pharmaceutical Medicine, 10.1007/s40290-023-00485-9, 37, 6, (405-416), (2023).
- Séverine Sarp, Ramona Reichenbach, Paul Aliu, An approach to data collection in compassionate use/managed access, Frontiers in Pharmacology, 10.3389/fphar.2022.1095860, 13, (2022).
- Laura Dormer, Celebrating 10 years of the Journal of Comparative Effectiveness Research, Journal of Comparative Effectiveness Research, 10.2217/cer-2021-0249, 11, 1, (1-6), (2021).
- Zhongwen Tang, Integrated evidence strategy and drug development efficiency, Biometrics & Biostatistics International Journal, 10.15406/bbij.2020.09.00315, 9, 5, (164-167), (2020).
- Melvin (Skip) Olson, Can real-world evidence save pharma US$1 billion per year? A framework for an integrated evidence generation strategy, Journal of Comparative Effectiveness Research, 10.2217/cer-2019-0162, 9, 2, (79-82), (2019).
