Open access

White Paper

9 November 2018

Real-world evidence for coverage decisions: opportunities and challenges

Authors: Grace Hampson [email protected], Adrian Towse, William B Dreitlein, Chris Henshall, and Steven D PearsonAuthor Info & Affiliations

Publication: J. Comp. Eff. Res.

Volume 7, Number 12

https://doi.org/10.2217/cer-2018-0066

PDF

Abstract

Aim: To explore current uses of real-world evidence (RWE) in the US healthcare system, summarize key concerns and highlight various opportunities that could be realized through best use of RWE. Materials & methods: Information was gathered via a literature review and interviews to generate a background paper for the 2017 Institute for Clinical and Economic Review Policy Summit meeting. Results: RWE is currently being utilized in drug development decisions, regulatory approval decisions, post-approval monitoring, payer coverage decisions (initial decisions and reassessments) and for outcomes-based contracting. Solutions to key challenges and opportunities for future development are presented. Conclusion: Exciting opportunities for the use of RWE exist, yet important reservations remain. Solutions are within reach if effective partnerships between stakeholders can be nurtured.

The capacity of the US healthcare system to generate and interpret large amounts of data is advancing exponentially. Computer learning, natural language processing and the evolution of electronic health records (EHRs) are revolutionizing potential availability and interpretability of real-world data to improve health. Patient wearables (such as Fitbits and the Apple Watch) [1] and additional social media sources of data (such as Facebook and Twitter [2,3]) are providing new functionality, and can, in principle, contribute to a much richer and larger dataset for predictive purposes.

Concurrently, reliance on traditional randomized controlled trial (RCT) evidence for coverage decisions may be declining for several reasons. First, with the growth of personalized medicine, patient population and study sizes decrease, making it harder to conduct RCTs. Second, the per-patient and setup costs of conducting RCTs have been rising with increases in complexity and the need to adhere to external standards such as those set by the International Conference on Harmonization. Pragmatic clinical trials (PCTs) offer the advantages of randomization combined with the added relevance of results obtained in more real-world clinical settings, but these trials are often more expensive and more complex to design than traditional RCTs.

It is, therefore, crucial that developers of innovative products and the Health Technology Assessment (HTA)/payer community understand the potential for real-world evidence (RWE) to inform coverage decisions and price negotiations. To understand this potential, the opportunities presented by RWE must be viewed in balance with its limitations, both conceptual and practical. Data from EHRs are often incomplete, incompatible with those from other institutions or hard to obtain given privacy concerns, while RWE arising from observational studies cannot escape concerns about biases introduced by unknown confounders.

The purpose of this paper is to explore the current uses of RWE in the US healthcare system, summarize key concerns that have been raised in this area and identify various opportunities that could be realized through best use of RWE for coverage decisions.

A version of this paper was developed as a background paper [4] for the Institute for Clinical and Economic Review (ICER) Policy Summit held in December 2017, to provide participants with a common foundation in some of the key debates around, and opportunities for, the use of RWE. A conceptual framework to guide optimal development and use of RWE that was developed from the meeting is reported elsewhere in the journal [5]. Together, these two papers can serve to educate readers of the critical issues involved with RWE, offer a collaborative way forward and inform the ongoing discussion about RWE.

Materials & methods

Information and insights were gathered via: interviews with nine experts in RWE across the pharmaceutical industry, payer organizations and academia; a literature review and the viewing of two webinars – the FDA–Duke Margolis meeting on the use of RWE in regulatory settings, and the National Academies of Sciences Engineering and Math meeting on incentives for the use of RWE to improve the efficiency of innovation.

The interviewees were selected and invited to interview by ICER or the Office of Health Economics. They included two representatives from US payers (one pharmacy benefit manager and one health insurer), one representative from the National Institute for Health and Care Excellence in the UK, three representatives from pharmaceutical manufacturers, one representative of an organization working on the collection of RWE and two academics. All were experienced in the field of RWE. The interviews were semi-structured, following an interview guide that was sent in advance of the meeting, but allowing for further discussion and debate according to the interviewees’ expertise and interests. Questions included the following:

What do you see as the main potential advantages of using RWE (for payers/manufacturers);

What are the opportunities for RWE to be used most effectively;

What are the potential risks or concerns with RWE and what is needed to address these.

We also asked the interviewees to share any key resources or publications, and, where relevant, these were used to inform the literature review. The interviews were conducted by telephone by two interviewers (one from the Office of Health Economics, one from ICER), one of which served as a primary note taker, and the notes were subsequently cross-checked with the second interviewer and with the interviewees for accuracy.

The literature review was designed to be pragmatic rather than systematic and was undertaken to identify key opportunities and challenges for the use of RWE, not to identify all papers on RWE. Key resources were identified from among those suggested by interviewees, supplemented via searches in Google and Google Scholar and by searching the US FDA website. Additional searches were undertaken (also in Google and Google Scholar) targeted toward specific issues raised in the interviews, relevant ongoing initiatives in the field or specific barriers to adoption. Reference lists and citation lists of relevant literature were searched and additional resources identified.

The webinars, both two-day workshops, were used to further inform the authors’ thinking around these issues and provided up-to-date information on progress and perceptions around the use of RWE in the USA.

Given that the purpose of this paper was to summarize current uses, challenges and opportunities for RWE, the results are organized by topic, rather than by source. Comments that were explicitly provided by interviewees are noted as such, and key sources of literature are specified throughout. We start by setting out a working definition of RWE for the purpose of this paper and the ICER policy summit meeting, followed by an exploration of current uses of RWE in the US healthcare system, challenges associated with the use of RWE and finally, opportunities to enhance the generation and use of RWE. The opportunities are separated into two categories based on our assessment of the literature and interviewees comments: measures that can be taken today, and opportunities to be explored over a longer time horizon, as our potential to generate and analyze new datasets develops further. Key discussion points that were highlighted at the meeting are raised here and developed further elsewhere in this issue [5].

Results

What is RWE?

For the sake of clarity throughout this paper, we adopted the following definitions:

‘Real-world data (RWD) are data relating to patient health status and/or the delivery of healthcare collected either prospectively or retrospectively from observations of routine clinical practice. Examples of RWD include data derived from EHRs, claims and billing data, data from product and disease registries, patient-generated data including in home-use settings, and data gathered from other sources that can inform on health status, such as mobile devices.

RWE is the clinical evidence regarding the usage, and potential benefits or risks, of a medical product derived from analysis of RWD’ [6].

The authors writing on behalf of the FDA clarify that the distinguishing feature of RWE is the setting in which the evidence is collected (i.e., it is collected in healthcare settings rather than in a research environment), and not the presence or absence of a planned intervention or the use of randomization [7]. As such, we believe that it is useful to include PCTs within this definition.

How is RWE used?

Interviewees identified six key areas in which RWE is being used in the USA currently.

Drug development

Epidemiological data help to identify targets for the development of new therapies and can help inform decisions around the most appropriate drug development pathway. For example, evidence on population size, clinical disease progression and adherence to similar therapies can help in making decisions about whether to move to the next development phase, and if so, what that phase should look like in terms of study design.

Regulatory approval decisions

Use of RWE in initial FDA regulatory decisions to date has been limited (see [8] for a summary). Importantly, the 21st Century Cures Act (2016) specifically directs the FDA to consider the use of RWE for approval of new indications for FDA-approved drugs. The FDA is currently working on how to implement this requirement. A guidance has been issued on using RWE in medical device development [6], but guidance on drug products has not yet been released.

Postapproval monitoring of safety signals

The FDA use of RWE to monitor postapproval drug safety is much more established. The FDA's Adverse Event Reporting System collects information on adverse event and medication errors from healthcare professionals and product manufacturers. The FDA uses data generated via these systems to monitor medical products postlaunch and take regulatory action if necessary. One study [9] reported a rate of 0.68 safety-related label changes per drug per year (for nonexpedited pathway drugs), based on a review of drugs approved by the FDA during the period 1997–2016.

Another example is the FDA's Sentinel Initiative, a national electronic system that collects evidence on events and other targeted outcomes from a range of data partners who compile their analysis using EHRs, claims data and other data generated within their system. There have been suggestions that Sentinel is not having a measurable impact, and the FDA have explained that this is partly because for the first 8 years Sentinel was a pilot program (Mini-Sentinel) and only became fully operational in 2016 [10].

HTA assessments & payer coverage decisions: initial decisions

US payers often use epidemiological data, based in part on claims data, at this stage to generate estimates of the potential population; they cover that could receive the treatment, and to explore potential cost offsets. RWE can be used to estimate differing patient numbers depending, for example, on a drug's use in second- or third-line treatment or off-label use.

In making assessments of comparative effectiveness at launch, however, both HTA organizations like ICER and payers usually rely largely on RCT or other controlled trial evidence generated for regulatory submission. ICER's reports seek to emphasize comparative clinical effectiveness, and routinely use network meta-analysis to conduct quantitative indirect comparisons when the patient populations and outcome measures are similar enough to make these analyses feasible. ICER has also begun working whenever possible with patient groups to analyze existing patient-reported data, or to gather new information from patients to complement other data.

The existence of RWE for existing treatment options creates a common dilemma when HTA groups and payers assess the comparative effectiveness of a new drug that has no RWE available yet. Even when RWE is viewed as providing important insights into the performance of existing treatments, HTA/payers must determine whether and how to use it to judge the comparative effectiveness of existing versus emerging treatment options.

HTA assessments & payer coverage decisions: reassessments

RWE gives HTA/payers the opportunity to reconsider coverage, formulary placement and price/payment terms in light of how the products are performing in their relevant population. In addition to evidence on real-world safety and effectiveness, manufacturers can use RWE to provide payers with evidence on dimensions of value (from either a clinical and economic perspective) that are not evaluated during drug development.

Interviewees suggested that evidence of real-world effect is what matters to payers. Uses can be as broad as: trying to identify or rule out safety signals; measuring adherence in order to evaluate whether a drug that is slightly more tolerable could be more effective in the real world if patients actually take it; establishing effectiveness and value for money within the health plan's specific population (i.e., RWE can be a good test of external validity, as the product is evaluated within a more representative sample); or establishing effectiveness within subpopulations.

Outcomes-based contracting

The final area in which RWE is currently used by payers is outcomes-based contracting, in which payment is linked via rebate levels or some other mechanism to the demonstrated real-world outcomes of patients. Outcomes-based agreements have not featured prominently in the US healthcare system, largely because of the difficulty of collecting the RWE that could support such agreements [11], but some examples do exist, including:

Novartis’ agreement with the US Centers for Medicare and Medicaid Services (CMS) where CMS will cover Kymriah^® (Novartis, Basel, Switzerland) chimeric antigen receptor T-cell therapy (CAR-T therapy) only if patients respond within the first month after treatment;

Merck's agreement with Cigna and Prime Therapeutics to provide rebate payments for Rebif (Merck, Darmstadt, Germany) if hospital visits were required due to relapses [12].

Ongoing initiatives

In addition to these uses of RWE, interviewees highlighted various ongoing initiatives that are being used to explore how RWE can be used in decision-making:

The International Society for Pharmacoeconomics and Outcomes Research & the International Society for Pharmacoepidemiology joint taskforce

This taskforce was created to make recommendations regarding good procedural practices that would enhance decision makers’ confidence in evidence derived from RWD studies [13]. The papers produced by the taskforce [13–15] cover study registration, replicability, stakeholder involvement, reporting and transparency in RWE studies. They suggest, among other things, that greater transparency in reporting could lead to ‘a substantial improvement in reproducibility, rigor and confidence [in RWE]’ [15].

The NEW Drug Development Paradigms Initiative WISDOM project [16]

The NEW Drug Development Paradigms Initiative (NEWDIGS) describes itself as a ‘think and do’ tank that aims to ‘reliably and sustainably deliver new, better, affordable therapeutics to the right patients faster’. The aim of the WISDOM project is to explore how new kinds of evidence (integrated with that from traditional RCTs) could impact regulatory and reimbursement decision-making. The work includes ‘efficacy to effectiveness’ exercises, which aim to explore the gaps in evidence generation across various case-based scenarios. The aim is to ensure that evidence produced throughout development and into practice is ‘meaningful, valid, expedited and transparent’ (https://newdigs.mit.edu/sites/default/files/documents/NEWDIGS%20WISDOM%20June%202017.pdf).

The Center for Medical Technology Policy & Green Park Collaborative's RWE decoder

This is a tool to facilitate review and evaluation of RWE to enable decision makers to feel confident when making decisions based on RWE. It contains various modules in which the user inputs details of the studies, and produces a visual summary of available evidence according to relevance and rigor. It is available for use from the Center for Medical Technology Policy website.

The Patient-Centered Outcomes Research Institute Pragmatic Clinical Studies initiative

Since the Patient-Centered Outcomes Research Institute (PCORI) launched its Pragmatic Clinical Studies initiative in 2014, it has commissioned 28 projects, although none have yet reported results. Several major payers have also commissioned pragmatic clinical trials, for a recent example, see Anthem (www.healthcore.com/anthem-healthcore-boehringer-ingelheim-initiate-worlds-largest-pragmatic-clinical-trial-study-people-living-copd-real-world-setting/), and there are also NIH and industry's commissioned studies. We are not aware, however, of any prelaunch PCT for a drug being conducted in the USA.

Finally, the interviewees indicated that there is a trend toward more reliance on RWE for payer decision-making outside the USA.

Challenges associated with the use of RWE

Several challenges were identified via the literature review and the interviews. These include:

Bias & confounding

While observational RWE offers advantages against RCTs in terms of external validity, questions are often raised about internal validity. This is because observational analyses are inherently vulnerable to selection biases and confounding. Indeed Garrison et al. [17] suggest that this potential for bias is the biggest concern in the use of RWE, as they are not considered to ‘meet the methodological rigor of RCTs’. This topic is well documented [18–21], and the debate is not repeated here.

Reporting bias, which occurs when some outcomes (or datasets) are selectively revealed or withheld, is also a potential problem. Evidence of reporting bias, such as that provided by McGauran et al. [22], damages trust between stakeholders.

In order to mitigate the impact of biases, RWE studies need to be rigorously designed and evaluated. A mandatory national registry for studies, such as is available for RCTs, could help mitigate concerns of reporting bias.

Incomplete data

Datasets, particularly RWD, are vulnerable to systematic omissions or misclassification. An example is central line-associated bloodstream infection, which have been used as a publicly reported indicator of healthcare quality, leading to fears that cases may be intentionally under-reported [23]. Clearly, analysis of data that has been intentionally manipulated would not be reliable.

In addition, there are often gaps in the data. Claims data, for example, may reveal that a patient had a test, but not the results. Interviewees reported that data gaps are particularly prevalent when relying on patients or physicians to submit their own data, rather than when it is proactively collected by researchers.

In order to mitigate the challenge of incomplete data, national data repositories and strict guidelines for reporting to reduce data gaps could be helpful. The ability to link datasets to one another would also be a useful way of filling in gaps and validating data.

Data mining

Data mining occurs when analysts re-examine existing datasets to generate new information. It is not inherently bad, but the concern in the context of RWE is that organizations can continue to reanalyze datasets using different modeling approaches until preferential outcomes are identified. This highlights the vulnerability of RWE to manipulation via repeat analyses with nondisclosure of unhelpful results, and underlines the need for strict protocols, analysis plans and well-defined research questions.

Access to data

Sharing of data across different healthcare organizations is not common in the USA, leading to gaps in data. The challenge is exacerbated by legal frameworks that restrict data sharing and access to patient identifiable information. Cole and colleagues [24] suggest that these problems arise because RWD is being used for purposes beyond those for which it was originally collected – to directly manage the care of the patient. As a result, legal frameworks are ‘playing catch-up’ in order to respond to data demands, which clearly benefit patients and society but in a different way.

Clearly, appropriate governance arrangements for RWE are crucial to facilitate evidence collection and to make the most of healthcare information for improving patient care.

Lack of universally accepted methodological standards

Many of the challenges outlined so far are exacerbated by a lack of universally accepted standards or principles for the design, conduct, analysis and/or reporting of RWE. There have been various efforts to set out best practices and standards [24], but a review of nine of these [25] found a lack of agreement between the various sets of principles. This undercuts the potential value of the information that is produced.

In order for a set of standards to be useful, all stakeholder groups are required to ‘buy into’ them and agree that they offer the most suitable guidance for the development, conduct, analysis and/or reporting of RWE. Ideally, all studies can then be conceptualized, designed, conducted, analyzed and reported according to this common set of standards, thereby increasing transparency, reliability and trust in the results of RWE.

Lack of investigator expertise

It is important that investigators understand RWD well in order to be able to interpret it properly and adjust for systematic omissions and confounding biases appropriately. Interviews indicated a perception that there is a lack of expertise in this area, which is an important challenge because, whether or not it is correct, it erodes trust in RWE and undermines its conclusions.

Strong methodological guidance is required to ensure rigorous standards for analysis, and as RWD becomes more readily available, expertise will spread.

Obsolete evidence hierarchies

Traditional evidence hierarchies that promote RCTs as the gold standard for evidence generation were developed for a world without big data, and do not necessarily account for the potential for RWE to supplement our understanding of the safety and effectiveness of treatments in different populations. Indeed, RWE studies are typically more useful than trials to address key questions regarding durability of effect, generalizability and long-term safety.

Opportunities to enhance the generation & use of RWE

As the nature of the information that we are able to gather evolves and improves, it is important to consider how we can integrate these into assessment processes to make sure we are making decisions based on the best available evidence. Interviewees and Policy Summit attendees described several measures, many of which could be taken today, to improve the generation and use of RWE for use in coverage decision-making.

Measures that can be taken today

Increase the quality & credibility of RWE studies

Measures could include: establishing a mandatory national registry for observational studies, such as is available for RCTs; developing national data repositories for datasets accessible for research purposes; investment in the quality and consistency of EHRs in a way that would transform the potential value of RWE; strict protocols, analysis plans and well-defined research questions; seeking consensus on good practice guidelines, to encourage participation in the above. The International Society for Pharmacoeconomics and Outcomes Research/International Society for Pharmacoepidemiology Task Force papers provide a good starting point for a definitive set of principles and procedures [13–15].

Establish effective governance arrangements that clarify how much data can be shared

Balance needs to be struck between protection of private information and informing real-world research. Clearly, appropriate governance arrangements for RWE are crucial to facilitate evidence collection, and to make the most of healthcare information and the role it can play in improving patient care. The FDA's Sentinel database uses a distributed network model that facilitates access to large amounts of healthcare data while safeguarding patient privacy. The database has been used primarily to monitor for safety issues related to medical products but a key challenge is whether such a model can also be used to address postlaunch effectiveness issues.

Focus RWE efforts on the development of pragmatic clinical trials

PCTs can offer a bridge between RCTs and observational RWE by combining randomization with elements of the real-world setting. As such they can, in principle, provide reassurance to payers about quality, while also providing relevance to the healthcare settings in which the drugs will be used [26]. The challenges in undertaking PCTs, however, are often cost, time and analytical.

The NIH Health Care Systems Research Collaboratory is investing in the development of PCTs. Its online resource, Rethinking Clinical Trials: A Living Textbook of Pragmatic Clinical Trials (www.rethinkingclinicaltrials.org/), guides stakeholders though each phase of the PCT process, from the development of a clinical question to the dissemination of results.

Going forward, there is a strong case for greater use of PCTs and nested designs that seek to combine randomization with real-life settings. The ability to embed the data needed for PCTs in the routine data collection processes of healthcare systems is key to getting costs down. The official position of the FDA is unclear, but a paper by Sherman et al. [7] and the discussions at the Duke-Margolis RWE Regulatory Framework Public Workshop, suggests that PCTs are under consideration (https://healthpolicy.duke.edu/events/public-workshop-framework-regulatory-use-real-world-evidence).

Additional opportunities to enhance the generation & use of RWE

Real-time evidence-based medicine

The biggest potential benefit of ‘Big Data’ may be health systems’ ability to combine this data with analysis that is translated into protocols and guidelines for health professionals that enable them to actively manage patients when entering key patient characteristics. Handheld devices can have software that, for example, stratifies patients and identifies the relevant set of interventions. This is an extension of analysis, payers are already undertaking routinely to identify subsets of patients who are most at risk of deteriorating health, becoming intensive users of healthcare services and resources if they are not proactively managed.

Real-time monitoring of patients

Wearables and the Apple Research Kit (Apple, CA, USA), for example, will enable some data collection to become routine. Indeed, one survey found that 70% of American adults are tracking at least one indicator of their health [1]. These can be viewed as one or more of tools that will reduce the costs of collecting evidence – along with the use of handheld devices by health professionals to input patient data; mechanisms for expanding evidence as to how patients are responding to drugs; or types of remote monitoring providing further opportunities for services designed to enable patients to better manage their medication, and alert health professionals as to when intervention is required.

Accelerated access to innovative therapies (adaptive pathways & coverage with evidence development)

The launch of new products under accelerated access paths (such as Fast Track designation, Breakthrough Designation and Accelerated Approval) means payers have at the point of initial drug-listing decisions less evidence than for ‘traditional’ new therapies. The ability to collect postlaunch RWE is crucial for payers, both observational and through PCTs.

Considering accelerated pathways alongside improvements in study design, arguably provides a new paradigm for payers to get evidence on the comparative effectiveness and cost–effectiveness of the drugs within their health systems, in response to the greater use of accelerated access regulatory pathways.

Discussion at the ICER Policy Summit meeting

This review of key opportunities and challenges led to a series of key discussions at the ICER Policy Summit meeting. The current uses were acknowledged and supported, and all parties reported using RWE in some form or another to inform decisions.

However, stakeholders reported being reluctant to accept RWE developed by other stakeholder groups in some instances. Much of the debate, therefore, focused on how transparency, replicability and adherence to standards of ‘good practice’ can be encouraged, and how the challenges (outlined above) could be overcome. Favorable options for increasing transparency and replicability appeared to include mandatory registration of RWE protocols, and replication of analyses by applying different analysis techniques to the same dataset or applying the same analysis (via code sharing) to different databases. Peer-review or reviews from other third parties were also discussed as potential options for checking validity, but it was felt that this type of review may not be detailed or timely enough for this purpose.

There was also debate around if and how the standards of good practice (i.e., the evidence level and the corresponding methodological and process standards that should be followed) should differ depending on the context and the question that is being addressed. For example, if RWE is being used to demonstrate superiority of one drug over another, or to expand coverage, the standards may be higher than if RWE was being used to demonstrate equivalent outcomes for drugs with similar mechanisms of action. If standards are to differ, then a clear framework is required to allow different stakeholder groups opportunity to develop evidence to the appropriate standard that is deemed credible by all parties. This debate is developed further and a resulting framework presented in the accompanying paper [5].

The discussions at the meeting were grounded in the attendees’ direct experience of uses of RWE to date, and as such the tendency was to focus on how to improve quality and credibility within the current uses and the measures that can be taken today to enhance the generation and use of RWE. These issues are, however, also relevant to the additional opportunities for the future that we identified, such as real-time monitoring of patients. Indeed, the accompanying framework [5] is not limited to the current uses of RWE, and its application is likely to expand as our potential to generate and analyze new datasets develops further.

Conclusion

Clearly, our potential for rapid accumulation and analysis of data is increasing alongside technological advancements. These developments provide exciting opportunities for the use of RWE, yet important reservations remain. Overcoming challenges is likely to require dialogue and collaboration between multiple stakeholders, notably payers and manufacturers, particularly in relation to improving credibility and replicability of RWE. Pragmatic clinical trials, adaptive pathways and real-time monitoring of patients all offer the potential for more efficient or applicable evaluations of health technologies, and are all within reach if such effective partnerships can be nurtured.

Executive summary

Background

The capacity of the US healthcare system to generate and interpret large amounts of data is advancing exponentially.

Concurrently, traditional randomized controlled trial evidence may be declining as smaller patient populations (related to more personalized medicine) make it harder to design studies, and per-patient and setup costs of conducting explanatory randomized controlled trials increase.

It is crucial that manufacturers and the HTA/payer community understand the potential for real-world evidence (RWE) to inform coverage decisions and price negotiations. The opportunities must be balanced with the limitations.

This paper explores current uses of RWE in the USA, summarizes key concerns and sets out various opportunities that could be realized through good practice use of RWE for coverage decisions.

Materials & methods

Information was gathered via a literature review and interviews with nine experts in RWE from industry, payer organizations and academia.

Additional input was provided at the Institute for Clinical and Economic Review Policy Summit by manufacturers, payers and invited academic experts.

Results

Current uses: RWE is being used in the USA currently in six key areas: drug development decisions; regulatory approval decisions; postapproval monitoring of safety signals; HTA assessments and payer coverage decisions (initial decisions); HTA and payer coverage decisions (reassessments); and outcomes-based contracting.

Ongoing initiatives: Various efforts are underway to explore how the use of RWE in decision-making can be improved. A joint International Society for Pharmacoeconomics and Outcomes Research/International Society for Pharmacoepidemiology taskforce published recommendations regarding good procedural practices that would enhance decision makers’ confidence in evidence derived from real-world data studies. The Patient-Centered Outcomes Research Institute's Pragmatic Clinical Studies initiative has commissioned 28 projects, although none have yet reported results.

Challenges: The challenges around the use of RWE are well documented in the literature. Key concerns include: the potential for bias and confounding; incomplete data; data mining; limited access to data; a lack of universally accepted methodological standards; a lack of investigator expertise; and a reliance on traditional evidence hierarchies, which do not promote RWE.

Opportunities: Various opportunities are available to enhance the generation and use of RWE in its current six key uses, including the use of innovative study designs, such as pragmatic clinical trials, that combine the benefits of RWE and best practice methods (i.e., randomization).

Ways to increase the quality and credibility of RWE studies include establishing a mandatory national registry for observational studies, and seeking consensus on good practice guidelines. In addition, effective governance arrangements need to be put in place that allow an appropriate balance between protection of private information and use of data to improve patient care.

Key additional opportunities for the future include real-time monitoring of patients through wearables and the use of RWE to update treatment algorithms and generate real-time evidence-based medicine. RWE increases the feasibility of using adaptive pathways and coverage with evidence development to provide accelerated access to innovative therapies by allowing payers to collect evidence on the comparative effectiveness and cost–effectiveness of the drugs within their health systems and renegotiate price and/or access depending on the RWE evidence.

Conclusion

The increasing potential for rapid accumulation and analysis of data provides exciting opportunities for the use of RWE, yet important reservations remain.

Pragmatic clinical trials, adaptive pathways and real-time monitoring of patients all offer the potential for improved evaluations of health technologies, and are within reach if effective partnerships between payers and manufacturers can be nurtured.

A separate paper is available that summarizes proposed ways forward based on the discussions that were had at the 2017 Institute for Clinical and Economic Review Policy Summit meeting.

Acknowledgments

The authors thank the meeting attendees for their input to the meeting and comments on a previous version of this paper.

Author contributions

G Hampson and A Towse conducted the literature review and interviews. All the authors contributed to the drafting of the manuscript.

Financial & competing interests disclosure

The Office of Health Economics received funding from the Institute for Clinical and Economic Review (ICER) for providing scientific content for the meeting. C Henshall received funding for his work as Chair. ICER is a non-profit organization; funding for the Summit was predominantly obtained from life science companies, health plans and pharmacy benefit management companies. G Hampson and A Towse are employees of the Office of Health Economics, a registered charity, which receives funding from a variety of sources, including the Association of the British Pharmaceutical Industry. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Open access

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/

References

Papers of special note have been highlighted as: • of interest; •• of considerable interest

Marchibroda J, Swope T, Watters S et al. (2016). Bipartisan policy center. Using real-world-evidence to accelerate safe and effective cures: advancing medical innovation for a healthier America. https://bipartisanpolicy.org/wp-content/uploads/2016/06/BPC-Health-Innovation-Safe-Effective-Cures.pdf.