The joint ISPOR–ISPE Special Task Force on real-world evidence in health care decision making: an interview with Marc Berger

Over the last few years, I had been attending a series of meetings in Washington, DC on enhancing the use of real-world evidence. At one of these meetings, I proposed that the trustworthiness of real-world evidence could be improved if decision makers expected observational studies to follow the same procedures as required for randomized controlled trials – that is, to be registered on a public website prior to their execution and have an explicit protocol and data analysis plan. In addition, it should be expected that the results of all studies should be made public. Last fall, I proposed to the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) that we create a special task force on good procedural practices for real-world data studies. This initiative was endorsed by the ISPOR leadership and I co-chaired the task force with Daniel Mullins. During the same period, we learned that the International Society for Pharmacoepidemiology (ISPE) had created a taskforce on the reproducibility of real-world data studies. ISPOR and ISPE decided to join forces [1], since the goals of these initiatives were complementary, and I participated in the ISPE task force.

Following the HITECH Act (2009) [2] in the USA, the use of electronic medical records became widespread. This increased interest in the emergence of a learning health care system that could utilize the wealth of digitized clinical data to inform health care policy decisions by payers, providers, HTA organizations and others. However, uptake has been variable due to concerns regarding the methodologic limitations in addressing bias and confounding in observational studies as well as the concern that published positive results might not represent the experience of therapeutics in the real world. The potential for real-world evidence to also support regulatory decision making was also recognized by the 21st Century Cures Act (2016) [3]. Given the burgeoning growth of real-world data, it appeared timely to provide guidance that would increase the credibility and trustworthiness of published real-world evidence.

This report focused on observational studies that were designed to evaluate the presence or absence of a pre-specified treatment effect and/or its magnitude [4]. It recommended pre-registration of study protocols and analysis plans on a public registration site, being transparent in publication if the protocol or analysis plan changed, providing opportunities for study replication, and including key stakeholders in the design, conduct and dissemination of the research. If the source of the hypothesis was an analysis performed on a particular data set, it also recommended that the hypotheses evaluation be performed on a different data set (unless not feasible).

This report focuses [5] on enhancing existing reporting guidelines (including RECORD [6], STROBE [7] and EnCePP [8]) by identifying a minimum set of items necessary to report in detail in order to achieve fully reproducible evidence from large health care database cohort studies. It encourages data and code sharing when data use agreements and IP permit; however, clear, natural language description of key operational and design details should be provided in publications so that the scientific thought processes of investigators can be fully understood.

The adoption and implementation of these recommendations will require efforts on the part of key decision-maker stakeholders including providers, payers, HTA authorities, regulatory authorities, and journal editors. Even if these recommendations are implemented, they will not guarantee that real-world evidence will be credible and trustworthy, although they are an excellent first step. The hope is that this will foster a culture of transparency and collaboration among the community of producers and users of real-world evidence. Only with successful experience in leveraging real-world evidence to inform health policy decisions will confidence in it grow [9].

We have already begun to present these reports at public meetings and we are holding a meeting with stakeholders to begin a conversation about how to foster their adoption, the ISPOR/ISPE “Summit on Real-World Evidence in Health Care Decision Making”. This is taking place on October 20, and more information can be found on the event website [10].

I have been working with real-world data for the last 25 years. The data is getting richer and better in quality. We must find a way to fully leverage this wealth of information to support continuous quality improvement of health care systems, to achieve the promise of personalized medicine, and to make the development of new therapeutics more efficient.

The opinions expressed in this interview are those of the interviewee and do not necessarily reflect the views of Future Medicine Ltd.

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.