Publicly funded practice-oriented clinical trials: of importance for healthcare payers

For society it is important to know what the real clinical therapeutic benefit is in comparison with best available alternatives. However, regulatory authorities require the demonstration of safety and efficacy, but not therapeutic benefit, before a medicinal product is granted marketing authorization. Trials are often performed in a highly selected population and placebo is often used as a comparator, although designs including an active comparator are encouraged. Ideally, the new intervention should be compared with the best available active comparator, which could be a non-drug intervention. Important statements in this regard are provided in a draft reflection paper of the European Medicines Agency (EMA): [12] “Where feasible, three-arm trials including experimental medicine, placebo and active control represent a scientific gold standard and there are multiple reasons to support their use in drug development … there are few circumstances where an indirect comparison might be considered sufficiently reliable”. Once marketing authorization and reimbursement are obtained, the company may not be willing to generate data that might hamper the marketing of their product. For example, there may be a commercial risk (promotion, price discussion) associated with the conduct of a direct head-to-head trial in case the company's product proves to be inferior (or not superior) to the existing alternative treatment (which may be less expensive). Industry might also be less interested in research to improve existing procedures and prescriptions, for example, finding the optimal drug combination or duration of treatment [13]. Furthermore, in addition to the selected patient population in Phase IIb/III clinical trials, it might be important to have a more pragmatic approach and to include a broad real-life population in comparative effectiveness trials.

Trials in children, women, older people or testing treatments for rare diseases have long been neglected. Despite incentives created to stimulate the development of medicinal products for children and in rare disorders, there remains a need for more clinical research in these areas. “The market-driven pharmaceutical industry does not pursue research and development for a number of diseases because of the small number of patients involved … and the insufficient profitability of the treatments (e.g., pediatric therapies…)” [13].

Non-commercial clinical trials may be necessary for medical devices since only safety and performance, and not efficacy, are assessed in the premarket phase in Europe [14,15]. In comparison with drug interventions, evidence generation is also less developed for diagnostics, which are very relevant as companion diagnostic for targeted therapy or as screening tools at the population level. In the field of companion diagnostics, there is a need for robust data on the test accuracy in the real life setting, as compared with centralized testing often used in confirmatory randomized controlled trials (RCTs) and used for the evaluation of the (cost–)effectiveness of the test–drug combination during the reimbursement procedure. Maintaining a high test specificity in routine care is crucial for the cost–effectiveness of the targeted treatment, much more than the cost of the test or even the cost of the drug [16].

In screening, the funding of a large-scale clinical trial may be the best strategy both from a healthcare perspective (the fastest route to obtain hard evidence) and from an economic perspective. For example, thanks to the existence of a large-scale clinical trial on prostate cancer screening (PROTECT), [17,18] policy makers were able to avoid the implementation of a non-evidence-based prostate cancer screening program in the UK for the past 20 years.

Finally, other fields have less or no medical industry support such as surgical techniques, diagnostic/imaging techniques, diagnostic strategies, radiation therapy, psychotherapy, physical therapy, lifestyle interventions or prevention, traditionally areas not ‘owned’ by private companies. Here too, the decision makers need high-quality data from comparative effectiveness trials to decide on the approval and financing of specific techniques and interventions. For example, as illustrated in a network of available comparisons, there is a need for more direct comparisons between exercise versus drug interventions in coronary heart disease, stroke, heart failure and prediabetes [19].

There are several studies examining the costs of a single RCT and the consequences for society. One of the most spectacular examples is the study analyzing the economic return from the Women's Health Initiative estrogen plus progestin clinical trial [21]. At a cost of approximately US$260 million (in 2012 US$, US$1 = €0.900 = GB£0.652; 23 September 2015), this Women's Health Initiative estrogen plus progestin trial was one of the most expensive studies ever funded by the US’ NIH [22]. After publication of RCT evidence of increased cardiovascular disease, venous thromboembolism and breast cancer risk among postmenopausal combined hormone therapy users, combined hormone therapy use decreased in the USA by approximately 50% and continued to decline at 5–10% annually as the US FDA and other groups endorsed the study conclusions [23–29]. According to the authors, the corresponding net economic return of the trial was US$37.1 billion (US$140 per dollar invested in the trial) at a willingness-to-pay level of US$100,000 per quality-adjusted life-year (QALY) [21].

Of course, a research program does not only include success stories. Costs of non-completed trials for which the analyses could not be performed as planned should also be taken into account and lessons should be learned from such failures. Seven studies were found, trying to calculate the return on investment of research programs. Evaluations of publicly funded clinical research and clinical trial programs were identified for selected NIH trial programs in the USA and clinical research programs funded by the National Health Service in the UK. Health economic evaluation reports of clinical trial programs in Australia, Sweden and The Netherlands were also identified.

In the UK ‘Medical Research: What's it worth?’ study, [30] the research programs of the 11 principal funders in cancer research, accounting for over 95% of research spending, were analyzed. Between 1970 and 2009, total expenditure on cancer-related research was GB£15 billion (2011/12 prices). Over the period 1991–2010, the interventions included in the study produced 5.9 million QALYs and health benefits equivalent to GB£124 billion (at a GB£25,000 per QALY value and including the costs of delivery). Smoking reduction accounted for around 65% of the net monetary benefit. The annual rate of return was estimated to be 10%. Spillover effects were described as the indirect impact of public and charitable research on the wider economy, such as leveraging private sector R&D activity. When these effects were included, the rate of return even increased to 40% [30].

A study on the effect of a US NIH program of clinical trials on public health and costs [31] looked at all Phase III randomized trials funded by the US National Institute of Neurological Disorders and Stroke between 1977 and 2000, including 28 trials with a total cost of US$335 million. The effects of a trial on total costs and savings or quality of life could only be assessed if such information was available. This was the case for eight trials, resulting in an estimated additional 470,000 QALYs at a total cost of US$3.6 billion (including costs of all trials and additional healthcare and other expenditures). At a per-head gross domestic product value of US$40,310 per QALY, this resulted in a net benefit to society by 10-years of US$15.2 billion.

In The Netherlands the ‘efficiency research’ program set up by the Dutch organization for health research and development ZonMw, was evaluated [32]. The return on investment of this program was calculated by comparing the costs of this program (2001–2015) with the expected cost savings in healthcare arising from the subsidized studies within the program. In a conservative scenario, assuming that 40% of the potential cost savings are actually achieved, the program shows a very high return of 327%. The authors conclude that this program pays for itself more than three times [32].

Several Australian studies [33–35] also report positive results, like a benefit/cost ratio of 2.17, [34] which means that a dollar invested in Australian health R&D returns $2.17 in health benefits on average. We do not discuss these studies further since the results hinge on some major assumptions (e.g., a time lag between the mid-point of the R&D expenditure and the mid-point of the wellbeing gains of on average 40 years, [35] or attributing half the historical gains in health span to global health R&D and allocating 2.5% to Australian R&D since this is Australia's share of global R&D activity [33]).

Finally, a Swedish study concludes that the positive effect of clinical research benefits exceeds costs, but acknowledges that an “accurate determination of the economic value of research would require significantly better basic data and better knowledge of relationships between research, implementation of new knowledge and health effects” [20].

While all identified studies are positive about public funding of trials, the results should be interpreted with caution. It is possible that there is a pro-innovation bias [20] in which authors have accepted in advance that research and innovation are profitable and have selected examples or made assumptions to support their view. Furthermore, all of the above studies have several important weaknesses. Both identification and valuation of benefits is very difficult and associated with very large uncertainty.

Even if the benefits can be identified, quantified and valued, it is difficult to know how much should be attributed to R&D efforts from the research program. For example, the US study [31] only included costs of Phase III clinical trials. Costs for basic research were not included. Nevertheless, as mentioned by the authors, “all the interventions from these clinical trials required understanding brought about through basic science research. Thus, the overall investment in basic and clinical research was important to achieving these health gains” [31]. On the other hand, they calculated that the benefits from the clinical trials alone (US$50 billion) were large enough to cover all the expenses of both basic and clinical research in the research program (US$29.5 billion) [31]. It is also not clear what the influence is of other elements, like an improved prosperity, lifestyle and diet, on delaying the onset of disease [36].

Another difficult to quantify important variable is the spillover effect. This externality may take different forms. For example, publicly funded research may improve the infrastructure and knowledge to perform trials and might have a positive influence on, for example, involving more physicians in clinical research and making them familiar with the foundations of evidence-based medicine. In addition this expertise might also attract industry-sponsored trials. In the UK study, [30] the largest part of the benefits comes from the spillover effect. While this might be an overestimation, other studies do not take this effect into account, [31,32] which results in a conservative estimation of the benefits of publicly funded research.

It may also be difficult to know whether the study conclusions were implemented in routine care and whether the results achieved are similar to those identified in the trial. The Dutch study [32] calculated the ‘potential’ cost savings, applying two scenarios in which 40 or 80% of the potential savings are achieved. The US study [31] also based its calculations on published economic evaluations, which were only available for part of the interventions in the research program. These projections from potential cost savings or published economic evaluations may be very different from the real-world impact and may over- or under-estimate the impact of publicly funded research. It would be desirable to check if the outcomes of this research had an impact on real-world practice, for example, by looking at practice guidelines, change in behavior, reimbursement decisions among others.

Finally, to avoid any misunderstanding, studies mentioning R&D investments result in cost savings do not always mean that, in the end, less money will be spent. Several studies provide a monetary value to the QALYs gained [21,30–31]. In the US study, [31] applying a US$40,310 value per QALY, the health benefits have a much greater value than the increased expenditures on health. For this aspect, the Dutch study [32] is more conservative by not including a valuation for the health benefits.