Skip to main content
Free access
2 October 2014

Incremental cost per quality-adjusted life year gained? The need for alternative methods to evaluate medical interventions for ultra-rare disorders


Drugs for ultra-rare disorders (URDs) rank prominently among the most expensive medicines on a cost-per-patient basis. Many of them do not meet conventional standards for cost–effectiveness. In light of the high fixed cost of R&D, this challenge is inversely related to the prevalence of URDs. The present paper sets out to explain the rationale underlying a recent expert consensus on these issues, recommending a more rigorous assessment of the clinical effectiveness of URDs, applying established standards of evidence-based medicine. This may include conditional approval and reimbursement policies, which should be combined with a firm expectation of proof of a minimum significant clinical benefit within a reasonable time. In contrast, current health economic evaluation paradigms fail to adequately reflect normative and empirical concerns (i.e., morally defensible 'social preferences') regarding healthcare resource allocation. Hence there is a strong need for alternative economic evaluation models for URDs.
Figure 1. Increasing acquisition cost per patient with decreasing prevalence as a result of fixed (i.e., largely volume-independent) research and development (R&D) expenditures.
Data taken from [26].
In the USA, the EU as well as in Japan, Australia and some other jurisdictions, legislation has been adopted to encourage the development of treatments for rare or 'orphan' diseases [1]. Under this legislation, developers and manufacturers of so-called orphan drugs to treat rare diseases benefit from a range of incentives, including reduced or waived licensing fees, extended market exclusivity periods and in the USA and Japan, tax relief on development costs.
In theory, there are no distinct (sub-)categories of rare and ultra-rare disorders (URDs) and treatments. Increasing rarity of a condition merely represents the end of a continuum, just like increasing severity and (in part) increasing comorbidities are continuous, not discrete phenomena. For policy-makers, it may nevertheless be pragmatic to define different categories of disorders and interventions, irrespective of the (absence of) theoretical merits of such an approach.
Definitions for 'orphan disorders' typically include a criterion of prevalence or incidence and differ somewhat between jurisdictions. In the USA, these are disorders with a prevalence of less than 200,000 affected persons (according to the Orphan Drug Act of 1983, and Orphan Drug Regulation of 1993) [1,2], in the EU, prevalence must be less than 1 per 2000 (or less than 0.05%) of the population (according to EU Regulation CE No. 141/2000 of 2000) [1,3]. Strict criteria have also been set in Japan (fewer than 4 per 10,000, according to Orphan Drug Regulation of 1993) and Australia (less than 1.1 per 10,000, according to Orphan Drug Policy of 1997) [1,4,5]. In Taiwan and South Korea, prevalence thresholds have been set at less than 1 per 10,000 and 1 per 20,000, respectively [6].
No official definition of 'ultra-orphan disorders' has yet been adopted globally. Rather, this informal subcategory was introduced by the National Institute for Health and Care Excellence (formerly, the Institute for Health and Clinical Excellence, and the Institute for Clinical Excellence; NICE), who applied it to drugs with indications for conditions with a prevalence of less than 1 per 50,000 persons [7,8]. The definition, albeit no less arbitrary than the definitions used for 'orphan disorders', corresponds to the even more restricted prevalence criteria adopted by England’s Advisory Group for National Specialist Services (AGNSS), that had been assigned the task of reviewing technologies for URDs until April 2013. The qualifier required by AGNSS was less than 500 persons affected in England (i.e., ˜1 in 100,000 of the English population) [9]. Publicly funded drug plans in the Canadian provinces of Alberta and Ontario also use very restrictive criteria, in other words, a prevalence of less than 1 per 50,000 in Alberta [10], and an incidence rate of fewer than 1 in 150,000 live births or new diagnoses per year in Ontario [11].
As judged by the number of granted orphan drug designations, regulations designed to spur the development of treatments for rare disorders by providing economic incentives may be considered a substantial success [12–14]. Notably, the number of orphan medicines approved by the European Medicines Agency (EMA) and by the US FDA has increased steadily since the enactment of legislation designed to mitigate the biopharmaceutical industry’s high risks and uncertain rewards when investing in the development of treatments for rare disorders [15–17]. This success notwithstanding, there remains an ongoing need for new orphan medicines, given that the vast majority of rare and URDs – with their total number estimated at approximately 7000 – still await the development of effective treatment [17]. However, these policies will remain of limited relevance if subsequent reimbursement of treatments for rare and URDs is denied on grounds of their high incremental cost–effectiveness ratios. Apparently, there is a lack of alignment between research funding policies and some reimbursement policies designed to address the need to set limits fairly in the light of resource scarcity.
A number of agencies in charge of health technology assessments (HTAs) have adopted cost-utility analysis as a method of choice to determine the 'value for money' offered by medical interventions [18]. For example, NICE expects that the use of drugs, to be recommended for reimbursement by the National Health Service of England and Wales, should not be associated with an incremental cost exceeding GBP£20,000 to GBP£30,000 per quality-adjusted life year (QALY) gained [19–21]. By adopting this approach, NICE followed the earlier models of Australia (with a benchmark in the range of AUS$42,000 to AUS$76,000 per life year gained) [22], New Zealand (with a reported cost–effectiveness benchmark of NZ$20,000 per QALY gained) [23] and Canada (with a suggested variable benchmark between CAN$ 20,000 and CAN$ 100,000 per QALY gained) [24].
Many drugs developed to treat URDs are unable to meet the cost–effectiveness thresholds stipulated by these and some other official regulatory bodies (Table 1) [8].
A key underlying reason for the failure of many orphan drugs to meet proposed – and, in some jurisdictions, actually applied – standards for cost–effectiveness is that manufacturers need to generate revenues that allow them to recoup the cost of R&D, which also include the costs of gaining regulatory approval, from a small number of patients during limited periods of market exclusivity. This challenge inevitably results in high acquisition costs for these products on a per patient basis [25,26]. Key drivers of R&D costs are well understood and include, beyond out-of-pocket expenditures, the cost of failures (or 'attrition rate', i.e., the proportion of development projects not leading to a marketable new product), regulatory obligations for Phase I to III trials, development times (averaging 12–13 years from discovery to marketing authorization) and the associated cost of capital [27–31]. Recent estimates accounting for these factors have indicated that the overall present value of R&D spending for a new molecular entity now might well exceed US$1 billion on average [32], although some observers have taken different views and raised objections against the interpretation of these numbers [33,34]. From a commercial perspective, however, this staggering figure does not even reflect the additional market risks, which are illustrated by the highly skewed distribution of returns (i.e., the fact that a large number of newly developed products do not generate a positive return on investment) [30,32].
It is, of course, legitimate to ask whether, or to what extent, the development of medical interventions for rare and URDs benefits from special conditions. For example, early consultation with regulatory authorities and protocol assistance (as provided under EU regulation 141/2000 [1,3] and the US Orphan Drug Act [1,2]) has been shown to improve the success rates of clinical development programs [35], and the number of patients included in clinical trials is typically smaller compared with more common diseases [12]. On the other hand, study logistics and patient recruitment may be more complex for rare and URDs, and mean development times appear to be similar for orphan and non-orphan development programs [36,37].
Nevertheless, given the largely fixed (i.e., independent from sales volume) nature of R&D costs, it seems plausible that the issue of not meeting conventional benchmarks for cost–effectiveness will only increase in relevance with decreasing prevalence rates, especially with drugs developed to treat very small patient populations [26,38–40]. Consistent with this, drug acquisition costs are inversely correlated with prevalence (Figure 1).
Obviously there is a serious mismatch between reimbursement policies based on the logic of cost–effectiveness, with cost-per-QALY benchmarks, on the one hand, and international policies designed to encourage R&D into rare and URDs and their effective treatment, on the other hand. There thus appears to be an unmet need for a coherent value framework reflecting all attributes of health technologies considered important by the public ('social preferences'), while at the same time remaining consistent with prior normative commitments as entailed by institutional and legal traditions. Such a framework should also enable to effectively address the specific challenges that are posed by HTAs of interventions for the diagnosis and treatment of rare and URDs, combining fair access to effective interventions (for patients) with incentives for research, development and 'innovation' (for manufacturers). At the same time, given concerns about high drug acquisition costs per patient and potential impact in healthcare budgets [40,41], the framework should specify clear principles for setting limits (for policymakers and payers).
Most prominently in England, there have been attempts to protect the logic of cost–effectiveness, built around cost-per-QALY benchmarks, against criticism by creating exemptions, thus isolating it from some of its most irritating implications. The introduction of an 'ultra-orphan' category by NICE, as well as the introduction of a second special category, so called 'end-of-life' treatments, can be interpreted as such a defensive move, responding to political and public pressures on NICE as a reaction to negative appraisals [7,8,42]. The need to create exceptions does, however, point to deeper issues regarding the generalizability of the 'logic of cost–effectiveness' as adopted by NICE. These may explain why estimating incremental cost per QALY ratios are not up to the task of evaluating drugs for rare and, especially, URDs, and may lead to recommendations that violate widely held fairness-related beliefs [43,44].

Objectives & methods

To address this mismatch between regulatory incentives and reimbursement hurdles for URD treatments, the not-for-profit Institute for Innovation & Valuation in Health Care (InnoValHC, Wiesbaden, Germany) convened an international expert workshop in Berlin, Germany, on 8 November 2012.
Objectives of the workshop were:
• To review the challenges that arise when applying conventional HTA methodologies to medical technologies for ultra-rare diseases (URDs);
• Given these challenges, to seek expert agreement on the need for (improved or) alternative evaluation methods, ideally in the form of a consensus statement;
• In light of this analysis, to initiate discussion of improved or alternative evaluation methods, including the advantages and disadvantages of different options and possible ways forward.
After the workshop, two documents summarizing the discussion were drafted and circulated among the participating experts, whose comments were integrated in an iterative process, which led to the final consensus document [45]. In the present paper we, the group of scientists with relevant backgrounds in health economics, HTA, clinical pharmacology and innovation management, set out to describe and explain the consensus achieved, and to outline potential ways forward that we agreed upon. In line with the timing of the workshop, an attempt was made to identify and discuss the relevant scientific literature available by November 2012 (effective cut-off date for inclusion).
Different potential levels of analysis were distinguished by the expert group, namely, a focus on:
• The principles underlying the current evaluation framework;
• The actual evaluation policies implemented by HTA agencies and regulatory bodies (primarily those concerned with pricing and reimbursement decisions);
• Evaluation practice when principles and policies are applied to real-world problems. The third level would have to include case studies, including cases where existing regulation has been potentially misused (see section below titled 'Definitions').
The group agreed that discussion should initially focus on fundamental principles. This was believed to be a reasonable approach, since policy implementation as well as evaluation practice (although clearly relevant dimensions) are hierarchically lower levels of analysis. The latter will have to be reviewed subsequently with reference to a set of prior higher level guiding principles. Thus, as a first step, a situation analysis was undertaken in order to establish common ground for future deliberation.


While recognizing the somewhat arbitrary nature of this cut-off criterion, the expert group agreed to focus on medical technologies targeting URDs (with a prevalence of less than 1 per 50,000), in other words, to exclude from further analysis the following related but different subject areas:
• Orphan disorders with a prevalence of less than 5 per 10,000 (or less than 1 per 2000), but higher than 1 per 50,000;
• Cancer medicines (given their distinct characteristics, including the frequently observed gradual expansion of indications, for example, by moving treatments from third or fourth line to second line, combined or adjuvant use in early-stage disease);
• The specific challenges posed by emerging concepts of 'personalized medicine';
• Also, for the time being, given the intended level of analysis (cf. above), potentially abusive commercial ploys such as 'indication slicing' (of a biologically much broader and more prevalent condition, in order to obtain orphan designation) and other potential strategic instrumentarium of some manufacturers [17,46].
Further characteristics of URDs under consideration should include that the conditions are severe, are chronic, represent clearly defined biological entities, and, hence, are associated with a broadly accepted high unmet medical need – whereas the absence of alternative treatment options was not considered a necessary defining condition of an URD (as the broader criterion of 'high unmet medical need' was believed to better capture the underlying rationale).
The subject of the analysis was specific (unique) condition/treatment pairs fulfilling the criteria listed above. The typical case the workshop participants had in mind were treatments that are effective for one URD only (such as enzyme replacement therapies for hereditary lysosomal storage disorders). The panel shared the view that certain adjustments would probably be necessary when one drug works in more than one URD indication, but these adjustments were considered likely to be of a rather technical nature and, hence, were not explored in detail at the workshop as its primary focus was on the underlying fundamental evaluation principles.


Specific challenges

The expert group recognized that 'ultra-rarity' merely represents the end of a continuous spectrum of decreasing prevalence. Thus any prevalence threshold was seen as arbitrary in nature and justifiable for pragmatic reasons only. As a matter of principle, any considerations regarding the evaluation of URDs should be applicable to less prevalent conditions, too. The group agreed, however, that with increasing 'rarity' a number of typical challenges must be expected when dealing with interventions for URDs. The most serious ones fall into one of two categories, the need to establish evidence of clinical effectiveness and/or the need to demonstrate 'value for money'.

Establishing evidence of clinical effectiveness

For several reasons, developing treatments for URDs is a more challenging, complex and sometimes more risky endeavor than developing treatments for more common diseases, as:
• Less clinical/medical research is often available for URDs, resulting in a limited clinical understanding;
• There is usually a very small number only of physicians with specialized expertise, who are based in few specialized centers;
• There exist unusual difficulties to produce robust clinical evidence, for example, because of limited understanding of the natural history of URDs and because of the often limited availability of validated instruments to measure disease severity/progression;
• This, combined with difficulties to generate a large volume of evidence for URDs based on randomized clinical trials may lead to higher levels of uncertainty surrounding effect size estimators;
• Significant hurdles exist when trying to identify and accurately diagnose patients with URDs;
• Because the small number of patients are often geographically dispersed, multiple clinical trial sites must be established for only a few patients [36–38,47–50].
Further to this, ongoing postmarketing requirements must be met, including the creation of registries and risk management plans, which must be maintained globally for only a small number of patients.
As a consequence of the above, in a significant number of cases, the safety and efficacy profiles of orphan drugs have been incomplete, and often marketing authorizations were based on small-scale studies addressing surrogate end points only [17,48,51,52].

Establishing 'value for money' (efficiency)

Further challenges are related to and extend beyond the sphere of generating evidence of clinical effectiveness. They are economic in nature and concern the efficiency or 'value for money' offered by URD treatments:
• Across healthcare systems, there is a marked heterogeneity regarding institutional arrangements. This is mirrored by the situation that currently established methodologies to determine 'value for money' vary internationally. A stronger utilitarian tradition (as for example in England) has generally led to a higher acceptance of 'efficiency first' evaluation principles, whereas stronger emphasis on a rights-based approach (and a corresponding legal tradition, as for example, in some continental European countries such as France and Germany) has made for a stronger reliance on approaches based on unmet medical need and on evidence of comparative clinical effectiveness for the allocation of healthcare resources [18,43,44,53,54];
• Neoclassical welfare theory built on the Pareto criterion for allocative efficiency is believed by many economists to represent “the theoretical high ground, […] although even the most committed Paretians acknowledge that distributional issues as well as efficiency issues need to be dealt with” [55]. Thus traditional economic theory, which uses maximum individual willingness-to-pay as universal measure of utility, has never gained real acceptance in health care, and has even been notably unpopular, among providers and policy makers because of the wide-spread belief that basic necessities “such as life, health and citizenship […] should be distributed less unequally than the ability to pay for them” – a view that has been referred to as 'specific egalitarianism' [56];
• Accordingly, the currently prevailing approach to the economic evaluation of healthcare programs does not consider utility but health gains as the central outcome used to assess the appropriateness (efficiency) of healthcare expenditures. Usually this paradigm is directly linked to the assumption that the objective of collectively financed health schemes ought simply to be maximization of the aggregate health gain for the population covered by the scheme [57–59]. If and when health gains are 'valued' (measured) in terms of quality-adjusted life years (QALYs), extrawelfarism then translates into QALY maximization, a normative hypothesis that has been endorsed by extrawelfarists on grounds of an alleged 'consensus in the literature' [60];
• From there it is straightforward to establish a ranking of medical interventions based on their efficiency as defined by their incremental cost per QALY gained (sometimes called QALY league tables, based on incremental cost–effectiveness ratios, ICERs), implying a presumably increasing social desirability of services associated with decreasing ICERs. In practice, this approach translates into the adoption of some sort of a benchmark for the maximum allowable cost per QALY, which may be interpreted as the social willingness to pay for, or the shadow price of, a QALY. Interventions meeting this benchmark criterion will then be deemed 'efficient' given a resource constraint [55,57,61];
• Claims of distributive neutrality notwithstanding ('a QALY is a QALY is a QALY, regardless of who gains or loses it' [62,63], a position termed by some economists as 'QALY egalitarianism' [64]), this approach implies considerable constraints on the preferences to be taken into account [44,65,66]. Any contextual variable(s) – apart from individual health gain – potentially influencing the social desirability of (and hence the social willingness to pay for) health services would necessarily violate the basic assumption that all QALYs are created equal [67] – or simply need to be assumed away [68,69]. The focus on individual QALY gains in applied health economics also ignores health effects occurring in other people than the index patient, for instance in carers [55];
• If there were other objectives beyond the maximization of population health (which represents the restricted extrawelfarist version of the goal of allocative efficiency), such as the wish to be treated with dignity and respect, or concerns about equity and fairness (e.g., with regard to equality of access to care, or equal access for equal need or more generally the recognition of differing moral claims of individuals, and so on), these quite obviously would either result in differential cost per QALY benchmarks as a function of these concerns, or might even require an entirely new evaluation paradigm. This issue has also been referred to using the notion of horizontal equity (i.e., the equal treatment of equals) versus vertical equity (i.e., the unequal but equitable treatment of unequals) [70]. As recognized early by philosopher economists campaigning to propagate the contribution of economics to applied ethics, “the principle of ‘horizontal equity’ … is … inconsistent with teleological maximizing.” [71] Teleological maximizing here refers to the overriding consequentialist principle to strive for 'the greatest good for the greatest number', regardless of distribution.
As noted earlier in the introductory section, many interventions for rare and URDs are unlikely (or altogether unable) to meet standard cost per QALY benchmarks. Hence, there is a need to examine the range of normative and empirical issues surrounding the application of the extrawelfarist logic of cost–effectiveness (intended to serve as a criterion for allocative efficiency) for the prioritization of healthcare programs. In an attempt to escape from contentious interpersonal comparisons, politicians and healthcare policy makers in some jurisdictions, such as the USA and Germany, have deliberately decided to refrain from the computation of cost per QALY gained. In effect, priority setting is based on an assessment of comparative effectiveness by the Patient-Centered Outcomes Research Institute (PCORI) in the US [72] and by the Joint Federal Committee (Gemeinsamer Bundesausschuss, GBA) in Germany [73], respectively. At best, technical efficiency will be taken into account as a secondary criterion (e.g., methods guidance by the Institute for Quality and Efficiency in Health Care, IQWiG in Germany, developed to avoid comparing interventions across different disorders, and thus by design falling short of an economic analysis of allocative efficiency) [74].
With either approach, there remains the need to establish fair boundaries with regard to coverage (reimbursement) and pricing, and, by implication, with regard to access to medical technologies. This challenge is an inevitable consequence of the scarcity of resources available for healthcare, given the limited willingness of the public to be taxed (or the limited social willingness to pay for health insurance).

Valuation: social norms & preferences

A number of specific normative as well as empirical (positive) and technical problems arise when traditional HTAs include cost–utility analyses, with QALYs as a measure of health-related outcomes (and their individual valuation) for URDs. In many ways, this can be interpreted as a reductionist simplification to react to the complexities of healthcare priority setting by attempting to reconstruct 'social value' as an aggregate of individual utility only [75,76]. In applied health economic analysis, in turn, analysts take individual preferences almost always as a proxy for individual utility [77,78]. Individual preferences for health states, then, can be elicited either from patients (if measured ex post, 'experienced utility') or, in practice more often after the consensus recommendations issued by the Washington Panel [79], from a representative sample of the general population (if ex ante, 'decision utility'), hereby employing a choice-based method such as the time trade-off or standard gamble [55,67,77–79]. This can be done directly, using health state vignettes, or indirectly, through the use of a validated health-related quality of life instrument [67]. All of these methodological choices, as well as a number of further 'technical' conventions broadly accepted and codified in health economic evaluation guidelines [44,45,55,67,79,80], are linked to normative implications. (Only some of the most salient ones will be briefly mentioned below, as this aspect – although clearly relevant – goes beyond the scope of the consensus to be explained here).

Normative ethics

The additive aggregation rule underlying the QALY maximization hypothesis corresponds to an (act) utilitarian calculus (sometimes referred to as 'medical utilitarianism' [68], because it values healthcare services only if and when they improve people’s health, and these health gains are considered as the one and only appropriate maximand). In effect, what matters under this rule is the distribution-independent sum total of population health only. The most serious normative issues arise, accordingly, when people are not only unwilling to subscribe to a pure utilitarian approach (in the case of traditional cost–benefit analysis, firmly grounded in welfare economics), but likewise are also reluctant to adopt a quasi-utilitarian (in the case of 'extrawelfarist' cost–utility analysis) approach. The massive and continuously growing evidence that this may indeed be the case will be discussed in the following section (see section titled 'Empirical ethics').
Amartya Sen’s objection against utilitarian welfare economics applies with equal force to extrawelfarism, too, in other words, that “each person deserves consideration as a person, and this militates against a distribution-independent view.” [81] His critique resembles closely John Rawls’ argument that distribution indifference does not take the distinction between persons adequately seriously [82]. A person group’s capacity to produce QALYs (i.e., individual health gain, valued or 'weighed' by the strength of individual preference, multiplied by the number of persons benefiting from a healthcare program) counts as the only input, but the fate (at an extreme, even the death) of any one individual person (as a consequence of the resulting distribution of health benefits among persons and person groups) remains totally irrelevant – in other words, as Sen put it, “if a person remains miserable or painfully ill, her deprivation is not obliterated or remedied or overpowered simply by making someone else happier or healthier.” [81]
In contrast to utilitarianism, nonconsequentialist or deontological rights or 'claims' based reasoning has a particularly strong tradition in medicine and health care [83]. Norman Daniels and others have extensively built and defended the argument that fairness requires to recognize a basic right of individuals to a 'normal range of opportunities' to realize their plans in life. Disease and disability may have a negative impact on this range of opportunities through impairment of normal functioning. Then this impairment should, according to this line of thought, serve as a “measure of the relative importance of health-care needs at the macro level” (however crude, as admitted by Daniels) [84].
Also the various legal systems, at both the national and at the supranational level, reflect deeply ingrained nonutilitarian value judgments, which cannot simply be sidelined in the present context. With particular reference to rare disorders, the following have all been cited as relevant to decisions determining levels of funding and commissioning [85]:
• Human rights legislation, particularly Article 2 of the European Convention on Human Rights (ECHR), stipulating that 'everyone’s right to life shall be protected by law';
• As well as the United Nations Convention on the Rights of Persons with Disabilities;
• EU disability legislation; and
• In the UK, the Equality Act of 2010 and UK 'tort law'.
Likewise, a recent Swiss consensus of stakeholders including physicians, payers (health insurance) and the research-based pharmaceutical industry, on the implementation of HTA in Switzerland [86] concluded that empirical preferences (neither individual nor social ones, see below) alone do not form a sufficient basis for decision-making on healthcare priorities; they rather need to be embedded in the context of a prior normative commitment. The consensus identified, in light of the national legal tradition, a number of fundamental principles guiding this prior commitment, namely, equality of rights, nondiscrimination (including that of persons with disabilities), special protection of the autonomy and the development opportunities of children and, more recently, an emphasis on procedural justice [86].
While individual claims commonly correspond with moral duties [87], philosophers have argued that individual preferences per se – in other words, the notion that something is valuable to a person simply because she wants it – do not imply direct moral importance for others. To illustrate the point with a few examples: a preference for food, shelter or health care in case of need – all of these have a different moral quality than a preference for a luxury car, a soccer championship by one’s favorite team and so on, no matter what the strength of the latter may be [88,89]. This line of thought inevitably leads to a serious challenge to the relevance of individual preference satisfaction for social policy in general [88,89], and by implication, ultimately, no less so for healthcare resource allocation policy in particular. In the present context, discussing economic evaluation principles heavily relying on individual preferences for health states, it is important to note that preferences and needs are not equivalent [90]. This constitutes a clear contradiction to attempts to redefine 'the concept of need', by asserting that “ill health does not necessarily indicate a need for health care, evidence of its cost–effectiveness being required to reach that conclusion,” thus creating a tautology in order to justify claiming alignment between needs-based allocation and the logic of cost–effectiveness using health gains valued by individual preferences as the maximand [91].

Empirical ethics

If however social value transcends QALY maximization, one would expect that decreasing social desirability of medical interventions should be out of line with the ranking of interventions based on their increasing incremental cost per QALY gained, sometimes referred to as 'QALY league tables' [92,93]. A few (admittedly extreme) examples may indeed suffice to illustrate the clash between the logic of cost–effectiveness, when applied to questions of allocative efficiency, and widely held moral intuitions. The removal of tattoos [94] or the prescription of sildenafil for the treatment of erectile dysfunction in elderly men suffering from diabetes [95] would clearly meet the criterion of cost–effectiveness as defined by maximum allowable cost per QALY gained, because in these cases strong individual preferences meet relatively low overall cost per patient. On the other hand, interventions near the end of life [42,66,96], including palliative care measures [96–99], and interventions for people in double-jeopardy, in other words, those unfortunate enough to be afflicted with disabilities or other severe comorbidities [66,68,100], may face major obstacles to meet prevalent benchmarks of cost–effectiveness. This list could be extended easily.
These examples suggest that the issue of how to properly evaluate interventions for URDs does pose much less of a singular problem (that, apart from temporary pragmatism, could be treated appropriately by way of exemption, in a way as NICE in the UK have tried to), and more so a symptom of deeper deficiencies inherent to the conventional evaluation paradigm. This violation of moral intuitions has led economists to wonder whether “economic evaluation [is] in touch with society’s health values,” [101] and more basically, “what values do the public want their healthcare systems to use in evaluating technologies?” [102].
If community values are to be respected, then a promising way to approach the issue of appropriate social objectives will be to use empirical evidence, instead of relying on the assumptions of economic welfare theory or its extrawelfarist variant [43,103]. The fact that most people apparently have preferences beyond narrow self-interest may indeed offer a partial explanation of the discrepancies between the prescriptive value judgments derived from an axiomatic theoretical framework and descriptive social values. For example, people have an interest in the social arrangements that shape the communities they live in. They are concerned with the well-being of other persons and frequently their decisions are shaped by their social preferences, leading to competition or cooperation. As citizens in a democratic society they might vote for a party that announced reforms that put them at a personal disadvantage because they believe these serve some greater public interest.
Important social preferences include reciprocity (or 'reciprocal fairness'), inequality aversion and altruism, but also spiteful or envious preferences. However, neither spiteful nor altruistic preferences alone can explain why one and the same person may be willing to support others at a personal cost in one situation, while harming other people in other situations [104]. Social preferences cannot be fully understood without reference to norms and institutions [105]. To complicate matters further, they cannot omit the need for a prior normative commitment [84,106]. For example, a majority might vote for slavery or other immoral policies. Hence, not unlike individual preferences (which, e.g., might be “evil, ignorant, adaptive, or otherwise mistaken preferences” [107], which may be based upon imperfect information, cognitive biases, prejudices, or simply sadistic tendencies [108,109]), social preferences need to be morally defensible, in other words, screened by ethical argument or 'laundered' [76,110]. However, if both social and individual preferences were to be deemed irrelevant in a social policy context merely because of that limitation, the unsatisfying yet likely consequence by policy makers would be to revert back to attitudes of either nihilism or reductionism [75].
As a starting point, there is the need to recognize that the rapidly growing body of literature on social preferences clearly shows that the 'QALY maximization hypothesis' must be considered 'descriptively flawed' [111]. This conclusion has received particularly strong support from a systematic review of 64 empirical studies, which in most cases employed trade-off techniques in attempts to quantify the importance of attributes that have a bearing on fairness [111]. This observation in fact eliminates the normative foundation of universal cost per QALY benchmarks as a valid criterion of economic efficiency, unless analysts were prepared to deliberately ignore the question of “what do people as citizens want from their health services?” [112], and replace it by an empirically falsified assumption regarding the objective of collectively financed health care. Rather, social preferences notably include equity concerns and a 'sharing' perspective [113]. Before briefly discussing some of the most intriguing results on social preferences, it should be noted that many studies have been small, have looked at particular aspects in isolation and need to be interpreted cautiously on account of methodological choices and framing effects. The next section will begin with a review of primarily health state related preferences and continue with a description of primarily person-related dimensions. Finally, some provocative views of the public will be offered regarding the relevance of treatment costs. An important limitation of this section is the impossibility, in the present context, to explore in depth the moral relevance or defensibility and, hence, any potential need for 'laundering' these preferences [87,106–110].

Social preferences primarily related to health state

Perhaps the best documented and least controversial contextual variable is severity of the initial health state. In studies, people consistently show a strong preference to prioritize health care for the worse off, and this priority has been found to be largely (although not totally) independent from the improvement achieved by an intervention (i.e., the difference between the pre- and post-intervention health state as captured by the conventional computation of incremental QALY gains) [43,68,100,103,114,115]. Variations in study design have proven a challenge for recent attempts to quantify concerns for severity by transforming the observed effects into a social value function, despite consistently being found [116]. The priority for interventions targeting more severe initial health states quite plausibly parallels the criterion of need proposed by moral philosophers [44,84,90].
A social preference has also been found for giving priority to those with more urgent conditions. The term 'rule of rescue' has been coined to describe the moral imperative people feel to rescue persons facing avoidable death, largely irrespective of considerations of cost–effectiveness. The moral relevance of being a visible victim – as opposed to being not identifiable – has been subject to debate and controversy [43,103,117–123].
In correctly applied conventional cost–utility analysis, the life of a permanently disabled patient would be valued lower than the life of a healthy person. This potential consequence raised concern and harsh controversy between philosophers and health economists [59,61,64,124,125]. Accordingly, in contrast to QALY-based valuation, the capacity to benefit seems to be relatively less relevant empirically, as people appear to value additional health gains lower, once a certain (however, not readily quantifiable) minimum effect has been shown to be achieved by an intervention [120,126–129]. This social preference reminds of the closely related legal norm prohibiting discrimination against persons with disabilities, and, by implication, persons in double-jeopardy, such as the chronically ill or the permanently disabled [43,85,86,103,130–133]. 'Double-jeopardy' refers to the situation of a chronically ill or permanently disabled person who acquires an unrelated serious disease. She suffers a double-jeopardy because she is disadvantaged by acquiring the serious disease, and she is additionally disadvantaged because treatment will not return her to full health. On top of that, under correct application of the extrawelfarist logic of cost–effectiveness, she would be given a lower priority for intervention because, even if successful, the intervention would only restore the previous state of illness or disability [130–133].
At the surface, the ad hoc convention accepted by many health economists to ignore co-existing conditions in standard cost–utility analyses may appear to circumvent this issue. However, it represents a clear violation of the QALY maximization principle, since many utility gains so calculated will never accrue in real life. Thus this approach can hardly be realigned with the theoretical underpinnings of cost–utility analysis and, albeit often convenient for the analyst, must be considered as conceptually incorrect [66].

Social preferences primarily related to patient attributes

Patient-related attributes that have been found to influence the public’s prioritization preferences include (younger) age, parent and caregiver status and (non)smoker [134].
With regard to age, the majority of – albeit not all – studies to date have been supportive of assigning a somewhat higher priority to younger patients [115,135–144], but it has been found difficult by scholars to quantify the magnitude of this preference [143]. From a normative perspective, such an approach ('ageism') has been controversial [124,125,145–151]. It might however be justifiable on grounds of both equity and (utilitarian) efficiency arguments (irrespective of whether focused on health maximization or productivity). Within the field of health economics, Alan Williams has been the key proponent of the 'fair innings' argument, which “reflects the feeling that everyone is entitled to some ‘normal’ span of health […] and anyone failing to achieve this has been cheated…”[147]. Similar arguments have been made by philosopher Daniel Callahan [152]. Compared to cost–utility analysis, which considers future costs and outcomes, “the fair innings argument adopts a different standpoint and takes a person’s whole lifetime experience of health as the appropriate concept to use when assessing inequalities in health…” [147]. The 'fair innings' argument is not identical to but shows parallels with the view of 'a decent minimum of health' as a prerequisite for individuals to have the opportunity to realize their life plans [84,90]. It has further been argued that Sen’s capabilities approach might provide “a more natural justification of age related access to health care” [151] – of note, applying orthodox welfare economic principles might lead to the opposite answer, since one would have to expect average individual willingness to pay for a QALY to increase with increasing age [65]. This is because, technically speaking, decreasing marginal utility of both health and income would translate into changing marginal rates of substitution between life expectancy and income as a function of age: on average, income and, in particular, wealth will tend to rise and remaining life expectancy will shrink with increasing age [65].
Other personal attributes influencing observed preferences for healthcare prioritization, such as social roles like parent and caregiver status, and (non)smoker can be broadly classified to fall into one of the following categories [134]: a person’s relation to others (including relative socioeconomic status, family, contribution to community, criminal record, and so on) [120,135,153–155], individual lifestyle choices contributing to ill health (such as nicotine or alcohol consumption, drug use, unhealthy diet, and so on) [120,135,155–158] and even self-related personal attributes (such as gender, sexual orientation and race) [135,155]. The potential moral relevance of such characteristics has been discussed extensively, for example by Jan Abel Olsen and colleagues [134]. In particular the latter group of examples reinforces the inescapable need of screening empirical preferences, in order to determine whether they are defensible against a prior normative commitment [76,95–97,134]. For these reasons, the 'empirical ethics project' was suggested to be a process combining empiricism and normative theory, that should, it is hoped for, result in the 'best available theory' at any given point in time [159].

Social preferences, allocation rules & 'rarity'

Finally, the decision rules of the logic of cost–effectiveness will lead to 'all-or-nothing' decisions on programs, depending on whether they fall above or below the cut-off line for efficiency (i.e., some maximum incremental cost–effectiveness ratio, ICER, for a QALY gained). Studies however have shown that people are not at all indifferent to the fact that this way certain groups of patients would be entirely excluded from receiving health benefits; rather, there was a consistent willingness to sacrifice some efficiency in order to achieve equity in access [68,113,127,155,158]. Clearly most people were unprepared to abandon whole groups of patients [113,159,160], whereas the logic of QALY maximization, which is a prerequisite for the existence of cut-off thresholds or 'benchmarks' separating efficient from nonefficient interventions, would necessarily result in a 'winners take it all' solution [68]. Of note, a feature of a presumably 'technical' nature associated with cost–effectiveness analysis contributes to this outcome, namely, the assumed perfect divisibility of programs and constant returns to scale [55,57,58,79] but implied indivisibility of resources across programs (i.e., one program is always better than another in terms of efficiency and therefore ranked higher) [68] (see below).
The question of the appropriate benchmark for the maximum allowable cost per QALY gained can only be addressed if one insists that such a universally applicable benchmark exists, and hereby is prepared to ignore the well-documented social values of the general public, as delineated above. Even then, the conventional decision rule based on ICER thresholds would not be sufficient to determine the efficiency of resource use [161]. The primary reason for this deficiency is that (under the assumption of a limited budget [162]) “the threshold approach does not consider where the additional resources are to be taken from, and hence the benefits foregone (or opportunity costs) by removing these resources from other uses” [163].
Often also the assumption of constant returns to scale [57,58] does not hold either. In the case of drug treatment, for example, there are almost always sharply diminishing costs per unit of output due to economies of scale, experience curve effects and the typical high fixed/low variable cost structure of the biopharmaceutical industry [38,164–166]. Likewise, orthodox microeconomic equilibrium theory has distinct difficulties accommodating this cost structure, as it is often conveniently expressing both short-run as well as long-run marginal and average cost curves as U-shaped, in other words, with an upward slope [167,168].
On the other hand, one “might expect that the marginal utility of QALYs diminishes with the size of QALY production” [169] – a possibility which has been assumed away by two linearity assumptions built into the QALY calculation algorithms, in other words, the 'constant proportional trade-off' [170–172] and the strict proportionality of 'social utility' and number of beneficiaries imposed by the additive aggregation rule [58,66,68,100,173].
All of this quite obviously has a direct bearing on the validity of cost-per-QALY gained-based evaluations of URD treatments. In particular, during ICER computation by definition patient numbers, or more generally, the absolute size of the numerator and the denominator, will cancel each other out. Hence, the ICER also will not indicate the size of the evaluated programs, and therefore the budget impact of a decision to fund a program within a collectively financed health scheme [169,174]. This has been referred to as 'the silence of the lambda', with the 'lambda' being the incremental cost–effectiveness threshold [169,174].
These observations naturally led health economists to the question of whether we should 'value rarity' [175]. Rarity of a condition, per se, is hardly an adequate reason for prioritization. It has been in fact been argued that the opportunity cost associated with the use of high-cost drugs for rare disorders will be substantial, and for the same amount of resources far more patients with more prevalent disorders could benefit [175,176]. To quote, “the costs [for orphan drugs] will be borne by other, unknown patients, with more common diseases who will be unable to access effective and cost-effective treatment as a result.” [175] Rejecting the argument of limited overall budget impact, the same authors illustrated their position by pointing out that “…the cost should not be considered without reference to the value of what is foregone, £2.5m would pay for over 520 hip replacements” (with reference to the opportunity cost of a hypothetical drug costing £50,000 per year if there were only 50 patients to be treated) [176]. Indeed, hardly anyone would suggest that rare disorders, or patients afflicted with one of these conditions, can be likened to collectors’ items, just as vintage cars or stamps are for some people. It comes as no surprise that the (still very limited) evidence to date has been interpreted as indicative of limited support from the public to put special value on 'rarity' as such [115,177,178].
In one Norwegian survey, more than 80% of 1479 respondents endorsed the statements, “all should have equal access to health care regardless of costs,” and “patients with rare diseases should have the same right to treatment as others even if more expensive.” When asked to choose between funding a rare or a common disease, about half of the respondents expressed indifference; 42% favored dividing the funds available equally between the two disease groups, even if the common disease affected 200 patients and only 50 suffered from the rare disease. Economically this implies acceptance of a fourfold higher cost per patient for the rare compared with the common disorder. The study authors were, however, “reluctant to accept this interpretation as indication of a societal preference for rarity” [177], because (among other potential reasons cited, including a preference for the middle option or 'central tendency') the result could be a manifestation of aversion to choice [177,179]. Quite interestingly, in a follow-up survey of 551 Norwegian doctors reported by the same author, no evidence was found of “a general preference for prioritizing rarity” [180]. This was a marked difference to the findings of the population survey, and it was suggested that the findings “raise important questions about whose values should be consulted in determining priorities in the health sector” and what “if there are large inconsistencies between the principles that an individual espouses and the choices made in allocation tasks?” [180].
A recent smaller Canadian study using a discrete choice method did not confirm respondents’ willingness to pay more per life year gained for a rare disease than a common disease [178]. A choice-based UK social preference study also showed no societal values favoring expensive treatments for rare diseases [115]. Because of the potential presence of confounders and framing effects, it is not straight-forward to interpret and to directly compare the results of these studies [181,182]. At least at this stage, it would also seem that the preferences elicited in the Canadian and UK studies were not meeting the criterion of being informed ones [71,109,110], as a sample of the general population participating in a survey – and even many healthcare specialists – cannot be expected to be familiar with the economics of biopharmaceutical R&D. However, in the absence of information about the cost structure of pharmaceutical R&D, respondents would be unaware of the resulting consequences for patients if willingness to reimburse remained unadjusted to prevalence.
Reframing the alternatives, putting both the difference between horizontal and vertical equity [69,70] and the likely consequences for certain patient groups center stage [183], might well have led to an entirely different set of responses. Further to this, the focus on opportunity costs from an individual perspective rests on the assumptions that all QALYs are created equal, and that the sum of QALYs produced for the total population is the proper maximand (which have been shown to be ‘descriptively flawed’ [111]). Owing to the fact that in economics opportunity costs are defined as value foregone, the appropriate perspective on costs will therefore deserve further attention and scrutiny [43].

Social preferences & treatment costs

Empirical studies suggest that the importance of costs per patient may be overstated by conventional health economic evaluations [184], since cost-minimization, cost–effectiveness, cost–utility and cost–benefit analyses, by definition, focus significantly on incremental cost per case. In contrast, the public does not appear to be prepared to deny patients treatment merely on the basis of cost – which constitutes a social preference closely related to principles of fairness or rights-based reasoning, in many respect similar to the dislike of 'all-or-nothing' decisions, which also imply acceptance of opportunity costs (lost efficiency) in exchange for fairness (increased equity). As such, it does not necessarily imply valuing 'rarity' per se.
Erik Nord and colleagues [184], in an Australian survey, collected 551 responses on a set of choices between two equally effective interventions for two groups of patients, the only difference being cost. Subjects were explicitly alerted to the fact that higher cost of treatment would mean that fewer patients could be helped, given that resources were limited. Nevertheless, more than 80% of the respondents rejected cost as a relevant criterion when selecting a medical technology. Remarkably, even when challenged by the researchers highlighting the budget limitation, still a solid majority of 70% preferred the equal priority option. When asked to explain their responses, survey participants emphasized that people cannot be blamed for contracting high-cost illnesses, that severity of illness should count rather than cost, and that people are equally entitled to treatment irrespective to cost.” [184]. In addition, when the implications of the different choices were illustrated by use of numerical examples, 94% of the subjects challenged still continued to prefer a budget allocation that did not maximize aggregate utility.
These results are not very dissimilar from some statements obtained in the Norwegian survey on orphan drugs mentioned above, such as “all should have equal access to health care regardless of cost” [177] – but Nord’s study appears much more robust due to the repeated challenging of respondents and alerting them to the issue of opportunity costs. Corresponding findings were reported by Abellan-Perpinan and Pinto-Prades [127], who asked a Spanish convenience sample of respondents how they would allocate a budget between two treatments with the same outcome but the second of them twice as expensive as the first one. Seventy-four percent of respondents preferred a 2:1 budget allocation, regardless of the opportunity cost [127]. These results are consistent with the findings obtained in studies reported earlier, for example, two studies examining social preferences for the allocation of scarce organs for transplantation in England [168] and the USA [160], and an Australian exercise in healthcare budget sharing [159].
In contrast, according to the logic of cost–effectiveness, with its strong focus on ICER estimates and derived measures of efficiency, incremental costs per patient should be the yardstick [55,57,58]. For example, NICE has adopted the view that its decisions on technology coverage by the National Health Service (NHS) should be guided by cost–effectiveness (“an additional QALY is of equal value regardless of other characteristic…”), whereas “the potential budget impact of the adoption of a new technology does not determine the Appraisal Committee’s decision” [185]. Rather budget impact analysis at NICE is intended to primarily serve as an implementation tool [186]. However, from the viewpoint of a healthcare policy maker (or, in the case of the UK, a budget holding Regional Health Authority) total cost – or budgetary impact – may be relevant not only from a pure implementation perspective [186]. After all, it is the impact on healthcare and social budgets that will determine opportunity costs at the program level [43,161,163,169,174].
These observations are of immediate relevance to the evaluation of interventions for URDs. Whereas costs per patient for URD treatment will necessarily tend to be (sometimes much) higher than the cost per patient for more common disorders [40,41,187], as discussed earlier, most technologies for URDs have a limited overall budget impact only.
While this is usually true for individual treatments, the combined budgetary impact of the health service costs for many URDs may be more profound: budget impact for rare disease drugs in Europe was around 3.3% of total drug spending in 2010, with available data indicating a range from 0.75% (for The Netherlands, 2006) to 6.6% (projection for Europe, 2016, upper extreme scenario) [188–193]. Owing to the expiry of market exclusivity for an increasing number of products, it has been predicted that European sales growth of orphan drugs should level off through 2020, after having reached a peak of 4.6% (range between 3.0% and 6.6%, according to sensitivity analyses) of total drug sales [192]. URD treatments will represent only a (presumably small) part of the entire group of 'orphan drugs'.

Some methodological issues

Although this is not the focus of the present paper, it seems worth mentioning a few salient methodological problems that many analysts have treated as purely 'technical' issues.
Central to the extrawelfarist logic of cost–effectiveness is the idea of some benchmark for the monetary value of a statistical life year adjusted by a utility weight representing the individual preference for (health-related) quality of life (or QALY). One way to think about this value is as the shadow price of a QALY in a given health scheme, subject to a fixed budget constraint. This however cannot be determined empirically as it would require perfect information about the cost–effectiveness of all interventions funded [55,161,163]. An alternative way to interpret this benchmark is as the social willingness-to-pay for a QALY [55,67].
Unfortunately, there are several different economic methods available to estimate this value – and even when one excludes the human capital approach, the median values obtained vary greatly by study type, depending on the method chosen. In a review published in 2000, for example, these values ranged from a median of US$93,000 in revealed preference nonoccupational safety studies over a median of US$161,000 in contingent valuation studies to a median of US$428,000 in revealed preference job risk studies [194]. Thus the choice of method is a major determinant of the cost–effectiveness benchmark obtained, revealing all currently used benchmarks [18–24] as entirely arbitrary ad hoc standards [195].
Major parts of the health economic literature, addressing the process of ranking intervention based on their cost per QALY (the 'league table' approach [92,93]) as well as possible solutions to approximate a valid benchmark over time [196], treat this problem as a presumably technical one. This way they gloss over the more fundamental normative issues intrinsic to the application of a quasi-utilitarian calculus to the allocation of healthcare resources [44,68,81,100,124,125,183].
Measurement problems also arise regarding the choice of generic index instrument to measure the utility weights required for health state valuation, as a prerequisite for the computation of QALYs. The available instruments differ in their descriptive systems, the number and levels of dimensions and items, the valuation method used and their sensitivity to dimensions, and hence the QALY differences resulting from a change in certain health states may also differ greatly depending on the choice of instrument [67]. Not very surprisingly, convergent validity of the instruments has been shown to be relatively low, with instruments explaining only 41 to 64% of each others’ variance according to the two five instrument comparison studies published to date [197,198].
A substantial number of further issues has been 'resolved' by standardization via the introduction of conventions and additional assumptions, which not only collectively impose a large number of largely ad hoc restrictions on the evaluation function, but at the same time represent normative choices (in effect, value judgments) under the guise of technical discussion [43,44,55,66–68,78,79,100,103,112,183,199]. These choices include, but are not limited to, who should be asked when eliciting utility values for health states, whether to adopt an ex ante or ex post perspective, which method to use (time trade-off, standard gamble, choice of multiattribute utility instrument, and so on), the appropriate perspective for costing and which costs to include, how to determine productivity loss for analysis, the treatment of future unrelated costs and many more [44,55,78–80,100,112,199]. Taken together, they characterize the currently prevailing health economic evaluation paradigm, in other words, logic of cost–effectiveness using cost per QALY benchmarks, as a pragmatic 'technocratic' attempt [200] to solve the complex problem of allocating healthcare resources fairly, and must cast serious doubt on its status as a coherent scientific theory.

Some key observations

At this point it can be concluded that:
• QALYs, conceptualized as a preference-based measure of individual health-related outcomes combining quality and length of life, seemingly fail to capture the full social value of URD technologies; hence they need to be complemented by or replaced with alternatives that include societal preferences, such as concerns for equity in access to treatment;
• Current (cost per QALY) ICER thresholds used for cost–effectiveness (or more precisely, cost-utility) analysis are largely arbitrary and particularly inappropriate when used to evaluate URD technologies; their application may lead to positively unethical conclusions that might deprive patients with URDs any chance of access to effective care, thus conflicting with fairness- and rights-based considerations;
• The very existence of these thresholds (outside the confines of the narrow extrawelfarist framework) depends on the validity of the QALY maximization hypothesis, whereas systematic reviews of the literature have convincingly shown that this assumption is 'descriptively flawed', in other words, these thresholds do not capture well-established social preferences beyond to the quasi-utilitarian (health outcomes) maximization principle (which, by design, is 'distribution-blind');
• Attempts to apply modifiers to account for severity of disease (so-called 'equity' or 'severity weights' [201–204]) in economic assessments of technologies for URDs have not fully reflected the large number of contextual variables, and cannot solve the underlying issues with regard to fair chances to have access to effective treatment.


Collectively, the findings and observations summarized above underscore the need for an evaluation paradigm capturing and reflecting social preferences better than the conventional logic of cost–effectiveness. Alternative approaches should fare better than the conventional approach in tests of so-called reflective equilibrium, examining the social acceptability of priority rankings of healthcare programs [82,205–207]. Accordingly, the considered moral judgments about justice in particular cases should carry weight [208]. We predict that this will be accompanied by far-reaching implications for the evaluation of URDs.

Evidence of clinical effectiveness

The starting point of any value analysis can only be clinical benefit. In their comprehensive review of the first decade of orphan drug legislation in the EU, Joppi and colleagues (2013) [17] found that many orphan drugs were approved with evidence of surrogate end point effects only. In the absence of sufficiently strong evidence for some minimum significant and clinically relevant benefit [209,210], which however is not easily quantifiable [211,212], there is no basis for any robust value determination.
While recognizing the challenges associated with developing clinical interventions for URDs, the panel agreed that evidence for improvement of surrogate end points only should be no more than an interim attitude (in contrast to recent demands to further ease requirements for marketing authorization [213]), providing a basis for provisional approval and reimbursement, in order to ensure patients’ fast access to new technologies. It should be linked to managed entry schemes such as 'coverage with evidence development' agreements in order to incentivize further research [214,215].
There is a need for ongoing R&D for highly innovative and life-saving products for URDs, in order to increase clinical disease understanding and produce robust evidence on the clinical effectiveness of interventions [17,47–49,216]. Clinical benefit needs to be proven not promised. In general, the current state of affairs regarding orphan medicines and the available evidence of clinical effectiveness is not satisfactory [17,47,48,51,52]. Yet there are examples that demonstrate that it is feasible to prove relevant patient benefits [17,52,217].
Even at a prevalence rate of a given condition as low as 1/50,000 (the URD qualifier), there will be about 10,000 patients in Europe. Thus it should be possible to set up multinational randomized controlled trials, including between 500 and 1000 patients, designed to show relevant clinical end point benefit. If necessary, such trials might be supported not-for-profit research centers such as the “European Clinical Research Infrastructures Network” (ECRIN) initiative devoted to promote multinational studies [218,219].

Perspectives on cost

As stated earlier, the cost per patient will tend to be higher with decreasing prevalence. Budget impact, however, can be looked at in various different ways:
• One prevalent view (consistent with the efficiency-first approach advocated by conventional health economics) is that budget impact should not be relevant to coverage decisions, which ought to be based on incremental cost–effectiveness [55,175,176]. For example, NICE has taken the position that budget impact analyses should not form part of the decision-making process; rather, they should be used as a tool aiding UK Regional Health Authorities in implementing NICE guidance locally [185,186].
• Given that ICERs by design provide no information on the dimension of a program, as the size of the numerator and the denominator cancels out (the so-called 'silence of the lambda') [161,163,169,174], healthcare policy makers are concerned with the budget impact of adopting a technology (consistent with the notion of 'affordability'), and methods have been proposed by health economists on how one might combine incremental cost–effectiveness and budget impact into one metric [220].
• If a social value perspective (instead of a focus on individual utility) was to be adopted in a consistent manner, then there could be simultaneous implications for the definition of social opportunity cost (or value foregone) [43], with social value being driven by the existence of a program (i.e., for example, the value people might attach to living in a society that does not simply abandon certain groups of patients, who are unfortunate enough to suffer from a high-cost illness) and opportunity cost by its budgetary impact (or the totality of transfers, whether in cash or benefits-in-kind). This would obviously shift the focus from cost per patient to cost on the program level, which indeed reflects the perspective of a real-world decision maker. Incidentally, this would also correspond to recent trends in commercial value management, which have been characterized by a shift from price maximization to revenue management for maximizing corporate profit [221].
• Finally, a more pragmatic approach might combine rights-based thinking in terms of a desire to offer fair chances to receive effective treatment also to patients with URDs with the realities of pharmaceutical R&D and its fixed cost structure; resulting in the implementation of price/volume trade-offs as realized; for example, in France [222]. Such arrangements might also take account of other relevant variables, such as the security of revenue streams to the pharmaceutical company over the life cycle of products like enzyme replacement therapies, which may ensure demand for the patient’s lifetime, in contrast to the more limited duration of treatment of an acute condition.

Valuation principles

According to recent reviews, orphan drugs often are not supported by favorable cost–effectiveness data, and in some cases, cost–effectiveness information simply does not exist [187,223]. This might at least in part be explained by the shortcomings of the prevailing evaluation paradigm discussed above, and in some cases by the impossibility to meet standard benchmarks for cost–effectiveness. It has also been argued that, if those benchmarks were considered inappropriate for rare and URDs, producing cost–effectiveness estimates might become superfluous [223]. This leaves policy makers without useful economic guidance when they have to make defensible decisions on market access, reimbursement, and appropriate use of interventions for URDs [40,41,224,225].
Hence there is a need for evaluation principles that better reflect the public’s social preferences (compared with the logic of cost–effectiveness using cost per QALY benchmarks). Examples for such approaches, which hold promise to overcome at least some of the weaknesses of the conventional logic, include (but are not limited to):
• Methods combining traditional cost–effectiveness with budget impact analysis [220], or cost value analysis by means of adjusting cost per QALY benchmarks according to multiple contextual variables [204];
• Using alternatives to QALYs as a measure of benefit, such as 'capability-adjusted life years' [226–228];
• Cost value analysis using the person trade-off method [100], or cost value (or social utility) analysis using the relative social willingness-to-pay (RS-WTP) instrument [229];
• A multi-criteria decision analysis (MCDA) framework [46,230–233].
All of those should be rigorously assessed for their potential to improve on the currently predominant standard, which is still represented by conventional cost–utility analysis. Given the limitations of the conventional approach, the strengths and weaknesses of each of the alternatives should be explored with high priority. The group believes that investigating this way forward should take precedence over the continued application of an obviously deficient evaluation model for URDs. Needless to say, the group further believes that the constraints on the application of the conventional logic of cost–effectiveness as well as any alternative evaluation principles should, in principle, also apply to the assessment of interventions for 'non-ultra-rare disorders'. These constraints should allow healthcare policy makers to make well-informed decisions, setting limits fairly.
Table 1. Preliminary cost per quality-adjusted life year incremental cost–effectiveness ratio estimates by NICE (2008).
ConditionPrevalence (England)ProductICER (preliminary estimated £ per QALY)
M. Gaucher type I and III270Imiglucerase (Ceredase®)391,200
MPS type 1130Laronidase (Aldurazyme®)334,900
M. Fabry200Agalsidase beta (Fabrazyme®)203,000
Hemophilia B350Nonacog alpha (BeneFIX®)172,500
M. Gaucher type I270Miglustat (Zavesca®)116,800
These examples from England illustrate the mismatch between ultra-orphan drug cost and conventional cost–effectiveness benchmarks as adopted by NICE (i.e., £20,000 to £30,000 per QALY gained) [8].
ICER: Incremental cost–effectiveness ratio; MPS: Mucopolysaccharidosis; QALY: Quality-adjusted life year.
Executive summary


• In light of the high fixed (volume-independent) cost of R&D, many new drug treatments for ultra-rare disorders (URDs) will be unable to meet conventional criteria for cost–effectiveness (i.e., commonly applied cost per quality-adjusted life year [QALY] benchmarks).

Specific challenges

• As to the demonstration of clinical effectiveness, developing treatments for URDs frequently is a particularly complex and risky endeavor, due to often limited clinical understanding and limited availability of validated instruments to measure disease severity/progression, very small numbers of physicians with specialized expertise and small patient numbers that are often geographically dispersed.
• With regard to the evaluation of efficiency or 'value for money', ranking the social desirability of medical interventions based on incremental cost–effectiveness ratios (often expressed in terms of cost per QALY gained) critically depends on the validity of the unproven assumption that the objective of collectively financed health schemes ought to be to maximize the number of QALYs produced, given a resource constraint – regardless of distribution.
• The quasi-utilitarian calculus underlying the 'QALY maximization hypothesis', which in principle rests on an additive aggregation of individual preferences for health states, does not adequately incorporate widely held fairness concerns, for example, regarding equality of access to care, equal access for equal need or more generally the recognition of moral claims of individuals – in other words, issues of vertical equity (the unequal but equitable treatment of unequals) as opposed to horizontal equity (the equal treatment of equals).

Underlying issues

• With regard to the currently prevailing health economic evaluation principles, there are serious normative concerns, which are reflected by the nonutilitarian value judgments deeply ingrained in largely rights-based legal systems at the national and international levels.
• Empirically, social preferences of the general public with regard to healthcare resource allocation have been shown to prominently include (but not to be limited to) giving priority to the worst-off and to those with more urgent conditions, as well as nondiscrimination against persons in 'double-jeopardy' (such as the permanently disabled, or more broadly, the chronically ill and persons with comorbid conditions). The public appears to give relatively less weight to capacity to benefit, in other words, the absolute difference between the health state after versus before an intervention, which is the standard measure of benefit in cost–effectiveness analysis.
• Recent studies of social preferences indicated that 'rarity' per se may not represent an empirically supported criterion for prioritization. However, there remain questions with regard to the validity of these findings, for example, concerning potential framing effects and the issue whether the elicitation process in these studies met the requirement of measuring informed public preferences. Earlier studies suggested that patients should not be disadvantaged on the grounds of having contracted a high-cost illness cost; consequently, treatment cost was rejected by a majority of respondents even after challenging them by highlighting the opportunity costs.


• From a medical perspective, also for treatments of URDs, unequivocal proof of clinical effectiveness should be expected as a starting point of any value analysis. Where more flexible provisional approval and reimbursement schemes have been adopted, accepting improvement of surrogate end points, such policies should be interim solutions only, being linked to mandatory further clinical effectiveness research (e.g., 'coverage with evidence development').
• From a health economic perspective, a new framework will be required to consistently assess the 'value for money' offered by URD treatments from a societal perspective. This should imply the adoption of broader social perspectives on both cost (i.e., transfer costs or budgetary impact as the appropriate resource constraint) and benefits (i.e., the social value of adopting a program) compared with conventional cost–utility analysis, with its relatively narrow focus on incremental cost per patient and individual utility gains.
• Candidates for advanced frameworks for the evaluation of medical interventions for URDs (and beyond) include more pragmatic formal multi-criteria decision analysis approaches as well as more rigorous variants of social cost value analysis, for example, using the person trade-off method or the relative social willingness-to-pay instrument.
• Investigating these potential ways forward should take precedence over the continued application of obviously deficient conventional evaluation models.

Financial & competing interests disclosure

Organization of the 1-day workshop was supported by two biopharmaceutical firms, Alexion, Cheshire, CT, USA and BioMarin, San Rafael, CA, USA under an unrestricted educational grant policy. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.


Papers of special note have been highlighted as: • of interest; •• of considerable interest
Rinaldi A. Adopting an orphan. Incentives to develop drugs for rare disorders raise hopes and controversy. EMBO Reports 6(6), 507–510 (2005).
US FDA. Orphan Drug Act: congresssional findings for the Orphan Drug Act. Amended. In: Federal Food, Drug, and Cosmetic Act (FD&C Act). Food & Drug Administration (FDA), MD, USA (2013). 
Orphan Druganaut Blog. Rare disease and orphan drug regulation in Japan. In: Orphan Druganaut Blog. NY, USA (2013). 
Orphan Druganaut Blog. Orphan drugs and rare diseases in Australia. In: Orphan Druganaut Blog. NY, USA (2013). 
Song PP, Gao JJ, Inagaki Y, Kokudo N, Tang W. Rare diseases, orphan drugs, and their regulation in Asia: current status and future perspectives. Intract. Rare Dis. Res. 1(1), 3–9 (2012).
NICE Citizens Council. NICE Citizens Council Report Ultra Orphan Drugs. London, UK (2004). 
NICE. Appraising orphan drugs [updated 14 April 2008]. National Institute for Health and Care Excellence, London, UK (2008).
NHS. About NHS specialised services. 
Alberta Blue Cross, Pharmacy Services. Rare diseases drug coverage program. In: Alberta Human Services Drug Benefit Supplement. Alberta Blue Cross, AB, Canada (2013). 
Ontario Ministry of Health and Long-Term Care. Ontario public drug programs. In: The Ministry of Health and Long-Term Care. Drugs for rare diseases (DRD) screening template, ON, Canada (2011). 
Meekings KN, Williams CS, Arrowsmith JE. Orphan drug development: an economically viable strategy for biopharma R&D. Drug Discov. Today 13–14, 660–664 (2012).
Woodcock J. The future of orphan drug development. Clin. Pharmacol. Ther. 92(2), 146–148 (2012).
Melnikova I. Rare diseases and orphan drugs. Nat. Rev. Drug Discov. 11(4), 267–268 (2012).
Bashaw ED, Fang L. Clinical pharmacology and orphan drugs: an informational inventory 2006–2010. Clin. Pharmacol. Ther. 91(5), 932–936 (2012).
Thorat C, Xu K, Freeman SN et al. What the Orphan Drug Act has done lately for children with rare diseases: a 10-year analysis. Pediatrics 129(3), 516–521 (2012).
Joppi R, Bertele' V, Garattini S. Orphan drugs, orphan diseases. The first decade of orphan drug legislation in the EU. Eur. J. Clin. Pharmacol. 69(4), 1009–1024 (2013).
Paris V, Belloni A. Value in pharmaceutical pricing. In: OECD Health Working Papers, No. 63. OECD Publishing, Paris, France (2013). 
Devlin N, Parkin D. Does NICE have a cost–effectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Econ. 13(5), 453–460 (2004).
Rawlins MD, Culyer AJ. National Institute of Clinical Excellence and its value judgments. Br. Med. J. 329, 224–227 (2004).
Dakin HA, Devlin NJ, Odeyemi IAO. “Yes”, “no” or “yes, but”? Multinomial modelling of NICE decision-making. Health Policy 77, 352–367 (2006).
George B, Harris A, Mitchell A. Cost–effectiveness analysis and the consistency of decision-making: evidence from pharmaceutical reimbursement in Australia. Pharmacoeconomics 19, 1103–1109 (2001).
Pritchard C. Overseas approaches to decision-making. In: Cost–effectiveness Thresholds. Economic and Ethical Issues. Towse A, Pritchard C, Devlin N (Eds). King’s Fund and Office of Health Economics, London, UK, 56–68 (2002).
Laupacis A, Feeny D, Detsky AS, Tugwell PX. How attractive does a new technology have to be to warrant adoption and utilization? Ten tentative guidelines for using clinical and economic evaluations. Can. Med. Assoc. J. 146(4), 473–481 (1992).
Médecins Sans Frontières. Fatal imbalance. The Crisis in Research and Development for Drugs for Neglected Diseases. Médecins Sans Frontières Access to Essential Medicines Campaign and the Drugs for Neglected Diseases Working Group, Geneva, Switzerland (2001).
Schlander M, Beck M. Expensive drugs for rare disorders: to treat or not to treat? The case of enzyme replacement therapy for mucopolysaccharidosis. Curr. Med. Res. Opin. 25(5), 1285–1293 (2009).
DiMasi JA, Hansen RW, Grabowski HG. The price of innovation: new estimates of drug development costs. J. Health Econ. 22(2), 151–185 (2003).
DiMasi JA, Grabowski H. The cost of biopharmaceutical R&D: is biotech different? Managerial and Decision Economics 28(4–5), 469–479 (2007).
Morgan S, Grootendorst P, Lexchin J, Cunningham C, Greyson D. The cost of drug development: a systematic review. Health Policy 100(1), 4–17 (2011).
Vernon JA, Golec JH, DiMasi JA. Drug development costs when financial risk is measured using the Fama-French three-factor model. Health Econ. 19(8), 1002–1005 (2009).
Jorge Mestre F. The R&D cost of a new medicine. OHE Report. Office of Health Economics, London, UK (2012).
Grabowski H, Vernon J, DiMasi JA. Returns on research and investment for the 1990s new drug introductions. Pharmacoeconomics 20(Suppl. 3), 11–29 (2002).
Light DW. Global drug discovery: Europe is ahead. Health Aff. (Milwood) 28(5), w969–w977 (2009).
Lanthier M, Miller KL, Nardinelli C, Woodcock J. An improved approach to measuring drug innovation finds steady reates of first-in-class pharmaceuticals, 1987–2011. Health Aff. (Milwood) 32(8), 1433–1439 (2013).
Regnstrom J. Factors associated with success of market authorization applications for pharmaceutical drugs submitted to the European Medicines Agency. Eur. J. Clin. Pharmacol. 66, 39–48 (2010).
Tambuyzer E. Rare diseases, orphan drugs and their regulation: questions and misconceptions. Nat. Rev. Drug Dev. 9(12), 921–929 (2010).
Orfali M, Feldman L, Bhattachariee V et al. Raising orphans: how clinical development programs of drugs for rare and common diseases are different. Clin. Pharmacol. Ther. 92(2), 262–264 (2012).
Rollet P, Lemoine A, Dunoyer M. Sustainable rare business and drug access: no time for misconceptions. Orphanet J. Rare Dis. 8(109), 1–9 (2013).
Messori A, Cicchetti A, Patregani L. Orphan drugs. Relating price determination to disease prevalence. Brit. Med. J. 341, c4615 (2010).
Simoens S. Pricing and reimbursement of orphan drugs: the need for more transparency. Orphanet J. Rare Dis. 6(42), 1–8 (2011).
Picavet E, Cassiman D, Simoens S. Do ultra-orphan medicinal products warrant ultra-high prices? A review. Orphan Drugs: Res. Rev. 3, 23–31 (2013).
Trowman R, Chung H, Longson C, Littlejohns P, Clark P. The National Institute for Health and Clinical Excellence and its role in assessing the value of new cancer treatments in England and Wales. Clin. Cancer Res. 17(15), 4930–4935 (2011).
Richardson J, McKie J. Economic evaluation of services for a National Health scheme: the case for a fairness-based framework. J. Health Econ. 26(4), 785–799 (2007).
•• Outlines an alternative evaluation approach that is driven by social preferences for fairness and emphasizes the role of social transfers.
Luebbe W. Can the QALY approach be amended to meet fairness objectives? In: Economic Evaluation for Health Technology Assessment: Foundations and Limitations. Schlander M (Ed.). Springer, NY, USA (2014).
Schlander M, Garattini S, Kolominsky-Rabas P et al. Determining the value of medical technologies to treat ultra-rare disorders (URDs) – consensus statement based upon an international expert workshop held in Berlin / Germany, November 08, 2012. Final version of July 19, 2013. Institute for Innovation & Valuation in Health Care, Berlin and Wiesbaden, Germany (2013). 
Hughes-Wilson W, Palma A, Schuurman A, Simoens S. Paying for the orphan drug system: break or bend? Is it time for a new evaluation system for payers in Europe to take account of new rare disease treatments? Orphanet J. Rare Dis. 7 (74), 1–8 (2012).
• Discusses the need for and proposes a multicriteria decision analysis framework instead of simple cost–effectiveness analysis for the economic evaluation of interventions for orphan disorders.
Joppi R, Bertele V, Garattini S. Orphan drug development is progressing too slowly. Br. J. Clin. Pharmacol. 61(3), 355–360 (2006).
Joppi R, Bertele V, Garattini S. Orphan drug development is not taking off. Br. J. Clin. Pharmacol. 67(5), 494–502 (2009).
Heemstra HE, van Weely S, Büller HA, Leufkens HGM, de Vrueh RLA. Translation of rare disease research into orphan drug development: disease matters. Drug Discov. Today 14(23/24), 1166–1173 (2009).
Korn EL, McShane LM, Freidlin B. Statistical challenges in the evaluation of treatments for small patient populations. Sci. Transl. Med. 5, 178 (2013).
Picavet E, Cassiman D, Hollak CE, Maertens JA, Simoens S. Clinical evidence for orphan medicinal products – a cause for concern? Orphanet J. Rare Dis. 164(8), 1–8 (2013).
Kanters TA, de Sonneville-Koedoot C, Redekop WK, Hakkaart L. Systematic review of available evidence on 11 high-priced inpatient orphan drugs. Orphanet J. Rare Dis. 8(124), 1–7 (2013).
Sorensen C, Drummond M, Kanavos P. Ensuring value for money in health care. The role of health technology assessment in the European Union. World Health Organization (WHO), on Behalf of the European Obervatory on Health Systems and Policies. WHO Regional Office for Europe, Copenhagen, Denmark (2008).
Schlander M. The use of cost–effectiveness by the National Institute for Health and Clinical Excellence (NICE): no(t yet an) exemplar of a deliberative process. J. Med. Ethics 34, 534–539 (2008).
Drummond MF, Sculpher MJ, Torrance GW, O'Brien BJ, Stoddart GL. Methods for the Economic Evaluation of Health Care Programmes (3rd Edition). Oxford University Press, Oxford, UK (2005).
Tobin J. On limiting the domain of inequality. J. Law Economics 13, 263–277 (1970).
Weinstein MC, Zeckhauser R. Critical ratios and efficient allocation. J. Public Econom. 2, 147–157 (1973).
Weinstein MC, Stason WB. Foundations of cost–effectiveness analysis for health and medical practices. N. Engl. J. Med. 296, 716–721 (1977).
Culyer AJ. The rationing debate: maximising the health of the whole community – the case for. Br. Med. J. 314, 667–669 (1997).
Torrance GW. Utility measurement in healthcare: the things I never got to. Pharmacoeconomics 24(11), 1069–1078 (2006).
Culyer AJ. Economics and ethics in health care. J. Med. Ethics 27, 217–222 (2001).
Weinstein MC. A QALY is a QALY is a QALY – or is it? J. Health Econ. 7, 289–290 (1988).
Torrance GW, Feeney D. Utilities and quality-adjusted life years. Int. J. Technol. Assess. Health Care 5(4), 559–575 (1989).
Culyer AJ. The morality of efficiency in healthcare – some uncomfortable implications. Health Econ. 1, 7–18 (1992).
Gyrd-Hansen D. Willingness to pay for a QALY – theoretical and methodological issues. Pharmacoeconomics 23, 423–432 (2005).
Schlander M. Measures of efficiency in healthcare: QALMs about QALYs? Z Evid. Fortbild. Qual. Gesundh. 104(3), 209–226 (2010).
• Discusses important limitations of the cost per QALY approach as a tool to set limits.
Brazier J, Ratcliffe J, Salomon JA, Tsuchiya A. Measuring and Valuing Health Benefits for Economic Evaluation. Oxford University Press, Oxford, UK (2007).
Ubel PA. Pricing life – why it’s time for health care rationing. The MIT Press, Cambridge, MA, USA (2000).
•• Offers a broad discussion of ethical and procedural issues that arise when cost–effectiveness analysis is used to inform healthcare rationing decisions.
Mooney G. Economics, Medicine, and Health Care (3rd Edition). Prentice Hall, Harlow, England (2003).
Mooney G. Vertical equity in health care resource allocation. Health Care Anal. 8, 203–215 (2000).
Broome J. Ethics out of Economics. Cambridge University Press, Cambridge, UK (1999).
Neumann PJ, Weinstein MC. Legislating against use of cost–effectiveness information. N. Engl. J. Med. 363, 1495–1497 (2010).
Glaeske G. The dilemma between efficacy as defined by regulatory bodies and effectiveness in clinical practice. Dtsch. Arztebl. Int. 109(7), 115–116 (2012).
Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG): Allgemeine Methoden zur Bewertung von Verhältnissen zwischen Nutzen und Kosten. Version 1.0 of October 10, 2009. 
Giacomini M, Hurley J, Gold I, Smith P, Abelson J. The policy analysis of ‘values talk’: lessons from Canadian health reform. Health Policy 67(1), 15–24 (2004).
Smith RD, Richardson J. Can we estimate the ‘social’ value of a QALY? Four core issues to resolve. Health Policy 74(1), 77–84 (2005).
Kahneman D, Wakker P, Sarin R. Back to Bentham? Explorations of experienced utility. Q. J. Econ. 112, 375–405 (1997).
Dolan P, Kahneman D. Interpretations of utility and their implications for the valuation of health. Econ. J. 118(1), 215–234 (2008).
Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost–effectiveness in Health and Medicine. Oxford University Press, New York, NY; Oxford, UK (1996).
Hjelmgren J, Berggren F, Andersson F. Health economic guidelines – similarities, differences and some implications. Value Health 4(3), 225–250 (2001).
Sen A. Why health equity? Health Econ. 11, 659–666 (2002).
Rawls J. A Theory of Justice. Harvard University Press, MA, USA (1971).
Schlander M. Economic evaluation of medical interventions: answering questions people are unwilling to ask? Paper presented to the 5th World Congress of the International Health Economics Association (iHEA), Barcelona. Book of Abstracts, 194–195 (2005).
Daniels N. Just Health Care. Cambridge University Press, Cambridge, UK (1985).
Hyry HI, Roos JCP, Manuel J, Cox TM. The legal imperative for treating rare disorders. Orphanet J. Rare Dis. 8(135), 1–7 (2013).
Schlander M, Affolter C, Sandmeier H et al. Swiss HTA Consensus Project: guiding principles. Basel, Bern, Solothurn and Wiesbaden. 13 March 2012. 
Broome J. Weighing Goods. Blackwell, Oxford, UK (1991).
Nagel T. The View from Nowhere. Oxford University Press, Oxford, UK (1986).
Hausman DM, McPherson MS. Economic Analysis, Moral Philosophy, and Public Policy. Cambridge University Press, Cambridge, UK (1996).
Daniels N. Just Health. Meeting Health Needs Fairly. Cambridge University Press, Cambridge, UK (2008).
Culyer AJ. Need – is a consensus possible? Editorial J. Med. Ethics 24, 77–80 (1998).
Drummond MF, Torrance G, Mason J. Cost–effectiveness league tables: more harm than good? Soc. Sci. Med. 37(1), 33–40 (1993).
Mauskopf J, Rutten F, Schonfeld W. Cost–effectiveness league tables: valuable guidance for decision makers? Pharmacoeconomics 21(14), 991–1000 (2003).
Drummond MF, Wilson DA, Kanavos P, Ubel P, Rovira J. Assessing the economic challenges posed by orphan drugs. Int. J. Technol. Assess. Health Care 23(1), 36–42 (2007).
Stolk E, van Busschbach JJ, Caffa M, Meuleman EJH, Rutten FFH. Cost utility analysis of sildenafil compared with papaverine-phentolamine injections. Br. Med. J. 320, 1–6 (2000).
Normand C. Setting priorities in and for end-of-life care: challenges in the application of economic evaluation. Health Econ. Policy Law 7(4), 431–439 (2012).
Yang YT, Mahon MM. Palliative care for the terminally ill in America: the consideration of QALYs, costs, and ethical issues. Med. Health Care Philos. 15(4), 411–416 (2012).
Kinghorn P, Coast J. A health economics response to the review of the Liverpool Care Pathway. J. Palliative Med. (2013) (In Press).
Sutton EJ, Coast J. Development of a supportive care measure for economic evaluation of end-of-life care using qualitative methods. Palliat. Med, 28(2), 151–157 (2013).
Nord E. Cost-value analysis in health care. Making Sense out of QALYs. Cambridge University Press, Cambridge, UK (1999).
•• Discusses limitations of the traditional approach to cost–effectiveness analysis and offers cost value analysis as a possible extension addressing some of its most severe limitations.
Coast J. Is economic evaluation in touch with society’s health values? Br. Med. J. 329, 1233–1236 (2004).
Buxton MJ, Chalmers JD. What values do the public want their health care systems to use in evaluating technologies? Eur. J. Health Econ. 12, 285–288 (2011).
Richardson J, McKie J. Empiricism, ethics and orthodox economic theory: what is the appropriate basis for decision-making in the health sector? Soc. Sci. Med. 60, 265–275 (2005).
• Delineates the need for an alternative framework for health care priority setting.
Fehr E, Fischbacher U. Why social preferences matter – the impact of non-selfish motives on competition, cooperation and incentives. Econ. J. 112, C1–C33 (2012).
Postlewaite A. . Social norms and preferences. In: Handbook for Social Economics, Vol. 1A. Benhabib J, Bisin A, Jackson M (Eds). Elsevier, CA, USA, 31–67 (2011).
Roth TP. The ethics and the economics of minimalist government. Edward Elgar, Cheltenham, UK (2002).
Adler MD, Posner EA. New Foundations of Cost Benefit Analysis. Harvard University Press, Cambridge, MA, USA (2006).
Bronsteen J, Buccafusco C, Masur JS. Welfare as happiness. Georgetown Law J. 98, 1583–1641 (2010).
Harsanyi J. Utilities, preferences and substantive goods. Social Choice Welfare 14(1), 129–145 (1997).
Goodin RE. Chapter 9. Laundering preferences. In: Utlitarianism as a Public Philosophy. Cambridge University Press, Cambridge, UK, 132–148 (1995).
Dolan P, Shaw R, Tsuchiya A, Williams A. QALY maximization and people’s preferences: a methodological review of the literature. Health Econ. 14, 197–208 (2005).
Mooney G. Challenging Health Economics. Oxford University Press, Oxford, UK (2009).
Richardson J, Sinha K, Lezzi A, Maxwell A. Maximising health versus sharing: measuring preferences for the allocation of the health budget. Soc. Sci. Med. 75, 1351–1361 (2012).
• Providing empirical data from an Australia survey showing that respondents shared resources rather than minimizing cost per QALY as the budget increased, implying a willingness to fund high-cost QALYs.
Shah KK. Severity of illness and priority setting in healthcare: a review of the literature. Health Policy 93, 77–84 (2009).
Linley WG, Hughes DA. Societal views on NICE, cancer drugs fund and value-based pricing criteria for prioritising medicines: a cross-sectional survey of 4118 adults in Great Britain. Health Econ. 22, 948–964 (2013).
Nord E, Johansen R. Concerns for severity in priotity setting in health care: a review of trade-off data in preference studies and implications for societal willingness-to-pay for a QALY. Health Policy 116(2–3), 281–288 (2014).
• Reviewing 15 other articles covering reports from 20 individual studies in nine countries and 116 individual observations of the social value of utility improvement and baseline severity; severity is shown to be highly significant.
Jonsen AR. Bentham in a box: technology assessment and health care allocation. Law, Med. Health Care 14, 172–174 (1986).
Hadorn DC. Setting health priorities in Oregon. Cost–effectiveness meets the rule of rescue. J. Am. Med. Assoc. 265, 2218–2225 (1991).
Eddy DM. Oregon’s methods: did cost effectiveness analysis fail? J. Am. Med. Assoc. (JAMA) 266, 2135–2141 (1991).
Nord E, Richardson J, Street A, Kuhse H, Singer P. Maximizing health benefits vs. egalitarianism: an Australian survey of health issues. Soc. Sci. Med. 41, 1429–1437 (1995).
Shmueli A. Survival vs. quality of life: a study of the Israeli public priorities in medical care. Soc. Sci. Med. 49, 297–302 (1999).
Jenni KE, Loewenstein G. Explaining the ‘identifiable victim effect’. J. Risk Uncertainty 14(3), 235–257 (1997).
McKie J, Richardson J. The rule of rescue. Soc. Sci. Med. 56(12), 2407–2419 (2003).
Harris J. The Value of Life. Routledge and Kegan Paul, London, UK (1985).
Harris J. QALYfying the value of life. J. Med. Ethics 13(3), 117–123 (1987).
Nord E. The relevance of health state after treatment in prioritising between different patients. J. Med. Ethics 19(1), 37–42 (1993).
Abellan-Perpiñán JM, Prades JLP. Health state after treatment: a reason for discrimination? Health Econ. 8(8), 701–707 (1999).
Dolan P, Cookson R. A qualitative study of the extent to which health gain matters when choosing between groups of patients. Health Policy 51(1), 19–30 (2000).
Ubel PA, Richardson J, Baron J. Exploring the role of order effects in person trade-off elicitations. Health Policy 61(2), 189–199 (2002).
Singer P, McKie J, Kuhse H, Richardson J. Double jeopardy and the use of QALYs in health care allocation. J. Med. Ethics 21(3), 144–150 (1995).
Harris J. Double jeopardy and the veil of ignorance: a reply. J. Med. Ethics 21(3), 151–157 (1995).
McKie J, Kuhse H, Richardson J, Singer P. Double jeopardy, the equal value of lives and the veil of ignorance: a rejoinder to Harris. J. Med. Ethics 22(4), 195–196 (1996).
Ubel PA, Richardson J, Prades JLP. Life-saving treatments and disabilities: are all QALYs created equal? Int. J. Technol. Assess Health Care 15(4), 738–748 (1999).
Olsen JA, Richardson J, Dolan P, Menzel P. The moral relevance of personal characteristics in setting health care priorities. Soc. Sci. Med. 57(7), 1163–1172 (2003).
Charny MC, Lewis PA, Farrow SC. Choosing who shall not be treated in the NHS. Soc. Sci. Med. 28(12), 1331–1338 (1989).
Busschbach JJV, Hessing DJ, de Charro FT. The utility of health at different stages of life: a quantitative approach. Soc. Sci. Med. 37(2), 153–158 (1993).
Cropper ML, Aydede SK, Portney PR. Preferences for life saving programs: how the public discounts time and age. J. Risk Uncertainty 8(3), 243–265 (1994).
Nord E, Street A, Richardson J, Kuhse H, Singer P. The significance of age and duration of effect in social evaluation of health care. Health Care Anal. 4(2), 103–111 (1996).
Johannesson M, Johansson PO. Is the valuation of a QALY gained independent of age? Some empirical evidence. J. Health Econ. 16(5), 589–599 (1997).
Tsuchyia A. Age-related preferences and age weighting health benefits. Soc. Sci. Med. 48(2), 267–276 (1999).
Rodríguez E, Pinto, JLP. The social value of health programmes: is age a relevant factor? Health Econ. 9(7), 611–621 (2000).
Tsuchiya A. The Value of Health at Different Ages. Discussion Paper No. 184, University of York, Centre for Health Economics, York, UK (2001).
Tsuchiya A, Dolan P, Shaw R. Measuring people’s preferences regarding ageism in health: some methodological issues and some fresh evidence. Soc. Sci. Med. 57(4), 687–696 (2003).
Diederich A, Winkelhage J, Wirsik N. Age as a criterion for setting priorities in health care? A survey of the German public view. PLoS ONE 6(8), 1–10 (2011).
Harris J. More and better justice. In: Philosophy and Medical Welfare. Bell JM, Mendus S (Eds). Cambridge University Press, Cambridge, UK, 75–96 (1988).
Shaw AB. In defence of ageism. J. Med. Ethics 20(3), 188–194 (1994).
Williams A. Intergenerational equity: an exploration of the ‘fair innings’ argument. Health Econ. 6(2), 117–132 (1997).
Rivlin MM. Why the fair innings argument is not persuasive. BMC Med. Ethics 1, 1 (2000).
Williams A. The ‘fair innings’ argument deserves a fairer hearing! Comments by Alan Williams on Nord and Johannesson. Health Econ. 10(7), 583–585 (2001).
Nord E. Concerns for the worse off: fair innings versus severity. Soc. Sci. Med. 60 (2), 257–263 (2005).
Anand P. Capabilities and health. J. Med. Ethics 31(5), 299–303 (2005).
Callahan D. Setting limits: medical goals in an aging society with “response to my critics”. Georgetown University Press, Washington, DC, USA (1995).
Mooney G, Jan S, Wiseman V. Examining preferences for health care gains. Health Care Anal. 3(3), 261–265 (1995).
Neuberger J, Adams D, MacMaster P, Maidment A, Speed M. Assessing priorities for allocation of donor liver grafts: survey of public and clinicans. Brit. Med. J. 317, 172–175 (1998).
Dolan P, Cookson R, Ferguson B. Effect of discussion and deliberation on the public’s views of priority setting in health care: focus group study. Brit. Med. J. 318, 916–919 (1999).
Bowling A. Health care rationing: the public’s debate. Brit. Med. J. 312, 670–674 (1996).
Jowell R, Curtice J, Park A, Brook L, Thomson K. British Social Attitudes: The 13th Report. SCPR, London, UK (1996).
Ratcliffe J. Public preferences for the allocation of donor liver grafts for transplantation. Health Econ. 9(2), 137–148 (2000).
Richardson J. Chapter 12. Empirical ethics or the poverty of ethical analyses in economics and the unwarranted disregard of evidence in ethics. In: Summary Measures of Population Health: Papers from the WHO Global Conference. Marrakech, December 1999. Murray C, Lopez A (Eds). WHO, Geneva, Switzerland, 627–640 (2002).
Ubel PA, Loewenstein G. Distributing scarce livers: the moral reasoning of the general public. Soc. Sci. Med. 42(7), 1049–1055 (1996).
Birch S, Gafni A. Decision rules in economic evaluation. In: The Elgar Companion to Health Economics. Jones AM (Ed.). Edward Elgar, Cheltenham, UK, 492–502 (2006).
Weinstein MC, Torrance G, McGuire A. QALYs: the basics. Value Health 12(Suppl.1), S5–S9 (2009).
Birch S, Gafni A. The ‘NICE’ approach to technology assessment: an economics perspective. Health Care Manage. Sci. 7, 35–41 (2004).
Scherer FM. The pharmaceutical industry – prices and progress. N. Engl. J. Med. 351(9), 927–932 (2004).
Schweitzer SO. Pharmaceutical Economics and Public Policy (2nd Edition). Oxford University Press, Oxford, UK (2007).
Blackstone EA, Fuhr Jr JP. Biopharmaceuticals: the economic equation. Biotechnol. Healthc. 4(6), 41–45 (2007).
Boyes W, Melvin M. Microeconomics (8th Edition). South-Western Cengage Learning, OH, USA (2011).
Perloff JM. . Microeconomics (6th Edition). Addison-Wesley, MA, USA (2012).
Gafni A, Birch S. Incremental cost–effectiveness ratios (ICERs): the silence of the lambda. Soc. Sci. Med. 62, 2091–2100 (2006).
Johannesson M, Pliskin JS, Weinstein MC. A note on QALYs, time tradeoff, and discounting. Med. Decis. Making 14(2), 188–193 (1994).
Garber AM. Advances in cost–effectiveness analysis of health interventions. In: Handbook of Health Economics. Culyer AJ, Newhouse JP (Eds). Elsevier, Amsterdam, The Netherlands, Volume 1A, 182–221 (2000).
Attema AE, Brouwer WB. The way you do it? An elaborate test of procedural invariance of TTO, using a choice-based design. Eur. J. Health Econ. 13(4), 491–500 (2012).
Weinstein MC. Decision rules for incremental cost–effectiveness analysis. In: The Elgar Companion to Health Economics. Jones AM (Ed.). Edward Elgar, Cheltenham, UK, 469–478 (2006).
Birch S, Gafni A. Information created to evade reality (ICER). Things we should not look to for answers. Pharmacoeconomics 24, 1121–1131 (2006).
McCabe C, Claxton K, Tsuchiya A. Orphan drugs and the NHS: should we value rarity? Br. Med. J. 331, 1016–1019 (2005).
McCabe C, Tsuchiya A, Claxton K, Raftery J. Orphan drugs revisited. Q. J. Med. 99, 341–345 (2006).
Desser AS, Gyrd-Hansen D, Olsen JA, Grepperud S, Kristiansen IS. Societal views on orphan drugs: cross sectional survey of Norwegians aged 40 to 67. Brit. Med. J. 341, c4715, 1–6 (2010).
Mentzakis E, Stefanowska P, Hurley J. A discrete choice experiment investigating preferences for funding drugs used to treat orphan diseases: an exploratory study. Health Econ. Policy Law 6, 405–433 (2011).
Tversky A, Shafir E. Choice under conflict: the dynamics of deferred decision. Psychol. Sci. 3, 358–361 (1992).
Desser AS. Prioritizing treatment of rare diseases: a survey of preferences of Norwegian doctors. Soc. Sci. Med. 94, 56–62 (2013).
Ryan M, Scott DA, Reeves C et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol. Assess. 5, 5 (2001).
Desser AS, Olsen JA, Gerpperud S. Eliciting preferences for prioritizing treatment of rare diseases: the role of opportunity costs and framing effects. PharmaceEconomics 31, 1051–1061 (2013).
Davis JB, McMaster R. The individual in mainstream health economics: a case of persona non-grata. Health Care Anal. 15(3), 195–210 (2007).
Nord E, Richardson J, Street A, Kuhse H, Singer P. Who cares about cost? Does economic analysis impose or reflect social values? Health Policy 34, 79–94 (1995).
National Institute for Health and Care Excellence (NICE). Process and methods guides: guide to the methods of technology appraisal 2013. NICE, London, UK (2013). 
National Institute for Health and Clinical Excellence (NICE). Developing costing tools: methods guide. NICE, London, UK (2008). 
Vegter S, Rozenbaum MH, Postema R, Tolley K, Postma MJ. Review of regulatory recommendations for orphan drug submissions in the Netherlands and Scotland: focus on the underlying pharmacoeconomic evaluations. Clin. Ther. 32(9), 1651–1661 (2010).
de Varax A, Letellier M, Börtlein G. Study on Orphan Drugs Pt. I. Alcimed, Paris, France (2005).
Orofino J, Soto J, Casado MA, Oyagüez I. Global spending on orphan drugs in France, Germany, the UK, Italy and Spain during 2007. Appl. Health Econ. Health Policy 8(5), 301–315 (2010).
Denis A, Mergaert L, Fostier C, Cleemput I, Simoens S. Budget impact analysis of orphan drugs in Belgium. Estimates from 2008 to 2013. J. Med. Econ. 13(2), 295–301 (2010).
Association Francaise contre les Myopathies (AFM). Médicament Orphelins, un Dispositive à Renforcer. Février 2011. 
Schey C, Milanova T, Hutchings A. Estimating the budget impact of orphan medicines in Europe: 2010–2020. Orphanet J. Rare Dis. 6(2), 1–10 (2011).
Schwabe U. Spezialpräparate. In: Arzneiverordnungsreport 2010. Schwabe U, Paffrath D (Eds). Springer, Berlin, Heidelberg, 127–155 (2010).
Hirth RA, Chernew ME, Miller E, Fendrick AM, Weissert WG. Willingness to pay for a quality-adjusted life year: in search of a standard. Med. Decis. Making 20, 332–342 (2000).
Ubel PA, Hirth RA, Chernew ME, Fendrick AM. What is the price of life and why doesn’t it increase at the rate of inflation? Arch. Intern. Med. 163, 1637–1641 (2003).
Culyer A, McCabe C, Briggs A et al. Searching for a threshold, not setting one: the role of the National Institute for Health and Clinical Excellence. J. Health Serv. Res. Policy 12(1), 56–58 (2007).
Hawthorne G, Richardson J, Day NA. A comparison of the Assessment of Quality of Life (AQoL) with four other generic utility instruments. Ann. Med. 33(5), 358–370 (2001).
Fryback DG, Palta M, Cherepanov D, Bolt D, Kim JS. Comparison of 5 health-related quality-of-life indexes using item response theory analysis. Med. Decis. Making 30(1), 5–15 (2010).
Richardson J. Cost utility analysis: what should be measured? Soc. Sci. Med. 39(1), 7–21 (1994).
Buxton MJ. Looking for a willingness-to-pay threshold for a QALY – does it make sense? A practical view. ISPOR Connect. 13, 9–11 (2007).
Stolk EA, Pickee SJ, Ament AH, van Busschbach JJ. Equity in health care prioritization: an empirical inquiry into social value. Health Policy 74(3), 343–355 (2005).
Bleichrodt H, Crainich D, Eeckhoudt L. Aversion to health inequalities and priority setting in health care. J. Health Econ. 27(6), 1594–1604 (2008).
Cookson R, Drummond M, Weatherly H. Explicit incorporation of equity considerations into economic evaluation of public health interventions. Health Econ. Policy Law 4(2), 231–245 (2009).
Johri M, Norheim OF. Can cost–effectiveness analysis integrate concerns for equity? Systematic review. Int. J. Technol. Assess. Health Care 28(2), 125–132 (2012).
Daniels N. Reading Rawls. Blackwell, Oxford, UK (1975).
Daniels N. Wide reflective equilibrium and theory acceptance in ethics. J. Med. Philos. 76, 256–282 (1979).
Nord E. Methods for quality adjustment of life years. Soc. Sci. Med. 34, 559–569 (1992).
Tersman F. Reflective Equilibrium: An Essay in Moral Epistemiology. Almqvist & Wiksell International, Stockholm, Sweden (1993).
Wyrwich KW, Wolinsky FD. Identifying meaningful intra-individual change standards for health-related quality of life measures. J. Eval. Clin. Pract. 6(1), 39–49 (2000).
Gyatt GH, Osoba D, Wu AW, Wyrwich KW, Norman GR, Clinical Significance Consensus Meeting Group. Methods to explain the clinical significance of health status measures. Mayo Clin. Proc. 77(4), 371–383 (2002).
Copay AG, Subach BR, Glassman SD, Polly DW Jr, Schuler TC. Understanding the minimum clinically important difference: a review of concepts and methods. Spine J. 7(5), 541–546 (2007).
King MT. A point of minimal important difference (MID): a critique of terminology and methods. Expert Rev. Pharmacoecon. Outcomes Res. 11(2), 171–184 (2011).
Miyamoto BE, Kakkis ED. The potential investment impact of improved access to accelerated approval on the development of treatments for low prevalence rare diseases. Orphanet J. Rare Dis. 6(49), 1–13 (2011).
Jaroslawski S, Toumi M. Market access agreements for pharmaceuticals in Europe: diversity of approaches and underlying concepts. BMC Health Serv. Res. 11, 259 (2011).
Levin L, Goeree R, Levine M et al. Coverage with evidence development: the Ontario experience. Int. J. Technol. Assess. Health Care 27(2), 159–168 (2011).
Remuzzi G, Garattini S. Rare diseases: what’s next? Lancet 371, 1978–1979 (2008).
Marchioli R, Landolfi R, Barbui T, Tognoni G. Feasibility of randomized clinical trials in rare diseases: the case of polycythemia vera. Leuk. Lymphoma 22(Suppl. 1), 121–127 (1996).
Demotes-Mainard J, Kubiak C. A European perspective – the European clinical research infrastructures network. Ann. Oncol. 22(Suppl. 7), vii44–vii49 (2011).
Demotes-Mainard J. ECRIN (European clinical research infrastructures network), a pan-European infrastructure for clinical research. Bull. Acad. Natl Med. 194(9), 1683–1694 (2010).
Nuijten MJC, Rutten F. Combining a budgetary-impact analysis and a cost–effectiveness analysis using decision-analytic modelling techniques. Pharmacoeconomics 20(12), 855–867 (2002).
Talluri KT, van Ryzin GJ. The Theory and Practice of Revenue Management. Springer, NY, USA (2004).
Sermet C, Andrieu V, Godman B, van Ganse E, Haycox A, Reynier JP. Ongoing pharmaceutical reforms in France: implications for key stakeholder groups. Appl. Health Econ. Health Policy 8(1), 7–24 (2010).
Connock M, Burls A, Frew E et al. The clinical effectiveness and cost–effectiveness of enzyme replacement therapy for Guacher’s disease: a systematic review. Health Technol. Assess. 10(24), iii–iv, ix–136 (2006).
Garattini S, Bertele' V, Godman B et al. Enhancing the rational use of new medicines across European health care systems. Eur. J. Clin. Pharmacol. 64, 1137–1138 (2008).
Garattini S. Time to revisit the orphan drug law. Eur. J. Clin. Pharmacol. 68, 113 (2012).
Coast J, Smith RD, Lorgelly P. Welfarism, extra-welfarism and capability: the spread of ideas in health economics. Soc. Sci. Med. 67(7), 1190–1198 (2008).
Coast J, Smith RD, Lorgelly P. Should the capability approach be applied in health economics? Health Econ. 17(6), 667–670 (2008).
Al-Janabi H, Flynn TN, Coast J. Development of a self-report measure of capability wellbeing for adults: the ICECAP-A. Qual. Life Res. 21(1), 167–176 (2012).
Richardson J, Lezzi A, Sinha K, Khan MA, McKie J. An instrument for measuring the social willingness to pay for health state improvement. Health Econ. doi:10.1002/hec (2013) (Epub ahead of print).
Winquist E, Bell CM, Clarke JTR et al. An evaluation framework for funding drugs for rare diseases. Value Health 15, 982–986 (2012).
Specialised Healthcare Alliance. The challenge of rarity – putting the N in the NHS. England’s new approach to commissioning services, products and technologies for small patient populations.
Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A. Multi-criteria Decision Analysis to Value Orphan Medicines. Research Paper 13/03. Office of Health Economics, London, UK, 2013.
Ishizaka A, Nemery P. Multi-criteria decision analysis. Method. Softw. John Wiley & Sons, Chichester, UK, 2013.

Information & Authors


Published In


Published online: 2 October 2014


  1. consensus statement
  2. cost–effectiveness
  3. economic evaluation
  4. fairness
  5. health technology assessment
  6. medical ethics
  7. medical need
  8. orphan medicines
  9. quality-adjusted life year
  10. social preferences



Michael Schlander [email protected]
1Mannheim Medical Faculty, Department of Public Health, University of Heidelberg, Ludolf–Krehl–Strasse 7–11, D-68167 Mannheim, Germany;
2Institute for Innovation & Valuation in Health Care (InnoValHC), An der Ringkirche 4, D-65197 Wiesbaden, Germany
Silvio Garattini
IRCCS – Istituto di Richerche Farmacologiche Mario Negri, Via Giuseppe La Masa 19, I-20156 Milan, Italy
Søren Holm
Centre for Social Ethics & Policy, University of Manchester, Oxford Road, Manchester, M13 9PL, UK
Peter Kolominsky-Rabas
Interdisciplinary Centre for Health Technology Assessment (HTA) & Public Health (IZPH), University of Erlangen-Nürnberg, Schwabachanalge 6, D-91054, Erlangen, Bavaria, Germany
Erik Nord
Norwegian Institute of Public Health & School of Pharmacy, University of Oslo, Sem Sælands vei 3 Farmasibygningen, N-0371 Oslo, Norway
Ulf Persson
The Swedish Institute for Health Economics (IHE), Box 2127, S-220 02 Lund, Sweden
Maarten Postma
Department of Pharmacy, University of Groningen, Antonius Deusinglaan 1, NL-9713 AV Groningen, The Netherlands
Jeff Richardson
Centre for Health Economics, Monash University, Clayton, Victoria 3800, Australia
Steven Simoens
KU Leuven Department of Pharmaceutical & Pharmacological Sciences, Herestraat 49, B-3000 Leuven, Belgium
Oriol de Solà Morales
Institut Investigació Sanitaria Pere Virgili (IISPV), Avenir 1 Ppal 1a, E-08006 Barcelona, Spain
Keith Tolley
Tolley Health Economics Ltd, 19 Bath Road, Buxton, Derbyshire, SK17 6HH, UK
Mondher Toumi
UFR d’Odontologie, University Claude Bernard Lyon I, 11 rue Guillaume Paradin, F-69372 Lyon Cedex 08, France


*Author for correspondence: [email protected]; [email protected]

Metrics & Citations


Article Usage

Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.

Downloaded 372 times


How to Cite

Incremental cost per quality-adjusted life year gained? The need for alternative methods to evaluate medical interventions for ultra-rare disorders. (2014) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer.14.34

Export citation

Select the citation format you wish to export for this article or chapter.

View Options

View options


View PDF

Get Access

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.







Copy the content Link

Share on social media