Skip to main content
Free access
5 March 2012

Newly marketed medications present unique challenges for nonrandomized comparative effectiveness analyses

The vast majority of medicines that come onto the market have met US FDA requirements for efficacy and safety. Of those, only half may have randomized evidence comparing the new agent to existing drugs, and of that group, only a small fraction may have fully evaluated a new drug’s relative effectiveness [1]. Clinicians, insurers, policy-makers and, ultimately, patients require timely results from post-marketing research to ascertain whether newly marketed drugs offer benefits in effectiveness and safety over existing standards of care. For this, nonrandomized (observational) studies using secondary healthcare data are key for assessment of drugs as they are used in routine care.
The general methodological challenges of performing comparative effectiveness research with nonrandomized data are magnified during this early marketing phase [2]. Challenges arise from one or more of the following sources: bias in initial treatment effects due to the differential channeling of sicker patients who may have waited for the arrival of the new drug in hope of improved effectiveness or safety; bias and treatment effect heterogeneity due to changing patient characteristics of the early users of a new drug compared with the broadening user base over the longer term; and the consequences of the potentially slow uptake of a new drug, leading to small numbers of users and thus imprecise effect estimates [3].
Perhaps the most serious of these challenges is the bias due to the differential nature of the early users of a new drug compared with the users of an existing standard of care. Early users of a new drug are likely to be a mix of two types of patients: those with an existing diagnosis that failed an earlier therapy or suffered from intolerable side effects, and those whose insurers and physicians are willing to initiate a newly diagnosed patient on a novel therapy. With the first group, the channeling of sicker patients – those with more severe disease and a poorer prognosis – toward the new drug will introduce confounding bias similar to that found in usual pharmacoepidemiology studies [4], but probably with even greater magnitude. With the second group, biases may result from physician- and system-level factors that go beyond the patient’s condition and characteristics. Those patients with generous insurance plans or doctors attuned to new developments in the drug market may be differential with respect to socioeconomic status, quality of care and other factors known to affect patients’ risk of a variety of outcomes. On another level, this heterogeneity of the patient population may lead to a heterogeneity of treatment effects; those patients that have failed earlier therapies may be less likely to be treated effectively with the new medication (or any second-line therapy, for that matter). Although profound, these biases, in many instances, can be controlled using sophisticated analytic techniques; in other cases, the biases may not be controllable without baseline randomization.
The observed effect of a newly marketed drug may change quickly as a consequence of the changing population of patients who are given the drug. While the confounding described in the previous paragraph will be most acute shortly after a drug goes on the market, there may be a lessening differential over time: a larger portion of the patient population may be homogenous with respect to baseline health status, more insurance companies may place the drug on their formularies, and prescribers may have greater familiarity with the new compound. Any easing of confounding will in turn cause a shift in the effect estimated in the early periods as compared with a later ‘quiescent state’.
The numbers involved in studies of newly marketed medications present statistical challenges. The speed of the uptake of the drug is strongly influenced by prevalence of the indication, drug retail price, insurance coverage, volume of marketing, and perceived effectiveness and safety profile compared with existing agents. If the uptake is slow, there will be few users in the early periods; combined with a rare outcome, there can be little information available for statistical inference on effectiveness. In the initial phases of marketing, population-wide effects, let alone subgroup analyses, can be difficult to estimate even in very large healthcare databases. Without sufficient numbers there are few alternatives other than altering the study design or gathering a larger study base through the passage of time.
Special challenges will arise in specific circumstances. In the case of a first-in-class medication, there may be no appropriate drug to compare to, which may in turn necessitate before/after studies. These will be valid if the uptake of the new drug is rapid and the new drug virtually replaces the old. Comparing drug therapy to medical management or a surgical procedure can be difficult due to very strong biases that often channel sicker patients away from interventional therapies. Drugs that treat diseases with time-delayed outcomes (e.g., cancer drugs and certain preventive medications) and those without easily measurable surrogate markers may be difficult to study in certain administrative healthcare databases, since long-term follow-up may be limited to a specific subpopulation. Limited-time-use drugs or drugs with long effect windows may be challenging to evaluate since the effect of the exposure can be hard to pinpoint.
While the list of challenges to performing comparative effectiveness research about recently marketed drugs may seem extensive, epidemiologists and biostatisticians have developed methods to overcome many of the issues faced. In cases where confounding is strong and fundamentally unmeasurable, or when populations on the therapies under study are simply incomparable, the answer will be to work with Phase IV randomized trials, simulation studies [5], or a combination of study techniques. However, in the majority of cases, appropriate study design combined with techniques that apply tested epidemiologic principles in new ways, such as sequential cohort studies [6], high-dimensional propensity scores [7], marginal structural models [8], self-controlled designs [9,10] and other approaches can mitigate many of the factors we describe and thus enable successful evaluation of newly marketed drugs without randomization.

Financial & competing interests disclosure

J Rassen was supported by a career development award from the Agency for Healthcare Research and Quality (K01-HS018088). S Schneeweiss is a member of the Methods Committee of the Patient-Centered Outcomes Research Institute and Director of the Brigham and Women’s Hospital DEcIDE Center for Comparative Effectiveness Research and the DEcIDE Methods Center, both funded by the Agency for Healthcare Research and Quality. The opinions expressed here are only those of the authors and not necessarily those of the aforementioned institutions. S Schneeweiss’s work was funded in part by investigator-initiated research grants from the Novartis Pharmaceuticals Corporation and from Pfizer, and was also supported by grants from the National Library of Medicine (R01-LM010213 and RC1-LM010351), the National Heart, Lung, and Blood Institute (RC4-HL102023), and the National center for Research Resources (RC1-RR028231). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.



Goldberg NH, Schneeweiss S, Kowal MK, Gagne JJ. Availability of comparative efficacy data at the time of drug approval in the United States. JAMA 305,1786–1789 (2011).
Schneeweiss S. Developments in post-marketing comparative effectiveness research. Clin. Pharmacol. Ther. 82,143–156 (2007).
Schneeweiss S, Gagne JJ, Glynn RJ, Ruhl M, Rassen JA. Assessing the comparative effectiveness of newly marketed medications: methodological challenges and implications for drug development. Clin. Pharmacol. Ther. 90,777–790 (2011).
Walker AM. Confounding by indication. Epidemiology 7,335–336 (1994).
Holford NH, Kimko HC, Monteleone JP, Peck CC. Simulation of clinical trials. Annu. Rev. Pharmacol. Toxicol. 40,209–234 (2000).
Gagne JJ, Rassen JA, Walker AM, Glynn RJ, Schneeweiss S. Active safety monitoring of new medical products using electronic healthcare data: selecting alerting rules. Epidemiology doi: 10.1097/EDE.0b013e3182459d7d (2012) (Epub ahead of print).
Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20,512–522 (2009).
Hernán MA, Brumback B, Robins JM. Marginal structural models to estimate the causal effect of zidovudine on the survival of HIV-positive men. Epidemiology 11,561–570 (2000).
Maclure M. The case-crossover design: a method for studying transient effects on the risk of acute events. Am. J. Epidemiol. 133,144–153 (1991).
Wang S, Linkletter C, Maclure M et al. Future cases as present controls to adjust for exposure trend bias in case-only studies. Epidemiology 22,568–574 (2011).

Information & Authors


Published In


Published online: 5 March 2012


  1. bias
  2. comparative effectiveness research/methods
  3. confounding factors
  4. drug approval
  5. epidemiologic methods
  6. humans
  7. propensity score
  8. statistical models time factors



Jeremy A Rassen* [email protected]
Division of Pharmacoepidemiology & Pharmacoeconomics, Department of Medicine, Brigham & Women’s Hospital & Harvard Medical School, Boston, MA, USA. [email protected]
Sebastian Schneeweiss
Division of Pharmacoepidemiology & Pharmacoeconomics, Department of Medicine, Brigham & Women’s Hospital & Harvard Medical School, Boston, MA, USA.


* Author for correspondence

Metrics & Citations


Article Usage

Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.

Downloaded 24 times


How to Cite

Newly marketed medications present unique challenges for nonrandomized comparative effectiveness analyses. (2012) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer.12.12

Export citation

Select the citation format you wish to export for this article or chapter.

Citing Literature

  • Overlap, matching, or entropy weights: what are we weighting for?, Communications in Statistics - Simulation and Computation, 10.1080/03610918.2024.2319419, (1-20), (2024).
  • Real-World Analysis of Clinical and Demographic Characteristics, Treatment Patterns, and Outcomes in Predominantly Older Patients with HR+/HER2− Metastatic Breast Cancer Receiving Abemaciclib in Routine Clinical Practice, Drugs - Real World Outcomes, 10.1007/s40801-023-00391-1, 10, 4, (589-603), (2023).
  • Real-world treatment patterns and outcomes of abemaciclib for the treatment of HR + , HER2- metastatic breast cancer patients in Japan, Breast Cancer, 10.1007/s12282-023-01461-6, 30, 4, (657-665), (2023).
  • Real world incidence and management of adverse events in patients with HR+, HER2− metastatic breast cancer receiving CDK4 and 6 inhibitors in a United States community setting, Current Medical Research and Opinion, 10.1080/03007995.2022.2073122, 38, 8, (1319-1331), (2022).
  • Real-World Patient Characteristics, Utilization Patterns, and Outcomes of US Patients with HR+, HER2− Metastatic Breast Cancer Treated with Abemaciclib, Drugs - Real World Outcomes, 10.1007/s40801-022-00327-1, 9, 4, (681-693), (2022).
  • Real-world treatment patterns and outcomes of abemaciclib for the treatment of HR+, HER2− metastatic breast cancer, Current Medical Research and Opinion, 10.1080/03007995.2021.1923468, 37, 7, (1179-1187), (2021).
  • Systematic review of Real-World effectiveness of eribulin for locally advanced or metastatic breast cancer, Current Medical Research and Opinion, 10.1080/03007995.2020.1835853, 36, 12, (2025-2036), (2020).
  • Current use of routinely collected health data to complement randomized controlled trials: a meta-epidemiological survey, CMAJ Open, 10.9778/cmajo.20150036, 4, 2, (E132-E140), (2016).
  • On the use of propensity scores in case of rare exposure, BMC Medical Research Methodology, 10.1186/s12874-016-0135-1, 16, 1, (2016).
  • Adjusting for Confounding in Early Postlaunch Settings, Epidemiology, 10.1097/EDE.0000000000000388, 27, 1, (133-142), (2016).

View Options

View options


View PDF

Get Access

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.







Copy the content Link

Share on social media