Skip to main content
Search
Search
Advanced Search
Skip main navigation
Free access
Review
21 February 2018

Evaluation of comparative effectiveness research: a practical tool

Authors: Debra A Schaumberg, Laura McDonald, Surbhi Shah, Michael Stokes, Beth L Nordstrom, and Sreeram V Ramagopalan [email protected]Author Info & Affiliations
Publication: J. Comp. Eff. Res.
Volume 7, Number 5
https://doi.org/10.2217/cer-2018-0007
PDFFormats

Abstract

Comparative effectiveness research (CER) guidelines have been developed to direct the field toward the most rigorous study methodologies. A challenge, however, is how to ensure the best evidence is generated, and how to translate methodologically complex or nuanced CER findings into usable medical evidence. To reach that goal, it is important that both researchers and end users of CER output become knowledgeable about the elements that impact the quality and interpretability of CER. This paper distilled guidance on CER into a practical tool to assist both researchers and nonexperts with the critical review and interpretation of CER, with a focus on issues particularly relevant to CER in oncology.

Supplementary Material

File (cer-2018-0007 supplementary data.docx)

References

Papers of special note have been highlighted as: •• of considerable interest
1.
Federal Coordinating Council for Comparative Effectiveness Research (FCCCER). FCCCER definition of Comparative Effectiveness. US National Library of Medicine (2017). https:/osp.od.nih.gov/wp-content/uploads/FCCCER-report-to-the-president-and-congress-2009.pdf
Google Scholar
2.
Price-Haywood EG. Clinical comparative effectiveness research through the lens of healthcare decision makers. Ochsner J. 15(2), 154–161 (2015).
PubMed
Google Scholar
3.
Blumenthal GM, Kluetz PG, Schneider J, Goldberg KB, McKee AE, Pazdur R. Oncology drug approvals: evaluating end points and evidence in an era of breakthrough therapies. Oncologist 22(7), 762–767 (2017).
PubMed
Google Scholar
4.
Patient-Centered Outcomes Research Institute (PCORI). Patient-Centered Outcomes Research Institute (PCORI) (2017). www.pcori.org/
Google Scholar
5.
International Society for Pharmacoeconomics and Outcomes Research (ISPOR). The leading global scientific and educational organization for HEOR and its use in decision making to improve health (2017). www.ispor.org/
Google Scholar
6.
International Society for Pharmacoepidemiology. International Society for Pharmacoepidemiology (ISPE), Home Page (2017). www.pharmacoepi.org
Google Scholar
7.
Agency for Healthcare Research and Quality (AHRQ). Agency for Healthcare Research and Quality (AHRQ). Home Page (2017). www.ahrq.gov/
Google Scholar
8.
European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). Home Page (2017). www.encepp.eu/
Google Scholar
9.
STROBE. STROBE Statement. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) (2017). www.strobe-statement.org/index.php?id=strobe-home
Google Scholar
10.
GRACE. Good Research for Comparative Effectiveness Initiative (GRACE) (2017). www.graceprinciples.org/
Google Scholar
11.
National Pharmaceutical Council (NPC). CER Collaborative Initiative (2017). www.npcnow.org/issue/cer-collaborative-initiative
Google Scholar
12.
Benchimol EI, Smeeth L, Guttmann A et al. The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. PLoS Med. 12(10), e1001885 (2015).
PubMed
Google Scholar
13.
Browman GP. Special series on comparative effectiveness research: challenges to real-world solutions to quality improvement in personalized medicine. J. Clin. Oncol. 30(34), 4188–4191 (2012).
PubMed
Google Scholar
14.
Dorey FJ. Statistics in brief: statistical power: what is it and when should it be used? Clin. Orthop. Relat. Res. 469(2), 619–620 (2011).
PubMed
Google Scholar
15.
Brookhart MA, Sturmer T, Glynn RJ, Rassen J, Schneeweiss S. Confounding control in healthcare database research: challenges and potential approaches. Med. Care 48(6 Suppl.), S114–S120 (2010).
PubMed
Google Scholar
•• Reviews approaches to control for confounding in pharmacoepidemiology database research.
16.
Suissa S, Garbe E. Primer: administrative health databases in observational studies of drug effects – advantages and disadvantages. Nat. Clin. Pract. Rheumatol. 3(12), 725–732 (2007).
PubMed
Google Scholar
17.
Lund JL, Richardson DB, Sturmer T. The active comparator, new user study design in pharmacoepidemiology: historical foundations and contemporary application. Curr. Epidemiol. Rep. 2(4), 221–228 (2015).
PubMed
Google Scholar
•• Reviews methodology to implement the active comparator, new user study design.
18.
Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am. J. Epidemiol. 158(9), 915–920 (2003).
PubMed
Google Scholar
19.
Setoguchi S, Gerhard T et al. Comparator Selection. Chapter 5. In: Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. Velentgas P, Dreyer NA, Nourjah P et al. Eds). Agency for Healthcare Research and Quality. Rockville, MD, USA (2013).
Google Scholar
20.
Groenwold RH, Hoes AW, Hak E. Confounding in publications of observational intervention studies. Eur. J. Epidemiol. 22(7), 413–415 (2007).
PubMed
Google Scholar
21.
Glynn RJ, Schneeweiss S, Sturmer T. Indications for propensity scores and review of their use in pharmacoepidemiology. Basic Clin. Pharmacol. Toxicol. 98(3), 253–259 (2006).
PubMed
Google Scholar
22.
Tan FES. Confounding in (non-) randomized comparison studies. OA Epidemiol. 1(3), 21 (2013).
Google Scholar
23.
Streeter AJ, Lin NX, Crathorne L et al. Adjusting for unmeasured confounding in nonrandomized longitudinal studies: a methodological review. J. Clin. Epidemiol. 87, 23–34 (2017).
PubMed
Google Scholar
24.
Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J. Clin. Epidemiol. 49(12), 1373–1379 (1996).
PubMed
Google Scholar
25.
Cepeda MS, Boston R, Farrar JT, Strom BL. Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders. Am. J. Epidemiol. 158(3), 280–287 (2003).
PubMed
Google Scholar
26.
Suissa S. Immortal time bias in pharmaco-epidemiology. Am. J. Epidemiol. 167(4), 492–499 (2008).
PubMed
Google Scholar
27.
Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. AHRQ. Agency for Healcare Research and Quality. Rockville, MD, USA (2013).
Google Scholar
•• Contains detailed information on best practices for comparative effectiveness research (CER) protocol development.
28.
Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J. Manag. Care Spec. Pharm. 22(10), 1107–1113 (2016).
PubMed
Google Scholar
29.
Shin S, Park CM, Kwon H, Lee KH. Erlotinib plus gemcitabine versus gemcitabine for pancreatic cancer: real-world analysis of Korean national database. BMC Cancer 16, 443 (2016).
PubMed
Google Scholar
30.
Moore MJ, Goldstein D, Hamm J et al. Erlotinib plus gemcitabine compared with gemcitabine alone in patients with advanced pancreatic cancer: a Phase III trial of the National Cancer Institute of Canada Clinical Trials Group. J. Clin. Oncol. 25(15), 1960–1966 (2007).
PubMed
Google Scholar
31.
American Cancer Society. Pancreatic Cancer Survival Rates, by Stage. (2016). www.cancer.org/cancer/pancreatic-cancer/detection-diagnosis-staging/survival-rates.html
Google Scholar
32.
Shin DW, Cho B, Guallar E. Korean National Health Insurance Database. JAMA Intern. Med. 176(1), 138 (2016).
PubMed
Google Scholar
33.
Yeh JM, Tramontano AC, Hur C, Schrag D. Comparative effectiveness of adjuvant chemoradiotherapy after gastrectomy among older patients with gastric adenocarcinoma: a SEER-Medicare study. Gastric Cancer 20(5), 811–824 (2017).
PubMed
Google Scholar
34.
Smalley SR, Benedetti JK, Haller DG et al. Updated analysis of SWOG-directed intergroup study 0116: a Phase III trial of adjuvant radiochemotherapy versus observation after curative gastric cancer resection. J. Clin. Oncol. 30(19), 2327–2333 (2012).
PubMed
Google Scholar
35.
Cunningham D, Allum WH, Stenning SP et al. Perioperative chemotherapy versus surgery alone for resectable gastroesophageal cancer. N. Engl. J. Med. 355(1), 11–20 (2006).
PubMed
Google Scholar
36.
Doria-Rose VP, Marcus PM. Death certificates provide an adequate source of cause of death information when evaluating lung cancer mortality: an example from the Mayo Lung Project. Lung Cancer 63(2), 295–300 (2009).
PubMed
Google Scholar
37.
Penson DF, Albertsen PC, Nelson PS, Barry M, Stanford JL. Determining cause of death in prostate cancer: are death certificates valid? J. Natl Cancer Inst. 93(23), 1822–1823 (2001).
PubMed
Google Scholar
38.
Percy C, Ries LG, Van Holten VD. The accuracy of liver cancer as the underlying cause of death on death certificates. Public Health Rep. 105(4), 361–367 (1990).
PubMed
Google Scholar
39.
Percy C, Stanek E 3rd, Gloeckler L. Accuracy of cancer death certificates and its effect on cancer mortality statistics. Am. J. Public Health 71(3), 242–250 (1981).
PubMed
Google Scholar
|