Propensity score matching versus coarsened exact matching in observational comparative effectiveness research
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim & methods: We compared propensity score matching (PSM) and coarsened exact matching (CEM) in balancing baseline characteristics between treatment groups using observational data obtained from a pan-Canadian prostate cancer radiotherapy database. Changes in effect estimates were evaluated as a function of improvements in balance, using results from randomized clinical trials to guide interpretation. Results: CEM and PSM improved balance between groups in both comparisons, while retaining the majority of original data. Improvements in balance were associated with effect estimates closer to those obtained in randomized clinical trials. Conclusion: CEM and PSM led to substantial improvements in balance between comparison groups, while retaining a considerable proportion of original data. This could lead to improved accuracy in effect estimates obtained using observational data in a variety of clinical situations.
Supplementary Material
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Schaumberg DA, Shah S, Nordstrom BL, McDonald L, Ramagopalan SV, Stokes M. Evaluation of comparative effectiveness research: a practical tool. J. Comp. Eff. Res. 7(5), 503–515 (2018).
2.
Etz A. Introduction to the concept of likelihood and its applications. Adv. Methods Pract. Psychol. Sci. 1(1), 60–69 (2018).
3.
Vittinghoff E, Glidden DV, Shiboski SC, McCulloch CE. Regression Methods in Biostatistics, 2nd Edition . Gail M, Krickeberg K, Samet JM, Tsiatis A, Wong W (Eds). Springer, NY, USA, 1–527 (2012).
4.
Greenland S, Schwartzbaum J, Finkle W. Problems due to small samples and sparse data in conditional logistic regression analysis. Am. J. Epidemiol. 151(5), 531–539 (2000).
5.
King G, Lucas C, Nielsen R. Optimizing balance and sample size in matching methods for causal inference (2013). https://gking.harvard.edu/files/gking/files/frontier_0.pdf
6.
Ho DE, Imai K, King G, Stuart EA. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Polit. Anal. 15(3), 199–236 (2007).
• Reviews the issue of model dependence and researcher bias and demonstrates how preprocessing through matching can reduce model dependence and thus researcher bias.
7.
Grijalva CG, Roumie CL, Murff HJ et al. The role of matching when adjusting for baseline differences in the outcome variable of comparative effectiveness studies. J. Comp. Eff. Res. 4(4), 341–349 (2015).
8.
Yao XI, Wang X, Speicher PJ et al. Reporting and guidelines in propensity score analysis: a systematic review of cancer and cancer surgical studies. J. Natl Cancer Inst. 109(8), 1–9 (2017).
9.
Rubin DB, Rosenbaum PR. Reducing bias in observational studies using score on the propensity subclassification. J. Am. Stat. Assoc. 79(387), 516–524 (1984).
10.
Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983).
• Defines what the propensity score estimates (i.e., probability of treatment decision given a patient’s baseline characteristics) and how balancing on the propensity score is an effective way to reduce confounding by indication.
11.
King G, Nielsen R. Why propensity scores should not be used for matching. Polit. Anal. 27(4), 435–454 (2019).
• Reviews the limitations of matching on the propensity score to improve balance in the distribution of baseline characteristics and demonstrates the propensity matching paradox, which motivates the need for a systematic development and evaluation of matching strategies to optimize balance and sample size trade-off.
12.
Fullerton B, Boris P, Krohn R, Adams JL, Gerlach FM, Erler A. The comparison of matching methods using different measures of balance: benefits and risks exemplified within a study to evaluate the effects of german disease management programs on long-term outcomes of patients with Type 2 diabetes. Health Serv. Res. 51(5), 1960–1980 (2016).
13.
Ripollone JE, Huybrechts KF, Rothman KJ, Ferguson RE, Jessica M. Evaluating the utility of coarsened exact matching for pharmacoepidemiology using real and simulated claims data. Am. J. Epidemiol. 189(6), 613–622 (2020).
•• Compares the performance of coarsened exact matching relative to propensity score preprocessing techniques, using simulated and observational data.
14.
Sturges H. The choice of a class interval. J. Am. Stat. Assoc. 21(153), 65–66 (1926).
15.
Rodrigues G, Gonzalez-Maldonado S, Lukka H et al. The Prostate Cancer Risk Stratification (ProCaRS) Project: database construction and outcome analysis. Int. J. Radiat. Oncol. 84(3), S57 (2012).
16.
Smith GD, Pickles T, Crook J et al. Brachytherapy improves biochemical failure-free survival in low- and intermediate-risk prostate cancer compared with conventionally fractionated external beam radiation therapy: a propensity score matched analysis. Int. J. Radiat. Oncol. Biol. Phys. 91(3), 505–516 (2015).
17.
Morris WJ, Tyldesley S, Rodda S et al. Androgen Suppression Combined with Elective Nodal and Dose Escalated Radiation Therapy (the ASCENDE-RT Trial): an analysis of survival endpoints for a randomized trial comparing a low-dose-rate brachytherapy boost to a dose-escalated external beam boost for high- and intermediate-risk prostate cancer. Int. J. Radiat. Oncol. Biol. Phys. 98(2), 275–285 (2017).
18.
Jones C, Hunt D, McGowan D et al. Radiotherapy and short-term androgen deprivation for localized prostate cancer. N. Engl. J. Med. 365(2), 107–118 (2011).
19.
Rodrigues G, Lukka H, Warde P et al. The prostate cancer risk stratification (ProCaRS) project: recursive partitioning risk stratification analysis. Radiother. Oncol. 109(2), 204–210 (2013).
20.
Stephenson AJ, Kattan MW, Eastham JA et al. Prostate cancer-specific mortality after radical prostatectomy for patients treated in the prostate-specific antigen era. J. Clin. Oncol. 27(26), 4300–4305 (2009).
21.
Brookhart MA, Schneeweiss S, Rothman KJ, Glynn RJ, Avorn J, Stürmer T. Variable selection for propensity score models. Am. J. Epidemiol. 163(12), 1149–1156 (2006).
22.
Austin PC. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat. Med. 28, 3083–3107 (2009).
23.
Austin PC. A comparison of 12 algorithms for matching on the propensity score. Stat. Med. 33(6), 1057–1069 (2014).
24.
Stuart E, Lee B, Leacy F. Prognostic score–based balance measures for propensity score methods in comparative effectiveness research. J. Clin. Epidemiol. 66(Suppl. 8), S84–S90 (2013).
25.
Franklin JM, Rassen JA, Ackermann D, Bartels DB, Schneeweiss S. Metrics for covariate balance in cohort studies of causal effects. Stat. Med. 33, 1685–1699 (2014).
26.
Belitser SV, Martens EP, Pestman WR, Groenwold RHH, De Boer A, Klungel OH. Measuring balance and model selection in propensity score methods. Pharmacoepidemiol. Drug Saf. 20, 1115–1129 (2011).
27.
Team RS. RStudio: Integrated Development Environment for R (2021). www.rstudio.com/
28.
Ho D, Imai K, King G, Stuart EA. MatchIt: nonparametric preprocessing for parametric causal inference. J. Stat. Softw. 42(8), 1–43 (2011).
29.
Therneau TM, Lumley T, Atkinson E, Crowson C. Package ‘survival’. 1–176 (2020). https://github.com/therneau/survival
30.
Gayat E, Resche-Rigon M, Mary J. Propensity score applied to survival data analysis through proportional hazards models: a Monte Carlo study. Pharm. Stat. 11(3), 222–229 (2012).
31.
Khor R, Duchesne G, Tai K et al. Direct 2-arm comparison shows benefit of high-dose-rate brachytherapy boost vs external beam radiation therapy alone for prostate cancer. Int. J. Radiat. Oncol. Biol. Phys. 85(3), 679–685 (2013).
32.
Westreich D, Cole SR. Invited commentary: positivity in practice. Am. J. Epidemiol. 171(6), 674–677 (2010).
33.
Ludwig M, Kuban D, Du X, Lopez D, Yamal J, Strom S. The role of androgen deprivation therapy on biochemical failure and distant metastasis in intermediate-risk prostate cancer: effects of radiation dose escalation. BMC Cancer 15(190), 1–8 (2015).
34.
Iacus SM, King G, Porro G. Multivariate matching methods that are monotonic imbalance bounding. J. Am. Stat. Assoc. 106(493), 345–361 (2011).
35.
Ripollone JE, Huybrechts KF, Rothman KJ, Ferguson RE, Franklin M. Implications of the propensity score matching paradox in pharmacoepidemiology. Am. J. Epidemiol. 187(9), 1951–1961 (2018).
Information & Authors
Information
Published In
Pages: 939 - 951
PubMed: 34060903
Copyright
© 2021 Future Medicine Ltd.
History
Received: 16 March 2021
Accepted: 10 May 2021
Published online: 1 June 2021
Keywords:
Topics
Authors
Funding Information
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Propensity score matching versus coarsened exact matching in observational comparative effectiveness research. (2021) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2021-0069
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Zhengsheng Yu, Tianle Liu, Land-use trade-offs of utility-scale solar power: An NDVI-based assessment of ecological impacts and recovery trajectories, Land Use Policy, 10.1016/j.landusepol.2026.108098, 168, (108098), (2026).
- Ioannis E. Livieris, Niki Kiriakidou, Christos Diou, Causal-siamese: similarity-based treatment effect estimation with limited data, Evolving Systems, 10.1007/s12530-026-09836-8, 17, 3, (2026).
- Nadeem E. Jones, Ying Yu, Alexander F. Dagi, Christine H. Rohde, H. John Cooper, Nana O. Sarpong, Jarrod T. Bogue, Strategic medial gastrocnemius flap in two-stage revision knee arthroplasty for periprosthetic joint infections: Outcomes and costs in a matched cohort, Journal of Plastic, Reconstructive & Aesthetic Surgery, 10.1016/j.bjps.2026.03.017, 117, (84-93), (2026).
- Hanifa Pilvar, Catia Nicodemo, Stavros Petrou, Brian A. Darlow, Paula van Dommelen, Kari Anne I Evensen, Sarah Harris, John Horwood, Karen Mathewson, Saroj Saigal, Louis Schmidt, Dieter Wolke, Lianne J. Woodward, Sungwook Kim, Outcomes in Early Adulthood for Individuals Born Very Preterm and/or with Very Low Birth Weight: Evidence from Multinational Cohorts, The Journal of Pediatrics: Clinical Practice, 10.1016/j.jpedcp.2025.200196, 19, (200196), (2026).
- Johannes Wienker, Kaid Darwiche, Rüdiger Karpf-Wissel, Dirk Westhölter, Erik Büscher, Sebastian Zensen, Johannes Haubold, David Kersting, Hubertus Hautzel, Josef Homola, Christian Taube, Marcel Opitz, Marc Struß, Comparative analysis of pulmonary function decline in patients undergoing bronchoscopic lung volume reduction with endobronchial valves versus conservative treatment in emphysema management: A longitudinal coarsened exact matched analysis, Chronic Respiratory Disease, 10.1177/14799731251350709, 22, (2025).
- Mauricio Perez-Alaniz, Helena Lenihan, Justin Doran, Christian Rammer, Subsidising innovation outside or within firms’ existing knowledge bases: which is best for radical innovation?, Economics of Innovation and New Technology, 10.1080/10438599.2025.2486324, 35, 3, (372-400), (2025).
- Nina M. Clark, Alex H. Hernandez, Lisa M. Knowlton, Barclay T. Stewart, Eileen M. Bulger, Alexander Malloy, Geoffrey Anderson, Joseph L. Dieleman, Douglas Zatzick, John W. Scott, Pre- and postinjury financial hardship among trauma survivors: A national survey study, Journal of Trauma and Acute Care Surgery, 10.1097/TA.0000000000004545, 98, 5, (720-728), (2025).
- Yichuan Zhu, Xin Zhang, Jianquan Wang, Guanying Gao, Yan Xu, Revision hip arthroscopy for hip synovial chondromatosis is effective despite inferior postoperative clinical outcomes compared to patients undergoing primary hip arthroscopy: a matched control study with minimum 2-year follow-up, Journal of Orthopaedic Surgery and Research, 10.1186/s13018-024-05298-3, 19, 1, (2024).
- Sarah Whitaker, Sarah Cole, Maria Peri, Brady Ernst, Conor O’Neill, James Satalich, Jibanananda Satpathy, Higher complication and readmission rates after total knee arthroplasty with discharge to inpatient facility vs. home: a propensity score matched analysis, Journal of Orthopaedic Surgery and Research, 10.1186/s13018-024-05294-7, 19, 1, (2024).
- Malia SQ. Murphy, Kasim E. Abdulaziz, Éric Lavigne, Erica Erwin, Yanfang Guo, Alysha LJ. Dingwall-Harvey, David Stieb, Mark C. Walker, Shi Wu Wen, Hwashin Hyun Shin, Association between prenatal air pollutant exposure and autism spectrum disorders in young children: A matched case-control study in Canada, Environmental Research, 10.1016/j.envres.2024.119706, 261, (119706), (2024).
- See more