A criterion-based approach to systematic and transparent comparative effectiveness: a case study in psoriatic arthritis
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: Indirect treatment comparisons are used when no direct comparison is available. Comparison networks should satisfy the transitivity assumption, that is, equal likelihood of treatment assignment for a given patient based on comparability of studies. Materials & methods: Seven criteria were evaluated across 18 randomized controlled trials in psoriatic arthritis: inclusion/exclusion criteria, clinical trial design and follow-up, patient-level baseline characteristics, disease severity, prior therapies, concomitant and extended-trial treatment and placebo response differences. Results: Across studies, placebo was a common comparator, and key efficacy end points were reported. Collectively, several potential sources of insufficient transitivity were identified, most often related to trial design and population differences. Conclusion: Potential challenges in satisfying transitivity occur frequently and should be evaluated thoroughly.
Supplementary Material
File (suppl_data.docx)
- Download
- 31.24 KB
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res. Synth. Methods 3(2), 80–97 (2012).
• Comprehensive review of indirect network comparisons that highlights important quality concerns in past designs. Definitions and descriptions of key terminology were applied within this report.
2.
Sutton A, Ades AE, Cooper N, Abrams K. Use of indirect and mixed treatment comparisons for technology assessment. Pharmacoeconomics 26(9), 753–767 (2008).
3.
The Airways Group: Cochrane Methods. Transitivity assumption (2014). https://airways.cochrane.org/sites/airways.cochrane.org/files/public/uploads/The%20transitivity%20assumption%20CJC.pdf
• Defines the transitivity assumption and highlights potential problems in network meta-analyses. These definitions and concepts were applied within this report.
4.
Hutton B, Salanti G, Caldwell DM et al. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann. Intern. Med. 162(11), 777–784 (2015).
5.
Statistical considerations in indirect comparisons and network meta-analysis. Cochrane Methods Training Event (PART 1) (2013). https://methods.cochrane.org/cmi/statistical-considerations-indirect-comparisons-and-network-meta-analysis
6.
Buckley F, Finckh A, Huizinga TW, Dejonckheere F, Jansen JP. Comparative efficacy of novel DMARDs as monotherapy and in combination with methotrexate in rheumatoid arthritis patients with inadequate response to conventional DMARDs: a network meta-analysis. J. Manag. Care Spec. Pharm. 21(5), 409–423 (2015).
7.
Jansen JP, Naci H. Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers. BMC Med. 11(1), 159 (2013).
•• Discusses validity issues in indirect comparsions and demonstrates graphically how imbalances in effect modifiers or patient characteristics can introduce different biases into comparative effect estimates.
8.
Lunt M, Solomon D, Rothman K et al. Different methods of balancing covariates leading to different effect estimates in the presence of effect modification. Am. J. Epidemiol. 169(7), 909–917 (2009).
9.
Jansen JP, Fleurence R, Devine B et al. Interpreting indirect treatment comparisons and network meta-analysis for health-care decision making: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: Part 1. Value Health 14(4), 417–428 (2011).
10.
Park HM. Comparing group means: T-tests and one-way ANOVA using STATA, SAS, R and SPSS. The University Information Techology Services (UITS) Center for Statistical and Mathematical Computing, Indiana University, IN, USA (2009). http://www.indiana.edu/∼statmath/stat/all/ttes
11.
Thompson SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309(6965), 1351–1355 (1994).
12.
Caldwell DM. An overview of conducting systematic reviews with network meta-analysis. Syst. Rev. 3(1), 109 (2014).
13.
Catala-Lopez F, Tobias A, Cameron C, Moher D, Hutton B. Network meta-analysis for comparing treatment effects of multiple interventions: an introduction. Rheumatol. Int. 34(11), 1489–1496 (2014).
14.
Nikolakopoulou A, Chaimani A, Veroniki AA, Vasiliadis HS, Schmid CH, Salanti G. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS ONE 9(1), e86754 (2014).
15.
Cutolo M, Myerson GE, Fleischmann RM et al. A Phase III, randomized, controlled trial of apremilast in patients with psoriatic arthritis: results of the PALACE 2 trial. J. Rheumatol. 43(9), 1724–1734 (2016).
• Apremilast trial used as a reference point to which the rest of baseline patient characteristics across the different treatments were compared in the proposed indirect comparison network.
16.
Edwards CJ, Blanco FJ, Crowley J et al. Apremilast, an oral phosphodiesterase 4 inhibitor, in patients with psoriatic arthritis and current skin involvement: a Phase III, randomized, controlled trial (PALACE 3). Ann. Rheum. Dis. 75(6), 1065–1073 (2016).
• Apremilast trial used as a reference point to which the rest of baseline patient characteristics across the different treatments were compared in the proposed indirect comparison network.
17.
Ungprasert P, Thongprayoon C, Davis JM 3rd. Indirect comparisons of the efficacy of subsequent biological agents in patients with psoriatic arthritis with an inadequate response to tumor necrosis factor inhibitors: a meta-analysis. Clin. Rheumatol. 35(7), 1795–1803 (2016).
18.
Betts KA, Griffith J, Friedman A, Zhou ZY, Signorovitch JE, Ganguli A. An indirect comparison and cost per responder analysis of adalimumab, methotrexate and apremilast in the treatment of methotrexate-naive patients with psoriatic arthritis. Curr. Med. Res. Opin. 32(4), 721–729 (2016).
• Presents different network meta-analyses consisting of the same, recent trials within psoriatic arthritis and states that differences in patient characteristics across the indirect networks were potential limitations in study interpretation.
19.
Gharaibeh M, Xu Y, Lee J, Chitnis M, Collier D. Efficacy of biologics and new anti-inflammatory agents used in the treatment of active psoriatic arthritis: systematic literature review and network meta-analysis of the evidence. Arthritis Rheumatol. 69(Suppl. 10), Abstract 1556 (2017). https://acrabstracts.org/abstract/efficacy-of-biologics-and-new-anti-inflammatory-agents-used-in-the-treatment-of-active-psoriatic-arthritis-systematic-literature-review-and-network-meta-analysis-of-the-evidence/
• Presents different network meta-analyses consisting of the same, recent trials within psoriatic arthritis and states that differences in patient characteristics across the indirect networks were potential limitations in study interpretation.
20.
Strand V, Elaine Husni M, Betts KA et al. Network meta-analysis and cost per responder of targeted Immunomodulators in the treatment of active psoriatic arthritis. BMC Rheumatol. 2(1), 3 (2018).
21.
Christensen AW, Tarp S, Furst DE et al. Most trial eligibility criteria and patient baseline characteristics do not modify treatment effect in trials using targeted therapies for rheumatoid arthritis: a meta-epidemiological study. PLoS ONE 10(9), e0136982 (2015).
22.
Cochrane Handbook for Systematic Reviews of Interventions. Higgins JPT, Green S (Eds). The Cochrane Collaboration (2011). https://handbook-5-1.cochrane.org/
23.
Donegan S, Williamson P, Gamble C, Tudur-Smith C. Indirect comparisons: a review of reporting and methodological quality. PLoS ONE 5(11), e11054 (2010).
24.
Song F, Loke YK, Walsh T, Glenny A-M, Eastwood AJ, Altman DG. Methodological problems in the use of indirect comparisons for evaluating healthcare interventions: survey of published systematic reviews. BMJ 338, b1147 (2009).
•• Provides numerous examples of potential validity and interpretive issues with indirect comparison networks. Some issues identified were used to inform this report's proposed transitivity criteria.
25.
Yildiz A, Nikodem M, Vieta E, Correll CU, Baldessarini RJ. A network meta-analysis on comparative efficacy and all-cause discontinuation of antimanic treatments in acute bipolar mania. Psychol. Med. 45(2), 299–317 (2015).
26.
Tonin FS, Rotta I, Mendes AM, Pontarolo R. Network meta-analysis: a technique to gather evidence from direct and indirect comparisons. Pharm. Pract. 15(1), 943 (2017).
27.
Chaimani A. Graphical tools for network meta-analysis in STATA. PLoS ONE 8(10), e76654 (2013).
28.
Phillippo DM, Ades AE, Dias S et al. NICE DSU Technical Support Document 18: methods for population-adjusted indirect comparisons in submissions to NICE. NICE (2019). http://nicedsu.org.uk/ 695 wp-content/uploads/2018/08/Population-adjustment-TSD-FINALref-rerun.pdf
29.
Rothman KJ, Greenland S, Walker AM. Concepts of interaction. Am. J. Epidemiol. 112(4), 467–470 (1980).
30.
Puhan MA, Schunemann HJ, Murad MH et al. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ 349, 5630 (2014).
31.
McInnes IB, Mease PJ, Kirkham B et al. Secukinumab, a human anti-interleukin-17A monoclonal antibody, in patients with psoriatic arthritis (FUTURE 2): a randomized, double-blind, placebo-controlled, Phase III trial. Lancet 386(9999), 1137–1146 (2015).
32.
Ritchlin C, Rahman P, Kavanaugh A et al. Efficacy and safety of the anti-IL-12/23 p40 monoclonal antibody, ustekinumab, in patients with active psoriatic arthritis despite conventional non-biological and biological anti-tumour necrosis factor therapy: 6-month and 1-year results of the Phase III, multicentre, double-blind, placebo-controlled, randomized PSUMMIT 2 trial. Ann. Rheum. Dis. 73(6), 990–999 (2014).
33.
Kavanaugh A, Mease PJ, Gomez-Reino JJ et al. Treatment of psoriatic arthritis in a Phase III randomized, placebo-controlled trial with apremilast, an oral phosphodiesterase 4 inhibitor. Ann. Rheum. Dis. 73(6), 1020–1026 (2014).
• Apremilast trial used as a reference point to which the rest of patient characteristics across the different treatments were compared in the proposed indirect comparison network.
34.
Schett G, Wollenhaupt J, Papp K et al. Oral apremilast in the treatment of active psoriatic arthritis: results of a multicenter, randomized, double-blind, placebo-controlled study. Arthritis Rheumatol. 64(10), 3156–3167 (2012).
• Apremilast trial used as a reference point to which the rest of patient characteristics across the different treatments were compared in the proposed indirect comparison network.
35.
McInnes IB, Kavanaugh A, Gottlieb AB et al. Efficacy and safety of ustekinumab in patients with active psoriatic arthritis: 1 year results of the Phase III, multicentre, double-blind, placebo-controlled PSUMMIT 1 trial. Lancet 382(9894), 780–789 (2013).
36.
Mease PJ, Fleischmann R, Deodhar AA et al. Effect of certolizumab pegol on signs and symptoms in patients with psoriatic arthritis: 24-week results of a Phase III double-blind randomized placebo-controlled study (RAPID-PsA). Ann. Rheum. Dis. 73(1), 48–55 (2014).
37.
Genovese MC, Mease PJ, Thomson GT et al. Safety and efficacy of adalimumab in treatment of patients with psoriatic arthritis who had failed disease modifying antirheumatic drug therapy. J. Rheumatol. 34(5), 1040–1050 (2007).
38.
Kavanaugh A, McInnes I, Mease P et al. Golimumab, a new human tumor necrosis factor alpha antibody, administered every four weeks as a subcutaneous injection in psoriatic arthritis: Twenty-four-week efficacy and safety results of a randomized, placebo-controlled study. Arthritis Rheumatol. 60(4), 976–986 (2009).
39.
Mease PJ, Gladman DD, Ritchlin CT et al. Adalimumab for the treatment of patients with moderately to severely active psoriatic arthritis: results of a double-blind, randomized, placebo-controlled trial. Arthritis Rheumatol. 52(10), 3279–3289 (2005).
40.
Mease PJ, Goffe BS, Metz J, Vanderstoep A, Finck B, Burge DJ. Etanercept in the treatment of psoriatic arthritis and psoriasis: a randomized trial. Lancet 356(9227), 385–390 (2000).
41.
Mease PJ, Kivitz AJ, Burch FX et al. Etanercept treatment of psoriatic arthritis: safety, efficacy, and effect on disease progression. Arthritis Rheumatol. 50(7), 2264–2272 (2004).
42.
Antoni C, Krueger GG, De Vlam K et al. Infliximab improves signs and symptoms of psoriatic arthritis: results of the IMPACT 2 trial. Ann. Rheum. Dis. 64(8), 1150–1157 (2005).
43.
Gottlieb A, Menter A, Mendelsohn A et al. Ustekinumab, a human interleukin 12/23 monoclonal antibody, for psoriatic arthritis: randomized, double-blind, placebo-controlled, crossover trial. Lancet 373(9664), 633–640 (2009).
44.
McInnes IB, Sieper J, Braun J et al. Efficacy and safety of secukinumab, a fully human anti-interleukin-17A monoclonal antibody, in patients with moderate-to-severe psoriatic arthritis: a 24-week, randomized, double-blind, placebo-controlled, Phase II proof-of-concept trial. Ann. Rheum. Dis. 73, 349–356 (2014).
45.
Antoni CE, Kavanaugh A, Kirkham B et al. Sustained benefits of infliximab therapy for dermatologic and articular manifestations of psoriatic arthritis: results from the infliximab multinational psoriatic arthritis controlled trial (IMPACT). Arthritis Rheumatol. 52(4), 1227–1236 (2005).
46.
Atteno M, Peluso R, Costa L et al. Comparison of effectiveness and safety of infliximab, etanercept, and adalimumab in psoriatic arthritis patients who experienced an inadequate response to previous disease-modifying antirheumatic drugs. Clin. Rheumatol. 29(4), 399–403 (2010).
47.
Kingsley GH, Kowalczyk A, Taylor H et al. A randomized placebo-controlled trial of methotrexate in psoriatic arthritis. Rheumatology (Oxford) 51(8), 1368–1377 (2012).
48.
Willkens RF, Williams HJ, Ward JR et al. Randomized, double-blind, placebo controlled trial of low-dose pulse methotrexate in psoriatic arthritis. Arthritis Rheumatol. 27(4), 376–381 (1984).
49.
Wells A, Edwards C, Adebajo AO et al. PALACE 4, a Phase III, randomized, controlled trial of apremilast, an oral phosphodiesterase 4 inhibitor, for treatment of psoriatic arthritis: long-term (52-week) improvements in physical function [abstract SAT0382]. Ann. Rheumatol. 73(Suppl. 2), 732 (2014).
50.
Chaimani A, Caldwell DM, Li T, Higgins JPT, Salanti G. Additional considerations are required when preparing a protocol for a systematic review with multiple interventions. J. Clin. Epidemiol. 83, 65–74 (2017).
51.
Ballegaard CJ, Jørgensen TS, Skougaard TS et al. Assessing the importance of trial characteristics as contextual factors when evaluating targeted therapies in patients with psoriatic disease: protocol for an exploratory systematic review and meta-research project. The Parker Institute (2016). http://www.parkerinst.dk/sites/default/files/study_protocol_3.pdf
52.
Briggs AM, March L, Lassere M et al. Baseline comorbidities in a population-based cohort of rheumatoid arthritis patients receiving biological therapy: data from the Australian rheumatology association database. Int. J. Rheum. 2009, 861481 (2009).
53.
Latimer NRA, Abrams KR. Adjusting survival time estimates in the presence of treatment switching. NICE DSU Technical Support Document 16. National Institute for Health and Care Excellence (NICE), London, UK (2014). http://nicedsu.org.uk/wp-content/uploads/2016/03/TSD16_Treatment_Switching.pdf
54.
Toh S, Li L, Harrold LR et al. Comparative safety of infliximab and etanercept on the risk of serious infections: does the association vary by patient characteristics? Pharmacoepidemiol. Drug Saf. 21(5), 524–534 (2012).
55.
Thorlund K, Druyts E, Avina-Zubieta JA, Wu P, Mills EJ. Why the findings of published multiple treatment comparison meta-analyses of biologic treatments for rheumatoid arthritis are different: an overview of recurrent methodological shortcomings. Ann. Rheum. Dis. 72(9), 1524–1535 (2013).
56.
Saad ED, Buyse M. Statistical controversies in clinical research: end points other than overall survival are vital for regulatory approval of anticancer agents. Ann. Oncol. 27(3), 373–378 (2016).
57.
Corbett M, Chehadah F, Biwas M et al. Certolizumab pegol and secukinumab for treating active psoriatic arthritis following inadequate response to disease modifying anti-rheumatic drugs: a systematic review and economic evaluation. Health Technol. Assess. 21(56), 1–326 (2017).
58.
Van SS, Diels J, Van LJ, Hemels M. Network meta-analysis with baseline risk adjustment to assess the relative efficacy of ustekinumab in adult patients with active psoriatic arthritis. Value Health 17(7), A373 (2014).
59.
Gladman DD, Thavaneswaran A, Chandran V, Cook RJ. Do patients with psoriatic arthritis who present early fare better than those presenting later in the disease? Ann. Rheum. Dis. 70(12), 2152–2154 (2011).
60.
Theander E, Husmark T, Alenius GM et al. Early psoriatic arthritis: short symptom duration, male gender and preserved physical functioning at presentation predict favourable outcome at 5-year follow-up. Results from the Swedish Early Psoriatic Arthritis Register (SwePsA). Ann. Rheum. Dis. 73(2), 407–413 (2014).
61.
Tillett W, Jadon D, Shaddick G et al. Smoking and delay to diagnosis are associated with poorer functional outcome in psoriatic arthritis. Ann. Rheum. Dis. 72(8), 1358–1361 (2013).
62.
McInnes IB, Nash P, Ritchlin C et al. Secukinumab for the treatment of psoriatic arthritis: comparative effectiveness results versus licensed biologics and apremilast from a network meta-analysis [abstract THU0437]. Ann. Rheum. Dis. 75(Suppl. 2), 348–349 (2016).
Information & Authors
Information
Published In
Pages: 1265 - 1298
PubMed: 31774340
Copyright
© 2019 Gabriel Tremblay. This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License
History
Received: 23 May 2019
Accepted: 20 August 2019
Published online: 5 September 2019
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
A criterion-based approach to systematic and transparent comparative effectiveness: a case study in psoriatic arthritis. (2019) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2019-0064
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Maria De Santis, Sara Martínez Breijo, Paul Robinson, Camille Capone, Katie Pascoe, Suzy Van Sanden, Mahmoud Hashim, Marco Trevisan, Caitlin Daly, Friso Reitsma, Sophie van Beekhuizen, Haoyao Ruan, Bart Heeg, Elena Verzoni, Feasibility of Indirect Treatment Comparisons Between Niraparib Plus Abiraterone Acetate and Other First-Line Poly ADP-Ribose Polymerase Inhibitor Treatment Regimens for Patients with BRCA1/2 Mutation-Positive Metastatic Castration-Resistant Prostate Cancer, Advances in Therapy, 10.1007/s12325-024-02918-6, 41, 8, (3039-3058), (2024).