Importance of assessing and adjusting for cross-study heterogeneity in network meta-analysis: a case study of psoriasis
Abstract
Aim: The importance of adjusting for cross-study heterogeneity when conducting network meta-analyses (NMAs) was demonstrated using a case study of biologic therapies for moderate-to-severe plaque psoriasis. Methods: Bayesian NMAs were conducted for Psoriasis Area and Severity Index 90 response. Several covariates were considered to account for cross-trial differences: baseline risk (i.e., placebo response), prior biologic use, body weight, psoriasis duration, age, race and baseline Psoriasis Area and Severity Index score. Model fit was evaluated. Results: The baseline risk-adjusted NMA, which adjusts for multiple observed and unobserved effect modifiers, was associated with the best model fit. Lack of adjustment for cross-trial differences led to different clinical interpretations of findings. Conclusion: Failure to adjust for cross-trial differences in NMA can have important implications for clinical interpretations when studying the comparative efficacy of healthcare interventions.
Supplementary Material
File (suppl_material.docx)
- Download
- 1.02 MB
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 331(7521), 897–900 (2005).
2.
Lu G, Ades A. Combination of direct and indirect evidence in mixed treatment comparisons. Stat. Med. 23, 3105–3124 (2004).
3.
Dias S, Sutton A, Welton N, Ades A. NICE DSU Technical Support Document 3: heterogeneity: subgroups, meta-regression, bias and bias-adjustment (2012). www.nicedsu.org.uk.
•• Provides key background on implementing meta-regressions.
4.
Salanti G. Indirect and mixed-treatment comparison, network, or multipletreatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res. Synth. Methods 3(2), 80–97 (2012).
5.
Salanti G, Marinho V, Higgins J. A case study of multiple-treatments meta-analysis demonstrates that covariates should be considered. J. Clin. Epidemiol. 62(8), 857–864 (2009).
• Provides case study of the importance of meta-regression adjustments.
6.
Cameron C, Fireman B, Hutton B et al. Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities. Syst. Rev. 4, 147 (2015).
7.
Donegan S, Williamson P, D'alessandro U, Tudur Smith C. Assessing key assumptions of network meta-analysis: a review of methods. Res. Synth. Methods 4(4), 291–323 (2013).
8.
Thorlund K, Druyts E, Aviña-Zubieta JA, Wu P, Mills EJ. Why the findings of published multiple treatment comparison meta-analyses of biologic treatments for rheumatoid arthritis are different: an overview of recurrent methodological shortcomings. Ann. Rheum. Dis. 72(9), 1524–1535 (2013).
9.
Loos AM, Liu S, Segel C, Ollendorf DA, Pearson SD, Linder JA. Comparative effectiveness of targeted immunomodulators for the treatment of moderate-to-severe plaque psoriasis: a systematic review and network meta-analysis. J. Am. Acad. Dermatol. 79(1), e7 (2018).
• Other key reviews in the area we cover that have considered network meta-analysis (NMA) with adjustments.
10.
Sawyer L, Fotheringham I, Wright E, Yasmeen N, Gibbons C, Holmen Moller A. The comparative efficacy of brodalumab in patients with moderate-to-severe psoriasis: a systematic literature review and network meta-analysis. J. Dermatolog. Treat., 1–12 (2018) (Epub ahead of print).
• Another key review in the area we cover that has considered NMA with adjustments.
11.
Signorovitch JE, Betts KA, Yan YS et al. Comparative efficacy of biological treatments for moderate-to-severe psoriasis: a network meta-analysis adjusting for cross-trial differences in reference arm response. Br. J. Dermatol. 172(2), 504–512 (2015).
• Other key reviews in the area we cover that have considered NMA with adjustments.
12.
Queiro R, Tejón P, Alonso S, Coto P. Age at disease onset: a key factor for understanding psoriatic disease. Rheumatology (Oxford) 53(7), 1178–1185 (2014).
13.
World Health Organization. Global Report on Psoriasis (2016). http://apps.who.int/iris/bitstream/10665/204417/1/9789241565189_eng.pdf.
14.
Di Meglio P, Villanova F, Nestle FO. Psoriasis. Cold Spring Harb. Perspect. Med. 4(8), pii: a015354 (2014).
15.
National Institute for Clinical Excellence in Health. Apremilast for treating moderate to severe psoriasis (2015). www.nice.org.uk/guidance/ta419/documents/committee-papers-2.
16.
National Institute for Clinical Excellence in Health. Ixekizumab for treating moderate to severe plaque psoriasis (2016). www.nice.org.uk/guidance/ta442/documents/committee-papers-3.
17.
National Institute for Clinical Excellence in Health. Secukinumab for the treatment of moderate to severe plaque psoriasis (2015). www.nice.org.uk/guidance/ta350/documents/secukinumab-for-treating-moderate-to-severe-plaque-psoriasis-committee-papers2.
18.
Jabbar-Lopez ZK, Yiu ZZN, Ward V et al. Quantitative evaluation of biologic therapy options for psoriasis: a systematic review and network meta-analysis. J. Invest. Dermatol. 137(8), 1646–1654 (2017).
19.
Sbidian E, Chaimani A, Garcia-Doval I et al. Systemic pharmacological treatments for chronic plaque psoriasis: a network meta-analysis. Cochrane Database Syst. Rev. 12, CD011535 (2017).
20.
Dias S, Welton N, Sutton A, Ades A. NICE DSU Technical Support Document 2: a generalised linear modelling framework for pairwise and network meta-analysis of randomised controlled trials (2011). http://www.nicedsu.org.uk.
21.
Asahina A, Nakagawa H, Etoh T, Ohtsuki M. Adalimumab M04-688 Study Group. Adalimumab in Japanese patients with moderate to severe chronic plaque psoriasis: efficacy and safety results from a Phase II/III randomized controlled study. J. Dermatol. 37(4), 299–310 (2010).
22.
Bachelez H, Van De Kerkhof PC, Strohal R et al. Tofacitinib versus etanercept or placebo in moderate-to-severe chronic plaque psoriasis: a Phase III randomised non-inferiority trial. Lancet 386(9993), 552–561 (2015).
23.
Blauvelt A, Papp KA, Griffiths CE et al. Efficacy and safety of guselkumab, an anti-interleukin-23 monoclonal antibody, compared with adalimumab for the continuous treatment of patients with moderate to severe psoriasis: results from the Phase III, double-blinded, placebo- and active comparator-controlled VOYAGE 1 trial. J. Am. Acad. Dermatol. 76(3), 405–417 (2016).
24.
De Vries AC, Thio HB, De Kort WJ et al. A prospective randomised controlled trial comparing infliximab and etanercept in patients with moderate to severe chronic plaque type psoriasis: the psoriasis infliximab versus etanercept comparison evaluation, the PIECE study. Br. J. Dermatol. 176(3), 624–633 (2016).
25.
Gordon KB, Blauvelt A, Papp KA et al. Phase III trials of ixekizumab in moderate-to-severe plaque psoriasis. N. Engl. J. Med. 375(4), 345–356 (2016).
26.
Gottlieb AB, Leonardi C, Kerdel F, Mehlis S, Olds M, Williams DA. Efficacy and safety of briakinumab vs. etanercept and placebo in patients with moderate to severe chronic plaque psoriasis. Br. J. Dermatol. 165(3), 652–660 (2011).
27.
Griffiths CE, Reich K, Lebwohl M et al. Comparison of ixekizumab with etanercept or placebo in moderate-to-severe psoriasis (UNCOVER-2 and UNCOVER-3): results from two Phase III randomised trials. Lancet 386(9993), 541–551 (2015).
28.
Griffiths CE, Strober BE, Van De Kerkhof P et al. Comparison of ustekinumab and etanercept for moderate-to-severe psoriasis. N. Engl. J. Med. 362(2), 118–128 (2010).
29.
Griffiths CE, Thaci D, Gerdes S et al. The EGALITY study: a confirmatory, randomised, double-blind study comparing the efficacy, safety and immunogenicity of GP2015, a proposed etanercept biosimilar, versus the originator product in patients with moderate to severe chronic plaque-type psoriasis. Br. J. Dermatol. 176(4), 928–938 (2016).
30.
Igarashi A, Kato T, Kato M, Song M, Nakagawa H, Japanese Ustekinumab Study Group. Efficacy and safety of ustekinumab in Japanese patients with moderate-to-severe plaque-type psoriasis: long-term results from a Phase II/III clinical trial. J. Dermatol. 39(3), 242–252 (2012).
31.
Langley RG, Elewski BE, Lebwohl M et al. Secukinumab in plaque psoriasis-results of two Phase III trials. N. Engl. J. Med. 371(4), 326–338 (2014).
32.
Lebwohl M, Strober B, Menter A et al. Phase III studies comparing brodalumab with ustekinumab in psoriasis. N. Engl. J. Med. 373(14), 1318–1328 (2015).
33.
Leonardi CL, Kimball AB, Papp KA et al. Efficacy and safety of ustekinumab, a human interleukin-12/23 monoclonal antibody, in patients with psoriasis: 76-week results from a randomised, double-blind, placebo-controlled trial (PHOENIX 1). [Erratum appears in Lancet 371(9627), 1838 (2008)]. Lancet 371(9625), 1665–1674 (2008).
34.
Leonardi CL, Powers JL, Matheson RT et al. Etanercept as monotherapy in patients with psoriasis. N. Engl. J. Med. 349(21), 2014–2022 (2003).
35.
Menter A, Feldman SR, Weinstein GD et al. A randomized comparison of continuous vs. intermittent infliximab maintenance regimens over 1 year in the treatment of moderate-to-severe plaque psoriasis. J. Am. Acad. Dermatol. 56(1), e1 (2007).
36.
Menter A, Tyring SK, Gordon K et al. Adalimumab therapy for moderate to severe psoriasis: a randomized, controlled Phase III trial. J. Am. Acad. Dermatol. 58(1), 106–115 (2008).
37.
Mrowietz U, Leonardi CL, Girolomoni G et al. Secukinumab retreatment-as-needed versus fixed-interval maintenance regimen for moderate to severe plaque psoriasis: a randomized, double-blind, noninferiority trial (SCULPTURE). J. Am. Acad. Dermatol. 73(1), e1 (2015).
38.
Papp K, Reich K, Leonardi CL et al. Apremilast, an oral phosphodiesterase 4 (PDE4) inhibitor, in patients with moderate to severe plaque psoriasis: results of a Phase III, randomized, controlled trial (Efficacy and Safety Trial Evaluating the Effects of Apremilast in Psoriasis [ESTEEM] 1). J. Am. Acad. Dermatol. 73(1), 37–49 (2015).
39.
Papp KA, Langley RG, Lebwohl M et al. Efficacy and safety of ustekinumab, a human interleukin-12/23 monoclonal antibody, in patients with psoriasis: 52-week results from a randomised, double-blind, placebo-controlled trial (PHOENIX 2). Lancet 371(9625), 1675–1684 (2008).
40.
Papp KA, Reich K, Paul C et al. A prospective Phase III, randomized, double-blind, placebo-controlled study of brodalumab in patients with moderate-to-severe plaque psoriasis. Br. J. Dermatol. 175(2), 273–286 (2016).
41.
Papp KA, Tyring S, Lahfa M et al. A global Phase III randomized controlled trial of etanercept in psoriasis: safety, efficacy, and effect of dose reduction. Br. J. Dermatol. 152(6), 1304–1312 (2005).
42.
Paul C, Cather J, Gooderham M et al. Efficacy and safety of apremilast, an oral phosphodiesterase 4 inhibitor, in patients with moderate-to-severe plaque psoriasis over 52 weeks: a Phase III, randomized controlled trial (ESTEEM 2). Br. J. Dermatol. 173(6), 1387–1399 (2015).
43.
Paul C, Lacour JP, Tedremets L et al. Efficacy, safety and usability of secukinumab administration by autoinjector/pen in psoriasis: a randomized, controlled trial (JUNCTURE). J. Eur. Acad. Dermatol. Venereol. 29(6), 1082–1090 (2015).
44.
Reich K, Armstrong AW, Foley P et al. Efficacy and safety of guselkumab, an anti-interleukin-23 monoclonal antibody, compared with adalimumab for the treatment of patients with moderate to severe psoriasis with randomized withdrawal and retreatment: results from the Phase III, double-blind, placebo- and active comparator-controlled VOYAGE 2 trial. J. Am. Acad. Dermatol. 76(3), 418–431 (2016).
45.
Reich K, Gooderham M, Green L et al. The efficacy and safety of apremilast, etanercept and placebo in patients with moderate-to-severe plaque psoriasis: 52-week results from a Phase IIIb, randomized, placebo-controlled trial (LIBERATE). J. Eur. Acad. Dermatol. Venereol. 31(3), 507–517 (2016).
46.
Reich K, Nestle FO, Papp K et al. Infliximab induction and maintenance therapy for moderate-to-severe psoriasis: a Phase III, multicentre, double-blind trial. Lancet 366(9494), 1367–1374 (2005).
47.
Saurat JH, Stingl G, Dubertret L et al. Efficacy and safety results from the randomized controlled comparative study of adalimumab vs. methotrexate vs. placebo in patients with psoriasis (CHAMPION). Br. J. Dermatol. 158(3), 558–566 (2008).
48.
Strober BE, Crowley JJ, Yamauchi PS, Olds M, Williams DA. Efficacy and safety results from a Phase III, randomized controlled trial comparing the safety and efficacy of briakinumab with etanercept and placebo in patients with moderate to severe chronic plaque psoriasis. Br. J. Dermatol. 165(3), 661–668 (2011).
49.
Strohal R, Puig L, Chouela E et al. The efficacy and safety of etanercept when used with as-needed adjunctive topical therapy in a randomised, double-blind study in subjects with moderate-to-severe psoriasis (the PRISTINE trial). J. Dermatolog. Treat. 24(3), 169–178 (2013).
50.
Thaci D, Blauvelt A, Reich K et al. Secukinumab is superior to ustekinumab in clearing skin of subjects with moderate to severe plaque psoriasis: CLEAR, a randomized controlled trial. J. Am. Acad. Dermatol. 73(3), 400–409 (2015).
51.
Torii H, Nakagawa H, Japanese Infliximab Study investigators. Infliximab monotherapy in Japanese patients with moderate-to-severe plaque psoriasis and psoriatic arthritis. A randomized, double-blind, placebo-controlled multicenter trial. J. Dermatol. Sci. 59(1), 40–49 (2010).
52.
Tsai TF, Ho JC, Song M et al. Efficacy and safety of ustekinumab for the treatment of moderate-to-severe psoriasis: a Phase III, randomized, placebo-controlled trial in Taiwanese and Korean patients (PEARL). J. Dermatol. Sci. 63(3), 154–163 (2011).
53.
Tyring S, Gottlieb A, Papp K et al. Etanercept and clinical outcomes, fatigue, and depression in psoriasis: double-blind placebo-controlled randomised Phase III trial. Lancet 367(9504), 29–35 (2006).
54.
Van De Kerkhof PC, Segaert S, Lahfa M et al. Once weekly administration of etanercept 50 mg is efficacious and well tolerated in patients with moderate-to-severe plaque psoriasis: a randomized controlled trial with open-label extension. Br. J. Dermatol. 159(5), 1177–1185 (2008).
55.
Yang HZ, Wang K, Jin HZ et al. Infliximab monotherapy for Chinese patients with moderate to severe plaque psoriasis: a randomized, double-blind, placebo-controlled multicenter trial. Chin. Med. J. 125(11), 1845–1851 (2012).
56.
Zhu X, Zheng M, Song M et al. Efficacy and safety of ustekinumab in Chinese patients with moderate to severe plaque-type psoriasis: results from a Phase III clinical trial (LOTUS). J. Drugs Dermatol. 12(2), 166–174 (2013).
57.
Blauvelt A, Prinz JC, Gottlieb AB et al. Secukinumab administration by pre-filled syringe: efficacy, safety and usability results from a randomized controlled trial in psoriasis (FEATURE). Br. J. Dermatol. 172(2), 484–493 (2015).
58.
Cai L, Gu J, Zheng J et al. Efficacy and safety of adalimumab in Chinese patients with moderate-to-severe plaque psoriasis: results from a Phase III, randomized, placebo-controlled, double-blind study. J. Eur. Acad. Dermatol. Venereol. 31(1), 89–95 (2017).
59.
Torres T, Puig L. Treatment goals for psoriasis: Should PASI 90 become the standard of care? Acta Dermosifiliogr. 106(3), 155–157 (2015).
60.
Higgins JP, Thompson SG. Controlling the risk of spurious findings from meta-regression. Stat. Med. 23(11), 1663–1682 (2004).
61.
Thompson SG, Higgins JP. How should meta-regression analyses be undertaken and interpreted? Stat. Med. 21(11), 1559–1573 (2002).
62.
Warn DE, Thompson SG, Spiegelhalter DJ. Bayesian random effects meta-analysis of trials with binary outcomes: methods for the absolute risk difference and relative risk scales. Stat. Med. 21(11), 1601–1623 (2002).
63.
Nikolakopoulou A, Chaimani A, Veroniki AA, Vasiliadis HS, Schmid CH, Salanti G. Characteristics of networks of interventions: a description of a database of 186 published networks. PLoS ONE 9(1), e86754 (2014).
64.
Gómez-García F, Epstein D, Isla-Tejera B, Lorente A, Vélez García-Nieto A, Ruano J. Short-term efficacy and safety of new biological agents targeting the interleukin-23-T helper 17 pathway for moderate-to-severe plaque psoriasis: a systematic review and network meta-analysis. Br. J. Dermatol. 176(3), 594–603 (2017).
65.
Gupta S, Faughnan ME, Tomlinson GA, Bayoumi AM. A framework for applying unfamiliar trial designs in studies of rare diseases. J. Clin. Epidemiol. 64(10), 1085–1094 (2011).
66.
Hartz S. Network meta-analysis to evaluate the efficacy of ixekizumab in the treatment of moderate-to-severe psoriasis. ISPOR 19th Annual European Congress. Vienna, Austria, 29 October–2 November (2016).
67.
Lin VW, Ringold S, Devine EB. Comparison of ustekinumab with other biological agents for the treatment of moderate to severe plaque psoriasis: a Bayesian network meta-analysis. Arch. Dermatol. 148(12), 1403–1410 (2012).
68.
Odom D, Brogan A, Talbird SE, Schenkel B. A network meta-analysis of randomized, controlled trials of ustekinumab and adalimumab for moderate-to-severe psoriasis. Value Health 16, A112 (2013).
69.
Peruzzi M, Colombo D, De Falco E, Chimenti I, Abbate A et al. Biologic therapy for psoriatic arthritis or moderate to severe plaque psoriasis: systematic review with pairwise and network meta-analysis. Int. J. Stat. Med. Res. 3, 74–87 (2014).
70.
National Institute for Clinical Excellence in Health. Final appraisal determination. Brodalumab for treating moderate to severe plaque psoriasis (2018). www.nice.org.uk/guidance/gid-ta10220/documents/final-appraisal-determination-document.
71.
Batson S, Score R, Sutton AJ. Three-dimensional evidence network plot system: covariate imbalances and effects in network meta-analysis explored using a new software tool. J. Clin. Epidemiol. 86, 182–195 (2017).
Information & Authors
Information
Published In
Copyright
© 2018 Janssen Inc.
History
Received: 9 July 2018
Accepted: 17 August 2018
Published online: 2 October 2018
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Importance of assessing and adjusting for cross-study heterogeneity in network meta-analysis: a case study of psoriasis. (2018) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2018-0065
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Yuning Zhao, Guangxin Luo, Can Huang, Zhenyang Chen, Yu Wei, Qiaosen Chen, Fang Wang, Yong Gan, Xiaoxv Yin, Comparative effects of some pharmacological and non-pharmacological interventions on cognitive function in Alzheimer’s disease: A Bayesian network meta-analysis, The Journal of Prevention of Alzheimer's Disease, 10.1016/j.tjpad.2026.100564, 13, 6, (100564), (2026).
- Keya Joshi, Aaron Situ, Wei-Jhih Wang, Mihaela Georgieva, Nicolas van de Velde, Ekkehard Beck, Letter to the Editor: comparative effectiveness of omicron XBB 1.5-adapted COVID-19 vaccines: a systematic literature review and network meta-analysis, Expert Review of Vaccines, 10.1080/14760584.2025.2554689, 24, 1, (870-872), (2025).
- Mark G. Lebwohl, André Carvalho, Akihiko Asahina, Jianzhong Zhang, Mir Sohail Fazeli, Ellen Kasireddy, Paul Serafini, Thomas Ferro, Ranga Gogineni, Diamant Thaçi, Biologics for the Treatment of Moderate-to-Severe Plaque Psoriasis: A Systematic Review and Network Meta-analysis, Dermatology and Therapy, 10.1007/s13555-025-01423-0, 15, 7, (1633-1656), (2025).
- Nils Erik Gilhus, Saiju Jacob, Mahmoud Hashim, Suzy Van Sanden, Christopher Drudge, Anna Nero, Sumeet Singh, Kavita Gandhi, Brian Hutton, Network connectivity, between-study heterogeneity and timepoint challenges in generalized myasthenia gravis: a feasibility assessment of indirect treatment comparisons, Journal of Comparative Effectiveness Research, 10.57264/cer-2025-0009, 14, 6, (2025).
- Nicholas Riley, Christopher Drudge, Morag Nelson, Anja Haltner, Michael Barnett, Simon Broadley, Helmut Butzkueven, Pamela McCombe, Anneke Van der Walt, Erin O. Y. Wong, Martin Merschhemke, Nicholas Adlard, Rob Walker, Imtiaz A. Samjoo, Comparative efficacy of ofatumumab versus oral therapies for relapsing multiple sclerosis patients using propensity score analyses and simulated treatment comparisons , Therapeutic Advances in Neurological Disorders, 10.1177/17562864241239453, 17, (2024).
- Carole Lunny, Areti-angeliki Veroniki, Julian P. T. Higgins, Sofia Dias, Brian Hutton, James M. Wright, Ian R. White, Penny Whiting, Andrea C. Tricco, Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool), Systematic Reviews, 10.1186/s13643-023-02388-x, 13, 1, (2024).
- April W. Armstrong, Richard B. Warren, Yichen Zhong, Joe Zhuo, Allie Cichewicz, Ananth Kadambi, Daniela Junqueira, Tracy Westley, Renata Kisa, Carolin Daamen, Matthias Augustin, Short-, Mid-, and Long-Term Efficacy of Deucravacitinib Versus Biologics and Nonbiologics for Plaque Psoriasis: A Network Meta-Analysis, Dermatology and Therapy, 10.1007/s13555-023-01034-7, 13, 11, (2839-2857), (2023).
- Christopher Drudge, Imtiaz A Samjoo, Róisín Brennan, Lohit Badgujar, Vivek Khurana, Santosh Tiwari, Nicholas Adlard, Judit Banhazi, An overview of reviews with network meta-analyses comparing disease-modifying therapies for relapsing multiple sclerosis, Future Neurology, 10.2217/fnl-2023-0009, 18, 3, (2023).
- Herman Sehmbi, Susanne Retter, Ushma J. Shah, Derek Nguyen, Janet Martin, Vishal Uppal, Epidemiological, methodological, and statistical characteristics of network meta-analysis in anaesthesia: a systematic review, British Journal of Anaesthesia, 10.1016/j.bja.2022.08.042, 130, 3, (272-286), (2023).
- Dan Pettitt, Michael Plotnick, Joshua Gagne, Letter to the Editor Regarding Long-Term Benefit–Risk Profiles of Treatments for Moderate-to-Severe Plaque Psoriasis: A Network Meta-analysis, Dermatology and Therapy, 10.1007/s13555-022-00809-8, 12, 12, (2863-2866), (2022).