Assessing the robustness of direct meta-analysis in the presence of heterogeneity
Abstract
Aim: To assess robustness of findings in recent systematic reviews of atypical antipsychotics in schizophrenia patients in the presence of heterogeneity. Methods: Relative efficacy was measured in seven direct comparisons of mean difference (MD) in average positive and negative syndrome scale total score and comparisons between conventional meta-analysis results and four alternate meta-analytic strategies using the difference in MD (ΔMD). Results: MDs in positive and negative syndrome scale total score were smaller in the conventional meta-analyses than those for three of the four analytic strategies. Such differences were small (all ΔMD <4). No considerable differences in effect size were observed in the limit meta-analysis comparison (|ΔMD| <1). Conclusion: The analyses validated the systematic review results and demonstrate the value of confirmatory sensitivity analysis.
Supplementary Material
File (supplementarymaterial.docx)
- Download
- 1.47 MB
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
WHO. Schizophrenia (2015). www.who.int/mental_health/management/schizophrenia/en/.
2.
Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
• Reports the Cochrane systematic review of aripiprazole versus comparators upon which this analysis was conducted.
3.
Asmal L, Flegar SJ, Wang J, Rummel-Kluge C, Komossa K, Leucht S. Quetiapine versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 11, CD006625 (2013).
• Reports the Cochrane systematic review of quetiapine versus risperidone upon which this analysis was conducted.
4.
Komossa K, Rummel-Kluge C, Schwarz S et al. Risperidone versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 19(1), CD006626 (2011).
• Reports the Cochrane systematic review of risperidone versus olanzapine upon which this analysis was conducted.
5.
Leucht S, Corves C, Arbter D, Engel RR, Li C, Davis JM. Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. Lancet 373(9657), 31–41 (2009).
6.
Citrome L. A systematic review of meta-analyses of the efficacy of oral atypical antipsychotics for the treatment of adult patients with schizophrenia. Expert Opin. Pharmacother. 13(11), 1545–1573 (2012).
7.
Leucht S, Tardy M, Komossa K et al. Antipsychotic drugs versus placebo for relapse prevention in schizophrenia: a systematic review and meta-analysis. Lancet 379(9831), 2063–2071 (2012).
8.
Geddes J, Freemantle N, Harrison P, Bebbington P. Atypical antipsychotics in the treatment of schizophrenia: systematic overview and meta-regression analysis. BMJ 321(7273), 1371–1376 (2000).
9.
Bartoli F, Crocamo C, Clerici M, Carra G. Second-generation antipsychotics and adiponectin levels in schizophrenia: a comparative meta-analysis. Eur. J. Neuropsychopharmacol. 25(10), 1767–1774 (2015).
10.
Deeks JJ, Altman DG, Bradburn MJ. Statistical methods for examining heterogeneity and combining results from several studies in meta-analysis. In: Systematic Reviews in Health Care: Meta-Analysis in Context. Egger M, Davey Smith G, Altman DG (Eds). BMJ Books, London, UK, 285–311 (2001).
11.
Thompson SG. Why sources of heterogeneity in meta-analysis should be investigated. BMJ 309(6965), 1351–1355 (1994).
12.
Eysenck HJ. Problems with meta-analysis. In: Systematic Reviews. Chalmers I, Altman DG (Eds). BMJ Publishing Group, London, UK, 64–74 (1995).
13.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Heterogeneity: overview. In: Introduction to Meta-Analysis. John Wiley & Sons, Ltd, Chichester, UK, 105–106 (2009).
14.
Cochrane Handbook for Systematic Reviews of Interventions. Higgins JPT, Green S (Eds). Version 5.1.0 [updated March 2011] (2011). www.cochrane-handbook.org.
15.
Fletcher J. What is heterogeneity and is it important? BMJ 334(7584), 94–96 (2007).
16.
Higgins JPT, Altman DG, Gøtzsche PC et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 343, d5928 (2011).
17.
Dechartres A, Altman DG, Trinquart L, Boutron I, Ravaud P. Association between analytic strategy and estimates of treatment outcomes in meta-analyses. JAMA 312(6), 623–630 (2014).
•• Presents the sensitivity approaches applied in this study. Reports general results across a large sample of meta-analyses.
18.
Egger M, Dickersin K, Davey Smith G. Problems and limitations in conducting systematic reviews. In: Systematic Reviews in Health Care: Meta-analysis in Context. Egger M, Davey Smith G, Altman DG (Eds). BMJ Books, London, UK, 43–68 (2001).
19.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Criticisms of meta-analysis. In: Introduction to Meta-Analysis. John Wiley & Sons, Ltd, Chichester, UK, 377–387 (2009).
20.
Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Publication bias. In: Introduction to Meta-Analysis. John Wiley & Sons, Ltd, Chichester, UK, 277–292 (2009).
21.
Sterne JAC, Egger M, Davey Smith G. Investigating and dealing with publication and other biases. In: Systematic Reviews in Health Care: Meta-Analysis in Context. Egger M, Davey Smith G, Altman DG (Eds). BMJ Books, London, UK, 189–208 (2001).
22.
Schwarzer G, Carpenter JR, Rucker G. Small-study effects in meta-analysis. In: Meta-Analysis With R. Springer International Publishing, Cham, Switzerland, 107–141 (2015).
• Describes the implementation of limit meta-analysis in R, as applied in this study.
23.
Rucker G, Carpenter JR, Schwarzer G. Detecting and adjusting for small-study effects in meta-analysis. Biomed. J. 53(2), 351–368 (2011).
•• Details the methodology underpinning the limit meta-analysis applied in the current study.
24.
Asenjo Lobos C, Komossa K, Rummel-Kluge C et al. Clozapine versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 10(11), CD006633 (2010).
25.
Komossa K, Rummel-Kluge C, Hunger H et al. Olanzapine versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 17(3), CD006654 (2010).
26.
Komossa K, Rummel-Kluge C, Hunger H et al. Ziprasidone versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 7(4), CD006627 (2009).
27.
Kay SR, Opler LA, Fiszbein A. Positive and Negative Syndrome Scale (PANSS) Manual. Multi-Health Systems, NY, USA (1986).
28.
R Development Core Team. R version 3.0.1: a language and environment for statistical computing (2013). http://CRAN.R-project.org.
29.
Glasziou PP, Shepperd S, Brassey J. Can we rely on the best trial? A comparison of individual trials and systematic reviews. BMC Med. Res. Methodol. 10, 23 (2010).
30.
Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ 346, f2304 (2013).
31.
Rucker G, Schwarzer G, Carpenter JR, Binder H, Schumacher M. Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 12(1), 122–142 (2011).
32.
Schwarzer G, Carpenter JR, Rucker G. Advanced statistical methods to model and adjust for bias in meta-analysis (2014). http://cran.r-project.org/web/packages/metasens/metasens.pdf.
33.
Haddad PM, Brain C, Scott J. Nonadherence with antipsychotic medication in schizophrenia: challenges and management strategies. Patient Relat. Outcome Meas. 5, 43–62 (2014).
34.
Acosta FJ, Hernández JL, Pereira J, Herrera J, Rodríguez CJ. Medication adherence in schizophrenia. World J. Psych. 2(5), 74–82 (2012).
35.
Furukawa TA, Levine SZ, Tanaka S et al. Initial severity of schizophrenia and efficacy of antipsychotics: participant-level meta-analysis of 6 placebo-controlled studies. JAMA Psych. 72(1), 14–21 (2015).
36.
Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 327(7414), 557–560 (2003).
37.
Liu S, Ren C, Song X. Comparative study between aripiprazole and clozapine in the treatment of first-episode schizophrenia. Chin. J. Health Psychol. 14(6), 658–659 (2006). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
38.
Zhang Q-l, Zhai J. Aripiprazole and clozapine in treatment refractory schizophrenia. West China Med. J. 25(6), 1033–1035 (2010). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
39.
Zhang Y, Yu Z. Aripiprazole and clozapine in the treatment refractory schizophrenia comparative observation. People's Military Surgeon 52(12), 822–823 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
40.
Yuan Cl. Quetiapine and aripiprazole in the treatment of first episode schizophrenia. Chinese Modern Medicine Application 2(3), 71 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
41.
Wei L, Zhu H, Zhao Q, Quan T. Aripiprazole and quetiapine in the treatment of schizophrenia. Shandong Archive of Psychiatry 22(1), 42–43 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
42.
Zhang D, Gan JL, Gao C, Duan H. Aripiprazole and risperidone on glucose and lipid. Linchuang Jinghen Yixue Zazhi 19(5), 343–345 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
43.
Luo L. Aripiprazole and risperidone in the treatment of schizophrenia efficacy analysis. Contemporary Medicine 15(170), 101–102 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
44.
Chen Y, Huang W, Ye X. Comparative study between aripiprazole and olanzapine in the treatment of agitated schizophrenia. Linchuang Jingshen Tizue Zazhi 19(4), 261–262 (2009). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
45.
Zimbroff D, Warrington L, Loebel A, Yang R, Siu C. Comparison of ziprasidone and aripiprazole in acutely ill patients with schizophrenia or schizoaffective disorder: a randomized, double-blind, 4-week study. Int. Clin. Psychopharmacol. 22(6), 363–370 (2007). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
46.
Potkin SG, Gharabawi GM, Greenspan AJ et al. A double-blind comparison of risperidone, quetiapine and placebo in patients with schizophrenia experiencing an acute exacerbation requiring hospitalization. Schizophr. Res. 85(1-3), 254–265 (2006). Cited in: Asmal L, Flegar SJ, Wang J, Rummel-Kluge C, Komossa K, Leucht S. Quetiapine versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 11, CD006625 (2013).
47.
Zhong KX, Sweitzer DE, Hamer RM, Lieberman JA. Comparison of quetiapine and risperidone in the treatment of schizophrenia: A randomized, double-blind, flexible-dose, 8-week study. J. Clin. Psych. 67(7), 1093–1103 (2006). Cited in: Asmal L, Flegar SJ, Wang J, Rummel-Kluge C, Komossa K, Leucht S. Quetiapine versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 11, CD006625 (2013).
48.
Atmaca M, Kuloglu M, Tezcan E, Ustundag B. Serum leptin and triglyceride levels in patients on treatment with atypical antipsychotics. J. Clin. Psych. 64(5), 598–604 (2003). Cited in: Komossa K, Rummel-Kluge C, Schwarz S et al. Risperidone versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. CD006626 (2011).
49.
Kilburg A, Galani Berardo C, Souto D, Llorca PM. Outcomes in schizophrenia: what does ‘clinically meaningful’ mean to payers? Value Health 17(3), A57 (2014).
50.
Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel RR. What does the PANSS mean? Schizophr. Res. 79(2–3), 231–238 (2005).
51.
Hermes EDA, Sokoloff DM, Stroup TS, Rosenheck RA. Minimum clinically important difference in the positive and negative syndrome scale using data from the CATIE schizophrenia trial. J. Clin. Psych. 73(4), 526–532 (2012).
52.
Leddy-Stacy MA, Rosenheck R. Obtaining employment as an anchor for estimating the minimum clinically important difference on the Positive and Negative Syndrome Scale (PANSS) in schizophrenia. Psychiatry Res. 238, 304–309 (2016).
53.
Meltzer HY, Risinger R, Nasrallah HA et al. A randomized, double-blind, placebo-controlled trial of aripiprazole lauroxil in acute exacerbation of schizophrenia. J. Clin. Psych. 76(8), 1085–1090 (2015).
54.
Kane JM, Potkin SG, Daniel DG, Buckley PF. A double-blind, randomized study comparing the efficacy and safety of sertindole and risperidone in patients with treatment-resistant schizophrenia. J. Clin. Psych. 72(2), 194–204 (2011).
55.
Shah S, Joshi D. Tolerability and efficacy of paliperidone ER compared to olanzapine in the treatment of schizophrenia: a randomized, double–blind, multicentric trial. Ind. Psychiatry J. 20(1), 25–31 (2011).
56.
Hoffmann TC, Erueti C, Glasziou PP. Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials. BMJ 347, f3755 (2013).
57.
Bristol-Myers Squibb. A multicenter, double-blind, randomized, comparative study of aripiprazole and olanzapine in the treatment of patients with acute schizophrenia. Clinical Study Report CN138003 (2005). Cited in: Khanna P, Suo T, Komossa K et al. Aripiprazole versus other atypical antipsychotics for schizophrenia. Cochrane Database Syst. Rev. 1, CD006569 (2014).
58.
Stafford MR, Mayo-Wilson E, Loucas CE et al. Efficacy and safety of pharmacological and psychological interventions for the treatment of psychosis and schizophrenia in children, adolescents and young adults: a systematic review and meta-analysis. PLoS ONE 10(2), e0117166 (2015).
59.
Hrdlicka M, Dudova I. Atypical antipsychotics in the treatment of early-onset schizophrenia. Neuropsychiatr. Dis. Treat. 11, 907–913 (2015).
60.
Suzuki T, Remington G, Arenovich T et al. Time course of improvement with antipsychotic medication in treatment-resistant schizophrenia. Br. J. Psych. 199(4), 275–280 (2011).
61.
Sherwood M, Thornton AE, Honer WG. A meta-analysis of profile and time-course of symptom change in acute schizophrenia treated with atypical antipsychotics. Int. J. Neuropsychopharmacol. 9(3), 357–366 (2006).
62.
Goldacre B. Missing data. In: Bad Pharma: How Medicine is Broken, and How We Can Fix it. Fourth Estate, London, UK, 1–100 (2013).
Information & Authors
Information
Published In
Copyright
© 2018 Future Medicine Ltd.
History
Received: 12 March 2018
Accepted: 30 July 2018
Published online: 22 August 2018
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Assessing the robustness of direct meta-analysis in the presence of heterogeneity. (2018) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2018-0024
Export citation
Select the citation format you wish to export for this article or chapter.