Skip to main content
Search
Search
Advanced Search
Skip main navigation
Open access
Methodology
29 June 2026

Considerations for assessing the feasibility of network meta-analysis of seasonal vaccines

Authors: Allie Cichewicz https://orcid.org/0000-0002-8528-9901 [email protected], Marissa Betts https://orcid.org/0000-0002-0557-3946, Narcisa Mesaros https://orcid.org/0000-0003-3992-9549, and Heather Burnett https://orcid.org/0000-0003-1269-4276Author Info & Affiliations
Volume 15, Number 7
https://doi.org/10.57264/cer-2026-0014
PDFFormats

Abstract

Aim: Network meta-analyses (NMAs) of seasonal vaccines face distinct challenges that can compromise the validity and relevance of findings. While established frameworks offer guidance for evaluating the feasibility of NMAs, they do not address factors specific to seasonal vaccines. This study aims to highlight unique methodological challenges related to conducting NMA feasibility assessments of seasonal vaccines. The considerations are framed to be compatible with existing guidance and recommendations for the conduct and reporting of NMAs. Materials & methods: We developed a set of key considerations that should be applied when assessing the feasibility and/or validity of NMAs comparing seasonal vaccines. The considerations were based on systematic reviews and critical appraisals of published NMAs of seasonal vaccines, hands-on experience performing feasibility assessments of seasonal vaccines, and input from consultations with vaccine experts. Results: Unique considerations for evaluating comparability across seasonal vaccine studies include: whether vaccines should be compared by platform, formulation, dose, and/or valence; the impact of seasonality, strain evolution and definitions of placebo/unvaccinated controls on network connectivity; target population characteristics including history and recency of prior vaccination and/or infection(s), and baseline infection/severe disease risk; antigenic match (i.e., the degree of concordance between vaccine composition and circulating viral strains), which directly influences effectiveness and outcome measurement approaches that consider time varying epidemiology and assay and measure discrepancy. Comprehensively integrating these elements into existing guidance frameworks ensures transparent assessment of the key assumptions underlying NMA (i.e., transitivity and homogeneity) within the context of unique study design and methodological features of seasonal vaccine studies. Conclusion: The concepts highlighted in this paper address important gaps in the feasibility assessment process for NMAs of seasonal vaccines, which are crucial for informing public health decisions and guiding vaccine policy and implementation.

Plain language summary: Considerations for comparing seasonal vaccines using network meta-analysis

What is this article about?

Existing network meta-analysis (NMA) frameworks were expanded to highlight considerations for NMAs of seasonal vaccines, such as those for COVID-19 and influenza, which require annual updates and evaluation across multiple formulations, platforms and populations.

What methodology is described?

Specific considerations related to challenges that are unique to NMAs of seasonal vaccine studies, including strain evolution, antigenic match, time varying epidemiology, assays and measures discrepancy, are considered alongside the potential need to rely on real-world evidence for an up-to-date understanding of vaccine efficacy/effectiveness.

Why is this important?

Currently, there is no existing guidance addressing the specific methodological challenges of conducting NMAs for seasonal vaccines. This paper provides researchers with key considerations to ensure their methodological approaches are robust, their analytical decisions are transparent, and their NMAs produce valid comparative evidence for informing vaccine policy and clinical practice.

Shareable abstract

Discusses key considerations for conducting network meta-analyses of seasonal vaccines by highlighting methodological challenges like antigenic match and population heterogeneity to fill a critical gap in guidance to ensure robust, valid comparative evidence for decision making.

Supplementary Material

File (supplementary materials.docx)

References

Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Igarashi A, Tanaka S, De Moor R et al. Indirect treatment comparisons in healthcare decision making: a targeted review of regulatory approval, reimbursement, and pricing recommendations globally for oncology drugs in 2021–2023. Adv. Ther. 42(1), 52–69 (2025).
PubMed
Google Scholar
2.
National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (2022). (Accessed 10 May 2026). https://www.nice.org.uk/process/pmg36/resources/nice-technology-appraisal-and-highly-specialised-technologies-guidance-the-manual-pdf-72286779244741
Google Scholar
3.
Hoaglin DC, Hawkins N, Jansen JP et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health 14(4), 429–437 (2011).
PubMed
Google Scholar
4.
Ahn E, Kang H. Concepts and emerging issues of network meta-analysis. Korean J. Anesthesiol. 74(5), 371–382 (2021).
PubMed
Google Scholar
5.
Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU technical support document 3: heterogeneity: subgroups, meta-regression, bias and bias-adjustment (2011). (Accessed 11 May 2025). http://www.nicedsu.org.uk/
Google Scholar
•• This document by The National Institute for Health and Care Excellence (NICE) Decision Support Unit provides core methodological guidance on assessment of whether heterogeneity and bias from effect modifiers can be explained or adjusted for sufficiently for valid network meta-analyses (NMAs) to inform medical decision-making.
6.
Member State Coordination Group on Health Technology Assessment (HTA CG). Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons (2024). (Accessed 18 January 2026). https://health.ec.europa.eu/document/download/4ec8288e-6d15-49c5-a490-d8ad7748578f_en?filename=hta_methodological-guideline_direct-indirect-comparisons_en.pdf
Google Scholar
• This guidance document from the Member State Coordination Group on Health Technology Assessment (HTA) describes the approach for conducting NMAs for HTA using direct and indirect evidence.
7.
Cope S, Zhang J, Saletan S, Smiechowski B, Jansen JP, Schmid P. A process for assessing the feasibility of a network meta-analysis: a case study of everolimus in combination with hormonal therapy versus chemotherapy for advanced breast cancer. BMC Med. 12, 93 (2014).
PubMed
Google Scholar
•• This seminal paper establishes the systematic methodology for NMA feasibility assessment, detailing procedures for visualizing treatment/outcome heterogeneity and operationalizing transitivity evaluation to determine if indirect comparisons are methodologically valid.
8.
Graham C, Barker E, Moss J et al. Opportunities for and challenges of conducting indirect treatment comparisons and meta-analyses for vaccines in post-EU HTA regulation era. JMAHP 13(2), 31 (2025).
Google Scholar
9.
Ainslie KEC, Haber M, Orenstein WA. Challenges in estimating influenza vaccine effectiveness. Expert Rev. Vaccines 18(6), 615–628 (2019).
PubMed
Google Scholar
• This expert review contextualizes the unique challenges and considerations for seasonal respiratory virus vaccines within the influenza vaccine landscape, providing essential background for adapting feasibility assessment frameworks to this therapeutic area.
10.
Beck E, Georgieva M, Wang W-J et al. Indirect comparison of the relative vaccine effectiveness of mRNA-1283 vs. BNT162b2 vaccines against symptomatic COVID-19 among US adults. Curr. Med. Res. Opin. 41(4), 721–732 (2025).
PubMed
Google Scholar
11.
Fahrbach K, Cichewicz A, Chu H et al. Comparative effectiveness of Omicron XBB 1.5-adapted COVID-19 vaccines: a systematic literature review and network meta-analysis. Expert Rev. Vaccines 24(1), 416–432 (2025).
PubMed
Google Scholar
12.
Ferdinands JM, Blanton LH, Alyanak E et al. Protection against influenza hospitalizations from enhanced influenza vaccines among older adults: a systematic review and network meta-analysis. J. Am. Geriatr. Soc. 72(12), 3875–3889 (2024).
PubMed
Google Scholar
13.
Veroniki AA, Thirugnanasampanthar SS, Konstantinidis M et al. Trivalent and quadrivalent seasonal influenza vaccine in adults aged 60 and older: a systematic review and network meta-analysis. BMJ Evid. Based Med. 29(4), 239–254 (2024).
PubMed
Google Scholar
14.
McGirr A, Widenmaier R, Burnett H et al. Network meta-analysis in the evaluation of vaccines. Value Health 20(9), A777 (2017).
PubMed
Google Scholar
15.
Institute for Clinical and Economic Review. Vaccines for Covid-19: draft background and scope (2025). (Accessed 11 May 2026). https://icer.org/wp-content/uploads/2025/11/ICER_Covid-19_Draft-Scope_For-Publication_111025.pdf?
Google Scholar
16.
Rind DM, McQueen RB, Zemplenyi A et al. Vaccines for Covid-19: effectiveness and value; draft evidence report. Inst. Clin.Econ. Rev. (2026). (Accessed 9 May 2026). https://icer.org/assessment/covid-19-2025/
Google Scholar
17.
Haute Autorité de Santé. EFLUELDA (vaccin grippal trivalent inactivé haute dose): immunisation active des personnes âgées de 60 ans et plus pour lesquelles la vaccination grippale est recommandée. Haute Autorité de Santé (2025). Available from: https://www.has-sante.fr/upload/docs/application/pdf/2025-05/efluelda_avis_economique_ceesp_795.pdf
Google Scholar
18.
World Health Organization. Evaluation of COVID-19 vaccine effectiveness in a changing landscape of COVID-19 epidemiology and vaccination (2022). https://www.who.int/publications/i/item/WHO-2019-nCoV-vaccine_effectiveness-VE_evaluations-2022.1
Google Scholar
• This guidance document by the WHO addresses methodological considerations for evaluating vaccine effectiveness amid evolving variants and waning immunity, providing frameworks for assessing temporal comparability and transitivity assumptions in dynamic epidemiological contexts.
19.
Roper LE, Godfrey M, Link-Gelles R et al. Use of additional doses of 2024–2025 COVID-19 vaccine for adults aged ≥65 years and persons aged ≥6 months with moderate or severe immunocompromise: recommendations of the Advisory Committee on Immunization Practices — United States, 2024. Morb. Mortal. Wkly Rep. 73(49), 1118–1123 (2024).
Google Scholar
20.
Grohskopf LA, Blanton LH, Ferdinands JM, Reed C, Dugan VG, Daskalakis DC. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices — United States, 2025–26 Influenza Season. Morb. Mortal. Wkly Rep. 74(32), 500–507 (2025).
PubMed
Google Scholar
21.
Higgins J, Thomas J, Chandler J et al. Cochrane Handbook for Systematic Reviews of Interventions (Version 5.5 Edition [updated August 2024]). Cochrane. (Accessed 11 May 2025). www.cochrane.org/handbook
Google Scholar
22.
Prasad V, Makary MA. An evidence-based approach to Covid-19 vaccination. N. Engl. J. Med. 392(24), 2484–2486 (2025).
PubMed
Google Scholar
23.
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. In: Research Ethics. Routledge (2008).
Google Scholar
24.
Menegale F, Manica M, Zardini A et al. Evaluation of waning of SARS-CoV-2 vaccine-induced immunity: a systematic review and meta-analysis. JAMA Netw. Open 6(5), e2310650 (2023).
PubMed
Google Scholar
25.
World Health Organization. Seasonal Influenza Vaccines: an Overview for Decision-Makers. (1st Edition). World Health Organization, Switzerland (2020).
Google Scholar
26.
World Health Organization. WHO roadmap on uses of COVID-19 vaccines in the context of Omicron and high population immunity (2023). https://www.who.int/publications/i/item/WHO-2019-nCoV-Vaccines-SAGE-Prioritization-2023.2
Google Scholar
27.
Kim SS, Flannery B, Foppa IM et al. Effects of prior season vaccination on current season vaccine effectiveness in the United States Flu Vaccine Effectiveness Network, 2012–2013 Through 2017–2018. Clin. Infect. Dis. 73(3), 497–505 (2021).
PubMed
Google Scholar
28.
Piret J, Boivin G. Viral interference between respiratory viruses. Emerg. Infect. Dis. 28(2), 273–281 (2022).
PubMed
Google Scholar
29.
Sacre A, Bambra C, Wildman JM et al. Socioeconomic inequalities in vaccine uptake: a global umbrella review. PLoS ONE 18(12), e0294688 (2023).
PubMed
Google Scholar
30.
Graham S, Walker JL, Andrews N et al. Quantifying and adjusting for confounding from health-seeking behavior and health care access in observational research. Open Forum Infect. Dis. 11(10), ofae598 (2024).
PubMed
Google Scholar
31.
Fukushima W, Hirota Y. Basic principles of test-negative design in evaluating influenza vaccine effectiveness. Vaccine 35(36), 4796–4800 (2017).
PubMed
Google Scholar
32.
Cao L, Lou J, Chan SY et al. Rapid evaluation of COVID-19 vaccine effectiveness against symptomatic infection with SARS-CoV-2 variants by analysis of genetic distance. Nat. Med. 28(8), 1715–1722 (2022).
PubMed
Google Scholar
33.
Manirambona E, John Okesanya O, Olabode Olaleke N, Adebimpe Oso T, Eliseo Lucero-Prisno D. Evolution and implications of SARS-CoV-2 variants in the post-pandemic era. Discover Public Health 21, 16 (2024).
Google Scholar
34.
Tricco AC, Chit A, Soobiah C et al. Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis. BMC Med. 11, 153 (2013).
PubMed
Google Scholar
•• This systematic review demonstrates maintained vaccine efficacy against mismatched influenza strains, providing critical empirical evidence for addressing strain variability challenges when assessing comparability and transitivity assumptions in seasonal vaccine evidence synthesis.
35.
GISAID. hCoV-19 (SARS-CoV-2) variants and lineages (2025). https://gisaid.org/hcov-19-variants-dashboard/
Google Scholar
36.
Higdon MM, Baidya A, Walter KK et al. Duration of effectiveness of vaccination against COVID-19 caused by the omicron variant. Lancet Infect. Dis. 22(8), 1114–1116 (2022).
PubMed
Google Scholar
37.
Ciabattini A, Nardini C, Santoro F, Garagnani P, Franceschi C, Medaglini D. Vaccination in the elderly: the challenge of immune changes with aging. Semin. Immunol. 40, 83–94 (2018).
PubMed
Google Scholar
38.
Varma A, Andrews NJ, Carazo S et al. Analytical approaches and examples of addressing time-varying factors in COVID-19 vaccine effectiveness studies: report from a meeting of the World Health Organization. Vaccine 62, 127567 (2025).
PubMed
Google Scholar
39.
Hollingsworth R, El Guerche-Séblain C, Tsai T et al. Assessment of the benefits of seasonal influenza vaccination: elements of a framework to interpret estimates of vaccine effectiveness and support robust decision-making and communication. Influenza Other Respir. Viruses 15(1), 164–174 (2021).
PubMed
Google Scholar
40.
Doll MK, Pettigrew SM, Ma J, Verma A. Effects of confounding bias in coronavirus disease 2019 (COVID-19) and influenza vaccine effectiveness test-negative designs due to correlated influenza and COVID-19 vaccination behaviors. Clin. Infect. Dis. 75(1), e564–e571 (2022).
PubMed
Google Scholar
41.
Mt-Isa S, Chumbley JR, Kelly MS, Weaver J, Banniettis N. Indirect comparison of the immunogenicity of 15-valent and 20-valent pneumococcal conjugate vaccines in children using a 2 + 1 schedule. Expert Rev. Vaccines 24(1), 946–957 (2025).
PubMed
Google Scholar
42.
European Medicines Agency, Committee for Medicinal Products for Human Use. Guideline on clinical evaluation of vaccines. European Medicines Agency, The Netherlands (2023).
Google Scholar
43.
Gilbert PB, Fong Y, Hejazi NS et al. Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controlled, vaccine efficacy trial. Vaccine 42(9), 2181–2190 (2024).
PubMed
Google Scholar
44.
Sicca F, Martinuzzi D, Montomoli E, Huckriede A. Comparison of influenza-specific neutralizing antibody titers determined using different assay readouts and hemagglutination inhibition titers: good correlation but poor agreement. Vaccine 38(11), 2527–2541 (2020).
PubMed
Google Scholar
45.
World Health Organization. COVID-19 Vaccines: Safety Surveillance Manual.. 1st edition. World Health Organization, Switzerland (2020).
Google Scholar
46.
US Food and Drug Administration. Development and licensure of vaccines to prevent COVID-19: guidance for industry (2023). (Accessed 11 May 2026). https://www.fda.gov/media/139638/download
Google Scholar
47.
Evrenoglou T, White IR, Afach S, Mavridis D, Chaimani A. Network meta-analysis of rare events using penalized likelihood regression. Stat. Med. 41(26), 5203–5219 (2022).
PubMed
Google Scholar
48.
Frieden TR. Evidence for health decision making – beyond randomized, controlled trials. N. Engl. J. Med. 377(5), 465–475 (2017).
PubMed
Google Scholar
49.
Bollaerts K, Wyndham-Thomas C, Miller E et al. The role of real-world evidence for regulatory and public health decision-making for accelerated vaccine deployment – a meeting report. Biologicals 85, 101750 (2024).
PubMed
Google Scholar
50.
de Waure C, Gärtner BC, Lopalco PL, Puig-Barbera J, Nguyen-Van-Tam JS. Real world evidence for public health decision-making on vaccination policies: perspectives from an expert roundtable. Expert Rev. Vaccines 23(1), 27–38 (2024).
PubMed
Google Scholar
51.
Thorlund K, Mills EJ. Sample size and power considerations in network meta-analysis. Syst. Rev. 1, 41 (2012).
PubMed
Google Scholar
52.
Ioannidis JPA. Factors influencing estimated effectiveness of COVID-19 vaccines in non-randomised studies. BMJ Evid. Based Med. 27(6), 324–329 (2022).
PubMed
Google Scholar
•• This study identifies critical bias sources in observational vaccine effectiveness studies, including pre-existing immunity, misclassification and outcome ascertainment, thus informing bias assessment when evaluating the feasibility of including nonrandomized evidence in NMA.
53.
Agampodi S, Tadesse BT, Sahastrabuddhe S, Excler J-L, Kim JH. Biases in COVID-19 vaccine effectiveness studies using cohort design. Front. Med. (Lausanne) 11, 1474045 (2024).
PubMed
Google Scholar
54.
Yuniar CT, Pratiwi B, Ihsan AF et al. Adverse events reporting quality of randomized controlled trials of COVID-19 vaccine using the CONSORT criteria for reporting harms: a systematic review. Vaccines (Basel) 10(2), 313 (2022).
PubMed
Google Scholar
55.
Barosa M, Prasad V. Characteristics of vaccine safety observational studies and authors' attitudes: a systematic review. Am. J. Med. 138(2), 254–261.e5 (2025).
PubMed
Google Scholar
56.
King DF, Groves H, Weller C et al. Realising the potential of correlates of protection for vaccine development, licensure and use: short summary. NPJ Vaccines 9(1), 82 (2024).
PubMed
Google Scholar
57.
Jin P, Li J, Pan H, Wu Y, Zhu F. Immunological surrogate endpoints of COVID-2019 vaccines: the evidence we have versus the evidence we need. Signal Transduct. Target Ther. 6(1), 48 (2021).
PubMed
Google Scholar
• This paper provides evidence that influenza vaccines maintain efficacy against mismatched strains, offering empirical support for treating strain variability as an addressable rather than prohibitive factor when evaluating transitivity and comparability assumptions in seasonal vaccine NMA.
|