Considerations for assessing the feasibility of network meta-analysis of seasonal vaccines
Abstract
Aim: Network meta-analyses (NMAs) of seasonal vaccines face distinct challenges that can compromise the validity and relevance of findings. While established frameworks offer guidance for evaluating the feasibility of NMAs, they do not address factors specific to seasonal vaccines. This study aims to highlight unique methodological challenges related to conducting NMA feasibility assessments of seasonal vaccines. The considerations are framed to be compatible with existing guidance and recommendations for the conduct and reporting of NMAs. Materials & methods: We developed a set of key considerations that should be applied when assessing the feasibility and/or validity of NMAs comparing seasonal vaccines. The considerations were based on systematic reviews and critical appraisals of published NMAs of seasonal vaccines, hands-on experience performing feasibility assessments of seasonal vaccines, and input from consultations with vaccine experts. Results: Unique considerations for evaluating comparability across seasonal vaccine studies include: whether vaccines should be compared by platform, formulation, dose, and/or valence; the impact of seasonality, strain evolution and definitions of placebo/unvaccinated controls on network connectivity; target population characteristics including history and recency of prior vaccination and/or infection(s), and baseline infection/severe disease risk; antigenic match (i.e., the degree of concordance between vaccine composition and circulating viral strains), which directly influences effectiveness and outcome measurement approaches that consider time varying epidemiology and assay and measure discrepancy. Comprehensively integrating these elements into existing guidance frameworks ensures transparent assessment of the key assumptions underlying NMA (i.e., transitivity and homogeneity) within the context of unique study design and methodological features of seasonal vaccine studies. Conclusion: The concepts highlighted in this paper address important gaps in the feasibility assessment process for NMAs of seasonal vaccines, which are crucial for informing public health decisions and guiding vaccine policy and implementation.
Plain language summary: Considerations for comparing seasonal vaccines using network meta-analysis
What is this article about?
Existing network meta-analysis (NMA) frameworks were expanded to highlight considerations for NMAs of seasonal vaccines, such as those for COVID-19 and influenza, which require annual updates and evaluation across multiple formulations, platforms and populations.
What methodology is described?
Specific considerations related to challenges that are unique to NMAs of seasonal vaccine studies, including strain evolution, antigenic match, time varying epidemiology, assays and measures discrepancy, are considered alongside the potential need to rely on real-world evidence for an up-to-date understanding of vaccine efficacy/effectiveness.
Why is this important?
Currently, there is no existing guidance addressing the specific methodological challenges of conducting NMAs for seasonal vaccines. This paper provides researchers with key considerations to ensure their methodological approaches are robust, their analytical decisions are transparent, and their NMAs produce valid comparative evidence for informing vaccine policy and clinical practice.
In the absence of head-to-head studies, network meta-analysis (NMA) is a method commonly used to estimate the relative effects of two or more interventions through a common comparator. Comparative efficacy/effectiveness estimates from NMAs are critical for informing healthcare decision making, including reimbursement, pricing and health policy decisions [1]. NMAs are well-established and generally accepted tools within health technology assessment (HTA); methods guidance from the National Institute and Health and Care Excellence (NICE) explicitly addresses acceptability of indirect and mixed treatment comparisons when the evidence base is suitable and the underlying assumptions, including homogeneity, transitivity and consistency, can be justified [2].
The feasibility of conducting an NMA needs to be carefully considered in the context of the core underlying assumptions to ensure the reliability and validity of the results [3–5]. Existing resources relevant to NMA feasibility assessment emphasize the systematic evaluation of clinical and methodological heterogeneity, and encourage transparent reporting of underlying assumptions [6,7]. However, these remain largely principles-based and do not provide universal thresholds, standardized decision rules or checklist-based criteria for determining feasibility. As a result, feasibility judgments require context-specific clinical and methodological interpretation, particularly when applied to preventative interventions, such as vaccines. A recent perspective piece acknowledged the gaps in vaccine-specific NMA guidance within the context of the European Union Joint Clinical Assessment and emphasized a few challenges with conducting indirect comparisons and meta-analyses of vaccines [8].
Unlike most routine vaccines that offer long-term protection (e.g., over several years or even a lifetime) and require less frequent boosters, vaccines against seasonal viruses (e.g., influenza and severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) are specifically tailored and adapted to address evolving viral strains each year. The need for annual adjustment introduces unique complexities for NMAs, demanding careful considerations beyond the conventional feasibility assessment frameworks. These challenges relate largely to varying risk profiles based on the study year, season, geography, population and outcome being used to assess vaccine performance (e.g., infection prevention vs hospitalization or severe outcomes) [9]. Therefore, NMAs of seasonal vaccines need to consider an expanded set of factors to ensure the underlying assumptions of a valid NMA are met, such as the impact of seasonality, strain evolution and definitions of placebo/unvaccinated controls on network connectivity, target population characteristics including history and recency of prior vaccination and/or infection(s), and baseline infection/severe disease risk, antigenic match (i.e., the degree of concordance between vaccine composition and circulating viral strains), which directly influences effectiveness and outcome measurement approaches that consider time varying epidemiology, and assay and measure discrepancy.
To date, detailed assessments of characteristics that are unique to seasonal vaccines and justification for underlying assumptions in published indirect comparisons has been limited. However, some recent published NMAs comparing COVID-19 and influenza vaccines have included detailed reporting of feasibility considerations, including network structure and node groupings, assessment of effect modifiers, acknowledgement of population comparability and antigenic match, outcome and data selection preferences [10–13]. Authors of these publications underscore the importance of transparent reporting the clinical and methodological judgments used to evaluate homogeneity, transitivity and provide examples of how uncertainty in how such judgements can be addressed using sensitivity and scenario analyses.
Clear examples of indirect comparisons being used in the context of vaccine policy decision-making are uncommon in the public domain. For seasonal vaccines, this may be driven by challenges in performing analyses utilizing evidence relevant for up-to-date for the decision context [14]. The Institute for Clinical and Economic Review (ICER) in the US has recognized the challenges with valid indirect comparisons of seasonal vaccines, specifically in the context of COVID-19 vaccines, claiming that comparative evidence across vaccines was insufficient for definitive comparative assessment [15,16]. On the contrary, the French National Authority for Health (Haute Autorité de santé [HAS]) recently criticized the evidence package submitted for influenza given the absence of an indirect comparison and indicated that the feasibility of such an analysis should have been explored [17]. Taken together, these examples suggest that the limited use of indirect comparisons in vaccine HTAs may reflect the methodological difficulty of constructing robust comparative evidence networks in this setting, rather than any lack of acceptance of indirect comparison methods by HTA bodies.
The aim of this publication is to expand upon existing feasibility assessment guidance to highlight challenges unique to the feasibility of NMAs of seasonal vaccines. The considerations draw on our experience assessing feasibility and conducting NMAs of seasonal vaccines, including clinical expert consultations; detailed evaluation of evidence bases identified through systematic literature reviews (SLRs), including randomized controlled trials (RCTs), observational studies and previously published indirect comparisons; and review of relevant evidence assessments and/recommendations from public health and immunization advisory bodies, such as the WHO [18] and the Advisory Committee on Immunization Practices (ACIP) [19,20], as well as HTA and value assessment bodies, such as HAS [17] and ICER [15,16], respectively.
Feasibility assessment considerations
Figure 1 summarizes key elements that may be considered when assessing the feasibility of NMAs of seasonal vaccines, using a stepwise approach. This structure is intended to complement the existing framework for NMAs of treatments [7] and extend it to seasonal vaccines to support evaluation of homogeneity-similarity of studies within each pairwise comparison based on clinical and methodological characteristics- and transitivity-whether imbalances exist across comparisons in clinical or methodological characteristics that are deemed effect modifiers (i.e., characteristics that modify the of relative effects across comparisons) in seasonal vaccine studies. These assessments inform the plausibility of consistency (i.e., whether direct and indirect evidence are expected to estimate the same underlying relative effects).
Figure 1. Feasibility assessment process with additional considerations for assessing comparability of seasonal vaccine studies.
The order in which each component is evaluated may vary depending on your research question or set of studies being considered for feasibility of NMA.
NMA: Network meta-analysis; RCT: Randomized controlled trial; VE: Vaccine efficacy/effectiveness.
This paper intentionally generalizes some key concepts to support broad applicability across different seasonal vaccines and anchored indirect comparisons. Viral strains are not differentiated from variants and ‘season’ is defined flexibly as either the period of viral predominance in a given geography (potentially differing across hemispheres) or the period of availability for that season’s vaccine, rather than a fixed calendar interval (e.g., fall/winter). While this paper is described in relation to NMAs, considerations also extend to anchored analyses, including pairwise anchored indirect comparisons, where two vaccines are compared indirectly via a single, common comparator rather than through a full network. Generalizability to unanchored analyses (e.g., population-adjusted indirect comparisons) may be limited, as such methods involve distinct methodological assumptions, data requirements and feasibility considerations beyond those discussed here.
Identification of the evidence base
Following best practices, studies to be considered for feasibility of NMA should be identified via an SLR, which is guided by a detailed protocol [21]. The protocol should include the objective of the SLR, relevant Population, Interventions, Comparators, Outcomes, and Study Design (PICOS) framework and outline a robust methodological approach in a replicable manner. The PICOS tailored for an NMA of seasonal vaccines should ensure the chosen study designs offer the most relevant evidence for the research question. While RCTs are the gold-standard for NMA, VE trials are rarely conducted for seasonal vaccine adaptations, which typically focus on immunogenicity end points (immunological bridges). In 2025, Prasad and Makary described the US FDA evidence-based framework for future COVID-19 evaluation, stating that immunogenicity is acceptable for decision-making of booster vaccines in elderly (65+ years) and high-risk populations (e.g., ≥1 risk factor for severe disease); however, approvals in the general population (6 months to 64 years and no risk factors) require RCT evidence of clinical outcomes, with control groups receiving placebo [22]. Historically, RCT evidence was often deemed not relevant for current decision making, given that VE trials did not consider the latest adapted vaccines or focus on current or recent vaccine seasons. In these cases, high-quality, observational studies – such as case–control or cohort designs – that compare vaccines or vaccination to no vaccination and include adequate adjustment for potential confounding variables may provide a more relevant, up-to-date evidence base and may be considered as a standalone evidence base, where appropriate [23]. When both RCTs and observational studies are available for the same research question, feasibility assessments should evaluate whether they are sufficiently compatible for a combined analysis or should be considered separately. Sensitivity analyses may also be explored to examine the impact of study design or risk of bias of individual studies on the robustness of the findings.
Network connectivity
The first step in assessing the feasibility of an NMA is to determine whether the identified studies can form a connected network of evidence through common comparators. When establishing a network of seasonal vaccine studies, differences in vaccine characteristics including vaccine product, platform, formulation, dose and/or valence must be taken into account.
Where head-to-head studies of active vaccines are not available, examining heterogeneity in the comparator group is particularly important, as the comparator group in many vaccine studies is composed of participants who never received a vaccine (unvaccinated), participants vaccinated in a prior season (no recent vaccine), a mix of both, or, in clinical trials, these participants may have been administered a placebo vaccine. In such instances where network connectivity requires placebo- or no (recent) vaccine-controlled studies, important considerations must be made to assess potential bias.
Six hypothetical studies of seasonal vaccines are presented which serve as examples of potential situations that may be encountered when working through a feasibility assessment. These aim to illustrate how some of the challenges faced may be addressed and what types of decisions may need to be made.
In Scenario A (Figure 2A), vaccines from the six studies are connected within a single network via placebo or no (recent) vaccine. This network requires an assumption of equivalence among placebo, no vaccine, and no recent vaccine and must consider whether the season in which the study was conducted and match of the vaccine to the circulating strain within that season is an effect modifier (Supplementary Table 1). The appropriateness of assuming equivalence between placebo/no vaccine and no recent vaccine as a common comparator relies on evidence indicating that vaccination in the prior season provides minimal or no residual protection at the time of the study (see population characteristics section) [12,24]. Evaluation of this assumption is required for each unique season represented in the network given that seasonal vaccines are re-formulated annually to match predicted circulating strains. Data on seasonal antigenic match – when the vaccine antigen matches that of circulating strains – by geography should be reviewed when available, and considered along with input from clinical experts.
Figure 2. Considerations for network connectivity.
Orange: studies conducted when Strain A was predominantly circulating; Green: studies conducted when Strain B was predominantly circulating. (A) Network assessing vaccines separately based on product, platform and strain with placebo/no recent vaccine as common comparator; (B) network assessing vaccines separately based on product, platform and strain without common comparator; (C) network assessing vaccines separately based on platform and strain; (D) network assessing vaccines targeting the same season/strain separately based on product, platform and strain with placebo/no recent vaccine as common comparator.
When limiting to product-specific, active-controlled studies, by removing the placebo/no (recent) vaccine comparator (Figure 2B), potential bias associated with vaccine matching is reduced, but all studies no longer connect to a single network. An alternative approach to the active-controlled network would be to collapse nodes by platform (e.g., egg based, cell based, recombinant, mRNA based, protein based), valence (e.g., monovalent, bivalent, trivalent, quadrivalent) and/or dose (e.g., standard dose, high dose) rather than product to provide greater opportunity for comparisons between key vaccine groups (Scenario C, Figure 2C). However, this network structure assumes equivalence between vaccine products and requires clinical validation (Supplementary Table 1).
To reduce bias from placebo-controlled studies while maintaining product-specific comparisons, consideration should be given to networks limited to studies evaluating vaccines targeting the same strains within the same season (Figure 2D). Such networks could consider placebo/no (recent) vaccine as a common comparator (as per Scenario A) or restrict the common comparator to either placebo/no vaccination alone or no recent vaccination alone. Restrictions to the common comparator are likely to limit the number of studies in the network, and subsequently the number of feasible vaccine comparisons.
An additional consideration regarding network connectivity by vaccine product may be needed when studies, primarily observational studies, do not report the vaccine product or only report results for groups that include a mix of products. In these situations, clinical and methodological input is needed to determine whether a vaccine group can be considered sufficiently representative of a single vaccine product within the context of the research question. This assessment should consider the distribution of vaccine products within the group, the comparison of interest, the outcome being evaluated, and the extent to which product-level differences may affect relative effects.
While this paper aims to cover most of the scenarios that researchers might encounter when undertaking an NMA of seasonal vaccines, selection of the best network structure and grouping of studies should be informed by the research question (i.e., what type of comparisons are of interest); potential biases that may arise from differences in populations and/or antigenic match of vaccines and circulating strains across the included studies and advice from clinical experts.
Population characteristics
Once network connectivity is established, the included studies should be thoroughly evaluated for similarities in population characteristics since imbalances in characteristics identified a priori as effect modifiers may violate the transitivity assumption.
When assessing the comparability of populations, it is important to consider standard demographic characteristics as well as factors particularly influential to seasonal vaccine studies such as age, underlying health conditions, occupation or exposure risk, prior vaccination and residual protection from prior infection or vaccination. Many of these factors contribute to vaccination guidance from major health authorities (i.e., World Health Organization [WHO]), with recommendations often varying for individuals at increased risk of infection and/or severe clinical outcomes (e.g., healthcare workers, pregnant women, children, elderly and those living with chronic conditions or who are immunocompromised) [25,26]. As such, it is important to consider whether differences in target populations, including those defined by risk level or clinical vulnerability, could affect the validity of the NMA.
Residual immunity derived from prior exposure to the virus through natural infection or previous vaccination should be carefully evaluated. The level, timing and type of prior exposure can affect immune responses and alter the baseline risk of infection or severe outcomes [27], while circulating viral strains across seasons, population-level vaccination coverage, time elapsed since previous exposure and the degree of immune waning may further influence residual protection (see study characteristics and outcomes sections). Interpretation of residual protection should consider atypical respiratory-virus circulation within or across study seasons, including substantial co-circulation or suppression of one respiratory virus during circulation of another. Such viral interference may contribute to season-specific differences in baseline risk, event rates, healthcare utilization and circulating strains or variants, which may affect study comparability and the validity of transitivity assumptions [28]. In addition, previous receipt of other seasonal vaccines or pneumococcal vaccines as well as previous antiviral therapy for infection should also be assessed where it could impact outcome severity or healthcare utilization, particularly in analyses of severe clinical outcomes.
Healthcare-seeking behavior is another critical consideration for seasonal vaccine NMAs, as it can substantially bias effectiveness estimates in observational studies [29]. Individuals who choose to receive annual seasonal vaccination may be systematically more likely to seek medical care when symptomatic, leading to differential outcome ascertainment between vaccinated and unvaccinated groups. This healthy user effect is particularly pronounced for seasonal vaccines given their voluntary, annual nature and variable uptake patterns across populations. When healthcare-seeking behavior is not consistently measured or reported across studies, proxies such as prior healthcare utilization (e.g., outpatient or preventive care visits, vaccination history, comorbidities or chronic conditions and indicators of access to care) may help characterize this behavior [30], or the use of test-negative control studies can reduce bias attributable to differences in healthcare-seeking behavior [31]. Depending on the design of the underlying study, these factors may be addressed through statistical adjustment techniques, such as randomization stratification, matching or weighting approaches, multivariable regression, propensity score methods or subgroup analyses.
As with any NMA, subgroup or sensitivity analyses should be considered to explore the potential impact of observed differences in key effect modifiers across study populations. Statistical heterogeneity arising from variations in population characteristics should be assessed alongside the interpretation of findings and contextualize the robustness and validity of the comparative effectiveness estimates.
Study characteristics
In seasonal vaccine studies, it is important to evaluate geographic location and study period to ensure they cover similar seasons or periods with comparable dominant strains. Viral mutations which emerge at different times based on geography can affect strain transmissibility and outcome severity, which influence VE estimates [32,33]. Antigenic mismatch between strains included in the vaccine formulation and those circulating during the study can also impact observed VE and, therefore, impact the validity of NMA estimates [34]. When assessing studies that evaluate different vaccine formulations within the same season, consideration must still be given to how well matched the vaccine was to the predominant circulating strain (Figure 3).
Figure 3. Comparability of studies based on seasonal vaccine, study period and prevalence of circulating strain.
Visualization of a hypothetical scenario using the six example studies. Vaccines against strain A were deployed in season 1, when strain A was still the more prevalent circulating variant. It then tapered off (representing <50% of circulating variants) while strain B started to emerge. Study 1 evaluated vaccines against strain A and were primarily conducted during periods of predominance of strain A (and its sub-strain A1), thus well-matched on vaccine strain and circulating strain. Study 2 evaluated vaccines against strain A and was conducted during a period when strain B became more prevalent, representing a mismatch between vaccine strain and circulating strain. Studies 3 and 4 evaluated vaccines against strain B and were well-matched with circulating strain B (and its substrains B1 and B2). The majority of Study 5 was conducted during a period of strain B predominance. The next seasonal vaccines (season 2) included strain B. Study 6 also evaluated vaccines against strain B but was conducted during a period when strain C began circulating and rose to predominance, thus representing a mismatch between vaccine and circulating strain.
M: Month.
This is especially important for networks that include studies comparing vaccines of different valencies or formulations or vaccine versus placebo or no (recent) vaccine groups. These types of studies are more susceptible to effect modification in the presence of antigenic mismatch. Specifically, when studies are conducted in different seasons, with different levels of antigenic matching (i.e., well-matched years vs mismatched years), the relative effect of vaccines versus placebo (or no [recent] vaccine) may differ (Figure 4). In networks including these types of studies, the match of each vaccine formulation to the year should be reviewed to inform study grouping and interpretation. In contrast, head-to-head studies of active vaccines adapted to the same strain are likely more robust to this effect modification and expected to have more stable relative effects, irrespective of seasonal match or mismatch. Ultimately, combining data from multiple studies, which occurred in different years or seasons or evaluated vaccines adapted to different strains, into a single network may be acceptable if studies only include head-to-head comparisons of vaccines, and within individual studies the head-to-head vaccines compared are adapted to the same strains and evaluated in the same season. If networks require a common comparator of placebo or no (recent) vaccine groups, it is imperative that the studies occur in the same season or are at least similar in terms of vaccine matching to the circulating strains.
Figure 4. Evidence of effect modification based on season when the comparator is placebo.
The percentages shown are the ratios among the groups, in this hypothetical scenario. For example, in the study conducted in a well-matched season (year 1), the percent of individuals receiving vaccine 1 is 50% lower than individuals who received placebo; the percent of individuals receiving vaccine 2 is 10% lower than those receiving vaccine 1. Even when illness rates are higher (year 2), the relative effectiveness of the vaccines versus each other and placebo are similar to year 1, as they are both well-matched seasons. In contrast, in a study conducted in a mis-matched season (year 3), illness rates in the vaccine arms are higher, leading to lower estimates of relative effectiveness versus placebo groups. However, because the vaccine groups, both adapted to the same strain, are equally affected by mismatch, the head-to-head relative effectiveness remains consistent (10% in this example) irrespective of mismatch versus well match (year 1 vs year 3) or overall illness rates (year 3 vs year 4).
Taken together with population characteristics, the setting in which the study is conducted should also be considered when assessing the comparability of seasonal vaccine studies for NMA. Studies carried out in or collecting data from hospitals or clinics, long-term care facilities or community-dwelling environments may include populations that differ in baseline risk of infection, patterns of exposure and likelihood of severe outcomes. For example, healthcare workers may have frequent occupational exposure to infected patients, whereas elderly individuals in nursing homes may have lower exposure but higher severity risk once infected; both should be assessed within the context of the full evidence base for NMA to assess the generalizability of the findings and whether sensitivity analyses should be explored to evaluate the impact of including studies conducted in different settings.
Other considerations may be needed around the size of seasonal vaccine studies in terms of efficacy/effectiveness. RCTs, especially those evaluating adapted vaccine formulations rather than novel formulations, are often small in size and, thus, may be underpowered to detect differences in outcomes that occur infrequently, such as severe infections, specific adverse events (AEs) or death. With these limitations potentially undermining the results of NMAs of RCTs, observational studies, such as those collecting data from large, national databases including hundreds of thousands or even millions of participants can serve as an alternative evidence base for NMA to potentially generate more reliable estimates of comparative effectiveness or safety, but with greater between-study heterogeneity. As with any NMA, it is important to understand the pros and cons of each study design as well as underlying limitations and potential impact on results.
Study design elements traditionally assessed for any NMA should also be evaluated, such as randomization or population matching and adjustment for confounders, inclusion and exclusion criteria, and study quality, to ensure comparability across studies and support the assumptions of similarity, transitivity and consistency.
Outcomes
Unlike traditional NMAs of treatments, efficacy/effectiveness outcomes for vaccines span both immunological and clinical measures, as vaccines act by inducing immune responses to prevent illness rather than treating an existing disease. A defining feature of seasonal vaccine studies is that both clinical (e.g., laboratory-confirmed infection, medically-attended illness) and immunogenicity (e.g., neutralizing antibody titers, seroconversion rate) outcomes are strain specific.
Clinical outcomes can differ substantially depending on the viral strains present during the study period; therefore, strain-specific outcome data are necessary for accurate interpretation. Similarly, immunogenicity responses vary across strains included in the vaccine, and strain-specific measurements provide a more precise understanding of immune protection.
Viral sequencing and detailed immunogenicity testing are often not reported for all studies, limiting the availability of strain-specific data to be considered for NMA. Where strain-specific data are unavailable, inconsistently reported, or based on limited sequencing or diagnostic confirmation, the suitability of the study for inclusion should be carefully evaluated; pooled estimates across different strains may introduce heterogeneity and not be clinically relevant for the research question. In these cases, geography and study period may provide proxy information to infer which strain was predominantly circulating during the time of the study for a particular country or region. If not specified by the study authors, publicly available global, regional and national surveillance datasets that report strain-specific prevalence by geography can also provide insight into predominant circulating variants [35]. Given this challenge, any assumptions should be vetted with clinical experts and/or virus specialists to ensure studies are appropriately grouped into strain-specific networks.
Time since current seasonal vaccination (i.e., timing of outcome assessment or duration of follow-up) is another main component of outcome assessment to consider during the feasibility assessment. VE is highly susceptible to waning of effect over time, especially among special target populations such as older adults or immunocompromised individuals [36,37]. Therefore, alignment of the timing of outcome assessment following vaccination across studies is critical to reduce potential bias [38]. Any assumptions made with respect to the comparability of studies and impact of potential biases on VE [39,40] should be clearly communicated in the context of the NMA findings.
It is also necessary to assess the types of data available for each outcome of interest and select appropriate summary statistics as inputs for the NMA. Special considerations by outcome category (e.g., efficacy/effectiveness, immunogenicity and safety) are described separately below.
Clinical efficacy/effectiveness outcomes
Vaccine protection against clinical efficacy/effectiveness outcomes (e.g., infection, medical visits, hospitalization) may be reported as crude event rates by group (arm level) or VE between groups (contrast level). VE is calculated as and is most translatable to relative risk (RR) where VE = (1 – RR) × 100%. VE may be calculated from other ratios (e.g., odds ratio [OR] or hazard ratio [HR]). ORs are approximately the same as RRs when event rates are low, whereas HRs, which are used for time-to-event data, may misrepresent vaccine efficacy if protection changes over time, thus overestimating long-term effects in longer studies with early events, or underestimating efficacy in shorter studies with delayed protection. Careful consideration is needed when using VE as the basis for inputs for NMA, since it can be calculated from different ratio types, which could introduce bias into the results. It is also important to note that VE itself cannot be used as data inputs because NMA relies on log-transformed relative measures [21]. VE must first be converted back to the logarithmic scale (i.e., log[RR]) before conducting analysis. NMA outputs can then be re-calculated back to VE for ease of interpretation.
In some instances, arm-level rates may be more suitable for calculating a consistent effect measure; however, unadjusted rates may also lead to bias in the NMA if potential confounders are not accounted for [18]. Selection of an appropriate effect measure should be carefully considered based on the outcome type, frequency of events, follow-up duration, censoring rules, availability of adjusted estimates and set of adjustment variables. When assumptions are needed for transforming, simulating or harmonizing effect measures across studies, assumptions should be clearly described with the appropriate reasoning. Sensitivity analyses may be considered to explore the impact of such data input decisions on the NMA results.
Immunogenicity outcomes
Vaccine efficacy/effectiveness can also be measured in terms of immunological response. Immunogenicity data are commonly reported using geometric mean titer (GMT) or concentration (GMC), which represent the mean of the titers of antibodies or other immunological markers on a logarithmic scale. GMT can be reported as an absolute value (e.g., prior to vaccination or after vaccination) or as a ratio (e.g., comparing GMT before and after vaccination or comparing GMT of a vaccine group to a control or other vaccine group). Careful review of GMT outcomes should ensure the same effect measures are reported or can be calculated as data inputs for NMA. Where available, measures reflecting relative effects between groups should be prioritized for NMA, such as the ratio of post-vaccination GMTs or GMCs between groups (GMR) or the rate difference between groups in seroconversion, seroresponse or seroprotection [41]. When an accepted correlate or threshold of protection exists, responder or seroprotection rates may be prioritized; otherwise, GMT/GMC ratios and justified seroresponse definitions provide a more general basis for noninferiority or superiority comparisons [42].
When conducting NMAs of immunogenicity outcomes, core challenges include that immunogenicity cannot reliably be used as a surrogate for clinical protection or efficacy [43], and that different laboratories use different immunogenicity assays (e.g., hemagglutination inhibitor, enzyme-linked immunosorbent assay) with distinct sensitivity thresholds and measurement approaches yielding incompatible data that cannot be reliably pooled without careful consideration of assay-specific performance characteristics [44].
Particular caution is warranted when evaluating strain-, variant- or antigen-specific immunogenicity outcomes for which little or no vaccine-induced response is expected. Measured values near the assay floor may largely reflect background signal, cross-reactivity, imputation of low values or other assay variability rather than a true immune response. In such settings, rules for handling values below the assay limit should be considered, and GMT/GMC ratios should be interpreted alongside absolute titers, proportions above the assay threshold and assay-performance characteristics [41].
As such, NMA findings must be interpreted cautiously, especially when immunogenicity data (in the absence of efficacy/effectiveness) are used as evidence to inform public health decisions.
Safety outcomes
A key challenge for NMAs of vaccine safety is the inconsistent definition, collection, adjudication and reporting of AEs and other safety outcomes across studies. Evaluation of AE comparability should include consideration of whether events are solicited or unsolicited, actively captured or passively reported, and whether severity is graded using consistent criteria [45]. Additionally, it is important to assess if events are classified into comparable clinical categories or defined as AEs of special interest and ensure that follow-up durations are adequately harmonized to support valid comparisons. Solicited local and systemic reactogenicity events are commonly collected over short post-vaccination windows, whereas unsolicited AEs, medically attended AEs, serious AEs and AEs of special interests may be collected over longer periods [46]. Studies reporting the same safety event over different timeframes (e.g., 0–7, 0–28 or 0–42 days after vaccination) may not be estimating the same outcome construct; longer follow-up may increase the opportunity to observe events, while shorter follow-up may underestimate events with delayed onset.
Vaccine safety data are also commonly characterized by sparse data, infrequent events and studies with zero occurrences, which present notable challenges for NMAs. The Cochrane Handbook [21] acknowledges that rare events commonly affect safety outcomes and rare adverse outcomes may yield unstable estimates. When conventional meta-analytic methods underperform when events are exceedingly rare or absent in certain study arms, alternative analytic approaches may be considered [47]. In such scenarios, the suitability of formal comparative analysis should be evaluated for each outcome individually. If event counts are insufficient, reporting varies or the assumptions necessary for NMA lack credibility, relative-effect estimates may need to be considered exploratory and interpreted with caution.
Lastly, some of the factors impacting NMA feasibility presented above that are considered effect modifiers for VE and immunogenicity outcomes (e.g., circulating strain and antigenic match) may not be applicable for or equally impact safety data. Therefore, networks of vaccine safety outcomes need separate considerations and justifications around homogeneity and transitivity assumptions. If differences in AE definitions, ascertainment, severity grading, event categorization or follow-up duration are suspected to violate these assumptions, sensitivity analyses focusing on studies with similar criteria or reporting may be considered if sufficient data are available to answer the research question. When safety data differ meaningfully between studies and cannot be aligned, NMA may be deemed not feasible and a descriptive summary of the results may be more appropriate.
Discussion
Feasibility assessments are critical for determining whether valid NMAs can be conducted. This paper expands upon existing guidance [7] by highlighting considerations for the unique methodological challenges associated with seasonal vaccine studies. Seasonal vaccines must be compared by formulation, dose, platform and valence, as these variables can influence outcomes. Seasonal aspects such as timing, strain variations and definitions of controls also affect study comparability. It is essential to account for population attributes, including prior vaccination, infection history and baseline risk, as well as ensuring antigenic compatibility with circulating strains to achieve optimal effectiveness. Furthermore, selection of suitable study designs and outcome measurement methods is critical, given evolving epidemiological trends and assay differences. These considerations highlight the complexity of NMAs for seasonal vaccines that demand careful evaluation of homogeneity, transitivity, consistency and the appropriateness of common comparators which can profoundly impact the validity of NMA findings. Comprehensively evaluating these factors helps to assess whether the fundamental assumptions of transitivity and similarity are sufficiently upheld, supports judgments about whether NMA is feasible when substantial heterogeneity is observed, and provides a basis for sensitivity analyses to examine the impact of decisions for analysis.
Unlike pharmacological treatments or traditional vaccines, seasonal vaccines are frequently reformulated in response to newly circulating pathogens, which means that timely RWE is especially valuable for informing public health decisions, often more so than RCTs, which may not keep pace with the speed of vaccine updates and deployment and can lack external validity [48,49]. To remain relevant for decision-making, NMAs of seasonal vaccines must be regularly updated to reflect these dynamic changes [50].
Furthermore, the use of RCTs in NMAs presents significant challenges for evaluating seasonal vaccines, primarily due to restricted sample sizes and insufficient statistical power to identify infrequent outcomes [51]. While observational studies provide access to larger sample populations and greater coverage, they are associated with increased heterogeneity and a higher risk of bias, both in methodology and reporting [52,53]. RWE often lacks sufficient information to properly examine heterogeneity or effect modification, or to conduct sensitivity and subgroup analyses for testing transitivity assumptions. Therefore, it is essential to thoroughly evaluate the respective strengths and weaknesses of these methodologies when selecting an evidence base for NMA.
Beyond clinical efficacy and effectiveness outcomes, which represent the primary evidence base for vaccine decision-making, this paper highlights methodological considerations for NMAs of vaccine safety and immunogenicity, which are outcomes that provide essential supporting evidence for regulatory review, clinical guideline development and vaccination program planning. Safety data present particular analytical challenges in seasonal vaccine studies, as they are frequently defined inconsistently and categorized variably across trials, resulting in substantial heterogeneity that complicates data synthesis and interpretation [54,55]. Immunogenicity data introduce a different set of considerations: while immune responses serve as important biological markers, a detectable immune response cannot be directly equated with clinical protection against infection without extensive research to establish it as a validated correlate of protection [56,57]. Consequently, immunogenicity data provide valuable mechanistic insights but remain insufficient as standalone evidence of vaccine effectiveness, underscoring the continued necessity for robust clinical outcome studies.
The considerations described were primarily culminated from the authors' experience with NMA feasibility assessments for influenza and COVID-19 vaccines, where annual strain updates necessitate frequent evidence synthesis. The applicability of the examples presented to long-term or single-series vaccines against infections such as respiratory syncytial virus, tetanus, diphtheria, pertussis (Tdap) or pneumonia which do not require annual reformulation due to conserved target antigens, may be limited. Additionally, the considerations may not adequately address NMAs of combination vaccines or co-administration strategies for multi-pathogen prevention, where network structures are more complex, comparative evidence may be even more sparse, and additional complexities arise from heterogeneous outcome measures. This paper also focuses on feasibility considerations for NMAs and, where relevant, anchored indirect comparisons, which rely on a connected evidence base to estimate relative effects, and is not generalizable to all indirect comparisons. Specifically, unanchored indirect comparisons are outside the scope of this paper because, in the absence of a common comparator, they require a different feasibility assessment focused on whether the available data can support adjustment for all relevant cross-study differences. In this setting, feasibility depends not only on the similarity of studies and distribution of potential effect modifiers, but also on whether the available data can support adjustment for all relevant cross-study differences needed to make the comparison interpretable. This introduces distinct considerations regarding the availability and compatibility of individual patient data and aggregate data, the completeness and measurement of both effect modifiers and prognostic factors, and the validity of estimating relative effects in the absence of an anchor.
Conclusion
This paper highlights key challenges with assessing the feasibility of NMAs for seasonal vaccines and provides clear considerations to support a valid methodological approach. Enhanced transparency and detailed reporting of feasibility methods and assumptions are critical NMA relevance and reliability, especially as viruses and vaccines continue to evolve. Given the dynamic nature of the seasonal vaccine landscape, this framework may require adaptation for future vaccines or virus seasons.
Beyond methodological rigor, comparative effect estimates for seasonal vaccines play a central role in HTA, economic evaluation, annual vaccination policy development and strategies to improve vaccine uptake. NMAs of vaccines facilitate evidence-based vaccine selection for clinicians, inform public health policy decisions and support targeted approaches to improve vaccination rates. The integration of NMA evidence into vaccination policy underscores the crucial role of robust comparative effectiveness research in supporting optimal vaccination strategies.
Summary points
•
This paper discusses critical methodological considerations for conducting network meta-analyses (NMAs) of seasonal vaccines, where unique characteristics like antigenic drift and temporal variation complicate standard NMA assumptions.
•
For seasonal vaccines, high-quality observational studies from recent seasons often provide more policy-relevant evidence than trials of outdated strain formulations, as manufacturers reformulate vaccines annually based on predicted circulating strains.
•
Antigenic match between vaccine composition and circulating viral strains is a fundamental determinant of effectiveness that varies by season and geography, requiring explicit evaluation as a potential effect modifier in NMAs.
•
Establishing valid network connectivity in seasonal vaccine comparisons requires assessing whether studies from different seasons with varying antigenic matches can share a common comparator, as ‘no recent vaccine’ in a high-match season differs fundamentally from placebo in a mismatch season.
•
Prior vaccination history and cumulative immune exposure are critical effect modifiers unique to seasonal vaccines, as repeated annual vaccination and natural infection create population-level immunity patterns that influence comparative effectiveness estimates.
•
Geographic and temporal alignment is essential because viral evolution occurs asynchronously across regions, creating strain diversity that makes effectiveness estimates from different locations or seasons potentially incomparable even for identically formulated vaccines.
•
Strain-specific outcome and immunogenicity data are necessary because multivalent seasonal vaccines perform differently against each included strain, and aggregate measures can mask clinically important variation in protection.
•
Vaccine effectiveness is a measure unique to vaccine studies that expresses proportional risk reduction; different vaccine effectiveness calculation methods and underlying study designs can yield incompatible effect estimates that bias comparative assessments.
•
Attention to these considerations can strengthen the validity and transparency of seasonal vaccine NMAs conducted to support evidence-informed decision-making by regulatory agencies, advisory committees and healthcare systems.
Author contributions
Conceptualization: A Cichewicz, M Betts, H Burnett. Methodology: A Cichewicz, M Betts, H Burnett. Visualization: A Cichewicz, M Betts. Writing – original draft: A Cichewicz. Writing – review and editing: A Cichewicz, M Betts, N Mesaros, H Burnett. All authors approved the final manuscript and agreed to be accountable for all aspects of the work.
Financial disclosure
Funding was provided by Thermo Fisher Scientific (MA, USA) through the authors' regular employment. No external funding was received.
Competing interests disclosure
A Cichewicz, M Betts and N Mesaros are employees of PPD™ Evidera™ Health Economics and Market Access, Thermo Fisher Scientific. H Burnett is an independent consultant in the field of health economics and outcomes research. The authors have no other competing interests or relevant affiliations with any organization/entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed. No external writing support was used for the development of this manuscript. The authors have no other competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript apart from those disclosed.
Writing disclosure
No funded writing assistance was utilized in the production of this manuscript.
Open access
This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/
Supplementary Material
File (supplementary materials.docx)
- Download
- 29.45 KB
References
Papers of special note have been highlighted as: • of interest; •• of considerable interest
1.
Igarashi A, Tanaka S, De Moor R et al. Indirect treatment comparisons in healthcare decision making: a targeted review of regulatory approval, reimbursement, and pricing recommendations globally for oncology drugs in 2021–2023. Adv. Ther. 42(1), 52–69 (2025).
2.
National Institute for Health and Care Excellence. NICE technology appraisal and highly specialised technologies guidance: the manual (2022). (Accessed 10 May 2026). https://www.nice.org.uk/process/pmg36/resources/nice-technology-appraisal-and-highly-specialised-technologies-guidance-the-manual-pdf-72286779244741
3.
Hoaglin DC, Hawkins N, Jansen JP et al. Conducting indirect-treatment-comparison and network-meta-analysis studies: report of the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices: part 2. Value Health 14(4), 429–437 (2011).
4.
Ahn E, Kang H. Concepts and emerging issues of network meta-analysis. Korean J. Anesthesiol. 74(5), 371–382 (2021).
5.
Dias S, Sutton AJ, Welton NJ, Ades AE. NICE DSU technical support document 3: heterogeneity: subgroups, meta-regression, bias and bias-adjustment (2011). (Accessed 11 May 2025). http://www.nicedsu.org.uk/
•• This document by The National Institute for Health and Care Excellence (NICE) Decision Support Unit provides core methodological guidance on assessment of whether heterogeneity and bias from effect modifiers can be explained or adjusted for sufficiently for valid network meta-analyses (NMAs) to inform medical decision-making.
6.
Member State Coordination Group on Health Technology Assessment (HTA CG). Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons (2024). (Accessed 18 January 2026). https://health.ec.europa.eu/document/download/4ec8288e-6d15-49c5-a490-d8ad7748578f_en?filename=hta_methodological-guideline_direct-indirect-comparisons_en.pdf
• This guidance document from the Member State Coordination Group on Health Technology Assessment (HTA) describes the approach for conducting NMAs for HTA using direct and indirect evidence.
7.
Cope S, Zhang J, Saletan S, Smiechowski B, Jansen JP, Schmid P. A process for assessing the feasibility of a network meta-analysis: a case study of everolimus in combination with hormonal therapy versus chemotherapy for advanced breast cancer. BMC Med. 12, 93 (2014).
•• This seminal paper establishes the systematic methodology for NMA feasibility assessment, detailing procedures for visualizing treatment/outcome heterogeneity and operationalizing transitivity evaluation to determine if indirect comparisons are methodologically valid.
8.
Graham C, Barker E, Moss J et al. Opportunities for and challenges of conducting indirect treatment comparisons and meta-analyses for vaccines in post-EU HTA regulation era. JMAHP 13(2), 31 (2025).
9.
Ainslie KEC, Haber M, Orenstein WA. Challenges in estimating influenza vaccine effectiveness. Expert Rev. Vaccines 18(6), 615–628 (2019).
• This expert review contextualizes the unique challenges and considerations for seasonal respiratory virus vaccines within the influenza vaccine landscape, providing essential background for adapting feasibility assessment frameworks to this therapeutic area.
10.
Beck E, Georgieva M, Wang W-J et al. Indirect comparison of the relative vaccine effectiveness of mRNA-1283 vs. BNT162b2 vaccines against symptomatic COVID-19 among US adults. Curr. Med. Res. Opin. 41(4), 721–732 (2025).
11.
Fahrbach K, Cichewicz A, Chu H et al. Comparative effectiveness of Omicron XBB 1.5-adapted COVID-19 vaccines: a systematic literature review and network meta-analysis. Expert Rev. Vaccines 24(1), 416–432 (2025).
12.
Ferdinands JM, Blanton LH, Alyanak E et al. Protection against influenza hospitalizations from enhanced influenza vaccines among older adults: a systematic review and network meta-analysis. J. Am. Geriatr. Soc. 72(12), 3875–3889 (2024).
13.
Veroniki AA, Thirugnanasampanthar SS, Konstantinidis M et al. Trivalent and quadrivalent seasonal influenza vaccine in adults aged 60 and older: a systematic review and network meta-analysis. BMJ Evid. Based Med. 29(4), 239–254 (2024).
14.
McGirr A, Widenmaier R, Burnett H et al. Network meta-analysis in the evaluation of vaccines. Value Health 20(9), A777 (2017).
15.
Institute for Clinical and Economic Review. Vaccines for Covid-19: draft background and scope (2025). (Accessed 11 May 2026). https://icer.org/wp-content/uploads/2025/11/ICER_Covid-19_Draft-Scope_For-Publication_111025.pdf?
16.
Rind DM, McQueen RB, Zemplenyi A et al. Vaccines for Covid-19: effectiveness and value; draft evidence report. Inst. Clin.Econ. Rev. (2026). (Accessed 9 May 2026). https://icer.org/assessment/covid-19-2025/
17.
Haute Autorité de Santé. EFLUELDA (vaccin grippal trivalent inactivé haute dose): immunisation active des personnes âgées de 60 ans et plus pour lesquelles la vaccination grippale est recommandée. Haute Autorité de Santé (2025). Available from: https://www.has-sante.fr/upload/docs/application/pdf/2025-05/efluelda_avis_economique_ceesp_795.pdf
18.
World Health Organization. Evaluation of COVID-19 vaccine effectiveness in a changing landscape of COVID-19 epidemiology and vaccination (2022). https://www.who.int/publications/i/item/WHO-2019-nCoV-vaccine_effectiveness-VE_evaluations-2022.1
• This guidance document by the WHO addresses methodological considerations for evaluating vaccine effectiveness amid evolving variants and waning immunity, providing frameworks for assessing temporal comparability and transitivity assumptions in dynamic epidemiological contexts.
19.
Roper LE, Godfrey M, Link-Gelles R et al. Use of additional doses of 2024–2025 COVID-19 vaccine for adults aged ≥65 years and persons aged ≥6 months with moderate or severe immunocompromise: recommendations of the Advisory Committee on Immunization Practices — United States, 2024. Morb. Mortal. Wkly Rep. 73(49), 1118–1123 (2024).
20.
Grohskopf LA, Blanton LH, Ferdinands JM, Reed C, Dugan VG, Daskalakis DC. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices — United States, 2025–26 Influenza Season. Morb. Mortal. Wkly Rep. 74(32), 500–507 (2025).
21.
Higgins J, Thomas J, Chandler J et al. Cochrane Handbook for Systematic Reviews of Interventions (Version 5.5 Edition [updated August 2024]). Cochrane. (Accessed 11 May 2025). www.cochrane.org/handbook
22.
Prasad V, Makary MA. An evidence-based approach to Covid-19 vaccination. N. Engl. J. Med. 392(24), 2484–2486 (2025).
23.
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. In: Research Ethics. Routledge (2008).
24.
Menegale F, Manica M, Zardini A et al. Evaluation of waning of SARS-CoV-2 vaccine-induced immunity: a systematic review and meta-analysis. JAMA Netw. Open 6(5), e2310650 (2023).
25.
World Health Organization. Seasonal Influenza Vaccines: an Overview for Decision-Makers. (1st Edition). World Health Organization, Switzerland (2020).
26.
World Health Organization. WHO roadmap on uses of COVID-19 vaccines in the context of Omicron and high population immunity (2023). https://www.who.int/publications/i/item/WHO-2019-nCoV-Vaccines-SAGE-Prioritization-2023.2
27.
Kim SS, Flannery B, Foppa IM et al. Effects of prior season vaccination on current season vaccine effectiveness in the United States Flu Vaccine Effectiveness Network, 2012–2013 Through 2017–2018. Clin. Infect. Dis. 73(3), 497–505 (2021).
28.
Piret J, Boivin G. Viral interference between respiratory viruses. Emerg. Infect. Dis. 28(2), 273–281 (2022).
29.
Sacre A, Bambra C, Wildman JM et al. Socioeconomic inequalities in vaccine uptake: a global umbrella review. PLoS ONE 18(12), e0294688 (2023).
30.
Graham S, Walker JL, Andrews N et al. Quantifying and adjusting for confounding from health-seeking behavior and health care access in observational research. Open Forum Infect. Dis. 11(10), ofae598 (2024).
31.
Fukushima W, Hirota Y. Basic principles of test-negative design in evaluating influenza vaccine effectiveness. Vaccine 35(36), 4796–4800 (2017).
32.
Cao L, Lou J, Chan SY et al. Rapid evaluation of COVID-19 vaccine effectiveness against symptomatic infection with SARS-CoV-2 variants by analysis of genetic distance. Nat. Med. 28(8), 1715–1722 (2022).
33.
Manirambona E, John Okesanya O, Olabode Olaleke N, Adebimpe Oso T, Eliseo Lucero-Prisno D. Evolution and implications of SARS-CoV-2 variants in the post-pandemic era. Discover Public Health 21, 16 (2024).
34.
Tricco AC, Chit A, Soobiah C et al. Comparing influenza vaccine efficacy against mismatched and matched strains: a systematic review and meta-analysis. BMC Med. 11, 153 (2013).
•• This systematic review demonstrates maintained vaccine efficacy against mismatched influenza strains, providing critical empirical evidence for addressing strain variability challenges when assessing comparability and transitivity assumptions in seasonal vaccine evidence synthesis.
35.
GISAID. hCoV-19 (SARS-CoV-2) variants and lineages (2025). https://gisaid.org/hcov-19-variants-dashboard/
36.
Higdon MM, Baidya A, Walter KK et al. Duration of effectiveness of vaccination against COVID-19 caused by the omicron variant. Lancet Infect. Dis. 22(8), 1114–1116 (2022).
37.
Ciabattini A, Nardini C, Santoro F, Garagnani P, Franceschi C, Medaglini D. Vaccination in the elderly: the challenge of immune changes with aging. Semin. Immunol. 40, 83–94 (2018).
38.
Varma A, Andrews NJ, Carazo S et al. Analytical approaches and examples of addressing time-varying factors in COVID-19 vaccine effectiveness studies: report from a meeting of the World Health Organization. Vaccine 62, 127567 (2025).
39.
Hollingsworth R, El Guerche-Séblain C, Tsai T et al. Assessment of the benefits of seasonal influenza vaccination: elements of a framework to interpret estimates of vaccine effectiveness and support robust decision-making and communication. Influenza Other Respir. Viruses 15(1), 164–174 (2021).
40.
Doll MK, Pettigrew SM, Ma J, Verma A. Effects of confounding bias in coronavirus disease 2019 (COVID-19) and influenza vaccine effectiveness test-negative designs due to correlated influenza and COVID-19 vaccination behaviors. Clin. Infect. Dis. 75(1), e564–e571 (2022).
41.
Mt-Isa S, Chumbley JR, Kelly MS, Weaver J, Banniettis N. Indirect comparison of the immunogenicity of 15-valent and 20-valent pneumococcal conjugate vaccines in children using a 2 + 1 schedule. Expert Rev. Vaccines 24(1), 946–957 (2025).
42.
European Medicines Agency, Committee for Medicinal Products for Human Use. Guideline on clinical evaluation of vaccines. European Medicines Agency, The Netherlands (2023).
43.
Gilbert PB, Fong Y, Hejazi NS et al. Four statistical frameworks for assessing an immune correlate of protection (surrogate endpoint) from a randomized, controlled, vaccine efficacy trial. Vaccine 42(9), 2181–2190 (2024).
44.
Sicca F, Martinuzzi D, Montomoli E, Huckriede A. Comparison of influenza-specific neutralizing antibody titers determined using different assay readouts and hemagglutination inhibition titers: good correlation but poor agreement. Vaccine 38(11), 2527–2541 (2020).
45.
World Health Organization. COVID-19 Vaccines: Safety Surveillance Manual.. 1st edition. World Health Organization, Switzerland (2020).
46.
US Food and Drug Administration. Development and licensure of vaccines to prevent COVID-19: guidance for industry (2023). (Accessed 11 May 2026). https://www.fda.gov/media/139638/download
47.
Evrenoglou T, White IR, Afach S, Mavridis D, Chaimani A. Network meta-analysis of rare events using penalized likelihood regression. Stat. Med. 41(26), 5203–5219 (2022).
48.
Frieden TR. Evidence for health decision making – beyond randomized, controlled trials. N. Engl. J. Med. 377(5), 465–475 (2017).
49.
Bollaerts K, Wyndham-Thomas C, Miller E et al. The role of real-world evidence for regulatory and public health decision-making for accelerated vaccine deployment – a meeting report. Biologicals 85, 101750 (2024).
50.
de Waure C, Gärtner BC, Lopalco PL, Puig-Barbera J, Nguyen-Van-Tam JS. Real world evidence for public health decision-making on vaccination policies: perspectives from an expert roundtable. Expert Rev. Vaccines 23(1), 27–38 (2024).
51.
Thorlund K, Mills EJ. Sample size and power considerations in network meta-analysis. Syst. Rev. 1, 41 (2012).
52.
Ioannidis JPA. Factors influencing estimated effectiveness of COVID-19 vaccines in non-randomised studies. BMJ Evid. Based Med. 27(6), 324–329 (2022).
•• This study identifies critical bias sources in observational vaccine effectiveness studies, including pre-existing immunity, misclassification and outcome ascertainment, thus informing bias assessment when evaluating the feasibility of including nonrandomized evidence in NMA.
53.
Agampodi S, Tadesse BT, Sahastrabuddhe S, Excler J-L, Kim JH. Biases in COVID-19 vaccine effectiveness studies using cohort design. Front. Med. (Lausanne) 11, 1474045 (2024).
54.
Yuniar CT, Pratiwi B, Ihsan AF et al. Adverse events reporting quality of randomized controlled trials of COVID-19 vaccine using the CONSORT criteria for reporting harms: a systematic review. Vaccines (Basel) 10(2), 313 (2022).
55.
Barosa M, Prasad V. Characteristics of vaccine safety observational studies and authors' attitudes: a systematic review. Am. J. Med. 138(2), 254–261.e5 (2025).
56.
King DF, Groves H, Weller C et al. Realising the potential of correlates of protection for vaccine development, licensure and use: short summary. NPJ Vaccines 9(1), 82 (2024).
57.
Jin P, Li J, Pan H, Wu Y, Zhu F. Immunological surrogate endpoints of COVID-2019 vaccines: the evidence we have versus the evidence we need. Signal Transduct. Target Ther. 6(1), 48 (2021).
• This paper provides evidence that influenza vaccines maintain efficacy against mismatched strains, offering empirical support for treating strain variability as an addressable rather than prohibitive factor when evaluating transitivity and comparability assumptions in seasonal vaccine NMA.
Information & Authors
Information
Published In
Copyright
© 2026 The authors. This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License
History
Received: 22 January 2026
Accepted: 5 June 2026
Published online: 29 June 2026
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Considerations for assessing the feasibility of network meta-analysis of seasonal vaccines. (2026) Journal of Comparative Effectiveness Research. DOI: 10.57264/cer-2026-0014
Export citation
Select the citation format you wish to export for this article or chapter.