Development and implementation of a living network meta-analysis framework for comparative effectiveness research: a case study in chronic venous insufficiency treatment

A four-phase living NMA framework was designed prospectively to accommodate ongoing evidence synthesis. The complete framework is illustrated in Figure 1, which serves as a visual roadmap for the methods described below. The framework addresses three key requirements for sustainable living evidence synthesis: resource commitment for ongoing surveillance and analysis, methodological consistency across update cycles and transparent dissemination to stakeholders. Phase I establishes the foundation, including protocol registration, sponsor commitment and prespecification of methods. Phase II comprises the baseline systematic review and NMA. Phase III activates the living framework through public website launch and documentation. Phase IV implements the ongoing surveillance and update cycle. Each phase is described below.

The baseline systematic review and NMA follow current best practices [14–16] and are described in detail elsewhere [8]. Key methodological elements are summarized here.

The ongoing surveillance and update cycle consists of six iterative steps, repeated at regular intervals. To ensure methodological consistency, each step applies the identical methods used in the baseline NMA.

The successful completion of the first update cycle demonstrates that a living NMA framework is practical to implement and sustain. The initial NMA in the setting of CVI showed that PEM was not significantly different statistically from ETA for venous closure or deep venous thrombosis [8]. These results remained unchanged in the first update cycle, despite the addition of seven new studies (54% increase in the total number of studies; less than 1% increase in total patient numbers due to several large real-world studies in the initial search). Nearly 2 years post-publication, the top-line conclusions remain unchanged, which should not be seen as wasted effort in a living NMA. On the contrary, robust findings upon subsequent updates, supported by a transparent and prospectively determined process, can increase confidence in the evidence base. These findings demonstrate consistency and durability of clinical outcomes across the spectrum of provider sites of service and physician techniques utilized over more than two decades of clinical practice.

Despite demonstrated theoretical feasibility [4], practical implementation of living NMAs has remained limited. Several barriers impede widespread adoption. Methodological consistency must be maintained across update cycles to ensure that evolving results reflect new evidence rather than analytical drift. Much attention has been paid to maintaining rigor in living systematic reviews and meta-analyses through use of prospective protocols [13,34], but fewer sources address the need for payment models to support this continued rigor. Much like long-term randomized clinical trials and clinical registries, sustained resource commitment is required for ongoing surveillance, screening, data extraction and analysis – activities that extend indefinitely beyond traditional project timelines. Grant-based funding mechanisms, which typically support discrete projects with defined end points, can be poorly suited to this continuous maintenance model without budget lines to finance these future updates. Additionally, a dissemination mechanism is needed that provides stakeholders with timely access to updated findings while maintaining scientific credibility. This work to maintain transparency cannot be an afterthought and also requires substantial resource commitments. While conventional systematic reviews and network meta-analyses summarize evidence at a single time point, their findings may lose relevance in rapidly evolving therapeutic areas. Living NMA addresses this limitation through continual evidence surveillance and predefined updates, maintaining the currency and usefulness of comparative estimates for clinical and policy decision-making, at significantly lower costs than alternative methodologies. Clinical registries offer prospective, real-world data collection but require substantial infrastructure investment, multi-site coordination, extended enrollment timelines and ongoing regulatory oversight. Living NMA offers a complementary alternative that synthesizes existing published comparative evidence without the need for prospective enrollment or site-level infrastructure. By continuously incorporating new evidence as it emerges from real-world practice and clinical trials alike, a living NMA can provide timely, updatable comparative effectiveness estimates at a fraction of the cost and time required to establish and maintain a dedicated registry. This approach is particularly well-suited to clinical areas such as CVI treatment, where a growing body of published comparative evidence already exists and new studies continue to emerge.

Continuation beyond the initial 3-year commitment will be assessed against three criteria specified at the outset: the rate at which new eligible evidence is generated, the stability of effect estimates across completed update cycles, and the continued clinical relevance of the research question. A declining rate of new evidence combined with stable estimates across successive cycles would indicate that the evidence base has matured and that less frequent updating, or transition back to static status, is appropriate. Should funding end before those conditions are met, the posted analyses, documentation and study list would remain publicly accessible with the search cut-off date displayed, allowing users to assess currency and permitting another group to resume updating from a documented position.

Industry sponsorship offers one potential solution to the sustainability challenge. Manufacturers of medical technologies have ongoing interest in current, rigorous comparative effectiveness evidence for their products, particularly when that evidence informs appropriate clinical use and supports payer medical policy decisions. Scholars, however, have repeatedly raised legitimate bias and independence concerns about industry-sponsored medical research [35,36]. Of particular interest to this project are the Veroniki et al. findings that pharmacologic industry-sponsored NMAs reach favorable results more frequently and report model details less completely compared with matched non-industry-sponsored NMAs [36]. These concerns motivated the safeguards in this framework which combine prospective protocol registration, pre-specified methods, peer-reviewed publication and transparent public dissemination of all results and documentation – creating conditions for accountability that make bias difficult to conceal. In this implementation, the sponsor committed to funding ongoing updates before the baseline NMA was completed, ensuring resources independent of initial findings. The protocol was registered in PROSPERO, and the baseline NMA was published in a peer-reviewed journal, with intent to update results stated a priori. Project documentation – including eligibility criteria, study citations, study attrition diagrams and forest plots – is posted publicly. This transparency enables stakeholders to evaluate the methodology and reach their own conclusions.

Elliott and colleagues outlined the rationale for living systematic reviews, emphasizing their value when evidence is rapidly evolving and critical clinical and policy decisions are being made [2]. Créquit and colleagues demonstrated that living NMA is theoretically feasible, with update workload representing approximately 10% of baseline effort and most clinical questions generating fewer than four new eligible trials per year [4]. This current study is consistent with these predictions in terms of the rate of new evidence: The first update cycle added seven studies over approximately 2 years of surveillance, a manageable evidence generation rate. The framework operationalizes what Créquit showed was possible and adds practical, step-by-step guidance. This study supports recommendations from various national and international agencies that analyses used for clinical decision-making should be informed by the best and most up-to-date evidence, generated using systematic approaches to evidence assessment [11].

To preserve scientific integrity, roles were prospectively defined. Under the funding agreement, the sponsor reviews analyses and manuscripts prior to public posting or submission, and sponsor input during this review is advisory. Decisions regarding study selection, statistical methods, effect estimates and interpretation rest with the analytic team and the author group, and no such decision was altered at the sponsor’s direction during the baseline analysis or the first update cycle. Because the protocol was registered prospectively, the methods pre-specified, and the complete study list and documentation posted publicly, departure from the prespecified analysis or withholding of an unfavorable result would be externally detectable.

Statistical code was maintained by the analytic team, and methodological decisions were documented in the registered protocol prior to review initiation. All authors had access to the data and participated in interpretation and manuscript development. These governance safeguards aim to reduce risk of sponsor-driven analytical drift while enabling sustained resource commitment.

Several limitations should be acknowledged. The framework depends on a single sponsor; sustainability would be threatened if that commitment ended. Multisponsor or consortium models might provide greater stability but would introduce coordination challenges and additional resource needs. The current implementation relies on published data only. Unpublished studies or individual patient data meta-analyses using registry data may exist that are not able to be included. Head-to-head studies and common comparators for PEM and ETA both remain limited, resulting in sparse data for some comparisons and preventing most sensitivity or subgroup analysis. The framework has been tested in one clinical domain; generalizability to other conditions, particularly those with different evidence generation rates, requires further evaluation. The approximately annual website update frequency, while practical, may not capture rapidly emerging evidence in fast-moving fields. Finally, effort/time was not tracked prospectively at the task level during the baseline analysis or first update cycle, and workload comparisons are therefore based on screening and extraction volume rather than person-time. We recommend that groups implementing a new living evidence syntheses in the future record task-level effort from the outset, as such data would substantially strengthen the evidence base on the sustainability of living methods.

Marshall and colleagues demonstrated that artificial intelligence (AI) tools can assist with living review surveillance, achieving high recall with reduced screening burden [37]. Because this four-phase living NMA framework relies on prespecified eligibility criteria, stable statistical methods, recurring literature surveillance and transparent public dissemination, AI may reduce operational burden by supporting literature surveillance, citation screening, data extraction, study classification, PRISMA documentation and preparation of recurring update materials. However, using AI as an augmentation tool with rigorous human oversight is critical, especially during framework development. The most credible future model is therefore a human-governed, AI-assisted living NMA process in which AI improves efficiency and consistency for repetitive tasks, while clinical, methodological, and statistical experts retain responsibility for framework design, evidence validation, model interpretation and final conclusions.

Validation of this human-governed, AI-assisted living NMA process could proceed by parallel operation. In a subsequent update cycle, AI-assisted screening and extraction would be run alongside the established human process, with human decisions retained as the reference standard. Performance would be assessed by recall for eligible studies at Level I screening, agreement with human decisions at Level II and field-level concordance for extracted data. Where AI and human outputs disagree, the human determination would govern, and discrepancies would be reviewed to identify systematic error patterns. Because a living framework repeats an identical workflow across cycles against a human-adjudicated record, it provides an unusually direct setting for this comparison. Adoption of any AI-assisted step would be contingent on demonstrated performance in such a parallel cycle rather than on assumed efficiency gains.

Another anticipated future opportunity is the inclusion of additional CVI clinical outcomes, including improvement in symptoms and quality of life and potentially valuable sensitivity or subgroup analysis, as the evidence base matures. More generally, the model could be extended to other clinical domains with multiple treatment options and ongoing evidence generation, such as comparative effectiveness of devices or biosimilars. Groups establishing similar frameworks would also benefit from specifying, at the outset, a mechanism for resolving disagreement between the analytic team and sponsor regarding results or their interpretation. Finally, linkage to living clinical practice guidelines could translate continuously updated evidence into actionable recommendations for clinicians and medical policy determinations.