Letter to the Editor: indirect treatment comparison of anifrolumab efficacy versus belimumab in adults with systemic lupus erythematosus

An important property of an ITC is that the patient populations are comparable [2]. Where this is not the case, but there is sufficient overlap of the populations, an adjusted analysis can be performed [3]. The authors are aware of this issue, citing ‘insufficient overlap in patient characteristics’ to reject otherwise eligible belimumab studies from the analysis [1]. However, while attempting to ‘mitigate the impact of between trial heterogeneity’ [1], the authors have overlooked the critical issue of insufficient overlap of the narrower TULIP population with the broader BLISS population to allow a robust adjustment to be performed.

The differences between the SLE populations in the TULIP versus BLISS studies are important: almost all patients in the TULIP studies had severe disease (British Isles Lupus Assessment Group [BILAG] index 1A and/or ≤2B), predominantly in the mucocutaneous and musculoskeletal domains [4,5]; in contrast, the BLISS studies included a broader SLE cohort with involvement beyond skin and joints [6–8]. This difference may be explained by the more restrictive inclusion criteria of the TULIP studies [4,5], resulting in an SLE population with more skin and joint involvement at the expense of patients with other serious organ involvement such as hematological and renal.

From the data presented in the paper [1], it is evident that the TULIP population is under-represented in the broader, more characteristic SLE population recruited to the BLISS studies. The most obvious illustration comes from the matching-adjusted indirect comparison (MAIC):

It is unclear how the feasibility assessment criteria were applied to each study [1]. This is critical since the analysis is highly dependent on the choice of studies. To date, belimumab has been evaluated in ten randomized controlled trials (including five placebo-controlled phase III trials in SLE), in long-term extensions and multiple post-marketing efficacy and safety studies [9]. As of April 2021, >7200 patients with SLE had received belimumab in clinical studies [9]. Yet the breadth of belimumab evidence has been excluded from the analysis [1]. Of particular note is the exclusion of the phase III BLISS-SC study (as noted by the authors as bioequivalent) [10,11]; no explanation is offered why the primary analysis is restricted to the belimumab intravenous formulation. BLISS-SC was included in the supplementary analysis [11] and, notwithstanding our overall concerns, these results change the paper’s conclusions since there was no significant difference between treatments across any outcomes. It is concerning that there is no comment on this, given the conclusion from the primary analysis.

There is no discussion in the paper that the BLISS and TULIP studies used different BILAG index versions [4,5,7,8]. This impacts not only the availability of effect modifiers common to both trials but also, potentially, the comparison of outcomes incorporating BILAG (SRI-4 response and BILAG flare at week 52).

The selection of effect modifiers is critical to the validity of the simulated treatment comparison (STC) [3]. The authors identified 14 effect modifiers, reducing to 12 to enable the MAIC [1]. However, the selection is unclear, supported by an unreferenced literature review of the predictors of SLE treatment outcomes. The authors note that prognostic variables are not required in the STC/MAIC and reflect on the challenges of differentiating between prognostic variables and effect modifiers in their discussion. This makes the absence of reporting of the ‘extensive sensitivity analyses… to explore the impact of adjusting for different subsets of TEMs’ troubling [1].

The authors assert their preference for STC methods based on recent research suggesting superior performance in eliminating bias when compared with MAIC. However, there are other simulation studies published recently that detail scenarios (similar to that investigated here; conducting the adjustment in presence of poor population overlap, comparing a conditional to marginal effect for the STC, and a small effective sample size in the MAIC) where, even in the presence of effect modifiers, both STC and MAIC can produce results with substantial bias and, in some cases, produce results that are less accurate than a standard ITC [12,13].

It has been demonstrated that MAIC analyses perform poorly when populations have limited overlap, generating results that, ‘…may even increase bias compared with standard indirect comparisons’ [12–14]. The authors acknowledge this concern and hence base the primary analysis on the STC [1]. Given this known bias, we find it curious that the MAIC results in the paper are highly consistent with the STC results. We believe rather than validating the robustness of the STC, as claimed by the authors, the MAIC results actually demonstrate that the STC is equally subject to bias arising from limited population overlap.

The pooled unadjusted OR for anifrolumab versus placebo for SRI-4 (Table 3; and likely used in the analysis) appears to be an overestimate [1]. The 1.63 OR is higher than what would be expected based on a pooling of the prespecified results in the primary publications (1.33) or if the revised post hoc definition for TULIP-1 is considered (1.56).

The authors adjust for between trial heterogeneity with regard to the TULIP and BLISS studies but appear to ignore heterogeneity between the two TULIP studies [4,5,7,8]: SRI-4 response favored placebo in TULIP-1 but anifrolumab in TULIP-2. This has implications for the plausibility of the STC/MAIC results and leads to overly narrow 95% CIs. Pooling the study results of TULIP-1 and TULIP-2 gives an OR of 1.33 (CI 0.99–1.79 fixed effects/CI 0.54–3.25 random effects). This is non-significant, in clear contrast to the statistically significant benefit for belimumab on SRI-4 response in the individual and pooled BLISS studies.

To perform the STC, the authors adjust for 12 baseline characteristics considered to be effect modifiers despite that only four from their selection are substantially different between BLISS and TULIP. Arguably increased disease activity (BILAG Index A and B, SLEDAI score) would be expected to result in a larger treatment effect for both belimumab and anifrolumab [15,16]. Thus, it is counterintuitive that this analysis generates a larger anifrolumab treatment effect in a lower disease activity population.

In summary, this ITC fails to meet fundamental requirements for a robust population adjusted analysis: that there is sufficient overlap that adjustments can be made [2,3]. BLISS-SC is a relevant study, and the rationale for its exclusion is not clearly articulated. We have highlighted other examples where the methodology may have introduced bias and lacks transparency. The cumulative effect of these issues leads to conclusions that are not in line with observations based on the results of the individual clinical studies. Without appropriate discussion of these limitations, we consider the findings of this paper are unhelpful to appropriate decision-making in the choice of SLE treatments.

All authors contributed equally to the development of this letter.

The production of this letter to the editor was funded by GSK. All authors are employees of GSK. N Ballew, RA Levy and M Bradley hold stocks and shares in GSK. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Editorial and submission support was provided by H Taylor, Fishawack Indicia Ltd., UK, part of Fishawack Health and was funded by GSK.

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/