Letter to the Editor: comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis

We read with interest the original article entitled ‘Comparison of ofatumumab and other disease-modifying therapies for relapsing multiple sclerosis: a network meta-analysis’ by Samjoo et al. [1]. Based on this network meta-analysis (NMA), the authors claim that for this population, for all outcomes (annualized relapse rate and confirmed disability progression [CDP] at 3 and 6 months [CDP-3 and CDP-6]), ofatumumab was as effective as other highly efficacious monoclonal antibody disease-modifying therapies (DMTs), i.e., alemtuzumab, natalizumab and ocrelizumab. NMA is a useful method for informing comparative effectiveness research; however, caution must be exercised when using certain methods and interpreting the results. To this end, we would like to highlight several limitations regarding this article, which we believe may lead to misinterpretation of the NMA methodology and final results, and hence, could be potentially misleading to the reader.

The authors present base case evidence network diagrams of the possible comparisons for evaluated outcomes. A key assumption of the NMA is that there is no difference between the trials in the distribution of effect-modifying variables; however, greater distances in the network diagrams increases the chance of an imbalance in effect modifiers, or a within-trial bias in one of the links. In the network diagrams for time to CDP-3 and time to CDP-6 (Figures 3 & 5, respectively; Samjoo et al. [1]), the distance in the networks between ofatumumab and ocrelizumab/alemtuzumab is large; in Figure 5, the shortest path from ofatumumab to ocrelizumab/alemtuzumab is via a chain of at least five pairwise links. Unfortunately, these large distances pose several limitations, which are not highlighted by the authors; importantly, they increase the risk of violating the key assumptions of the NMA, which could introduce bias into the effect estimate of ofatumumab versus ocrelizumab/alemtuzumab, i.e., the greater the distance between two treatments in the network, the less reliable the estimate is. Furthermore, in networks consisting of only one or two trials per treatment comparison, as per the majority included in the networks in Samjoo et al. [1], indirect comparisons are highly vulnerable to systematic variation resulting from imbalances in effect-modifier distributions [2]. As acknowledged by the authors, population-adjusted indirect comparisons, such as Matching-Adjusted Indirect Comparison or Simulated Treatment Comparison [2], represent alternatives to an NMA, which may help to address these limitations. Population-adjusted indirect comparisons use individual patient data from one or more trials to adjust for between-trial differences in the distribution of variables that influence outcome. These can adjust for imbalances in measured-effect modifiers and do not use intermediate linking studies, hence, can provide a valid estimate of relative treatment effects when standard indirect comparisons are either inappropriate, unreliable or infeasible. It must be acknowledged, however, that such analyses rely on their own set of strong assumptions that should also be carefully evaluated.

We would also like to highlight our concern regarding lack of details around the rederivation of the CDP outcomes. The NMA includes CDP data from ASCLEPIOS [3] that has been aligned with the CDP definition from the OPERA I and OPERA II trials [4] (‘OPERA-aligned CDP’); however, the authors failed to mention several limitations of these OPERA-aligned analyses. Notably, making outcome definitions more consistent across trials can reduce bias; however, the OPERA-aligned CDP definition could only be applied to the ASCLEPIOS trial, meaning that the several linking trials between ASCLEPIOS and OPERA did not have this consistent definition applied. Moreover, the NMA protocol containing a definition of the post-hoc OPERA-aligned CDP does not appear to have been prospectively registered, and the authors failed to mention the different variations of the definition that were evaluated; therefore, the variations in the OPERA-aligned CDP analyses cannot be reconciled, as it is not known how many variations of the aligned definitions were analyzed to model the change in hazard ratio prior to the final definition that was included in the NMA. The post-hoc OPERA-aligned CDP analyses for ASCLEPIOS have been previously presented only at investor relations calls [5–7], but these were not included in the article’s reference list or in the systematic literature review. As per NICE guidelines (Guide to the methods of technology appraisal 2013 [8]), attempts should be made to identify unpublished and part-published evidence that is not in the public domain, as well as evidence from sources other than randomized controlled trials. Such information must be critically appraised and, when appropriate, sensitivity analysis conducted to examine the effects of its incorporation or exclusion. Notably, the OPERA-aligned input data were not critically appraised; therefore, the quality of the data may be disputed. Furthermore, sensitivity analyses were not carried out to assess the effects of including the OPERA-aligned CDP data in the NMA; thus, the results cannot be interpreted reliably and may be misleading, particularly when presented in the main body of evidence. To avoid any potential misinterpretations, the effect of including the OPERA-aligned CDP data should ideally be assessed using sensitivity analyses, which included input data for all DMTs using the OPERA-aligned definition and for all DMTs using the ASCLEPIOS-aligned definition; however, we acknowledge that without access to individual patient data this is not possible. The authors should recognize this limitation in the discussion.

Finally, we believe that in order to make a more balanced conclusion, the authors should have reported the safety end points in the main article alongside the efficacy outcomes, as this would have allowed the readers to assess and understand the benefit–risk profile of each DMT.

We agree with the authors’ statement that effective treatment of patients with relapsing multiple sclerosis requires an understanding of the comparative efficacy and safety of different DMTs; payers and other decision makers highly rely on NMAs; therefore, we believe that these important transparency concerns should be addressed and key limitations highlighted, so that the results of the NMA can be correctly interpreted to help guide treatment decisions.