Increasing transparency in indirect treatment comparisons: is selecting effect modifiers the missing part of the puzzle? A review of methodological approaches and critical considerations

Failure to adjust for effect modifiers (EMs) in indirect treatment comparisons (ITCs) can produce biased and uncertain effect estimates. This is particularly important for health technology assessments (HTAs), where the availability of new treatments is based on comparative effectiveness results. Much emphasis has been placed on advancing ITC methods to adjust for EMs, yet whether EMs are appropriately identified for the conduct of ITCs in the first place is unclear. To understand the extent of guidance and requirements for the selection of EMs for ITCs currently available and if and how this guidance is applied in practice, a series of pragmatic reviews of guidance documents from HTA and non-payer organizations, primary published ITC analyses, and prior HTA submissions in two indications (non-small cell lung cancer and psoriasis) was conducted. The reviews showed that current ITC guidance mainly focused on developing analytical methods to adjust for EMs. Some organizations, such as HTA bodies in the UK, France and Germany, recommended the use of literature reviews, expert opinion and statistical methods to identify EMs. No detailed guidance on the selection process or the appropriate literature review approach was found. Similar trends were identified through the database search and review of prior HTA submissions; only few published ITCs and submissions included information on the EM selection process which was either based on findings from the literature, trial subgroup analyses, or clinical input. No reference to a systematic selection approach was found.

There is an urgent need to fill the guidance gap identified across the reviews by including a step in ITC guidelines on how EMs should be identified through systematic reviews, formal expert elicitation, and a quantitative assessment of the EM distribution. Researchers and manufacturers are also encouraged to improve transparent reporting and justification of their selection of EMs to allow for an independent review of the set of factors being considered for adjustment. Both will contribute toward reducing bias in the ITC results and ultimately increase confidence in decision-making.

The lack of guidance on how to identify effect modifiers (EM) for indirect treatment comparisons (ITC) in health technology assessments (HTA) led to a considerable underreporting of the EM selection process in published ITCs and HTA submissions. Further HTA guidance is needed.

Certain variables that can affect the outcomes of new therapies must be accounted for to avoid bias when comparing the effectiveness of new and existing treatments. This is especially important in indirect treatment comparisons that are included in submissions to agencies which recommend whether a medicine should be financed through the local healthcare system. Previous research has focused on how to adjust for these variables to create an unbiased evaluation, but little information exists on how to appropriately identify them in the first place. A review of guidance documents from reimbursement bodies and other relevant publications demonstrated that there was no detailed guidance on the selection process or a systematic approach to handle this issue. These findings highlight an urgent need to develop guidance that will reduce bias in indirect treatment comparisons and increase confidence in the evidence needed to make new therapies available to the public.