Open access

Systematic Review

21 October 2021

Antibacterial data synthesis challenges: a systematic review of treatments for complicated Gram-negative urinary tract infections

Authors: Ryan Dillon https://orcid.org/0000-0002-6817-327X [email protected], Jennifer Uyei https://orcid.org/0000-0003-4397-8554, Rajpal Singh https://orcid.org/0000-0001-9760-2152, and Eilish McCann https://orcid.org/0000-0001-7156-9390Author Info & Affiliations

Publication: Journal of Comparative Effectiveness Research

Volume 10, Number 18

https://doi.org/10.2217/cer-2021-0138

PDF

Abstract

Aim: To determine the suitability of network meta-analysis (NMA) using antibacterial treatment evidence in complicated urinary tract infection. Materials & methods: We conducted a systematic literature review to identify published clinical trial data for complicated urinary tract infection treatments. We performed a feasibility assessment to determine whether the available evidence would support the creation of a robust NMA, considering key assumptions of homogeneity, similarity and consistency. Results: Twenty-five trials met eligibility criteria. Risk of bias was low, and individual studies met their primary end point(s). Assumptions central to the conduct of a robust NMA were not met. Heterogeneity was ubiquitous, including baseline pathogen, treatment and patient characteristics. Conclusion: Limited and heterogeneous data identified make the use of NMA to compare novel antibacterial agents impractical and likely unreliable.

Urinary tract infections (UTIs) are burdensome and one of the most common bacterial infections in the USA [1]. Clinically, UTIs are categorized as uncomplicated or complicated, with factors including genitourinary abnormalities, underlying disease (e.g., stone disease, diverticulum, neurogenic bladder), immunocompromised status, indwelling catheters and/or infection with multidrug-resistant pathogens being associated with complicated UTIs (cUTIs) [2]. According to the Premier Healthcare Database, 70,298 adults were hospitalized for cUTI between 2013 and 2015 in the USA [3]. A meta-analysis of US literature showed that catheter-associated UTIs were the third most common healthcare-associated infections (HAIs) at acute care hospitals, with an incidence rate of 1.87 cases per 1000 device days [4]. Among all HAIs, cUTIs account for approximately 40% of all severe sepsis cases, including urosepsis, which can have a mortality rate as high as 40% [5]. Additionally, cUTIs are frequently associated with complications including renal scarring and end-stage renal disease [5].

cUTIs are frequently caused by Gram-negative pathogens, predominantly Enterobacterales (e.g., Escherichia coli and Klebsiella pneumoniae) or Pseudomonas aeruginosa [6]. Data from the National Healthcare Safety Network indicate that Gram-negative pathogens are responsible for more than 30% of HAIs and 45% of UTIs in the US [7].

The treatment of a cUTI is typically empiric (i.e., antibacterial treatment is selected prior to results from microbiological and antibacterial susceptibility testing of the culture obtained from the patient), guided by an awareness of local ecology, antimicrobial resistance data, clinical presentation of the patient and known risk factors for a particular pathogen or resistance status [8]. Once, the infecting pathogens have been identified from the obtained culture, therapy may be adjusted if the empiric therapy was deemed to be inappropriate [9–12]. Selecting an appropriate therapy is critical in order to minimize the potential for the development of antibacterial resistance and to optimize patient outcomes, with meta-analyses demonstrating that initial inappropriate antibacterial therapy increases the risk of mortality and increases costs and duration of hospital stay compared with appropriate therapy [13].

Widespread emergence of antibacterial resistance has compromised the effectiveness of many available cUTI treatments and has created a significant unmet need for alternative safe and effective therapies, particularly for infections caused by Gram-negative pathogens [14]. In order to address this unmet need rapidly, policy efforts have allowed promising drug candidates to be designated as a streamlined development program initiative. This designation permits smaller, more rapid randomized controlled trials (RCTs) to be conducted and to support regulatory approval. Regulatory agencies have also developed guidance for the conduct of RCTs to substantiate the efficacy and safety of new antibacterial agents, including specific guidance to aid the investigation of treatments for cUTIs [15]. As a consequence of these regulatory-specific requirements, the available evidence typically only establishes the noninferiority of competing interventions; this is also complicated by the use of different noninferiority threshold values, and different standard of care therapies. The reliance on noninferiority RCTs is in part due to the nuanced and evolving nature of infectious disease (e.g., selection of an appropriate/relevant comparator, changing ecology [time and geography], and/or key patient and pathogen characteristics known to affect clinical outcomes). As a result, the available data may not be sufficient to support other forms of decision-making (e.g., the development of clinical guidelines or health technology assessments [HTA]).

In particular, HTA agencies seek to determine the comparative effectiveness of new intervention(s) versus relevant comparators by leveraging methods such as indirect comparison by means of a network meta-analysis (NMA) [16]. The validity of NMA is based on three key assumptions: homogeneity (the absence of relevant heterogeneity between trials assessing the same intervention), similarity (the inclusion of studies based on rigorous criteria reporting similar populations, outcomes and effect modifiers in an attempt to minimize bias) and consistency (no relevant discrepancy or inconsistency between direct and indirect comparative estimates between the same interventions) across those studies being incorporated into a putative evidence network [17–19]. This requires the inclusion of evidence that are sufficiently similar (i.e., minimal heterogeneity exists in the clinical [patient population, baseline characteristics, interventions and outcome ascertainments] and methodological [study design, conduct or quality] criteria) [20,21].

We hypothesized that it may be challenging to use the available evidence base for cUTI antibacterial treatments to support decision-making beyond the initial regulatory approval; specifically, the creation of a robust NMA to support decision makers in their consideration of value and access. To explore this, we conducted a systematic literature review (SLR) of published clinical evidence and performed a feasibility assessment to determine whether the creation of a connected evidence network is plausible in an attempt to assess the comparative effectiveness of antibacterial agents used in the management of cUTI.

Materials & methods

This SLR was conducted according to a prespecified protocol following guidelines published by the Cochrane Collaboration and PRISMA group [22,23]. The SLR aimed to identify Phase II and III RCTs and controlled clinical trials that assessed antibacterial therapy for the treatment of cUTI in adults requiring hospitalization. A comprehensive search of EMBASE^®, MEDLINE^®, MEDLINE In-Process^®, the Cochrane Central Register of Controlled Studies and Cochrane Database of Systematic Reviews was conducted via OVID^® on 5 April 2017, and then updated on 17 May 2019. Searches were limited to English-language papers. The intent of the 2019 search was to identify additional trials that were published since the original search, using the same search strategy and eligibility criteria as the original SLR. The complete search strategy can be found in Supplementary Tables 1 & 2. Database searches were supplemented by additional free-text searches of the European Congress of Clinical Microbiology and Infectious Diseases (ECCMID), Infectious Diseases Society of America (ID Week) and American Society for Microbiology Microbe (ASM Microbe) conference proceedings held in 2018 and 2019.

Two reviewers evaluated study eligibility reviewing the title, abstract and full text against the eligibility criteria defined by population, intervention, comparator, outcome, time and study design (Supplementary Table 3). Differences between reviewers were resolved through reconciliation or arbitration by a third reviewer. Distiller^® and EndNote^® were used to manage references and the screening process. Data were extracted from studies and entered into a predesigned, pilot-tested data extraction form. For each study, key methodological characteristics, patient characteristics and outcomes were extracted and, when possible, calculated. Data were extracted from eligible papers by one reviewer and independently checked against the source document by a second.

The risk of bias in individual studies was assessed for all included trials using the Cochrane Collaboration’s tool for assessing risk of bias [22]. The internal and external validity of studies was assessed based on selection bias, performance bias, measurement of detection bias, attrition bias and reporting bias (Supplementary Table 4).

A full feasibility assessment was conducted using data from included studies to assess whether it was possible to create an evidence network of interlinked studies for the outcome of clinical response (as defined by each trial). Second, any differences in study and/or patient characteristics across included studies that are likely or known modifiers of the relative treatment effects of the interventions of interest were assessed. The effect modifiers assessed included parameters of study design and patient (demographic and clinical), pathogen (type and susceptibility status) and treatment characteristics (dosing schedule and position in therapy) [16].

Results

Systematic literature review

The total evidence base for this SLR comprises 37 publications, representing 25 unique studies (Supplementary Figure 1). The 2017 SLR identified 15 unique studies (15 publications). The SLR update conducted in April 2019 identified 22 citations representing 14 unique studies that met eligibility criteria. Of these, four studies had already been identified in the initial SLR; therefore, a total of ten additional unique studies were identified during the 2019 SLR update, with additional publications identified for previously included studies.

The eligible studies (n = 25) assessed three major efficacy outcomes in patients with cUTI: clinical response (n = 22), microbiological response (n = 22), and composite clinical and microbiological response (n = 14). As clinical outcomes are typically used to drive HTA discussions related to value, and form key inputs within economic models; we focused on the 22 studies reporting clinical response as a test case for our feasibility assessment.

Of the 22 studies reporting clinical response, a low risk of bias for the majority of domains was determined and, thus, the overall quality of studies was deemed to be high. Four studies [24–27] demonstrated high risk of performance bias and two studies demonstrated high risk of detection bias due to nonblinding of participants, physicians and outcome assessors [24,26]. Supplementary Table 4 summarizes the risk of bias assessment of the included studies.

Assessing the available evidence base

We evaluated the available evidence from 22 studies that reported the clinical response outcome (Table 1). Across studies, a marked level of heterogeneity was observed. This was present in terms of study design (analysis population, statistical analysis methods, geography), patient characteristics (risk factors, underlying comorbidity), treatment characteristics (empiric or confirmed, dose, schedule), disease parameter(s) (severity, causative pathogen, susceptibility profile), outcome definition and assessment timepoint. Table 1 reports the definitions and criteria for clinical response as reported within the original publication(s). It is apparent that the definition for this outcome is inconsistent between studies. Furthermore, there is additional complexity added by the definitions of the analysis population and outcome assessment timepoint (e.g., the baseline patient characteristics were typically reported for the overall population, whereas outcomes were assessed for various subgroups of the overall population such as the modified intention-to-treat population; Table 2 & Supplementary Table 5). There were many sources of heterogeneity, and often there were multiple components within each study, causing a multiplicative effect.

Table 1. Study characteristics.

Study (year)	Country/region	Intervention, dose and dosing interval	Comparator, dose and dosing interval	Empiric/ confirmed	Clinical assessment	Definition of clinical assessment	Ref.
Wagenlehner et al. (2019), EPIC N = 604	Multinational	Plazomicin 15 mg/kg iv. o.d.	Meropenem 1000 mg iv. q8h	Empiric	Clinical cure	Reduction in severity (at day 5 and at the end of intravenous therapy) or complete resolution (at the test-of-cure visit) of all core symptoms with no new symptoms or as a return to the patient's status before development of the UTI, with no use of nontrial antibiotics for the current cUTI	[38]
Kaye et al. (2019), ZEUS N = 464	Multinational	ZTI-01 6000 mg iv. q8h	Piperacillin 4000 mg/tazobactam 500 mg iv. q8h	Confirmed	Clinical cure	Investigator defined: Yes Complete resolution or significant improvement of signs and symptoms such that no further antimicrobial therapy is warranted	[32]
Wunderink et al. (2018), TANGO II N = 72	USA, Argentina, Brazil, Colombia, Greece, Israel, Italy, United Kingdom	Meropenem 2000 mg/ vaborbactam 2000 mg iv. q8h	Best available therapy (as described)	Empiric and confirmed	Clinical cure	Complete resolution of signs/symptoms of the index infection such that no further antimicrobial therapy (and/or surgical intervention for cIAI) was warranted	[27]
Wagenlehner et al. (2018), N = 223	Germany, Poland	Finafloxacin 800 mg iv./oral o.d.	Ciprofloxacin 400 mg or 500 mg oral b.i.d.	NR	Clinical response	Complete resolution of cUTI or resolution of cUTI to baseline symptoms with no new symptoms developing	[37]
Motsch et al. (2020), RESTORE-IMI 1 N = 41	Multinational	Imipenem-cilastatin 500 mg/relebactam 250 mg iv. q6h	Colistimethate sodium (colistin) loading dose 300 mg, maintenance 150 mg iv. q12h, plus imipenem 500 mg iv. q6h	Confirmed	Favorable clinical response	Resolution of baseline signs and symptoms	[28]
Portsmouth et al. (2018), APEKS-cUTI N = 452	Multinational	Cefiderocol 2000 mg iv. q8h	Imipenem 1000 mg/cilastatin 1000 mg iv. q8h	Confirmed	Clinical response	Investigator defined: Yes Resolution or improvement of cUTI infection symptoms present at study entry and the absence of new symptoms	[34]
Mir et al. (2018), PLEA N = 230	NR	Ceftriaxone 1000 mg/ sulbactam 500 mg/ EDTA 37 mg iv. q12h	Meropenem 1000 mg iv. q8h	NR	Clinical cure	Complete resolution of signs/symptoms such that no further antimicrobial therapy was required	[41]
Kaye et al. (2018), TANGO I N = 550	Multinational	Meropenem 2000 mg/vaborbactam 2000 mg iv. q8h	Piperacillin 4000 mg/tazobactam 500 mg iv. q8h	Empiric or confirmed	Clinical cure	Complete resolution or significant improvement of baseline signs and symptoms of complicated UTI or acute pyelonephritis	[31]
Connolly et al. (2018), N = 145	US, India, Columbia, Chile	Plazomicin 10 or 15 mg/kg iv. o.d.	Levofloxacin 750 mg iv. o.d.	NR	Clinical cure	Complete resolution of all baseline signs and symptoms at TOC without the use of additional antibiotic therapy or an AE leading to premature discontinuation of the study drug	[30]
Chaudhary et al. (2018), N = 204	India	Ceftriaxone/sulbactam/disodium EDTA 3000 mg iv. o.d.	Ceftriaxone 2000 mg iv. o.d.	Empiric	Clinical assessment	Improvement in the signs and symptoms was performed throughout the treatment regimen	[25]
Sims et al. (2017), N = 302	Multinational	Imipenem 500 mg/cilastatin 500 mg/relebactam 125 mg or 250 mg iv. q6h, subsequent ciprofloxacin after 96 h of iv. treatment	Imipenem 500 mg/cilastatin 500 mg + placebo iv. q6h, subsequent ciprofloxacin after 96 h of iv. treatment	Empiric or confirmed	Favorable response ‘cure’	Resolution all or most pretherapy signs and symptoms of the index infection or returned to ‘preinfection status’ and no additional antibiotic therapy (beyond iv. study therapy and oral ciprofloxacin) was required	[35]
Wagenlehner et al. (2016), RECAPTURE 1 & 2 N = 1033	Multinational	Ceftazidime 2000 mg/avibactam 500 mg iv. q8h	Doripenem 500 mg iv. q8h	Confirmed	Clinical cure	All or most pretherapy signs and symptoms of the index infection had improved or resolved such that no additional antibiotics (not protocol specified) were required	[39]
Carmeli et al. (2016), REPRISE N = 333	Multinational	Ceftazidime 2000 mg/avibactam 500 mg q8h	Best available therapy (as described)	Confirmed	Clinical response	Complete resolution or substantial improvement of signs and symptoms of the index infection, such that no further antibacterial therapy (other than those allowed per protocol) was necessary	[24]
Wagenlehner et al. (2015), ASPECT-cUTI N = 1083	Eastern Europe, North America, South America, India, Israel, South Africa, South Korea, Thailand	Ceftolozane/tazobactam 1500 mg iv. q8h	Levofloxacin 750 mg iv. o.d.	Confirmed	Clinical cure	Complete resolution, substantial improvement (i.e., reduction in severity of all baseline signs and symptoms and worsening of none) or return to preinfection signs and symptoms of cLUTI or pyelonephritis without the need for additional antibiotic therapy	[40]
Vazquez et al. (2012), N = 137	US, Guatemala, India, Jordan, Lebanon	Ceftazidime 500 mg/ avibactam 125 mg iv. q8h	Imipenem/cilastatin 500 mg iv. q6h	Confirmed	Clinical response	Resolution of all or most pretherapy signs and symptoms with no further requirement for antibiotics	[36]
Sandberg et al. (2012) N = 248	Sweden	Ciprofloxacin 500 mg oral b.i.d.	Ciprofloxacin 500 mg oral b.i.d. + placebo	Empiric	Clinical cure	Complete resolution of symptoms during treatment with no recurrence of symptoms or signs of UTI during follow-up	[44]
Naber et al. (2009), DORI-05 N = 753	North America, South America, Europe	Doripenem 500 mg iv. q8h	Levofloxacin 250 mg iv. q8h	Confirmed	Clinical cure	Investigator defined: Yes Signs or symptoms of cLUTI or pyelonephritis (fever, flank pain, costovertebral angle tenderness, suprapubic pain, dysuria and urgency or frequency of urination) improved or resolved or returned to preinfection status (if known) and if nonstudy antibacterial therapy had not been administered for the treatment of the baseline urinary infection	[33]
Peterson et al. (2008) N = 1102	US	Levofloxacin 750 mg iv. or oral o.d.	Ciprofloxacin 400 mg iv. or 500 mg oral BD	Confirmed	Clinical cure	Investigator defined: Yes Resolution of preclinical signs and symptoms without additional antibacterial therapy	[42]
Klausner et al. (2007) N = 311	US	Levofloxacin 750 mg iv. or oral o.d.	Ciprofloxacin 400 mg iv. or 500 mg oral BD	Confirmed	Clinical cure	Investigator defined: Yes Resolution of pretreatment clinical signs and symptoms without additional antibacterial therapy	[43]
Carmignani et al. (2005) N = 257	Italy, France	Prulifloxacin 600 mg oral o.d.	Ciprofloxacin 500 mg oral b.i.d.	Confirmed	Clinical cure or improvement	End point reported, but not clearly defined	[29]
Naber et al. (2002) N = 337	NR	Piperacillin 4000 mg/tazobactam 500 mg iv. q8h	Imipenem 500 mg/cilastatin 500 mg iv. q8h	Confirmed	Clinical cure	Resolution of all signs and symptoms of the UTI without any further antibacterial treatment, improvement as residual symptoms present, but no pain in the upper urinary tract and no necessity for further antibacterial treatment	[45]
Hou et al. (2002) N = 7	China	Meropenem 500 mg or 100 mg iv. b.i.d.	Imipenem 500 mg or 1000 mg/cilastatin 500 mg or 1000 mg iv. b.i.d.	Empiric or Confirmed	Clinical cure	Complete resolution of signs and symptoms of infection, eradication of the pathogens and normal laboratory test results. Marked improvement, patient improving in ≥3 of four criteria (resolution of signs and symptoms of infection, eradication of the pathogens and normal laboratory test results)	[26]

AE: Adverse event; b.i.d.: Twice daily; cIAI: Complicated intra-abdominal infection; cLUTI: Complicated lower urinary tract infection; cUTI: Complicated urinary tract infection; iv.: Intravenously; NR: Not reported; o.d.: Once daily; q6h: Once every 6 h; q8h: Once every 8 h; TOC: Test of cure; UTI: Urinary tract infection.

Table 2. Study definitions.

Study (year)	Definition of cUTI	Definition of clinical response	Timepoint for clinical response	Definition of population	Ref.
Wagenlehner et al. (2019) EPIC	≥2 signs or symptoms (chills, rigors or fever [temperature >38°C]; dysuria or urinary frequency or urgency; lower abdominal or pelvic pain; or nausea or vomiting) and ≥1 complicating factor (a history of urinary retention [in male patients], current indwelling urinary catheter, obstructive uropathy or any functional or anatomical abnormality)	Clinical cure was defined as a reduction in severity (at day 5 and at the end of intravenous therapy) or complete resolution (at the test-of-cure visit) of all core symptoms with no new symptoms or as a return to the patient’s status before development of the UTI, with no use of nontrial antibiotics for the current cUTI	Day 5, EOT, TOC, LFU (days 24–32)	mMITT: Patients had ≥1 qualifying baseline pathogen (≥10⁵ CFU/ml) that was susceptible to both meropenem (MIC of ≤1 μg/ml) and plazomicin (MIC of ≤4 μg/ml)	[38]
Kaye et al. (2019) ZEUS	≥2 signs or symptoms, urine specimen with evidence of pyuria and one associated risk factor	Clinical cure, investigator defined as complete resolution or significant improvement of signs and symptoms such that no further antimicrobial therapy is warranted	TOC, LFU	MITT: Patients who met ITT criteria and received any amount of study drug mMITT: Patients who met MITT criteria and had ≥1 baseline Gram-negative pathogen from an appropriately collected pretreatment baseline urine or blood sample CE: Patients who met inclusion and exclusion criteria, received ≥9 doses of study drug, had the EOT, TOC, LFU visits occur within the window and did not have an indeterminate clinical response at the specified visit ME: Patients who met mMITT criteria and CE criteria and had an appropriately collected urine culture specimen and interpretable urine culture result at the EOT, TOC and LFU visits, respectively	[32]
Wunderink, 2018 TANGO II	NR	Clinical cure was defined as complete resolution of signs/symptoms of the index infection such that no further antimicrobial therapy (and/or surgical intervention for cIAI) was warranted	EOT, TOC	mMITT: Received ≥1 dose of study drug and had a baseline qualifying Gram-negative pathogen mCREMITT: Patients who received ≥1 dose of study drug and had a baseline qualifying isolate confirmed as CRE by local or central laboratory	[27]
Wagenlehner et al. (2018)	NR	Clinical response was defined as complete resolution of cUTI or resolution of cUTI to baseline symptoms with no new symptoms developing	Visit 2 (day 3), EOT (day 10), TOC (day 17), day 24	MITT: Microbiological intent to treat	[37]
Motsch et al. (2020), RESTORE-IMI CSR, 2018	NR	Favorable clinical response was defined as resolution of baseline signs and symptoms	EOT, day 3, EFU (days 5–9)	mMITT: Patients were eligible if pathogens were imipenem-NS (but CST- and IMI/REL-susceptible) based on central lab MIC SmMITT: Comprised mMITT plus all patients who met inclusion criteria only based on local lab MIC	[28]
Portsmouth et al. (2018), APEKS-cUTI	FDA diagnostic criteria: clinical syndrome characterized by pyuria and a documented or suspected microbial pathogen on culture of urine or blood, accompanied by local and systemic signs and symptoms, including fever (temperature ≥38°C), chills, malaise, flank pain, back pain or costovertebral angle pain or tenderness that occurred in the presence of a functional or anatomical abnormality of the urinary tract or in the presence of catheterization (catheter had to be replaced when necessary), and required iv. treatment	Clinical response, assessed by the investigator, defined as resolution or improvement of complicated urinary tract infection symptoms present at study entry and the absence of new symptoms	Days 3–5, EOT, TOC (7 ± 2 days), follow-up (14 days)	MITT: All randomly assigned individuals who received ≥1 dose of study drug and had a qualifying Gram-negative uropathogen (≥1 × 10⁵ CFU/ml)	[34]
Mir et al. (2018), PLEA	NR	Clinical cure was defined as a complete resolution of signs/symptoms such that no further antimicrobial therapy was required	TOC	mMITT: Patients with ≥1 baseline Gram-negative pathogen at ≥10⁵ CFU/ml, susceptible to both study drugs	[41]
Kaye et al. (2018), TANGO I	≥2 signs or symptoms and one associated risk factor	Clinical cure was defined as complete resolution or significant improvement of baseline signs and symptoms of cUTI or acute pyelonephritis	EOT	MITT: All patients who received one or more doses of the study drug mMITT: All patients in the MITT population with ≥1 bacterial pathogens of 10⁵ CFU/ml or more in baseline urine culture or the same bacterial pathogen present in concurrent blood and urine cultures	[31]
Connolly et al. (2018)	The presence of pyuria (≥5 WBC per high-power field in urine sediment and/or a positive leukocyte esterase test on urinalysis) and ≥1 of the following signs or symptoms: fever (oral temperature ≥38.5°C or ≥101.3°F), elevated WBC count (≥10,000/mm³ or a left shift of ≥15% immature polymorphonuclear leukocytes), dysuria, increased urinary frequency, urgency or lower abdominal pain	Clinical cure is the complete resolution of all baseline signs and symptoms at TOC without the use of additional antibiotic therapy or an AE leading to premature discontinuation of the study drug	TOC (5–12 days)	MITT: Randomized patients with ≥1 isolated causative bacterial pathogen at ≥10⁵ CFU/ml from an appropriately collected pretreatment urine specimen ME: Clinically evaluable patients with a causative pathogen isolated at the baseline and results obtained from a noncontaminated urine culture collected at TOC within a specified window (5–12 days after the end of treatment, except in cases of early failure, in which the TOC visit was performed early)	[30]
Chaudhary et al. (2018)	At least two of the following signs and symptoms: fever (>38°C), dysuria, costovertebral angle tenderness or suprapubic tenderness, flank pain, increased urinary frequency and urgency, pyuria, and bacterial colony count of ≥10⁵ CFU/ml	Clinical assessment for improvement in the signs and symptoms was performed throughout the treatment regimen	EOT (6–15 days), TOC (16–25 days), LFU (23–32 days)	MITT: Received ≥1 dose of study medication and the population was used for safety analysis mMITT: Patients with a confirmed cUTI diagnosis and with growth of one or no more than two Gram-negative uropathogens of ≥10⁵ CFU/ml in urine culture CE: mMITT patients who received therapy for ≥48 h, with ≥80% of the scheduled drug administered or received therapy <48 h before discontinuing treatment due to an AE, had no protocol deviations that would affect the assessment of efficacy ME: Patients in the CE population who had a microbiological outcome response at TOC and LFU visits and had no protocol deviations	[25]
Sims et al. (2017)	≥2 specific signs and symptoms of UTI and ≥1 risk factor	Favorable response ‘cure’ reported at different follow-up periods was defined as resolution of all or most pretherapy signs and symptoms of the index infection or returned to ‘preinfection status’ and no additional antibiotic therapy (beyond iv. study therapy and oral ciprofloxacin) was required	DCiv. (5–14 days), LFU (28–42 days)	ME: Excluded patients with protocol deviations that could substantially affect results of the primary efficacy end point (i.e., not meeting protocol definitions of cUTI/acute pyelonephritis; prestudy urine culture failing to grow ≥1 Gram-negative and/or anaerobic pathogen at sufficient quantity; inclusion/exclusion violations impacting efficacy assessment; receiving <96 h of iv. study therapy) MITT: Patients who received ≥1 dose of iv. study therapy and had a prestudy urine culture growing ≥1 Gram-negative and/or anaerobic pathogen at any quantity	[35]
Wagenlehner et al. (2016), RECAPTURE 1 & 2	cUTI without pyelonephritis was defined as presence of ≥2 symptoms, including ≥1 UTI-specific symptom (dysuria, urgency, frequency and suprapubic pain with onset/worsening within the previous 7 days) as well as ≥1 complicating factor	All or most pretherapy signs and symptoms of the index infection had improved or resolved such that no additional antibiotics (not protocol specified) were required	EOT, TOC, LFU	mMITT: Patients with symptomatic resolution (or return to premorbid state) of UTI-specific symptoms, except flank pain, with resolution or improvement in flank pain from baseline at the day 5 visit (based on the PSAQ); patients with both microbiological eradication and symptomatic resolution (or return to premorbid state) of all UTI-specific symptoms at TOC (21–25 days postrandomization)	[39]
Carmeli et al. (2016), REPRISE	NR	The complete resolution or substantial improvement of signs and symptoms of the index infection, such that no further antibacterial therapy (other than those allowed per protocol) was necessary	TOC (7–10 days)	mMITT: All patients who received ≥1 dose of the study drug who had a diagnosis of complicated urinary tract infection or complicated intra-abdominal infection with ≥1 ceftazidime-resistant Gram-negative pathogen	[24]
Wagenlehner et al. (2015), ASPECT-cUTI	Fever (oral temperature higher than 38°C) accompanied by rigors, chills or warmth; flank pain; costovertebral angle or suprapubic tenderness on physical examination; or nausea or vomiting plus suprapubic pain, dysuria, urinary frequency or urgency, and ≥1 of the following: male sex with urinary retention, indwelling urinary catheter, current obstructive uropathy or any functional or anatomical urogenital tract abnormality	The complete resolution, substantial improvement (i.e., reduction in severity of all baseline signs and symptoms and worsening of none) or return to preinfection signs and symptoms of complicated lower urinary tract infections or pyelonephritis without the need for additional antibiotic therapy	EOT, TOC, LFU	MITT: Patients who received ≥1 dose of study drug mMITT: All patients in the MITT population with growth of one or two uropathogens of ≥10⁵ CFU/ml in urine culture	[40]
Vazquez et al. (2012)	Confirmed by signs and symptoms consistent with a UTI (defined as body temperature >37.8°C orally, >38.2°C by tympanic measurement or >38.4°C by rectal measurement, chills, flank pain, costovertebral angle tenderness, dysuria, urgency, frequency, incontinence, suprapubic pain, nausea or vomiting), pyuria (≥10 WBC/mm³) and a positive urine culture (≥10⁵ CFU/ml of a uropathogen presumed or known to be susceptible to study drugs). Male patients with other cUTIs were also required to have a diagnosis of acute prostatitis excluded by physical examination. Female patients with other cUTIs were also required to have a history or clinical evidence of one or more urological abnormalities (indwelling or intermittent use of bladder catheter, instrumentation of urinary tract, including urogenital surgery in the 7 days prior to study entry) and/or functional or anatomical abnormalities of the urinary tract	Clinical response was defined as resolution of all or most pretherapy signs and symptoms with no further requirement for antibiotics	EOT, TOC, LFU	ME: Patients who had no major protocol violations, had a positive urine culture on enrollment that contained ≥10⁵ CFU/ml (>10⁴ CFU/ml if bacteremic) of ≥1 uropathogen presumed or known to be susceptible to the study antibiotics, had a clinical and microbiological assessment at the TOC visit (including a quantitative urine culture) and had either received ≥7 days of study therapy (iv. or iv. plus oral) or were classified as failures after completing ≥48 h of iv. therapy CE: Patients who had clinical evidence of cUTI, were compliant with study drug therapy (received ≥7 days of study antibiotic) or were classed as an evaluable clinical failure after completing ≥48 h of iv. study therapy and had a clinical outcome assessment at the TOC visit MITT: Patients who received ≥1 dose of study medication and who had a pretreatment urine culture containing >10⁵ CFU/ml of ≥1 uropathogen	[36]
Sandberg et al. (2012)	Diagnosis of community-acquired acute pyelonephritis: fever of ≥38°C (measured at home or in the emergency department) and ≥1 symptom or sign relating to the urinary tract such as flank pain, costovertebral angle tenderness, dysuria, urgency or frequency	Clinical cure was defined as complete resolution of symptoms during treatment with no recurrence of symptoms or signs of urinary tract infection during follow-up	Days 10–14, LFU (42–63 days)	PP: Patients were not eligible for per-protocol analysis for the following reasons: no follow-up visit; systemic treatment with other antimicrobial drugs up to day 28 (visit 3); or missing more than one dose of the study drug during the first week of treatment or more than two doses during the whole treatment period	[44]
Naber et al. (2009), DORI-05	≥1 UTI symptom (dysuria, frequency, suprapubic pain or urgency) and ≥1 complicating factor (male gender, current bladder instrumentation or indwelling catheter that was to be removed during iv. study drug administration, obstructive uropathy that was to be medically or surgically treated during iv. study drug administration, urogenital surgery within 7 days of the first dose of study drug, or a functional or anatomical urogenital tract abnormality with a voiding disturbance)	Clinical cure, assessed by the investigator, if signs or symptoms of cLUTI or pyelonephritis (fever, flank pain, costovertebral angle tenderness, suprapubic pain, dysuria and urgency or frequency of urination) improved or resolved or returned to preinfection status (if known) and if nonstudy antibacterial therapy had not been administered for the treatment of the baseline urinary infection	EOT iv., TOC (5–11 days), LFU (28–42 days)	mMITT: Patients with baseline urine culture results qualifying them for participation in the study, regardless of the susceptibility of the baseline uropathogen(s) to either study drug ME: Patients met the protocol definition of cUTI, had a bacterial uropathogen (≥10⁵ CFU/ml) isolated from a baseline urine culture, were compliant with study drug therapy or were classified as an evaluable failure after completing ≥3 days of iv. study drug therapy, had no significant protocol deviation from the inclusion–exclusion criteria or in-study procedures (including previous or concurrent antibiotics), and had an interpretable urine culture result at the specified visit	[33]
Peterson et al. (2008)	≥1 complicating factor: neurogenic bladder or urinary retention; partial obstruction, renal tumor or fibrosis, distorted urethral structure; and/or intermittent catheterization	Clinical cure, assessed by the investigator, defined as resolution of preclinical signs and symptoms without additional antibacterial therapy	EOT, PT	MITT: All subjects with an appropriate clinical diagnosis of AP or cUTI and who had a positive (≥10⁵ CFU/ml) urine culture and one or two uropathogens at study entry, regardless of other analysis criteria ME: Patients met additional major study criteria (e.g., not lost to follow-up; no dosing deviations; appropriate evaluation dates for the PT visit)	[42]
Klausner et al. (2007)	Positive for leukocyte esterase, and/or ≥5 WBC/high-power field on examination of centrifuged urine sediment, and/or ≥10 WBC/mm³ of uncentrifuged urine and ≥1 of the following symptoms: nausea, vomiting, dysuria, increased urinary frequency compared with a historical baseline and/or urgency	Clinical cure, assessed by the investigator, defined as resolution of pretreatment clinical signs and symptoms without additional antibacterial therapy	EOT (5–7 days), PT (10–14 days)	MITT: MITT population included all ITT subjects who had a clinical diagnosis of AP and who had a positive urine culture (≥10⁵ CFU/ml) and ≤2 uropathogens at study entry ME: MITT subjects who additionally met all other major evaluability criteria	[43]
Carmignani et al. (2005)	The presence of indwelling catheter, intermittent catheterization, residual urine ≥50 ml after voiding, prostatic hypertrophy, obstructive uropathy, vesicoureteral reflux or other urologic abnormalities was eligible in the trial	Clinical cure or improvement end point reported, but not clearly defined	Days 5–7	ITT: All the randomized patients with baseline urine culture exhibiting an infecting strain >10³ CFU/ml and final microbiological assessment PP: All the randomized patients with an infecting strain ≥10⁵ CFU/ml and pyuria, treatment compliance ≥80%, not assuming concomitantly other antibacterial agents and with the final assessments	[29]
Naber et al. (2002)	Defined by the presence of complicating factors and by ≥1 of the following symptoms: fever >38°C, pain in the upper urinary tract, pain in the lower urinary tract (perineal pain, urethralgia, pain in the vesical region) and micturition difficulties (dysuria, urgency, frequency)	Clinical cure was defined as resolution of all signs and symptoms of the UTI without any further antibacterial treatment, improvement as residual symptoms present, but no pain in the upper urinary tract and no necessity for further antibacterial treatment	Days 1–2, TOC (5–9 days), weeks 4–6	NR	[45]
Hou et al. (2002)	NR	Clinical cure: patient showing complete resolution of signs and symptoms of infection, eradication of the pathogens and normal laboratory test results. Marked improvement: patient improving in ≥3 of four criteria (resolution of signs and symptoms of infection, eradication of the pathogens and normal laboratory test results)	EOT	NR	[26]

AE: Adverse event; AP: Acute pyelonephritis; CE: Clinically evaluable; CFU: Colony-forming unit; cIAI: Complicated intra-abdominal infection; cLUTI: Complicated lower urinary tract infection; CRE: Carbapenem-resistant Enterobacterales; CST: Cilastatin; cUTI: Complicated urinary tract infection; DCiv.: Discontinuation of intravenous therapy; EFU: Early follow-up; EOT: End of therapy; IMI: Imipenem; ITT: Intention to treat; iv.: Intravenous; LFU: Late follow-up; mCREMITT: Microbiologic carbapenem-resistant Enterobacterales modified intent to treat; ME: Microbiologically evaluable; MIC: Minimum inhibitory concentration; mITT: Modified intention to treat; mMITT: Microbiological modified intent to treat; NR: Not reported; NS: Nonsusceptible; PP: Per protocol, PSAQ: Patient symptom assessment questionnaire; PT: Post-therapy; REL: Relebactam; SmMITT: Supplemental microbiological modified intent to treat; TOC: Test of cure; UTI: Urinary tract infection; WBC: White blood cell.

In addition to the aforementioned challenges, there are external factors unique to the antibacterial therapeutic space, such as evolving disease biology, geographic location, changes in clinical practice, trial design requirements, changing diagnostic criteria and variability in local resistance patterns. Although recognized within the literature, the impact of these confounding factors has not been assessed, nor is it possible to explore their effect as per the data presented within this study.

The majority of evidence from the 22 studies reporting the clinical response outcome was obtained from multinational trials (n = 15) [24,27–40], with additional studies from individual countries (India, n = 2 [25,41]; US, n = 2 [42,43]; China, n = 1 [26]; and Sweden, n = 1 [44]); study location was not reported in one study [45]. Sample size also varied across studies, from 7 to 1102 patients [26,42].

Studies differed by whether the tested interventions were used in an empiric or a confirmed setting and by treatment characteristics (Table 1). Eleven of the 22 studies [24–26,28–38,42–45] reported treatment in the confirmed setting with a known pathogen at baseline, while three studies [27,39,40] reported the use of empiric therapy (Table 1). A single study by Mir et al. did not report any information relating to the baseline pathogen and, therefore, it was not possible to determine whether treatment was empiric or confirmed [41]. Twelve studies investigated antibacterial monotherapies [25,26,29,30,32–34,37,38,42–44], and ten studies investigated combination therapies (two or more antibacterial agents) [24,27,28,31,34–36,39,40,45]. Furthermore, there was variability among the same intervention in terms of treatment dose and schedule within and across studies (e.g., plazomicin 10 mg/kg [30] or 15 mg/kg once daily [39] and ceftazidime 500 mg/avibactam 125 mg q8h [36] or ceftazidime 2000 mg/avibactam 500 mg q8h) [39].

Differences and/or the absence of data relating to the severity of cUTI, patient renal status and the top three causative pathogens at baseline are summarized in Figure 1. The proportion of patients with acute pyelonephritis at baseline varied considerably, ranging from 13.3 to 100.0% [44,45]. Patient renal status was unknown in the majority of studies. However, nine studies using various methods reported renal status at baseline, and/or the proportion of patients with normal renal function or mild impairment (creatinine clearance ≥50 ml/min) ranged from 30.9 to 93.0% [38,40]. Causative baseline pathogen was available for 16 of the 22 studies reporting clinical response. Of those, E. coli was most frequently reported (N = 14) [24,25,27,29,31–36,39,40,43,44], with proportions in each study that ranged from 9.3 to 93.6% [27,36]. Other commonly reported causative bacteria were K. pneumoniae [24,25,27–29,31–35,39,40,44] and P. aeruginosa [28,34–36,39].

Figure 1. Complicated urinary tract infection renal status and pathogen distribution by study.
CrCl: Creatine clearance; cUTI: Complicated urinary tract infection; NR: Not reported.

NMA feasibility assessment

Assuming a best-case network using the studies (N = 22) reported in Table 1 for the outcome ‘clinical response,’ a fully connected network would be possible. However, prior to the formation of a network, it is important to consider the available evidence with regards to the key principles of NMA.

First, there are limited studies that could contribute to the formation of a potential network. Table 1 demonstrates that only a single study would be available for each comparison between the different interventions identified in the SLR, and thus the assumption of homogeneity typically assumed for pooled estimates in a pairwise comparison is not relevant [18,46]. Second, the assumption of similarity between studies cannot be supported. The available evidence highlights considerable variation among reported effect modifiers and patient characteristics, including outcome definition, outcome assessment timepoint, trial population, baseline pathogen, proportion of patients with acute pyelonephritis and treatment characteristics (dose and schedule; Table 1, Figure 1 & Supplemental Table 5); all of these factors, individually or cumulatively, would be expected to impact or modify the treatment effect observed within each trial. This variability of likely effect modifiers between studies may create biased comparisons between therapies. Unfortunately, as there is only a single study contributing to each treatment node within the potential evidence network, methods such as meta-regression or bias adjustment cannot be used to evaluate the impact of the reported effect modifiers, while maintaining a connected network. Finally, due to a lack of closed loops within this potential network, it is not possible to assess the assumption of consistency, in other words, the consensus of direct and indirect treatment effect estimates.

Given the violation of the applicable key assumptions, it does not seem possible to create a robust NMA using the identified data for the treatment of cUTI. The validity, generalizability and clinical relevance of any findings from such a network would be questionable.

Discussion

We conducted an SLR and feasibility assessment to determine the validity of an NMA to compare the clinical effectiveness of antibacterial agents for the treatment of patients with cUTI. The credibility of an NMA relies on the assumptions of homogeneity, similarity and consistency across the included studies. Unfortunately, the published evidence identified suggests that none of these assumptions would be met or, in some cases, could not be assessed. Of particular concern were the multiple sources of heterogeneity, which challenge the formation of a scientifically robust network. If a network were to be established, regardless of this, it is unlikely that results would be meaningful or generalizable to the clinical setting or form a solid base on which to make decisions on which treatments to adopt into clinical practice and for which to provide reimbursement.

All identified and included studies discussed here met their primary efficacy end points of noninferiority. Studies showed a low risk of bias, and the overall quality of the individual studies was deemed high. Differences were observed in study design, patient characteristics, treatment characteristics, disease parameters, outcome definition and assessment timepoint, and causative pathogen. Studies also differed with respect to treatment dosing and schedule; differences were even observed within and across multiple studies evaluating the same intervention.

Understanding that the available data, when considered in isolation, do satisfy the requirements of regulatory authorities but may pose a challenge for traditional data synthesis methods relied upon by HTA bodies and/or payor and reimbursement groups, is worrisome. Although the value of NMA as a means of comparative effectiveness is well established, the aforementioned challenges associated with the antibacterial evidence base has the potential to generate spurious results that could undermine the value of novel antibacterials. Furthermore, reliance on outcome of such methodology could lead to negative access and decisions that detract from the ethos of antimicrobial stewardship. In an era of growing antimicrobial resistance, a lack of novel antimicrobial agents being developed, and rising patient need, decision makers may need to assess the value of these important agents differently, and without relying on indirect evidence comparisons.

Challenges similar to those described in our study have been documented in the literature [47–54]. A 2013 SLR and meta-analysis of studies reporting on patients with cUTI was conducted to inform the future development of noninferiority trials [53]. The authors discussed a number of limitations including reporting biases due to the presence of heterogeneity, inconsistencies within the reported study measure(s), end points and the recognition of limited evidence that may hamper a full assessment of other potential sources of heterogeneity [53].

Similarly, reviews across other infection types, such as complicated intra-abdominal infection, identified variability among multiple data points including but not limited to definitions for the main clinical outcome (clinical response/failure) [47,49,51] and details relating to treatment dose and schedule [49]. The authors have also commented on external pressures unique to the antibacterial space, such as the evolving pathogen ecology, resistance/susceptibility profiles and the variability that exists within treatment protocols in different geographic regions, all of which can impair the generalizability of findings [48–50,54]. Finally, a recent poster by Reason et al. commented that traditional methodologies for NMA present significant limitations when applied to antimicrobials and that ways to overcome these challenges are important [54].

There were a few limitations in this SLR and feasibility assessment. First, searches were conducted in 2017 and 2019 to identify articles reported in English only, which could mean that potentially relevant articles were missed. Although this SLR may not have included the most recent literature for cUTI, the challenges described herein are likely to remain. Second, studies were assessed using a reputable risk of bias assessment tool, and although shown to be of high quality with a low risk of bias, some studies had missing information and could not be evaluated comprehensively. Additionally, we did not limit or exclude any references based on the quality assessment findings; this may have introduced elements of bias across multiple domains including outcome assessment, incomplete data and participant blinding. Also, although there were several differences across studies, there were a number of potential effect modifiers that were not well reported. These unreported effect modifiers could have an impact on clinical outcomes and lead to further heterogeneity and inconsistency in a potential network.

Conclusion

The results of this study provide a comprehensive overview of the published RCT and controlled trial literature for patients with Gram-negative cUTI treated with an antibacterial agent. Heterogeneity in the evidence base suggests that the use of NMA when comparing the clinical effectiveness of antibacterial agents used in the management of Gram-negative cUTI is impractical and likely unreliable. An incomplete and heterogeneous evidence base, such as this, should not be used to pool evidence in order to make inferences of superiority, particularly when the individual trials are typically designed to assess noninferiority or to support the preferential use of specific antimicrobials, as this runs counter to the ethos of good antimicrobial stewardship. Furthermore, the perceived suitability of these antibacterial data, using an established NMA methodology and without careful consideration of the clinical and microbiological environment, could prove detrimental to patient outcomes. This inability to draw meaningful conclusions from an established methodology such as NMA highlights the deep-rooted issues associated with antibacterial therapeutic trial designs that are considered suitable from a regulatory standpoint, but are problematic for inclusion into the traditional comparative effectiveness framework favored by HTA bodies in this context.

Future perspective

Access to novel antibacterial therapy for patients with, or who are at risk of resistant infection(s) is paramount. The importance of effective antibacterial therapy goes beyond the patient treated and can impact local ecology and rates of resistance. Realistically, it may not be possible or appropriate to expect alignment in the evidence-generation process due to the distinct role of regulators and HTA reimbursement bodies [55,56]. Instead, efforts to recognize the suitability of clinical trial data for comparative effectiveness within this therapeutic space should be considered on a case-by-case basis. The availability of real-world evidence (RWE) also provides decision makers with supplemental data relevant to their individual decision problem(s). Recognizing that RWE may not sit at the top of an evidence hierarchy, careful attention must be paid to the inclusion of these and/or any trial data when making determinations relating to access, value and reimbursement. The use of RWE may make it possible to enhance and/or contextualize clinical trial data using data specific to local epidemiology, pathogen susceptibility data and/or patient risk factors.

Summary points

•

Widespread emergence of antibacterial resistance has reduced the effectiveness of many treatments for complicated urinary tract infections (cUTI) and has created a significant unmet need for safe and effective therapies, particularly for infections caused by Gram-negative pathogens; however, the evidence required by regulatory agencies to support the use of a given treatment does not always align with the needs of health technology assessment agencies.

•

The findings of this systematic literature review suggested that data for treatment of patients with Gram-negative cUTI are limited and heterogeneous, thereby reducing the utility and reliability of a network meta-analysis to compare novel antibacterial agents as typically required by reimbursement and health technology assessment agencies concerned with comparative clinical effectiveness versus relevant comparators.

•

Real-world evidence may better contextualize clinical trial data to assess the value of novel antibacterial agents, in that these data are specific and relevant to local epidemiology, pathogen susceptibility and/or patient risk factors.

Author contributions

R Dillon, E McCann and J Uyei participated in the conception and design of the study. All the authors helped analyze and interpret the data, participated in critical revision of the manuscript, approved the final version for publication and agree to be accountable for all aspects of the work. They are responsible for the work described in this paper. They were involved in at least one of the following: conception, design of work or acquisition, analysis, interpretation of data, and drafting the manuscript and/or revising/reviewing the manuscript for important intellectual content. They provided final approval of the version to be published. They agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Financial & competing interests disclosure

This work was supported by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., NJ, USA. R Dillon is an employee of Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Kenilworth, NJ, USA (MSD). E McCann was an employee of MSD at the time of study conduct. J Uyei is an employee of IQVIA, Inc. (CA, USA) and R Singh is an employee of IQVIA, Inc. (Mumbai, India). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Medical writing and/or editorial assistance were provided by Arianna Nevo dcxof IQVIA, Inc., CA, USA. This assistance was funded by Merck Sharp & Dohme Cop., a subsidiary of Merck & Co., Inc., NJ, USA.

Open access

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/

Supplementary Material

File (supplemental material.docx)

Download
262.01 KB

References

Papers of special note have been highlighted as: • of interest

Medina M, Castillo-Pino E. An introduction to the epidemiology and burden of urinary tract infections. Ther. Adv. Urol. 11, 1756287219832172 (2019).