An economic and health outcome evaluation of telehealth in rural sepsis care: a comparative effectiveness study

This study is a multicenter retrospective propensity-matched comparative effectiveness study of tele-ED use for age-qualifying Medicare beneficiaries who present to a rural ED with sepsis from 2017 through 2019. We have identified hospitals that subscribe to a single large tele-ED provider as intervention hospitals, and we will identify similar hospitals without tele-ED services as control hospitals. This study will be reported using the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) [23,24].

All age-qualifying (age ≥65 years) Medicare beneficiaries with at least one rural hospital ED visit followed by hospital admission for sepsis between 1 January 2017 and 31 December 2019 will be identified in Medicare beneficiary administrative claims data and included in the analysis. We defined a cohort of non-federal rural EDs that comprised all rural EDs subscribing to the single tele-ED service (n = 166, Table 1), and we will match these hospitals in a 1:2 ratio with a set of similar control hospitals. All included hospitals will be general medical and surgical hospitals with an on-campus ED in the rural USA. Hospitals will be classified as rural if their Rural Urban Commuting Area Code designates them as one of the categories of: large rural, small rural, or isolated, based on the hospital zip code [25,26]. Figure 1 shows the geographic distribution of intervention hospitals.

In our study, we will define sepsis according to International Classification of Diseases, 10th edition (ICD-10) diagnosis codes (Supplemental Appendix 1). We will use an inclusive definition of sepsis, using both explicit and implicit (coexisting infection and organ failure codes) diagnosis codes from hospital inpatient discharge [27]. Further, we will then require that the corresponding rural ED visit includes either an explicit diagnosis code of sepsis or infection, to exclude hospital-acquired cases of sepsis (Figure 2). We will also conduct a sensitivity analysis using only cases diagnosed with severe sepsis, with or without shock. Cases will be included only if the initial ED visit occurs in hospitals identified as part of our initial cohort of hospitals (either tele-ED intervention hospitals or matched control hospitals); however, inpatient admission may have occurred at the same or a different hospital after inter-hospital transfer. We will exclude patients with hospital length-of-stay of greater than 30 days. Considering that this study relies on administrative claims data alone, no clinical or physiologic severity of illness criteria will be applied.

All tele-ED-capable hospitals for this study are those hospitals that subscribe to a single tele-ED provider. Avera eCARE, based in Sioux Falls, South Dakota, is the largest tele-ED service provider in North America. The provider-to-provider tele-ED service provides 24-h high-definition on-demand video telehealth services to subscribing hospitals. Local clinical staff connect virtually with an ED nurse and board-certified emergency physician by activating a button on the ED wall. Tele-ED staff have access to the local electronic medical record, and they make recommendations on clinical care, provide nursing charting, physician order entry and coordination of care for inter-hospital transfers. eCARE uses standardized call log software with decision-support, screening protocols, and checklists designed to enable adherence with treatment guidelines. In 2017, eCARE implemented a standard sepsis screening and treatment tool for use in tele-ED subscribing hospitals. This process recommended that triage nurses using a standard SIRS-based electronic trigger, and it recommended tele-ED activation for sepsis screen-positive cases with hypotension, altered mental status or organ failure evidenced on standard sepsis laboratory screening [28].

Patients and the public have not contributed to the study design. They also will not be engaged in the implementation of the study.

Data for this project will be obtained from telehealth provider data, Medicare claims and hospital-level data:

Our cohort will ultimately be generated by matching in a 1:2 ratio all identified tele-ED-capable hospitals subscribing to the Avera eCARE service between 2017 and 2019 with non-tele-ED control hospitals using variables from the AHA Survey and US Census data, matching only to hospitals confirmed not to have another telehealth provider based on the NEDI-USA data. The justification for 1:2 matching is that tele-ED cases in participating hospitals are limited, so we want to maximize power for analytic efficiency and sample size reduction while excluding hospitals that are qualitatively different.

For this study, we are taking the perspective of the payer. Medicare reimbursements are expected to approximate actual costs of hospital care. Reimbursement will be obtained from the Medicare Claims data.

Total costs will be measured between hospital admission and 30 days after hospital discharge (to align with the Medicare readmission metrics). Mortality rates will be assessed at 90 days post-hospital admission.

We will calculate all costs in 2019 US dollars (2019 $USD). Costs will use a discount rate calculated from the Consumer Price Index for Medical Care, as it contains medical care services and medical care commodities and reflects temporal changes in the cost of medical care broadly [37].

Additional variables are defined for use in the propensity score and risk-adjustment models. For hospital-level matching, potential covariates will be identified based on theory, and we will include variables in the final model that have an association (p < 0.20) with Avera tele-ED subscription. We plan to use the following hospital-level covariates:

For patient-level severity of illness adjustment, we will use the following covariates:

This analysis will be conducted as a complete case analysis. No imputation techniques will be used. Medicare beneficiaries not included in the call log will be assumed not to have had telehealth used.

We hypothesize that the use of tele-ED for sepsis patients in rural hospitals will lead to reduced costs and more favorable long-term outcomes. This pathway is hypothesized to result from more timely high-quality resuscitation care (early appropriate antibiotics, fluid resuscitation, improved use of vasoactive medications, rapid diagnostic studies) associated with tele-ED use that leads primarily to reductions in organ failure. Prior data have shown that early appropriate antibiotics are associated with a lower rate of subsequent septic shock [40]. We expect that avoiding organ failure will lead to fewer complications, fewer procedures, fewer ICU admissions, shorter length-of-stay in the ICU and hospital and fewer discharges to skilled nursing facilities. We also expect cost-savings to result from decreased debility, fewer readmissions and improved functional outcomes at discharge.

Because it is likely that the highest severity patients are the ones that receive tele-ED care, our primary analysis will be done at the level of the hospital assignment. We assume that severity of illness will be similar across rural hospitals, which minimizes the effect of selection bias from only the most severe patients being treated with tele-ED. This assumption is necessary because there is no physiologic severity of illness measure provided in the claims data. We hypothesize that having tele-ED available will lead to better aggregate outcomes in the hospital-level analysis, and that the effect size would be even larger in a risk-adjusted patient-level analysis.

We assume that Medicare reimbursement is a close approximation of cost of care, and we assume that all healthcare for Medicare beneficiaries is billed to Medicare. We assume that all deaths are reported to and validated by Medicare with accurate death dates. In obtaining our cohort of matched hospitals, we assume that the covariates most useful in identifying similar hospitals are those that are the most strongly associated with a hospital subscribing to tele-ED services. While we recognize that tele-ED is likely used for patients with greater illness severity [41], we assume that the aggregate illness severity across all the patients treated in a participating hospital is similar between hospitals. This assumption is based on our prior work showing that rural hospital bypass is rare in sepsis care [42].

In a sample of tele-ED CAHs (n = 18) in the upper Midwest that subscribed to Avera tele-ED, we found that these rural EDs had a mean of 16 sepsis cases annually [43]. Extended to our cohort of 500 hospitals for Medicare beneficiaries only (estimated 60% of sepsis cases), this would result in an estimated 14,400 sepsis cases over the years of 2017–2019 [44]. Using Medicare cost data, with 80% power, this would allow for a minimal detectable difference of US$878 (α = 0.05, μ = US$28,242, σ = $17,719, allocation ratio = 2), with the mean case cost of US$28,242.

Through prior work demonstrating the association of tele-ED use in treating sepsis with improved short-term outcomes, the long-term perspective on costs is important. Prior quality improvement initiatives that optimize early sepsis care have been shown to decrease healthcare costs, through decreased health resource utilization through improved clinical outcomes [45,46]. Furthermore, earlier sepsis diagnosis is also associated with decreased cost [47]. Because the tele-ED intervention is expected to improve adherence with sepsis guidelines very early in treatment, we expect that tele-ED will also be associated with reduced need for ICU care, less discharge debility, improved long-term survival and decreased index hospitalization costs, all of which are expected to lead to a decrease in overall expenditures [48]. We expect to see a modest decrease in expenses in our primary hospital-level analysis. If we see that our patient-level analysis shows a more substantial decrease, that finding will further support our hypothesis that telehealth use in sepsis care is associated with less disability and lower costs. If our patient-level analysis shows that tele-ED-enabled cases have higher expenses, we will conclude that our severity of illness measure was imperfect, but we will use the secondary outcomes to better understand the mechanism of the effect (e.g., categories in which expenses were different). If we see that expenses are higher in our primary analysis and that mortality is lower, we will conclude that increased costs are related to improved survival – a patient-relevant and important explanation for the overall effect observed (Table 3). Figure 3 outlines the analysis proposed for this project.

Data will be stored on a secure, restricted access, password-protected server accessible only to the study team. The original data will be stored in a locked vault with keycard and biometric access required to enter. All Medicare Beneficiary ID numbers will be encrypted, and the dataset will be deidentified prior to analysis. All identifiable data will be transmitted by encrypted portable media or secure encrypted file transfer. We will report only aggregate data, and cells with fewer than ten observations will be concealed to minimize the risk of loss of patient confidentiality. The study principal investigator (PI) will have access to the final dataset and will take ultimate responsibility for its integrity.

The study protocol was reviewed and approved by the institutional review board (IRB) at both the University of Iowa and Avera eCARE under waiver of informed consent.

We will disseminate the results of this analysis by publication in peer-reviewed journals, presentations at scientific meetings, and production of research and policy briefs. Data sets will not be available for dissemination because of the data use requirements for each of our data sets, but we encourage cross-institution collaborations. The study team will collaborate with interested investigators to conduct additional analyses while maintaining the security of the dataset using a standard written request process that is evaluated by the study executive committee.

The main strength of this study is that we will be measuring the real-world application of tele-ED care in an existing and functioning tele-ED network. Other studies have reported the results of pilot programs under highly controlled conditions unlikely to be replicated in real-world practice. Our work will report the actual experience in a network functioning at scale.

The second strength is that we will be conducting this work using data available from non-tele-ED hospitals for comparative-effectiveness analysis. Care in hospitals with telehealth services can evolve even in non-telehealth patients over time because of interactions with the hub network, limiting the ability to observe differences between telehealth-enabled care and control cases [21,34,49–51]. By using a large database that compares outcomes within hospitals with EDs that participate in the network and others with EDs without telehealth capabilities, we will be able to see differences that may not be apparent within facilities already participating in a telehealth program alone.

Third, conducting this analysis at the hospital level, then using a propensity score to limit cases in non-tele-ED capable hospitals for the participant-level analysis is a strong strategy to reduce the effect of confounding by indication. Telehealth studies in acute care are often challenged by the strong relationship between severity, indication and telehealth use. By selecting only cases in non-telehealth capable EDs to compare with telehealth-enabled cases will reduce confounding by removing cases very unlikely to have had telehealth used. Comparing cases within a hospital only can be subject to confounding by indication because cases that are seemingly similar on measured covariates still had a reason telehealth was used – and this reason could be associated with our outcomes. This effect is not observable when controls come from other hospitals.

Finally, our use of cost as an outcome is a strength. Cost–effectiveness is one of the most important, yet difficult questions in telehealth policy. Using a Medicare-based analysis with a robust control group is a powerful way to estimate the cost avoidance associated with improved care from telehealth use.

The two greatest limitations of this work are its observational design and the lack of a robust severity of illness variable. Because our primary data source is administrative data, we will be able to access a very large sample, even in hospitals that do not participate in a telehealth network. The disadvantage, though, is that administrative claims contain minimal information about the clinical interventions performed as part of clinical care or physiologic severity of illness measures. This limitation may allow some residual confounding that cannot be adjusted with the variables we include in our propensity score methods. Another limitation is that our intervention hospitals all come from participation in a single tele-ED network and all patients are age-qualifying Medicare beneficiaries. The findings from that network may not generalize to all tele-ED interventions or to non-Medicare patients.