Open access

Research Article

2 November 2021

Missing data methods for intensive care unit SOFA scores in electronic health records studies: results from a Monte Carlo simulation

Authors: Daniel L Brinton https://orcid.org/0000-0002-7888-6708 [email protected], Dee W Ford, Renee H Martin, Kit N Simpson https://orcid.org/0000-0002-1713-0632, Andrew J Goodwin, and Annie N Simpson https://orcid.org/0000-0002-7216-1036Author Info & Affiliations

Publication: Journal of Comparative Effectiveness Research

Volume 11, Number 1

https://doi.org/10.2217/cer-2021-0079

PDF OtherFormats

Abstract

Aim: Missing data cause problems through decreasing sample size and the potential for introducing bias. We tested four missing data methods on the Sequential Organ Failure Assessment (SOFA) score, an intensive care research severity adjuster. Methods: Simulation study using 2015–2017 electronic health record data, where the complete dataset was sampled, missing SOFA score elements imposed and performance examined of four missing data methods – complete case analysis, median imputation, zero imputation (recommended by SOFA score creators) and multiple imputation (MI) – on the outcome of in-hospital mortality. Results: MI performed well, whereas other methods introduced varying amounts of bias or decreased sample size. Conclusion: We recommend using MI in analyses where SOFA score component values are missing in administrative data research.

Supplementary Material

File (supplementary material.docx)

Download
431.52 KB

References

Papers of special note have been highlighted as: • of interest; •• of considerable interest

1.

Vincent JL, De Mendonca A, Cantraine F et al. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Working group on “sepsis-related problems” of the European Society of Intensive Care Medicine. Crit. Care Med. 26(11), 1793–1800 (1998).

• Validation study of the SOFA score

2.

Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol. Scand. 52(4), 467–478 (2008).

3.

Buyse S, Teixeira L, Galicier L et al. Critical care management of patients with hemophagocytic lymphohistiocytosis. Intensive Care Med. 36(10), 1695–1702 (2010).

4.

Neto AS, Barbas CSV, Simonis FD et al. Epidemiological characteristics, practice of ventilation, and clinical outcome in patients at risk of acute respiratory distress syndrome in intensive care units from 16 countries (PRoVENT): an international, multicentre, prospective study. Lancet Resp. Med. 4(11), 882–893 (2016).

5.

Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA 286(14), 1754–1758 (2001).

6.

Singer M, Deutschman CS, Seymour CW et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 315(8), 801–810 (2016).

•• Sepsis-3 guidelines, which state one is to assume no organ derangement unless the patient has a known organ dysfunction

7.

Teasdale G, Jennett B. Assessment of coma and impaired consciousness. A practical scale. Lancet 2(7872), 81–84 (1974).

8.

Sessler CN, Gosnell MS, Grap MJ et al. The Richmond Agitation–Sedation Scale: validity and reliability in adult intensive care unit patients. Am. J. Respir. Crit. Care Med. 166(10), 1338–1344 (2002).

9.

Sessler CN, Grap MJ, Brophy GM. Multidisciplinary management of sedation and analgesia in critical care. Presented at: Semin. Respir. Crit. Care Med. (2001).

10.

Ely EW, Inouye SK, Bernard GR et al. Delirium in mechanically ventilated patients: validity and reliability of the confusion assessment method for the intensive care unit (CAM-ICU). JAMA 286(21), 2703–2710 (2001).

11.

Charlson ME, Pompei P, Ales KL, Mackenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J. Chronic Dis. 40(5), 373–383 (1987).

12.

Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J. Clin. Epidemiol. 45(6), 613–619 (1992).

13.

Burton A, Altman DG, Royston P, Holder RL. The design of simulation studies in medical statistics. Stat. Med. 25(24), 4279–4292 (2006).

14.

Rubin DB. Multiple Imputation after 18+ years. J. Am. Stat. Assoc. 91(434), 473–489 (1996).

•• Review of multiple imputation (MI) framework and gives response to criticism of MI, comparing alternative strategies.

15.

Rubin DB. Multiple imputation for nonresponse in surveys. John Wiley & Sons, NY, USA (1987).

• Seminal article on MI in research.

16.

Molenberghs G, Beunckens C, Sotto C, Kenward MG. Every missingness not at random model has a missingness at random counterpart with equal fit. J. Roy. Stat. Soc. Ser. B. (Stat. Method.) 70(2), 371–388 (2008).

17.

Bell ML, Fairclough DL, Fiero MH, Butow PN. Handling missing items in the Hospital Anxiety and Depression Scale (HADS): a simulation study. BMC Res. Notes 9(1), 479 (2016).

18.

Schafer JL. Analysis of Incomplete Multivariate Data. Chapman & Hall, Boca Raton, FL. (1997).

19.

Von Hippel PT. Regression with missing Ys: an improved strategy for analyzing multiply imputed data. Sociological Methodol. 37, 83–117 (2007).

20.

White IR, Daniel R, Royston P. Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Comput. Stat. Data Anal. 54(10), 2267–2275 (2010).

21.

White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat. Med. 30(4), 377–399 (2011).

22.

Li P, Stuart EA, Allison DB. Multiple imputation: a flexible tool for handling missing data. JAMA 314(18), 1966–1967 (2015).

•• A very approachable primer on MI and missing data mechanisms

23.

Yourman LC, Lee SJ, Schonberg MA, Widera EW, Smith AK. Prognostic indices for older adults: a systematic review. JAMA 307(2), 182–192 (2012).

|