Mortality risk estimation for autistic older adults: comparing novel machine-learning derived weights versus standard weights for the Charlson Comorbidity Index

Data used in this study were derived from 2021 through 2023 Medicare Standard Analytical Files (SAF), which include limited dataset (LDS) information on 100% of Medicare beneficiaries for these years. SAF includes beneficiary-level healthcare claims data from inpatient, outpatient and skilled nursing facility records. Outpatient files contain all encounters with an institutional-outpatient facility such as hospital outpatient departments, rural health clinics, renal dialysis facilities, outpatient rehabilitation facilities, federally qualified health centers and community mental health centers. They do not include free-standing facilities (e.g., independent clinical laboratories, ambulance providers, free-standing ambulatory surgical centers and free-standing radiology centers).

Autistic older adults were included in this study if they were aged 65 years or older; were enrolled in Medicare Part A and Part B for 12 months consecutively between 2021 and 2023; had at least one inpatient hospitalization in 2022 to serve as an index hospitalization to anchor extraction of co-occurring conditions and mortality, aligned with the methods of Quan et al. [9]; and had at least one inpatient or two outpatient encounter with a diagnosis code for autism during 1 January 2021 through 31 December 2023. For those with more than one hospitalization in 2022, we used the last observed hospitalization as the index hospitalization, consistent with Quan et al. [9]. Autism diagnoses were identified using ICD-10 codes F84.0, F84.1, F84.5 or F84.9. Those enrolled in Medicare Advantage Plans were excluded from this study.

Our dependent variables were all-cause 30-day and 1-year mortality after the index hospitalization. Because Medicare receives beneficiary death dates nightly from the United States Social Security Administration, the date of death in the LDS SAF denominator file is up to date as of the day the files were created and only contains dates of death that have been independently validated using the LDS SAF standard procedures. Therefore, we assumed beneficiaries who had recorded no date of death during 2021–2023 were still alive at the end of the observational period. Our independent variables were the established CCI and a novel risk index we constructed as the weighted sum of the 12 CCI conditions using autism-specific weights. Consistent with Quan et al. [9], we used the year prior to index hospitalization as a look-back period to identify the 12 CCI conditions based on diagnosis codes in inpatient healthcare claims: congestive heart failure, dementia, chronic pulmonary disease, rheumatologic disease, liver disease (mild or severe), diabetes with complications, hemiplegia or paraplegia, renal disease, malignancy, metastatic solid tumor and acquired autoimmune deficiency syndrome/human immunodeficiency virus. Supplementary Table 1 provides the ICD-10 codes used to identify these conditions [9,15].

Demographic characteristics, including sex, race/ethnicity, age and US region of residence, were extracted from the Master Beneficiary Summary File as of the date of the index hospitalization. Rurality was determined based on the 2023 USDA Rural-Urban Continuum Code of ‘large metro area – urban population of greater than 1000,000’, ‘small metro area – urban population of less than 1000,000’, ‘metro area adjacent’, ‘non-metro area – urban population of greater than 2500’ and ‘rural area – urban population of less than 2500’ [16].

We randomly divided our dataset into two mutually exclusive groups: a 75% training cohort used to build the model, and a 25% validation cohort used to test the model performance. We applied a machine learning method known as stochastic hill climbing (see Figure 1) to iteratively adjust the weights for each condition in the CCI [17]. The algorithm began by assigning an initial weight of 2 to all conditions and computing a mortality risk score for each individual as a weighted sum of binary indicators (presence/absence of each condition). The risk score was then used as the sole predictor in a logistic model for mortality, and model performance was evaluated using the AUC, a standard measure of predictive ability. At each iteration, one condition was randomly selected, and its weight was adjusted by a small random value drawn from a normal (0, 0.2) distribution. The updated weights were retained only if they improved the AUC; otherwise, the previous weights were preserved. To identify an appropriate number of iterations, two preliminary runs with 10,000 iterations were conducted, and convergence in AUC was observed at approximately 1000 iterations, which was used for the final analysis. The stochastic hill climbing algorithm was repeated 50-times to ensure stability, and the final weights were derived from the median of the weights across these 50 runs. We applied the autism-specific and established weights in the validation cohort using logistic regression to predict 30-day and 1-year mortality, controlling for sex and age. We compared the AUC of the models based on the established and autism-specific weights. AUC is interpreted such that 0.5–0.6 was fail, 0.6–0.7 was poor, 0.7–0.8 was fair, 0.8–0.9 was considerable and 0.9–1.0 was excellent predictive ability. We used SAS v9.4 statistical software for all analyses.

Several methodologic considerations should be acknowledged when interpreting the results of this study. First, because we aimed to derive autism-specific weights that could be compared as directly as possible to the established weights from Quan et al. [9], we made a number of methodologic decisions based on their precedent. This included using only data from inpatient hospitalizations to identify comorbidities, using a 1-year look-back period, using only the last-observed inpatient hospitalization during 2022, and restricting our predictors to the 12 CCI conditions, age and sex. Future studies aiming to further improve the predictive ability of the CCI could consider including both inpatient and outpatient data to identify comorbidities, extending the look-back period beyond 1-year, and including multiple hospitalizations. Additionally, demographic factors and social determinants of health such as sex, race, location, and income may have significant predictive value in comorbidity indices [26,27]. Incorporating these variables into future models may enhance their applicability and predictive ability. Second, the use of administrative claims data introduces potential biases related to unmeasured clinical and social factors, under coded or miscoding of co-occurring conditions, and possible misclassification of diagnoses, all of which may affect the accuracy of mortality risk estimation [28,29]. Third, our findings are based on autistic adults aged 65 and older enrolled in Medicare, which limits generalizability to younger autistic individuals, those with private insurance, and populations with different healthcare access or utilization patterns. Fourth, we applied only stochastic hill climbing to derive autism-specific weights. There are many other machine learning methods available. Other studies seeking to improve the predictive accuracy of the CCI for autistic older adults or other clinical populations may wish to compare different types of machine learning methods. Finally, the autism-specific weights require validation in independent datasets and across more diverse autistic populations to determine their broader applicability. Continued research is needed to refine and validate mortality risk prediction tools tailored to the unique health profiles of autistic individuals.