The value of linking hospital discharge and mortality data for comparative effectiveness research

We used the California Office of Statewide Health Planning and Development Patient Discharge Database, which consists of records for all discharges from all nonfederal hospitals located in California. The records are linked to the California State Death Statistical Master File, a database of death certificates for all California residents dying in or out of the state. For any given year, approximately 3.5 million discharges are reported within the Patient Discharge Database, and there are 240,000 reported deaths within the Death Statistical Master File.

The hospital discharge and death files were linked by the California Office of Statewide Health Planning and Development using an automated linkage algorithm, primarily created with the patient’s Social Security number (SSN) [4]. This linked data file is called ‘Linked Patient Data and Vital Statistics Death Statistical Master File’. According to Zingmond et al., approximately 20–30% of patient discharge records do not have a valid SSN and are not matched [5]. The majority of the records lacking SSNs (∼65%) were for births, and the availability of SSN improved in later years. In addition, residents dying outside of California did not have SSNs and were not linked.

We obtained linked Patient Discharge Data and Vital Statistics Death Files from OSPHD from 2000 to 2007 for any individual with an International Classification of Diseases Ninth Revision Clinical Modification (ICD-9-CM) diagnosis code of 441.xx (aortic aneurysm and dissection) appearing anywhere in the diagnosis array. Ultimately, we focused on a subset of these patients with the more specific diagnosis of abdominal aneurysm without mention of rupture as a principal diagnosis (ICD-9-CM code, 441.4). We focused on this group of patients so that the analysis targets patients with planned aneurysm repairs rather than high-risk emergent replacements with a higher risk of in-hospital mortality.

The study design was reviewed and approved by the Committee for the Protection of Human Subjects – the institutional review board for the State of California.

Out of the 379,334 discharge records in the linked database with ICD-9-CM diagnosis code 441.xx appearing anywhere in the diagnosis array, we identified 24,295 unique patients with abdominal aneurysm without mention of rupture as a principal diagnosis (ICD-9-CM code 441.4).

Table 1 lists the exclusion criteria and their effect on the sample size. We excluded patients with an AAA OAR or EVAR admission in the first 6 months of 2000 to increase the probability of identifying the patient’s first OAR rather than a revision of a repair. We also excluded patients with these procedures in 2007 – the last year for which we had data – to ensure that patients could be followed in the data for at least 1 year. These two exclusion criteria reduced the sample by 4754 patients. All patients were required to have ICD-9-CM procedure codes indicating either OAR (38.44 or 39.25) or EVAR (39.71), which reduced the sample by 4058 patients. Patients were excluded if they had both OAR and EVAR procedures during the same admission because we could not attribute the effectiveness to a particular intervention (114 patients met this exclusion criteria). In order to improve the homogeneity of the two cohorts, we also excluded patients with procedural codes for repair of the thoracic aorta (38.35, 38.45 or 39.73) and/or visceral or renal bypass (38.46, 39.24 or 39.26), polyarteritis nodosa (446.0), coarctation of the aorta (747.1) or Marfan syndrome and other congenital anomalies (759.8). These criteria resulted in a total of 744 excluded patients. A total of 92 patients were excluded owing to anomalies in the data set, including patients who had multiple death records, a death date prior to an admission date, or multiple values for sex. Finally, we excluded 881 patients with OAR or EVAR procedures who were admitted through the emergency room, because they are more likely to have AAA rupture and the study focus was on elective AAA repair. All exclusions were implemented to identify a cohort of reasonably homogenous patients with a similar risk of mortality following these procedures. The trade-off in implementing these exclusions to improve homogeneity and interval validity is that it may reduce the study’s generalizability. In particular, the results may not be applicable to patients with AAA rupture or with the excluded comorbid conditions.

The final patient count for this study was 13,652; of these, 7606 had an OAR procedure and 6046 had an EVAR procedure.

The primary outcome measure was mortality, which was measured during the inpatient stay and at 30 days, 1 year and 5 years following discharge, and was obtained from the vital statistics records. The other variables that were used in the logistic regression were obtained from the hospital discharge records. The broad categories of patient characteristics were: sociodemographic characteristics (sex, age, race, ethnicity, median household income for the patient’s ZIP code and primary health insurance type); year of first OAR or EVAR AAA procedure; whether or not the patient had a hospitalization with a principal AAA diagnosis in the prior 6 months; whether or not the patient had a hospitalization for any other cause in the prior 6 months; and comorbidities. Two sets of criteria were used to identify comorbid conditions associated with AAA: one suggested by Schwarze et al. [11] and the other calculated using the Healthcare Cost and Utilization Project Comorbidity Software [102]. We used the version of this software that was appropriate to the set of Medicare Severity-Diagnosis Related Group codes in each database. Hospital characteristics, specifically bed size and teaching status, were based on the American Hospital Association Annual Survey data [12]. We calculated from the discharge data the volume of AAA OAR and EVAR repairs conducted over the study period at the hospital where the patient was treated.

We compared the characteristics of the OAR and EVAR cohorts using means and frequencies, and determined the statistical significance of group differences using χ² and t-statistics. The two cohorts were matched using propensity scores for the purpose of obtaining homogenous samples with which to compare outcomes. To derive the propensity score, a logistic model calculated the conditional probability of being in the OAR versus the EVAR cohort.

The predicted propensity score was then used to create a 1:1 match between the OAR and EVAR cohorts using greedy matching with a caliper of 0.25 standard deviation of the propensity score. Balance on the covariates was assessed by computing standardized differences for each covariate; ‘balanced’ was defined as <10 standardized differences. Finally, we conducted Kaplan–Meier survival analyses on the unmatched and propensity score matched cohorts. t-tests were used to test for statistical significance (p < 0.05).

Table 2 contains the patient and hospital characteristics for individuals undergoing EVAR or OAR, before and after propensity score matching. After propensity score matching, the cohorts were considered balanced (all standardized differences were <10).

In the matched sample, in-hospital and 30-day postdischarge mortality rates were significantly lower for patients undergoing EVAR than for patients undergoing OAR (1.2 vs 2.5% and 1.5 vs 2.8%, respectively). For 1- and 5-year mortality in the matched EVAR and OAR cohorts, the difference in the probability of death between cohorts was not statistically significant (Table 3 & Figure 1). The size of the differences in mortality rates between EVAR and OAR was significantly reduced after matching (Figure 1 vs Figure 2).

We found the linked database relatively easy to use for observational comparative effectiveness analysis. Importantly, our results are consistent with those of prior randomized clinical trials that find an advantage of EVAR over OAR with respect to immediate inpatient and perioperative mortality, but no longer-term advantage [13,14].

Two studies have also compared EVAR with OAR using Medicare claims linked to vital statistics mortality data. Schermerhorn and colleagues, using data for 2001–2004, reported an advantage of EVAR over OAR in survival up until 3 years, at which time the survival rates converge [15]. By contrast, Jackson and colleagues, using 2003–2007 data, found a survival advantage for EVAR that was maintained throughout the 5.7-year follow-up [16]. Our analysis is consistent with these two prior studies in showing a perioperative survival advantage of EVAR over OAR. The absolute 30-day mortality differences are also similar; the advantage of EVAR over OAR was 1.2 versus 4.7% (present study), 1.2 versus 4.8% [15] and 1.2 versus 5.8% [16].

In our propensity score-matched analyses, the survival difference is not significant after 1 year, whereas in the unmatched analyses the survival advantage lasts beyond 1 year. Lack of a survival advantage after 1 year is consistent with results from randomized clinical trials, but not with the two prior studies using Medicare claims. Additionally, our unmatched results indicate a long-term survival advantage. Such discrepancies in long-term survival advantage of EVAR over OAR suggest that the longer-term outcomes may vary owing to patient heterogeneity (e.g., age and comorbidities) or other factors (e.g., surgical techniques) that need to be more fully understood. For example, the descriptive statistics comparing the EVAR and OAR populations reveal that EVAR tends to be more commonly used among older patients who may be less able to tolerate an open surgery. In general, these finding support the assertion that observational data are a cost-effective supplement to clinical trials to evaluate interventions in real-world settings and a potential alternative when randomized clinical trials are infeasible.

Clearly, the data will only be useful for some types of comparative effectiveness analyses. In particular, it will be most useful when evaluating interventions that are provided primarily in inpatient hospital settings for which ICD-9-CM procedure codes or other standardized codes exist to identify the interventions in the discharge records. Additionally, the data will be useful for comparative effectiveness studies where the conditions, procedures and posthospital mortality being evaluated are sufficiently prevalent to provide adequate statistical power to detect differences in mortality outcomes. In contrast to Medicare claims data, which primarily capture data for individuals aged 65 years and older, discharge data capture all payers. Given that approximately 66% of individuals discharged from community hospitals are younger than 65 years, linked discharge data can be particularly informative for researching interventions relevant to this population. These include percutaneous transluminal coronary angioplasty, spinal fusion, laminectomy, hip replacement and prostatectomy procedures (authors’ calculation of frequent procedures among populations younger than 65 years using [101]).

As with all observational data, the linked discharge–mortality data will face the issue of unobserved confounding from selection bias. The ability of researchers to address confounding is somewhat more limited with linked discharge and mortality data – as compared with claims data linked to mortality data – because the linked discharge–mortality data lack information on prescription drug and outpatient utilization that can be used for statistical adjustment. Lack of outpatient data also precludes examination of outpatient interventions, either as alternatives to hospital interventions or as indications of complicated cases. A second limitation of the linked discharge–mortality data is the lack of clinical detail. For example, for this particular study of AAA procedures, information about aneurysm size or anatomical features was not available in the hospital discharge records. However, the Medicare claims data include the same limitation, and they have continued to be utilized for observational studies. The third common criticism of administrative data concerns errors in coding of diagnoses or procedures. We conducted a literature search and did not find any published studies that examined the validity of ICD-9 coding of AAA or OAR and EVAR, although we have no reason to believe that the data were incorrectly coded.

We discovered a small number of data errors from the OSPHD database across the 5 years; there were 92 erroneous records out of a total of 13,652 (0.67%). These errors primarily represented individuals who were listed as male on one discharge record and female on another. In a few cases, there were multiple death records, and reports of deaths that occurred prior to an admission. Dr Zigmond and OSPHD staff members who conducted the data linkage reported that errors may occur either because miscoded SSNs slip through the linkage and validation process or because of fraudulent uses of another individual’s SSN upon admission to a hospital. These small errors did not appear to have a significant impact on our ability to conduct the study, but this issue needs to be evaluated further. Another consideration is that the linkage process excludes patients without SSNs. The majority of missing records are for young children and new births; however, the majority of missing deaths are for older adults. More research is needed to understand the implications of these missing records for introducing biases or limiting the generalizability of the results. Moreover, if government agencies start to place more limits on SSNs for vital statistics, the deterministic approach used to link hospital discharge and mortality data may not be viable, and alternative methods such as probabilistic linking may need to be employed [103].

In addition to observational comparative effectiveness studies, linked discharge–mortality data may be of value for a number of other purposes. First, the data can be used to understand the epidemiology of diseases or treatments, such as incidence rates of particular outcomes (e.g., rates of conversion from EVAR to OAR, rates of complication and postdischarge mortality rates) overall or for different subpopulations of patients. This information may be helpful in: anticipating and mitigating clinical complications of the procedures for certain subgroups; discharge planning and timing of follow-up monitoring for adverse outcomes; avoiding a specific type of procedure in some complicated cases; developing hypotheses and planning further research on the condition and alternative treatments using more rigorous research designs; and examining analyses within subgroups, in particular for individuals not covered by Medicare.

Second, another potential application is in pragmatic randomized clinical trials where data from administrative records linked to vital statistics are used to track outcomes following randomization. Examples of such ‘naturalistic’ and ‘streamlined’ studies are available from the UK, The Netherlands and Denmark, among other countries [104]. For example, in the FAST trial being conducted in the UK and Denmark, medications are provided to patients by mail, and hospitalization and mortality outcomes are monitored via linked records. The usefulness of the data for this type of research is predicated on the feasibility of establishing processes to identify patient records.

A third usage may be to improve knowledge of hospital-level outcomes and practice patterns. For example, the state hospital discharge data capture all procedures conducted at the hospital; thus, one can create measures of total hospital volume and compare them with postdischarge mortality, which has only been feasible to date for the subgroup of Medicare beneficiaries. For example, OSPHD has used the data to compare heart attack mortality rates for Californian hospitals [17,18].