Using Hospital Outcomes to Predict 30-Day Mortality Among Injured Patients Insured by Medicare

Hypothesis  Survival until a fixed time after injury is a more useful outcome variable than survival until hospital discharge.

Design  We sought to determine whether 30-day survival could be accurately predicted by hospital discharge status.

Setting  Academic research.

Patients  We analyzed Medicare fee-for-service records for patients 65 years or older admitted with a principal diagnosis of injury (International Classification of Diseases, Ninth Revision, Clinical Modification codes 800-959, excluding 905-909, 930-939, and 958).

Main Outcome Measures  Patients were classified by maximum Abbreviated Injury Score (range, 1-5) and Charlson comorbidity score (0, 1, 2, or ≥3). We modeled the conditional probability of survival at 30 days given hospitalization survival (P[S30SH]) as a function of census region, age, sex, maximum Abbreviated Injury Score, Charlson comorbidity score, length of stay, and discharge home or not.

Results  A total of 436 104 patients met inclusion criteria, and a model was created using half the sample. For northeastern women aged 65 to 69 years with a maximum Abbreviated Injury Score of less than 3, Charlson comorbidity score of 0, and discharge home with length of stay less than 3 days, the model predicted P (S30SH) to be 0.998. The P (S30SH) was lower for other census regions, male sex, older age, more severe injury, and greater comorbidity. The equation had modest predictive ability when applied to individuals in the other half of the sample (area under the receiver operating characteristic curve, 0.75) and closely predicted P (S30SH) within numerous subpopulations.

Conclusion  For injured patients insured by Medicare, P (S30SH) can be estimated using administrative data known at the time of hospital discharge.

In-hospital mortality has been the principal outcome used to measure quality of care for hospitalized injured patients. Trauma center performance has been commonly determined by comparing predicted deaths with observed deaths before hospital discharge. Many risk-adjusted outcome models of hospitalization survival based on logistic regression analysis have been created, and proponents have argued for specific models based on their predictive accuracy. However, a major limitation in all these analyses is that some of the short-term deaths among seriously injured patients occur after discharge and will not be observed if only hospital data are available.

Although most younger patients who survive hospitalization have a high likelihood of prolonged survival, about half of the 30-day mortality among older injured patients occurs after hospital discharge.^1,2 As the population older than 65 years increases, accurate evaluation of hospital quality must account for these postdischarge deaths. Otherwise, an apparently successful trauma center could simply be a hospital that expeditiously, or even prematurely, discharges high-risk patients home or to alternative health care facilities.

Several options are theoretically available to analysts who seek a more comprehensive measure of quality. Hospital personnel can apply intensive follow-up measures to every trauma patient after hospital discharge and add these outcomes to their trauma registry. However, this approach is time-consuming, and trauma patients are notoriously difficult to find after leaving the hospital, so that incomplete follow-up will undermine data validity. Cross-linking hospital trauma registries with death certificate data¹ provides a less expensive method but is still laborious and requires a long delay until the death certificate information accumulates, is entered into a database, and becomes available for analysis.

An inexpensive alternative to these methods might be to estimate the conditional probability that a patient with specified characteristics will survive 30 days given that he or she initially survived to be admitted to a hospital. This may be envisioned as the product of the following 2 factors: (1) the predicted or observed probability of hospitalization survival given hospital admission and (2) the conditional probability of survival at 30 days given hospitalization survival, abbreviated herein as P (S30SH).

For the United States, Medicare billing data contain information that can be used to predict the outcomes of trauma patients after discharge. A limitation of the Medicare database is that it generally applies only to patients older than 65 years. However, important strengths of the Medicare data are that they represent the population at greatest risk for death after hospital discharge, that the federal government monitors and records when beneficiaries die (and are no longer eligible for benefits), and that an anonymous but unique identifier allows hospital claims data for individuals to be linked to their survival data.

Our intent in this study was to analyze Medicare data to identify hospitalized injured patients who died or had survived up to a fixed point, namely, 30 days following hospital admission for acute injury. A specific goal of our analyses was to determine among patients discharged alive from an acute care hospital whether a logistic regression model using information customarily available at the time of discharge could accurately predict the probability that an individual would be alive or dead within the specified time frame. Such a model might provide a valuable additional measure in the attempt to determine variability in the quality of care delivered to hospitalized injured patients.

Medicare Provider Analysis and Review and Denominator files for 1999 were obtained from the Centers for Medicare & Medicaid Services through a cooperative agreement with Dartmouth Medical School, Hanover, New Hampshire. Institutional review boards at the Maine Medical Center and the Harvard School of Public Health judged this study exempt from further review.

The Medicare Provider Analysis and Review file contained hospital discharge abstracts summarizing acute care inpatient stays for fee-for-service Medicare beneficiaries. Records for each discharge contained hospitalization and outcome data, with up to 10 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) discharge diagnoses. Patients were selected if the principal diagnosis code was between 800 and 959 (injuries), excluding 905 to 909 (late effects of injury), 930 to 939 (foreign bodies), and 958 (complications of injury). By means of a unique patient identification number, a corresponding record in the Denominator file had to show that the beneficiary was at least 65 years old at the time of admission, lived in one of the 50 states or District of Columbia, and was enrolled in fee-for-service Medicare hospital insurance.

Commercially available software (ICDMAP-90; Tri-Analytics, Bel Air, Maryland) mapped each ICD-9-CM diagnosis code to an Abbreviated Injury Score (AIS) when possible and then determined the maximum AIS for each patient.³ The AIS classifies severity of injuries as 1 (minor), 2 (moderate), 3 (severe, not life-threatening), 4 (severe, life-threatening, survival probable), or 5 (critical, survival uncertain).⁴ The injury resulting in the maximum AIS for a given patient is usually, but not necessarily, the principal diagnosis. The “Ignore Unknown” and “Low Severities” options for the software were used, thus ignoring any ICD-9-CM code that could not be classified and selecting the less severe AIS when this mapping was equivocal.

Because preexisting medical conditions may have an important role in the outcome of older trauma patients, a modification of the Charlson comorbidity score (CCS) was calculated for each patient.⁵ The CCS adds weights of 6 for AIDS or metastatic solid tumor; 3 for severe liver disease; 2 for any malignant neoplasm, renal failure, or complications of diabetes mellitus; and 1 for a history of myocardial infarction, peripheral vascular disease, dementia, chronic lung disease, rheumatic disease, mild liver disease, or uncomplicated diabetes. Diagnosis codes identifying these conditions were obtained using the index modifications described by Romano et al.⁶

Based on US census definitions, hospitals were assigned to one of the following 4 regions: Northeast, South, Midwest, and West. Hospitalization survival was determined from the discharge status in the Medicare Provider Analysis and Review file. Survival until 30 days after admission was computed from the admission date in the Medicare Provider Analysis and Review file and the date of death in the Denominator file.

For the predictive models, we excluded patients who died in the hospital, patients who were still hospitalized after 30 days, and patients who had a principal diagnosis of hip fracture (ICD-9-CM code 820.x). Half the remaining sample was used to construct logistic regression models predicting the probability of 30-day mortality as a function of census region, age, sex, maximum AIS, CCS, length of stay (LOS), and discharge home or not. Predictive capability of this model was measured using pseudo- R² and the area under a receiver operating characteristic curve (measured using the other half of the sample for validation). The conditional probability of survival at 30 days given hospitalization survival, or P (S30SH), was calculated as 1 minus the probability of 30-day mortality.

A total of 436 104 patients met inclusion criteria (Table 1 and Table 2). Of injured patients 65 years or older, 73.9% were women and 59.1% were 80 years or older. Patients with a primary diagnosis of hip fracture represented 50.3% of the population. Sixty percent had a maximum AIS of 3, and 46.9% had a CCS of 1 or higher. A total of 30 452 patients in the overall population died within 30 days of being hospitalized for their injuries. These decedents can be divided into 14 895 who died in the hospital and 15 557 who died after discharge but within the 30-day threshold.

For the predictive modeling, only 221 003 patients remained after excluding those who died in the hospital, those who were still in the hospital at 30 days, and those who had hip fractures. Of this subgroup, 6835 (3.1%) died after hospital discharge but on or before the 30th day following hospital admission (Table 1). Logistic regression analysis was performed using a randomly selected half of this sample to predict mortality and to evaluate the effect of covariates. Postestimation diagnostic tests showed a pseudo- R² of 0.10 and an area under the receiver operating characteristic curve of 0.75 (using the other half of the sample for validation), indicating that a modest amount of individual variability was explained by the model.

Odds ratios obtained from the predictive model are given in Table 3. For the baseline group of northeastern women in good health aged 65 to 69 years with maximum AIS of less than 3, CCS of 0, and discharge home with LOS of less than 3 days, the model predicted the odds of mortality at 30 days after admission for injury to be 0.00194, corresponding to a P (S30SH) of about 0.998. The 30-day odds of mortality for other patient cohorts (given discharge alive from the hospital) can be obtained by multiplying the baseline odds times each of the applicable factors in Table 3. The probability of mortality can be obtained from the odds using the following standard formula:

Probability = Odds/ (1 + Odds).

For example, the odds for an 88-year-old southern man in good health whose most severe injury was a fractured femur (AIS, 3) and who was discharged to a nursing home on the 10th hospital day would be 0.00194 × 1.54 × 5.82 × 1.19 × 1.51 × 3.21 = 0.100. Therefore, the P (S30SH) for such a patient (ie, the probability that he would be alive at 30 days after admission given the information available at the time of hospital discharge) would be about 90.9%.

Using this approach, P (S30SH) calculated for numerous subpopulations of interest within the validation sample was found to closely approximate actual 30-day survival. These results are summarized in Table 4.

Calculation of the estimated survival at 30 days given hospitalization survival, which we designate as P (S30SH), is a measure of quality separate from hospitalization survival after admission. Our calculation of P (S30SH) depends only on information recorded in trauma registries or hospital administrative data and does not require observation beyond hospitalization. This measurement provides the opportunity of new methods to evaluate performance for health care policy makers who seek to measure the outcomes of hospitalized injured patients.

The simplest application of our results would be as a modification of the observed outcome at the time of hospital discharge, which instead of simply being 0 or 1 (corresponding to dead or alive) could be multiplied by the calculated P (S30SH) to estimate the probability of 30-day survival given hospital admission. Comparison of hospitals could then use a linear regression model predicting this estimated conditional probability that a patient with specified characteristics will survive 30 days given that he or she initially survived to be admitted to a hospital as a substitute for the customary logistic regression model predicting hospitalization survival as a binary event. Calculated P (S30SH) could also be used as a “score” or “utility” in decision analyses or cost-benefit analyses if actual 30-day survival data are unavailable.

The regional variations in outcome observed in this study have been previously reported. In an analysis of in-hospital mortality for all age groups using the Nationwide Inpatient Sample, hospitalized injured patients in the western region of the United States had a lower adjusted odds of mortality compared with those in the northeastern region.⁷ However, the contrasting pattern of regional variation that we observed in this study, with the lowest odds of mortality in the northeastern region, is consistent with the 30-day mortality reported previously for injured patients insured by Medicare.⁸

Inappropriate conclusions may be reached if regions or hospitals are compared on the basis of a measured mortality rate that does not control for variations in discharge patterns and LOS. In a 1988 study of medical patients, Jencks et al⁹ also demonstrated a disparity between hospital mortality and 30-day mortality. They found a 99% longer LOS and 25% higher inpatient mortality in New York than in California. On the other hand, the 30-day mortality rate in New York was actually 1.6% lower than that in California. Jencks et al argued that a difference in inpatient mortality can be an artifact of a difference in LOS patterns and that 30-day mortality may be a more accurate way to compare hospital outcomes.

Similarly, Johnson and colleagues¹⁰ compared 30-day mortality with all procedure-related mortality rates at 43 Veterans Affairs medical centers and showed that hospital performance rankings could change dramatically based on how mortality was defined. Carey and colleagues¹¹ also demonstrated a significant increase in apparent mortality following cardiac surgery after accounting for postdischarge mortality in addition to in-hospital mortality.

The improved accuracy obtained from a specified period of observation, rather than depending on hospital discharge data alone, has been noted in cardiac surgery^12,13 and in general surgery.^14,15 Indeed, the 30-day observation period required for the American College of Surgeons National Surgical Quality Improvement Program is becoming a standard. The choice of a 30-day period of observation is arbitrary, and it may be that a longer period will be found more appropriate for trauma patients.

One implication of this and other studies of long-term outcome is that discharge planning can have a direct influence on the quality of care delivered to injured patients. Much of the emphasis on measuring the optimal quality of care delivered to injured patients has been on events during the initial resuscitation, surgery, and treatment in an intensive care unit. Several authors have pointed out that discharge destination is an important outcome to consider because observed hospital mortality may depend on the availability of skilled nursing beds in a given community or region.^11,16-18 Although hospital administrators may strive for shorter LOS, this cannot be considered a good measure of clinical quality if it leads to increased mortality after hospital discharge.^19,20 Conversely, hospitals that tend to have longer LOS owing to limitations in the availability of subacute facilities should not necessarily be considered inferior simply because their measured in-hospital mortality may be greater than that in hospitals with shorter LOS.

There have been previous efforts to predict long-term survival after hospitalization among older patients. Teno and colleagues²⁰ demonstrated a Hospitalized Elderly Longitudinal Project model to estimate 1- and 2-year survival probabilities for certain older nonsurgical nontrauma patients using data obtainable at the time of hospital admission. Walter and colleagues²¹ presented a prognostic index to estimate 1-year survival probabilities for older inpatients with acute medical illness based on functional status and other variables identified at the time of hospital discharge. These authors reviewed some of the few earlier efforts in this area, which were focused on making predictions that could guide the care of individual patients.

Our objective was more limited, namely, to approximate the expected 30-day survival for a cohort of injured patients admitted to a given hospital. For this purpose, patients younger than 65 years can be assumed to have a P (S30SH) close to 1, and those 65 years or older can be assigned a P (S30SH) using the Medicare-based model that we describe herein. The mean P(S30SH) over all patients for a given hospital can then be considered the estimated survival at 30 days after injury, allowing fair comparison with another hospital or with a group mean, rather than comparing only survival at the time of hospital discharge.

It is probable that improved models will be created in the future using more recent data, additional variables, longer periods of observation, or outcomes other than mortality. Registry data or medical records data would likely provide more accurate prediction. However, our results allow initial exploration of this general concept and can be immediately applied without the expense of identifying and collecting postdischarge outcomes. If this approach is validated in further studies, it may present a cost-effective way to minimize the information bias resulting from measuring only outcomes observed during acute hospitalization.

The use of Medicare data to study injury outcomes has definite limitations, which have been discussed in more detail elsewhere.^2,22 Approximately 23% of Medicare beneficiaries, disproportionately those residing in western states, are covered by health maintenance organizations and do not generate fee-for-service Medicare claims. Furthermore, Medicare records are based on billing data, and the accuracy of extrapolating an AIS from ICD-9-CM codes is dependent on the accuracy and validity of the coding used in hospital billing systems.

We intentionally excluded patients with hip fracture from this study (but may report on them separately) because we were most interested in the more typical trauma center population and the inclusion of this very frequent injury would have made it more difficult to evaluate other risk factors. Because Medicare data cover only a select group of disabled persons younger than 65 years, we also do not know whether these results are applicable to younger patients. Additional studies to determine factors leading to early postdischarge mortality or other outcomes may be useful not only for injured patients but also for other medical and surgical conditions in which recovery is often less than complete at the time patients leave an acute care hospital.

In conclusion, this study demonstrates that Medicare data can be converted into an alternative measure of outcome for hospitalized injured patients that incorporates the risk of death after discharge. As the population of patients treated in regional trauma centers ages, health care policy makers seeking information on the quality of trauma care need to expand their measures beyond the traditional focus on hospitalization survival. Although administrative data can provide modestly precise prediction, a similar method using registry or medical records data might be more accurate. The additional information obtained in this way may provide a more valid measure of the performance of a hospital or trauma system than simply using survival observed at the time of hospital discharge.

Correspondence: David E. Clark, MD, Department of Surgery, Maine Medical Center, 887 Congress St, Portland, ME 04102 (clarkd@mmc.org).

Accepted for Publication: January 25, 2010.

Author Contributions: Dr Clark had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Gorra and Clark. Acquisition of data: Clark. Analysis and interpretation of data: Gorra, Clark, and Mullins. Drafting of the manuscript: Gorra and Clark. Critical revision of the manuscript for important intellectual content: Gorra, Clark, and Mullins. Statistical analysis: Clark. Obtained funding: Clark. Administrative, technical, and material support: Clark. Study supervision: Clark.

Financial Disclosure: None reported.

Funding/Support: This study was supported in part by grant R49/CCR115279 from the National Center for Injury Prevention and Control to the Harvard Injury Control Research Center (Dr Clark).

Disclaimer: This article reflects the views of the authors but not necessarily those of the National Center for Injury Prevention and Control.

Previous Presentations: This study was presented in part at the 67th Annual Meeting of the American Association for the Surgery of Trauma; September 24, 2008; Maui, Hawaii, and at the American College of Surgeons Committee on Trauma Region I 2008 Resident Trauma Papers Competition; November 10, 2008; Boston, Massachusetts.

Additional Contributions: Edward L. Hannan, PhD, provided statistical advice and reviewed the manuscript.