Context For many elderly patients, an acute medical illness requiring hospitalization
is followed by a progressive decline, resulting in high rates of mortality
in this population during the year following discharge. However, few prognostic
indices have focused on predicting posthospital mortality in older adults.
Objective To develop and validate a prognostic index for 1 year mortality of older
adults after hospital discharge using information readily available at discharge.
Design Data analyses derived from 2 prospective studies with 1-year of follow-up,
conducted in 1993 through 1997.
Setting and Patients We developed the prognostic index in 1495 patients aged at least 70
years who were discharged from a general medical service at a tertiary care
hospital (mean age, 81 years; 67% female) and validated it in 1427 patients
discharged from a separate community teaching hospital (mean age, 79 years;
61% female).
Main Outcome Measure Prediction of 1-year mortality using risk factors such as demographic
characteristics, activities of daily living (ADL) dependency, comorbid conditions,
length of hospital stay, and laboratory measurements.
Results In the derivation cohort, 6 independent risk factors for mortality were
identified and weighted using logistic regression: male sex (1 point); number
of dependent ADLs at discharge (1-4 ADLs, 2 points; all 5 ADLs, 5 points);
congestive heart failure (2 points); cancer (solitary, 3 points; metastatic,
8 points); creatinine level higher than 3.0 mg/dL (265 µmol/L) (2 points);
and low albumin level (3.0-3.4 g/dL, 1 point; <3.0 g/dL, 2 points). Several
variables associated with 1-year mortality in bivariable analyses, such as
age and dementia, were not independently associated with mortality after adjustment
for functional status. We calculated risk scores for patients by adding the
points of each independent risk factor present. In the derivation cohort,
1-year mortality was 13% in the lowest-risk group (0-1 point), 20% in the
group with 2 or 3 points, 37% in the group with 4 to 6 points, and 68% in
the highest-risk group (>6 points). In the validation cohort, 1-year mortality
was 4% in the lowest-risk group, 19% in the group with 2 or 3 points, 34%
in the group with 4 to 6 points, and 64% in the highest-risk group. The area
under the receiver operating characteristic curve for the point system was
0.75 in the derivation cohort and 0.79 in the validation cohort.
Conclusions Our prognostic index, which used 6 risk factors known at discharge and
a simple additive point system to stratify medical patients 70 years or older
according to 1-year mortality after hospitalization, had good discrimination
and calibration and generalized well in an independent sample of patients
at a different site. These characteristics suggest that our index may be useful
for clinical care and risk adjustment.
People aged 65 years or older make up about 13% of the US population,
but they account for 37% of discharges from acute care hospitals.1 For many elderly patients, an acute medical illness
requiring hospitalization is followed by a progressive physical decline, resulting
in high rates of mortality during the year following discharge.2
Since hospitalization is frequently a major health transition for older adults,
reassessing goals of care at this juncture is often necessary. Prognostic
information can provide the basis for discussions about the goals of care
and therapy.3 However, few prognostic indices
have focused on prediction of posthospital mortality in the elderly population.
A prognostic index that estimates long-term mortality in older adults
following hospitalization may be useful to clinicians for many reasons. Such
an index can provide objective prognostic estimates to supplement clinicians'
intuition and judgment when counseling patients and their families about the
meaning of health problems and utility of treatment options. Prognostic indices
also can be useful in identifying groups at high risk for poor outcomes in
whom targeted treatment interventions may be indicated4
or for whom palliative care may be most appropriate.5
Also, prognostic indices are essential for comparing outcomes among
different physicians, hospitals, or systems of care.6
For example, indices that correct for baseline risk differences among patients
are needed to draw fair inferences from observed mortality data about the
quality of patient care provided by different health plans following hospitalization.
Fair comparisons can stimulate improvements in quality of care, but such comparisons
are not possible without accurate methods of risk adjustment.7
The few prognostic indices that stratify hospitalized general medical
patients into risk groups for long-term mortality have a number of limitations.
Some only apply to the critically ill,8-10
or require complex calculations and data that would not be routinely available
to clinicians. Only a few include functional status,11-13
despite its association with mortality in older patients who are hospitalized.14,15 Also, many indices have not been
developed for ethnically diverse groups of patients or validated in independent
samples, limiting their generalizability.16
To address these issues, we developed a prognostic index for 1-year
mortality following hospital discharge in a large heterogeneous group of older
adults with medical illnesses, in whom we measured multiple potential prognostic
factors, including functional status. We then validated the index in an independent
sample. Our goal was to provide an accurate and easy-to-use index that could
stratify older adults into groups by their risk of mortality after hospital
discharge.
This study includes individuals enrolled in 2 randomized trials of an
intervention to improve functional outcomes of hospitalized older adults.
The trials were conducted at the University Hospitals of Cleveland (UHC),
a tertiary care hospital, and the Akron City Hospital (ACH), a community teaching
hospital in Ohio, between 1993 and 1997. Each trial enrolled patients who
were aged 70 years or older and who were admitted to the general medical service.
Patients admitted to intensive care units (ICUs) or subspecialty services
or elective admissions were excluded, as were patients with lengths of stay
fewer than 2 days. Study protocols randomly selected a subset of eligible
patients to be representative of the general medical wards since it was not
possible to enroll all eligible patients because of logistic constraints.
Of a possible 11 475 eligible patients, 3163 were randomly selected for
enrollment. The demographic, clinical, and functional characteristics of patients
enrolled in the study were similar to those not enrolled.17
After 1 year, there was no difference in mortality or functional status between
the control and intervention groups, so they were combined for this analysis.
We used patients from the UHC to derive the prediction model and then
used patients from the ACH to validate the model. The UHC trial enrolled 1632
patients and the ACH trial enrolled 1531 patients. The potential analytic
cohorts for this study included the 1565 UHC patients and 1482 ACH patients
who survived to hospital discharge. We excluded 70 patients (4%) in the UHC
cohort because they were missing data on comorbid conditions or functional
status, leaving 1495 patients, and 55 patients (4%) from the ACH cohort who
were missing data on these risk factors, leaving 1427 patients.
Data Collection and Measurements
Predictors of Mortality. We obtained data from standardized interviews with patients and surrogates
and from medical records. We interviewed surrogate respondents when the patient
scored more than 5 errors on the 10-point Short Portable Mental Status Questionnaire18 or was too ill to communicate at the time of admission
(40%). We interviewed participants at both admission and discharge. The interviews
included demographic characteristics and reports of independence in 5 activities
of daily living (ADLs): bathing, dressing, using the toilet, transferring
from bed to chair, and eating. We used a modified version of the Katz Index
of ADLs19 to assess independence in ADLs by
asking the patient or surrogate at the time of discharge whether the patient
needed help from another person to perform each activity. A patient who required
personal assistance to perform a particular ADL was classified as dependent
in that ADL. A patient who used an assistive device to perform an ADL but
did not require help from another person was considered independent.
Information obtained from medical records by trained chart abstractors
included laboratory values on admission comprising the APACHE (Acute Physiology
and Chronic Health Evaluation) II score,20
medical diagnoses comprising the Charlson comorbidity index,21
reason for admission, length of hospital stay, and discharge destination.
We used laboratory values from the time of admission, because in clinical
practice they are routinely obtained at that time but not always at the time
of discharge.
We grouped the risk factors that we hypothesized were associated with
1-year mortality into 4 broad categories: demographic variables, medical diagnoses,
functional status, and laboratory values. Race was identified by the patient.
Specific risk factors were chosen based on clinical relevance, previous studies
of predictors of mortality, and prevalence greater than 10% in our sample.
Age was coded into 5-year intervals. Length of hospital stay was divided
into 7 days or fewer or more than 7 days, based on the mean length of stay.
Categorical variables, such as comorbid conditions, were coded as present
or absent, except that cancer was coded as absent or solitary or metastatic
solid tumor. Hematologic malignancies were coded as solitary cancer. Functional
status was categorized as totally independent (independent in all ADLs), partially
dependent (dependent in 1-4 ADLs), or totally dependent (dependent in all
ADLs). Analyses that used individual ADL items or total ADL scores produced
models with virtually the same discrimination as our final model. Creatinine
and albumin levels were also recoded into intervals based on clinically relevant
cut points.22,23
Definition of Outcome. The outcome of interest was defined as death within 1 year after hospital
discharge. We also used Kaplan-Meier curves to examine the performance of
our prognostic index over time. We obtained information about vital status
through follow-up interviews with participants and family members and a search
of the National Death Index.24 Deaths were
classified based on matches of the National Death Index record with the subject
according to name, sex, date of birth, and Social Security number. We achieved
100% follow-up for vital status.
We measured the bivariable relationship between each risk factor and
mortality in the derivation cohort using logistic regression models containing
only the risk factor of interest. We then entered all risk factors associated
with 1-year mortality (at P<.20) into a multivariable
logistic regression model with backward elimination (P<.05
to retain) to select the final set of risk factors. The same multivariable
model was chosen using forward selection (P<.05
to enter). After developing the final model, we assessed interactions between
sex and age with other risk factors. None were significant at P<.05.
We describe the results of our predictive model in 2 ways. First, we
estimated the predicted risk of death for each subject, based on the final
logistic regression model, and divided the subjects into quartiles of risk.
Second, we constructed a bedside risk scoring system in which we assigned
points to each risk factor by dividing each β coefficient in the final
model by the lowest β coefficient (male sex) and rounding to the nearest
integer.25 A risk score was assigned to each
subject by adding up the points for each risk factor present. Subjects were
then divided into approximate quartiles based on their risk scores.
The predictive accuracy of the logistic model and the point scoring
system was determined by comparing predicted vs observed mortality in the
ACH validation cohort (calibration), and by calculating the area under the
receiver operating characteristic (ROC) curves (discrimination) in both the
derivation and validation cohorts. Discrimination reflects the ability of
the prognostic index to distinguish between patients at high and low risk
of death and is often described in terms of the area under the ROC curve (ROC
area), which is related to the relative probability that in all possible pairs
of patients in which one patient lives and the other dies, a higher risk was
assigned to the patient who died than to the one who lived.26
We chose to validate our predictive model at a different site (ACH) than where
it was developed (UHC) since this form of prospective validation not only
tests the accuracy of the model but also tests its geographic and methodologic
transportability.16,27,28
Characteristics of Participants
The mean (SD) age of patients in the UHC derivation cohort was 81 (7)
years. Sixty-seven percent were women, 60% were white, and 30% were discharged
to a nursing home or skilled nursing facility. Forty-one percent were independent
in all ADLs at discharge, 32% were dependent in 1 to 4 ADLs, and 27% were
dependent in all ADLs (Table 1).
During 1-year follow-up, 492 patients (33%) died.
The mean (SD) age of patients in the ACH validation cohort was 79 (7)
years. Sixty-one percent were women, 88% were white, and 14% were discharged
to a nursing home or skilled nursing facility. Fifty percent were independent
in all ADLs at discharge, 35% were dependent in 1 to 4 ADLs, and 15% were
dependent in all ADLs (Table 1).
During the year following hospital discharge, 398 patients (28%) died.
Risk factors associated with 1-year mortality in the bivariable analyses
(P<.20) included age of 80 years or older, male
sex, history of myocardial infarction, congestive heart failure, cerebrovascular
disease, dementia, cancer, ADL function at discharge, length of hospital stay
of more than 7 days, discharge to a nursing home or skilled nursing facility,
creatinine level of 1.5 mg/dL (132.6 µmol/L) or more, and albumin level
of less than 4.0 g/dL (Table 2).
Six of these 12 risk factors were independently associated with mortality
in multivariable analysis (Table 3),
including 1 demographic variable (male sex), 2 medical diagnoses (congestive
heart failure and cancer), functional dependency in any ADL at discharge,
and 2 laboratory values (creatinine level >3.0 mg/dL [265.2 µmol/L]
and albumin level ≤3.4 g/dL). Many of the risk factors significantly associated
with 1-year mortality in bivariable analyses were not independently associated
with 1-year mortality after adjustment for discharge functional status. These
included age, dementia, and discharge to a nursing home.
By quartiles of predicted risk, 1-year mortality ranged from 13% in
the lowest-risk quartile to 63% in the highest-risk quartile in the derivation
cohort and from 9% to 64% in the validation cohort (Table 4). There was good calibration of the model, with close agreement
between observed and predicted mortality. The discrimination of the final
model was better in the validation cohort (ROC area = 0.80) than in the derivation
cohort (ROC area = 0.75). The model also retained good discrimination in the
validation cohort within sex and age subgroups. The ROC area was 0.80 for
women, 0.78 for men, 0.79 for patients aged 70 to 79 years, and 0.79 for patients
aged 80 years or older.
Bedside Risk Scoring System
The points assigned to each of the final 6 risk factors in the bedside
scoring system are listed in Table 3.
A risk score was calculated for each patient by adding the points of each
risk factor that was present. For example, a 70 year-old man (1 point) admitted
to a general medical service with functional dependency in 3 ADLs (2 points),
an albumin level of 2.9 g/dL (2 points), and a normal creatinine level would
have a risk score of 5 points. Derivation cohort risk scores ranged from 0
to 16 points (mean [SD], 4.0 [3]).
Patients were divided by risk scores into 4 risk groups of roughly equal
size. In the UHC derivation cohort, mortality ranged from 13% in the lowest-risk
group (0-1 point) to 68% in the high-risk group (>6 points). Within these
groups, patients with 0 points had a mortality rate of 11% (22/197) while
patients with more than 9 points had a mortality rate of 82% (55/67). Similar
results were seen in the validation cohort, except that the low-risk group
had only a 4% mortality (Table 4).
The point system had better discrimination in the validation cohort (ROC area
= 0.79) than the derivation cohort (ROC area = 0.75). Kaplan-Meier survival
curves of the 4 risk groups in the validation cohort demonstrate that the
groups have markedly different survival trajectories and that the mortality
differences between risk groups are persistent over the 1 year of follow-up
(Figure 1). In addition, the point
system retained good discrimination in age and sex subgroup analyses (ROC
area = 0.79 for women, 0.78 for men, 0.79 for patients aged 70-79 years, and
0.79 for patients aged 80 years or older).
We have developed a prognostic index that can be used as a simple point
scoring system at the bedside to stratify elderly medical patients into high-,
intermediate-, and low-risk groups for mortality during the year following
hospital discharge. This index includes risk factors from each of the 4 domains
that we hypothesized were associated with 1-year mortality: demographic variables,
medical diagnoses, functional status, and laboratory values. This finding
is consistent with the clinical scenario that in many older adults the cause
of death is multifactorial.29 Our index emphasizes
the importance of considering multiple domains when assessing prognosis in
older patients and adds to our understanding of the complexity of mortality
prediction in the elderly population.
Our study, by demonstrating the prognostic importance of ADL function,
provides further evidence supporting routine assessment of functional status
in hospitalized older adults. Consistent with other studies, we found that
measures of functional status add important information about risk for 1-year
mortality beyond that provided by medical diagnoses or physiologic measures.13-15 This is probably
because functional status reflects the severity and end result of many different
illnesses and psychosocial factors. However, the importance of assessing functional
status extends well beyond its value as a prognostic measure. Assessing ADL
function of hospitalized older adults is essential for providing quality care
after discharge. Without assessing ADL function, it is difficult to advise
a patient about long-term care needs, assess the need for home care and other
supportive services, or evaluate the needs of a patient's caregiver.30,31 While physicians often fail to assess
their patients' functional status,32 the ADL
questions we asked in this study took only a few minutes to administer. Also,
the ease of reviewing functional information routinely obtained by other disciplines,
such as nursing or physical therapy, should improve as more hospitals are
developing systematic methods for measuring and recording functional status
in older adults.13
Only 2 of the medical diagnoses from the Charlson comorbidity index
(congestive heart failure and cancer) remained independently associated with
mortality. Other illnesses, such as dementia and cerebrovascular disease,
which were highly associated with mortality in bivariate analyses, no longer
added to the prognostic estimate after adjustment for functional status. This
suggests that decrements in functional status reflect the severity of dementia
and cerebrovascular disease better than they reflect the severity of congestive
heart failure or cancer.
Additional risk factors that remained associated with an increased risk
for mortality after adjustment for comorbid illness and functional status
included male sex and laboratory values for creatinine and albumin. Others
have argued that the association between creatinine and mortality may be explained
by the direct negative effects of renal dysfunction on multiple organ systems
or may be reflective of generalized decreased tissue perfusion.33
Albumin also is a strong predictor of mortality in this and other studies
probably because it is both a marker of malnutrition as well as general disease
severity.23 In contrast, age did not add to
the predictive power of our index after we adjusted for comorbidity and functional
status. This suggests that the association of older age with mortality may
be explained by greater disease burden and functional impairment in older
patients consistent with other studies.12,34,35
By combining functional status, comorbid illnesses, sex, and laboratory
values, our index performed better in predicting 1-year mortality than other
available prognostic indices that focus only on comorbid illnesses or physiologic
measures. For example, the Charlson comorbidity index had a ROC curve area
of 0.68 for 1-year mortality in the validation cohort, and APACHE II, a physiologic
index developed for ICU patients, had a ROC area of 0.59.15
Since mortality in older adults is often dependent on many factors, it makes
sense that an index combining multiple domains of risk would have better discrimination
than indices that consider only a single domain.
In comparison with other prognostic indices that consider multiple domains
of risk,8,11,13 our
index is easier to use while maintaining prognostic accuracy. Our prognostic
index, based on 6 risk factors and an additive point system, performed well
in stratifying older adults into risk groups for 1-year mortality. Our index
had good discrimination, with large differences in 1-year mortality between
the low-risk and high-risk groups. Our index was successfully validated in
an independent patient sample from a different site with no decrement in discrimination
(ROC area = 0.79) and only a mild decrease in calibration, demonstrating our
index's generalizability to another location and patient group.16
Clinicians should use our index to supplement and lend confidence to
their judgments about prognosis, rather than to replace their clinical judgment.
Previous work suggests that clinicians' abilities to estimate prognoses are
about equal to that of prognostic indices. However, combining prognostic indices
and clinician estimates results in more accurate estimates than either alone.8,11,36 Further, a recent
survey of clinicians suggests that many clinicians do not fully consider prognosis
in their clinical decision making and avoid discussing prognosis with patients
because they lack confidence in their prognostic estimates.37
This is despite evidence that most patients would like clinicians to discuss
prognosis with them.3,8 One use
of objective prognostic indices may be to increase clinicians' confidence
in their own prognostic estimates, enhancing their willingness to discuss
prognosis with their patients.
Many patients may be concerned about their prognosis when they experience
a major event like hospitalization. Our index may be useful to clinicians
in initiating and guiding discussions about prognosis with patients at both
low and high risk for 1-year mortality. For example, an 80-year-old woman
admitted for pneumonia with no ADL dependencies at discharge and no major
comorbid conditions may be relieved to know that her 1-year risk of death
is similar to an 80-year-old woman living in the general community who has
not been hospitalized (<10%).38 In contrast,
an 80-year-old man who is dependent in 3 ADLs at discharge, has a creatinine
level of 3.5 mg/dL (309.4 µmol/L) and an albumin level of 2.8 g/dL has
a greater than 60% risk of death in the ensuing year. Such information may
stimulate a conversation about the goals of care.
Our study has several limitations. First, we did not have information
about clinical care or patient preferences after discharge so that in some
cases poor survival may have been affected by decisions to limit treatment.
Also, there are different ways to ask about ADLs. For example, inquiring about
difficulty instead of dependence would have resulted in higher levels of ADL
impairment.39 Users of our index should be
aware that the performance of our index will differ if the way of inquiring
about ADLs is changed from our method. Finally, since the patients were involved
in a study to improve functional outcomes, it is possible that the selection
process for the study or the process of being observed in a study could affect
the generalizability of our index. However, this seems unlikely because the
intervention did not significantly improve outcomes after 1 year and the patients
randomly selected for the study were representative of those admitted to the
medical services of the 2 hospitals.17 As with
all prognostic indices, the true validity and generalizability of our index
needs to be established by cumulative testing to determine if the index remains
accurate in other locations and groups of patients.16,28
In summary, our index provides a potentially useful prognostic tool
to estimate the likelihood of 1-year mortality after hospitalization for older
medical patients. The index uses 6 risk factors, all of which are easily available
at hospital discharge, and a simple additive point system. The index had good
discrimination and calibration, and it generalized well in an independent
sample of patients at a different site. These characteristics suggest that
our index may be useful for guiding clinical care and for risk adjustment.
1.Palmer RM. Acute care. In: Hazzard WR, Blass JP, Ettinger WH, Halter JB, Ouslander JG, eds. Principles of Geriatric Medicine and Gerontology. 4th ed.
New York, NY: McGraw-Hill; 1999:483-492.
2.Creditor MC. Hazards of hospitalization of the elderly.
Ann Intern Med.1993;118:219-223.Google Scholar 3.Christakis NA. Death Foretold: Prophecy and Prognosis in Medical
Care. Chicago, Ill: University of Chicago Press; 1999.
4.Winograd CH. Targeting strategies: an overview of criteria and outcomes.
J Am Geriatr Soc.1991;39:25S-35S.Google Scholar 5.Lynn J, Teno JM, Harrell FE. Accurate prognostications of death: opportunities and challenges for
clinicians.
West J Med.1995;163:250-257.Google Scholar 6.Iezzoni LI. Risk Adjustment for Measuring Healthcare Outcomes. 2nd ed. Chicago, Ill: Health Administration Press; 1997.
7.Dudley RA, Miller RH, Korenbrot TY, Luft HS. The impact of financial incentives on quality of health care.
Milbank Q.1998;76:649-686.Google Scholar 8.Knaus WA, Harrell FE, Lynn J.
et al. The SUPPORT prognostic model: objective estimates of survival for seriously
ill hospitalized older adults.
Ann Intern Med.1995;122:191-203.Google Scholar 9.Knaus WA, Wagner DP, Draper EA.
et al. The APACHE III prognostic system: risk prediction of hospital mortality
for critically ill hospitalized adults.
Chest.1991;100:1619-1636.Google Scholar 10.Lemeshow S, Teres D, Klar J, Avrunin JS, Gehlbach SH, Rapoport J. Mortality probability models (MPM II) based on an international cohort
of intensive care unit patients.
JAMA.1993;270:2478-2486.Google Scholar 11.Teno JM, Harrell FE, Knaus WA.
et al. Prediction of survival for older hospitalized patients: the HELP survival
model.
J Am Geriatr Soc.2000;48:S16-S24.Google Scholar 12.Fried LP, Kronmal RA, Newman AB.
et al. Risk factors for 5-year mortality in older adults: the cardiovascular
health study.
JAMA.1998;279:585-592.Google Scholar 13.Inouye SK, Peduzzi PN, Robison JT, Hughes JS, Horwitz RI, Concato J. Importance of functional measures in predicting mortality among older
hospitalized patients.
JAMA.1998;279:1187-1193.Google Scholar 14.Reuben DB, Rubenstein LV, Hirsch SH, Hays RD. Value of functional status as a predictor of mortality: results of
a prospective study.
Am J Med.1992;93:663-669.Google Scholar 15.Covinsky KE, Justice AC, Rosenthal GE, Palmer RM, Landefeld CS. Measuring prognosis and case mix in hospitalized elders: the importance
of functional status.
J Gen Intern Med.1997;12:203-208.Google Scholar 16.Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information.
Ann Intern Med.1999;130:515-524.Google Scholar 17.Counsell SR, Holder CM, Liebenauer LL, Palmer RM, Fortinsky RH, Kresevic DM. Effects of a multicomponent intervention on functional outcomes and
process of care in hospitalized older patients.
J Am Geriatr Soc.2000;48:1572-1581.Google Scholar 18.Pfeiffer E. A short portable mental status questionnaire for the assessment of
organic brain deficit in elderly patients.
J Am Geriatr Soc.1975;23:433-441.Google Scholar 19.Katz S, Ford AB, Moskowitz RW, Jackson BA, Jaffe MW. Studies of illnesses in the aged: the index of ADL: a standardized
measure of biological and psychosocial function.
JAMA.1963;185:914-919.Google Scholar 20.Knaus WA, Draper EA, Wagner WP, Zimmerman JE. APACHE II: a severity of disease classification system.
Crit Care Med.1985;13:818-829.Google Scholar 21.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal
studies: development and validation.
J Chronic Dis.1987;40:373-383.Google Scholar 22.Meyer BR, Bellucci A. Renal function in the elderly.
Cardiol Clin.1986;4:227-234.Google Scholar 23.Corti M, Guralnik JM, Salive ME, Sorkin JD. Serum albumin level and physical disability as predictors of mortality
in older persons.
JAMA.1994;272:1036-1042.Google Scholar 24.Rich-Edwards JW, Corsano KA, Stampfer MJ. Test of the National Death Index and Equifax Nationwide Death Search.
Am J Epidemiol.1994;140:1016-1019.Google Scholar 25.Concato J, Feinstein AR, Holford TR. The risk of determining risk with multivariable models.
Ann Intern Med.1993;118:201-210.Google Scholar 26.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic
(ROC) curve.
Radiology.1982;143:29-36.Google Scholar 27.Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules: applications and methodological standards.
N Engl J Med.1985;313:793-799.Google Scholar 28.McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG, Richardson WS. Users' guides to the medical literature, XXII: how to use articles
about clinical decision rules.
JAMA.2000;284:79-84.Google Scholar 29.Kohn RR. Cause of death in very old people.
JAMA.1982;247:2793-2797.Google Scholar 30.Fortinsky RH, Covinsky KE, Palmer RM, Landefeld CS. Effects of functional status changes before and during hospitalization
on nursing home admission of older adults.
J Gerontol A Biol Sci Med Sci.1999;54:M521-M526.Google Scholar 31.Covinsky KE, Palmer RM, Counsell SR, Pine ZM, Walter LC, Chren M. Functional status before hospitalization in acutely ill older adults:
validity and clinical importance of retrospective reports.
J Am Geriatr Soc.2000;48:164-169.Google Scholar 32.Calkins DR, Rubenstein LV, Cleary PD.
et al. Failure of physicians to recognize functional disability in ambulatory
patients.
Ann Intern Med.1991;114:451-454.Google Scholar 33.Browner WS, Li J, Mangano DT. In-hospital and long-term mortality in male veterans following noncardiac
surgery.
JAMA.1992;268:228-232.Google Scholar 34.Davis RB, Iezzoni LI, Phillips RS, Reiley P, Coffman GA, Safran C. Predicting in-hospital mortality: the importance of functional status
information.
Med Care.1995;33:906-921.Google Scholar 35.Hamel MB, Davis RB, Teno JM.
et al. Older age, aggressiveness of care and survival for seriously ill, hospitalized
adults.
Ann Intern Med.1999;131:721-728.Google Scholar 36.Charlson ME, Hollenberg JP, Hou J, Cooper M, Pochapin M, Pecker M. Realizing the potential of clinical judgment: a real-time strategy
for predicting outcomes and cost for medical inpatients.
Am J Med.2000;109:189-195.Google Scholar 37.Christakis NA, Iwashyna TJ. Attitude and self-reported practice regarding prognostication in a
national sample of internists.
Arch Intern Med.1998;158:2389-2395.Google Scholar 39.Gill TM, Robison JT, Tinetti ME. Difficulty and dependence: two components of the disability continuum
among community-living older persons.
Ann Intern Med.1998;128:96-101.Google Scholar