Bars depict the relative percentage that adjusted same-hospital readmissions (SHRs) underestimate AHRs, calculated as |(AHR − SHR)/AHR|, across all 177 hospitals. Rates are adjusted for case mix (age, sex, and chronic conditions). The median percentage of underestimation was 25.2% (interquartile range, 16.6%-34.3%).
eFigure 1. Initial Exclusions
eFigure 2. Calculation of Excess Readmission Ratios
eFigure 3. Adjusted Same-Hospital Versus All-Hospital Readmission Rates
eTable. Model Used for Case-Mix Adjustment of Same-Hospital and All-Hospital Readmission Rates
Alisa Khan, Mari M. Nakamura, Alan M. Zaslavsky, Jisun Jang, Jay G. Berry, Jeremy Y. Feng, Mark A. Schuster. Same-Hospital Readmission Rates as a Measure of Pediatric Quality of Care. JAMA Pediatr. 2015;169(10):905–912. doi:10.1001/jamapediatrics.2015.1129
Health care systems, payers, and hospitals use hospital readmission rates as a measure of quality. Although hospitals can track readmissions back to themselves (hospital A to hospital A), they lack information when their patients are readmitted to different hospitals (hospital A to hospital B). Because hospitals lack different-hospital readmission (DHR) data, they may underestimate all-hospital readmission (AHR) rates (hospital A to hospital A or B).
To determine the prevalence of 30-day pediatric DHRs; to assess the effect of DHR on readmission performance; and to identify patient and hospital characteristics associated with DHR.
Design, Setting, and Participants
We analyzed all-payer inpatient claims for 701 263 pediatric discharges (patients aged 0-17 years) from 177 acute care hospitals in New York State from January 1, 2005, through November 30, 2009, to identify 30-day same-hospital readmissions (SHRs), DHRs, and AHRs. Data analysis was performed from March 12, 2013, through April 6, 2015. We compared excess readmission ratios (calculated per the Medicare formula) using SHRs and AHRs to determine what might happen if the federal formula were applied to a specific state and to evaluate how often hospitals might accurately anticipate—using data available to them—whether they would incur penalties (excess readmission ratio >1) for readmissions. Using multivariate logistic regression, we identified patient- and hospital-level predictors of DHR vs SHR.
Main Outcomes and Measures
The proportion of DHRs vs SHRs, AHR and SHR rates, and excess readmissions.
Different-hospital readmissions constituted 13.9% of 31 325 AHRs. At the individual hospital level, the median (interquartile range) percentage of DHRs was 21.6% (12.8%-39.1%). The median (interquartile range) adjusted AHR rate was 3.4% (3.0%-4.1%), 38.9% higher than the median adjusted SHR rate of 2.5% (2.0%-3.4%) (P < .001). Excess readmission ratios using SHRs inaccurately anticipated penalties (changed from >1 to ≤1 or vice versa) for 20 of the 177 hospitals (11.3%); all were nonchildren’s hospitals and 18 of 20 (90.0%) were nonteaching hospitals. Characteristics associated with higher odds ratios (ORs) (reported with 95% CIs) of DHR in multivariate analyses included being younger (compared with age <1 year, ORs [95% CIs] for the other age categories ranged from 0.76 [0.66-0.88] to 0.85 [0.73-0.99]); being white (ORs [95% CIs] for nonwhite race/ethnicity ranged from 0.74 [0.65-0.84] to 0.88 [0.79-0.99]); having private insurance (1.14 [1.04-1.24]); having a chronic condition indicator for a mental disorder (1.33 [1.13-1.56]) or a disease of the nervous system (1.37 [1.20-1.57]) or circulatory system (1.20 [1.00-1.43]); and admission to a nonchildren’s (1.62 [1.01-2.60]), urban (ORs for nonurban hospitals ranged from 0.35 [0.24-0.52] to 0.36 [0.21-0.64]), or lower-volume (0.73 [0.64-0.84]) hospital (P < .05 for each).
Conclusions and Relevance
Different-hospital readmissions differentially affect hospitals’ pediatric readmission rates and anticipated performance, making SHRs an incomplete surrogate for AHRs—particularly for certain hospital types. Failing to incorporate DHRs into readmission measurement may impede quality assessment, anticipation of penalties, and quality improvement.
Readmissions have become a standard measure of the quality of the US health care system.1 The Centers for Medicare & Medicaid Services publicly reports 30-day readmission rates for Medicare beneficiaries2 and reduces Medicare payments to hospitals with excess readmissions.3- 5 State Medicaid programs seek to reduce readmissions6 and use readmission rates in accountability programs, with some imposing financial penalties for high rates.7- 10
Among pediatric patients admitted to acute care children’s hospitals, 6.5% experience an unplanned readmission within 30 days.11 National reporting agencies place increasing attention on pediatric readmissions.3,5 The National Quality Forum recently endorsed its first pediatric all-condition readmission measure.12
Hospitals themselves increasingly focus on measuring and reducing readmissions as part of quality improvement efforts. However, when calculating readmission rates and assessing performance over time or against external benchmarks, hospitals typically can identify only readmissions back to themselves (same-hospital readmission [SHR]; ie, hospital A to hospital A) because they lack data on readmissions to different hospitals (different-hospital readmission [DHR]; ie, hospital A to hospital B).13 As a result, hospitals may underestimate true readmission rates (all-hospital readmission [AHR]; ie, hospital A to hospital A or B).
Studies find that approximately 20% of adult readmissions are to a different hospital,13- 16 and SHR rates underestimate mean AHR rates by approximately 5% in adults with heart failure and those who have undergone surgery, with the degree of underestimation varying across hospitals.14,17 Little is known about the prevalence of DHR among pediatric patients. To evaluate the importance of including DHRs in pediatric readmission rates, we estimated the prevalence of 30-day pediatric DHR, assessed the effect of DHRs on hospitals’ estimated readmission performance, and identified patient and hospital characteristics associated with DHR.
Although payers have access to all-hospital readmissions (readmissions to the same or a different hospital), hospitals can generally only track readmissions back to themselves (same-hospital readmissions).
Different-hospital readmissions constituted 13.9% of all-hospital pediatric readmissions in New York State during a 5-year period and varied by patient and hospital characteristics.
For individual hospitals, the median adjusted all-hospital readmission rate was 38.9% higher than the median adjusted same-hospital readmission rate.
Same-hospital readmissions underestimate true all-hospital readmission rates in a variable fashion by 0.6 to 68 relative percentage points across hospitals, with certain hospital types (eg, nonchildren’s hospitals) being particularly prone to underestimation.
Failing to incorporate different-hospital readmissions into readmission measurement may impede hospitals in their quality assessment, anticipation of penalties, and benchmarking efforts.
We analyzed hospital discharges from January 1, 2005, through November 30, 2009, in the Agency for Healthcare Research and Quality’s New York State Inpatient Database,18,19 an all-payer claims database with patient linkages across hospitalizations. We included patients aged 0 to 17 years at the time of admission to 177 general acute care hospitals, including 12 children’s hospitals. We combined the records of patients transferred between hospitals into a single admission and attributed readmission to the second hospital. The institutional review board of Boston Children’s Hospital approved the study with a waiver of informed consent. Patient data were deidentified.
We excluded admissions with primary diagnoses of healthy newborn births (not for disease management), obstetric care (not within pediatric purview), and mental health conditions (different hospital-level readmission patterns than admissions for other conditions). We excluded index admissions ending in death or departure against medical advice and records with incomplete or inconsistent data for key variables (ie, hospital type, patient identifier, admission and/or discharge date, disposition status, age, diagnosis code, or sex) (eFigure 1 in the Supplement).
We also excluded readmissions for planned procedures and chemotherapy. To identify planned procedures, pediatric specialists indicated procedure codes from the International Classification of Diseases, Ninth Revision, Clinical Modification that are scheduled at least 24 hours in advance in more than 80% of cases and that might necessitate hospitalization.
Our primary outcome consisted of proportion of DHRs vs SHRs. Secondary outcomes included SHR and AHR rates and hospital excess readmissions.
An index admission—defined as an eligible initial admission—served as the starting point for enumerating readmissions. A readmission was defined as an unplanned admission within 30 days of discharge from an index admission. Admissions were treated as new index admissions only if they occurred more than 30 days after the previous index admission.11,14 We calculated overall readmission rates as the number of index admissions followed by at least 1 readmission (same + different) within 30 days divided by the total number of index admissions.
We defined an SHR as a first readmission (even if preceded by a DHR) within 30 days to the same hospital from which the patient was discharged. The SHR rate was calculated as the number of index admissions with an SHR within 30 days divided by the total number of index admissions. We defined a DHR as a readmission within 30 days that was to a different hospital and not preceded by an SHR. The DHR rate was calculated as the number of index admissions with a DHR within 30 days divided by the total number of index admissions.
To assess potential policy impact, we calculated excess readmission ratios (ERRs) using the Medicare formula2 (eFigure 2 in the Supplement) used by the Centers for Medicare & Medicaid Services to calculate which hospitals incur federal penalties (ERR >1). The ERR is calculated as risk-adjusted predicted readmissions divided by risk-adjusted expected readmissions. We compared ERRs using SHRs and AHRs to determine—using data available to them—how often hospitals might accurately anticipate whether they might incur penalties if this federal formula was applied to a specific state.
Patient populations vary across hospitals in their distributions of age, sex, and chronic conditions, characteristics thought to be associated intrinsically with readmission risk, independent of quality of care.20 We adjusted readmission rates for age, sex, and chronic conditions as recorded during the index hospitalization using a previously developed case-mix model for pediatric readmissions (eTable in the Supplement).12 We categorized age using clinically and developmentally appropriate ranges as less than 1, 1 to 4, 5 to 7, 8 to 11, and 12 to 17 years. We used the Agency for Healthcare Research and Quality’s Chronic Condition Indicator (CCI) system,21 which dichotomizes diagnosis codes from the International Classification of Diseases, Ninth Revision, Clinical Modification, as chronic or nonchronic and places each code into 1 of 18 organ systems, disease categories, or other categories. We included 17 of the 18 CCIs, excluding CCI 11 (“complications of pregnancy, childbirth, and the puerperium”), which appeared only 4 times as a secondary diagnosis. Although we excluded hospitalizations with a primary mental health diagnosis, we included CCI 5 (“mental disorders”) because many records remained with a secondary mental health diagnosis. To account for risk associated with multiple chronic conditions, we adjusted for the number of body systems in which a patient had a chronic condition (0-1, 2, 3, or ≥4).
To determine the effect of DHRs on individual hospital readmission rates, we calculated crude and case-mix–adjusted SHR and AHR rates and assessed hospital variation in readmission rates, excess readmissions, and magnitude of random and systematic error. We quantified the increase in the coefficient of variation of random error in SHR rates compared with AHR rates by comparing the variability of these rates. We performed a bivariate hierarchical model of SHR with hospital random intercept only to estimate the variance of hospital random intercepts. Using this hospital variance, along with the ratio of SHR to AHR, we estimated the factor (mean [SD]) by which SHR rates would need to be multiplied to generate AHR rates. We compared the relative percentage increase between SHR and AHR rates; a higher percentage of difference indicated a larger potential for bias when using SHR instead of AHR.
We assessed patient and index admission hospital characteristics associated with DHR vs SHR. In addition to age, sex, and chronic conditions, patient characteristics included insurance status, race/ethnicity (using categories derived from definitions of the Agency for Healthcare Research and Quality22), and length of stay. Hospital characteristics included teaching status, children’s hospital status, annual admission volume (logarithm transformed), and location defined using rural-urban commuting area codes (large rural city or town, small and isolated small rural town, and urban).23
We modeled bivariate associations between patient or hospital characteristics and DHR vs SHR using logistic regression with variances adjusted for clustering by hospital and selected as candidates for multivariate modeling variables at P < .2. We evaluated multivariate associations using hierarchical logistic regression with a random effect for each hospital and fixed effects for patient and hospital characteristics. We used SAS software (version 9.3; SAS Institute Inc) for statistical analysis.
We analyzed 701 263 index admissions, of which 56.4% were for nonwhite patients and 47.2% were for Medicaid beneficiaries (Table 1). Most index admissions (54.2%) were for patients with no chronic conditions. Admissions were predominantly to urban (88.5%), nonchildren’s (63.1%), and teaching (65.3%) hospitals (Table 2).
Of 31 325 AHRs, 26 985 (86.1%) were SHRs and 4340 (13.9%) were DHRs. The overall 30-day unadjusted mean SHR rate was 3.8%, whereas the overall unadjusted mean AHR rate was 4.5%. At the individual hospital level, DHRs constituted a median 21.6% of AHRs (interquartile range [IQR], 12.8%-39.1%).
Individual case-mix–adjusted AHR rates ranged from 1.2% to 8.5%, and SHR rates ranged from 0.6% to 8.8% (eFigure 3 in the Supplement). The median adjusted AHR rate was 3.4% (IQR, 3.0%-4.1%), 38.9% higher than the median adjusted SHR rate of 2.5% (IQR, 2.0%-3.4%) (P < .001). In other words, the median observed SHR rate was 28.0% lower than the median true AHR rate. We present relative differences by the inverse (the percentage by which the observed SHR rates are lower than the true AHR rates).
By comparing the coefficients of variation of random error between SHR and AHR rates, we found that not including DHRs and thus reducing the number of observed readmissions led to a 14.5% increase in measurement error for SHR rates compared with AHR rates. We found significant hospital-level variability in the contribution of DHRs to AHR rates (covariance of hospital intercept, 0.83 [P < .001]). The percentage of relative difference between the adjusted SHR and AHR rates varied by hospital (Figure); the median difference was 25.2% (IQR, 16.6%-34.3%). In addition, using the hospital variance and mean SHR:AHR ratio, we estimated that SHR rates for most hospitals require multiplication by a factor of 1.12 to 1.75 to estimate true AHR rates.
The contribution of DHRs to AHR rates varied by hospital characteristic (P < .05 for children’s and teaching hospital status). Nonchildren’s hospitals, nonteaching hospitals, and lower-volume hospitals had greater underestimation of true AHR rates when using SHR rates than their counterparts. For instance, excluding DHRs underestimated rates differentially for nonchildren’s vs children’s hospitals (by 25.67% vs 9.68%, respectively) and for nonteaching vs teaching hospitals (by 27.41% vs 14.20%, respectively). Excluding DHRs underestimated the mean adjusted readmission rate more for low- than high-volume hospitals (by 30.49% for those in the lowest-volume quartile vs 15.56% for those in the highest quartile) (Table 3).
When we compared anticipated ERRs (calculated using SHRs) with true ERRs (calculated using AHRs), ERRs moved from greater than 1 to 1 or less (penalty to no penalty) or vice versa for 20 of 177 hospitals (11.3%; Table 3). Among the 10 hospitals whose ERRs shifted to 1 or less (improved), all were nonchildren’s and 8 were nonteaching hospitals. The 10 hospitals whose ERRs shifted to greater than 1 (worsened) were all nonchildren’s and nonteaching hospitals, and 6 were urban hospitals.
In bivariate analyses, patient characteristics associated with greater odds of DHR vs SHR included younger age (<1 year), certain CCI categories, and fewer (0-1) body systems affected by chronic conditions (Table 1). Hospital characteristics associated with higher odds of DHR included nonchildren’s or nonteaching status, lower volume, and rural location (Table 2).
In multivariate analyses, age, race/ethnicity, insurance status, certain CCI categories, and certain hospital characteristics were significantly associated with DHR (P < .05 for each). Patient characteristics associated with higher odds of DHR included age younger than 1 year, white race, private insurance, and mental health, nervous system, and circulatory disorders (Table 1); hospital characteristics associated with higher odds of DHR included nonchildren’s status, lower volume, and urban location (Table 2).
We found that approximately 1 in 7 pediatric readmissions in New York during a 5-year period were to a different hospital. For individual hospitals, the median adjusted AHR rate (3.4%) was 38.9% higher than the median adjusted SHR rate (2.5%). We found substantial hospital-level variability in contributions of DHRs to AHR rates, with individual hospitals underestimating AHR rates by 0.6 to 68 relative percentage points. In general, DHRs constituted a greater proportion of readmissions for nonchildren’s, lower-volume, and urban hospitals and for patients who were younger, were white, were privately insured, or had certain chronic conditions.
Excluding DHRs contributed differentially based on hospital characteristics to random and systematic error in calculating readmission rates. When SHR rates were used instead of AHR rates, a 15% increase in the random error of measurement resulted owing to reduced numbers of readmissions. Same-hospital readmission rates systemically underestimated AHR rates more for nonchildren’s and nonteaching hospitals. For instance, nonteaching, nonchildren’s, urban hospitals had a 30% lower mean adjusted SHR rate (2.40%) than AHR rate (3.40%), whereas teaching, children’s, urban hospitals had a 10% lower mean adjusted SHR rate (4.35%) than AHR rate (4.82%).
We are aware of no other studies that examine SHR vs AHR rates in pediatrics. Our overall SHR rate of 3.8% is lower than the 6.5% pediatric readmission rate reported by Berry et al,11 perhaps because that study examined children’s hospitals across the United States whereas this study assessed children’s and nonchildren’s hospitals in 1 state.
In pediatrics, as in adults, SHR appears to be a suboptimal surrogate for AHR. We found an all-condition DHR rate of 13.9%, in contrast to DHR rates of approximately 20% for adult conditions.13- 17 The relative percentage by which our mean adjusted SHR rates underestimated AHR rates (24.2%) was similar to relative differences found in adult patients with heart failure and those who had undergone surgery (20% and 36%, respectively),14,17 and we also found substantial variability in the degree of underestimation across hospitals.
A number of reasons for variability in DHR risk exist. Variability by patient characteristics may reflect differences in access to and quality of hospitals. Patients may be more likely to experience DHR if they have greater access to a variety of hospitals, for example, because their insurance allows more flexibility or they live in an area with higher hospital density. Mental health, nervous system, and circulatory disorders—CCIs associated with higher odds of DHR—may be conditions with a higher likelihood of acute emergencies for which patients may go to the nearest hospital rather than a specialized center. For instance, a patient with a seizure may be rushed to the closest hospital regardless of where the prior admission occurred. In contrast, patients with chronic conditions that require highly coordinated multispecialty care—for example, oncologic, hematologic, or genitourinary conditions—may be less likely to be readmitted to a different hospital because their subspecialty and follow-up care is concentrated at a particular hospital. Hospitals may therefore systematically underestimate rates for certain populations and fail to target them in readmission interventions.
Different-hospital readmissions may be more likely for nonchildren’s and low-volume hospitals because families may prefer readmission to hospitals with greater expertise with pediatric, complex, rare diagnoses. In addition, families dissatisfied with the quality of care during the initial hospitalization may seek a different hospital on readmission. The limited ability of nonchildren’s and nonteaching hospitals to measure readmissions accurately is especially problematic because it impairs their ability to recognize the need for improvement.
Although payers have access to DHR information, hospitals’ lack of access to these data prevents them from assessing accurate readmission rates. This gap may undermine quality improvement and benchmarking efforts, which increasingly include initiatives to measure and reduce readmissions. For example, based on SHR rates, hospitals may conclude that quality improvement efforts have had a limited effect on readmission rates, when in truth their efforts may simply be preventing DHRs more than SHRs. Hospitals typically use SHR data when pooling data for benchmarking purposes. Children’s hospitals around the country share SHR rates in the Solutions for Patient Safety National Children’s Network for the purpose of reducing readmissions.24 However, without DHR data, hospitals, networks, and patients are hampered in assessing and comparing hospital performance. Furthermore, the 38.9% relative difference in the median AHR vs SHR rate has potentially actionable consequences, given that quality assessment, accountability, and benchmarking efforts focus on even small changes in readmission performance.
Being handicapped in self-assessment of their performance places hospitals in the difficult position of being accountable for an outcome they do not have the ability to measure accurately and cannot calculate without other hospitals’ information. Hospitals typically provide SHR rates for voluntary participation in public reporting, which may falsely reassure some hospitals and their patients about readmission performance. If readmission penalties were determined using ERRs as in Medicare’s Hospital Readmissions Reduction Program,4 exclusion of DHRs would substantially affect whether hospitals could predict penalties, especially for nonchildren’s and nonteaching hospitals, with 11.3% of hospitals drawing incorrect conclusions.
In addition to being necessary for accurate evaluation of readmission performance, recognition and prevention of DHRs is important because DHRs have undesirable consequences for individual patients and the overall health system. Because communication and transfer of records among hospitals is often suboptimal, DHRs may lead to duplication of tests, procedures, and treatments, perhaps resulting in unnecessary interventions and costs or delays in diagnosis and treatment. In 1 adult study,15 DHRs were related to increased overall payments with no associated reduction in mortality.
Our study has several limitations. We analyzed data from a single state, which may limit generalizability. Our data set does not capture readmission information for patients readmitted to hospitals located in different states, which may lead to underestimation of the effect of DHRs on rates for hospitals that have higher out-of-state readmissions. In addition, we relied on administrative data to identify admissions, patient characteristics, and linkage of records; such data provide incomplete clinical information and are subject to errors and variation in hospital coding practices.25 Although the quality of linkages in this data set is quite good, nonlinkages do exist. Because nonlinkages are probably more common for DHRs, they may lead to underestimation of DHRs and affect predictors of DHR if certain patient or hospital types are associated with more failed linkages.
This study highlights the usefulness of aggregated pediatric data sets, such as state all-payer databases, that facilitate accurate and meaningful comparisons of readmissions and other quality measures. Such centralized databases provide hospitals with AHR data and permit measurement of readmission rates that are adjusted for case mix across hospitals. At present, even in states that have such data sets, individual hospitals do not typically receive timely enough feedback to inform quality efforts. Using such data sets to provide hospitals with centrally estimated readmission rates at timely intervals would allow hospitals to better assess their quality improvement efforts and anticipate rates used in accountability programs.
To our knowledge, this study is the first to examine the effect of DHR on pediatric readmission rates. We found that DHRs account for 13.9% of pediatric readmissions and affect readmission rates and error in their estimation in a variable fashion, with a disproportionate effect on certain hospital and patient types. Without DHR data, hospitals cannot track their readmission performance accurately over time, conduct meaningful hospital comparisons, or anticipate rates that could be used to assign financial penalties. As increasing attention is directed toward improving quality by reducing readmissions, readmissions should be enumerated accurately—namely, DHRs should be included in readmission rate calculations. Data sets that allow calculation of AHRs by incorporating DHRs bolster the validity of readmission rates as a quality measure.
Accepted for Publication: April 16, 2015.
Corresponding Author: Alisa Khan, MD, MPH, Division of General Pediatrics, Boston Children’s Hospital, 21 Autumn St, Room 200.2, Boston, MA 02215 (firstname.lastname@example.org).
Published Online: August 3, 2015. doi:10.1001/jamapediatrics.2015.1129.
Author Contributions: Dr Khan and Ms Jang had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Khan, Nakamura, Schuster.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analyses: Khan, Zaslavsky, Jang.
Obtained funding: Berry, Schuster.
Administrative, technical, or material support: Feng.
Study supervision: Nakamura, Schuster.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by the US Department of Health and Human Services Agency for Healthcare Research and Quality (AHRQ) and Centers for Medicare & Medicaid Services, CHIPRA (Children’s Health Insurance Program Reauthorization Act) Pediatric Quality Measures Program Centers of Excellence, under grant U18 HS 020513 (principal investigator, Dr Schuster). Dr Khan was supported by grant T32HS000063 from the AHRQ.
Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ.