Association of the 2011 ACGME Resident Duty Hour Reforms With Mortality and Readmissions Among Hospitalized Medicare Patients

Importance  Patient outcomes associated with the 2011 Accreditation Council for Graduate Medical Education (ACGME) duty hour reforms have not been evaluated at a national level.

Objective  To evaluate the association of the 2011 ACGME duty hour reforms with mortality and readmissions.

Design, Setting, and Participants  Observational study of Medicare patient admissions (6 384 273 admissions from 2 790 356 patients) to short-term, acute care, nonfederal hospitals (n = 3104) with principal medical diagnoses of acute myocardial infarction, stroke, gastrointestinal bleeding, or congestive heart failure or a Diagnosis Related Group classification of general, orthopedic, or vascular surgery. Of the hospitals, 96 (3.1%) were very major teaching, 138 (4.4%) major teaching, 442 (14.2%) minor teaching, 443 (14.3%) very minor teaching, and 1985 (64.0%) nonteaching.

Exposure  Resident-to-bed ratio as a continuous measure of hospital teaching intensity.

Main Outcomes and Measures  Change in 30-day all-location mortality and 30-day all-cause readmission, comparing patients in more intensive relative to less intensive teaching hospitals before (July 1, 2009–June 30, 2011) and after (July 1, 2011–June 30, 2012) duty hour reforms, adjusting for patient comorbidities, time trends, and hospital site.

Results  In the 2 years before duty hour reforms, there were 4 325 854 admissions with 288 422 deaths and 602 380 readmissions. In the first year after the reforms, accounting for teaching hospital intensity, there were 2 058 419 admissions with 133 547 deaths and 272 938 readmissions. There were no significant postreform differences in mortality accounting for teaching hospital intensity for combined medical conditions (odds ratio [OR], 1.00; 95% CI, 0.96-1.03), combined surgical categories (OR, 0.99; 95% CI, 0.94-1.04), or any of the individual medical conditions or surgical categories. There were no significant postreform differences in readmissions for combined medical conditions (OR, 1.00; 95% CI, 0.97-1.02) or combined surgical categories (OR, 1.00; 95% CI, 0.98-1.03). For the medical condition of stroke, there were higher odds of readmissions in the postreform period (OR, 1.06; 95% CI, 1.001-1.13). However, this finding was not supported by sensitivity analyses and there were no significant postreform differences for readmissions for any other individual medical condition or surgical category.

Conclusions and Relevance  Among Medicare beneficiaries, there were no significant differences in the change in 30-day mortality rates or 30-day all-cause readmission rates for those hospitalized in more intensive relative to less intensive teaching hospitals in the year after implementation of the 2011 ACGME duty hour reforms compared with those hospitalized in the 2 years before implementation.

Quiz Ref IDOn July 1, 2011, the Accreditation Council for Graduate Medical Education (ACGME) implemented new duty hour reforms for all ACGME-accredited residency programs.¹ The revisions maintain the weekly limit of 80 hours set forth by the 2003 duty hour reforms² but reduced the work hour limit from 30 consecutive hours to 16 hours for first-year residents (interns) and 24 hours for upper-year residents (with an additional 4 hours to perform transitions of care and participate in educational activities).¹ In parallel, new standards for enhanced faculty supervision were also introduced to further efforts to improve patient safety.

Initial duty hour reforms in 2003 were prompted by widespread concern about the effects of resident fatigue, including deaths due to medical errors.³ However, previous research evaluating the 2003 duty hour reforms has found no significant associations of the reforms with changes in mortality,^4-7 hospital readmission rates,⁸ indicators of patient safety,⁹ or probability of prolonged length of stay¹⁰ in the first few years after reforms.

Quiz Ref IDProgram directors and residents remain concerned that the 2011 duty hour reforms may adversely affect the quality of resident education, increase handoffs in care, and put both patient safety and outcomes at risk.^11-13 One year before the reforms went into effect, 94% of residency programs reported exceeding the 16-hour limit for first-year residents,¹² indicating that teaching hospitals in the United States would need to undergo a major restructuring to become compliant. Furthermore, such changes are associated with significant cost burdens without additional funding available.^14,15 Given the significance of duty hour reforms, it is of vital importance that the immediate associations of these changes be evaluated. In this study, our objective was to evaluate the association of the 2011 ACGME duty hour reforms with mortality and readmissions among hospitalized Medicare patients during the first year after the reforms.

This study was approved by the institutional review boards at the University of Pennsylvania and The Children’s Hospital of Philadelphia. A waiver of consent was approved by the institutional review boards because this study, which analyzes millions of patients, poses minimal risk to confidentiality and it would not have been feasible to obtain informed consent from each individual.

The main outcome measure was all-location mortality within 30 days of hospital admission for patients admitted with a principal medical diagnosis of acute myocardial infarction, stroke, gastrointestinal bleeding, or congestive heart failure or a Diagnosis Related Group (DRG) classification of general, orthopedic, or vascular surgery. We assessed all-location mortality (both in-hospital and postdischarge deaths) within 30 days of admission because it is a measure that eliminates bias due to length-of-stay differences between hospitals or over time.¹⁶ This measure is consistent with previous research^4-7 and supported by the Centers for Medicare & Medicaid Services (CMS).¹⁷

A secondary outcome measure was all-cause readmission within 30 days of hospital discharge for all patients discharged alive after admission with a principal medical diagnosis of acute myocardial infarction, stroke, gastrointestinal bleeding, or congestive heart failure or a DRG classification of general, orthopedic, or vascular surgery. This measure is consistent with previous research⁸ and supported by the CMS.¹⁷

Patients were selected using an approach described in previous studies^4-8 and included all Medicare patients admitted to short-term, acute care, general US nonfederal hospitals from July 1, 2009, to June 30, 2012. Admissions were excluded from analysis if any of the following criteria were met: (1) a hospital was non–acute care, not in a US state or the District of Columbia, or opened or closed during the study period; (2) a hospital had fewer than 350 Medicare admissions in any year (a mean of less than 1 admission per day across all conditions), to eliminate those too small to yield stable estimates in a fixed-effects analysis; (3) Medicare Cost Report data or resident-to-bed (RB) ratio were missing; (4) a patient was enrolled in a health maintenance organization at any point during the study period, because these admissions did not have complete claims data for our analysis; (5) a hospital was missing more than 2 months of data in the prereform period or 1 month of data in the postreform period; (6) a patient’s hospitalizations spanned July 1, 2011 (the timing of the implementation of new duty hour reforms); (7) a patient had a reported date of death before an admission date; (8) a patient was younger than 66 years (to allow a 180-day look back for risk adjustment) or older than 90 years (because changes in the proportion of such patients treated aggressively may not be well reflected in administrative data); (9) a patient admitted for acute myocardial infarction was discharged alive in fewer than 2 days (because such cases may not represent actual acute myocardial infarctions)^18,19; or (10) a patient was transferred from one hospital to another for a qualifying condition or surgical procedure (in which case the entire episode of care was evaluated as a single admission rather than 2 admissions). Additional details are available in eTable 1 in the Supplement.

To evaluate 30-day all-cause readmissions, each admission was considered an index admission if there were no admissions for that patient in the previous 30 days. A 30-day look back was instituted to reduce the likelihood that readmissions were selected as index admissions, which otherwise might have skewed readmission rates.⁸ When there was a transfer of care during an index admission from one hospital to another, the admission date from the first hospitalization was used along with the discharge date from the second hospitalization. This allowed us to capture the entire episode of care and removed the possibility that the second hospitalization could be counted as a readmission for the index admission at the first hospitalization.

The risk adjustment approach was identical to previous studies,^4-10 a modification of methods of Elixhauser et al.²⁰ The Elixhauser method used 27 comorbidities excluding fluid and electrolyte disorders and coagulopathy, which should not be used in quality indicator risk adjustment,^21,22 and has been shown to achieve better discrimination than other approaches.^23,24 This was augmented with adjustments for age, sex, and race/ethnicity. For surgical categories, we also adjusted for DRGs, grouping DRGs with and without complications or comorbidities into 1 variable. We performed a 180-day look back, including data from previous hospitalizations, to obtain more comprehensive information on comorbidities than available by using the index admission alone.²⁵

Data on patient characteristics were obtained from the Medicare Provider Analysis and Treatment File (MEDPAR), which included information on principal and secondary diagnoses, age, sex, comorbidities, and discharge status, including dates of death.²⁶ Data on health maintenance organization enrollment were obtained from the CMS Denominator files. Data on self-reported race/ethnicity was obtained from the CMS Master Beneficiary File and coded by the study team as white, black, Hispanic, and other (included other, Asian, and North American Native). The number of residents at each hospital was obtained from CMS Medicare Cost Reports.

The primary measure of teaching intensity was the RB ratio, calculated at a defined point in time as the number of interns plus residents divided by the mean number of staffed beds. The RB ratio has been used to differentiate hospitals by teaching intensity in previous studies^27-29 and in the approach used in previous research.^4-10 Teaching hospitals were defined as hospitals with RB ratios greater than 0 as follows: very minor (RB ratio, >0 to <0.05), minor (RB ratio, 0.05 to <0.25), major (RB ratio, 0.25 to <0.60), and very major (RB ratio, ≥0.60).

Quiz Ref IDWe used a multiple time series research design,^4-10,30 also known as difference in differences, and examined whether the implementation of duty hour reforms was associated with a change in the underlying trend in patient outcomes for more intensive relative to less intensive teaching hospitals, an approach that reduced potential biases from unmeasured variables.^31,32 The multiple times series research design compared each hospital with itself, contrasting the changes in more intensive teaching hospitals to the changes in less intensive teaching hospitals, and adjusted for observed differences in patient risk factors. The design also adjusted for changes in outcomes over time (trends) common to all hospitals and minimized bias from 3 possible sources. First, a difference between hospitals that is stable over time cannot be mistaken for an effect of the reform because hospital fixed effects are used to compare each hospital with itself before and after reform. Second, universal changes affecting all hospitals similarly, such as technological improvements or pay-for-performance initiatives, cannot be mistaken for an effect of the reform because the logit model includes year indicators. Third, if the mix of patients is changing differently among hospitals, and if these changes are accurately reflected in measured risk factors, this cannot be mistaken for an effect of the reform because the logit model adjusts for these measured risk factors. Although the difference-in-differences model offers these advantages, it has limitations. Any diverging trend in mortality or readmission rates over time for more intensive relative to less intensive teaching hospitals already in progress or coincident with the initiation of the reform could be mistaken for an effect of the reform, although we adjusted for any observed differences in prereform trends.

Logistic regression models were fit with patient outcomes data from July 1, 2009, to June 30, 2012, using the RB ratio as a continuous variable to measure hospital teaching intensity. The dependent variables in the models were all-location mortality within 30 days of hospital admission and all-cause readmission within 30 days of hospital discharge. We report the C statistic, which represents model fit.³³ Similar to previous research,^4-10 30-day all-location mortality and 30-day all-cause readmissions were assumed to have a common time trend until implementation of the duty hour reforms, after which the trends were allowed to diverge. To assess whether underlying trends in 30-day all-location mortality and 30-day all-cause readmissions were similar among more intensive relative to less intensive teaching hospitals during the 2 years before duty hour reform, a test of controls was performed.³⁴ Parameters were added to the model for interactions between the RB ratio and indicators for prereform year 2 and prereform year 1. A Wald test was used to determine whether these interactions significantly improved model fit,³⁵ which would suggest that more intensive teaching hospitals relative to less intensive teaching hospitals had differing trends in the 2 years before reform independent of the reform. Similar to previous research,^4-10 if a test of controls was significant for differing prereform trends, then instead of using both prereform years as a control, only prereform year 1 (July 1, 2010, to June 30, 2011) was used as a control in the main model.

The association of duty hour reforms with patient outcomes was estimated using the coefficients of RB ratio interacted with a dummy variable representing postreform year 1 (July 1, 2011, to June 30, 2012). These coefficients, presented as odds ratios (ORs), measure the degree to which 30-day all-location mortality and 30-day all-cause readmissions differed over time in more intensive relative to less intensive teaching hospitals. Medical conditions were assessed both individually and together as combined medical conditions. Surgical categories were also assessed individually and as combined surgical categories. All hypothesis tests were 2-sided and used a significance level of P<.05. All statistical analyses were conducted using SAS version 9.3 (SAS Institute Inc).

We performed a series of independent sensitivity analyses to further evaluate model estimates. First, we excluded patients from the state of New York, where duty hour reforms have historically been implemented differently from other states. Second, we excluded patients transferred from nursing homes, whose care may be less aggressive. Third, we used the bootstrap procedure to estimate confidence intervals and P values that account for the correlation between patients with multiple observations.^7,36-38 Fourth, we estimated outcomes for 30-day in-hospital mortality. Fifth, we estimated all-cause 7-day readmissions, which may more directly reflect associations with inpatient care.³⁹

The study sample comprised 2 790 356 patients with 6 384 273 admissions from 3104 hospitals. The number and proportion of hospitals and associated admissions by teaching intensity were as follows: 96 very major teaching hospitals (3.1% of all hospitals in the sample) comprising 353 040 admissions (5.6% of all admissions in the sample), 138 major teaching hospitals (4.4%) comprising 513 704 admissions (8.1%), 442 minor teaching hospitals (14.2%) comprising 1 330 993 admissions (20.9%), 443 very minor teaching hospitals (14.3%) comprising 784 125 admissions (12.3%), and 1985 nonteaching hospitals (64.0%) comprising 3 402 411 admissions (53.3%). Table 1 and Table 2 show patient characteristics and comorbidities for the sample during the entire study period. Sample patient characteristics by study year are available in eTables 2 through 4 in the Supplement.

In the 2 years before duty hour reforms, there were 4 325 854 admissions with 288 422 deaths and 602 380 readmissions. In the first year after the reforms, there were 2 058 419 admissions with 133 547 deaths and 272 938 readmissions. Unadjusted trends in 30-day all-location mortality and 30-day all-cause readmissions declined over time for most medical conditions and surgical categories (Table 3). Unadjusted trends for both outcomes also declined when evaluated by hospital teaching intensity (Table 4).

Between prereform year 2 and prereform year 1, a test of controls did not find diverging trends for 30-day all-location mortality for any medical conditions or surgical categories. For 30-day all-cause readmissions, a test of controls found diverging trends only for the surgical categories of orthopedic surgery and vascular surgery (estimates available in eTables 5 and 6 in the Supplement), so for these categories, prereform year 1 was used as the reference group in adjusted analyses.

In adjusted analyses of 30-day all-location mortality, there were no significant differences over time in more intensive relative to less intensive teaching hospitals for combined medical conditions (OR, 1.00; 95% CI, 0.96-1.03; P = .75), combined surgical categories (OR, 0.99; 95% CI, 0.94-1.04; P = .64), or any of the individual medical conditions or surgical categories (Table 5). In adjusted analyses of 30-day all-cause readmissions, there were no significant differences over time in more intensive relative to less intensive teaching hospitals for combined medical conditions (OR, 1.00; 95% CI, 0.97-1.02; P = .71) or combined surgical categories (OR, 1.00; 95% CI, 0.98-1.03; P = .88) (Table 6). There were higher odds of 30-day all-cause readmissions for the medical condition of stroke (OR, 1.06; 95% CI, 1.001-1.13; P = .047) but not for any other individual medical condition or surgical category. The C statistics for the 30-day all-location mortality models ranged from 0.68 to 0.86, while the C statistics for the 30-day all-cause readmission models ranged from 0.59 to 0.68. This indicates that the available covariates were reasonably strong predictors of mortality but less strong predictors of readmission.

Sensitivity analysis for combined medical conditions and combined surgical categories did not differ from the main models for 30-day all-location mortality or 30-day all-location readmissions (eTables 7-14 in the Supplement). Thirty-day all-cause readmission estimates had significantly higher odds for stroke when excluding the state of New York but were not significant when excluding patients transferred to nursing homes, when using the bootstrap procedure, or for 7-day all-cause readmission estimates.

Duty hour reform is arguably one of the largest efforts ever undertaken to improve the quality and safety of patient care in teaching hospitals. To our knowledge, this is one of the first national evaluations of the association of the 2011 ACGME duty hour reforms with patient outcomes. Quiz Ref IDIn the first year after the reforms, we found no significant positive or negative associations of duty hour reforms with 30-day all-location mortality for any of the medical conditions or surgical categories in this study. We also found no significant positive or negative associations of duty hour reforms with 30-day all-cause readmissions for combined medical conditions or combined surgical categories. We did find that patients admitted with stroke had higher odds of 30-day all-cause readmission after reform for more intensive relative to less intensive teaching hospitals. However, this finding was not found to be significant in several of the sensitivity analyses.

There are several potential reasons why duty hour reforms were not found to be significantly associated with changes in patient outcomes, as measured by 30-day all-location mortality and 30-day all-cause readmissions. First, although the 2011 duty hour reforms were a major change for US teaching hospitals, they may not have been larger in magnitude than the 2003 duty hour reforms, in which there has also been little evidence of associations of reforms with mortality or readmissions.^4-8 Second, although 94% of residency programs reported exceeding 2011 duty hour reform work limits in the year before implementation, little is known about adherence to the new reforms. Some programs may not have yet adapted to the new reforms, thereby decreasing their potential association with patient outcomes.

Quiz Ref IDThird, hospitals that did adopt the new reforms may have leveraged faculty or hospitalists with greater experience than residents to care for Medicare patients. Fourth, the 2011 reforms focused on improving faculty supervision that could have resulted in improved patient outcomes, potentially compensating for any adverse effects of increased patient handoffs. Fifth, although our study focused on mortality and readmissions, measurements of other outcomes such as patient safety indicators or complications may better elucidate the relative effects of decreased resident fatigue and increased patient handoffs.

Sixth, unadjusted trends for mortality and readmissions were declining throughout the study period. This trend is consistent with other analyses of outcomes for Medicare beneficiaries evaluating mortality⁴⁰ and readmissions^40,41 during a similar period. This may reflect other concurrent policy initiatives by the CMS targeted toward Medicare beneficiaries, such as Partnership for Patients⁴² or the Hospital Readmissions Reduction Program.⁴³ If these initiatives led to general improvements in outcomes across all hospitals, duty hour reforms may be less likely to be directly associated with changes in patient outcomes at more intensive relative to less intensive teaching hospitals.

Despite these potential considerations, our findings suggest that in the first year after the 2011 duty hour reforms, the goals of improving the quality and safety of patient care,¹ as measured by decreased 30-day all-location mortality and 30-day all-cause readmissions, were not being achieved. Conversely, concerns^11-13,44-46 that outcomes might actually worsen because of decreased continuity of care have not been borne out.

This study is subject to several limitations. First, we evaluated the outcomes of 30-day all-location mortality and 30-day all-cause readmissions. Although the duty hour standards were an attempt to improve the quality and safety of patient care,¹ measurements of other outcomes such as patient safety indicators or complications may better elucidate the relative effects of decreased continuity of care compared with decreased resident fatigue. Nonetheless, we chose to report 2 outcomes that are most accurately observed with claims data to minimize the likelihood of underreporting or overreporting. Second, any observational study is susceptible to unmeasured confounding. We used administrative data, so risk adjustment is more limited than with clinical data; however, by comparing outcomes over time within each hospital in more intensive relative to less intensive teaching hospitals, potential bias from unmeasured confounders is diminished. Third, survival bias associated with admissions resulting in death being differentially more likely after reform than before reform in more intensive relative to less intensive teaching hospitals could still have affected the study results. Fourth, the study population was limited to Medicare beneficiaries and, therefore, our sample may not be representative of the general US population. Fifth, we are unable to disentangle other major policy initiatives directed at this population concurrent with implementation of duty hour reforms. Sixth, this study evaluated outcomes in only the first year after the reforms, and these findings should be considered to be representative only of this time period. Further analysis of changes in patient outcomes several years after the reforms will be important to evaluate changes in associations over time.

Among Medicare beneficiaries, there were no significant differences in changes in 30-day mortality rates or 30-day all-cause readmission rates for those hospitalized in more intensive relative to less intensive teaching hospitals in the year after implementation of the 2011 ACGME duty hour reforms compared with those hospitalized in the 2 years before implementation.

Corresponding Author: Mitesh S. Patel, MD, MBA, MS, University of Pennsylvania, 423 Guardian Dr, 12th Floor, Blockley Hall, Philadelphia, PA 19104 (mpatel@upenn.edu).

Author Contributions: Dr Patel had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Patel, Volpp, Small, Rosenbaum, Bellini, Silber.

Acquisition, analysis, or interpretation of data: Patel, Volpp, Small, Hill, Even-Shoshan, Ross, Zhu, Silber.

Drafting of the manuscript: Patel, Rosenbaum, Bellini, Silber.

Critical revision of the manuscript for important intellectual content: Patel, Volpp, Small, Hill, Even-Shoshan, Ross, Bellini, Zhu, Silber.

Statistical analysis: Patel, Small, Hill, Ross, Bellini, Zhu, Silber.

Obtained funding: Volpp, Even-Shoshan, Bellini, Silber.

Administrative, technical, or material support: Patel, Volpp, Even-Shoshan, Bellini, Silber.

Study supervision: Patel, Volpp, Even-Shoshan, Bellini, Silber.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

Funding/Support: This project was funded in part through National Heart, Lung, and Blood Institute grant R01-HL094593. Dr Patel was supported by the Department of Veteran Affairs and the Robert Wood Johnson Foundation.

Role of the Funder/Sponsor: The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.