The flowchart highlights discrepancies between administrative and clinical diagnosis, planned readmissions captured by the readmission measure, and prevalence of unrelated readmissions.
eAppendix 1. Procedure categories considered planned.
eAppendix 2. List of readmission diagnoses that force readmission to be classified as unplanned, regardless of operation performed.
Sacks GD, Dawes AJ, Russell MM, Lin AY, Maggard-Gibbons M, Winograd D, Chung HR, Tomlinson J, Tillou A, Shew SB, Hiyama DT, Cryer HG, Brunicardi FC, Hiatt JR, Ko C. Evaluation of Hospital Readmissions in Surgical PatientsDo Administrative Data Tell the Real Story?. JAMA Surg. 2014;149(8):759-764. doi:10.1001/jamasurg.2014.18
The Centers for Medicare & Medicaid Services has developed an all-cause readmission measure that uses administrative data to measure readmission rates and financially penalize hospitals with higher-than-expected readmission rates.
To examine the accuracy of administrative codes in determining the cause of readmission as determined by medical record review, to evaluate the readmission measure’s ability to accurately identify a readmission as planned, and to document the frequency of readmissions for reasons clinically unrelated to the original hospital stay.
Design, Setting, and Participants
Retrospective review of all consecutive patients discharged from general surgery services at a tertiary care, university-affiliated teaching hospital during 8 consecutive quarters (quarter 4 [October through December] of 2009 through quarter 3 [July through September] of 2011). Clinical readmission diagnosis determined from direct medical record review was compared with the administrative diagnosis recorded in a claims database. The number of planned hospital readmissions defined by the readmission measure was compared with the number identified using clinical data. Readmissions unrelated to the original hospital stay were identified using clinical data.
Main Outcomes and Measures
Discordance rate between administrative and clinical diagnoses for all hospital readmissions, discrepancy between planned readmissions defined by the readmission measure and identified by clinical medical record review, and fraction of hospital readmissions unrelated to the original hospital stay.
Of the 315 hospital readmissions, the readmission diagnosis listed in the administrative claims data differed from the clinical diagnosis in 97 readmissions (30.8%). The readmission measure identified 15 readmissions (4.8%) as planned, whereas clinical data identified 43 readmissions (13.7%) as planned. Unrelated readmissions comprised 70 of the 258 unplanned readmissions (27.1%).
Conclusions and Relevance
Administrative billing data, as used by the readmission measure, do not reliably describe the reason for readmission. The readmission measure accounts for less than half of the planned readmissions and does not account for the nearly one-third of readmissions unrelated to the original hospital stay. Implementation of this readmission measure may result in unwarranted financial penalties for hospitals.
Reducing hospital readmissions after discharge is a pivotal component of the quality improvement and cost-containment measures in the Patient Protection and Affordable Care Act.1 Readmission rates for certain medical conditions are now publically reportable outcomes, and readmissions after certain surgical procedures are soon to follow suit.2,3 In anticipation, the Centers for Medicare & Medicaid Services (CMS) sponsored the development of a standardized measure to evaluate hospital readmissions for any cause.4Quiz Ref IDThe Hospital-Wide All-Cause Unplanned Readmission Measure, which is yet to be implemented, uses Medicare claims data to calculate a hospital-level, risk-standardized rate of all unplanned readmissions within 30 days of discharge. This rate will be used to compare hospital performance and penalize those hospitals with higher-than-expected readmission rates.
Several studies5- 9 have suggested that administrative claims data, as used by the readmission measure, correlate poorly with clinical data derived directly from medical records. Clinical data, although more expensive and time-consuming to collect, may offer a more accurate estimate of outcomes, including accurate clinical information necessary for risk adjustment. The reliability of administrative data to accurately describe hospital readmissions is yet to be determined.
Quiz Ref IDThis study seeks to evaluate the readmission measure in 3 ways. First, because the readmission measure relies exclusively on Medicare claims data, we sought to determine the accuracy of each readmission diagnosis in administrative claims data compared with the clinical diagnosis derived from the actual medical records. Second, we tested the readmission measure’s ability to appropriately exclude planned readmissions compared with those identified through medical record review. Finally, we sought to determine the frequency of readmissions that were clinically unrelated to the original hospitalization.
The study design was reviewed and approved by the David Geffen School of Medicine at UCLA Institutional Review Board. We retrospectively queried the University HealthSystem Consortium (UHC) database for all readmissions within 30 days after discharge from general surgery services at a single academic medical center. We selected general surgery patients because hospital readmissions in this cohort have not been thoroughly studied, and readmissions for these patients will soon be a publically reportable outcome. The UHC database allowed us to analyze an administrative data source, similar to that analyzed by Medicare. All readmissions during 8 consecutive quarters, from quarter 4 (October through December) of 2009 through quarter 3 (July through September) of 2011, were analyzed. We included patients who were on a surgical service but did not receive an operation on their index hospitalization (eg, conservative management of small-bowel obstruction and nonoperative trauma). Readmissions for chemotherapy, radiation therapy, rehabilitation, dialysis, labor and delivery, or psychiatric reasons were excluded from our analysis.
For each readmission, 2 physicians (G.D.S. and A.J.D.) independently reviewed the patient’s medical record. A third physician (M.M.-G.) reviewer resolved any discrepancy between reviewers (47 cases [15%]). We collected data on patient demographic characteristics, including age, sex, race (as identified by the UHC database), operation performed during hospitalization, and length of stay. We also recorded the number of days between hospital discharge and readmission and the hospital length of stay during the readmission and calculated median values with interquartile ranges (IQRs). We used χ2 tests to compare categorical variables and Wilcoxon–Mann-Whitney tests to compare continuous variables.
First, we performed a medical record review to assess the accuracy of administrative codes in determining the cause of readmission. A primary diagnosis for the readmission as recorded in the UHC database (administrative diagnosis recorded by International Classification of Diseases, Ninth Revision [ICD-9] code) was compared with the diagnosis determined by medical record review (clinical diagnosis). We assigned each discrepancy between administrative and clinical diagnosis into 1 of 3 categories: (1) secondary diagnosis that was relevant to the readmission but not the primary reason for readmission, (2) previous diagnosis given to the patient before the readmission, and (3) inaccurate code that did not match any records in the patient’s medical record. These categories were further subdivided to reflect the clinical diagnosis category. We then identified and excluded all coding errors from further analysis. Coding errors occurred when a readmission was identified by the UHC database, but clinical medical record review revealed that no readmission had taken place.
Second, we applied the readmission measure to determine whether each readmission satisfied one of the measure’s predefined criteria as a planned readmission. According to the readmission measure, a planned readmission is one in which a patient undergoes 1 of 36 predefined planned procedures (eAppendix 1 [Supplement]) and is admitted for an elective procedure. The measure considers these planned readmissions as an essential component of patient care and are not reflective of poor quality. Readmissions for emergency procedures (eAppendix 2 [Supplement]) are characterized as unplanned, even if 1 of the 36 procedures is performed. For example, a patient readmitted within 30 days to undergo an elective cholecystectomy would be labeled a planned readmission. The readmission would be unplanned, however, if the patient were readmitted with a diagnosis of biliary sepsis and underwent a cholecystectomy during the hospitalization. Using the clinical data abstracted from the medical record, we determined whether each readmission was planned. We identified a readmission as planned if the treatment received during the hospital stay was a necessary part of the treatment plan, was scheduled ahead of time, and did not involve an emergency procedure. The McNemar test was used to compare planned readmissions identified by the readmissions measure and those identified by medical record review.
Third, we determined the frequency of unplanned readmissions that were clinically unrelated to the original hospitalization. Our definition of unrelated readmission was developed by consensus among the authors. A readmission was labeled as unrelated if it met the following criteria: (1) the indication for readmission was neither addressed nor caused during the original hospital stay, (2) the readmission diagnosis was not related to the underlying surgical disease, and (3) the readmission diagnosis was not a recognized surgical complication. Planned and unrelated readmissions were measured as a fraction of total readmissions. Unrelated readmissions were divided into 3 categories: (1) acute illness that was unrelated to the original hospital stay, (2) treatment of a chronic medical condition, and (3) social reasons. All statistical analyses were performed using Stata/IC statistical software, version 11.0 (StataCorp).
Quiz Ref IDDuring the study period, 3788 patients were discharged from general surgical services. Of these, 315 (8.3%) were readmitted within 30 days (Figure). Of the 315 readmissions, 14 (4.4%) were excluded for coding errors, and 43 (13.7%) were determined to be planned readmissions. Of the remaining 258 readmissions, 188 (72.9%) were related to the original readmission, whereas 70 (27.1%) were unrelated. Most readmitted patients were male (160 [50.8%]) and white (212 [67.3%]); the mean age was 55 years, with a range of 15 to 88 years (Table 1). Operations performed in the original admission included elective general surgery (gastrointestinal or abdominal wall) operations (69 [21.9%]), elective colorectal operations (48 [15.2%]), elective noncolonic oncologic operations (42 [13.3%]), emergency nontrauma operations (37 [11.7%]), trauma operations (17 [5.4%]), and endocrine operations (6 [1.9%]). Almost one-third of patients (96 [30.5%]) did not undergo an operation during their original hospital stay. Length of stay during the original hospitalization was longer for readmitted patients (6 days; IQR, 3-10 days) than for patients who were not readmitted (3 days; IQR, 2-6 days; P < .001).
Quiz Ref IDThe readmission diagnosis listed in the administrative claims data differed from the clinical diagnosis in 97 readmissions (30.8%). For these discrepancies, the administrative claims data most often recorded a secondary acute condition diagnosis (56 [57.7%]). Administrative data recorded a previous diagnosis in 12 conflicting cases (12.4%) and an inaccurate code in 29 conflicting cases (29.9%). In all 3 groups, gastrointestinal was the diagnosis category that most commonly conflicted with the clinical diagnosis (Table 2).
According to the criteria outlined by the readmission measure, there were 15 planned readmissions (4.8%). On review of the clinical data, 43 (13.7%) were planned (P < .001), including all 15 identified by the readmission measure. Therefore, the readmission measure accounted for approximately one-third (15 of 43 [34.9%]) of the total planned readmissions.
Quiz Ref IDOf the remaining 258 unplanned readmissions, 70 (27.1%) were for reasons unrelated to the original hospital stay. A typical example of an unrelated readmission is a 52-year-old man who underwent an elective laparoscopic cholecystectomy and was then readmitted 21 days after discharge with a kidney stone. An example of a related readmission is a 73-year-old man who underwent a Whipple operation for pancreatic adenocarcinoma and was readmitted 20 days after discharge with a pancreatic fistula. Unrelated readmissions were due to an acute condition unrelated to the original hospital stay (28 [40.0%]), chronic disease management (39 [55.7%]), and social factors (3 [4.3%]) (Table 3).
Reducing hospital readmissions is a major target for current quality improvement and cost-containment efforts. National policy makers have developed a tool that uses administrative data to analyze all-cause hospital readmissions and determine whether each readmission is susceptible to financial penalties.4 This tool is yet to be implemented, however.
Using a cohort of general surgery patients, the current study demonstrates that administrative data failed to correctly identify the reason for hospital readmission in nearly one-third of cases. Similarly, the readmission measure failed to identify nearly two-thirds of planned readmissions. Clinical data also demonstrated that more than one-quarter of unplanned readmissions were for reasons unrelated to the original hospital stay. These findings identify the limitations of using administrative claims data to evaluate hospital performance for readmissions.
Several studies5- 11 comparing administrative and clinical data for measuring health outcomes have found marked discrepancies similar to those found in our study. These studies have focused on the accuracy of administrative data to identify preoperative risk factors and postoperative complications, including mortality.5,10 Lawson et al7 demonstrated a range of 0.27 to 0.78 for the sensitivity of Medicare claims data to detect complications recorded in the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP). Similarly, Koch et al8 compared ACS-NSQIP data with administrative data from the Agency for Healthcare Research and Quality (AHRQ) and found poor concordance for the diagnosis of postoperative hemorrhage, sepsis, deep vein thrombosis, and respiratory failure. To our knowledge, our study is the first of its kind to compare the relative accuracy of administrative and clinical data sources for evaluating hospital readmissions. Most notably, we found that in one-third of discrepant diagnoses between UHC administrative and clinical data from medical record review, there was no clinical evidence in the patient medical records to support the primary diagnosis coded by the administrative claims data. Cima et al6 identified similar discrepancies in a comparison of ACS-NSQIP and AHRQ data.
Determining whether a readmission represents a planned part of a patient’s care or an unexpected complication can be challenging. Review of the literature presents a diverse approach to making this determination. In the comprehensive study on hospital readmissions, Jencks et al12 examined the diagnosis-related group for each readmission and identified 10% of all readmissions as planned. In contrast, van Walraven et al13 identified a readmission as planned if it was arranged at the time of discharge from the original hospitalization. Yermilov et al14 described planned readmissions for patients undergoing adjuvant cancer therapy, rehabilitation, or both. Planned readmissions, in our methodology, refer to elective readmissions that represent a necessary part of comprehensive treatment, usually scheduled in advance. The readmission measure, on the other hand, uses common procedure codes and nonurgent ICD-9 codes to identify planned readmissions. According to the measure, these planned readmissions represent an essential component of patient care and are not reflective of poor quality. Our findings suggest that this method, which relies on administrative claims data, fails to recognize nearly two-thirds of planned readmissions.
The literature generally agrees that planned readmissions do not reflect poor-quality care and therefore should be excluded from analysis.12- 18 This belief is clearly reflected by the methods outlined in the readmission measure.4 There is, however, no consensus on how to account for those readmissions that are unrelated to the original hospital stay. Few studies have attempted to quantify the frequency of these readmissions. A study by Dharmarajan et al2 found that the proportion of patients readmitted for the same condition treated during the original hospitalization ranged from 10% to 35%. Other studies analyzing readmissions after Whipple operations19 and spine surgery20 found the rates of unrelated readmissions to be 20% and 5.7%, respectively. In our study, nearly one-third (70 [27.1%]) of readmissions were for reasons unrelated to the original hospital stay. The relatively high frequency of unrelated readmissions brings into question the extent to which hospitals should be held accountable for patient well-being after discharge. Even if hospitals maximize the quality of care during an inpatient stay and the subsequent postdischarge period, unrelated health issues may arise for which the hospital would be penalized. Indeed, the readmission measure concedes that there is no reliable way to determine whether a readmission is related to the original hospitalization based on the documented admission diagnosis in the administrative data. Our study, on the other hand, suggests that when clinical data are taken into consideration, it may be possible to determine whether a readmission is in fact related to the original hospital stay. To better inform policy on this issue, further research using clinical data is needed to characterize unrelated hospital readmissions in a larger population.
Our study has several limitations. Our cohort of readmitted patients and our overall readmission rate were obtained from the UHC database, which, on a number of occasions, inaccurately identified a readmission when none had occurred. Conversely, it is possible that not all readmissions were accurately captured by the database. We also may have underreported some readmissions because we were unable to identify readmissions to another hospital, which may occur in 19%21 to 47%14 of cases. This finding may also account for our relatively low overall readmission rate. Because it is unlikely that coding of administrative data would differ for patients readmitted to other hospitals, this limitation is unlikely to bias our findings. This study is a single-institution study; therefore, the generalizability of our findings is not known. We did, however, examine readmissions for a wide range of general surgery specialties, procedures, and surgeons. We also included all surgical patients undergoing elective, urgent, and emergency procedures, as well as conditions that were managed nonoperatively. Analyzing data from another center may reveal even higher rates of unrelated readmissions. Patients may be more likely to return to a tertiary referral center for problems that relate to surgery and to present to another hospital, possibly closer to their home, for unrelated medical problems. Finally, our analysis of unrelated readmissions relies on a subjective interpretation of the patient medical record. To address this limitation, we outlined explicitly defined criteria that an unrelated readmission must meet. We also ensured that there was agreement between the 2 physician reviewers.
Our study demonstrates significant limitations of the Hospital-Wide All-Cause Unplanned Readmission Measure developed by the CMS. The use of administrative claims data to describe and evaluate hospital readmissions is often inaccurate and fails to identify the true number of planned readmissions. In addition, the high frequency of readmissions that are clinically unrelated to the original hospital stay raises the question of whether these should be excluded from financial penalties implemented by policy makers. Although decreasing readmissions is an important target for quality improvement and cost containment, further work is needed in this area before hospital readmissions can be used as a quality measure. Addressing the issue of risk adjustment to fairly and accurately compare hospitals based on their readmission rates is a potential next step.
Accepted for Publication: November 18, 2013.
Corresponding Author: Greg D. Sacks, MD, MPH, David Geffen School of Medicine at UCLA, Department of Surgery, 10833 Le Conte Avenue, 72-215 CHS, Los Angeles, CA 90095 (email@example.com).
Published Online: June 11, 2014. doi:10.1001/jamasurg.2014.18.
Author Contributions: Dr Sacks had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Sacks, Dawes, Russell, Lin, Maggard-Gibbons, Winograd, Tomlinson, Tillou, Shew, Hiyama, Cryer, Ko.
Acquisition, analysis, or interpretation of data: Sacks, Dawes, Russell, Maggard-Gibbons, Winograd, Chung, Tomlinson, Tillou, Hiyama, Cryer, Brunicardi, Hiatt.
Drafting of the manuscript: Sacks, Dawes, Cryer.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Sacks, Dawes.
Administrative, technical, or material support: Winograd, Chung, Cryer.
Study supervision: Russell, Lin, Maggard-Gibbons, Tomlinson, Tillou, Shew, Hiyama, Brunicardi, Ko.
Conflict of Interest Disclosures: None reported.
Previous Presentation: This study was presented as a podium presentation in the Quality Outcomes and Costs I session at the American College of Surgeons Clinical Congress Meeting; October 2, 2012; Chicago, Illinois.