[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.147.238.168. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure.
True vs NSQIP 30-Day Readmission Rates
True vs NSQIP 30-Day Readmission Rates

The National Surgical Quality Improvement Program (NSQIP) method (30-day postoperative readmission) undercounts true 30-day postdischarge readmission. This effect worsens with longer lengths of stay and is statistically significant at 25 days or longer (P = .012, determined by use of the Fisher exact test).

1.
Jencks  SF, Williams  MV, Coleman  EA.  Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428.
PubMedArticle
2.
American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). User guide for the 2011 Participant Use Data File. http://site.acsnsqip.org/wp-content/uploads/2012/03/2011-User-Guide_Final.pdf. Published October 2012. Accessed May 28, 2013.
3.
Lucas  DJ, Haider  A, Haut  E,  et al.  Assessing readmission after general, vascular, and thoracic surgery using ACS-NSQIP. Ann Surg. 2013;258(3):430-439.
PubMedArticle
4.
Suissa  S.  Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492-499.
PubMedArticle
5.
Hechenbleikner  EM, Makary  MA, Samarov  DV,  et al.  Hospital readmission by method of data collection. J Am Coll Surg. 2013;216(6):1150-1158.
PubMedArticle
6.
Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53(282):457-481. doi:10.1080/01621459.1958.10501452.Article
Research Letter
August 2014

Avoiding Immortal Time Bias in the American College of Surgeons National Surgical Quality Improvement Program Readmission Measure

Author Affiliations
  • 1Department of Surgery, Walter Reed National Military Medical Center, Bethesda, Maryland
  • 2Department of Surgery, Johns Hopkins Hospital, Baltimore, Maryland
  • 3Department of Surgery, Georgetown University Hospital, Washington, DC
JAMA Surg. 2014;149(8):875-877. doi:10.1001/jamasurg.2014.115

Readmission has become a key quality metric because it is a frequent and costly adverse event for patients.1 Medicare penalizes hospitals if they have excess numbers of readmissions for certain diagnoses, including some within surgery. The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) began tracking readmission rates in 2011.2

While conducting research on readmission after surgery, we noted a problem with the NSQIP’s definition of readmission.3 The NSQIP only counts readmissions during the 30 days following surgery, consistent with the interval they use for all postoperative outcomes. However, the standard period for readmission used by Medicare and others is 30 days after hospital discharge. This discrepancy creates an immortal time bias—patients cannot be readmitted before hospital discharge and are therefore “immortal” for this outcome until they leave the hospital.4 Including immortal time when calculating readmission rates leads to a bias between the measured value and the true value for readmission rates. Logically, 30-day postoperative readmission should systematically undercount 30-day postdischarge readmission. In addition, this bias should worsen for a longer length of stay. In our previous NSQIP analysis, we noted that readmission rates increased with longer lengths of stay, peaked at 13 days, and decreased thereafter. We hypothesized that readmission rates would increase continuously with longer lengths of stay if this immortal time bias were not present. Here we report data supporting this hypothesis and demonstrate how to avoid this bias.

Methods

We examined patients who had a colorectal resection at Johns Hopkins Hospital in Baltimore, Maryland, between 2009 and 2011, survived to hospital discharge, and were captured by the NSQIP. Dates of admission, discharge, and readmission were obtained from the NSQIP, medical record review, and administrative data, as previously described.5 The NSQIP 30-day postoperative readmission rate was compared with the standard 30-day postdischarge readmission rates obtained from medical record review. A Kaplan-Meier method to estimate the true 30-day postdischarge readmission rate using only data collected by the NSQIP was then evaluated.6 Our study was approved by the Johns Hopkins Hospital institutional review board. Consent was waived owing to the retrospective nature of the analysis. Patients were not compensated.

Results

We identified 708 patients, whose characteristics were described previously.6 Of these 708 patients, 134 (18.9%) were readmitted within 30 days after hospital discharge, but only 114 (16.1%) were readmitted within 30 days after surgery. Therefore, the NSQIP method had a sensitivity of 85% for 30-day postdischarge readmission in this population. This undercounting worsened with longer lengths of stay and was statistically significant at 25 days (P = .01, determined by use of the Fisher exact test) (Figure).

The NSQIP began reporting the day of readmission in 2012. This information can be used to overcome the immortal time bias. Truncating our data at 30 days after surgery as if it were from the NSQIP, we estimated the 30-day postdischarge readmission rate using Kaplan-Meier methods. This resulted in an estimated readmission rate of 17.2% (95% CI, 14.5%-20.3%), which included the true value of 18.9%.

Conclusions

The NSQIP systematically undercounts 30-day postdischarge readmissions, and this bias worsens with longer lengths of stay. The Medicare Hospital Readmissions Reduction Program enforces larger and more widespread financial penalties than any other quality measure of any type, and it is expanding into surgery. The NSQIP and other programs like it, which are designed to help hospitals improve quality, should “teach to the test” and adopt the 30-day postdischarge readmission definition. Until this occurs, research on readmission using NSQIP data should use Kaplan-Meier methods to avoid underestimating readmission rates.

Back to top
Article Information

Corresponding Author: Timothy M. Pawlik, MD, MPH, PhD, Department of Surgery, Johns Hopkins Hospital, 600 N Wolfe St, Blalock 688, Baltimore, MD 21287 (tpawlik1@jhmi.edu).

Published Online: July 2, 2014. doi:10.1001/jamasurg.2014.115.

Author Contributions: Drs Lucas and Pawlik had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Lucas, Haut, Pawlik.

Acquisition, analysis, or interpretation of data: Lucas, Hechenbleikner, Wick, Pawlik.

Drafting of the manuscript: Lucas, Hechenbleikner, Pawlik.

Critical revision of the manuscript for important intellectual content: Lucas, Haut, Wick, Pawlik.

Statistical analysis: Lucas, Pawlik.

Administrative, technical, or material support: Hechenbleikner.

Study supervision: Haut.

Conflict of Interest Disclosures: Dr Haut receives royalties from Lippincott Williams & Wilkins for a book he coauthored (Avoiding Common ICU Errors). He has received honoraria for various speaking engagements regarding clinical and quality and safety topics and has given expert witness testimony in various medical malpractice cases. No other disclosures are reported.

Disclaimer: The views expressed in this article are those of the authors and do not represent the official policy of the US Navy, the Department of Defense, or the US government.

Additional Information: Dr Haut is the primary investigator of the Mentored Clinician Scientist Development Award K08 1K08HS017952-01 from the Agency for Healthcare Research and Quality entitled “Does Screening Variability Make DVT an Unreliable Quality Measure of Trauma Care?” Dr Haut is the primary investigator of a contract with The Patient-Centered Outcomes Research Institute entitled “Preventing Venous Thromboembolism: Empowering Patients and Enabling Patient-Centered Care via Health Information Technology.”

References
1.
Jencks  SF, Williams  MV, Coleman  EA.  Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428.
PubMedArticle
2.
American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). User guide for the 2011 Participant Use Data File. http://site.acsnsqip.org/wp-content/uploads/2012/03/2011-User-Guide_Final.pdf. Published October 2012. Accessed May 28, 2013.
3.
Lucas  DJ, Haider  A, Haut  E,  et al.  Assessing readmission after general, vascular, and thoracic surgery using ACS-NSQIP. Ann Surg. 2013;258(3):430-439.
PubMedArticle
4.
Suissa  S.  Immortal time bias in pharmaco-epidemiology. Am J Epidemiol. 2008;167(4):492-499.
PubMedArticle
5.
Hechenbleikner  EM, Makary  MA, Samarov  DV,  et al.  Hospital readmission by method of data collection. J Am Coll Surg. 2013;216(6):1150-1158.
PubMedArticle
6.
Kaplan  EL, Meier  P.  Nonparametric estimation from incomplete observations. J Am Stat Assoc. 1958;53(282):457-481. doi:10.1080/01621459.1958.10501452.Article
×