Weiss A, Anderson JE, Chang DC. Comparing the National Surgical Quality Improvement Program With the Nationwide Inpatient Sample Database. JAMA Surg. 2015;150(8):815-816. doi:10.1001/jamasurg.2015.0962
Both raw and risk-adjusted outcomes are increasingly being made publicly available.1- 3 The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) is heralded as the most robust database to examine surgical outcomes. However, enrollment in the NSQIP is expensive, and the use of administrative databases may be more cost-effective.2- 4
In our study, we compare the receiver operating characteristic curves of the Nationwide Inpatient Sample (NIS) with those of the NSQIP to determine which is superior at performing analyses of risk-adjusted outcomes for several operations.
Our study uses 2010 data from both the NIS and the NSQIP. Inpatients older than 18 years of age were included. Patients were identified by International Classification of Diseases, Ninth Revision codes (NIS) and Current Procedural Terminology codes (NSQIP): abdominal aortic aneurysm repair, appendectomy, aortic valve replacement, coronary artery bypass graft, carotid endarterectomy, laparoscopic cholecystectomy, total and partial colectomy, esophagectomy, sleeve gastrectomy, pancreatectomy, and ventral hernia repair. Outcomes included inpatient death and complications. Patients were classified as having a complication if they had one of the following: infection (surgical site, deep incisional, and organ/space), wound disruption, pneumonia, pulmonary embolism, acute renal failure, urinary tract infection, cerebrovascular accident, myocardial infarction, and blood loss requiring transfusion.
We performed multivariate logistic regressions predicting inpatient mortality and complication by procedure. In the NSQIP, the models were adjusted for preoperative risk variables in the database. In the NIS, the models were adjusted for age, race, sex, insurance status, and Charlson comorbidity index. The area under the receiver operating characteristic curve (C statistic) was calculated for each model. Statistical analysis was performed using STATA 64-bit Special Edition, version 11.2 (StataCorp). Our study was exempt from review as designated by the University of California, San Diego Human Research Protections Program.
There were 242 584 patients in the NIS and 73 130 patients in the NSQIP. Unadjusted complication rates were higher in the NIS than in the NSQIP for 7 surgical procedures. Mortality rates were higher for most procedures in the NIS; they were similar for appendectomy, laparoscopic cholecystectomy, and coronary artery bypass graft (Table 1). The C statistic was much higher in most logistic regressions for both mortality and complications in the NSQIP database (Table 2).
Our study shows that the NSQIP is superior to the NIS administrative database as represented by higher C statistic values. Our study also finds that both mortality and complication rates were higher in the NIS than in the NSQIP. It is possible that hospitals participating in the NSQIP have lower mortality rates because they systematically examine their surgical outcomes. However, an alternative explanation is that hospitals in the NSQIP underreport their complications. Nurse abstractors are able to reason and exclude complication rates or mortality rates that are not directly related to a procedure. This is not true of the NIS. As the landscape for postoperative complication reimbursement changes, it will be prudent to repeat our study in several years and note if complication rates decrease.
Our study is limited by differences in coding. The NIS relies on automated data extraction from discharge diagnoses, whereas the NSQIP relies on trained nurses to manually extract information. Variations exist between International Classification of Diseases, Ninth Revision and Current Procedural Terminology codes. Also, each database was developed for different purposes and may not correlate.3 The NIS was developed for reimbursement purposes, whereas the NSQIP was developed to evaluate and improve outcomes.1 These databases also include different sample hospitals. The hospitals involved in the NSQIP tend to be academic centers, whereas the NIS includes all US hospital discharges. This may create a sampling error by comparing a consistent NSQIP patient cohort with a relatively varied NIS cohort. In conclusion, the NSQIP allows for a more robust risk-adjusted analysis compared with the NIS, as evidenced by higher C statistic values. Wider participation in the NSQIP could allow more hospitals to participate in robust research in surgical outcomes and quality.
Corresponding Author: Anna Weiss, MD, Department of Surgery, University of California, San Diego, 200 W Arbor Dr, 8402, San Diego, CA 92103 (firstname.lastname@example.org).
Published Online: June 10, 2015. doi:10.1001/jamasurg.2015.0962.
Author Contributions: Dr Weiss had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Weiss, Anderson.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Anderson.
Administrative, technical, or material support: Weiss, Chang.
Study supervision: Weiss, Chang.
Conflict of Interest Disclosures: None reported.
Previous Presentation: This paper was presented at the 86th Annual Meeting of the Pacific Coast Surgical Association; February 19, 2015; Monterey, California.