A, Composite serious adverse events. B, Hospital-acquired infections. In both graphs, the bars and left y-axes measure relative contributions; the curve and right y-axes, total patients.
Shah RK, Stey AM, Jatana KR, Rangel SJ, Boss EF. Identification of Opportunities for Quality Improvement and Outcome Measurement in Pediatric Otolaryngology. JAMA Otolaryngol Head Neck Surg. 2014;140(11):1019-1026. doi:10.1001/jamaoto.2014.2067
Despite increased emphasis on measuring safety outcomes and quality indicators for surgical care, little is known regarding which operative procedures should be prioritized for quality-improvement initiatives in pediatric otolaryngology.
To describe the 30-day adverse event rates and relative contributions to morbidity for procedures in pediatric otolaryngology surgery using data from the American College of Surgeons’ National Surgical Quality Improvement Program Pediatric database (ACS-NSQIP-P).
Design, Setting, and Participants
Analysis of records contained in the ACS-NSQIP-P 2011-2012 clinical database. The ACS-NSQIP-P is a nationwide risk-adjusted, clinical outcomes–based program aimed at measuring and improving pediatric surgical care. Fifty hospitals participated in the 2011-2012 ACS-NSQIP-P program. Medical records of patients who underwent tracked otolaryngologic procedures were accrued in the ACS-NSQIP-P database. These were inclusive of specific otolaryngologic surgical procedures and do not represent the entire spectrum of pediatric otolaryngology surgical procedures.
Main Outcomes and Measures
Individual 30-day adverse events, composite morbidity, composite serious adverse events, and composite hospital-acquired infections were compiled. Clinically related procedure groups were used to broadly evaluate outcomes. Procedures and groups were evaluated according to their relative contribution to otolaryngologic morbidity and their incidence of major complications.
A total of 8361 patients underwent 1 of 40 selected otolaryngology procedures; 90% were elective; 76% were performed on an outpatient or ambulatory basis; and 46% were American Society of Anesthesiologists (ASA) class 2 cases. Individual 30-day adverse event rates were highest for return to the operating room (4%), surgical site infection (2%), pneumonia (1%), sepsis (1%), and reintubation (1%). The highest rates of composite morbidity were seen for tracheostomy in patients younger than 2 years (23%), airway reconstruction (19%), and tympanoplasty with mastoidectomy (2%). Airway reconstruction procedures had the highest rates of composite serious adverse events (16%), followed by tracheostomy (13%) and abscess drainage (5%). Tracheostomy (31%) and airway reconstruction (16%) made the largest relative contributions to composite morbidity rate of the procedures studied. Tracheostomy in patients younger than 2 years had the highest composite hospital-acquired infection rate (14%), followed by airway reconstruction procedures (11%) and tympanoplasty with mastoidectomy (2%).
Conclusions and Relevance
While the overall rate of major postoperative morbidity in pediatric otolaryngology is low, areas for targeted quality-improvement interventions include tracheostomy, airway reconstruction, mastoidectomy, and abscess drainage. Measurement of outcomes specific to otolaryngologic procedures will be necessary to further identify and measure the impact of quality-improvement initiatives in pediatric otolaryngology.
In surgical practice, significant attention is paid to understanding causes of and reducing perioperative adverse events. However, in pediatric otolaryngology, it is sometimes difficult for individual institutions to reliably perform quality assessments and interventions within a reasonable time frame because the frequency of adverse events within any single institution is low. National aggregate databases can be used to identify variances from best practices and subsequently identify improvement opportunities for each institution.1 One successful example of such a database is the American College of Surgeons (ACS) National Surgical Quality Improvement Program (ACS-NSQIP).2 This landmark platform was successfully used in the Veterans Administration hospitals and from there was translated to encompass the broad field of surgical care in adults. In partnership with the American Pediatric Surgical Association and the ACS, the platform was further expanded to include pediatric surgical care (ACS-NSQIP-P) and now serves as the only nationwide, risk-adjusted, outcomes-based program evaluating pediatric surgical care in the United States.3 A large spectrum of pediatric hospitals, from free-standing children’s hospitals to general acute care hospitals with a pediatric wing, participate in the program allowing for more reliable composite data and improvement analysis.
The value of the ACS-NSQIP in the Veterans Administration, and now in pediatric surgical care, has been established by its use in successful pilot and beta studies demonstrating how to use this database to potentially improve quality of care.3- 5 As the ACS-NSQIP-P has matured, the involvement of more surgical subspecialties has become a priority, and the leadership of the ASC-NSQIP-P has been welcoming and inclusive of surgical specialties as they realize the value of specialists within the scope of pediatric surgical care.5 Surgical specialty participation is critical within the ASC-NSQIP-P because specialty content experts help to determine and shape how these data can be used to improve the care of patients in their fields. In adult care, otolaryngologists have used the ACS-NSQIP to help prioritize quality improvement using risk-adjusted outcomes data reported across this platform.6- 8
To our knowledge, the present study is the first to evaluate perioperative outcomes in pediatric otolaryngology through analysis of records residing in the ACS-NSQIP-P database. We describe herein the 30-day adverse events rates and relative contributions of pediatric otolaryngology procedures to morbidity using data from the ACS-NSQIP-P. We hypothesize that analysis of these data will identify specific procedure groups that contribute significantly to morbidity, therefore facilitating prioritization of future quality-improvement efforts in pediatric otolaryngology.
This study was reviewed by the RAND corporation institutional review board and qualified for exemption. Patient informed consent was also waived.
We performed a cohort study of children of all ages who underwent otolaryngology procedures at participating ACS-NSQIP-P hospitals in 2011 and 2012. Dedicated clinical abstractors collect clinical data at each participating institution, including demographic information, comorbid diagnoses, laboratory results, and 30-day postoperative adverse events. Operations are recorded by their primary procedure Current Procedural Terminology (CPT) codes.
The sampling methodology for ACS-NSQI-P is to include 35 different types of procedures over an 8-day cycle at each participating institution. As such, not all procedures are equally represented in the ACS-NSQIP-P database. The CPT codes of lower-risk, higher-volume procedures (eg, adenotonsillectomy and tympanostomy tube placement) are not collected in order to increase the capture of rarer procedures that may involve higher risk. The rationale for such collection methodology is that inclusion of high-volume, low-risk procedures would affect the integrity of the ACS-NSQIP-P as a quality-improvement tool.
There are 21 individual postoperative adverse events collected in the ACS-NSQIP-P database as dichotomous variables. All postoperative adverse events are actively followed up to 30 days postoperatively. There are no missing data on adverse events and no loss of follow-up over the 30-day study period. Otolaryngology-related CPT codes were initially selected and approved for collection through an ACS-NSQIP leadership committee process. Individual procedures were chosen to be comprehensively inclusive of specific subspecialties of otolaryngology care.
The primary predictor in this analysis was the operative procedure, as designated by CPT code. Forty CPT codes were the initial units, and these were studied and sorted into 9 distinct clinically related procedure groups based on clinical relevance (Table 1). Procedures that were clinically distinct but unrelated to the 9 distinct categories, with fewer than 100 patients, were put into the “other” category, included to ensure the integrity of the proportions and statistical analyses.
The 3 outcomes of interest in this analysis judged to be of significant public health relevance were the composite morbidity, composite serious adverse events, and composite hospital-acquired infections. It has been shown that 3 composite outcomes may be more reliable in estimating surgical quality than single adverse events.9,10
Composite morbidity encompasses all 30-day adverse events commonly used by the ACS-NSQIP in benchmarking hospital morbidity, including the occurrence of any 1 or more of the following within 30 days postoperatively: surgical site infection, pneumonia, reintubation, renal insufficiency, urinary tract infection, coma, seizure, peripheral nerve injury, intraventricular hemorrhage and/or intracranial hemorrhage, cardiac arrest, intraoperative and postoperative transfusion, graft failure, venous thrombosis requiring therapy, sepsis, and central line–associated bloodstream infection. Composite serious adverse event includes the following unplanned serious events with medium- and long-term implications: 30-day reoperation, 30-day reintubation, and 30-day cardiac arrest. Composite hospital-acquired infection included the following nosocomial infections not present preoperatively: 30-day urinary tract infection, 30-day surgical site infection, and 30-day pneumonia.
Descriptive statistics (count and frequency) of the preoperative clinical characteristics of the patients who underwent otolaryngologic procedures were calculated. The number of patients in each procedure category was calculated. The adverse event rate was then calculated for each procedure category. This calculation was performed by dividing the number of patients who underwent the procedure and experienced the event by the total number of patients who underwent the procedure.
The relative contribution of each procedure group to the total number of all composite morbidities in otolaryngology was calculated. This calculation was performed by dividing the number of patients with any composite morbidity undergoing each procedure by the total number patients with composite morbidity undergoing all procedures. The same was done to identify which procedures made the largest relative contribution to serious adverse events and hospital-acquired infections. Procedures were then ranked based on the relative contribution to composite morbidity, composite serious adverse events, and composite hospital-acquired infections and plotted in a Pareto chart with stacked histogram for composite serious adverse events and hospital-acquired infections. All data management and analyses were performed using SAS software, version 9.3 (SAS Institute Inc).
A total of 8361 patients underwent 1 of 40 individual otolaryngology procedures categorized into 10 procedure groups at one of the 50 ACS-NSQIP-P participating hospitals between 2011 and 2012. There was a median of 591 patients per procedure group (interquartile range, 183-1247) (Table 2). A total of 90% of procedures were elective, and 76% were performed on an outpatient or ambulatory basis. Most cases (46%) had an American Society of Anesthesiologists (ASA) classification of 2. Pulmonary comorbid conditions were relatively common: 3% of children were ventilator dependent; 4% required oxygen supplementation; and 2% had a tracheostomy preoperatively. Of note, 6% of children were septic preoperatively. Subcategory percentages may not tally exactly to category totals owing to rounding.
The average rates of individual 30-day adverse events were highest for return to the operating room (4%), surgical site infection (2%), pneumonia (1%), sepsis (1%), and reintubation (1%) (Table 3). Tracheostomy in children younger than 2 years had the highest rate of composite morbidity (23%), followed by airway reconstruction procedures (19%) and tympanoplasty with mastoidectomy procedures (2%).
Airway reconstruction procedures had the highest rates of composite serious adverse events (16%): return to the operating room (13%), reintubation (6%), and cardiac arrest (2%). Tracheostomy in children younger than 2 years had the next highest rate of serious adverse events (13%): return to the operating room (11%), reintubation (1%), and cardiac arrest (3%). Abscess drainage procedures had the third highest serious adverse event rate (5%): return to the operating room (4%) and reintubation (<1%).
Tracheostomy in patients younger than 2 years had the highest hospital-acquired infection rate (14%): surgical site infection (5%), pneumonia (7%), and urinary tract infections (2%). Airway reconstruction procedures had the second highest rate of hospital-acquired infections (11%): surgical site infection (6%) and pneumonia (6%). Tympanoplasty with mastoidectomy had the third highest hospital infection rate (2%): surgical site infection (2%) and pneumonia (<1%).
Tracheostomy in patients younger than 2 years made the largest relative contribution (31%) to composite morbidity of all the studied procedures within otolaryngology (Figure, A). Airway reconstruction procedures made the second largest relative contribution (16%) to composite morbidity in otolaryngology.
Tracheostomy in patients younger than 2 years also made the largest contribution (27%) to serious adverse events in otolaryngology, followed by abscess drainage procedures (27%) and airway reconstruction procedures (21%) (Figure, B). Analysis of the individual adverse events that made up the composite serious adverse events demonstrated that abscess drainage procedures made the largest relative contribution to unplanned reoperation (30% of reoperations), followed by tracheostomy (25%) and airway reconstruction procedures (19%). Airway reconstruction procedures made the largest relative contribution to unplanned reintubation (63% of reintubations), followed by tracheostomy (11%) and abscess drainage procedures (11%). Tracheostomy made the largest relative contribution to cardiac arrest (67% of all cardiac arrests) followed by airway reconstruction procedures (33%).
Tracheostomy in children younger than 2 years also made the largest contribution (26%) to hospital-acquired infections in otolaryngology (Figure, B). Tracheostomy in children younger than 2 years made the largest relative contribution to pneumonia (55%), followed by airway reconstruction procedures (28%) and cochlear device implantation (8%). Patients undergoing tracheostomy accounted for all urinary tract infections in otolaryngology.
This study examined the outcomes of 40 pediatric otolaryngology procedures performed at 50 participating institutions in 2011 and 2012 using the ACS-NSQIP-P platform. To our knowledge, this study is the first to evaluate utility of ACS-NSQIP-P for measuring 30-day adverse events in pediatric otolaryngology. The ACS-NSQIP-P is a robust clinical data source that contains 30-day perioperative outcomes.2- 8
This analysis demonstrates that there is an immediate opportunity for focusing quality-improvement efforts in pediatric otolaryngology on patients younger than 2 years undergoing tracheostomy, airway reconstruction procedures, otologic surgery, and abscess drainage. These procedures have comparatively high adverse event rates and account for the largest proportion of adverse events in otolaryngology. Further examining these procedures in depth within ASC-NSQIP-P, through expansion of variables to include procedure-specific outcome measurement, could enable targeted quality-improvement efforts. The present study is invaluable in highlighting these 4 procedures as areas for improvement.
As expected in our specialty, pediatric otolaryngology procedures are commonly high volume and frequency and are associated with very low morbidity. The overall safety and low adverse event rate is not a novel finding in pediatric surgery and indeed has been shown across the overall platform.3- 5 This finding represents a difference between adult and pediatric surgical care, and this disparity has been corroborated in the literature using the ACS-NSQIP-P. Issues of high frequency and low morbidity were outlined in a publication3 that described the potential challenges in developing a pure pediatric surgical database and were a priority considered by the leadership of the ACS-NSQIP-P. Specifically, issues of concern included marked variances in outcomes, very low overall mortality in pediatrics, and the need for caregiver assessments because of the age and cognitive limitations of children.3
No single platform or analysis technique will address all areas of improvement. Each tool will produce the results and findings that it is built to provide. The ACS-NSQIP-P sampling and prioritization strategy identifies the opportunities in pediatric otolaryngology discussed herein, but these are not the only zones of risk in our specialty. For example, to properly study and perhaps improve outcomes in adenotonsillectomy (high volume, relatively low risk), another platform or methodology than ACS-NSQIP-P will be necessary.
There are some comparisons between the data reported in the present study from the ACS-NSQIP-P and that in the adult NSQIP literature. An outpatient readmission rate for adult otolaryngology patients was found to be 2.01% in a study evaluating the 2011 NSQIP.7 For inpatient otolaryngologic surgery, the most common postdischarge complications were surgical site infections, other infections, and deep venous thrombosis.8 In the present analysis using ACS-NSQIP-P, we identified that the most common 30-day adverse event rates were for return to the operating room, surgical site infections, pneumonia, sepsis, and reintubation (Table 3). The disparity between adult and pediatric outcomes demonstrates the need for pediatric-focused investigations to identify specific and actionable improvement opportunities.
The initial phase 1 report on the feasibility and initial outcomes for the ASC-NSQIP-P demonstrated the relative importance of otolaryngology, where otolaryngologic procedures represented 23.2% of all sampled cases.4 Interestingly, despite the high volume of otolaryngology procedures sampled and included in that study, the 1.5% rate of complication was the lowest of the pediatric surgical specialties. For that reason, improving outcomes in pediatric otolaryngology will require focused initiatives to further reduce the adverse event rate of 1.5%.
Alternatively, the outcomes evaluated may not be meaningful or fully representative of quality surgical care in pediatric otolaryngology. For example, return to the operating room following airway reconstruction may not represent a useful measure of morbidity because scheduled postoperative endoscopy and debridement are often standard components of these procedures; the ASC-NSQIP-P platform does not allow for nuances and interpretations for such returns to the operating room. A more useful outcome measure would be procedure specific and may relate to quality of the graft placement or rate of restenosis. Likewise, a measure that evaluates hearing or speech outcomes following otologic surgery may be more useful than generalized measures establishing a low overall rate of morbidity.
In the future, the onus will rest on each individual surgical specialty society to use the data within ACS-NSQIP-P to identify, measure, and quantify outcomes relevant to that specialty. The data highlighted herein evaluate 30-day adverse events and surgical outcomes for pediatric otolaryngology with ACS-NSQIP-P. An institution, whether or not they participate in ACS-NSQIP-P, can use the present study to establish normative baselines and focus their quality-improvement initiatives on specific procedure groups. As a specialty, pediatric otolaryngology can use these baseline data to inform future inclusion of procedures within ACS-NSQIP-P and generate measures that are more specific and meaningful to otolaryngology procedures that will ultimately result in a more robust platform and more meaningful metrics.
There are several limitations of this study. First, a potential limitation of any observational study is the integrity of the data acquisition process. The administrators of the ACS-NSQIP platform make great efforts to ensure the data accuracy, including random sampling of cases, training of abstractors, and regular data audits. A second limitation is that despite the ability for all institutions to participate, the majority of ACS-NSQIP-P hospitals are large academic pediatric hospitals. This composition may make the findings of this study less generalizable to smaller institutions or different practice settings. A third limitation is that this study combines and analyzes procedures based on CPT codes, not the surgical service performing the procedure. In the ACS-NSQIP platform, the goal is to improve the outcomes for procedures, regardless of the surgical service performing the procedure.
A fourth limitation is that ACS-NSQIP-P does not collect all procedures performed by otolaryngologists but instead, by design, gathers data only on procedures with high rates of adverse events. Certain very common procedures performed in otolaryngology with low adverse event rates (such as tonsillectomies) were excluded because they skewed the overall data for the entire platform. As a result, the actual estimate of the relative contribution of specific procedure groups to adverse events in otolaryngology will be overestimated. Nonetheless, the adverse event rate in these procedures is so low that the overestimation is most likely negligible.
A final and significant limitation is the definition of a morbidity. For example, it was reported that myringoplasty had a high rate of 30-day surgical site infections. This was most likely because those entering the data were not otolaryngology-trained nurses and might have classified otologic discharge as an infection after speaking with the patient and reviewing pertinent outpatient documentation. However, the practicing otolaryngologist knows that otorrhea is common in patients after myringoplasty, and the outcome is not affected. Unfortunately, ACS-NSQIP does not collect outcomes specific to ultimate success of the operation.
It is also important to clarify that, for procedures, it is the relative contribution to the variable (eg, total hospital-acquired infections), not the absolute rate, that is reported in our results and illustrated in Figure, B. The reason this relative rate is so high for myringoplasty is because myringoplasty was the second most frequently performed procedure; a low rate in common procedure means that more patients will have that variable than in the case where there is a procedure with a higher rate in a smaller number of patients.
While the overall rate of major postoperative morbidity in pediatric otolaryngology is low, areas for targeted quality-improvement interventions include tracheostomy, airway surgery, otologic surgery, and abscess drainage. The data reported herein suggest that such interventions could be focused on these procedures to reduce adverse events in pediatric otolaryngology. Future directions include further examination of these procedures within the ASC-NSQIP-P system to identify, measure, and quantify outcomes relevant to otolaryngology. Moreover, significant efforts should be made to ensure that we are measuring and reporting on pertinent procedures and metrics of value to improve surgical care in pediatric otolaryngology. While the findings from this study provide baseline data on general safety outcome measures, whether or not ACS-NSQIP-P is appropriate for meaningful quality improvement in pediatric otolaryngology remains to be seen.
Submitted for Publication: June 1, 2014; final revision received July 16, 2014; accepted August 5, 2014.
Corresponding Author: Emily F. Boss, MD, MPH, Johns Hopkins University, 601 N Caroline St, Sixth Floor, Baltimore, MD 21287 (email@example.com).
Published Online: September 25, 2014. doi:10.1001/jamaoto.2014.2067.
Author Contributions: Drs Stey and Boss had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Shah, Rangel, Boss.
Acquisition, analysis, or interpretation of data: Shah, Stey, Jatana, Boss.
Drafting of the manuscript: All authors.
Critical revision of the manuscript for important intellectual content: Jatana, Boss.
Statistical analysis: Shah, Stey, Boss.
Obtained funding: Boss.
Administrative, technical, or material support: Shah, Jatana, Rangel, Boss.
Study supervision: Boss.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported in part by the Robert Wood Johnson Foundation Clinical Scholars program and the US Department of Veterans Affairs (Dr Stey) and by the Johns Hopkins Clinician Scientist Award and grant 1K08 HS22932-01 from the Agency for Healthcare Research and Quality (Dr Boss).
Role of the Funder/Sponsor: The supporting institutions had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Previous Presentation: This article was presented at the American College of Surgeon’s National Surgical Quality Improvement Program Annual Meeting; July 27, 2014; New York, New York.