Endovascular aneurysm repair (EVAR) and open aneurysm repair (OAR) components are shown.
The first row shows endovascular aneurysm repair (EVAR) over time with 2003 as the reference. The second row shows open aneurysm repair (OAR) over time with 2003 as the reference. The third row shows EVAR compared with OAR stratified for each year.
Rose J, Evans C, Barleben A, Bandyk D, Wilson SE, Chang DC, Lane J. Comparative Safety of Endovascular Aortic Aneurysm Repair Over Open Repair Using Patient Safety Indicators During Adoption. JAMA Surg. 2014;149(9):926-932. doi:10.1001/jamasurg.2014.1018
In 2003, the Agency for Healthcare Research and Quality established Patient Safety Indicators (PSIs) to monitor preventable adverse events during hospitalizations.
To evaluate the comparative safety of endovascular aneurysm repair (EVAR) vs open aneurysm repair (OAR) of abdominal aortic aneurysm by measuring PSIs associated with each procedure over time.
Design, Setting, and Participants
Cases of abdominal aortic aneurysm repair were extracted from the Nationwide Inpatient Sample (2003-2010). Patient Safety Indicators were calculated using Agency for Healthcare Research and Quality software (Win QI, version 4.4). Unadjusted analysis included year, age, sex, race/ethnicity, comorbidities, rupture status, hospital teaching status, and emergency status. Multivariable analysis was stratified by year for any PSI in EVAR vs OAR. Postoperative mortality was analyzed to control for the overall safety.
Main Outcomes and Measures
Patient Safety Indicators and mortality.
In total, 43 385 EVARs and 27 561 OARs were documented, with 1289 (3.0%) and 3094 (11.2%) associated PSIs, respectively. Compared with those receiving OAR, patients receiving EVAR were more likely to be male, older, and of white race/ethnicity; have a lower Charlson Comorbidity Index; and seek care at teaching hospitals (P < .001 for all). Patients were less likely to have a PSI after EVAR than after OAR. Overall, multivariable analysis showed that EVAR was associated with a 42.1% decrease in the risk-adjusted odds of any PSI compared with OAR (odds ratio, 0.58; 95% CI, 0.51-0.65). Stratified by year, the risk-adjusted odds of any PSI after EVAR were comparatively less likely than after OAR every year except for 2007, and the odds of death were comparatively less every year. The annual percentage of PSIs among all aortic repairs decreased from 7.4% in 2003 to 4.4% in 2010, while the proportion of total repairs that were EVARs increased from 41.1% in 2003 to 75.3% in 2010.
Conclusions and Relevance
Patient Safety Indicators can be used to monitor the comparative safety of emerging surgical technologies. Herein, EVAR was safer than OAR. The adoption of minimally invasive technology can improve safety among surgical admissions.
Studies1,2 have documented the rapid adoption of endovascular aneurysm repair (EVAR) of abdominal aortic aneurysm (AAA) in the United States. The procedure was initially described in a 1991 study,3 and the Food and Drug Administration approved the aortic stent graft in 1999. Since that time, EVAR began to replace open aneurysm repair (OAR), accounting for more than half of elective AAA repairs nationwide by 2003 and 72% by 2006 according to a 2009 study.1 During this period of adoption, the Society for Vascular Surgery reported an overall decline in aneurysm-related deaths after repair of intact or ruptured AAA.4 Randomized clinical trials have shown that EVAR reduces length of stay and postoperative complications.5- 10 However, the effects of rapid adoption on patient safety have not been formally evaluated within existing regulatory frameworks, an important aspect of surgical care to the patient consumer and to health care systems.
In 2003, the Agency for Healthcare Research and Quality (AHRQ)11 established Patient Safety Indicators (PSIs) to standardize monitoring of preventable adverse events during hospitalizations in the United States. In contrast to traditional surgical complications, these quality measures capture adverse events that result from exposure to the health care system itself. Patient Safety Indicators were developed by expert consensus, were validated with empirical analysis, and were shown to have greater specificity and less bias than other measures.12- 14 They are widely endorsed and used by agencies monitoring quality in cases such as the National Quality Forum and the Centers for Medicare and Medicaid Services’ Hospital Compare Program.15 Multiple PSIs were designed specifically for surgery but have only recently been used to evaluate the dissemination of innovative technology.16,17
The evidence base driving the popularity of EVAR did not include comparative evaluation of PSIs after open and endovascular AAA repair. This is true in large part because PSIs were not introduced until 2003. However, recent literature shows higher rates of PSIs after vascular surgery and usefulness in monitoring PSIs after surgery for peripheral vascular disease and unruptured cerebral aneurysm repair.18- 20 This study evaluates the comparative safety of endovascular over open AAA repair by measuring PSIs associated with each procedure over time.
Data from 2003 to 2010 were extracted from the Nationwide Inpatient Sample (NIS).21 The NIS is a public deidentified database. As such, this research did not include contact with any patients or identifiable patient data, and the institutional review board process was waived. The NIS is the largest all-payer inpatient database in the United States, containing data from approximately 1000 hospitals each year. The database is a 20% representative sample of inpatient records from 45 states. In the NIS, hospitalizations are recorded at the patient level with associated diagnoses and procedures using International Classification of Diseases, Ninth Edition (ICD-9) codes. The NIS also merges data from the American Hospital Association to include pertinent hospital variables such as geographic region and teaching status. We used data from 2003 to 2010 because PSIs were introduced in 2003 and because 2010 was the most recent year available.
Data on patients with an AAA diagnosis who underwent EVAR or OAR were extracted from the NIS database using ICD-9 codes 4413 and 4414 for AAA, code 3971 for EVAR, and codes 3834, 3844, 3864, and 3925 for OAR. Cases were included regardless of rupture status and emergency status.
Patient Safety Indicators were defined according to the AHRQ technical specifications and were calculated using AHRQ software (Win QI, version 4.4).11,22 We excluded experimental safety indicators (ie, anesthesia related) and all obstetrics-related indicators. Patient Safety Indicators were calculated as the proportion of inpatients with a given adverse event or PSI (numerator) among all patients at risk for developing the same PSI (denominator). Not all patients are at risk for all PSIs. In fact, the algorithm for each PSI excludes certain patients from the denominator who are unique to that individual PSI. For example, PSI 13 captures postoperative sepsis and uses as a denominator all elective surgical discharge patients 18 years or older and defined by specific diagnosis related groups and ICD-9 codes for a procedure in the operating room. In addition, PSI 13 excludes patients with codes for the following: (1) preexisting sepsis or infection; (2) immunocompromised state or cancer; (3) pregnancy, childbirth, or puerperium; and (4) length of stay less than 4 days. In this way, the PSI algorithms exclude preexisting diagnoses to minimize the influence of patient risk factors. As such, PSIs evaluate the safety of hospital admissions by emphasizing the identification of preventable adverse events, as opposed to merely documenting all postoperative adverse events. This highlights the fundamental difference between PSIs and traditional postoperative complications.
We used t test and Pearson χ2 test to perform univariate and bivariate analyses to identify the likelihood of PSIs 2 through 16 associated with each procedure and for the covariates of year, age, sex, race/ethnicity, Charlson Comorbidity Index, rupture status, hospital teaching status, and emergency status. Multivariable analysis was performed with the above covariates (including year) and any PSI as the outcome to trend the comparative safety of each procedure (with itself) over time. We then stratified the multivariable analysis by year for any PSI for EVAR vs OAR to trend the comparative effectiveness of endovascular vs open repair over time. Mortality after each procedure was also trended with multivariable analysis using the same covariates as a control for the overall safety of each procedure. Statistical significance was defined as P < .05. Statistical analysis was performed using commercial software (STATA SE, version 11.1; StataCorp LP).
In total, 70 946 patients were hospitalized for AAA repair between 2003 and 2010. Of these, 43 385 EVARs (61.2%) and 27 561 OARs (38.8%) were documented (Table 1). Compared with those undergoing OAR, patients undergoing EVAR were more likely to be male, older, and of white race/ethnicity; have a lower Charlson Comorbidity Index; and seek care at teaching hospitals (P < .001 for all). They were also more likely to undergo a procedure in an unruptured, nonemergency clinical setting. The proportion of all AAA repairs that were EVARs increased steadily during the study period from 41.1% in 2003 to 75.3% in 2010 (Figure 1). The total number of AAA repairs did not change significantly during the study period (8609 in 2003 vs 8524 in 2010).
In total, 4383 PSIs were documented during the study period (Table 1). The total number of PSIs annually declined from 641 (7.4%) in 2003 to 377 (4.4%) in 2010. Patients undergoing EVAR were less likely to have any PSI (3.0%) compared with patients undergoing OAR (11.2%). Specifically, patients undergoing EVAR were less likely than those undergoing OAR to experience the following: failure to rescue (12.2% vs 19.9%), iatrogenic pneumothorax (0.1% vs 0.2%), central venous catheter–related bloodstream infection (0.2% vs 1.1%), postoperative hemorrhage (0.4% vs 1.4%), postoperative physiological or metabolic derangement (0.3% vs 1.4%), postoperative respiratory failure (3.3% vs 14.0%), postoperative pulmonary embolism or deep vein thrombosis (0.5% vs 2.0%), postoperative sepsis (3.0% vs 3.8%), and accidental puncture or laceration (1.1% vs 3.0%) (Table 2). No significant differences between EVAR and OAR were observed for the following PSIs: decubitus ulcer, postoperative hip fracture, postoperative wound dehiscence, or transfusion reaction. Aortic aneurysm repairs do not qualify for the category of death in low-mortality diagnosis related groups, and it is included herein only for completeness. The low denominator for postoperative respiratory failure is also owing to strict inclusion and exclusion criteria, available in the AHRQ technical specifications.22
Multivariable analysis showed an overall 42.1% reduction in the likelihood of any PSI after EVAR (odds ratio [OR], 0.58; 95% CI, 0.51-0.65). In fact, when stratified by year with OAR as the reference group, the risk-adjusted odds of any PSI after EVAR were comparatively less likely than after OAR every year except for 2007 (Figure 2). In 2007, the odds of PSIs with EVAR were less than with OAR, but the difference was not significant (OR, 0.82; 95% CI, 0.62-1.10). Multivariable analysis with year as a covariate showed that, when using 2003 as the reference, the risk-adjusted odds of any PSI with EVAR declined by 37.3% between 2003 and 2010 (OR, 0.63; 95% CI, 0.45-0.87 in 2010). This decline was not linear and occurred only in 2004 and 2010, with no difference in other years. The risk-adjusted odds of any PSI after OAR did not change except for 2008, when any PSI was 29.0% more likely (OR, 1.29, 95% CI, 1.07-1.56).
In total, 3717 deaths were documented during the study period. Patients were less likely to die after EVAR (1.8%) than after OAR (10.6%) (Table 1). The risk-adjusted odds of dying after EVAR were 72.3% less than after OAR (OR, 0.28; 95% CI, 0.24-0.32). When stratified by year with OAR as the reference group, the odds of death with EVAR were comparatively less for every year studied (Figure 2). Using year as a covariate with 2003 as the reference, the risk-adjusted odds of death after EVAR decreased in 2004 (OR, 0.57; 95% CI, 0.35-0.92), 2007 (OR, 0.58; 95% CI, 0.38-0.89), 2008 (OR, 0.62; 95% CI, 0.41-0.95), and 2010 (OR, 0.57; 95% CI, 0.39-0.85), while no change was observed in other years. The odds of death after OAR did not change significantly during any of the years studied.
Ensuring the highest standards of patient safety in vascular surgery lies at the core of promoting public health and consumer confidence. Hernandez-Boussard et al19 recently created a safety profile of 8 vascular operations by calculating PSIs after EVAR, OAR, carotid endarterectomy, lower extremity endarterectomy, lower extremity bypass, aortobifemoral bypass, below-knee amputation, and above-knee amputation. Using the NIS database from 2005 to 2009, they found that PSIs were much more likely to occur after these vascular operations than following nonvascular surgery. Pertinent to our discussion, they found that 10.9% of OARs resulted in a PSI, while only 2.8% of EVARs did. Our results (11.2% after OAR and 3.0% after EVAR) may be influenced by the inclusion of emergency and ruptured cases, but they confirm this disparity and strengthen the profile of clinical efficacy of endovascular AAA repair. For example, one might expect fewer accidental lacerations and postoperative respiratory failures with EVAR because access is extraperitoneal via a femoral artery cutdown or percutaneous approach, whereas in OAR the aorta is accessed via transperitoneal or retroperitoneal exposure. In this study, we demonstrate that patient safety is improved when a less invasive technique is used for AAA repair. By calculating PSIs for each procedure over time, we report an overall decline in PSIs associated with AAA repair from 2003 to 2010 that correlated with increased adoption of EVAR technology by vascular surgeons.
The promotion of patient safety is especially important during the dissemination of innovative technology, and PSIs are a growing part of the safety literature in other surgical specialties as well. For example, laparoscopic surgery conferred a 57% reduction in PSIs for left colectomies compared with traditional open procedures.16 Notably, this reduction was achieved as the percentage of laparoscopic vs open colectomies increased from 2% in 2003 to 29% in 2009. Similarly, Parsons et al17 described a 28% reduction in the likelihood of PSI after laparoscopic partial nephrectomy compared with after open surgery between 1998 and 2009, when the proportion of laparoscopy procedures increased from less than 1% to 30%. Furthermore, Stroup et al23 noted the same trend in radical nephrectomy, whereby the adoption of laparoscopy was associated with a 32% reduction in the odds of PSIs after resection of renal tumors between 1998 and 2008, when the prevalence of laparoscopic radical nephrectomy rose from 0.6% to 18.9%. Our results show that PSIs were 42.1% less likely to occur after EVAR compared with after OAR. This association was risk adjusted from a large patient population between 2003 and 2010, when the nationwide proportion of AAAs treated with EVAR increased from 41.1% to 75.3%. As such, this study supports the growing conclusion that the adoption of minimally invasive techniques improves the overall safety of surgical care.
Understanding the dynamics of surgical evolution is key to the study of patient safety. Our retrospective review confirms that the safety of AAA repair improved nationwide with the adoption of EVAR. However, the driving factors making EVAR safer in some years compared with others are unclear. Variation in performance with EVAR may correlate with changes in practice patterns such as adaptations of EVAR devices (ie, percutaneous techniques) or expansion of indications to new patient cohorts (ie, emergency cases). Although beyond the scope of this study, changes in performance over time merit further investigation. The adoption of laparoscopic partial nephrectomy, laparoscopic radical nephrectomy, and laparoscopic colectomies represents the greatest reduction in PSIs after passing a critical “tipping point.”16,17,23 Theoretically, the tipping point corresponds to a transition as early adopters master the skill set of innovators.24,25 In practice, these factors remain elusive and understudied. Future studies could benefit from the standardized PSI algorithms to monitor trends in safety over time and to promote the adoption of the safest techniques.
This study also provides an opportunity to consider the appropriate pace and scope of adoption in relation to randomized clinical trials. Timing was especially relevant in the context of robotics-assisted radical prostatectomy,26 whereby the adoption proceeded to 28% before the first large outcomes study,27 which documented an increased risk of perioperative patient safety events. Rapid dissemination of novel technologies without a foundation of level 1 evidence creates the potential for large-scale disappointment and public harm.28- 30 This contrasts with the experience of laparoscopic colectomy, whereby the results of randomized clinical trials were available before the proportion of laparoscopic techniques exceeded 10% of all colectomies.16 In the case of EVAR, our study captures the period when publications from multiple randomized clinical trials were published.5- 10 When level 1 data became available, EVAR already constituted 50% of all AAA repairs nationwide. However, unlike robotics-assisted prostatectomy, the overall number of AAA repairs did not increase during the adoption of EVAR (Figure 1). This is likely the result of strong indications from the United Kingdom Small Aneurysm Trial31 and the Aneurysm Detection and Management Veterans Affairs Cooperative Study,32 which showed no benefit for repair of AAAs less than 5.5 cm in diameter. The appropriate pace of dissemination remains unknown but would ideally strike a balance between the benefits of new expertise and the cost of the learning curve, while maintaining clear indications for surgery.
The use of PSIs to monitor the safety of innovative technologies carries significant policy implications as well. As hospital performance is increasingly scrutinized, bundled reimbursement plans will favor surgical procedures that add value instead of volume. It is also true that high-level health system infrastructure and financing affect the likelihood of PSI occurrence over time.33 Clement et al33 showed that changes to the reimbursement climate altered occurrence rates in 11 states for select PSIs between 1995 and 2000. Although we do not evaluate systems-level factors herein, standardized metrics to ensure safe delivery of care will be a vital component of quality control as the era of accountable care organizations matures. This crucial component of the future clinical research agenda will allow academic surgeons to have a strong role in producing research that encourages self-regulation and autonomy in the United States health care system.
This study has several inherent limitations that stem from the use of administrative databases. While discharge data sets standardize diagnosis codes (ie, ICD-9), no verification process validates that the codes are used correctly. The ICD-9 codes can represent a broad range of problems, with variation between hospitals, whereby coding is known to be inconsistent.34,35 Our analysis limits this type of coding bias by linking AAA diagnosis to EVAR and OAR procedure codes, which are subject to more oversight because they are frequently tied to billing and reimbursement.36 It is also likely that coding errors are distributed evenly across the database, minimizing quantitative differences in outcomes. Herein, we validated our PSI results by reporting inpatient mortality because it is less ambiguous than other diagnoses. Notably, our mortality after OAR (10.6%) is significantly greater than the mortality after OAR in randomized clinical trials5- 10 (approximately 4%), which may reflect our inclusion of emergency and ruptured cases. This also points to a relative strength of administrative data sets because we show general population results in real-world settings, whereas performance in clinical trials5- 10 may be artificial in that patients and surgeons are rigorously preselected. However, standardized quality indicators rooted in administrative databases are not intended to replace randomized clinical trials, which are better suited to monitor traditional postoperative complications.
Another weakness lies in the timing of this study in relation to the adoption of EVAR. Since its introduction in 1991, EVAR has been noted to follow the standard gaussian pattern of dissemination of technology.2 Studies37,38 of adoption describe 5 stages that correspond to their relationship (in standard deviations) to the mean time of adoption: innovators (>2 SD from the mean), early adopters (1-2 SD), early and late majority (±1 SD), and laggards (>1 SD behind the mean). In our study period, the proportion of AAA repairs that were EVARs increased steadily from 41.1% in 2003 to 75.3% in 2010. The mean time of adoption occurred in 2005, when EVARs accounted for 52.3% of all AAA repairs. This means that our study examined the early and (mostly) late majority. Given known relationships between volume and outcomes,39 it is likely that different phases of the adoption curve display unique safety profiles that are unaccounted for in the present analysis. Our narrow focus on the late majority is due to the fact that an ICD-9 code for EVAR did not exist until 2000 and PSI algorithms were unavailable until 2003. Future studies of this kind should incorporate early phases of the adoption, when the technology transitions from the experienced hands of expert researchers to the inexperienced hands of early adopters.
In summary, EVAR technology is increasingly the preferred method for AAA repair, and this study is the first to date to report PSIs as indicators of the comparative safety of EVAR over OAR in the context of rapid EVAR adoption. Using PSIs as a surrogate for the comparative safety, EVAR was safer than OAR every year and failed to achieve statistical significance only in 2007. In addition, EVAR became safer during the study period, and this finding was validated with corresponding declines in mortality. The volume and safety of open repair remained unchanged. Future monitoring is necessary to identify the driving factors of improvements in safety along the adoption curve. In showing that EVAR is safer than OAR, these results confirm the findings of multiple randomized clinical trials published during the study period and imply a role for PSIs and administrative databases in evaluating the safety of emerging technologies. In addition, this study adds to the evidence that the adoption of minimally invasive surgical techniques improved the overall safety of AAA repairs nationwide.
Accepted for Publication: April 3, 2014.
Corresponding Author: John Rose, MD, MPH, Department of Surgery, University of California, San Diego, 200 W Arbor Dr, San Diego, CA 92103 (firstname.lastname@example.org).
Published Online: July 9, 2014. doi:10.1001/jamasurg.2014.1018.
Author Contributions: Drs Rose and Chang had full access to all the data and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Administrative, technical, or material support: All authors.
Conflict of Interest Disclosures: None reported.
Previous Presentation: This study was presented at the 85th Annual Meeting of the Pacific Coast Surgical Association; February 15, 2014; Dana Point, California.