eAppendix. Templated Peer Review Form for the VA CART MAE Program
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Doll JA, Plomondon ME, Waldo SW. Characteristics of the Quality Improvement Content of Cardiac Catheterization Peer Reviews in the Veterans Affairs Clinical Assessment, Reporting, and Tracking Program. JAMA Netw Open. Published online August 02, 20192(8):e198393. doi:10.1001/jamanetworkopen.2019.8393
Do peer reviews of cardiac catheterization laboratory complications provide useful content for quality improvement?
In this quality improvement study, 152 cardiac catheterization cases selected for peer review because of the occurrence of a major adverse event were analyzed, and only 16.4% of cases were adjudicated as not meeting the standard of care. Concerns about operator performance and judgment were more common, and reviewers recommended improvements in operator performance and care processes in 41.4% and 38.2% of cases, respectively.
Peer review programs should focus on maximizing quality improvement opportunities even when the standard of care is met.
Peer review is recommended for quality assessment in all cardiac catheterization programs, but, to our knowledge, the content of peer reviews and the potential for quality improvement has not been described.
To characterize the quality improvement content of cardiac catheterization peer reviews.
Design, Setting, and Participants
This quality improvement study used retrospective case review of diagnostic angiography and percutaneous coronary intervention procedures to characterize the major adverse event review process of the US Department of Veterans Affairs Clinical Assessment, Reporting, and Tracking (CART) program from January 1, 2012, to December 31, 2016. Data review and analysis took place from November 2017 to August 2018.
Main Outcomes and Measures
Percentage of peer reviews reporting substandard care and opportunities for quality improvement.
A total of 196 643 diagnostic coronary angiograms and 62 576 percutaneous coronary interventions were performed in the Department of Veterans Affairs. Of these, 168 (0.1%) were triggered for review because of a self-reported major adverse event during the procedure. Of 152 cases with complete peer review data, care was adjudicated as not meeting the standard of care in 25 cases (16.4%). Concerns about operator judgment were identified in 46 cases (30.3%), about case selection in 26 (17.1%), about trainee supervision in 21 (13.8%), and about technical performance in 46 (30.3%). Reviewers made recommendations to improve operator performance in 63 cases (41.4%) and catheterization laboratory or hospital processes in 58 (38.2%).
Conclusions and Relevance
While substandard care is infrequently identified in peer review of catheterization laboratory complications in the Department of Veterans Affairs, the process often generates recommendations for quality improvement. Peer review programs should focus on identifying quality improvement opportunities and providing meaningful feedback to operators.
Peer review is recommended for all cardiac catheterization programs as a mechanism to promote clinical proficiency and identify opportunities for quality improvement (QI).1 Review of cases with poor outcomes or unexpected complications (ie, morbidity and mortality conference) is a common form of peer review,2,3 but whether this activity yields content that may guide performance improvement has not been described, to our knowledge. In 2004, the US Department of Veterans Affairs (VA) launched the Clinical Assessment, Reporting, and Tracking (CART) program to study and address concerns about cardiovascular care quality.4 Since 2011, the VA CART program has convened a committee of interventional cardiologists to review major complications that arise during percutaneous coronary intervention (PCI). We sought to characterize the QI content of these peer reviews via a retrospective analysis of program records.
We examined records from all peer-reviewed cases in the VA CART major adverse event (MAE) program from January 1, 2012, to December 31, 2016. Data review and analysis took place from November 2017 to August 2018. Cases were triggered for review by the in-laboratory occurrence of death, stroke, need for urgent coronary artery bypass graft surgery (CABG), or need for urgent unplanned PCI during diagnostic angiography (a trigger since 2015). Percutaneous coronary intervention operators reported the adverse event by selecting the complication from a templated list on the VA CART procedure report. Staff with the VA CART program compiled procedure reports and images from the diagnostic angiography or PCI procedure for dissemination to the MAE peer review committee. The peer review committee is composed of 8 to 12 VA interventional cardiologists, 2 of whom are designated to review each case and given access to the angiography files and all clinical documentation available on the VA national integrated electronic health record. The primary reviewer communicates with the operator to supplement the case documentation, and the secondary reviewer relies on documentation alone. The reviewers write an assessment of the procedure, including recommendations for improvement, using a standard questionnaire that solicits feedback on various aspects of the procedure (eAppendix in the Supplement). Each reviewer independently adjudicates the case using a 3-level rating system: level 1, indicating most experienced competent practitioners would have managed the case similarly in all aspects; level 2, indicating most experienced competent practitioners would have managed the case differently in 1 or more aspects; and level 3, indicating most experienced competent practitioners would have managed the case differently in several aspects. If either reviewer indicates a level 2 or 3 rating, the case proceeds to full committee review during a monthly teleconference. For full committee review, all members are provided access to the case documentation and angiography and invited to submit a level rating in advance of the teleconference. Committee discussion is focused on developing a consensus regarding the final rating and appropriate feedback to the operator. If consensus cannot be reached, a final rating is determined by majority vote. A report is issued to the operator, who can appeal the determination and address the committee personally on a subsequent teleconference. After additional discussion, a final report is issued to the operator and the hospital, including a final rating, a description of the committee discussion, and recommendations for improvement.
In this study, individual reviewer reports and the committee summary report (when present) were reviewed for each case. We described the frequency and indications of MAE cases, reviewer ratings, and content of feedback with summary statistics. To assess the completeness of event capture, we subsequently identified all deaths that occurred on the same day as a diagnostic catheterization or PCI procedure but were not reported to VA CART. We reviewed relevant clinical documentation to understand the cause of death and association with the catheterization procedure. This review was considered a QI activity within the scope of VA CART program operations and not subject to review by the institutional review board. This report adheres to the Revised Standards for Quality Improvement Reporting Excellence (SQUIRE) reporting guideline.
Overall, 196 643 diagnostic coronary angiograms and 62 576 PCIs were performed in the VA from January 1, 2012, to December 31, 2016. Of these, 168 (0.1%) were triggered for review by the VA CART program. Full documentation was available for review of 152 cases, including 78 deaths (51.3%), 47 strokes (30.9%), 24 emergent CABG surgical procedures (15.8%), and 3 unplanned PCIs during diagnostic angiography (2.0%). Documentation for the remaining 16 cases (5 deaths [31.3%], 6 strokes [37.5%], 1 emergent CABG surgical procedure [6.3%], and 4 unplanned PCIs [25.0%]) was irrevocably lost in a software failure. After review, 127 cases (83.6%) were rated level 1, 19 (12.5%) were level 2, and 6 (3.9%) were level 3 (ie, care was adjudicated as not meeting the standard of care in 25 cases [16.4%]). All cases rated level 3 were deaths. Reviewers were concordant in 121 cases (79.6%), but 31 cases (20.4%) were discordant and required full committee review to determine a final rating (Table). The primary reviewer (who spoke with the operator) was more likely to give a level 1 rating than the secondary reviewer (126 cases [82.9%] vs 116 cases [76.3%]).
Concerns about operator judgment were identified in 46 cases (30.3%), about case selection in 26 (17.1%), about trainee supervision in 21 (13.8%), and about technical performance in 46 (30.3%). Reviewers identified potential for improvement in catheterization laboratory processes and operator performance in 58 cases (38.2%) and 63 cases (41.4%), respectively.
Feedback to the operators, when provided, generally focused on the following: (1) case selection, including the indication for revascularization and appropriate discussion of CABG surgery candidacy prior to PCI; (2) preprocedural planning, including need for hemodynamic support or atherectomy; and (3) procedural technique, including use of devices and adjunctive antithrombotic medications. In 2 cases (1.3%), the committee recommended a more extensive evaluation of PCI operator and catheterization laboratory quality.
An additional 107 deaths occurred on the same day as diagnostic coronary angiography or PCI during the study period but were not reported to VA CART. Of these, 10 (9.4%) were in-laboratory deaths and should have been reported by the operator to the MAE program. An additional 66 deaths (61.7%) were considered potentially associated with the cardiac procedures, and 31 deaths (28.9%) were considered unrelated to the procedure.
These data describe the processes of the only national peer review program for cardiac catheterization laboratories, functioning in the largest integrated health care system in the United States. Overall, peer review programs generally target 2 related, but sometimes competing, priorities: (1) quality assurance and (2) QI.5 Quality assurance activities are designed to assess and maintain care standards, with a focus on identifying and correcting outlier performance. In contrast, QI activities avoid negative attribution while targeting opportunities for systems change and education. In the VA CART program, the quality assurance function of peer review is infrequently needed. Only 25 cases (16.4%) over 5 years were determined to not meet the standard of care, and concerns about physician competency were very rare. This is similar to a single-center report of morbidity and mortality activities for PCI,2 in which physician performance was judged “very controversial or unacceptable” in only 5.9% of reviewed cases.
However, the VA CART MAE peer review program often generated QI content even when the standard of care was met. More than 40% of cases prompted recommendations for improved operator performance, and feedback regarding case selection, procedural planning, and technical performance was common. Despite the ubiquity of morbidity and mortality and case review conferences in US catheterization laboratories, there has been little research to describe successful peer review programs. Prior reports of peer review processes for PCI have focused on structural aspects of peer review and have not examined the QI output of the programs.2,3 For example, a 2017 report described review of 157 PCI cases over 10 years at a single academic center.2 Cases were triggered for review by the occurrence of at least 1 of 9 clinical triggers entered into the clinical record by catheterization laboratory staff. Cases were reviewed monthly by faculty using a standardized scoring system. The VA MAE program uses many of the same structural elements as this program but applies them on a national level, using anonymous reviews from clinical experts. Further research is required to determine the strengths and limitations of local vs national systems for peer review. The program assesses only MAEs, and it is unknown if more frequent random case review (as recommended by clinical societies5) or review of less serious complications would yield similar QI content. Complications were self-reported by the operator, and therefore may be subject to underreporting, although only 10 in-laboratory deaths were unreported over a 5-year period. In addition, we only captured complications occurring in the catheterization laboratory. This resulted in complication rates that are low compared with other health systems.2,6 A review of deaths occurring on the same day of the procedure but after the patient was transferred out of the catheterization laboratory revealed that most were potentially associated with care received in the laboratory. We are therefore exploring alternative mechanisms to identify poor procedural outcomes and improve capture of MAE.
This study has several important limitations. We cannot determine the effect of this program on overall cardiac catheterization outcomes at the VA, although peer review is an important component of a unique quality framework that may contribute to improved PCI outcomes at lower cost compared with non-VA facilities.7 Despite progressively increasing preprocedural risk, outcomes of patients treated with PCI at VA hospitals remained constant from 2009 to 2015, a period that spans the initiation of the VA CART MAE program in 2011.8 However, a randomized clinical trial may be required to determine the effect of peer review of clinical outcomes. It is also unclear if this VA experience is generalizable to other settings. The VA is a large national network of hospitals and physicians, all connected through a common electronic health record. Other health systems may require alternative solutions to monitor and improve catheterization laboratory quality.
Peer review of catheterization laboratory complications serves an important quality assurance and QI role in the VA. Opportunities for improvement are often identified even when the standard of care is met. Peer review programs should focus on identifying QI opportunities and providing meaningful feedback to operators.
Accepted for Publication: June 13, 2019.
Published: August 2, 2019. doi:10.1001/jamanetworkopen.2019.8393
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Doll JA et al. JAMA Network Open.
Corresponding Author: Jacob A. Doll, MD, Section of Cardiology, VA Puget Sound Health Care System, 1660 S Columbian Way, S111-CARDIO, Seattle, WA 98108 (firstname.lastname@example.org).
Author Contributions: Dr Doll had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Doll, Waldo.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Doll.
Critical revision of the manuscript for important intellectual content: All authors.
Administrative, technical, or material support: Plomondom, Waldo.
Supervision: Doll, Waldo.
Conflict of Interest Disclosures: None reported.
Funding/Support: Dr Doll is supported by Career Development Award 1IK2HX002590 from the US Department of Veterans Affairs.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of Veterans Affairs or the US government.
Additional Contributions: Candice Mueller, BA, Clinical Assessment, Reporting, and Tracking (CART) program, Veterans Affairs, provided CART peer review information and overall program administration. Ms Mueller is an employee of the CART program and was not otherwise compensated for her assistance with this study.
Create a personal account or sign in to: