Quality of Evidence Underlying the American Heart Association/American College of Cardiology/Heart Rhythm Society Guidelines on the Management of Atrial Fibrillation

Question  How have the American Heart Association/American College of Cardiology/Heart Rhythm Society clinical practice guidelines on atrial fibrillation changed over time with respect to the distribution of recommendations across classes of recommendations and levels of evidence?

Findings  This review of guideline recommendations found that despite a significant increase in atrial fibrillation research from 2001 to 2014, there was a nonsignificant increase in the use of level B evidence, an insignificant decrease in the use of level C evidence, and the use of level A evidence did not appreciably change. The 2014 guideline was largely supported by level C evidence with few level A recommendations (8.8%); no rate control recommendations were supported by level A evidence.

Meaning  Significant opportunities exist to improve the evidence base on the management of atrial fibrillation, specifically within rate control.

Importance  The joint American College of Cardiology (ACC), American Heart Association (AHA), and Heart Rhythm Society (HRS) guidelines on the management of atrial fibrillation (AF) are used extensively to guide patient care.

Objective  To describe the evidence base and changes over time in the AHA/ACC/HRS guidelines on AF with respect to the distribution of recommendations across classes of recommendations and levels of evidence.

Data Sources  Data from the AHA/ACC/HRS guidelines on AF from 2001, 2006, 2011, and 2014 were abstracted. A total of 437 recommendations were included.

Data Extraction and Synthesis  The number of recommendations and distribution of classes of recommendation (I, II, and III) and levels of evidence (A, B, and C) were determined for each guideline edition. Changes in recommendation class and level of evidence were analyzed using the 2001 and 2014 guidelines.

Results  From 2001 to 2014, the total number of AF recommendations increased from 95 to 113. Numerically, there was a nonsignificant increase in the use of level of evidence B (30.5% to 39.8%; P = .17) and a nonsignificant decrease in the use of level of evidence C (60.0% to 51.3%; P = .21), with limited changes in the use of level A evidence (8.4% to 8.8%; P = .92). In the 2014 guideline document, 10 of 113 (8.8%) recommendations were supported by level of evidence A, whereas 58 of 113 (51.3%) were supported by level of evidence C. Most recommendations were equally split among class I (49/113; 43.4%) and class IIa/IIb (49/113; 43.4%), with the minority (15/113; 13.3%) assigned as class III. Most class I recommendations were supported by level of evidence C (29/49; 59.2%), whereas only 6 of 49 (12.2%) were supported by level of evidence A. No rate control category recommendations were supported by level of evidence A.

Conclusions and Relevance  Some aspects of the quality of evidence underlying AHA/ACC/HRS AF guidelines have improved over time. However, the use of level of evidence A remains low and has not increased since 2001. These findings highlight the need for focused and pragmatic randomized studies on the clinical management of AF.

Since 2001, the American College of Cardiology (ACC) and American Heart Association (AHA) have published clinical practice guidelines on atrial fibrillation (AF). In 2011, the Heart Rhythm Society (HRS) joined as a key author and stakeholder. The AHA/ACC/HRS guidelines on AF are widely used to guide patient care and have recently inspired a hospital-based, national performance initiative.¹ Given the level of importance these guideline documents have assumed in recent years, it is vital to understand how the AF guideline documents have changed over time and to identify gaps in the evidence underlying specific recommendations. The objective of this study was to determine whether there have been meaningful changes in the strength of recommendations and quality of evidence used in the AF guidelines since 2001. Our goal was to identify gaps in the evidenced-based foundation of the guidelines and thereby highlight areas in need of further research.

For full details regarding the methods used in this study, see the eAppendix in the Supplement. Briefly, all recommendations from the AHA/ACC/HRS clinical practice guidelines on AF from 2001,² 2006,³ 2011,⁴ and 2014⁵ were abstracted. For each recommendation, class of recommendation (COR), level of evidence (LOE), and AF category (antithrombotic therapy, rate control, maintenance of sinus rhythm, and miscellaneous) were recorded. The distribution of recommendations among various COR (class I, II, or III) and LOE (level A, B, or C) classifications was determined for each document. This analysis was repeated for specific COR-LOE combinations (eg, I-A and I-B) and by AF categories. Changes in proportions were reported as absolute differences (2014 vs 2001) and statistical testing was performed using the 2-proportion z test (2014 vs 2001). All calculations were performed in Microsoft Excel. To quantify the amount of clinical research on AF in recent years, the MEDLINE database was searched for studies on AF from 2001 to 2014 using various Medical Subject Headings terms (eAppendix in the Supplement).

From 2001 to 2014, the annual number of publications on AF and the annual number of randomized clinical trials on AF both increased significantly (255% and 244% increase, respectively; Figure). The distribution of AHA/ACC/HRS guideline recommendations on AF categorized by COR and LOE over this same period is shown in Table 1 and eFigure 1 in the Supplement. The total number of guideline recommendations increased from 95 to 113. There were no statistically significant changes in the proportions of recommendations assigned to each COR or LOE. Numerically, the use of LOE B increased (30.5% to 39.8%) and the use of LOE C decreased (60.0% to 51.3%). The use of LOE A did not change significantly (8.4% to 8.8%). In the 2014 guideline, LOE A was the least common designation (8.8%) and LOE C the most common (51.3%). The distribution of recommendations across COR did not significantly change. In the 2014 guideline, most recommendations were equally split among class I (43.4%) and class IIa/IIb (43.4%), with the minority (13.3%) assigned as class III.

The distribution of recommendations organized by specific COR-LOE combinations is shown in Table 2 and eFigure 2 in the Supplement. There were no statistically significant changes in the proportions of recommendations within each COR-LOE combination from 2001 to 2014. The largest absolute changes were observed in II-B (increase from 11.6% to 20.4%) and II-C (decrease from 28.4% to 19.5%). In the 2014 guideline edition, I-C was the most common designation (25.7%) followed by II-B (20.4%).

Within specific AF guideline categories, there were several notable changes in the use of COR and LOE, although no changes reached statistical significance (Table 1; eFigure 3 in the Supplement). Level C evidence decreased in all categories except antithrombotic therapy, in which it remained relatively constant. Level B evidence increased in all categories except miscellaneous. Level A evidence decreased within antithrombotic therapy but remained constant or increased in the other categories. Rate control was noted to have zero level A recommendations, which has not changed since 2001. Class I recommendations increased in antithrombotic therapy but remained constant or decreased in other categories. Class II recommendations increased in miscellaneous but decreased or remained constant in other categories. Class III recommendations increased in rate control but remained constant in other categories.

The COR and LOE for 21 individual recommendations that were present in both the 2001 and 2014 guidelines are shown in the eTable in the Supplement. Level of evidence was downgraded in 6 of 21 recommendations and upgraded in 2 of 21. Class of recommendation was not downgraded in any recommendation and was upgraded in 2 of 21. The distribution of recommendations across different guideline categories in 2001 and 2014 is shown in eFigure 4 in the Supplement.

This study presents several key findings. First, from 2001 to 2014, the quality of evidence underlying AF guideline recommendations improved. Specifically, there was a decrease in the use of LOE C and an increase in the use of LOE B over time, indicating less reliance on expert consensus and a larger role of medium-quality clinical evidence. However, despite these favorable changes, the use of LOE A did not increase, and only 8.8% of all recommendations in the 2014 guideline were supported by LOE A. This is despite the large increase in the number of published randomized trials on AF per year over this period (Figure), indicating that these studies may not be of sufficient size or methodologic rigor, or address a pragmatic decision/treatment, to result in an LOE A guideline recommendation.

Rate control was identified as a specific area with limited supporting evidence and recommendation strength. In the 2014 guideline, rate control had the fewest recommendations supported by LOE A (0%) and the lowest proportion of class I recommendations (25%). Neither figure has improved appreciably since 2001. These findings highlight the uncertainty regarding optimal rate control in AF. Despite numerous studies in this area,^6,7 it remains unclear whether strict rate control is beneficial in certain patient populations.

Our findings are similar to those of a previous analysis of all AHA/ACC guidelines published between 1984 and 2008, which found that 11% of all recommendations were supported by LOE A and 48% by LOE C.⁸ However, it is notable that the AF guideline lags behind those for several other common diseases cited in this study, such as heart failure and unstable angina, which have a much higher proportion of class I and LOE A recommendations, likely owing to the unique challenges in studying and treating AF.

We did not observe any significant change in the distribution of recommendations across COR. Class II recommendations, which are subject to uncertainty regarding the appropriate medical decision, continue to comprise a large part of the document. In addition, I-C, which combines the strongest recommendation with the weakest level of evidence, remains the most common COR-LOE combination. These factors highlight the uncertainty and lack of supporting evidence for many recommendations in the 2014 guideline. Changes regarding antithrombotic therapy occurred mostly between 2011 and 2014, likely owing to the recent availability of data regarding non–vitamin K antagonist oral anticoagulants.

In our view, these findings do not diminish the value or importance of the guidelines but rather highlight the need for additional high-quality studies targeted to specific and pragmatic clinical questions in AF. However, it should be noted that not all topics or recommendations may be amenable to clinical studies. Accordingly, it may be reasonable for guideline authors to use LOE C and class II recommendations to provide guidance in these areas. However, these findings do emphasize that quality metrics based on guideline adherence should carefully take into account the level of supporting evidence for specific recommendations.

This analysis has several limitations. We did not take into account any revisions to the aims and methodology used by ACC/AHA/HRS over the study period. These revisions include those outlined in the 2012 American College of Cardiology Foundation/AHA guideline methodology summit report,⁹ such as the inclusion of patient representatives on the guideline committee. Clinical practice guidelines from other organizations, such as the European Society of Cardiology or the American College of Chest Physicians, were not included in this analysis. We also did not consider areas of AF evaluation and management not currently represented in the guidelines.

In summary, there was improvement in the quality of evidence underlying the AHA/ACC/HRS AF guidelines since 2001, as evidenced by an increase in LOE B and decrease in LOE C. However, the use of LOE A remains low and has not increased appreciably, and class II and I-C recommendations continue to comprise a large part of the guideline document. These findings highlight the need for focused, randomized, and pragmatic studies on the clinical management of AF.

Accepted for Publication: October 20, 2016.

Corresponding Author: Jonathan P. Piccini, MD, MHS, Electrophysiology Section, Duke Center for Atrial Fibrillation, Duke University Medical Center, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27710 (jonathan.piccini@duke.edu).

Published Online: December 14, 2016. doi:10.1001/jamacardio.2016.4936

Author Contributions: Drs Barnett and Piccini had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Barnett, Fonarow, Steinberg, Piccini.

Acquisition, analysis, or interpretation of data: Barnett, Lewis, Field, Fonarow, Gersh, Page, Calkins, Peterson, Piccini.

Drafting of the manuscript: Barnett, Piccini.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Barnett, Peterson.

Administrative, technical, or material support: Piccini.

Supervision: Steinberg, Peterson, Piccini.

Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Fonarow provides consulting to Janssen Pharmaceuticals, Medtronic, and St Jude Medical, and he has received personal fees from Janssen Pharmaceuticals and Medtronic. Dr Calkins has received personal fees for serving as a consultant to Medtronic, Abbott Medical, and Boehringer Ingelheim. Dr Steinberg receives research support from Janssen Pharmaceuticals and Boston Scientific, provides consulting to BMS/Pfizer, and receives educational support from Medtronic and Biotronik. Dr Peterson has received personal fees and/or grants from AstraZeneca, Bayer Pharmaceuticals, Janssen Pharmaceuticals, Merck & Co, Regeneron Pharmaceuticals, Sanofi-Aventis, and Valeant Pharmaceuticals. Dr Piccini receives research funding from Agency for Healthcare Research and Quality, ARCA biopharma, Boston Scientific, Gilead Sciences, Janssen Pharmaceuticals, Johnson & Johnson, ResMed, Spectranetics, and St Jude Medical and provides consulting to BMS/Pfizer, GlaxoSmithKline, Janssen Pharmaceuticals, Johnson & Johnson, Medtronic, and Spectranetics. No other disclosures were reported.

Disclaimer: Dr Fonarow is the Associate Editor for Health Care Quality and Guidelines for JAMA Cardiology but was not involved in the editorial review or the decision to accept the manuscript for publication.