Assessing the Trustworthiness of the Guideline for Management of High Blood Pressure in Adults

The article by James and colleagues¹ published in JAMA contains the long-awaited guideline for hypertension issued by the panel members appointed by the National Heart, Lung, and Blood Institute (NHLBI) to the Eighth Joint National Committee (JNC 8). In the past decade, the effect of guidelines on clinical practice has increased because organizations that develop quality measures and make coverage decisions depend on guideline developers to uncover the best evidence and make specific recommendations. Because guidelines matter more, critics have questioned the processes that professional associations have used to develop guidelines. The title of the 2011 Institute of Medicine (IOM) report on quality standards for practice guidelines, Clinical Practice Guidelines We Can Trust,² captures current concerns. Despite the efforts of the expert panel that developed the new guideline for management of hypertension, some aspects of the external review process may undermine public confidence.

The NHLBI convened the JNC 8 panel in 2008. The panel decided to adhere to the standards set by the IOM study committee² instead of following the practices of earlier JNC panels. The JNC 7 committee produced a comprehensive overview of the management of hypertension. The committee did not commission systematic reviews of the evidence. The report did not state the extent and quality of the evidence for many topics, so that the logical connection between the evidence and some recommendations was not clear. The report described conflicts of interest for committee members but did not say whether members recused themselves from voting when they had a conflict.³ As part of the guideline development process, the present hypertension guideline panel commissioned a systematic review of randomized trial evidence, evaluated the evidence, made recommendations, indicated the level of evidence supporting those recommendations, reported conflicts of interest, and recused conflicted panel members from voting. Because of these changes, this guideline adheres much more closely to the IOM standards than the JNC 7 guideline.

The external review process began when the panel submitted the draft guideline to the NHLBI. According to James et al,¹ the NHLBI submitted the guideline to 16 federal agencies and to 20 reviewers, all of whom were experts in hypertension. In addition, individual reviewers were expert in cardiology, nephrology, primary care, pharmacology, research (including clinical trials), biostatistics, and other related fields. The authors report that they received responses from 16 individual reviewers and 5 reviewers from federal agencies. The panel revised the guideline, completing its work in June 2013, just as the NHLBI announced that it was turning the guideline development process over to the American Heart Association and the American College of Cardiology, which jointly sponsor a respected guidelines program.⁴ Rather than submit the hypertension guideline for review by these organizations, the panel members submitted the guideline to JAMA, where it underwent both internal and external peer review.

Thus, physicians and other readers are confronted with an important report that, although it has undergone extensive review, has not been evaluated by the specialty societies that the NHLBI designated to take responsibility for the guidelines program. The panel’s departure from usual practice leads to 4 questions. First, what are the key elements of trustworthiness in a guideline? Second, how does this guideline measure up? Third, what is the role of expert review of guidelines? Fourth, what is the pathway to guidelines that the public can trust?

The IOM committee listed 8 standards. This editorial focuses on 4 of them. First, active management of conflict of interest is essential. The IOM committee thought that conflicts should be disclosed to the public and panelists should not vote on questions if they have a conflict. Second, a systematic review should be the starting point for developing a guideline. An IOM committee worked in parallel with the IOM guidelines committee to develop standards for systematic reviews.⁵ A systematic review should minimize the possibility that the guideline panel will fail to take account of relevant evidence. Third, the panel must explain the reasoning that led from the evidence to its recommendation. Fourth, the panel should involve relevant stakeholders in the external review process, publish a draft of the report for public comment, and record reviewer comments and the panel’s response.²

The committee appointed by the NHLBI met the first 3 standards with room to spare but may have fallen short of the IOM standards for external review. The published report includes important details of the review process, such as the reviewing organizations and the expertise of the reviewers. However, these hypertension guidelines, like the recently released guidelines on assessing cardiovascular risk⁶ and management of blood cholesterol,⁷ were not published in draft form to solicit public comment. These concerns notwithstanding, the committee met the first 3 standards so well that it is highly likely that the recommendations in the guideline accurately reflect strong evidence.

External reviewers may point out evidence that the committee missed, identify flawed studies, or cite illogical reasoning, poor writing, or unjustified recommendations. The credibility of a guideline depends on the thoroughness of the external reviewers and the integrity of the panel in responding to concerns. In the case of this hypertension guideline, the reader has a rare opportunity to see the review process in action and judge its integrity. The authors kept detailed records of the comments of the reviewers, the panel’s responses to those comments, and how the panel considered the comments when it revised the guidelines. Remarkably, the authors have said that they will send this record to interested parties on request. This form of transparency sets an important precedent that, if adopted widely, could increase public confidence in guidelines. In this case, the record of verbatim reviewer comments and the panel’s response should convince most readers that little would have been gained by additional review by specialty society–designated reviewers.

Expert review is necessary, but the review process must be managed so that unfair criticism by reviewers is exposed while the committee is held accountable for its response to credible criticism. This approach sounds difficult to achieve in practice, but one organization, the US National Academies, which includes the IOM, uses it routinely for its reports.⁸ According to the approach of this organization, good management of review should include 4 features. First, reviewers should declare all of their financial conflicts of interest. Second, the review process should be managed by an impartial organization. Third, the dialogue between reviewers and guidelines committees should distinguish evidence-based assertions from those based on opinion and personal clinical experience. This principle can help to distinguish a credible criticism from one that is suspect. Fourth, if the organization that manages the review is dissatisfied with the committee’s response to reviewers’ comments, it should be empowered to withhold its endorsement of the review process. The hypertension guideline committee’s willingness to make public the content of the review process suggests a fifth operational principle: publish the dialogue between reviewers and guidelines committees as an online appendix to the guidelines report. This suggestion is in keeping with an overarching principle of guidelines, which is to help the health professions and the public make informed choices, not simply tell them what to do.

In response to the uneven quality of practice guidelines, an IOM study in 2006 proposed creating a marketplace in which guideline users can compare guidelines from different organizations.⁹ If users could identify the best-quality guidelines, they would gravitate toward them, which would compel guideline developers to raise their game or be ignored. With the publication of the GRADE standards¹⁰ and the IOM committee’s quality standards,² this concept is gaining momentum. This approach requires metrics of adherence to the quality standards, which are still in the early stages of development.¹¹

A rigorous, transparent process for developing and reviewing guidelines matters a great deal because guidelines are increasingly driving the practice of medicine. This Editorial focuses on the external review process for the hypertension guideline because it raised some concerns. The panel addressed them head-on by agreeing to share its record of the review process with anyone who asks. Reading the critiques and responses, many readers will conclude that the panel was on solid ground in its interpretation of high-quality evidence about the limited but important set of questions that it chose to address. However, a discussion of external review should include several additional questions. Should guideline users shun a guideline that has not been posted in draft form for public comment per the practice of the US Preventive Services Task Force? Have practice guidelines become so important that they require a review process managed by an impartial third party, like that used by the National Academies? The answers to those questions await public debate. Meanwhile, the panel’s decision to open the review process to public scrutiny challenges other guidelines programs to follow suit.

Corresponding Author: Harold C. Sox, MD, Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine, 31 Faraway Ln, West Lebanon, NH 03784 (hsox@comcast.net).

Published Online: December 18, 2013. doi:10.1001/jama.2013.284429.

Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Sox reports that he was a member of the IOM committees cited in references 8 and 9 and has been a member of the Report Review Committee of the National Academies.

Editorials represent the opinions of the authors and JAMA and not those of the American Medical Association.