Cost Consideration in the Clinical Guidance Documents of Physician Specialty Societies in the United States

Importance Despite increasing concerns regarding the cost of health care, the consideration of costs in the development of clinical guidance documents by physician specialty societies has received little analysis.

Objective To evaluate the approach to consideration of cost in publicly available clinical guidance documents and methodological statements produced between 2008 and 2012 by the 30 largest US physician specialty societies.

Design Qualitative document review.

Main Outcomes and Measures Whether costs are considered in clinical guidance development, mechanism of cost consideration, and the way that cost issues were used in support of specific clinical practice recommendations.

Results Methodological statements for clinical guidance documents indicated that 17 of 30 physician societies (57%) explicitly integrated costs, 4 (13%) implicitly considered costs, 3 (10%) intentionally excluded costs, and 6 (20%) made no mention. Of the 17 societies that explicitly integrated costs, 9 (53%) consistently used a formal system in which the strength of recommendation was influenced in part by costs, whereas 8 (47%) were inconsistent in their approach or failed to mention the exact mechanism for considering costs. Among the 138 specific recommendations in these guidance documents that included cost as part of the rationale, the most common form of recommendation (50 [36%]) encouraged the use of a specific medical service because of equal effectiveness and lower cost.

Conclusions and Relevance Slightly more than half of the largest US physician societies explicitly consider costs in developing their clinical guidance documents; among these, approximately half use an explicit mechanism for integrating costs into the strength of recommendations. Many societies remain vague in their approach. Physician specialty societies should demonstrate greater transparency and rigor in their approach to cost consideration in documents meant to influence care decisions.

Studies have shown that the United States spends far more per capita on health care than any other developed nation without reaping commensurate gains in life expectancy or other health-related outcomes.^1,2 As rising health care costs threaten efforts to improve access to care and place an increasing burden on public and private finances, the need to control costs has become widely acknowledged,^3-6 but the questions of who should take responsibility for controlling costs, and how, remain highly controversial.^7-9 The 2010 health care reform debate highlighted public concern over health care rationing, whether through rumored “death panels” to be created by the government, coverage denials by private health plans, or claims that there would be “bedside rationing” by individual physicians.^10,11 With scrutiny focused on these controversial mechanisms to control costs, the role of another important source of clinical and health system policy has received far less attention: physician specialty societies. Medical and surgical societies differ in size and scope but generally assume many responsibilities and functions, including research, training, certification, and policy advocacy. Many of the larger societies also convene clinical experts in organized programs to review evidence and promulgate clinical guidelines and other forms of guidance for their members.

Despite the important role that physician specialty societies play in the US health care system, the extent to which they have explicitly addressed health care costs in their clinical guidance documents has received little study. To help understand the current landscape and to underscore the question of what the appropriate role of these societies should be, we performed an empirical analysis of the largest US medical and surgical physician specialty societies to determine (1) what percentage of societies explicitly consider costs in developing clinical guidance documents, (2) how costs are considered by those developing guidance documents for societies that explicitly consider costs, and (3) how cost considerations are manifested in formal recommendations in the documents themselves.

We identified all physician specialty societies in the United States that produced clinical guidance documents in the past 5 years by performing an advanced search of the Agency for Healthcare Research Quality National Guideline Clearinghouse database.¹² We selected the checkboxes for the following criteria: (1) Organization Type: Medical Specialty Society, (2) Intended Users: Physicians, and (3) Publication Year: 2008, 2009, 2010, 2011, 2012, resulting in a list of 813 documents published by 128 societies. We excluded societies that are veterinary, dental, or nonclinical; are based outside the United States; operate only on the regional, state, or local level; primarily serve residents, medical students, or allied health professionals; or are based on a specific procedure or diagnostic test.

To limit the societies to a practical number for the purposes of this study and believing that larger societies were more likely to produce clinical guidance documents, we also set an a priori threshold of at least 10 000 members for inclusion in this study. We acknowledge that societies may report total membership numbers that include professionals other than physicians, but we did not adjust for this factor because many societies do not report the proportion of nonphysicians in their census. Applying all exclusion and inclusion criteria resulted in an inventory of 25 societies. We cross-referenced this list with the Federation of Medicine's listing of medical specialty society websites published on the American Medical Association website¹³ and the Yahoo! Directory,¹⁴ and we identified another 5 societies that met our inclusion criteria. This produced a final population of 30 US physician societies reviewed in this study.

We reviewed the methodological statements for clinical guidance development registered in the Agency for Healthcare Research Quality National Guideline Clearinghouse¹² and/or published on a society's official website for each of the 30 societies. We also searched each society's website for separate articles or statements on methodology that outline the process for developing guidance documents. A clinical guidance document was considered to be any form of evidence-based and/or expert consensus document published or otherwise made publicly available with the endorsement of the society. Types of clinical guidance documents included clinical practice guidelines, consensus statements, appropriate use criteria, and quality measures. The consideration of costs in the development of clinical guidance documents was classified as (1) explicit (including a clear statement that costs were integrated in the development of at least 1 of the society's clinical guidance documents); (2) implicit (including a statement that costs were not routinely included but might be considered on occasion); (3) excluded (including a statement that costs were intentionally ignored); or (4) not mentioned (no language indicating the role of cost consideration in developing clinical guidance documents).

For the societies whose general methodological statements indicated that they explicitly integrated costs in developing clinical guidance documents, we examined all of the most recently published clinical guidance documents since January 1, 2008 (with an a priori cap set at 50 documents per society), to determine the exact mechanism used for considering costs in the development of each guidance document. These mechanisms were classified according to whether the document developers (1) used a formal grading system developed by a professional society or international collaboration in which the strength of the recommendation is influenced in part by costs, such as GRADE (Grading of Recommendations Assessment, Development and Evaluation),¹⁵ SORT (Strength of Recommendation Taxonomy),¹⁶ AGREE (Appraisal of Guidelines, Research and Evaluation),¹⁷ or a similar process; (2) reviewed cost data to inform their conclusions but did not use a formal grading system as just described; or (3) made no mention of the exact mechanism of cost consideration in the methodological statements for individual documents.

We searched for the use of GRADE, SORT, AGREE, or similar systems because they represent well-known structured approaches to rate the strength of clinical guideline recommendations. The GRADE system was developed by an international collaborative group that uses a stepwise approach to grading the strength of a clinical recommendation by considering evidence quality, patient preferences and values, and resources.¹⁵ The SORT system was developed by several family practice and primary care journals in the United States.¹⁶ It addresses the quality, quantity, and consistency of evidence and allows authors and reviewers to rate individual studies or bodies of evidence. The taxonomy is built around the information mastery framework, which emphasizes the use of patient-oriented outcomes that measure changes in morbidity or mortality. The AGREE tool is a broader set of criteria for evaluating the quality of guidelines that was developed by an international collaboration centered on European countries.¹⁷ We limited our subanalysis on the mechanism of cost consideration of individual guidance documents to societies whose statements on methodology were classified as explicit.

We also examined each clinical guidance document to identify statements regarding costs as part of the justification for specific recommendations. Based on our reading of all recommendations, we determined 5 categories: (1) recommending use of an intervention because of equal effectiveness and lower cost; (2) recommending use because the incremental benefit justified the extra cost; (3) recommending use to prevent future costs; (4) recommending nonuse because there was no clinical benefit and costs could be avoided; and (5) recommending nonuse because the incremental clinical benefit did not justify the higher costs. All clinical guidance documents were reviewed independently by both of us, and any discrepancies in classification were resolved by consensus.

Of the 30 societies identified (Table 1), 17 (57%) stated in either a document on general guideline development strategy or at least 1 of their specific clinical guidance documents that they explicitly integrated cost considerations. Among the other societies, 6 (20%) did not describe whether or how costs were considered in any documents, 4 (13%) indicated that costs might be considered by guidance developers but gave no further information (implicit cost consideration), and 3 (10%) said that they routinely excluded costs from consideration in any clinical guidance development. Examples of language representing each category are provided in Table 2.

With our maximum of 50 clinical guidance documents per society, we found 279 clinical guidance documents produced within the last 5 years by the 17 societies that explicitly indicated that costs were considered as part of guidance development. Nine (53%) of these societies consistently used a formal grading system in which the strength of recommendation was influenced in part by costs. The remaining 8 (47%) used various methods for incorporating costs or failed to mention the exact mechanism of cost consideration in the individual documents they produced (Table 3).

We also examined the 279 clinical guidance documents to determine how cost was used to justify specific recommendations. Of the 279 guidance documents, 98 (35%) had at least 1 cost-related recommendation, and all 17 societies made at least 1 specific recommendation using cost issues as part of its justification. We identified 138 specific recommendations within the 279 guidance documents in which cost was mentioned. Of these 138 specific statements, 50 (36%) encouraged use of an intervention because of equal effectiveness and lower cost compared with other alternatives, 26 (19%) advised nonuse because there was no clinical benefit and costs could be avoided, 24 (17%) recommended nonuse because the incremental clinical benefit did not justify the higher costs, 23 (17%) favored use because the incremental benefit did justify the extra cost, and 15 (11%) encouraged use to prevent future costs. The breakdown of these specific recommendations by society is shown in Table 4, and examples of the actual language used in each of these categories are given in Table 5.

Our study has several important limitations. Because our review of physician societies focused on those with at least 10 000 members, we cannot comment on the approach to cost considerations in smaller societies. Nor did we assess the overall quality of guidance documents according to standards such as those published by the Institute of Medicine.²⁷

We chose to examine the individual recommendations only in guidance documents produced by societies whose methodological statements on cost consideration were categorized as explicit, and therefore we cannot comment on the possible use of cost in guidance documents produced by other societies. In addition, our study was limited to evaluation of publicly available information, so we may have missed important internal communication between societies and their members in “members only” portions of society websites and in other media not available to us. Our analysis therefore did not include other mechanisms through which societies might consider costs in their efforts to improve the quality and value of health care services, including training programs, continuing medical education offerings, and certification requirements. Although we did not review these other activities, the focus on publicly available clinical guidance documents presents a way to measure how physician societies perceive and publicly present their own role in addressing costs.

Our results suggest that approximately half of the largest physician specialty societies in the United States indicate publicly that their methodological approach to clinical guidance development explicitly integrates cost considerations. A few societies intentionally exclude costs from consideration, whereas several others are ambiguous on this point, failing to explain the exact role of costs or whether costs are considered at all.

On closer examination of the individual clinical guidance documents produced by the societies that explicitly integrated costs in developing these documents, we found that approximately half used a systematic grading system in which cost is a factor that affects the strength of the recommendations. Some of these societies were inconsistent in their approach, using a systematic approach for some, but not all, of their clinical guidance documents. Other societies routinely used less rigorous and transparent mechanisms or did not give any further information clarifying how costs were considered during guidance development.

Should costs be considered in clinical guideline development? Some physician societies recently joined the Choosing Wisely initiative with the goal of identifying “five tests or procedures commonly used in their field, whose necessity should be questioned and discussed” to help reduce unnecessary health care expenditures.²⁸ Societies that participate in this kind of initiative and that produce clinical guidance documents that include formal cost considerations may be able to help reduce costs, promote quality patient care, and participate in self-regulation.^29,30 Opponents of explicit cost consideration, however, believe that physicians should place individual patient needs ahead of societal needs, regardless of cost.³¹ Critics fear that the introduction of costs into clinical decision making at any level will ultimately lead to bedside rationing and cause a rift in the physician-patient relationship that will foster public mistrust of the medical community.^32-34 Moreover, historical patterns of financial incentives for physicians have largely favored higher use, creating potential conflicts for physician societies considering steps to reduce marginally beneficial care.

Consideration of costs may be more relevant for some kinds of clinical guidance (eg, practice guidelines) than for others, but our findings suggest such consideration is already a routine element of guidance development among some physician societies, and we recommend that other societies adopt similar approaches. Some societies may not explicitly state whether costs are considered in developing clinical guidance documents while still using cost to help justify some of their recommendations, but this seems neither good methodological practice nor good public policy. Costs should be considered through an explicit, rigorous process as part of guidance development, focusing not only on short-term costs but on longer-term cost-effectiveness, because some interventions that are more costly in the short term can provide substantial longer-term patient and health system benefits.^35,36

One article published more than a decade ago found a slightly lower percentage (31%) of all clinical guidelines citing at least 1 economic analysis, but the authors did not attempt to determine whether cost considerations were used to justify specific recommendations.³⁷ All 17 physician societies in our study indicating that they considered costs in developing guidance documents made at least 1 recommendation during the past 5 years in which cost was part of the justification; overall, more than one-third of their clinical guidance documents contained at least 1 specific recommendation with a basis in cost considerations.

In our analysis, the most common way to use cost in justifying a recommendation was to state that an intervention was recommended because it was as effective as other options but less costly. This approach has been touted as one of the best ways to reduce waste in the US health care system and is generally considered less contentious than attempts to identify other kinds of waste.^38,39 Nonetheless, 17% of all recommendations that included cost considerations took a very different approach by recommending nonuse of an intervention with an incremental clinical benefit only because the additional benefit was not enough to justify a much higher cost. Although this kind of value judgment is controversial, we recommend that physician societies explicitly pursue this approach and consider in their guidance the relative cost-effectiveness of beneficial medical interventions because the judgment of physician societies is preferable to that of individual physicians “rationing at the bedside.”^36,40,41

In the near future, more clinical care will be delivered by physician-led organizations, such as accountable care organizations that bear responsibility for health care costs as well as clinical outcomes. It thus seems timely for all physician specialty societies to consider costs when developing clinical guidance to help set standards for appropriate care. Our findings suggest the need for greater dialogue and the sharing of best practices within the clinical community and among physician societies on this issue. It is vital that physicians—and the public—understand clearly whether and how costs are considered in authoritative clinical guidance that influences patient care.

Correspondence: Steven D. Pearson, MD, MSc, National Institutes of Health, 10 Center Dr, Bldg 10, Room 1C118, Bethesda, MD 20892-1156 (pearsonsd@cc.nih.gov).

Accepted for Publication: January 10, 2013.

Published Online: May 6, 2013. doi:10.1001/jamainternmed.2013.817

Author Contributions:Study concept and design: All authors. Acquisition of data: Schwartz. Analysis and interpretation of data: All authors. Drafting of the manuscript: All authors. Critical revision of the manuscript for important intellectual content: All authors. Statistical analysis: Schwartz. Administrative, technical, and material support: Schwartz. Study supervision: Pearson.

Disclaimer: The opinions expressed are those of the authors and do not reflect the position or policy of the National Institutes of Health, the Public Health Service, or the Department of Health and Human Services.