The Anatomy of a US Preventive Services Task Force Recommendation: Lipid Screening for Children and Adolescents

Differing methods for guideline development result in conflicting recommendations and clinical practice variation. This article details the approach used by the US Preventive Services Task Force to issue recommendation statements, using the 2007 recommendation for screening of lipid levels as an example. An analytic framework served as the source of key questions for a systematic review of the evidence on lipid screening in children and adolescents. Evidence was insufficient, of poor quality, or conflicting to answer 7 of the 10 questions. There was no direct evidence of the benefit of lipid screening, and insufficient evidence existed in the indirect chain of evidence to support a recommendation. In 2008, the American Academy of Pediatrics issued a clinical policy statement recommending screening for targeted children. We discuss the contrasting approach to the development of this guideline. The use of a standardized method to develop clinical guidelines promotes trust and credibility among patients and clinicians.

Recent moves to promote research on the comparative effectiveness of preventive, diagnostic, and therapeutic interventions for patients will ultimately lead to further evidence synthesis and practice recommendations. Clinicians receive practice recommendations and guidelines from many different types of organizations, including federal bodies, professional practice organizations, health plans, and hospitals. However, clinicians frequently receive conflicting recommendations regarding the best approach to care. For example, clinicians caring for adults were recently informed about a modification to the breast cancer screening recommendation by the US Preventive Services Task Force (USPSTF),¹ provoking considerable controversy among different professional and advocacy groups in the medical community. This controversy underscored some of the underlying differences in methods and grading that exist between organizations that produce guidelines for physicians and other clinicians.

Such controversies and conflicts in recommendation statements also exist in children's health care. One example concerns whether some children or adolescents should receive routine screening of lipid levels as part of preventive care. This question is currently facing many pediatricians, family physicians, and nurse practitioners in the United States after the American Academy of Pediatrics (AAP) Committee on Nutrition published a Clinical Report regarding pediatric screening of lipid levels and cardiovascular health.² This report recommended screening of lipid levels for children aged 2 to 10 years with risk factors for cardiovascular disease (eg, overweight, hypertension) and those with a positive family history of premature cardiovascular disease or dyslipidemia. Immediately after its publication, the popular media reported controversy among physicians regarding the recommendations contained in this statement.³ Around the same time, the USPSTF reached a different conclusion, stating that the evidence was insufficient to recommend routine screening for children and adolescents.

The USPSTF is an independent panel of experts in primary care and prevention that systematically reviews the evidence of effectiveness and develops recommendations for clinical preventive services. The USPSTF membership is composed of primary care physicians, nurse practitioners, epidemiologists, and behavioral scientists and is funded, staffed, and appointed by the US Department of Health and Human Services' Agency for Healthcare Research Quality (AHRQ). As USPSTF members, we wish to highlight major differences between the clinical recommendations in the 2008 AAP-sponsored report and the 2007 recommendations published by the USPSTF^4,5 and the differences in methods and processes that were used to generate the 2 sets of recommendations. Clinicians who depend on clinical guidelines for their practice may find it helpful to understand these differences as they decide which clinical guideline recommendations to incorporate into practice.

In its July 2008 report, the AAP Committee on Nutrition recommended “targeted cholesterol screening” of high-risk children and adolescents “with a positive family history of dyslipidemia or premature (<55 years of age for men and <65 years of age for women) cardiovascular disease or dyslipidemia.”^2(p205) In addition to this group, fasting lipid profile screening was also recommended for children “for whom family history is not known or those with CVD [cardiovascular disease] risk factors, such as overweight, obesity, hypertension, cigarette smoking, or diabetes mellitus, be screened with a fasting lipid profile.”^2(p205) Recommended therapies for children and adolescents who have dyslipidemia range from lifestyle modification to pharmacologic treatment with medications, such as statins (hydroxymethyl glutaryl coenzyme A reductase inhibitors). These recommendations updated previous AAP recommendations that were more limited in scope, one of the major changes being the addition of overweight and obese children and teenagers to the targeted groups for screening.^6,7

The recommendations of the AAP Committee on Nutrition focus on interventions in the following 2 domains: (1) whether to encourage clinicians to promote exercise and healthy patterns of eating among high-risk children and (2) whether to screen using a lipid panel. We focus only on the controversies associated with the question concerning lipid screening and not other parts of the policy statement.

Using the same available evidence base as the AAP Committee on Nutrition, the USPSTF concluded that there was insufficient evidence to recommend for or against routine screening of lipids for any group of children and teenagers. The USPSTF found that the net balance of potential benefits and harms of screening was too uncertain to warrant a recommendation for routine screening in this population.⁴

The differences between the 2 recommendations on lipid screening by these well-respected organizations provide an opportunity to compare and contrast the methods used by the USPSTF and the AAP Committee on Nutrition.

The development of the USPSTF recommendations is anchored on the following set of key principles:

• Use of a reproducible evidence-based method. All USPSTF recommendations are based on full or targeted systematic evidence reviews using clear and explicit methods, including an analytic framework, key questions, and a standard approach to grading evidence and recommendation statements.^8,9

• Scientific peer review of the methods, evidence summaries, and recommendations.¹⁰

• Recommendations that hinge on net health benefit to people, with consideration of potential harms and benefits of screening and therapeutic interventions. Where insufficient data exist to estimate the magnitude of benefit or harm, recommendations are not made.

• Rigorous conflict-of-interest policy. Potential conflicts of interest among USPSTF members are routinely screened and reviewed by the scientific director and USPSTF chairs for each topic review.

These principles drive a sequential standardized process composed of these main steps (Table 1): (1) key question formulation; (2) evidence collection, assessment, and synthesis (including meta-analysis, if appropriate); (3) recommendation development^11-13; and (4) external peer and public review. The current USPSTF method documentation for clinical recommendations is summarized in a continuously updated procedure manual available to the public on its Web site.¹⁴

This process of evidence-based guideline development closely parallels that of the traditional scientific inquiry, in which key study questions are identified, and a transparent method is used to drive the data collection process and analysis of retrieved literature. In the case of guideline development, the data collection process is a comprehensive, systematic search of the scientific literature, with clear decision rules designed to capture the highest possible proportion of relevant information.

The USPSTF developed its recommendations for lipid screening on the basis of a specific analytic framework used to narrow the evidence review to the key questions that would have the greatest leverage on a decision to recommend, not recommend, or abstain from recommending lipid screening. Ten possible questions (Figure and Table 2) were derived for the analytic framework, all of which can be reviewed in the USPSTF evidence synthesis.¹⁵ The searches for relevant studies were based on these questions. Evidence from key question 1 would provide direct evidence of the effectiveness of lipid screening among children by identifying controlled trials of screening compared with no screening, with information on long-term health outcomes, reaching into adulthood. Because no literature of this type was found in this review, the USPSTF focused on the remaining key questions (2-10) in an effort to assemble a chain of indirect evidence, using the sequential steps of the screening process. An example of an indirect chain of evidence to support screening is one that shows that a screening test accurately identifies persons with or at high risk for the target condition, that effective treatments are available to treat the target condition, and that harm due to screening and treatment is outweighed by the benefits. For adult obesity screening, the USPSTF recommendations were based on indirect evidence of health benefits of screening using body mass index as an outcome associated with important clinical outcomes. To achieve this threshold of benefit, evidence was independently assembled to assess the accuracy and reliability of the screening test, the effectiveness of interventions on weight loss, the effectiveness of weight loss on intermediate outcomes (eg, blood pressure and lipid levels), and the effectiveness of weight loss on clinical outcomes (eg, mortality, daily functioning, prevention of chronic disease).¹⁶ Gaps in any sequence of the evidence chain because of insufficient or poor-quality studies affect the level of certainty regarding the balance of benefit and/or harm associated with the screening process.

As part of its recommendation process, the USPSTF publishes the complete literature synthesis or a targeted update of a previous synthesis. Literature syntheses are performed by researchers from the AHRQ (the federal sponsor of the USPSTF) or one of the academic evidence-based practice centers under contract to the AHRQ. Evidence grading of individual studies, using a standard explicit criteria appropriate to study design (available at: http://www.ahrq.gov/clinic/uspstf08/methods/procmanualap7.htm), is performed by the group performing the synthesis and summarized in the published evidence tables.

The USPSTF found important gaps in the scientific literature concerning many of these key questions regarding pediatric lipid level screening (see Table 2 for itemization of key questions and evidence conclusions). Of the 10 key questions, 7 contained evidence that was insufficient, of poor quality, or conflicting regarding the answers. No studies directly evaluated benefits of screening compared with no screening. Regarding the indirect evidence on screening, evidence was most abundant for the question that focused on the effect of statin medications on lipid levels. However, lipid levels are an intermediate (nonclinical) outcome. There was insufficient evidence to determine the benefits of screening on clinical outcomes (eg, future cardiovascular events) or risk of adult dyslipidemia, or to accurately estimate long-term harms of treatments that might be initiated as a result of screening (eg, effects of long-term therapy with statins). In addition, the trials were conducted among children with dyslipidemia syndromes and may not be applicable to children with multifactorial dyslipidemia, as seen with obesity. Although evidence was available to address certain questions about lipid screening in children and teenagers, a complete chain of indirect evidence could not be assembled that would support a recommendation for screening among average or high-risk children and teenagers.

The USPSTF voted to rate pediatric lipid screening with an “I” statement (Insufficient evidence to recommend for or against screening), concluding that the current evidence was insufficient to assess the balance of benefits and harms of the service. Because the balance of benefits and harms could not be determined, the certainty of net benefit was low (Table 3).¹⁷ The “I” statement for this recommendation recognizes that clinicians take into consideration the individual child and any comorbidity that exists and discuss the evidence gaps and the pros and cons of screening with patients and their families. Using a shared decision-making approach helps to facilitate these conversations. The USPSTF has prepared aids for clinicians in the interpretation of an “I” statement (available at: http://www.ahrq.gov/clinic/ivideos.htm) and emphasizes that recommendations with an “I” statement offer an enormous opportunity for researchers interested in key questions salient to prevention in primary care.

The use of an explicit analytic framework and set of key questions is only 1 part of the process to improve the salience, transparency, and validity of the guideline development process. All comprehensive literature reviews are sent to USPSTF members and external peer reviewers with special expertise in the scientific area under review. The AHRQ and the evidence-based practice center are keenly interested in ensuring that the literature synthesis reflects the current state of the art and that important studies that have been or are about to be published are not missing. Based on the systematic review of the literature and evidence synthesis, these pediatric lipid recommendations were jointly developed by the USPSTF members, representatives from the evidence-based practice center, and AHRQ medical officers.

Potential conflicts of interests, including financial, intellectual (eg, patent holder), and academic (eg, grant funding) conflicts, are surfaced through a standard review process conducted jointly by the USPSTF's chair and scientific officer; all significant financial, intellectual, and professional conflicts (the disclosure form is available at: http://www.ahrq.gov/clinic/uspstf08/methods/procmanualap2.htm) are shared with all other USPSTF members and USPSTF partner organizations, and a determination is made regarding whether participation in the discussion and/or voting is permitted. In the case of this specific recommendation, 1 USPSTF member was excused from discussion and voting because of potential conflicts of interest. Drafts of the recommendation were presented to the full USPSTF for review, modification, and finalization of a recommendation grade by vote. The USPSTF strives for a supermajority consensus in finalizing its recommendations. The USPSTF did not consider cost-effectiveness in making this recommendation, although the comparative efficiency of strategies may be modeled for other USPSTF recommendations. All drafts of recommendations and statements undergo external peer review by partner organizations of the USPSTF. Recently, a new process was implemented to gather public feedback to draft recommendation statements before publication in a peer-reviewed journal.

The AAP Committee on Nutrition clinical report containing the lipid screening recommendations was an update of previous reports published in Pediatrics and based on the original findings from the National Cholesterol Education Program in 1992, updated in 1998.⁷ The original recommendations by the National Cholesterol Education Program were based on those of an expert consensus panel, in which scientific and clinician experts participated in a commonly used process to arrive at clinical guidelines. As with many expert panels, the methods used to identify and analyze scientific evidence were not explicitly defined in the report. In its subsequent updated clinical reports, the Committee on Nutrition did not reference or outline a clear method for its updated literature review. The process for literature appraisal or grading the quality or sufficiency of evidence was not specified. Other processes, including vetting potential conflicts of interests and external peer reviews, were not reported.

The process used by the Committee on Nutrition to arrive at its recommendation appears to have followed the pathway of a clinical report rather than a guideline, which by AAP standards does not require an explicit review. However, such documents are viewed as policy statements from the AAP and perceived by patients and clinicians as guidelines—suggesting that similarly rigorous methods should be applied. The AAP guideline regarding the diagnosis and management of otitis media followed the specifications of a clinical practice guideline and is an example of a multidisciplinary, explicit, evidence-driven process that is based on the key principles of guideline development.¹⁸ The AAP has also developed a highly organized and thoughtful set of recommendations regarding the formulation of clinical practice guidelines; many of the principles and methods are an integral part of these recommendations.¹⁹

For clinicians to have confidence in the use of guidelines issued by professional organizations, they must know about the quality of evidence and the process used to develop the recommendations. The process should be transparent and rigorous and include safeguards from conflicts of interest. Many professional medical groups have adopted a highly standardized approach to clinical practice guideline development and clearly indicate the strength of evidence behind recommendations. It is possible that the AAP would have reached a different conclusion if the Committee on Nutrition followed the approach used to develop an AAP-endorsed clinical practice guideline. Because this AAP clinical recommendation appears to have been based on less systematic and transparent methods, users may not know whether the recommendation is consistent with the evidence, whether guideline developers factored in important research gaps when developing the recommendation, or how potential conflicts of interest may have been managed.

Other recent reports regarding the impact of process variation in clinical guideline development outcomes (eg, colorectal cancer screening in adults) warrant attention by practicing clinicians who depend on such recommendations to standardize and improve their practice of medicine.²⁰ Primary care clinicians ultimately depend on the organizations who sponsor these guidelines to use evidenced-based and standardized methods to produce them. Their patients and patients' families deserve nothing less.

Correspondence: David C. Grossman, MD, MPH, Group Health Research Institute, 1730 Minor Ave, Ste 1600, Seattle, WA 98101 (grossman.d@ghc.org).

Accepted for Publication: September 9, 2010.

Author Contributions:Study concept and design: Grossman, Moyer, Melnyk, and DeWitt. Analysis and interpretation of data: Grossman, Chou, and DeWitt. Drafting of the manuscript: Grossman, Moyer, Melnyk, and Chou. Critical revision of the manuscript for important intellectual content: Grossman, Moyer, Chou, and DeWitt. Administrative, technical, and material support: Grossman. Study supervision: Grossman.

Financial Disclosure: Dr Chou is affiliated with the Oregon evidence-based practice center that provides evidence reviews under contract to the USPSTF.

Additional Information: Drs Grossman and Melnyk are current members of the USPSTF and participate in the USPSTF Child Health Workgroup. Drs Moyer and DeWitt were USPSTF members at the time the lipid statement was finalized.

Additional Contributions: Robert Reid, MD, PhD, Eric Larson, MD, MPH, Mary Barton, MD, MPP, Iris Mabry-Hernandez, MD, MPH, Joan DeClaire, BA, and Rebecca Hughes, BA, provided comments on earlier drafts of this article.