To assess improvement in documentation of asthma indicators using the Asthma Toolbox, an asthma decision-making tool developed in accord with National Asthma Education and Prevention Program guidelines.
Retrospective medical record review using cross-sectional, independent, random samples. Reviews were conducted for 1-year periods before and after implementation and after revision reflecting 2007 guideline modifications.
Two inner-city, federally qualified health center programs providing pediatric primary care to housed and homeless populations.
A total of 1246 patients aged 6 months to 18 years with at least 1 asthma visit to a community health center using paper records (n = 600) or a mobile medical program serving family homeless shelters using an electronic health record (EHR; n = 646).
Implementation of the Asthma Toolbox incorporated into paper encounter forms and embedded in the EHR to guide providers (ie, physicians and nurse practitioners) through pediatric asthma assessment and management.
Main Outcome Measures
Documentation of a subset of asthma severity/control measures, emergency department visits, hospitalizations, and percentage of persistent asthmatic patients prescribed controller medications.
Documentation of each asthma indicator increased significantly after implementation (χ2 tests; P < .001 all comparisons) for both programs. Documentation of severity/control increased from 25.5% to 77.5% in paper records and from 11.7% to 85.1% in the EHR (P < .001). Increases were sustained after Asthma Toolbox revision for all indicators. The percentage of patients with persistent/uncontrolled asthma prescribed controller medications reached 96% to 97% in both programs.
Use of the Asthma Toolbox, an asthma decision-making tool, significantly increased documentation of pediatric asthma management among providers working in high-disparity, urban primary care settings.
National Asthma Education and Prevention Program (NAEPP) Guidelines for the Diagnosis and Management of Asthma aim to improve clinical outcomes by bridging the gap between research and clinical practice.1 Yet, the morbidity and mortality associated with childhood asthma remain substantial, and racial/ethnic and socioeconomic disparities in asthma outcomes persist.2- 6 Achieving improvement in asthma care and closing disparities in outcomes require translating complex guidelines into primary care practice, where most asthmatic children receive care. It has been suggested that the length, complexity, and changing nature of the guidelines have hindered their adoption, contributing to persistence of poor asthma outcomes.7
Several studies8- 14 have found poor adherence to NAEPP guidelines in primary care settings. Barriers include lack of familiarity or agreement with guidelines, lack of self-efficacy with assessment and treatment recommendations, lack of time, and poor outcome expectancy.14- 16 Although provider education alone has not been shown to consistently improve asthma guideline adherence,17,18 a systems-based approach combining provider training with standardized assessment tools has demonstrated improved adherence to asthma guidelines and improved clinical outcomes.8- 10,19- 23
As migration from paper to electronic records becomes universal, attention to documentation and collection of clinical data will be needed to comply with Centers for Medicare and Medicaid Services criteria for meaningful use. Clinical quality measures for meaningful use, released as part of an electronic health record (EHR) incentive program, include 3 that focus on documentation of guideline-based care of asthmatic patients.24 In addition, as primary care practices seek recognition as patient-centered medical homes, documentation of asthma care can be used to satisfy elements proving best practice–based care.
To facilitate provider adherence and optimize care, we developed the Asthma Toolbox, a decision-making tool initially based on the 2002 NAEPP guideline update. The Asthma Toolbox is incorporated into paper and electronic records to efficiently guide providers through assessment, monitoring, and treatment of pediatric asthma during primary care visits. The objective of this study was to assess change in documentation of key asthma indicators after implementation in 2 New York programs: a community health center in the South Bronx and a mobile medical program serving homeless families and youth throughout New York City.
We hypothesized that implementation of the Asthma Toolbox would (1) increase documentation of asthma severity classification, emergency department (ED) visits, and hospitalizations and (2) increase the percentage of patients with persistent or uncontrolled asthma prescribed controller medications. We further hypothesized that improvement would be sustained after revision of the tool to reflect 2007 NAEPP guidelines.
Community Pediatric Programs, a partnership of the Children's Hospital at Montefiore and the Children's Health Fund, provides comprehensive primary care to approximately 6000 pediatric patients (0-19 years old) annually through 2 federally qualified health center programs. South Bronx Health Center (SBHC) is a community health center that serves residents in the poorest congressional district in the country. Of patients seen at SBHC in 2009, 74% were living at or below the federal poverty level; 68% were Hispanic and 26% were black. Among pediatric patients, 67% received Medicaid or other public insurance and 17% were uninsured. Paper medical records were used at SBHC.
New York Children's Health Project (NYCHP) provides health care to homeless children and families via mobile medical and on-site clinics at family shelters, domestic violence shelters, and a drop-in center/shelter for homeless youth. Almost all the patients (96%) seen by NYCHP in 2009 were living at or below the federal poverty level; 42% were Hispanic and 42% were black. Among pediatric patients, 76% received Medicaid and 24% were uninsured. The NYCHP has been using EHRs since 1998.
Residents of public housing,25 the family shelter system,26 and the South Bronx27 have among the highest pediatric asthma rates in New York City. Annually, approximately 30% of pediatric patients at SBHC and 20% at NYCHP have a current diagnosis of asthma compared with a 9% prevalence in children (0-17 years old) in New York City27 and nationally.28
The Asthma Toolbox is a boxed set of questions printed on every encounter form and programmed into the EHR as a drop-down checklist. It provides a concise visual reminder for providers to address asthma at each visit and leads them through standardized assessment, classification, and management. The asthma indicators included in the tool were chosen to capture essential elements required to categorize severity/control and to gauge the impact of asthma on the child's life while recognizing time constraints in busy primary care settings (Figure 1). Other aspects of asthma management (eg, education, self-management plans, and trigger assessment) are documented elsewhere in the paper and electronic records. The NYCHP upgraded its EHR during the study. In the first EHR, providers who did not voluntarily use the Asthma Toolbox were shown a prompt, regardless of diagnosis. Providers could bypass this prompt if not applicable or if asthma was not addressed. The newer EHR did not have this prompt function, and use was strictly voluntary.
Asthma Toolbox incorporated into age- and visit-specific clinical encounter forms in the paper medical record. The version for well-child care visits for children younger than 4 years is shown.
The Asthma Toolbox was based on the 2002 NAEPP guideline update for severity classification and optimal asthma management and was revised according to 2007 NAEPP guidelines that included assessment of control status, evaluation of risk by frequency of exacerbations requiring oral corticosteroid use, and classification algorithms for severity and control according to age group (0-4, 5-11, and ≥12 years). We, therefore, developed age-specific versions of the Asthma Toolbox to be used at well-child care (WCC) visits and a version for walk-in/follow-up visits. Physicians are trained to use all the criteria (daytime and nighttime symptoms, risk, and exercise and activity impairment) in their assessment of patients' severity or control classification.
The Asthma Toolbox was developed by clinical champions with feedback from primary care providers in our practices. Physicians received group trainings on the 2002 and 2007 NAEPP guidelines and the use of each Asthma Toolbox version before implementation. During the study, 25 clinicians (21 physicians and 4 nurse practitioners) provided care to pediatric asthmatic patients.
This study was a retrospective medical record review using cross-sectional independent samples. Medical record reviews were conducted for 3 measurement periods at each site: 1 year before initial implementation (pre-Toolbox), 1 year after initial implementation (post-1), and 1 year after revision per 2007 NAEPP guidelines (post-2). Owing to phased implementation, measurement periods were as follows: for SBHC, November 1, 2004, to October 31, 2005; December 1, 2005, to November 30, 2006; and May 1, 2008, to April 30, 2009; and for NYCHP, April 1, 2005, to March 31, 2006; May 1, 2006, to April 30, 2007; and June 1, 2008, to May 31, 2009. This study was approved by the institutional review board of Montefiore Medical Center.
Eligible patients were aged 6 months to 18 years, with at least 1 clinical visit coded for asthma (International Classification of Diseases, Ninth Revision, codes 493.xx). Patients were excluded if found not to have a diagnosis of asthma or if visits were incorrectly coded for asthma. The population of eligible patients was determined using Clinical Looking Glass, a software application that integrates systemwide clinical and administrative data sets. Separate queries were performed by site and measurement period. If seen in multiple measurement periods, selected patients were removed from the subsequent universe to ensure independent samples. To explore patterns of provider documentation at WCC, acute, and nonacute visits, 200 patients were randomly selected for each measurement period at both sites (assuming approximate equal distribution of visit types based on previous medical record reviews).
We conducted medical record reviews using a standardized data extraction tool and coding sheet that operationalized the variables collected. A pilot study of 10 randomly selected medical records from the pre-Toolbox period indicated interrater reliability of 0.947, assessed by intraclass correlation using a 2-way random-effects model. All the visits coded for asthma were reviewed to determine whether variables were documented at any time during a measurement period (patient-level analysis) and at the last visit in a measurement period (visit-level analysis). The following variables were recorded as dichotomous (yes/no): severity documented (defined as intermittent, mild persistent, moderate persistent, or severe persistent) or control documented during post-2 (defined as well controlled, not well controlled, or very poorly controlled), ED visits for asthma documented, hospitalization for asthma documented, asthma classified as persistent or uncontrolled (not well/poorly controlled), and patient prescribed controller medication (defined as an inhaled corticosteroid or montelukast sodium).
Assessment outcomes included the proportion of patients with documentation of severity and/or control, ED visits, hospitalization, and all 3 of these indicators. The treatment outcome was prescription of a controller medication for patients with persistent or uncontrolled asthma at the visit when severity and/or control were assessed.
Independent variables included age, sex, type of visit, and number of asthma visits. Visits were categorized as WCC (International Classification of Diseases, Ninth Revision, codes v20.2 and v70.0), acute asthma, or nonacute asthma. Acute visits were defined by documentation of asthma symptoms within 1 week, positive findings on lung examination, nebulizer treatment given during the visit, prescription of oral corticosteroids, or diagnosis of asthma exacerbation. Nonacute visits included asthma follow-up and visits for primary concerns other than asthma in which asthma was addressed.
Means and frequencies are used to describe the samples. χ2 Tests were conducted for categorical variables and 1-way analysis of variance for continuous variables to compare characteristics of samples by site. All analyses compared pre-Toolbox vs post-1 and post-1 vs post-2 to assess change in documentation after initial implementation and whether the change was sustained after revision. χ2 Tests (2-sided) were used to compare the percentage of patients with dependent variables documented at least once during a measurement period and at their last visit in a measurement period. Significance (type I error) was set at α = .05. Binary logistic regression was performed to examine the effect of the Asthma Toolbox on documentation, adjusting for age, sex, type of visit, and total number of asthma visits in the measurement period. All the analyses were conducted using a commercially available software program (SPSS for Windows, version 15; SPSS Inc, Chicago, Illinois).
For SBHC, 600 of 1962 eligible patients' medical records were included in the analysis (200 for each period). For NYCHP, 646 of 772 eligible patients were included; all the medical records were included for the pre-Toolbox (n = 197) and post-1 (n = 249) periods due to small numbers of patients; a random sample of 200 patients was reviewed at post-2 per protocol. Generally, patient characteristics were similar across measurement periods for both programs (Table).
At both study sites, the proportion of patients with documentation of asthma severity and/or control, ED visits, and hospitalization at least once during the measurement period increased significantly from pre-Toolbox to post-1 (P < .001 all comparisons) (Figure 2). Documentation of severity and/or control increased from 25.5% to 77.5% at SBHC (paper record) and from 11.7% to 85.1% at NYCHP (EHR). The proportion of patients with all 3 assessment indicators documented increased from 6.5% to 76.0% at SBHC and from 5.6% to 82.3% at NYCHP. Although rates of documentation were not compared statistically between sites, the results were similar. In post-1, documentation increased to 78% to 88% at SBHC and to 85% to 90% at NYCHP. There were no differences comparing post-1 with post-2 except for a significant increase in documentation of severity and/or control at SBHC (P = .03); approximately 86% of patients at both sites were assessed for asthma severity and/or control during post-2.
Proportion of patients with documentation at any visit during each measurement period at South Bronx Health Center (A) and New York Children's Health Project (B). All indicates severity, emergency department (ED) visits, and hospitalization combined. * P < .001. † P < .05. χ2 Test pre-Toolbox vs post-1 (1 year before the initial implementation vs 1 year after the initial implementation), and post-1 vs post-2 (1 year after revision per 2007 National Asthma Education and Prevention Program guidelines).
To gauge provider use of the Asthma Toolbox at the visit level, documentation at patients' last visit during each measurement period was examined and yielded similar results (Figure 3). At both sites, documentation of each assessment indicator increased significantly in post-1 vs pre-Toolbox (P < .001 for all), with no significant decreases in post-2. At their last visit in post-2, 62.0% of patients at SBHC and 71.5% at NYCHP had all 3 assessment indicators documented. The last visit consisted of all types, with significant differences in their distribution across measurement periods only at SBHC (Table).
Proportion of patients with documentation at their last visit during each measurement period at South Bronx Health Center (A) and New York Children's Health Project (B). All indicates severity, emergency department (ED) visits, and hospitalization combined. * P < .001. χ2 Test pre-Toolbox vs post-1 (1 year before the initial implementation vs 1 year after the initial implementation), and post-1 vs post-2 (1 year after revision per 2007 National Asthma Education and Prevention Program guidelines).
In binary logistic regression models of assessment indicators documented at any time and at the last visit during the measurement period, the Asthma Toolbox at post-1 and post-2 remained a significant predictor of documentation (P < .001), adjusted for age, sex, type of visit, and total number of asthma visits. Having a WCC visit also was an independent predictor of documentation for assessment indicators (P < .001). Results were consistent for both sites.
Owing to the low rate of severity classification pre-Toolbox, the proportion of patients with persistent asthma prescribed controller medication could not be reliably determined. In post-1, 95.7% of patients (66 of 69) with persistent asthma were prescribed controller medications at SBHC and 81.3% (61 of 75) at NYCHP. In post-2, the proportion of patients with persistent or uncontrolled asthma prescribed controller medications was sustained at SBHC at 96.2% (50 of 52 patients) and rose significantly to 97.3% (107 of 110 patients) at NYCHP (P < .001 vs post-1).
Incorporating a concise asthma decision-making tool into paper records and EHRs consistently improved documentation of key asthma indicators during pediatric primary care visits. Results were sustained at both programs after the Asthma Toolbox was revised to reflect the more complex 2007 NAEPP guidelines. In addition, changing EHRs did not affect results, despite the fact that use of the tool was prompted in the first EHR but not in the second. Results were consistent across the 2 clinical settings with different populations, delivery systems, providers, and methods of recording medical information. These findings are salient because the study was conducted in the context of disadvantaged patient populations in challenging clinical settings.
To further assess use, we examined provider behavior at a single visit (ie, last visit in each measurement period). In post-1, 60% of patients at SBHC and 75% at NYCHP had documentation of all asthma indicators at their last visit, which was sustained after revision of the tool. The somewhat higher rates for the homeless program may result from use of an EHR or efforts to provide the most comprehensive care possible at each visit for a transient population. In regression analyses, the presence of the Asthma Toolbox was a significant predictor of documentation regardless of visit type. Having a WCC visit also was a significant independent predictor of documentation, suggesting that complete asthma assessments are most routinely performed in that context.
Properly classifying children's asthma severity or control status is the key step for determining the need for controller medication. The use of inhaled corticosteroids has been shown to be the most effective therapy for reducing asthma severity and morbidity.1 In this study, the percentage of children with persistent or uncontrolled asthma prescribed a controller medication increased from 81% in post-1 to 97% in post-2 (P < .01) at NYCHP and was 96% at SBHC in both postimplementation periods. The significant increase in controller medication prescriptions in patients with persistent or uncontrolled asthma at NYCHP may be attributed to ongoing training or improved documentation of medications in the later EHR. Consistent with other studies conducted in primary care settings,8,11 prescribing practices could not be reliably assessed pre-Toolbox because few patients had severity classified. The Asthma Toolbox increased documentation of severity and/or control to 86% in both programs, enabling assessment of appropriate prescribing in most patients.
Prescribing patterns after Asthma Toolbox implementation are encouraging; however, a study19 of guideline use by primary care providers in a similar population of poor, minority, urban children found that prescribing a corticosteroid was insufficient to reduce medical service use. A reminder tool should be a component of comprehensive asthma care that includes education, asthma action plans, trigger/environmental screens, allergy testing, spirometry, and potential referral to asthma management programs. Children are referred for the previously mentioned interventions within our primary care settings.
Studies have shown that guideline-based asthma decision-making tools should be concise,21 readily available to the provider during the encounter,29 and focused on diagnosis and therapy.19 To our knowledge, 1 other study23 evaluated the effectiveness of an asthma decision-making tool incorporated into the medical record in a pediatric primary care setting, demonstrating small but significant increases in prescription of controller medications and spirometry. Another study30 found significant increases in asthma classification (from 24% to 44%) and appropriate prescription of controller medications (from 37% to 71%) after training medical residents to use the asthma template in an EHR.
Typically, studies10,19,20,31 have evaluated assessment and decision-making tools administered in the context of asthma management programs, therefore not reaching all children seen in primary care. Furthermore, the contribution of various tools to the overall effect of multifaceted interventions was not determined. The Asthma Toolbox was designed to make guideline-based asthma care accessible to all asthmatic children as a routine part of the clinical encounter. In addition to provider training, incorporating a reminder tool on every paper encounter form or as a template in the EHR, rather than as a separate tool, likely accounts for its effectiveness.
This study has several limitations. Documentation of selected asthma indicators was used as a proxy measure of delivery of guideline-based care. We did not assess other aspects of care captured in the Asthma Toolbox or whether symptoms, ED utilization, hospitalization, or school absenteeism were reduced. The cross-sectional design allowed us to examine processes but not outcomes of care. Incorporating other elements of the guidelines into the tool may affect use and outcomes; however, this was beyond the scope of this study. This study has several strengths. Looking at all visit types, the results reflect provider behavior in the typical setting of primary care. Although the small samples in this study limit generalizability, consistent results across substantially different settings suggest utility to other pediatric primary care practices.
In conclusion, this study shows that complex guideline-based care can be adopted into pediatric primary care practice when translated into a concise tool embedded in the medical record. Moreover, the Asthma Toolbox demonstrates adaptability, an important feature that facilitates evidence-based care in the face of changing guidelines and technologies. In 2004, the Institute of Medicine identified asthma as a national priority area for quality improvement and recommended developing data collection systems to assess effectiveness of such efforts.32 The Asthma Toolbox facilitated quality improvement activities by serving as a data collection tool and a decision-making tool. Standardized asthma quality measures are based on the assumption that symptoms and severity are routinely documented,33,34 which has been shown not to be the case.8,10,11 Effective quality improvement efforts must first address process outcomes (eg, documentation of severity) to evaluate clinical outcomes. The effectiveness of the Asthma Toolbox is 1 step toward providing quality, evidence-based care to underserved populations. Further study is needed to determine whether the use of decision-making tools, incorporated into routine pediatric care, improves asthma outcomes.
Correspondence: Alan Shapiro, MD, Senior Medical Director, Community Pediatric Programs, Montefiore Medical Center, 853 Longwood Ave, Bronx, NY 10459 (firstname.lastname@example.org).
Accepted for Publication: January 5, 2011.
Author Contributions: Dr Shapiro had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Shapiro, Gracy, Quinones, Applebaum, and Sarmiento. Acquisition of data: Shapiro, Gracy, Quinones, Applebaum, and Sarmiento. Analysis and interpretation of data: Shapiro, Gracy, and Applebaum. Drafting of the manuscript: Shapiro, Applebaum, and Sarmiento. Critical revision of the manuscript for important intellectual content: Shapiro, Gracy, Quinones, and Applebaum. Statistical analysis: Applebaum and Sarmiento. Administrative, technical, and material support: Gracy, Quinones, and Applebaum. Study supervision: Shapiro.
Financial Disclosure: None reported.
Funding/Support: This study was supported by the Children's Health Fund.
Role of the Sponsor: The Children's Health Fund had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.
Shapiro A, Gracy D, Quinones W, Applebaum J, Sarmiento A. Putting Guidelines Into PracticeImproving Documentation of Pediatric Asthma Management Using a Decision-Making Tool. Arch Pediatr Adolesc Med. 2011;165(5):412-418. doi:10.1001/archpediatrics.2011.49