[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.163.129.96. Please contact the publisher to request reinstatement.
[Skip to Content Landing]
Download PDF
Figure.
Flow of practice clusters, participating internists, and patients in the practice improvement module intervention trial. Eligible patients for assessing outcomes were those who were seen for asthma in the postintervention period by participating internists. Values for eligible patients and responders are expressed as mean ± SD.

Flow of practice clusters, participating internists, and patients in the practice improvement module intervention trial. Eligible patients for assessing outcomes were those who were seen for asthma in the postintervention period by participating internists. Values for eligible patients and responders are expressed as mean ± SD.

Table 1. 
Components of Asthma Practice Improvement Module Completed by 19 Physicians Randomized to the Intervention Groupa
Components of Asthma Practice Improvement Module Completed by 19 Physicians Randomized to the Intervention Groupa
Table 2. 
Comparison Between Study Arms of Baseline Patient-Level Variables Used to Adjust Postintervention Resultsa
Comparison Between Study Arms of Baseline Patient-Level Variables Used to Adjust Postintervention Resultsa
Table 3. 
Comparison Between Study Arms of Patient-Level Sociodemographic Characteristics for Patients Seen in the Postintervention Period
Comparison Between Study Arms of Patient-Level Sociodemographic Characteristics for Patients Seen in the Postintervention Period
Table 4. 
Unadjusted and Adjusted Odds of Study Outcomes in the Postintervention Perioda
Unadjusted and Adjusted Odds of Study Outcomes in the Postintervention Perioda
Table 5. 
Differences in Measures of Asthma Severity and Asthma Control in the Postintervention Perioda
Differences in Measures of Asthma Severity and Asthma Control in the Postintervention Perioda
Table 6. 
Unadjusted and Adjusted Odds of Study Outcomes in the Postintervention Period: Per-Protocol Analysisa
Unadjusted and Adjusted Odds of Study Outcomes in the Postintervention Period: Per-Protocol Analysisa
Table 7. 
Differences in Measures Asthma Severity and Asthma Control in the Postintervention Period: Per-Protocol Analysisa
Differences in Measures Asthma Severity and Asthma Control in the Postintervention Period: Per-Protocol Analysisa
1.
Holmboe  ESMeehan  TPLynn  LDoyle  PSherwin  TDuffy  FD Promoting physicians' self-assessment and quality improvement: the ABIM diabetes practice improvement module. J Contin Educ Health Prof 2006;26 (2) 109- 119
PubMedArticle
2.
Kassirer  JP ABIM looks toward the future. Am J Med 1996;100 (2) 123- 124
PubMedArticle
3.
Kassirer  JP The quality of care and the quality of measuring it. N Engl J Med 1993;329 (17) 1263- 1265
PubMedArticle
4.
Greco  PJEisenberg  JM Changing physicians' practices. N Engl J Med 1993;329 (17) 1271- 1273
PubMedArticle
5.
 Promoting excellence in health care: self-evaluation component overview. American Board of Internal Medicine Web site. http://wwwabimorg/moc/semovshtm. Accessed October 30, 2006
6.
Casalino  LP Disease management and the organization of physician practice. JAMA 2005;293 (4) 485- 488
PubMedArticle
7.
Berwick  DKilo  C Idealized design of clinical office practice: an interview with Donald Berwick and Charles Kilo of the Institute for Healthcare Improvement. Manag Care Q 1999;7 (4) 62- 69
PubMed
8.
Mohr  JJBatalden  PB Improving safety on the front lines: the role of clinical microsystems. Qual Saf Health Care 2002;11 (1) 45- 50
PubMedArticle
9.
Nelson  ECBatalden  PBHuber  TP  et al.  Microsystems in health care, I: learning from high-performing front-line clinical units. Jt Comm J Qual Improv 2002;28 (9) 472- 493
PubMed
10.
Campbell  MKElbourne  DRAltman  DG CONSORT statement: extension to cluster randomised trials. BMJ 2004;328 (7441) 702- 708
PubMedArticle
11.
Turk  A Overview of HEDIS 2000 Asthma Measurement. Am J Manag Care 2004;6 ((suppl 2)) S342- S346
12.
Waring  JKosinski  MDewey  JE How to Score Version 2 of the SF-36 Health Survey.  Lincoln, RI Quality Metric Inc2000;
13.
Bousquet  JKnani  JDhivert  H  et al.  Quality of life in asthma, I: internal consistency and validity of the SF-36 questionnaire. Am J Respir Crit Care Med 1994;149 (2, pt 1) 371- 375
PubMedArticle
14.
Donner  AKlar  N Design and Analysis of Cluster Randomization: Trials in Health Research.  London, England Arnold Publishers2000;
15.
Nathan  RASorkness  CAKosinski  M  et al.  Development of the asthma control test: a survey for assessing asthma control. J Allergy Clin Immunol 2004;113 (1) 59- 65
PubMedArticle
16.
Revicki  DALeidy  NKBrennan-Diemer  FSorensen  STogias  A Integrating patient preferences into health outcomes assessment: the multiattribute Asthma Symptom Utility Index. Chest 1998;114 (4) 998- 1007
PubMedArticle
17.
Deyo  RACherkin  DCCiol  MA Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45 (6) 613- 619
PubMedArticle
18.
Grant  ENLyttle  CSWeiss  KB The relation of socioeconomic factors and racial/ethnic differences in US asthma mortality. Am J Public Health 2000;90 (12) 1923- 1925
PubMedArticle
19.
Mannino  DMHoma  DMAkinbami  LJMoorman  JEGwynn  CRedd  SC Surveillance for asthma—United States, 1980-1999. MMWR Surveill Summ 2002;51 (1) 1- 13
PubMed
20.
Akinbami  LJ The state of childhood asthma, United States, 1980-2005. http://www.cdc.gov/nchs/data/ad/ad381.pdf. CDC–National Center for Health Statistics Advance Data Web site. Accessed August 10, 2007
21.
SAS Institute Inc, SAS/STAT Users Guide: Version 8 Cary, NC SAS Institute Inc1999;
22.
Davis  DAThomson  MAOxman  ADHaynes  RB Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274 (9) 700- 705
PubMedArticle
23.
Davis  DO'Brien  MAFreemantle  NWolf  FMMazmanian  PTaylor-Vaisey  A Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282 (9) 867- 874
PubMedArticle
24.
Tu  KDavis  D Can we alter physician behavior by educational methods? lessons learned from studies of the management and follow-up of hypertension. J Contin Educ Health Prof 2002;22 (1) 11- 22
PubMedArticle
25.
Campbell  CParboosingh  JGondocz  TBabitskaya  GPham  B A study of the factors that influence physicians' commitments to change their practices using learning diaries. Acad Med 1999;74 (10) ((suppl)) S34- S36
PubMedArticle
26.
Davis  DAMazmanian  PEFordis  MVan Harrison  RThorpe  KEPerrier  L Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006;296 (9) 1094- 1102
PubMedArticle
27.
Duffy  FDHolmboe  ES Self-assessment in lifelong learning and improving performance in practice: physician know thyself. JAMA 2006;296 (9) 1137- 1139
PubMedArticle
28.
Weingarten  SRHenning  JMBadamgarav  E  et al.  Interventions used in disease management programmes for patients with chronic illness—which ones work? meta-analysis of published reports. BMJ 2002;325 (7370) 925
PubMedArticle
29.
Haahtela  TJarvinen  MKava  T  et al.  Comparison of a beta 2-agonist, terbutaline, with an inhaled corticosteroid, budesonide, in newly detected asthma. N Engl J Med 1991;325 (6) 388- 392
PubMedArticle
30.
Kerstjens  HABrand  PLHughes  MD  et al.  A comparison of bronchodilator therapy with or without inhaled corticosteroid therapy for obstructive airways disease: Dutch Chronic Non-Specific Lung Disease Study Group. N Engl J Med 1992;327 (20) 1413- 1419
PubMedArticle
31.
Pauwels  RAPedersen  SBusse  WW  et al.  Early intervention with budesonide in mild persistent asthma: a randomised, double-blind trial. Lancet 2003;361 (9363) 1071- 1076
PubMedArticle
32.
Sin  DDMan  JSharpe  HGan  WQMan  SF Pharmacological management to reduce exacerbations in adults with asthma: a systematic review and meta-analysis. JAMA 2004;292 (3) 367- 376
PubMedArticle
33.
Milgrom  HBender  BAckerson  LBowry  PSmith  BRand  C Noncompliance and treatment failure in children with asthma. J Allergy Clin Immunol 1996;98 (6 Pt 1) 1051- 1057
PubMedArticle
34.
Bender  BWamboldt  FSO'Connor  SL  et al.  Measurement of children's asthma medication adherence by self report, mother report, canister weight, and Doser CT. Ann Allergy Asthma Immunol 2000;85 (5) 416- 421
PubMedArticle
35.
Gilbert  JREvans  CEHaynes  RBTugwell  P Predicting compliance with a regimen of digoxin therapy in family practice. Can Med Assoc J 1980;123 (2) 119- 122
PubMed
36.
Morgan  WJCrain  EFGruchalla  RS  et al.  Results of a home-based environmental intervention among urban children with asthma. N Engl J Med 2004;351 (11) 1068- 1080
PubMedArticle
37.
Shojania  KGGrimshaw  JM Still no magic bullets: pursuing more rigorous research in quality improvement. Am J Med 2004;116 (11) 778- 780
PubMedArticle
Original Investigation
November 12, 2007

Improving Asthma Care Through RecertificationA Cluster Randomized Trial

Author Affiliations

Author Affiliations: Center for Health Services Research (Mss Simpkins and Wang and Drs Pladevall and Williams) and Departments of Biostatistics and Research Epidemiology (Drs Divine and Williams) and Internal Medicine (Dr Williams), Henry Ford Hospital, Detroit, Michigan; and American Board of Internal Medicine, Philadelphia, Pennsylvania (Dr Holmboe).

Arch Intern Med. 2007;167(20):2240-2248. doi:10.1001/archinte.167.20.2240
Abstract

Background  As part of recertification, the American Board of Internal Medicine requires its diplomats to complete at least 1 practice improvement module (PIM). We assessed whether completing an asthma-specific PIM resulted in improved patient outcomes.

Methods  Practices were the unit of randomization in this cluster randomized trial. Physicians in the intervention group were asked to complete the PIM through its planning phase. The primary outcome was the dispensing of an inhaled corticosteroid (ICS) after a postintervention visit for asthma. Secondary outcomes included patient reported processes of care, asthma-related heath care use, and asthma severity. Analyses were adjusted for baseline rates at the cluster-level as well as for individual sociodemographic characteristics.

Results  Eight practices (19 internists) were randomized to the intervention group and 8 practices (21 internists) to the control group. For the primary outcome, ICS fill rates, patients seen by intervention group physicians were not more likely to fill an ICS prescription in the postintervention period than patients seen by control group physicians (adjusted odd ratio [AOR], 1.00; 95% confidence interval [CI], 0.64-1.56). Patients seen for asthma by intervention group physicians were less likely to receive a written action plan than patients seen by control group physicians (AOR, 0.67; 95% CI, 0.48-0.93); however, they were more likely to discuss potential asthma triggers (AOR, 1.62; 95% CI, 1.08-2.42) and had lower self-reported asthma severity measures (unadjusted P = .03). Per-protocol analysis supported the latter 2 associations.

Conclusion  A PIM designed to improve asthma care did not improve filling of ICS prescriptions but may have lessened asthma severity through an increased discussion of asthma triggers.

Trial Registration  clinicaltrials.gov Identifier: NCT00115284

In 1995, the American Board of Internal Medicine (ABIM) ad hoc Committee on Assessment of Practice Performance recommended that the ABIM assess practice performance in addition to assessing knowledge in its maintenance of certification program.1,2 These recommendations were developed in the setting of a national movement to measure and monitor quality3 and with the hope that physician participation would be more likely to effect practice change.4 Initially, maintenance of certification included the option of completing a practice improvement module (PIM); however, in January 2006, this became a recertification requirement for all internists with time-limited certification.5 Modules have been developed for a number of conditions, including asthma, diabetes, and hypertension.

The overall design of each PIM was based on the chronic care model6 and idealized design of the clinical office practice.7 Each PIM is intended to improve care by having physicians adopt quality improvement in their practice, in part through increased awareness of the office microsystem.8,9 In completing a PIM, physicians evaluate their (and their practice’s) management of a specific disease condition; they develop and implement a plan to improve care for that condition; and they measure the impact of the implemented plan on subsequent care. However, despite the current requirement that recertifying internists complete a PIM, it is not currently known whether these modules result in objective improvement in disease outcomes. We chose a cluster-randomized study design to determine whether an asthma PIM improved disease-specific outcomes.

METHODS
STUDY SETTING AND POPULATION

The trial was performed at a large vertically integrated health care system in southeastern Michigan. All participating providers were practicing, board-certified internists and members of a large multispecialty medical group. Written consent was required of all participating physicians. The study was approved by the health system's institutional review board and was compliant with its Health Insurance Portability and Accountability Act policy.

RANDOMIZATION

This study was designed in accordance with the recommendations of the Consolidated Standards of Reporting Trials for cluster randomized trials.10 Practicing, board-certified general internists were recruited at the multiple practice sites (ie, clinics) throughout the health system. In the consent process, physicians were notified that if they were randomized to the intervention group they would be asked to complete an asthma PIM. The recruitment goal was 40 internists (20 internists per study arm), although the number of consenting physicians and their practice location determined the number and size of the practice clusters. As PIMs were designed to improve the disease-specific performance at both the physician level and the practice level, participating internists were grouped according to practice, and the practices were randomized such that all participating physicians at a given practice site were randomized to either the intervention arm or the control arm. Likewise, participating physicians within a practice who were randomized to the intervention group could work together to develop a practice improvement plan for their practice. Outcomes were assessed in patients seen for asthma by both intervention and control group physicians and accounted for higher-level clustering by physician and practice. Neither physicians nor researchers were informed of their intervention assignment until after randomization.

To ensure comparable patient populations in each study arm, practice randomization was stratified by rate of use of asthma controller medications (ie, greater than and less than 70% of asthmatic patients who were using a controller medication)11 and practice location (ie, urban vs suburban). Patients with asthma who received their care from enrolled providers were notified by mail of their physician's participation in the study; however, patients were not aware of their physician's group assignment.

INTERVENTION

There were 2 phases of the asthma PIM: a data collection phase (ie, physicians collected a minimum of 10 patient surveys, performed at least 10 chart reviews, and completed 1 practice review) and an improvement plan phase (ie, physicians reviewed the results of the data collection phase, selected practice processes to improve, submitted the plan to the ABIM, implemented the plan in their practice, and measured the impact of the plan). We requested that participating physicians in the intervention group complete the PIM through plan implementation. All participating physicians in the intervention group received the asthma PIM contemporaneously and were initially allowed 45 days to complete the PIM; this period was extended to 90 days when it was clear that additional time was required. Control group physicians continued to provide their usual care and did not complete an asthma PIM. After this 90-day period, we began assessing the differences between the patients who were seen by intervention group physicians and the patients who were seen by control group physicians.

OUTCOME MEASURES

The primary outcome of this study was the likelihood of filling 1 or more inhaled corticosteroid (ICS) prescriptions after a visit for asthma. Secondary patient-level outcomes included the likelihood of each of the following after a physician visit for asthma: (1) filling the prescription for 1 or more short-acting β-agonist canisters; (2) filling the prescription for 1 or more oral corticosteroid medications; (3) reporting a prior prescription of an asthma controller medication (defined as an “inhaler or pill that is not used for quick relief, but instead is used to control asthma”); (4) receiving a written action plan for asthma exacerbations; (5) receiving instruction on the use of a peak flow meter; (6) reporting instructions regarding proper inhaler technique; (7) having a discussion regarding asthma triggers; (8) experiencing nocturnal asthma symptoms in the 4 weeks before the survey; (9) using a rescue inhaler regularly in the 4 weeks before the survey; and (10) being instructed to quit smoking (among current smokers with asthma). Additional secondary outcomes that were examined but not prespecified included the likelihood of asthma-related hospitalization in the 3 months after the asthma visit; the likelihood of an asthma-related emergency department visit in the 3 months after the asthma visit; measures of asthma severity and control; and both the Physical Functioning Scale and the General Health Scale from the 36-Item Short-Form Health Survey (SF-36).12 These latter 2 dimensions of the SF-36 were used because they were the most strongly correlated with asthma severity in a previous study.13

DATA SOURCES

Patient-level data regarding sociodemographics, processes of care, medication dispensings, asthma control and symptoms, and perception of general health were assessed by both patient surveys and electronic claims data. Because this study was a cluster-randomized trial, with practice as the unit of randomization, we assessed both utilization data and survey data on 600 patients with asthma who were seen at these practices to adjust postintervention study results. This double cross-sectional approach for cluster-randomized studies has been described in detail elsewhere.14 The survey assessed process indicators consistent with the primary and secondary outcomes, as well as asthma control, asthma severity, and general health, using validated instruments from the Asthma Control Test,15 the Asthma Symptom Utility Index,16 and the SF-36, respectively. A total of 358 of 600 patients (60%) responded to the baseline survey; 14 of 358 respondents (4%) reported that they did not have a diagnosis of asthma. Baseline rates of medication dispensing were assessed using electronic claims data. These data were used to identify the proportion of patients with a prescription fill in the 3 months after their last asthma-related visit to a study physician in the preintervention period.

To assess study outcomes, the same survey was administered to all patients seen by intervention and control group physicians for an asthma-related visit (N = 741) (ie, primary or secondary diagnosis coded as International Classification of Diseases, Ninth Revision, 493.xx) in the 6 months after the intervention (ie, the postintervention period). Some patients from the preintervention cohort may have been included in the postintervention cohort owing to the double cross-sectional design of the trial. Of the 741 postintervention surveys that were mailed, 496 (67%) were returned; 49 of the 496 respondents (10%) reportedthat they did not have a diagnosis of asthma. Electronic claims data were used to identify the proportion of patients with an ICS fill in the 3 months after their first asthma-related visit to a study physician in the postintervention period. Information on patient age, sex, race, and marital status was also available from electronic data. Race/ethnicity categories were entered at the time of registration into the health system; usually this information was self-identified but, on occasion, could have been assigned by health care personnel. We used diagnosis and procedural codes to calculate an adaptation of the Charlson Comorbidity Index, an estimate of concomitant comorbidity.17 To estimate median household income and other socioeconomic status variables, we used a commercially available package (Mapping Solutions LLC, Lansing, Michigan). The components included MapInfo Professional and MapMarker (MapInfo Corp, Troy, New York), bundled with Allocate/Solocast (SRC LLC, Orange, California). MapMarker geocoding software assigned longitude and latitude coordinates to the address record, which was then mapped to the census block using MapInfo Professional. Allocate/Solocast are data retrieval and reporting engines, which then linked census-block level demographic information to the mapped address. An individuals' household income was taken as the median household income for their census block using year 2000 data from the US Census Bureau.

STATISTICAL ANALYSIS

Analyses were specific to the cluster-randomized design and accounted for correlation of outcome measures within practice clusters.14 This study had 80% power to detect an 18% difference in ICS/controller use (50% vs 68%) between treatment arms, assuming an intraclass correlation coefficient of 0.05 for patients seen in the same practice. Differences in the sociodemographic characteristics of patients seen for asthma in the preintervention and postintervention periods in the 2 study arms were compared using generalized estimating equation (GEE) logistic regression for categorical variables and GEE linear regression for continuous variables. The clusters for the GEE analyses were defined by physicians nested within practice locations. Differences in the primary and secondary outcomes between study arms were fit first with a univariable model and then with a multivariable model using logistical regression to estimate adjusted odds ratios (AORs) and 95% confidence intervals (CIs). Multivariable models were adjusted for baseline asthma severity and stratifying variables in practice clusters, baseline rates for the outcomes of interest in practice clusters, and individual sociodemographic characteristics, including age, sex, race, and median household income. We selected these individual characteristics to include in the models based on their prior association with asthma outcomes.1820 Differences in asthma control, asthma severity, physical functioning, and general health were assessed using linear regression. Because many physicians in the intervention arm did not complete the PIM as directed, we repeated our analyses comparing patients who were seen by the 5 intervention physicians who completed the intervention as directed (ie, per-protocol analysis) with patients who were seen by control group physicians. P < .05 was considered statistically significant. Data were analyzed using a commercially available software package.21

RESULTS

Eight practices, consisting of 19 participating physicians, were randomized to the intervention arm, and 8 practices, consisting of 21 physicians, were randomized to the control arm (Figure). No differences in age, sex, length of board certification, or location of practice (ie, urban or suburban) were found between intervention arm and control arm physicians (data not shown). Of the 19 physicians in the intervention group, only 5 (26%) completed the PIM through submitting the practice improvement plan (Table 1). Of the remaining 14 physicians, 10 initiated some portion of the PIM; 4 physicians (21%) in the intervention group did not submit any material.

Baseline differences between patients seen by intervention and control group physicians are shown in Table 2. No statistically significant differences were observed in the baseline variables that were used to adjust the postintervention results. Postintervention differences in the sociodemographic composition of patients seen by physicians in both study groups are shown in Table 3. There was no statistically significant difference in the sociodemographic characteristics between patients seen by intervention group and control group physicians (P values not shown).

Table 4 presents the intention-to-treat analysis comparing the odds of study outcomes between treatment arms. For the primary outcome, ie, ICS use, patients seen by intervention group physicians were not more likely than patients seen by control group physicians to have a fill of an ICS in the 3 months after the visit for asthma in the postintervention period (AOR, 1.00; 95% CI, 0.64-1.56). Patients of physicians in the intervention group were less likely to report discussing a written action plan for asthma exacerbations (36% vs 46%; AOR, 0.67; 95% CI, 0.48-0.93) but more likely than patients of physicians in the control group to discuss possible asthma triggers with their physician (77% vs 70%; AOR, 1.62; 95% CI, 1.08-2.42). In the unadjusted analysis, patient-reported asthma severity (ie, Asthma Symptom Utility Index score) was significantly lower in patients seen by physicians in the intervention group (P = .03) but was of borderline significance after adjustment (P = .09) (Table 5).

Because many physicians in the intervention arm did not complete the PIM as directed, we compared postintervention study outcomes between patients who were seen by the 5 intervention physicians who completed the intervention as requested and all patients who were seen by control group physicians (ie, per-protocol analysis). Patients who were seen by intervention physicians were more likely to report discussing potential asthma triggers with their physician (AOR, 1.55; 95% CI = 1.01-2.37) (Table 6). Also, patients who were seen by physicians in the intervention group had a lower index of asthma severity and a better index of disease control after intervention (P< .001 and P = .01, respectively) than patients who were seen by control group physicians (Table 7).

COMMENT

In this study, we did not observe an improvement in our primary outcome measure: filling of ICS prescriptions. However, we did observe an improvement in patient self-reported asthma severity in patients who were seen by intervention group physicians compared with patients who were seen by control group physicians. Although asthma severity was an a priori outcome at the time of data collection (and all measured outcomes are shown), it was not prespecified at the time of study registration, nor was it the primary outcome. These findings, albeit speculative, provide the first evidence that recent recertification requirements implemented by the ABIM may improve relevant disease outcomes.

If traditional, didactic continuing medical education may be of limited utility in improving physician behavior or health outcomes, practice-based interventions may be more effective.2224 In 1 study, physicians were asked to record those learning activities that prompted practice changes.25 Investigators found that learning by reviewing the treatment of 1 or more patients was 37% more likely than medical literature review to result in a change in practice. In light of the recent evidence suggesting that physicians do not accurately self-assess their performance,26 structured feedback, as occurs with the PIM, may provide a stronger impetus for behavioral change.27 In a meta-analysis of interventions used in disease management, provider feedback appeared effective in improving adherence to guidelines but resulted in modest improvements in disease control.28

In this study, the exact process through which the PIM may have improved asthma severity was not known. The only significant postintervention differences that we noted for patients seen by intervention group physicians were increased discussions of asthma triggers but lower rates of receiving a written action plan when compared with patients seen by control group physicians. We did not observe a significant difference in the primary outcome measure: ICS use.

Despite the limited differences in process measures, there are a number of reasons why we believe that the observed differences in asthma severity resulted from the intervention. First, differences in asthma severity between treatment arms persisted after baseline severity measures within practice clusters were controlled for. Second, the strength of the association increased after limiting the analysis to those physicians who completed the PIM as requested (this finding also included other measures of disease control, such the Asthma Control Test score). Third, other measures of asthma severity that preceded the postintervention index visit, such as historical asthma-related emergency department visits and hospitalizations, did not differ between patients seen by physicians in both study arms, suggesting that patients seen by both groups of physicians were similar in disease severity before the postintervention index visit. Finally, patterns of rescue medication use (ie, lower short-acting β-agonist and oral steroid use) after the postintervention visit were consistent with subsequent less severe asthma in patients seen by intervention group physicians when compared with those seen by control group physicians.

It is important to note that in completing the PIM, physicians chose the process measure that they wished to improve. Accordingly, physicians may have identified and targeted different deficiencies to improve. We assumed that given the well-described benefits of ICS use,2932 most physicians would choose ICS use as the process to improve. In particular, more than 90% of patients reported a previous prescription for a controller medication, yet only 50% of patients with asthma filled an ICS prescription at baseline. This suggested that actual ICS use was inappropriately low even after other asthma controller medication use was accounted for (data not shown). However, physicians completing the asthma PIM adjudicated ICS use by abstracting patient charts and were not privy to pharmacy fill information or rates of adherence. Because patients and physicians tend to overreport33,34 and overestimate35 compliance, respectively, it is quite possible that physicians did not perceive a deficiency in ICS use and therefore did not target it for improvement.

Although we did not observe a change in the primary outcome measure, if the net effect of multiple process changes (measured and unmeasured) resulted in improved asthma care, we might expect to see an improvement in an end measure, such as asthma severity, rather than a single, intermediate-process measure. However, patients who were seen by intervention physicians were significantly more likely to report discussing potential asthma triggers with their physician than were patients who were seen by control group physicians. Therefore, it is also plausible and possible that a resulting decrease in trigger exposure among patients seen by intervention group physicians resulted in less severe asthma symptoms, as has been demonstrated in intensive home-based interventions.36

In focusing on 1 aspect of asthma-related care to improve, such as discussing asthma triggers, other processes may have been neglected by physicians in the intervention group. Neglecting to discuss these other aspects of asthma care may explain why patients who were seen by intervention physicians were significantly less likely to report receiving written action plans. The long-term effects of these potential trade-offs were not evaluated and will need further study.

This study must be interpreted in light of its other limitations as well. Randomizing practices may result in an inequitable distribution of patients, which can confound outcome differences between treatment arms.14 To prevent and control potential confounding, we stratified our randomization and adjusted for baseline intercluster differences. Although we found no difference in baseline or patient-level sociodemographic characteristics by study arm, other important differences may have persisted in spite of our efforts. Despite this limitation, we believe that the cluster-randomized design was the most appropriate model for a practice-based intervention.

Next, most of the physicians in the intervention group did not complete the intervention as intended. One of the barriers identified was the time that physicians had to complete the PIM through the planning stage, which was initially shorter than the time allowed by the ABIM in actual practice. As a result, time was extended (ie, from 45 days to 90 days) to more closely replicate the actual time allowed when a module for maintenance of certification is being completed. Extending the time allotted to complete the module did not markedly increase the number of physicians who completed the required PIM components. Because some physicians in our study had time-unlimited ABIM certification, we also examined whether PIM completion rates were different for this group when compared with the intervention group physicians who were required to complete a PIM for recertification. Surprisingly, completion rates were not different between groups (P > .99, data not shown). Therefore, allocated time and current ABIM requirements did not seem to affect completion rates.

Anecdotally, some clinicians expressed difficulty in having sufficient numbers of patients return surveys, a problem well known to researchers. Given low patient response rates on surveys in general, it may be onerous for physicians to meet the requisite number of surveys to complete a PIM. This component of the PIM also had the lowest rate of completion in our study. However, now that PIMs are a requirement for ABIM recertification, a formal analysis of factors related to completion could be performed on a much larger group of physicians than was involved in our study.

Although there were few intervention physicians who completed the PIM as requested, our results suggested that the intervention may have affected clinically important outcomes (ie, asthma severity and control), especially among patients whose physicians developed a plan to improve asthma care in their practice. However, the overall clinical relevance of these changes is uncertain. Moreover, the lack of multiple-process measure improvements proximal to the time of the intervention could also be construed as evidence against the intervention's robustness, since this would have been the time of peak effect. Finally, given sample size limitations, there was no prespecified plan to adjust for the multiple comparisons of secondary outcomes. However, in this study, we show all outcome measures examined, and post hoc adjustment using a conservative criterion, such as the Bonferroni correction, would still suggest that the PIM reduced asthma severity.

In summary, despite many physicians in our study not completing the PIM as directed, our findings suggest that asthma severity may be lowered in patients whose physicians complete an asthma PIM. Although the effect appeared modest, as pointed out by Shojania and Grimshaw,37 perhaps this is the magnitude of improvement that we should rationally expect from quality improvement efforts. Further studies will be needed to confirm our findings and to identify modifiable barriers to physicians completing PIMs.

Back to top
Article Information

Correspondence: L. Keoki Williams, MD, MPH, Center for Health Services Research, Henry Ford Hospital, One Ford Place, 3A CHSR, Detroit, MI 48202 (kwillia5@hfhs.org).

Accepted for Publication: June 13, 2007.

Author Contributions: Ms Simpkins had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Simpkins, Divine, and Williams. Acquisition of data: Simpkins, Wang, and Williams. Analysis and interpretation of data: Simpkins, Divine, Wang, Holmboe, Pladevall, and Williams. Drafting of the manuscript: Simpkins, Divine, Holmboe, and Williams. Critical revision of the manuscript for important intellectual content: Simpkins, Divine, Holmboe, Pladevall, and Williams. Statistical analysis: Divine, Wang, and Pladevall. Obtained funding: Williams. Administrative, technical, and material support: Simpkins. Study supervision: Williams.

Financial Disclosure: Dr Holmboe is an employee of the ABIM.

Funding/Support: This research was funded by the ABIM Foundation. Additional funding was provided by the National Institute of Heart, Lung, and Blood (Drs Pladevall and Williams); the National Institute of Allergy and Infectious Diseases (Dr Williams); and the National Institutes of Diabetes, Digestive, and Kidney Disease (Drs Pladevall and Williams), National Institutes of Health, Bethesda, Maryland.

Role of the Sponsor: Dr Holmboe, acting in the role of coinvestigator, was involved in the analysis and interpretation of data and critical revision of the manuscript for important intellectual content.

Previous Presentation: This work was presented in part at the 13th Annual Health Maintenance Organization Research Network Conference; March 20, 2007; Portland, Oregon.

References
1.
Holmboe  ESMeehan  TPLynn  LDoyle  PSherwin  TDuffy  FD Promoting physicians' self-assessment and quality improvement: the ABIM diabetes practice improvement module. J Contin Educ Health Prof 2006;26 (2) 109- 119
PubMedArticle
2.
Kassirer  JP ABIM looks toward the future. Am J Med 1996;100 (2) 123- 124
PubMedArticle
3.
Kassirer  JP The quality of care and the quality of measuring it. N Engl J Med 1993;329 (17) 1263- 1265
PubMedArticle
4.
Greco  PJEisenberg  JM Changing physicians' practices. N Engl J Med 1993;329 (17) 1271- 1273
PubMedArticle
5.
 Promoting excellence in health care: self-evaluation component overview. American Board of Internal Medicine Web site. http://wwwabimorg/moc/semovshtm. Accessed October 30, 2006
6.
Casalino  LP Disease management and the organization of physician practice. JAMA 2005;293 (4) 485- 488
PubMedArticle
7.
Berwick  DKilo  C Idealized design of clinical office practice: an interview with Donald Berwick and Charles Kilo of the Institute for Healthcare Improvement. Manag Care Q 1999;7 (4) 62- 69
PubMed
8.
Mohr  JJBatalden  PB Improving safety on the front lines: the role of clinical microsystems. Qual Saf Health Care 2002;11 (1) 45- 50
PubMedArticle
9.
Nelson  ECBatalden  PBHuber  TP  et al.  Microsystems in health care, I: learning from high-performing front-line clinical units. Jt Comm J Qual Improv 2002;28 (9) 472- 493
PubMed
10.
Campbell  MKElbourne  DRAltman  DG CONSORT statement: extension to cluster randomised trials. BMJ 2004;328 (7441) 702- 708
PubMedArticle
11.
Turk  A Overview of HEDIS 2000 Asthma Measurement. Am J Manag Care 2004;6 ((suppl 2)) S342- S346
12.
Waring  JKosinski  MDewey  JE How to Score Version 2 of the SF-36 Health Survey.  Lincoln, RI Quality Metric Inc2000;
13.
Bousquet  JKnani  JDhivert  H  et al.  Quality of life in asthma, I: internal consistency and validity of the SF-36 questionnaire. Am J Respir Crit Care Med 1994;149 (2, pt 1) 371- 375
PubMedArticle
14.
Donner  AKlar  N Design and Analysis of Cluster Randomization: Trials in Health Research.  London, England Arnold Publishers2000;
15.
Nathan  RASorkness  CAKosinski  M  et al.  Development of the asthma control test: a survey for assessing asthma control. J Allergy Clin Immunol 2004;113 (1) 59- 65
PubMedArticle
16.
Revicki  DALeidy  NKBrennan-Diemer  FSorensen  STogias  A Integrating patient preferences into health outcomes assessment: the multiattribute Asthma Symptom Utility Index. Chest 1998;114 (4) 998- 1007
PubMedArticle
17.
Deyo  RACherkin  DCCiol  MA Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol 1992;45 (6) 613- 619
PubMedArticle
18.
Grant  ENLyttle  CSWeiss  KB The relation of socioeconomic factors and racial/ethnic differences in US asthma mortality. Am J Public Health 2000;90 (12) 1923- 1925
PubMedArticle
19.
Mannino  DMHoma  DMAkinbami  LJMoorman  JEGwynn  CRedd  SC Surveillance for asthma—United States, 1980-1999. MMWR Surveill Summ 2002;51 (1) 1- 13
PubMed
20.
Akinbami  LJ The state of childhood asthma, United States, 1980-2005. http://www.cdc.gov/nchs/data/ad/ad381.pdf. CDC–National Center for Health Statistics Advance Data Web site. Accessed August 10, 2007
21.
SAS Institute Inc, SAS/STAT Users Guide: Version 8 Cary, NC SAS Institute Inc1999;
22.
Davis  DAThomson  MAOxman  ADHaynes  RB Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA 1995;274 (9) 700- 705
PubMedArticle
23.
Davis  DO'Brien  MAFreemantle  NWolf  FMMazmanian  PTaylor-Vaisey  A Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999;282 (9) 867- 874
PubMedArticle
24.
Tu  KDavis  D Can we alter physician behavior by educational methods? lessons learned from studies of the management and follow-up of hypertension. J Contin Educ Health Prof 2002;22 (1) 11- 22
PubMedArticle
25.
Campbell  CParboosingh  JGondocz  TBabitskaya  GPham  B A study of the factors that influence physicians' commitments to change their practices using learning diaries. Acad Med 1999;74 (10) ((suppl)) S34- S36
PubMedArticle
26.
Davis  DAMazmanian  PEFordis  MVan Harrison  RThorpe  KEPerrier  L Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. JAMA 2006;296 (9) 1094- 1102
PubMedArticle
27.
Duffy  FDHolmboe  ES Self-assessment in lifelong learning and improving performance in practice: physician know thyself. JAMA 2006;296 (9) 1137- 1139
PubMedArticle
28.
Weingarten  SRHenning  JMBadamgarav  E  et al.  Interventions used in disease management programmes for patients with chronic illness—which ones work? meta-analysis of published reports. BMJ 2002;325 (7370) 925
PubMedArticle
29.
Haahtela  TJarvinen  MKava  T  et al.  Comparison of a beta 2-agonist, terbutaline, with an inhaled corticosteroid, budesonide, in newly detected asthma. N Engl J Med 1991;325 (6) 388- 392
PubMedArticle
30.
Kerstjens  HABrand  PLHughes  MD  et al.  A comparison of bronchodilator therapy with or without inhaled corticosteroid therapy for obstructive airways disease: Dutch Chronic Non-Specific Lung Disease Study Group. N Engl J Med 1992;327 (20) 1413- 1419
PubMedArticle
31.
Pauwels  RAPedersen  SBusse  WW  et al.  Early intervention with budesonide in mild persistent asthma: a randomised, double-blind trial. Lancet 2003;361 (9363) 1071- 1076
PubMedArticle
32.
Sin  DDMan  JSharpe  HGan  WQMan  SF Pharmacological management to reduce exacerbations in adults with asthma: a systematic review and meta-analysis. JAMA 2004;292 (3) 367- 376
PubMedArticle
33.
Milgrom  HBender  BAckerson  LBowry  PSmith  BRand  C Noncompliance and treatment failure in children with asthma. J Allergy Clin Immunol 1996;98 (6 Pt 1) 1051- 1057
PubMedArticle
34.
Bender  BWamboldt  FSO'Connor  SL  et al.  Measurement of children's asthma medication adherence by self report, mother report, canister weight, and Doser CT. Ann Allergy Asthma Immunol 2000;85 (5) 416- 421
PubMedArticle
35.
Gilbert  JREvans  CEHaynes  RBTugwell  P Predicting compliance with a regimen of digoxin therapy in family practice. Can Med Assoc J 1980;123 (2) 119- 122
PubMed
36.
Morgan  WJCrain  EFGruchalla  RS  et al.  Results of a home-based environmental intervention among urban children with asthma. N Engl J Med 2004;351 (11) 1068- 1080
PubMedArticle
37.
Shojania  KGGrimshaw  JM Still no magic bullets: pursuing more rigorous research in quality improvement. Am J Med 2004;116 (11) 778- 780
PubMedArticle
×