Accreditation Council for Graduate Medical Education Accreditation and Influence on Perceptions of Pediatric Otolaryngology Fellowship Training Experience

Importance  The American Society of Pediatric Otolaryngology (ASPO) has set a goal of universal accreditation of fellowship programs by the Accreditation Council for Graduate Medical Education (ACGME) by 2014. This study offers data comparing trainee experience at accredited vs nonaccredited programs.

Objective  To evaluate perceptions of pediatric otolaryngology fellowship training experience and to elucidate differences between those who trained in ACGME-accredited fellowships vs those who did not.

Design and Participants  Web-based survey sent to all members of ASPO, as well as recent fellowship graduate ASPO-eligible physicians. Responses were obtained in an anonymous fashion. The study population comprised 136 ASPO members who recently graduated from pediatric otolaryngology fellowship programs (36 from ACGME-accredited fellowships and 100 from nonaccredited programs).

Main Outcomes and Measures  Difference in perceived fellowship experience between graduates of accredited vs nonaccredited programs, specifically, differences in service vs education perceptions.

Results  Overall, a majority (64%) of respondents agreed that standardizing the pediatric fellowship curriculum through ACGME accreditation is a worthwhile goal. Those who attended ACGME-accredited fellowships were more likely to favor accreditation vs non-ACGME graduates (83% vs 58%; P = .006). Graduates of ACGME-accredited programs were also more likely to agree that their fellowship provided adequate preparation for a career in academic medicine (100% vs 89%; P = .04), protected time for research (94% vs 60%; P < .001), vacation and academic time (94% vs 78%; P = .03), and opportunities to formally evaluate their superiors (72% vs 32%; P < .001). Non-ACGME graduates reported higher primary call frequency (0.8 days per week vs 0.2 days per week; P = .01), and attending physician participation in rounds (71% vs 53%; P = .05).

Conclusions and Relevance  Most respondents were in agreement with universal ACGME accreditation. Those having trained in accredited programs cite increased allowance for research, academic and vacation time, more formal opportunities to evaluate their faculty, and decreased primary call burden.

Bluestone and Stool began the first fellowship in pediatric otolaryngology in 1975.¹ The field has flourished since, with 31 fellowships currently available (Table 1). The Accreditation Council for Graduate Medical Education (ACGME) laid out the initial criteria for accreditation of pediatric otolaryngology training programs in 1995 and revised them to the current guidelines in 2005.² As of the 2012 academic year, 18 of 31 fellowship programs are accredited by the ACGME, and one other program (The Hospital for Sick Children, Toronto, Ontario, Canada) is accredited by the Canadian equivalent Royal College of Physicians and Surgeons of Canada. The American Society of Pediatric Otolaryngology (ASPO) has espoused the goal of universal accreditation by 2014. As a comparison, all fellowship programs for pediatric surgery and pediatric urology are ACGME accredited, whereas accreditation for pediatric orthopedics and plastic surgery training programs remains voluntary. Notably, both pediatric surgery and pediatric urology offer the opportunity to become board certified for these subspecialties.

Informal polling of the programs not currently accredited by ACGME reveals that the primary reason cited to not pursue accreditation is financial. Indeed, many of the nonaccredited programs use income generated from fellows’ billings to support the fellowship because they face difficulties garnering this financial support for the fellowship from their local associated academic institution. Thus, financial issues remain a significant impediment toward accreditation for some of the programs. It is unclear what the educational implications are for the trainee in a more traditional ACGME-accredited residency training paradigm vs a junior partner financially independent training paradigm.³ We hypothesized that there would be substantive differences in service vs education perceptions when surveying recent graduates of both types of training programs, with those having attended ACGME-accredited fellowships citing an increased educationally focused training experience.

To test this hypothesis, we undertook a survey study of ASPO members and recent graduates of pediatric otolaryngology fellowships to evaluate perceptions of fellowship training experience and to elucidate differences between those who trained in ACGME-accredited fellowships vs those who did not (non-ACGME graduates).

A web-based survey (Figure 1) was sent to all members of ASPO, as well as recent fellowship graduate ASPO-eligible physicians. Unless otherwise listed, participants responded to each question by choosing on a 5-point scale from 1 (strongly disagree) to 5 (strongly agree). Respondents were categorized by ACGME accreditation status of their fellowship (at the time they attended) and by time since graduation from fellowship. The χ² test was used to determine a statistically significant difference in the groups for categorical data, and the Mann-Whitney test was used for continuous data.

There were 136 responses for a response rate of 38%. Of these respondents, 100 (74%) graduated from nonaccredited programs and 36 (26%) graduated from ACGME-accredited fellowship programs. Of the 136 total responses, 72 came from graduates in the past 10 years. Of these, 29 (40%) were from ACGME programs. Therefore, most (84%) respondents who attended ACGME-accredited fellowships graduated in the past 10 years. All respondents, regardless of the accreditation status of their fellowships, believed that their fellowship experience met or exceeded their expectations and would recommend their program to future applicants. Overall, graduates of ACGME-accredited programs were more likely to agree that their fellowship provided adequate preparation for a career in academic medicine (100% vs 89%; P = .04).

In terms of supervision and autonomy, we excluded respondents who graduated more than 10 years ago (when most programs were not accredited and training was inherently different). Table 2 summarizes the responses to supervision experience on a rating scale from 0 (never) to 5 (always). A similar percentage of graduates from both categories believed that they had adequate supervision in the operating room and in managing inpatients. Interestingly, non-ACGME graduates reported a higher rate of attending physician participation in rounds (rounding on inpatients everyday or most of the time, responding 4 or 5 on the rating scale) (85% vs 44%; P < .001, χ² test).

Overall, graduates of ACGME-accredited programs were significantly more likely to agree that their fellowship provided adequate protected time for vacation and attending academic meetings (94% vs 78%; P = .03); however, for respondents over the past 10 years, this difference did not reach statistical significance (96% vs 81%; P = .06). Non-ACGME graduates reported higher primary call frequency (0.8 days per week vs 0.2-days per week; P = .01). For graduates in the past 10 years, ACGME fellows reported more formal opportunities for didactic interactions per month (6.6 vs 3.6 events per month; P = .04). Similarly, there was a statistically significant difference in the reported protected time for research. This time for research opportunity was reportedly more abundant in ACGME programs both overall (94% vs 60% adequate; P < .001) and for graduates in the past 10 years (96% vs 60% adequate; P < .001). These results are summarized in Figure 2A and B.

Those who graduated in the past 10 years, regardless of accreditation status, felt satisfied in regard to understanding expectations and receiving feedback on their performance. Table 3 summarizes the responses in this category. Importantly, opportunities for fellows to formally evaluate their superiors were reportedly twice as likely to occur in fellow graduates of ACGME-accredited programs, both overall (72% vs 32%; P < .001) and for graduates in the past 10 years (83% vs 46%; P = .002) (Figure 2C).

All graduates in the past 10 years, regardless of program accreditation, would recommend their programs for training. Overall, the majority of respondents agreed that standardizing the pediatric fellowship curriculum through ACGME accreditation is a worthwhile goal (64%). Those who attended ACGME-accredited fellowships were significantly more likely to agree than those who did not (83% vs 58%; P = .006). This difference remained significant when looking only at recent graduates (86% vs 53%; P = .007) (Figure 2D).

The main objective of this study was to determine if there were any differences in the way graduates perceived their fellowship training between those who participated in ACGME-accredited programs vs nonaccredited ones. Broad areas in which we believed there may be differences included formal didactics, the amount of direct supervision vs autonomy, and balance of service vs educational requirements.

It is noteworthy to discover that overall graduates are pleased with their training experience and would recommend it to others. Furthermore, it appears that existing programs provide adequate didactics, regardless of accreditation status. There were significant differences in areas such as time off for meetings, primary call, and protected research time that could represent a heightened awareness of the education-service balance among accredited programs. Slightly more graduates of accredited programs felt prepared for a career in academic medicine. This survey was not designed to examine whether this led to a difference in practice type after graduation.

While nearly two-thirds of respondents agreed that ACGME accreditation is a worthwhile goal for the pediatric otolaryngology community, this approval varied sharply based on fellowship training type. This is hardly surprising, especially given the universally high regard that survey participants had for their fellowship training. Because the cost of accreditation remains burdensome for fellowship programs that lack institutional financial backing and depend on fellows’ billings for program support, financial issues are frequently cited as a significant impediment toward accreditation. Furthermore, the absence of subcertification for pediatric otolaryngology makes the incentive for ACGME accreditation elusive for some. Despite these facts, a majority of pediatric otolaryngology programs are now accredited, and many others are in the process of applying for accreditation.

Our study has several limitations. Survey-based studies can suffer from nonresponse bias. The survey responses were collected anonymously because we believed that respondents would feel more comfortable honestly evaluating their fellowship experience in this fashion. This unfortunately prevents us from comparing characteristics of survey responders with nonresponders and with graduates of pediatric otolaryngology fellowships as a whole. Second, the survey asked respondents about events that potentially occurred many years prior, introducing the chance that certain aspects of their education may be misremembered. While we recognize these limitations, we believe that these data are worth examining and discussing as a specialty, especially as we move to universal accreditation of pediatric otolaryngology fellowship programs.

In conclusion, most respondents were in agreement with universal ACGME accreditation. Survey responses highlight perceived differences in fellowship training areas related to service requirements vs education based on status of ACGME accreditation. Those having trained in accredited programs cite increased allowance for research, academic and vacation time, more formal opportunities to evaluate their faculty, and decreased primary call burden.

Corresponding Author: Diego Preciado, MD, PhD, Division of Pediatric Otolaryngology, Children’s National Medical Center, 111 Michigan Ave NW, Washington, DC 20010 (dpreciad@cnmc.org).

Submitted for Publication: January 17, 2013; June 4, 2013; accepted June 17, 2013.

Published Online: August 15, 2013. doi:10.1001/jamaoto.2013.4064.

Author Contributions: Drs Bedwell and Preciado had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Bedwell, Choi, Preciado.

Acquisition of data: Bedwell, Chan, Preciado.

Analysis and interpretation of data: Bedwell, Choi, Preciado.

Drafting of the manuscript: Bedwell, Preciado.

Critical revision of the manuscript for important intellectual content: Bedwell, Choi, Chan, Preciado.

Statistical analysis: Bedwell, Preciado.

Study supervision: Choi, Preciado.

Conflict of Interest Disclosures: None reported.

Previous Presentation: This study was presented as a poster presentation at the 2011 American Society of Pediatric Otolaryngology Meeting; April 29, 2011; Chicago, Illinois.