Customize your JAMA Network experience by selecting one or more topics from the list below.
Akinboyewa I, Cabrera-Muffly C. Association of Fellowship Training With Otolaryngology Training Examination Scores. JAMA Otolaryngol Head Neck Surg. 2016;142(3):274–277. doi:10.1001/jamaoto.2015.3570
Copyright 2016 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
No prior studies have evaluated whether residents who pursue fellowship training achieve higher performance on the Otolaryngology Training Examination (OTE) and whether a specific fellowship will demonstrate a correlation with the corresponding specialty-specific OTE score.
To determine whether residents pursuing fellowship training achieve higher performance on the OTE and whether fellowship choice is correlated with higher scores on the related subspecialty section of the OTE.
Design, Setting, and Participants
This retrospective analysis included 35 residents training in an academic otolaryngology residency program from July 1, 2003, to June 30, 2014. The OTE scores for postgraduate years 2 through 5 and the type of fellowship were collected for all residents meeting inclusion criteria. Data were collected from September 1 to October 15, 2014, and analyzed from October 16 to December 1, 2014.
Residents were divided by whether they pursued fellowship training and by the type of fellowship chosen.
Main Outcomes and Measures
Outcome measures included comparison of scores between residents who pursued vs those who did not pursue fellowship training and comparison of subspecialty OTE scores between residents who pursued the corresponding fellowship and those who did not.
Of the 35 residents who met the inclusion criteria (24 men and 11 women), 17 (49%) pursued fellowship training. The 3 most common fellowship choices were facial plastic and reconstructive surgery, pediatric otolaryngology, and rhinology (4 residents each [24%]). For all residents, mean scores on the OTE improved each subsequent training year, but this difference was only significant between postgraduate years 2 and 3 (from 60.9% to 68.6% correct; P < .001). Residents who pursued fellowship training did not perform better than those who did not (mean correct responses in postgraduate years 2-5, respectively, 59.6% vs 62.6% [P = .40]; 67.6% vs 71.2% [P = .33]; 71.2% vs 72.0% [P = .71]; and 71.4% vs 73.7% [P = .20]). Among residents who pursued the most common fellowships, no correlation was found between postgraduate year 5 scores on the related subspecialty section compared with the rest of the study group (mean correct responses for facial plastic and reconstructive surgery, 72.5% vs 74.7% [P = .67]; for pediatric otolaryngology, 72.9% vs 71.3% [P = .79]; and for rhinology, 72.2% vs 71.2% [P = .91]).
Conclusions and Relevance
Residents who pursued fellowship training did not achieve higher scores on the OTE in any examination year compared with residents who did not pursue fellowship training and did not achieve higher scores within the OTE section that matched their chosen subspecialty in their last year of training. Fellowship choice does not appear to influence OTE scores among residents.
Every March, otolaryngology residents in the United States take the Otolaryngology Training Examination (OTE). The American Board of Otolaryngology offers this examination and reports that its purpose is to provide an “assessment of current knowledge in all areas of otolaryngology–head and neck surgery.”1 Residents (and program directors) are expected to “use the OTE to establish a baseline of specialty knowledge at the beginning of training, and to monitor changes in that knowledge over time”.1 Since 2010, the questions have been divided into 9 sections based on subspecialty areas within otolaryngology. These sections include allergy, fundamentals, head and neck, laryngology, otology, pediatrics, plastic and reconstructive, rhinology, and sleep. In addition to obtaining an overall examination score, residents receive scores within each specialty section. These sections correlate well with the 8 surgical fellowships offered within otolaryngology programs, which include allergy and immunology, head and neck surgery, laryngology, otology and/or neurotology, pediatric otolaryngology, facial plastic and reconstructive surgery, rhinology, and sleep.
Several studies2,3 have investigated factors that influence fellowship choice within other medical specialties. These factors have been found to be loan indebtedness, prospects of future income, desire to acquire refined skills, work hours, career in a community vs a university setting, and the prestige of the subspecialty field. The role of the American Board of Surgery In-Training Examination in fellowship applications has also been reviewed, but performance of residents in subspecialty sections of that examination has not been scrutinized or correlated with final fellowship choice.4 Within otolaryngology, a cross-sectional survey was used to assess fellowship training and career track preferences among residents. The investigators5 reported that fellowship interest decreased and academic career interest increased during the course of residency and that men had an increased interest in fellowship and academic practices compared with women by the end of residency training. Factors affecting these trends were not identified.
In the field of higher education, we may expect students to gravitate toward a particular area of interest. Often, the student will perform well within that area of study and subsequently follow a career path in that field. In addition, a resident may pursue fellowship training to receive more training in an area of perceived deficit, but we anticipate that, in most cases, an affinity for the subspecialty drives fellowship choice. We hypothesize that residents pursuing specific fellowship training will demonstrate a correlation between their fellowship choice and the corresponding specialty-specific OTE score. We expect this correlation to occur later in residency once applicants have decided the fellowship to which they will apply. To our knowledge, no study has attempted to assess this relationship. Given that fellowship application is a competitive process, we also wished to determine whether residents pursuing fellowship training perform better overall on the OTE.
We collected OTE scores and fellowship data from residents who were enrolled in the University of Colorado otolaryngology residency program from July 1, 2003, to June 30, 2014. Three residents were excluded from the analysis because they did not complete residency. Residents who graduated in the years 2003 to 2005 did not have OTE scores from all the program years and were excluded from the comparison of yearly OTE scores. The Colorado Multiple Institutional Review Board approved this study and waived the need for consent.
We collected the percentage of correct scores and scaled scores for each subspecialty section of the OTE and overall. Study data were collected from September 1 to October 15, 2014, and managed using Research Electronic Data Capture (REDCap) tools hosted at the University of Colorado.6 REDCap is a secure, web-based application designed to support data capture for research studies. Data were analyzed using JMP software (version 11.0; SAS Institute, Inc) from October 16 to December 1, 2014. Univariate analysis was performed with a P value of .05 indicating significance.
A total of 35 residents met inclusion criteria, including 24 men (69%) and 11 women (31%). Demographic characteristics of these participants are listed in Table 1. A mean of 3 residents graduated per year, with exceptions occurring in 2008 and 2013, when 2 residents each graduated, and in 2014, when 4 residents graduated. Among the 17 residents who chose fellowship training, the most common fellowship subspecialty choices were facial plastic and reconstructive surgery, pediatric otolaryngology, and rhinology, each with 4 residents (24%). Eleven men and 6 women pursued fellowship training, with no significant difference by sex (P = .63).
We examined mean OTE scores for each year beginning with postgraduate year (PGY)–2. This analysis excluded residents who graduated before 2006 because we did not have all yearly scores for these residents. Mean scores increased with each program year, with the largest improvement from PGY-2 to PGY-3. Matched-pair analysis revealed a significant change in the percentage of correct responses from PGY-2 to PGY-3 (60.9% to 68.6%; P < .001), whereas the changes for other years were not significant (Table 2). Mean scaled scores were different for PGY-2 to PGY-3 (5.86 vs 6.20; P < .001) and for PGY-3 to PGY-4 (6.20 vs 6.37; P = .03) comparisons but not for PGY-4 to PGY-5 (6.37 vs 6.33; P = .53) comparisons. We then compared overall yearly scores between residents by pursuit of fellowship training. The percentage of correct responses and scaled scores were not different when comparing residents who pursued fellowship training vs those who did not seek further training (Table 3).
Finally, we analyzed scores between residents who pursued the 3 most common fellowship types in our data set, that is, facial plastic and reconstructive surgery, pediatric otolaryngology, and rhinology. We did not analyze other fellowship specialties because these groups were believed to be too small for meaningful comparisons. For each group, we analyzed mean PGY-5 scores in the corresponding subspecialty section; for example, for rhinology, we compared the PGY-5 rhinology subspecialty scores for residents with a rhinology fellowship with PGY-5 rhinology scores for all residents without a rhinology fellowship. Residents pursuing any of the 3 most common fellowship specialties did not have higher mean PGY-5 specialty-specific scores compared with the other residents (Table 4).
Our study population included more than 10 years of residents at a single academic otolaryngology training program. Half of the participants attended a fellowship program, with facial plastic and reconstructive surgery, pediatric otolaryngology, and rhinology as the most common fellowship types. This finding is similar to those from the Section for Residents and Fellows-in-Training (SRF) Annual Survey, which reported that 60% to 70% of respondents had already matched or planned to pursue fellowship training.7 Two of the 3 most popular fellowship choices in our data set matched with the 3 most popular choices in the SRF Annual Survey (facial plastic and reconstructive surgery and pediatric otolaryngology).7 Pursuit of fellowship training did not vary by the sex of the trainee in our database or the SRF Annual Survey.7
We expected overall mean OTE scores to increase in each program year. Although we found this increase, a significant increase was present for the percentage of correct responses and mean scaled scores only from PGY-2 to PGY-3. We found no change in the percentage of correct responses for the other years, whereas the change in the mean scaled score was significant from PGY-3 to PGY-4 but not from PGY-4 to PGY-5. The lack of significant change during the later years of residency is most likely owing to a narrower spread of scores. For example, the mean percentage of correct responses on the examination for the PGY-2 class was 60.9%, whereas for the PGY-3 class it was 68.6%. This difference is intuitively a significant jump. The percentage of correct responses for PGY-4 and PGY-5 were 71.2% and 71.6%, which constitute a less robust increase. We were not able to analyze the difference between PGY-1 and PGY-2 because PGY-1 scores were not available, but we would expect a significant change between these mean scores. A steeper learning curve appears to exist during the first 2 years of dedicated otolaryngology residency (PGY-1 having limited exposure to our field), and our findings likely reflect this statement.
When comparing residents who pursued fellowship training of any type with those who did not, we found no difference in overall OTE scores. This difference is likely owing to several reasons. First, unlike fellowships in general surgery,4 otolaryngology fellowship directors do not review OTE scores, so fellowship applicants may sacrifice time studying for the examination to spend more time on research and publications that will strengthen their applications. Second, even if fellowship applicants are spending more time studying their subspecialty topic of choice, this increase will lead to less time spent on other subjects and an unchanged overall OTE score. Finally, all residents, regardless of fellowship aspirations, are similarly driven to perform well on the OTE as practice for board certification, making differences between groups less likely.
We compared mean scores for residents who completed a specific fellowship with mean scores for all residents who did not. This comparison was possible owing to a division of the OTE by topics that closely reflect fellowship choices. We only analyzed the most popular fellowship specialties. Even this restriction provided complete data only for the residents who completed a facial plastic and reconstructive surgery fellowship. Because the OTE examination topics did not include pediatric otolaryngology or rhinology before 2010, each of these fellowship specialties had score data for only 2 of the 4 residents. We compared PGY-5 scores because, by this time within residency, residents already knew they had matched for their fellowship. We found no difference in PGY-5 OTE scores within the facial plastic and reconstructive surgery, pediatric otolaryngology, or rhinology sections when comparing residents who had pursued the corresponding fellowship training vs the rest of the resident group. In fact, the PGY-5 facial plastic and reconstructive surgery subspecialty scores for residents who did not pursue similar fellowship training were higher than those who did (but this difference was not statistically significant). Again, this finding likely reflects the fact that OTE scores are not evaluated by fellowship directors, so residents may have less incentive to study. Another possible reason for this finding is that, despite an increased interest by a fellowship applicant within a specific topic, training programs are required to have parity, or equal numbers of cases and exposures, for each resident. Residents are unlikely to be able to skew the amount of knowledge significantly within a topic before fellowship training owing to other duty requirements and rotation assignments.
This study had several limitations. The number of participants in this study limited its power. Specifically, the number of residents pursuing each type of fellowship training was small. Changes to the topics within the OTE that occurred in 2010 meant that our already small number of fellows within each specialty were further limited for analysis. Inclusion of similar data from other residency programs would be useful to confirm our results. We were also unable to obtain data about residents who applied to fellowship training but did not match. We do not know whether any of our included residents did not match. Finally, OTE scores are affected by numerous factors, all of which can confound our analysis of one specific variable.
Within otolaryngology, OTE scores do not correlate with pursuit of fellowship training or fellowship specialty choice. In our study, we found a significant improvement in total OTE scores from PGY-2 to PGY-3. The OTE scores should continue to be used for personal assessment of otolaryngology resident knowledge but are not helpful in predicting postresidency career choices.
Corresponding Author: Cristina Cabrera-Muffly, MD, Department of Otolaryngology–Head and Neck Surgery, University of Colorado School of Medicine, 12631 E 17th Ave, Mail Stop B205, Aurora, CO 80045 (email@example.com).
Submitted for Publication: October 13, 2015; final revision received November 18, 2015; accepted November 25, 2015.
Published Online: February 4, 2016. doi:10.1001/jamaoto.2015.3570.
Author Contributions: Dr Cabrera-Muffly had access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Cabrera-Muffly.
Acquisition, analysis, or interpretation of the data: Both authors.
Drafting of the manuscript: Akinboyewa.
Critical revision of the manuscript for important intellectual content: Cabrera-Muffly.
Statistical analysis: Cabrera-Muffly.
Administrative, technical, or material support: Cabrera-Muffly.
Study supervision: Cabrera-Muffly.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by Colorado Clinical and Translational Science Institute grant UL1 TR001082 from the National Center for Advancing Translational Sciences, National Institutes of Health (NIH).
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Disclaimer: The contents are the authors’ sole responsibility and do not necessarily represent official views of the NIH.
Previous Presentation: This paper was presented at the 2015 annual meeting of the American Academy of Otolaryngology–Head and Neck Surgery Foundation; September 29, 2015; Dallas, Texas.
Create a personal account or sign in to: