The top 5 most often cited study sources indicated as a primary or secondary source for the American Board of Surgery In-Training Examination (ABSITE) or patient care (4 sources cited by each respondent). SCORE indicates Surgical Council on Resident Education; SESAP, Surgical Education and Self-Assessment Program.
Respondent level of satisfaction with their primary or secondary study source, whether being used for patient care or American Board of Surgery In-Training Examination (ABSITE) studying, had a direct positive correlation with median ABSITE percentile scores. The perceived level of importance placed on ABSITE results also had a significant effect on the median percentile scores.
eAppendix. Survey distributed to residents
Kim JJ, Kim DY, Kaji AH, Gifford ED, Reid C, Sidwell RA, Reeves ME, Hartranft TH, Inaba K, Jarman BT, Are C, Galante JM, Amersi F, Smith BR, Melcher ML, Nelson MT, Donahue T, Jacobsen G, Arnell TD, de Virgilio C. Reading Habits of General Surgery Residents and Association With American Board of Surgery In-Training Examination Performance. JAMA Surg. 2015;150(9):882-889. doi:10.1001/jamasurg.2015.1698
Copyright 2015 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Few large-scale studies have quantified and characterized the study habits of surgery residents. However, studies have shown an association between American Board of Surgery In-Training Examination (ABSITE) scores and subsequent success on the American Board of Surgery Qualifying and Certifying examinations.
To identify the quantity of studying, the approach taken when studying, the role that ABSITE preparation plays in resident reading, and factors associated with ABSITE performance.
Design, Setting, and Participants
An anonymous 39-item questionnaire including demographic information, past performance on standardized examinations, reading habits, and study sources during the time leading up to the 2014 ABSITE and opinions pertaining to the importance of the ABSITE was administered August 1, 2014, to August 25, 2014, to 371 surgery residents in 15 residency programs nationwide.
Main Outcomes and Measures
Scores from the 2014 ABSITE.
A total of 273 residents (73.6%) responded to the survey. Seven respondents did not provide their January 2014 ABSITE score, leaving 266 for statistical analysis. Most respondents were male (162 of 266 [60.9%]), with a mean (SD) age of 29.8 (2.6) years. The median number of minutes spent studying per month was 240 (interquartile range, 120-600 minutes) for patient care or clinical duties and 120 for the ABSITE (interquartile range, 30-360 minutes). One hundred sixty-four of 266 respondents (61.7%) reported reading consistently throughout the year for patient care or clinical duties. With respect to ABSITE preparation, 72 of 266 residents (27.1%) reported reading consistently throughout the year, while 247 of 266 residents (92.9%) reported preparing between 1 and 8 weeks prior to the examination. Univariate analysis (with results reported as effect on median ABSITE percentile scores [95% CIs]) identified the following factors as positively correlated with ABSITE scores: prior United States Medical Licensing Examination (USMLE) 1 and 2 scores (per 1-point increase: USMLE 1, 0.1 [0.02-0.14], P = .03; USMLE 2, 0.3 [0.19-0.44], P < .001), prior Medical College Admission Test (MCAT) scores (per 1-point increase, 1.2 [1.3-2.0]; P = .002), high opinion of ABSITE significance (P < .001), surgical textbook use (11 [6-16]; P = .02), daily studying (13 [4-23]; P = .02), and high satisfaction with study materials (P < .001). On multivariable analysis, USMLE 2 score (per 1-point increase, 0.4 [0.2-0.6]; P < .001), MCAT score (0.6 [0.2-1.0]; P = .003), opinion of ABSITE significance (9.2 [6.9-11.6]; P < .001), and having an equal focus on patient care and ABSITE preparation during study (6.1 [0.6-11.5]; P = .03) were identified as positive predictors of ABSITE performance.
Conclusions and Relevance
Most residents reported reading consistently for patient care throughout the year. Daily studying and textbook use were associated with higher ABSITE scores on univariate analysis. Scores on the USMLE 2 and MCAT, as well as resident attitude regarding the importance of the ABSITE results, were independent predictors of ABSITE performance.
In 2003, the Accreditation Council for Graduate Medical Education implemented the 80-hour work week for all US residents. One of the perceived potential benefits of this duty-hour restriction was an increase in the time available for residents to read and strengthen clinical knowledge, participate in educational activities, and prepare for standardized examinations.1
The American Board of Surgery In-Training Examination (ABSITE) is one such standardized examination that is administered to all general surgery residents in the United States each year. The stated purpose of the ABSITE is evaluation of resident progress with respect to surgical knowledge during the formative training years.2 Performance on the ABSITE also has been shown to affect future performance on the American Board of Surgery Qualifying Examination (ABS-QE).3- 5 Furthermore, a survey of fellowship program directors revealed a strong emphasis on applicants’ ABSITE scores during the selection process.6
To date, to our knowledge, there has been little research reporting on study habits and reading practices of surgery residents, especially in the era of the 80-hour work week. Furthermore, despite the importance of the ABSITE, there is a lack of consensus on preparation strategies that are associated with higher performance. Therefore, the objectives of our study were to identify the quantity of studying, the approach taken when studying, the role that ABSITE preparation plays in resident reading, and factors associated with ABSITE performance.
A 39-item questionnaire was developed for administration to general surgery residents who had taken the ABSITE during the previous academic year (January 2014) to identify factors that were potentially associated with ABSITE performance. Background information collected included postgraduate training year, age, sex, subspecialty interest, and whether the resident was in a research year. The survey also included questions about study habits of residents, including whether the primary focus of reading was directed toward clinical patient care, toward the ABSITE, or equally divided between the two; primary and secondary study sources used for reading (eg, standard surgery textbook, Surgical Council on Resident Education [SCORE] curriculum, ABSITE review book); satisfaction with study material; and a quantification of reading effort (frequency per week, reading throughout the year vs just before the examination, and average duration of each reading session); as well as potential barriers to studying for the 2014 ABSITE, such as children at home. Prior performance on standardized tests (United States Medical Licensing Examination [USMLE] and Medical College Admission Test [MCAT]) was also requested. Finally, residents were asked their opinion with respect to the significance of the ABSITE, history of remediation owing to poor ABSITE score, and their actual 2014 ABSITE score (percentage correct and percentile rank). Question formats were multiple choice, Likert scale, yes or no, and free text or numerical response. Questions answered in Likert scale format included 4 response choices (1, strongly disagree; 2, disagree; 3, agree; and 4, strongly agree). The survey (eAppendix in the Supplement) was uploaded and disseminated via SurveyMonkey (https://www.surveymonkey.com) in an anonymous format, although response rates from specific programs were tracked.
Program directors from general surgery residencies across the nation were recruited for participation in the study. Programs were chosen based on a history of cooperation with such surveys, a desire to represent both academic and independent programs, program size (both large and small programs), and geographical diversity of the program. Participating program directors were then asked to send emails to residents within their program with a brief statement of purpose and a link to the anonymous survey from August 1, 2014, to August 25, 2014. A single study coordinator (J.J.K.) was responsible for dissemination of surveys to each program director, tracking response rates from each program, and management of response data. The study was approved by the Los Angeles Biomedical Institute at Harbor–University of California at Los Angeles Human Subjects Institutional Review Board prior to initiation of the research.
Data management was performed with Microsoft Excel 2007 (Microsoft Corp). Statistical analysis of survey responses was performed with SAS, version 9.3 (SAS Institute, Inc). The total number of minutes spent studying per month by each resident was extrapolated by multiplying the frequency of study sessions and the self-reported average length of each session. Descriptive statistics were performed on the data. In addition, to assess the association with ABSITE scores, we used the Kruskal-Wallis test, Fisher exact test, and Wilcoxon rank sum test, as appropriate. The ABSITE scores were treated as continuous variables, with percentile score used for statistical analysis. Variables that were significant on univariate analysis and determined to be appropriate for inclusion were entered into a multivariable analysis. Adjustment for clustering of scores within programs was also performed using generalized estimating equations. The effect of the presence or absence of categorical variables on median percentile ABSITE scores was calculated, along with 95% CIs. For continuous variables, the effect that every 1-point change had on median ABSITE percentile scores was calculated with 95% CIs. P < .05 was considered statistically significant.
Program directors at 15 general surgery residencies within the United States were contacted about participation in the study, and all 15 agreed to participate. The survey was sent to 371 general surgery residents who had taken the 2014 ABSITE. The response rate was 73.6% (273 respondents) overall. Seven respondents did not provide their January 2014 ABSITE score, leaving 266 for statistical analysis.
The mean (SD) age of respondents was 29.8 (2.6) years, and 162 (60.9%) were male (Table 1). Few respondents reported having children at home (47 [17.7%]) or having taken an ABSITE preparatory course (16 [6.0%]). Prior mandatory remediation was self-reported by 39 residents (14.7%), with most reporting low ABSITE scores as the cause of remediation (35 [13.2%]). There was an even distribution of postgraduate levels 1 through 4 (54 [20.3%] to 68 [25.6%]), with 24 respondents (9.0%) reporting postgraduate year 5 or above at the time of the January 2014 ABSITE. Mean (SD) MCAT score was 32.1 (4.3), mean (SD) USMLE 1 score was 231.1 (36.0), and mean (SD) USMLE 2 score was 238.9 (24.3). The mean (SD) self-reported ABSITE percentile score was 61.0 (26.5), and the median was 67. Quiz Ref IDIn terms of study focus, 49 residents (18.4%) indicated that their studying was focused primarily on ABSITE preparation, 153 (57.5%) indicated that their studying was primarily for patient care or clinical duties, and 63 (23.7%) indicated equal weight between the two. When extrapolated from the frequency and duration of self-reported studying, the total median number of minutes spent studying for any reason was 480 (interquartile range, 200-960 minutes).
One hundred seven (40.2%) respondents indicated having a year-round reading schedule that they followed and most (218 [82.0%]) read on a weekly or daily basis (Table 2). Quiz Ref IDWhen extrapolated from the frequency and duration of self-reported studying, the median number of minutes studied per month for clinical duties was 240 (interquartile range, 120-600 minutes). The most often cited primary study sources used for patient care or clinical duties included a surgical textbook (139 [52.3%]), SCORE curriculum (43 [16.2%]), and an ABSITE review book (40 [15.0%]).
Fifty-six residents (21.1%) reported adhering to a year-round ABSITE study schedule and 40 (15.0%) reported studying daily for the ABSITE (Table 3). Many residents reported studying sporadically (91 [34.2%]) or once weekly (106 [39.8%]). Quiz Ref IDThe average length of time spent during each study session varied, but most residents indicated study sessions up to 30 minutes (74 [27.8%]) or between 31 and 60 minutes (116 [43.6%]). The median number of minutes spent studying per month for the ABSITE was 120 (interquartile range, 30-360 minutes) when extrapolated from the frequency and duration of study sessions. An ABSITE review book was the most often selected primary study source (126 [47.4%]). When asked about perceived importance of the ABSITE, most respondents indicated a desire to perform well on the examination by responding, “I want to do well, but don’t feel it will significantly affect my career goals” (148 [55.6%]), followed by “I must do well because it is important in achieving future career goals” (96 [36.1%]). Finally, when asked about the timing of ABSITE preparation, 187 respondents [70.3%] indicated “agree” or “strongly agree” to the statement, “I prepared by reading regularly for 1-2 months before the ABSITE.” Fifteen residents (5.6%) agreed with the statement, “I did not prepare at all.”
Responses given when asked to rate the level of satisfaction with primary and secondary study sources are shown in Figure 1. Although rapid-access Internet sources (search engine, PubMed, and peer-reviewed Internet sources) were not chosen as often as formal review materials, they were rated with the highest satisfaction levels, with 94.3% to 97.1% of residents indicating “satisfied” or “strongly satisfied.” Those using a surgical textbook (n = 254) also indicated “satisfied” or “strongly satisfied” 92.9% of the time.
Factors positively associated with ABSITE scores on univariate analysis included (with results reported as effect on median ABSITE percentile scores [95% CIs]) prior USMLE 1 and 2 scores (per 1-point increase: USMLE 1, 0.1 [0.02-0.14], P = .03; USMLE 2, 0.3 [0.19-0.44], P < .001), prior MCAT scores (per 1-point increase, 1.2 [1.3-2.0]; P = .002), having an equal study focus on ABSITE and patient care (11 [7-15]; P = .009), daily studying for patient care or clinical duties (13 [4-23]; P = .02), use of a surgical textbook (11 [6-16]; P = .02), level of satisfaction with study materials, and perceived ABSITE significance (Table 4). Quiz Ref IDFactors negatively correlated with ABSITE scores included prior ABSITE remediation (–26 [–36 to –16]; P = .002), lack of studying as indicated by agreement to the statement, “I did not prepare at all” (–12 [–21 to –9]; P < .001), use of SCORE questions as a primary source (–14 [–19 to –9]; P = .01), use of an Internet search engine as a primary source (–21 [–30 to –13]; P = .04), and primary focus on patient care or clinical duties when studying (–9 [–14 to –5]; P = .009).
The positive correlation exhibited between level of satisfaction and perception of ABSITE significance to ABSITE percentile scores was especially strong (Figure 2). Respondent level of satisfaction with their primary or secondary source, whether being used for patient care or ABSITE studying, had a direct positive correlation with median ABSITE percentile scores (P < .001). Furthermore, the perceived level of importance placed on ABSITE results (possible responses: 1, “my ABSITE score doesn't matter to me at all;” 2, “I just need to pass to avoid disciplinary measures;” 3, “I want to do well, but don’t feel it will significantly affect my career goals;” and 4, “I must do well because it is important in achieving future career goals”) had a significant effect on the median percentile score (P < .001). Despite the strong correlation that satisfaction with source material had on ABSITE scores, use of an Internet search engine (top satisfaction level [97.1%]) actually had a detrimental effect on ABSITE performance.
Factors without a significant association with ABSITE scores included age, sex, having children at home, specialty interest, use of an ABSITE review book, having taken an ABSITE preparatory course, adhering to a year-round study schedule, and average length of study sessions. In addition, research year status was also not associated with ABSITE scores.
The most commonly cited barrier to studying was resident work hours (85 responses [32.0%]), followed by the desire to spend time with a significant other or spouse (67 [25.2%]). Lack of motivation was chosen least often as a significant barrier to studying (25 [9.4%]).
Based on significance from univariate analysis, the following variables were entered into multivariable linear regression analysis: USMLE 1 and 2 scores, MCAT scores, prior ABSITE remediation, opinion of ABSITE’s significance, use of clinical textbook, use of SCORE questions, whether reading was clinically focused, having an equal study focus on ABSITE and patient care, as well as potential within-program resident correlations.
Quiz Ref IDMultivariable analysis identified 4 independent predictors of ABSITE performance, all with a positive correlation (with results reported as effect on median ABSITE percentile scores [95% CIs]): USMLE 2 score (per 1-point increase, 0.4 [0.2-0.6]; P < .001), MCAT score (0.6 [0.2-1.0]; P = .003), having an equal study focus on ABSITE and patient care or clinical duties (6.1 [0.6-11.5]; P = .03), and perceived ABSITE significance (9.2 [6.9-11.6]; P < .001) (Table 5). Of these predictors, resident perception of the importance of the ABSITE results seemed to have the most significant effect, with a 9.2–percentile point increase in ABSITE score for each increase in perceived level of significance (of 4 levels).
This study reviewed perceptions, attitudes, and study habits of 266 residents at 15 general surgery residency programs across the United States. Respondents reported a median of 480 minutes spent studying in total for any reason each month (median for patient care, 240 minutes; for ABSITE, 120 minutes). Most residents (61.7%) reported consistently studying throughout the year for patient care, while only 27.1% reported consistent year-round studying for the ABSITE. Overall, residents expressed a desire to perform well on the ABSITE and a lack of motivation was rarely cited as a barrier to preparation. Specific factors associated with success were identified on univariate analysis. These factors included a history of high performance on standardized testing (MCAT and USMLE 2), studying on a daily basis, use of a surgical textbook, and a high level of satisfaction with the study source being used. However, multivariable analysis showed that the effect of resident perception of how important the ABSITE results were for achieving future goals was particularly strong. Indeed, these perceptions likely set the foundation on which ABSITE-related study habits and strategies are built. Therefore, it is not surprising that daily studying and textbook use are not independent predictors of performance, as these variables are probably closely linked to the attributes that lead to high ABSITE performance in the first place. Interestingly, the overall minutes per month spent studying or having taken an ABSITE preparatory course did not correlate with scores.
Currently, a comprehensive assessment of surgical residents’ study habits is lacking. Previous studies assessing surgery residents’ study patterns have focused on ABSITE-related preparation7,8 and have not necessarily distinguished between preparation for clinical duties vs for ABSITE. Furthermore, the sample sizes of these studies have been small, generally less than 60 residents. A larger-scale study investigating the use of various study sources was conducted by Glass et al,9 with 773 respondents. However, that study did not quantify the amount of studying, and associations with respect to ABSITE performance were not investigated. Therefore, the question of how much residents study and how their time is distributed with respect to study materials and ABSITE preparation has not previously been answered.
There is mounting evidence suggesting that ABSITE scores play a meaningful role in achieving future career goals. A 2008 study showed that scoring below the 35th percentile on the ABSITE on more than 1 occasion was predictive of failing the ABS-QE.3 Furthermore, scoring below the 25th percentile was predictive of failing both the ABS-QE and the Certifying Examination (CE) in that study. A subsequent retrospective review of 607 surgical residents further validated these findings and showed that scoring below the 35th percentile on the ABSITE at any point was predictive of failing both the ABS-QE and ABS-CE.4 A more recent study reviewing more than 6000 residents’ ABSITE, ABS-QE, and ABS-CE scores concluded that high ABSITE scores are highly predictive of success on the ABS-QE, but low ABSITE scores were less reliable in predicting subsequent ABS-QE failure.5 Given the literature, ABSITE scores clearly have a distinct association with performance on the ABS examinations. Furthermore, a survey-based study of 148 surgical fellowship program directors across all subspecialties ranked the ABSITE as the third most important component of acceptance to a fellowship, behind letters of recommendation and residency program attended.6 In that report, the ABSITE was noted to carry greater weight with fellowship directors than did publications. Given the increasing number of general surgery graduates entering fellowship programs, the importance of the ABSITE is likely to become more relevant to current surgical residents.
Despite its significance, residents currently are not knowledgeable about how to best prepare for the ABSITE. In 2014, Simpson-Camp and colleagues10 showed that residents were unable to accurately predict their ABSITE performance immediately prior to, or even after taking the examination. Most residents overestimated their performance. This overestimation reflects the poor understanding possessed by residents of what constitutes adequate vs inadequate preparation. To exacerbate the matter, contemporary studies on the topic have assessed factors that are not necessarily under the control of residents. Programmatic factors assessed include weekly reading assignments and quizzes administered by the program11 and structured study materials provided to residents by the program director.12 Furthermore, reports of USMLE scores as predictors of ABSITE performance are also unhelpful in guiding examination preparation strategies.13- 15 Studies that have reported on modifiable factors have been small (<60 residents) and have only addressed a single aspect of study strategy (amount of studying or number of practice questions completed during intern year).8,16 Needless to say, an important aim of our study was to comprehensively review resident study strategies to identify the most influential methods of preparation, whether advantageous or detrimental.
There are some limitations to our study. The potential for recall bias is present with any study involving self-reported information. Respondents with poor ABSITE scores may have underreported the amount of studying or their perceived importance of the examination, either knowingly or subconsciously. However, by addressing and asking for ABSITE scores and opinions after all other questions have been answered, we hope to have minimized the effects of recall bias. Furthermore, owing to the voluntary basis of the survey we do not know if data from nonresponders would change our findings. Our respondents’ mean ABSITE percentile score of 61.0 and median of 67 may indicate lower performance in the nonresponders. Alternatively, the mean and median scores may have been owing to not capturing a broad enough sample. Fortunately, the high response rate (73.6%) likely mitigated this issue as well.
Residents spent more consistent time reading for patient care or clinical duties while ABSITE preparation was more focused prior to the examination. Programs must place greater emphasis on the significance of the ABSITE to shift resident perceptions and increase performance. Residents should evaluate and reassess the importance that the examination has on their future career goals. In addition, daily reading, the use of a surgical textbook, and studying from materials that residents were satisfied with had positive correlations with ABSITE performance on univariate analysis.
Accepted for Publication: March 25, 2015.
Corresponding Author: Christian de Virgilio, MD, Department of Surgery, Harbor–University of California at Los Angeles Medical Center, 1000 W Carson St, Torrance, CA 90502 (email@example.com).
Published Online: July 15, 2015. doi:10.1001/jamasurg.2015.1698.
Author Contributions: Drs de Virgilio and J. J. Kim had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: J. J. Kim, D. Y. Kim, Gifford, Reid, Jarman, Galante, Melcher, Nelson, de Virgilio.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: J. J. Kim, Reid, de Virgilio.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: J. J. Kim, Kaji, Donahue.
Administrative, technical, or material support: J. J. Kim, Gifford, Reid, Reeves, Nelson, de Virgilio.
Study supervision: Jarman, Smith, Nelson, Donahue, Jacobsen, Arnell, de Virgilio.
Conflict of Interest Disclosures: Dr de Virgilio is the author of an American Board of Surgery In-Training Examination review book.
Previous Presentation: This study was presented at the 86th Annual Meeting of the Pacific Coast Surgical Association; February 22, 2015; Monterey, California.