A clinical score was given to each student during the Pediatric clerkship and is shown as a point in the appropriate specialty group. The Pediatrics group had significantly higher scores when compared with the other groups. Data are displayed graphically to demonstrate the distribution of the individual points around the 50th quantile, or the median, located in the middle of each quantile box and the total sample mean represented by the horizontal line that crosses all 3 quantile boxes. Statistical significance is also graphically displayed.
The mean medical history and physical examination scores for each student during the pediatric clerkship are shown as a point in the appropriate specialty group. The pediatric and internal medicine group had significantly higher scores when compared with the family medicine group. Data are displayed graphically to demonstrate the distribution of the individual points around the 50th quantile, or the median, located in the middle of each quantile box and the total sample mean represented by the horizontal line that crosses all 3 quantile boxes. Statistical significance is also graphically displayed.
Deterding R, Kamin C, Barley G, Adams L, Dwinnell B, Merenstein G. Effect of a Longitudinal Course on Student Performance in Clerkships. Arch Pediatr Adolesc Med. 1999;153(7):755-760. doi:10.1001/archpedi.153.7.755
To determine the effect that a 3-year primary-care course experience with family medicine, internal medicine, or pediatric preceptors would have on clerkship performance in pediatrics and internal medicine.
In 1 academic year, third-year students were divided retrospectively into 3 groups based on preceptor type in the primary care course. An analysis of variance was conducted. When the analysis of variance showed statistical significance, a multiple-comparison t test was performed.
University medical school with a longitudinal preceptor experience.
One hundred nine third-year medical students who participated in the primary care course and completed the pediatric and internal medicine clerkships. Fifty-six students took part in the self-assessment portion of the study.
Main Outcome Measures
Student performance scores in the pediatric clerkship and internal medicine clerkship were analyzed for significant differences based on preceptor type. Student self-assessment on pediatric objectives was analyzed for significant differences based on preceptor experience.
Students with pediatric preceptors received higher clinical scores in the pediatric clerkship (P=.04) and perceived themselves as more advanced on 18 of the 39 pediatric curriculum pretest self-assessment items. Students with pediatric or internal medicine preceptors received significantly higher scores on the written patient medical history and physical examinations (P=.02). There were no significant differences on the pediatric written examination. There were no significant performance differences in the internal medicine clerkship. All hypothesis testing was conducted at the 95% confidence level.
Experiences with pediatric preceptors in the early years of medical school may improve a student's performance and confidence in the pediatric clerkship.
MEDICAL schools around the country have initiated curriculum reform to incorporate principles of generalism as a foundation for all medical students.1 Educators at the University of Colorado, Denver, as part of the Interdisciplinary Generalist Curriculum Project funded by the Health Resources and Services Administration, implemented a longitudinal primary care course that would expose first-, second-, and third-year students to generalist role models in family medicine, internal medicine, and pediatrics.2 This change was based on new assumptions that all students should have early and longitudinal community-based experience to increase interest in generalist careers and to promote greater integration of clinical medicine and basic science.3
As part of the primary care curriculum (PCC), students spent an average of a half day per week for 3 weeks per month in their assigned preceptor's office during the first 3 years of medical school. Preceptor specialty type was determined by the request of the student and the preceptor was assigned by the course director from a pool of volunteers recruited by the departments. During the fourth week of the month, the afternoon was spent on activities in small groups. Other than a lecture series designed to review physical examination skills that included only 1 session on the pediatric examination, physical examination skills could only be learned from the preceptor. At the end of each year, assessments of the students' ability to take a patient's medical history and to perform a physical examination, and of their professional behavior learned through clinical experiences, were made using adult standardized patients and objective structured clinical examinations.
In the 1996 academic year, the first group of students from the PCC entered third-year clinical clerkship rotations. Little data existed to determine the effect that this curriculum change would have on student performance during the clerkships. For example, how would students with an internal medicine preceptor and no experience in examining an infant or child perform in pediatrics? Conversely, how would students with a pediatric preceptor and no opportunities to evaluate adults perform in internal medicine? With these questions in mind, we hypothesized that students with no experience examining children would be limited in their abilities to achieve pediatric clerkship objectives, while those with extensive experience examining children would be more proficient and at a distinct advantage. A similar hypothesis was held for the reverse effect. In other words, students with no experience examining adults would be limited in their ability to achieve the internal medicine clerkship objectives, while those with extensive experience examining adults would be more proficient and at a distinct advantage.
We sought to test our hypotheses by investigating all measures of student performance that compose the final grade of both the pediatric and internal medicine clerkship. At the beginning of the pediatric clerkship, we also asked all students to rate their experience with the pediatric objectives to determine if students' perceptions of their clinical objectives in pediatrics would be significantly different based on preceptor type.
Third-year clerkship students for the 1996 academic year were grouped into a pediatrics group, a family medicine group, and an internal medicine group depending on a student's preceptor specialty in the PCC. Each entering student's preceptor specialty type was based on the personal choice of the student when possible. Eighteen students changed preceptors during the 3 years, but they were distributed evenly between all combinations of specialty. In the first year of the PCC, some students were placed with obstetricians. All but 1 student placed with an obstetrician switched in subsequent years to another specialty preceptor. The 1 student who remained with an obstetrician was eliminated from the study. Also excluded were any MD or PhD students or fourth-year students (n=10) who had not participated in the longitudinal preceptor experience. Seven students were excluded from the data analysis for the student performance scores in the pediatric clerkship due to incomplete course work. All 7 of these students had a family medicine preceptor.
For the pediatric clerkship analysis, those who spent any documented preceptor time with pediatricians were placed in the pediatrics group because they had some clinical experience with children (n=31). Those with a combination of experience in family medicine and internal medicine were put in the family medicine group (n=53). Students with family practice clerkships were presumed by the investigators to have had an opportunity to see children during their clerkship. The third group were students with internal medicine preceptors (n=25).
The internal medicine clerkship groups were developed using a similar rationale. Those with a combination of pediatric and family medicine preceptors were in the family medicine group (n=46) because it was expected that they would have had exposure to adult medicine. Those who had spent some PCC time with internal medicine preceptors were put in the internal medicine group (n=47). Those who had only been with pediatricians were put in the pediatric group (n=23).
Student assessment scores in the pediatric and internal medicine clerkships were retrospectively examined for differences in the groups of students who had been with a pediatric, internal medicine, or family medicine preceptor. The different components of each clerkship's individual assessment were analyzed to determine whether differences existed between these groups based on preceptor type.
The pediatric clerkship at the University of Colorado was a 6-week clinical experience that was conducted at 2 major teaching hospital sites and 3 to 4 ambulatory sites. Regardless of location, all students followed the National General Pediatric Core Curriculum4 and participated in ambulatory, newborn, and inpatient pediatric care. Student grades of honors, high pass, pass, and fail were assessed on the following:
Faculty and residents were asked by the student to evaluate his or her knowledge and critical thinking, clinical skills in taking a patient's medical history and performing a physical examination, attitude, and professional behavior. Evaluators used a scoring system from 1 to 6 for each area, with 1 as a fail and 6 as honors. Evaluators were blinded to the specialty type of the preceptor in the PCC. In fact, many of the evaluators were unaware of the change in the undergraduate curriculum. The mean of 3 to 6 clinical evaluations per student was calculated into the final grade. This mean was recorded as the individual student score on clinical performance for the purpose of this study.
Four complete patient histories and physical examination write-ups with discussions were evaluated by a resident or attending physician of the student's choice using a Likert scale from poor (0) to outstanding (10) based on established criteria for each component of the write-up. The mean of these write-ups was calculated into the student's final grade. This mean score was recorded as the individual score on write-ups for the purpose of this study.
A multiple-choice examination was given at the end of the clerkship on knowledge learned primarily from lectures, readings, and clinical experiences. We used this raw score as the individual score for the purpose of this study.
Midway into the academic year, preliminary data supported our hypothesis that students with pediatric preceptors would receive significantly higher clinical evaluations in the pediatric clerkship than students with other preceptors. At the beginning of the pediatric clerkship, all students ranked their experience or confidence in pediatric skills on a 39-item pretest to determine if students began the clerkship with different perceived levels of experience or confidence. These 39 items were the basic objectives and objectives for the clerkship and were based on the National General Pediatric Core Curriculum.4 Each student rated himself or herself on each item using a Likert scale from 1 (very experienced) to 5 (no experience). Because data collection began midyear, the number of students pretested (n=56) was lower than the number who completed the clerkship during this academic year (n=109). All items were analyzed for each student to determine if there were differences in the groups by preceptor type.
The internal medicine clerkship was a 12-week clinical experience that consisted of 3 rotations of 4 weeks each. Two of these rotations were hospital inpatient rotations in 1 of 6 teaching hospitals in the Denver metropolitan area. The third month varied: either another month at a hospital, a month of combined ambulatory/inpatient care at a rural preceptorship site, or a month of ambulatory primary care in the Denver metropolitan area. Topics covered were outlined in the Clerkship Directors in Internal Medicine/Society of General Internal Medicine Core Medical Clerkship Curriculum Guide.5 Grades of honors, high pass, pass, and fail were assessed using the following measures.
Attending faculty and housestaff evaluated students on a numerical scale from 1 (poor) to 6 (honors) in 4 areas: data gathering, problem solving, communication and interaction, and professional behavior. Scores from attending physicians were weighted higher than those from residents, whose scores were weighted higher than those from interns. The scores were calculated for each month separately and then averaged.
General internal medicine faculty conducted 30-minute standardized oral examinations. The examinations included 4 common internal medicine problems that students discussed in relationship to patients seen during their clerkships. Students were evaluated using a numerical scale from 1 (poor) to 6 (honors).
The National Board of Medical Examiners Self-Examination in Internal Medicine, a 2-hour multiple-choice examination of 150 questions, was taken by all clerkship students. A ranked score between 1 (poor) and 6 (honors) was assigned based on a combination of the percentile rank provided by the National Board of Medical Examiners and the rotation's distribution of scores.
A self-assessment of clerkship objectives was not completed before the internal medicine clerkship.
All hypothesis testing was performed at the 95% significance level. Statistical testing was computed in JMP Statistical Discovery Software6 and was performed for individual item scores as follows. An omnibus analysis of variance was performed to test the null hypothesis that the mean score for each group is equal. Variances within groups were tested for homogeneity, and if significant, a Welch analysis of variance, testing the same hypothesis but allowing for different variances, was used. If the analysis of variance showed statistical significance, then the Tukey test was performed to test for individual differences between all pairs of groups. This procedure allowed for conservation of the 95% significance level across multiple comparisons.
Students in the pediatrics group obtained significantly (P=.04) higher clinical performance evaluation scores in the pediatric clerkship, with a mean of 5.61, than students in either the family medicine or internal medicine groups, who scored means of 5.29 and 5.17, respectively (Figure 1). Even when the outlier from the family medicine group was removed, there were still significant differences between the groups.
As shown in Figure 2, evaluation scores for written patient histories and physical examinations in the pediatric clerkship were also statistically (P=.02) better in the pediatrics group (mean, 8.52) and in the internal medicine group (mean, 8.04) compared with the family medicine group (mean, 7.68). When the outlier from the family medicine group was removed, there were still significant differences between the family medicine group and the pediatrics and internal medicine groups.
The mean scores on the pediatric multiple-choice examination at the end of the clerkship were 83.16 for the pediatrics group, 81.34 for the family medicine group, and 82.68 for the internal medicine group. These scores were not statistically different among the 3 groups.
There were significant differences found between the student groups on 18 of the 39 self-assessment items administered at the beginning of the pediatric clerkship. The pediatrics group scored themselves significantly higher than the family medicine and internal medicine groups on the 8 items listed in Table 1. Students ranked their experience in taking a newborn's medical history and in performing an examination, doing a growth assessment, evaluating failure to thrive, asking about behavioral concerns, conducting developmental screening, calculating nutritional requirements, and asking about health maintenance issues for a adolescent. The pediatrics and family medicine groups assessed themselves as significantly more experienced than the internal medicine group in 8 areas (Table 2). These areas included assessment of growth and diet, home safety, generic medications for uncomplicated pediatric conditions, fluids and electrolyte requirements, medical problems specific to newborns, pediatric medical history taking, and physical examinations. The pediatrics group rated themselves significantly lower than both the family medicine and the internal medicine groups on the following 2 items: completing a written summary of the medical history and physical examination and orally presenting it in a focused and chronological manner; and effectively communicating the diagnosis and treatment to the patient and caregivers. Finally, there were no significant differences between the groups on the remaining 21 items (Table 3).
In contrast to the pediatric clerkship, there were no statistical differences between the groups based on preceptor type on any of the student assessment measures used in the internal medicine clerkship. On the clinical performance evaluations, mean scores were 5.47 for the pediatrics group, 5.46 for the family medicine group, and 5.48 for the internal medicine group. On the oral examination, mean scores were 5.25 for the pediatrics group, 5.24 for the family medicine group, and 5.37 for the internal medicine group. On the written examination, mean scores were 4.94 in the pediatrics group, 5.17 in the family medicine group, and 5.03 in the internal medicine group.
The experience with a pediatric primary care preceptor had a positive effect on clinical performance evaluations in the pediatric clerkship, pediatric written medical history and physical examinations, and the student's overall perceptions of their clinical knowledge and skills in pediatrics. Students with family medicine or internal medicine preceptors did not receive these same benefits. All specialties did not seem equal in their ability to enhance a student's performance in pediatrics. Curiously, preceptor type did not influence performance evaluations in the internal medicine clerkship.
Why, then, did the selection of a pediatric preceptor positively influence a student's performance in the pediatric clerkship? An obvious answer is that students in the pediatrics group had a longitudinal clinical experience with a preceptor whose focus is the well-being of children. This experience was not reproduced by internal medicine preceptors, who may provide care to adolescents but not children, or family medicine preceptors, who may provide care to a varying number of children and whose practice is less focused on the care of children. The students in the family medicine and internal medicine groups were not able to compensate for their lack of clinical pediatric experience through other medical school activities in the first 2 years of medical school. Pediatric patient skills were not assessed before the third-year clerkships, while adult patient skills were assessed.
Physical examination skills are routinely taught and tested with adult patients at most medical schools. Adolescent or adult patients are typically used in objective structured clinical examinations and as standardized patients to evaluate clinical performance, because it is unrealistic to expect an infant or toddler to participate in student examinations.
Many students also do not have much experience with children outside of medical school. A survey done at the DuPont Hospital for Children, Wilmington, Del, found that a large percentage of medical students lacked basic pediatric experiences outside of the medical school environment.7 The lack of experience was attributed to smaller families and postponement by medical students in having children.
Although we believe that differing levels of experience with children was the biggest reason for differences between the groups, 2 other possible explanations deserve mention. First, students selected a preceptor specialty type. Perhaps the students who selected pediatricians intended to pursue pediatrics as a career and were more motivated to perform well in the pediatric clerkship. Second, the pediatric clerkship is only 6 weeks long. Students with other specialty preceptors may not have had sufficient time in the clerkship to establish the confidence or skill level that the pediatrics group achieved.
Why, then, were there no differences between the groups in performance evaluations for the internal medicine clerkship? It is possible that when students have a longer clerkship, the 12 weeks spent in the department of internal medicine at the University of Colorado enabled them to "catch up" to their classmates in the internal medicine group. Also, students with pediatric preceptors supplement early clinical experiences by learning how to take medical histories from adult patients and how to perform physical examinations on adults in preparation for the adult standardized patient examinations at the end of each year. Additionally, the clinical experiences in the PCC are almost exclusively in the ambulatory setting. The internal medicine clerkship is predominately in an inpatient setting, possibly diminishing the effect of all student experiences in the PCC. This, however, was not observed in the pediatric clerkship, which also had components of inpatient and ambulatory experiences.
We believe that the self-assessment reflects a student's confidence level in addressing specific pediatric objectives and that this confidence has its basis in previous clinical experiences. As expected on the pretest, students in the pediatrics group rated their experience with pediatric skills higher overall. The pediatrics group had significantly higher scores than the family medicine group in objectives that seemed to require a more in-depth understanding of pediatric issues. For example, both pediatrics and family medicine groups rated themselves as having more experience than the internal medicine group on measuring and plotting growth parameters. However, the pediatrics group rated themselves as having more experience than the family medicine and internal medicine groups on including an assessment of a child's growth in a workup and in the evaluation of a child with failure to thrive. This may be one reason why the family medicine group did not have higher performance evaluations in the pediatric clerkship, despite rating themselves as experienced on 8 self-assessment items. The 21 items that demonstrated no difference between the groups may represent items that all students feel similarly confident about and that should have been experienced with any primary care preceptor, such as professional conduct, critical thinking skills, or generic clinical skills; or experiences in pediatrics that are not often seen by the student in a ambulatory pediatric office, such as emergent clinical situations or child abuse. The pediatrics group rated their skills in taking written and oral medical histories and in conducting physical examinations lower, although this was not reflected in their written medical histories and physical examinations in the clerkship, or in communicating information about the diagnosis and treatment to patients and caregivers. It may be that family medicine or internal medicine preceptors emphasized these areas.
Clinical clerkship evaluations entail some degree of subjective clinical evaluations when assessing performance. Studies have shown a poor correlation between evaluators' ratings of students' knowledge and subsequent performance on standardized testing or the national boards.8,9 This would seem to be true of the pediatric clerkship's clinical evaluations when compared with the multiple-choice test scores, which did not differ between the 3 groups. The pediatric multiple-choice questions may have more accurately reflected rote memorization of knowledge without the integrated knowledge necessary to perform clinically at a superior level. Furthermore, subjective clinical evaluations were subject to grade inflation when a student's scores were inadvertently elevated by an instructor's overall impression of the student's personality or work ethic. The high average mean score for all students in the pediatric clerkship indicated that grade inflation might have occurred. If students with pediatric preceptors were more motivated during the clerkship, this may have resulted in higher overall pediatric clinical scores for this group. This explanation might be possible for clinical evaluations, but would not apply to the students' pretest results.
The implementation of the longitudinal PCC at the University of Colorado School of Medicine has been a successful, innovative curriculum initiative. Reports from medical student focus groups indicated that students were enthusiastic about early clinical experiences and appreciated the long-term mentoring relationship with a primary care preceptor in the early years of medical school. In this article, we demonstrated that the PCC has also had a significant effect on issues related to pediatric education.
Confronted with these data, the PCC has been adjusted to provide pediatric exposure for all students in the first 2 years of medical school. Students who do not have pediatric exposure with their assigned preceptors are required to "switch" for 1 month to a pediatric preceptor. The reverse switch also occurs for students with no clinical experience with adult patients. Furthermore, students are now required to have a preceptor directly observe and document their ability to perform the steps in taking a medical history and performing a physical examination on an infant aged 0 to 6 months. Problem-based learning cases focusing on pediatric problems and computer-based pediatric cases, incorporating digital video, are being developed that may be integrated in the PCC. Data collection during the next 2 academic years will determine if these changes can adequately bridge the gap in pediatric clerkship performance between these groups.
Because PCC experiences at different institutions may vary especially in regard to pediatric exposure, the results of a similar analysis at a different institution may also vary. Regardless, we believe that this study has important implications for generalist curricula and pediatric education. Students with more clinical exposure to pediatric preceptors had higher clinical performance scores and were more confident in the pediatric clerkship. Our data suggest that medical educators should be aware of a student's need for early pediatric experiences in a longitudinal PCC and plan more opportunities for students to develop pediatric skills.
Accepted for publication December 16, 1998.
Presented at the 1998 Annual Meeting of the Council of Medical Student Education in Pediatrics (COMSEP), Miami, Fla, March 7, 1998.
We thank Loralee Logan for her assistance with the statistical analysis.
Reprints: Robin Deterding, MD, The Children's Hospital, Campus Box B395, 1056 E 19th Ave, Denver, CO 80218.
Editor's Note: I'm very interested in seeing the results of similar studies performed in a variety of medical schools. It's a good bet the findings would be comparable.—Catherine D. DeAngelis, MD