Pediatric Clerkship Experience and Performance in the Nebraska Education Consortium: A Community vs University Comparison

Objective  To compare the reported experiences and performance on end-of-course examinations of students completing their pediatric clerkship at the University of Nebraska Medical Center (UNMC), Omaha, with that of students completing their clerkship in a community pediatrician's practice (CPP) outside the Omaha metropolitan area.

Design  Cohort study.

Setting  Private and/or institutional practices with both ambulatory and hospital components.

Participants  For the academic year 1996-1997, all 113 students completing the 8-week third-year pediatric clerkship returned a questionnaire detailing their opinions of the experience. They also completed written (multiple-choice and essay questions) and oral (standardized parent interview) examinations, locally prepared and based on clerkship curriculum objectives provided to the students at orientation. Prior to student placement in the CPP, the clerkship goals, content, and evaluation methods as well as techniques for teaching in a busy office practice were reviewed with the CPP physicians. Eighty-one students performed their clerkship at UNMC while 31 spent all but the first week of the clerkship in the CPP.

Main Outcome Measures  The students' opinions about their experiences and their performances on the end-of-course examinations were compared. Statistical analysis of the questionnaire was done using the Fisher exact test and the Mantel-Haenszel χ² test while examination performance was compared using the t test and the Wilcoxon rank sum test.

Results  The UNMC and CPP groups reported similar opinions of their experiences in the newborn nursery and the inpatient portion of the clerkship, but the CPP students were much more positive about their learning experience in the clinic (P=. 001). The CPP students reported more involvement in the patient's overall care (P<.001) and in other aspects of clinic operation (P<.001). The UNMC and CPP students had similar opinions of curriculum content, reading material, and didactic instruction. No group differences were found regarding interest in pediatrics as a career. Most importantly, no group differences were found in performance on any portion of the end-of-course examinations.

Conclusions  Community-based education at the third-year clerkship level can be accomplished without a significant effect on student examination performance if students and faculty are aware of and adhere to a common set of goals. The end result is a much more robust experience for students who spend the clerkship in the practice of a community-based pediatrician.

MOST THIRD-YEAR pediatric clerkships are organized so that students spend equal amounts of time in the clinic and on the inpatient service. For years this has been a successful model for clinical training of medical students.¹ However, the forces of change have begun to affect this manner of training. No longer are pediatric patients hospitalized for long periods, the absolute numbers of children being admitted to hospitals has declined dramatically,² and there have been numerous calls for reforming the way medical students are trained.^3,4 The result has, at least for some pediatric departments, forced a reexamination of the overall organization of the clerkship and a greater focus on ambulatory settings. The pediatrics department at the University of Nebraska Medical Center (UNMC), Omaha, was faced with exactly this challenge. In a way we were forced to consider this change by virtue of the fact that there are 2 medical schools in Omaha—a relatively small metropolitan area—with each school having equal numbers of medical students who need a pediatric clerkship experience.

Before ever considering sending third-year students to private offices for their entire clerkship, we conducted a "pilot project" in which fourth-year medical students performed a required pediatric rotation in a private practice office. The students and preceptors participating in this pilot project were very positive about their experience. We were encouraged in this process when we discovered that the students performed equally well on the structured final oral examination irrespective of where they completed their clerkship. Further motivation for changing the clerkship focus and structure came from favorable reports of other schools who had students off-campus for their entire clerkship.^5-8 However, we were cautioned about placing students in offices by others who reported having third-year students return from a community pediatric experience only to perform poorly on their end-of-course examination.⁹ Therefore, we looked to the community pediatricians across the entire state of Nebraska for help in teaching our students. They responded with enthusiasm.

This report is a critical appraisal of student perceptions of their experiences in this "distributed" clerkship as well as a comparison analysis of performance for university- vs community-trained students.

A needs assessment had been previously performed before the beginning of the pilot project. Thus, we had already outlined a strategy and timeline for this new curriculum. The program was introduced to the faculty by a series of conferences, meetings, seminars, and workshops. All paid faculty and pediatric house staff were familiarized with the concept and content of the program by a series of seminars; most were combined with regular faculty meetings. House staff were given instruction in preceptor techniques at intern retreats and resident conferences. Additional instruction in teaching techniques (eg, precepting in a busy office) was given to selected UNMC full-time (n=2) and volunteer faculty members (n=6) through 2 invited workshops. All volunteer faculty (n=68) were visited in their offices by 2 of us (F.A.M. and G.L.B.) at times that were convenient for them. Topics covered were the overall curriculum design, specific curriculum content, and precepting techniques for the office. Of the 68 volunteer faculty members involved in this project, 35 have private practices outside Omaha (community pediatrician practice [CPP] volunteer faculty).

All students were oriented to the course and curriculum through attendance at a mandatory 4-hour orientation on the first day of the clerkship. Figure 1 shows the overall course layout for both UNMC and CPP students. The UNMC students spent 2 weeks on the inpatient service, 1 week in the normal newborn nursery, 3 weeks in the general pediatric clinic, 1 week at Meyers Rehabilitation Clinic, and 1 week at a private pediatrician's office in Omaha. The CPP students were expected to totally integrate themselves into the community practice as prearranged with the preceptors. The CPP students (on average 6 per 8-week rotation) were given 9 hours of case discussions the first week of the rotation with 1 of us (F.A.M. or J.L.H.). This orientation introduced students to the care of children in the areas of health care maintenance; fever; fluids; common ear, nose, and throat problems; vomiting; abdominal pain; abdominal mass; wheezing; and stridor and respiratory emergencies. The CPP students also spent 1 day in the general pediatric clinic, 1 day in the newborn nursery, and 2 hours with a member of the pharmacy faculty for introduction to pediatric pharmaceutical preparations and dosing. The UNMC students received the same case discussions during the 8 weeks of the clerkship during seminars given each Wednesday and Thursday afternoon of the clerkship (Figure 1).

Shortly after students received their schedule for the third year, a memo was sent to them explaining the CPP and UNMC training opportunities. They were asked to indicate their top 3 preferences of training site locations. Site assignments were based on a first-come, first-served basis with the final approval of the clerkship director. Nonresponders were randomly assigned to either the UNMC or the CPP rotation. However, this process was seldom necessary. The CPP students lived in the community of assignment the entire length of their rotation. Housing was provided either by the practice or by the community hospital at no cost to the student. The self-selected nature of student assignments quite likely affected some of our outcome data.

The clerkship curriculum was contained in a looseleaf binder and was divided into modules. Each module dealt either with an organ system or a unique issue within pediatrics (eg, fluid and electrolytes, child abuse, and so on). A companion self-assessment guide was also included with the curriculum. This guide consisted of questions organized into modules corresponding to the modules in the curriculum notebook. Each of these was given to the students at the clerkship orientation. The recommended textbook was also loaned to each student at orientation. For those students training at UNMC, a lecture series was provided that was intended to expand on the concepts presented in the curriculum notebook. A videotape of each lecture (recorded during the previous academic year) was sent to each practice for viewing by the CPP students. In addition, CPP preceptors were supplied with all of the printed materials and had been extensively involved in the development of this program and the pilot project noted previously.

All students were surveyed at the end of the clerkship using an end-of-course critique. All clerks finishing the third-year pediatric clerkship since 1994 had used this survey instrument. It consisted of 15 questions that asked the student to rate all aspects of the clerkship (eg, inpatient, nursery, clinic, curriculum content and layout, various conferences, and patient care activities) using either a yes/no format or a Likert scale where the highest numbers were the highest ratings. There were slight differences between the UNMC and CPP questionnaires, but where those differences existed we did not include this in our analysis.

Each student completed 2 examinations the last 2 days of the clerkship. All students returned to the university for these examinations. The written examination was created locally and had both multiple-choice and essay questions. The examination content was based on the objectives of the curriculum. The multiple-choice portion was electronically graded and the essays were scored using the modified essay question scoring technique. The students also completed a structured oral examination. This was scored for the students' ability to obtain a focused history as well as their interviewing behavior. We trained pediatric house staff to act as simulated parents for the oral examination.

For questions with 2 choices (yes/no), the Fisher exact test was used to see if there was a difference in the distribution of responses between student groups. For questions with more than 2 choices (Likert scale responses) each choice was assigned an ordinal value and a trend test, the Mantel-Haenszel χ² test, was used to determine if there were differences between the responses of the 2 student groups. A comparison of oral and written examination scores between the 2 student groups was performed using a t test.

There were no apparent differences in opinion between UNMC and CPP students regarding their experiences on the inpatient and nursery portions of the rotation. However, there was conclusive evidence of a difference between the responses and the location of the students, with the CPP students responding more favorably toward their learning experience in general clinic (Table 1).

All students spent time in a private pediatrician's office (UNMC students, 1 week; CPP students, 6.5 weeks). All students recorded in a log book the age, sex, chief complaint, final diagnosis, and location of the encounter for all patients they saw during the 8-week clerkship. This report details the self-reported experience of each student. The log book data are currently being analyzed and will be reported later. The questionnaire asked them to place a check mark beside each activity (Table 2) they engaged in with their preceptor. From their self-report there was conclusive evidence that more CPP students saw patients before their preceptor as well as with their preceptor. The CPP students reported that they were allowed to do more procedures than the UNMC students and this was again statistically significant. It should also be noted that the students were asked to comment on their involvement in daily procedures throughout the entire clerkship. The CPP students were statistically more likely to make hospital and nursery rounds with their preceptor and to spend time in their preceptor's after-hours clinic. The CPP students also spent more time in their preceptor's business office.

Both groups of students were asked to rate the objectives of the curriculum as "helpful to clarify expectations," "confusing," or "did not read or use." The majority of students (75%) rated the curriculum objectives as helpful and felt that the course content satisfied the curriculum objectives. Many students in each group felt that there was too much to read. They found the curriculum supplements (eg, a self-assessment guide, clinical cases, and lectures/videotapes) helpful in enhancing their understanding of pediatrics.

When asked which aspect of the clerkship was most helpful in teaching them how to conduct a pediatric history and physical examination, the CPP students ranked clinic patient contact as most important while the UNMC students ranked the inpatient experience as most beneficial (Table 3). Again these differences were statistically significant. The CPP students rated night call as valuable while the UNMC students disagreed (P=.01).

The most encouraging results came when we analyzed the differences between CPP and UNMC students' performances on the end-of-course examinations. We could detect no differences between the 2 groups on either the oral or the written examination (Table 4). There were also no differences in the students' final grades.

The results of our study support our contention that third-year medical students can receive an equivalent clinical education irrespective of whether that experience occurs in a private office setting or within the university. Two important factors influence this. First, all faculty involved in the students' education must teach using the same basic set of goals/objectives and subscribe to the same basic principles for the clerkship. Second, all students must have equal access to educational materials irrespective of the site of training.¹⁰ We believe that we accomplished both through the methods used in implementing and supporting this widely distributed clerkship experience.

While this may be restating the obvious, it is very important to involve all of the affected people in the initial process of curriculum revision and planning. Our approach to implementing this revised clerkship followed a traditional plan of curriculum design by first identifying all of the stakeholders in the curriculum (eg, students, house staff, university faculty, volunteer faculty [private-practice physicians], and the clerkship administration). This was followed by a needs assessment and a stepwise process to full implementation.¹¹ Even now, we monitor student reports and performance as an ongoing process of continuous quality improvement.

Setting expectations was another important "front-end" task. Even before we embarked on this project, we consciously devised a plan for introducing this new approach to all faculty.¹² Through a series of faculty and house staff presentations, small group discussions, and committee work, the curriculum content and implementation plan was developed. The entire university pediatric faculty and house staff were involved from the very beginning. Once a consensus was developed within the faculty, the private practice pediatricians were approached. The private practice pediatricians were contacted individually and site visits were made to every office. At those meetings, the plan was discussed, questions were answered, and return visit plans were made. On the first return visit, the concept of the 1-minute preceptor was introduced. The private practice physicians were constantly reassured that curriculum content issues were the university faculty's responsibility and that our expectation was that they would allow students to see patients in their office and to focus teaching on the patients' problems.

Everyone involved in this clerkship was kept informed and apprised at every step along the way. Students knew on the first day of the clerkship what was expected of them and what we expected of the faculty involved in their education. They were oriented to the curriculum content, supplemental study materials, and the complete grading policy of the department. The clerkship coordinator was also involved in all curriculum design decisions. He attended all meetings and was integral in presenting each part of the plan as it was devised. He accompanied the clerkship director at the time (F.A.M.) on all site visits and participated in briefing the faculty on the administrative aspects of the curriculum. He established and maintained contact with key office personnel so as to establish a direct personal link to every office involved. We piloted the program using fourth-year medical students. From that experience we made necessary corrections before the next academic year when we had projected sending third-year students to the same offices.

Providing a good overall course design (curriculum guide, supplemental study materials, videotapes, and so on) as well as having an interest in student education enhances the quality of education.^13,14 This is supported by the fact that all students performed equally well on all end-of-course examinations and that a frequent comment by all students was how they valued any one-on-one instruction they received. Furthermore, the CPP students felt that their most valuable experience was the one-on-one instruction they received. They placed great value on a close personal relationship with a practicing pediatrician. The CPP students also reported that they were involved in the medical care of many patients in the private practice offices. There is proven educational value in evaluating patients and dealing with their medical problems in a repetitive as well as a longitudinal fashion,^15,16 and a private practice setting can provide both. The CPP students witnessed firsthand how continuity of care is important in the care of children. They were also able to experiment with different behavior models as demonstrated by their preceptors. Working in a private practice offered the student a fresh and sometimes totally different perspective on the practice of medicine.

Our greatest success in implementing this program has been having all private practices outside Omaha participating as teaching sites. In addition, most pediatric practices in Omaha also work with our students. We are fortunate in having many of our residency graduates remain in the state of Nebraska. This gives us a pool of private practitioners who are familiar with the university and its overall educational mission. We have rewarded them for their participation by presenting each office and each practitioner with a certificate of appreciation and participation suitably framed for display in their office and waiting room, formally recognizing an outstanding preceptor from Omaha as well as one from outside the city, and providing Internet access for practices outside Omaha through a small grant from the dean's office. We have created a pediatrics Web site where any student or faculty member can access the clerkship curriculum, self-study questions, clinical case discussions, student lectures, and Children's Hospital grand rounds discussions. We can also use the Web site as a resource for e-mail correspondence between the distant clinical teaching sites and the university. A study is currently being conducted of Web site use. And finally, we continue to work with the faculty on developing their own teaching style. This is done with the volunteer faculty through travel to their offices to work with them one on one. Through all of these efforts, we have been able to be very effective at maintaining a "far-flung" clerkship without compromising the learning of students.

Accepted for publication January 12, 1999.

Presented at the annual meeting of the Ambulatory Pediatrics Association, New Orleans, La, May 3, 1998.

We gratefully acknowledge the secretarial assistance of Heidi Mautner.

Kim Allen, DO; Deborah Anderson, MD; David Rohwer, MD (Columbus); Terry Wooldridge, MD; Paige Groppe-Giesselmann, MD (Fremont); Karen Higgins, MD; Agnes Gomes, MD; Larry Marshall, MD; Janice Kutilek, MD; Shelly Klein, MD; Doug Boon, MD (Grand Island); Bruce Sheffield, MD; Pat Doherty, MD; Janet Howe, MD; Kristen Johnson, MD; Ken Zoucha, MD (Hastings); Kenton Shaffer, MD; Michelle Collignon-Howe, MD; Susan Greenwald, MD; Katherine Keifer, MD; Deann Psota, MD; Phil Gasseling, MD (Kearney); Chuck Erickson, MD; Mike Germer, MD; Stacie Bleicher, MD; Sue Springman, MD; Larry Fletcher, MD; Jo Kinberg, MD; Hal Shriner, MD (Lincoln); Rich Votta, MD; Norm Kelley, MD (Norfolk); Sam Perry, MD; Dell Shepherd, MD (North Platte); Lynn Williams, MD; Steve Parys, MD (Scottsbluff); Michael Grush, MD; Shirley Miller, MD; Greg Penny, MD; Donald Uzendoski, MD; Ann Russell, MD; Patricia Hammett, MD; Jill Reel, MD; Nancy VanderSluis, MD; Charles Rush, MD; Michael Moore, MD; Jillyn Kratochvil, MD; Otto Rath, MD; Thomas Byrne, MD; Mark Domet, MD; Cynthia Bowling, MD; Kurt Davey, MD; Elizabeth Larson, MD; John Moore, MD; Fran Harrison, MD; Janie Mikuls, MD; Laura Nielsen, MD; Deb McCaslin, MD; Joseph Ellison, MD; Philip Itkin, MD; Betsy Stepehnson, MD; Robert Woodford, MD; Charles Look, MD; Lisa Whitcomb, MD; John Schiffbauer, MD; Gary Lerner, MD; John Vann, MD; Mary Lou Flearl, MD; Kenton Amstutz, DO (Omaha).

Editor's Note: Willie Sutton said when asked why he robbed banks, "That's where the money is." Community sites are where the real practice is—where general pediatricians most likely will see patients.—Catherine D. DeAngelis, MD

Reprints: Fredrick A. McCurdy, MD, PhD, Department of Pediatrics, 982184 Nebraska Medical Center, Omaha, NE 68198-2184.