Describe the effect of an educational intervention on medical provider knowledge and competency regarding child sexual abuse.
Using a before and after trial design with an educational intervention, the study assesses knowledge changes in specific content areas and describes a postintervention competency assessment.
Voluntary participation of practicing medical providers and pediatric residents.
Completion of a self-study, case-based, published learning curriculum on child sexual abuse, including a workbook and videotaped genital examinations.
Main Outcome Measures
Pre- and postintervention multiple choice and short answer (30 questions) test results as well as a written response to a clinical case scenario.
Sixty-four participants completed pre- and post-tests. The average posttest score (26.9/30, SD = 4.13) was significantly higher (P<.001) than the average pretest score (20.4/30, SD = 1.65). More than half (59.4%) of providers did not correctly interpret the exam findings, 28.1% did not correctly reassure the child and family, and 39.1% did not indicate an appropriate understanding of the legal implications.
Motivated medical providers demonstrated significant knowledge gains regarding the evaluation of child sexual abuse following participation in the educational program. This new knowledge was not enough to provide competency in the interpretation of genital findings or in offering legal advocacy to the families. Competence in these areas may in fact represent the domain of experts, not primary care providers, and further studies are needed to determine how much experience is necessary to provide competency in these areas.
There are few studies of effective educational interventions for teaching child sexual abuse medical evaluations. Active interventions, such as use of standardized documentation forms, chart reviews with feedback, and peer review have met with some success.1- 3 Continuing medical education has been an accepted strategy for ongoing learning once medical providers have left the structured educational venues of medical school and is intended to improve medical provider knowledge and lead to improved patient outcomes. Self-study modules for emergency medicine physicians have been shown to be an efficient and effective method of delivering continuing medical education on child abuse.4 Faculty-dependent educational interventions are difficult to replicate.5,6 Most programs focus on limited content areas such as recognition and reporting of child maltreatment. However, a more comprehensive approach that includes interviewing techniques, mental health issues, child development, prevention, treatment, and legal aspects is necessary.7
We present a comprehensive educational intervention for generalist pediatric providers. This published, standardized curriculum is based on recommended adult learning strategies, including self-assessment of learning needs, interactive activities, sequenced learning modules, and recommended resources.8 This program presents evidence-based medicine through common-case examples to incrementally build knowledge in 4 core areas of child sexual abuse: process, history, physical exam, and legal issues. This intervention assumes that the participants are already able to recognize when to report child sexual abuse. The course was developed for the provider who is interested in a more comprehensive education, learning how to manage the patient beyond reporting. The program assists providers in the following basic evaluations: how to perform an exam for child sexual abuse without tampering with evidence, correctly document findings, prevent further physical or emotional trauma, offer reassurance, address legal issues and refer to a medical expert in child sexual abuse. We report the recruitment effort, the effect of the self-study course on medical provider child sexual abuse knowledge, and results of a post-course assessment of competency.
There were 2 groups of subjects, practicing medical providers and residents. The medical providers were recruited through marketing at local conferences and referrals from advocacy centers (1999-2002). They included physicians, physician assistants, nurse practitioners, and nurses. Pediatric residents from State University of New York, Upstate Medical University, Syracuse, and Children’s Hospital at Montefiore, Bronx, NY, were given the opportunity to voluntarily use the program.
The Child Abuse Medical Provider (CHAMP) Program consisted of course materials published in Evaluating Child Sexual Abuse: Education Manual for Child Sexual Abuse Medical Professionals.9 These materials used case studies and a question and answer format designed to facilitate self-paced learning by building each case on previously learned concepts. An accompanying videotape provided approximately 10 minutes of genital examination findings, highlighting normal variations of the hymen.
The program is in a workbook format and relevant supplemental materials are referenced. Successful completion requires the learner to actively participate in the learning process through self-assessment with questions and answers regarding a series of cases. The program uses principles of adult learning including listing of objectives and key points, recency and primacy, digestible pieces of information, feedback through self-examination, and overlearning through repetition on sequential cases.7 Completion of the entire manual (240 pages) qualifies for 21 credit hours in Category 1 of the American Medical Association Physician’s Recognition Award.
Medical provider pre-CHAMP and post-CHAMP knowledge was assessed to evaluate the effectiveness of the training program. The pretest and posttest questions assessed knowledge pertaining to the 4 content areas: protocol and process decision points, history, medical exam, and legal issues. There were 6 process, 5 history, 17 examination, and 2 legal questions resulting in a total of 30 multiple choice and short answer, 1-word, fill-in questions. The posttest contained an additional question designed to assess competency in evaluation of a case presentation and 1 still colposcopic photograph of female adolescent genitalia. The learner was asked to provide an essay response covering 6 competency areas (documentation, interpretation, ability to reassure the patient, and understanding of legal, medical, and follow-up issues) that are considered necessary and sufficient for a child sexual abuse examination. Each area was graded on a scale of 0 to 2, 0 indicating a blank or incorrect answer, 1 indicating a partially correct answer, and 2 indicating a completely correct answer. These tests were scored by the lead author (A.S.B.) who was blinded to the participant’s score on the pre- or posttest.
The pretest included demographic data collection: type of practitioner (physician, nurse practitioner, nurse, physician assistant, or resident); training (pediatrics, family medicine, general medicine, internal medicine, or other); affiliation (community hospital, university hospital/medical school, practice, or other); previously performed sexual abuse evaluations (yes/no); acknowledgment of prior education or training in child sexual abuse (yes/no); worked with someone who was a forensic expert in child sexual abuse (yes/no); and owned a copy or had access to a copy of the New York state Child and Adolescent Sexual Offense Medical Protocol10 (yes/no). This protocol is no longer in print but was sent by mail to all medical providers in New York state in 1996, prior to the course implementation.
To assess cognitive gains from the intervention, we submitted CHAMP completers’ pre- and posttest data to a repeated measures analysis of variance, setting α to reject the null hypothesis of no knowledge gain to .05. The statistical model was a 2 × 3 (time[pre × post] × practitioner type) mixed-model analysis of variance. This analysis was conducted on overall knowledge and 4 knowledge subscore data.
Posttest competency data (essay scores) were submitted to a 1-way analysis of variance comparing competency among the 3 types of practitioners described above, again setting critical α as .05. Competency was not assessed prior to the course.
The Institutional Review Board at the State University of New York, Upstate Medical University approved this study.
Of the total 189 providers who participated in the course, 6 were eliminated from the data because of missing practitioner identifying information. A total of 64 medical providers completed both a pre- and posttest, including 30 physicians, 24 physician extenders, and 10 pediatric residents. The main study sample included these 64 providers who completed the CHAMP course.
Table 1 shows subject characteristics and baseline knowledge summaries of completers (pretest and posttest data available) and noncompleters (only pretest or posttest data). The comparison between completers vs noncompleters on pre-CHAMP knowledge data demonstrates no significant difference on overall knowledge or subscales assessing process, relevant medical history, or legal issues. The analysis did reveal that completers had significantly higher physical findings subscale data than noncompleters; mean (SD) = 11.52(3.37) vs 10.23(3.60), respectively, P<.05.
Except for the lack of nurses, the providers who completed the course were not significantly different in area of practice, affiliation, reported previous experience in child sexual abuse examinations, reported formal training in these evaluations, reported working relationship with a child abuse or forensic pediatrician, or reported access to the New York state protocol from those who did not complete the course (Table 1).
The practice and affiliation demographic information is also summarized in Table 1. In general, most of the participants were pediatric providers and the distribution of affiliation was nearly evenly divided between those most often based at a community hospital, university hospital, or office setting. Those who designated themselves in the “other” category for area of practice included write-in responses for obstetrics/gynecology, emergency medicine, and public health. The “other” category for affiliations included write-in responses for urgent care settings and public health clinics.
Since the primary purpose of this article is to evaluate the effectiveness of a training program on pre- and postknowledge and competency, all subsequent results focus only on learners for whom both pre- and posttest data are available (n = 64).
The average pretest or baseline scores (Table 1) for the 64 participants in the study sample was significantly lower than the average posttest scores on the 30 multiple choice/short answer questions (26.9/30, SD = 1.65, P<.001). The posttest scores ranged from a high of 29 out of 30 to a low of 20 out of 30. The results of the cognitive assessment findings (Table 2) show that there was significant improvement overall and in all 4 areas (process, history, physical, and legal).
Additionally, our analyses of practitioners’ knowledge gains overall and on all 4 subscaled content areas revealed no statistical interaction effects, meaning that physicians, residents, and physician extenders all improved their knowledge similarly from baseline by taking the course.
Although all participants including physicians gained knowledge, we did find 1 significant main effect for process knowledge (P<.05) indicating an overall knowledge difference among practitioner types averaged across pre- and postscores. Subsequent Tukey Honestly Significant Difference post-hoc analysis revealed that physicians showed significantly higher pre- and postprocess related subscore data than physician extenders (P<.02). This effect demonstrates only a process knowledge difference between physicians and physician extenders.
Posttest gains were not significantly associated with reported educational experiences, affiliations, areas of practice, or access to the New York state protocol.
The posttest scores indicate that there were significant gains in the physical examination subscale. A detailed analysis of the pretest questions in this subscale indicates that baseline participant ability to label the female genital anatomy was good since the hymen was correctly labeled by 95% of the participants, the labia minora correctly labeled by 81%, and the urethra correctly labeled by 98% of the 171 who answered these questions. Significant knowledge gains were noted despite the better than expected baseline skills in genital anatomy recognition.
Posttest essay results were provided by 63 of the 64 completers who averaged 7 of 12 potential points. There was no significant difference in essay performances by practitioner type. Table 3 shows the results of the essay by competence area. Despite improved knowledge as shown in the multiple choice/short answer questions, 59.4% of providers did not correctly interpret the findings, 28.1% did not correctly reassure the child and family, and 39.1% did not document an appropriate understanding of the legal implications.
This study demonstrates 2 important findings. First, interested primary care medical providers and residents showed significant cognitive gains following this self-study course. Second, although basic child sexual abuse information was learned, knowledge did not imply competence, particularly for interpretation of findings and providing legal advocacy.
The problem of physician inexperience, lack of understanding, and lack of education in child abuse is not new.11- 13 Continuing medical education course work has previously concentrated primarily on recognition and reporting abuse and has been shown to improve knowledge of these topics.14 Except for the finding that physicians (compared with other providers) had a better baseline and posttest understanding of sexual abuse physical examination findings, our results did not demonstrate a significant relationship between educational background in child sexual abuse and overall scores on the pretest.
Less than 20% of the participants acknowledged access to the New York state protocol regarding child sexual abuse. This supports the previously demonstrated notion that distribution of guidelines alone are generally ineffective educational strategies.8 Our results further support this since there was not a significant difference in pretest scores between those that had access or did not have access to the protocol. The low number of individuals who still had access to the protocol indicates that this is not a very effective method of providing lasting information.
Others have demonstrated that many providers lack baseline knowledge of genital anatomy.11,12 Recognition of normal anatomy was not generally a precourse problem for this study group.
Some educators have focused on the physical examination findings and have shown that common gynecologic disorders can be recognized if learners complete a program in pediatric gynecology.15 Focusing on examination findings, as is commonly found in most child sexual abuse conferences and courses, may result in the neglect of the other important aspects of the evaluation, such as taking a sexual abuse history, understanding legal issues, and providing therapeutic reassurance for the child and family. Although the CHAMP course had more physical examination test questions than questions in the other areas, the course did not emphasize one area over another and, in fact, results showed improvement in all knowledge areas.
Despite these gains, providers did not demonstrate competency on the case example regarding interpretation of findings and legal aspects. The posttest essay question referred to a paragraph describing a case of sexual abuse and a single colposcopic photograph. The case was an adolescent with a history of forced sexual intercourse. The photograph shows a normal examination, with an anterior notch in the estrogenized hymen. In general, the participants correctly documented the decreased anterior hymenal tissue. However, this finding was frequently misinterpreted as abnormal. Participants were expected to write their suggestions for future legal advocacy, such as assisting the family with law enforcement reporting and providing testimony regarding the normal or abnormal finding. In most cases, the participants left a blank answer regarding the legal issues.
It is possible that providers would have demonstrated improved competency if asked to examine a real patient or by utilizing a videotaped case example.16 Still photographs are often difficult to interpret and do not always demonstrate the exact findings identified by carefully observing the edge of the hymen. The hymen is a dynamic structure that can change appearance with changes in relaxation and positioning of the patient. Even skilled experts might disagree in their interpretation of a still photograph.17- 19 However, the CHAMP photograph had been previously viewed by 3 authors (A.S.B., J.H.R., L.C.) and 2 other experts in the field of child sexual abuse with consensus on the findings and interpretation. The influence of the history on the physician’s interpretation of findings, or expectation bias, is another possible reason that participants misinterpreted normal findings as being abnormal.19,20 Provider competency might be more accurately assessed using a more complete evaluation of a series of cases. Despite these limitations, the essay results suggest that more experience is necessary to achieve an adequate level of competency.
The learner’s self-assessed need for training is associated with effective continuing medical education.21 Our pretests were sent to medical providers who were identified as having some motivation to learn about child sexual abuse. Yet only 35% completed the program. This suggests that if the materials and pretest were provided to the general population of providers, a smaller percentage would have completed the program, and the course would not have been as effective. We anticipated that continuing medical education credits would provide incentive for completion of the program.
In an informal telephone survey of providers, lack of time was implicated as the most common reason for noncompletion. Concerns about the effect on their practices and financial loss if they were to become “local experts” were also raised. In some states, education of medical providers regarding child sexual abuse has been linked to enhanced reimbursement for these examinations.22- 25 At the time of this study, there was no system in New York state to cover the costs of these examinations. Improving reimbursement might serve as an incentive for course completion.
Incorporating child sexual abuse evaluation experiences into residency training is recommended.26,27 As early as 1998, Dubowitz28 surveyed residencies to assess training and resources for pediatric residents in the area of child maltreatment and found that 79% of respondents wanted to strengthen their teaching efforts. A more recent survey of residency program directors and residents to assess perceptions regarding training for child sexual abuse evaluations found that more than half of faculty and residents rated the quality of the training as less than adequate for expected needs after residency.13
In our study, half of the residents did not complete the posttest. We theorize that this is most likely because of time constraints of the residency and lack of incentive for completion since it was not a part of a required rotation. Residency education in child sexual abuse is essential, not only because residents will need this knowledge and related skills after graduation, but also because residents are often the first and main medical providers in the emergency and outpatient settings of many educational institutions.28,29 Abused children, for reasons often related to their abuse experience, miss appointments or arrive late, creating difficulties in scheduling teaching during rotations in outpatient settings. Incorporating child sexual abuse education into the standard residency curriculum can be challenging as the overall learning requirements of residency programs continue to increase. Utilizing a self-paced program that is independent of faculty skills, real case availability, or scheduled resident time constraints has potential for success as the first step in improving medical provider knowledge regarding child sexual abuse.
This curriculum was developed for the pediatric generalist. Case examples were clear and did not require the participant to distinguish ambiguous physical findings. It is the uncertain nonspecific findings that are expected to require expertise for interpretation. However, the essay results indicate that even normal findings can be misinterpreted by the generalist. The characteristics of a competent child abuse expert remain ill-defined by courts and the American Board of Pediatrics. Yet, the need for 2 levels of educational programs is apparent.30 This study suggests that the role of the generalist could include documentation, treatment, and referral but not necessarily interpretation of findings or legal advocacy.
This self-study program meets the goals of effective medical education by showing knowledge gains by the participants. The program does not appear to enable competency in the skills of legal advocacy and interpretation of findings. Competence in these areas may represent the domain of experts, not primary care providers, and further studies are needed to determine how much experience is necessary to provide competency in these areas. Residents and primary care providers should learn to manage all common cases of child sexual abuse effectively and refer unusual cases or those with specific issues to expert forensic pediatricians and/or child advocacy centers. Educational programs that include guided experiences in examining abused children need to be studied in order to determine if this type of education improves competency and patient outcomes as well as cognitive skills.
Correspondence: Ann S. Botash, MD, Associate Professor of Pediatrics, State University of New York, Upstate Medical University, 750 E Adams St, Syracuse, NY 13210 (firstname.lastname@example.org).
Disclaimer: The content of the manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the Centers for Disease Control or the New York State Department of Health, Rape Crisis Program. The funding organizations did not participate in the design, conduct, interpretation, and analysis or review of the study. Any income generated from the sale of Evaluating Child Sexual Abuse: Education Manual for Medical Professionals is reimbursed to the CHAMP program through the Research Foundation.
Accepted for Publication: January 6, 2005.
Funding/Support: This study was supported in part by funding from the Centers for Disease Control and rape prevention education funding administered by the New York State Department of Health, Rape Crisis Program, Albany, NY.
Acknowledgment: Special thanks to Christine Schoonmaker of Safe Horizon Inc, Brooklyn, NY, and Lauren Arbolino, psychology student at Syracuse University, for their assistance. Thanks also to Drs Joyce Adams (Clinical Professor of Pediatrics, Division of Adolescent Medicine, University of California, San Diego) and Lori Frasier (Center for Safe and Health Families, Primary Children’s Medical Center, Salt Lake City, Utah) for their review of the essay case photograph.
Botash AS, Galloway AE, Booth T, Ploutz-Snyder R, Hoffman-Rosenfeld J, Cahill L. Continuing Medical Education in Child Sexual AbuseCognitive Gains but Not Expertise. Arch Pediatr Adolesc Med. 2005;159(6):561-566. doi:10.1001/archpedi.159.6.561