Performance on the pretest for diagnosis of infection with 4 category A bioterrorism agents is compared with performance after completing the didactic section. P < .001 for posttest vs pretest results for smallpox, botulism, and plague; and P = .002 for posttest vs pretest results for anthrax.
Performance on the pretest for management of infection with 4 category A bioterrorism agents is compared with performance after completing the didactic section. P < .001 for posttest vs pretest results for infection with all 4 agents.
Cosgrove SE, Perl TM, Song X, Sisson SD. Ability of Physicians to Diagnose and Manage Illness Due to Category A Bioterrorism Agents. Arch Intern Med. 2005;165(17):2002-2006. doi:10.1001/archinte.165.17.2002
Early recognition of a terrorist attack with biologic agents will rely on physician diagnosis. Physicians’ ability to diagnose and care for patients presenting after a bioterror event is unknown. The role of online case-based didactics to measure and improve knowledge in the diagnosis and treatment of these patients is unknown.
A multicenter online educational intervention was completed by 631 physicians at 30 internal medicine residency programs in 16 states and Washington, DC, between July 1, 2003, and June 10, 2004. Participants completed a pretest, assessing ability to diagnose and manage potential cases of smallpox, anthrax, botulism, and plague. A didactic module reviewing diagnosis and management of these diseases was then completed, followed by a posttest. Pretest performance measured baseline knowledge. Posttest performance compared with pretest performance measured effectiveness of the educational intervention. Results were compared based on year of training and geographic location of the residency program.
Correct diagnoses of diseases due to bioterrorism agents were as follows: smallpox, 50.7%; anthrax, 70.5%; botulism, 49.6%; and plague, 16.3% (average, 46.8%). Correct diagnosis averaged 79.0% after completing the didactic module (P<.001). Correct management of smallpox was 14.6%; anthrax, 17.0%; botulism, 60.2%; and plague, 9.7% (average, 25.4%). Correct management averaged 79.1% after completing the didactic module (P<.001). Performance did not differ based on year of training (P = .54) or geographic location (P = .64). Attending physicians performed better than residents (P<.001).
Physician diagnosis and management of diseases caused by bioterrorism agents is poor. An online didactic module may improve diagnosis and management of diseases caused by these agents.
The anthrax attacks against the United States in 2001 emphasized the need for preparedness for bioterrorism-related events within the medical community. All of the patients who had anthrax presented for care to clinics or hospitals, and at least 4 were initially sent home without a diagnosis, demonstrating the importance of education regarding the clinical presentation and treatment of patients infected with potential bioterrorism agents.1 In the event of a biologic attack, physicians will be first responders and represent the front line of defense.2 Various educational resources, ranging from Web sites to training seminars, have been developed by the Centers for Disease Control and Prevention, local health departments, and medical institutions to educate physicians.3 However, despite the availability of such resources, little is known about the ability of physicians to appropriately diagnose and treat potential patients infected with bioterrorism agents.
Six agents have been identified as having a high potential for use as bioweapons (anthrax, botulinum toxin, plague, smallpox, tularemia, and viral hemorrhagic fevers) because they can be easily disseminated or transmitted, have high morbidity or mortality rates, and would cause widespread social disruption.4 We hypothesized that physicians have not been adequately educated about the diagnosis and management of 4 of the diseases caused by these category A bioterrorism agents (smallpox, anthrax, botulism, and plague), and that computer-assisted instruction could be used to effectively teach physicians to diagnose and treat patients presenting with any of these diseases.
A didactic module on bioterrorism was developed using a 6-step approach to curriculum development.5 Twenty-four references were used to support the content of the didactic module and questions. Sixteen questions of clinical cases were written describing patients presenting with anthrax, botulism, plague, smallpox, or a common disorder that has clinical overlap with these diseases. These 4 pathogens due to category A bioterrorism agents were chosen because they have well-defined clinical presentations that are similar to common disorders but can also be distinguished from common disorders based on specific diagnostic clues.6 Multiple answer choices were provided, requesting the learner to select a diagnosis or make an initial decision on medical management. No time limitations were made on completion of the questions. Face validity and content validity of questions were obtained by having diagnosis and management questions reviewed by 10 infectious diseases specialists with expertise in bioterrorism; questions were revised until agreement was reached that sufficient information was provided to correctly diagnose and manage each case. Content validity was further verified by having 6 clinical experts in bioterrorism answer the multiple choice questions; the average score among these experts was 97%. One set of cases and questions on diagnosis and management was used as the module pretest, and a different set was used as the posttest. To educate learners on diagnosis and treatment of patients presenting with infection from a bioterrorism agent, additional cases with didactic text were written on each of the 4 agents, including the differential diagnosis, important clues to the diagnosis, and principles of treatment. Registered users began the module by completing the pretest, which enabled them to access the didactic section. The posttest could not be accessed unless the didactic section had been completed. For the pretest and posttest, learners were informed if their answer choice was correct, and if incorrect, which answer choice was correct.
The bioterrorism module was used by 30 internal medicine residency training programs in 16 states and Washington, DC. Study subjects were 2407 physicians at each participating residency training program (internal medicine house staff and faculty) who had registered and been approved to use the curriculum Web site (available at: http://www.hopkinsilc.org) and completed the module voluntarily. Participating residency training programs included primary affiliates of medical schools with extensive National Institutes of Health funding and several community hospitals.7 The Johns Hopkins University School of Medicine Office of Human Subjects Research granted exemption to this project, because this survey of educational tests had all identifiers of individual human subjects removed.
Performance data were tabulated by the Web site from July 1, 2003, through June 10, 2004, when the module was removed for updating for the new academic year. Results of partially completed modules were not included. Responses to the case scenario and management questions were tabulated electronically by the Web site, and then analyzed based on year of training, attending physician status, and residency training program. Statistical analyses were performed with Stata software, version 8.2 (Stata Corp, College Station, Tex). Comparisons between pretest and posttest scores and comparisons of pretest scores at different training levels were performed using the χ2 test. Comparisons of pretest scores at different locations and comparisons of ratings were performed using the t test. All P values were 2-sided, and P<.05 was considered significant.
Of 2407 eligible physicians, 631 (26.2%) completed the module. Of these 631 physicians, 192 (30.4%) were in postgraduate year (PGY)–1, 217 (34.4%) were in PGY-2, 183 (29.0%) were in PGY-3, 30 (4.8%) were attending physicians, and 9 (1.4%) did not state their level of training.
Average pretest and posttest scores for questions regarding diagnosis of smallpox, anthrax, botulism, and plague (or common syndromes with clinical overlap) are shown in Figure 1. The highest average pretest scores for recognition were for anthrax (70.5%) and the lowest were for plague (16.3%); half of respondents answered smallpox (50.7%) and botulism (49.6%) diagnosis questions correctly. Scores on diagnosis improved significantly after completion of the didactic section of the module (Figure 1). The average pretest score on the diagnostic questions was 46.8%, and the average posttest score was 79.0% (an increase of 32.2%, P<.001).
The ability to differentiate smallpox from other common infections required diagnosing a patient who presented with fever and a rash, with the diagnosis based on the appearance of the rash. Although the description of the rash was consistent with varicella, 42.6% of respondents incorrectly diagnosed the patient as having smallpox (Table 1). In contrast, 70.5% of respondents recognized that a presentation consisting of fever and a left lower lobe infiltrate without mediastinal widening was more consistent with Streptococcus pneumoniae pneumonia than anthrax (19.0%). Although many respondents recognized acute descending paralysis involving multiple cranial nerves as botulism (49.6%), this presentation was diagnosed as Guillain-Barre syndrome by 22.5% and as myasthenia gravis by 17.0%. Only 16.3% of respondents identified a 23-year-old woman with an acute illness consisting of fever, hemoptysis, and right middle lung pneumonia whose grandmother died of pneumonia 4 days prior as more likely to have pneumonic plague than anthrax (58.2%) or brucellosis (15.1%).
The average pretest and posttest scores for questions regarding initial management of anthrax, botulism, plague, and smallpox are shown in Figure 2. With the exception of botulism (60.2%), average scores for correct initial management of smallpox (14.6%), anthrax (17.0%), and plague (9.7%) were low. Scores on management improved significantly after completion of the didactic section of the module (Figure 2). The average pretest score on management was 25.4%, and the average posttest score was 79.1% (an increase of 53.7%, P<.001).
The smallpox pretest management question asked for the appropriate treatment for a patient who developed a fever 3 days after exposure to a patient with smallpox (too short of an incubation period for the fever to represent smallpox infection) (Table 2). Although 76.5% of respondents recognized the importance of administering a smallpox vaccination to a patient with recent smallpox exposure, most respondents also favored giving vaccinia immune globulin and cidofovir (62.0%), which would not be indicated as part of smallpox prophylaxis. Similarly, when asked for the best initial treatment for pulmonary anthrax, 71.2% of respondents knew that ciprofloxacin was appropriate therapy, but only 17.0% knew that combination therapy with another agent was recommended. Most respondents (91.6%) knew the importance of rapid administration of botulinum antitoxin for the management of botulism; however, 31.4% also favored giving antibiotic therapy, which would not be indicated. When asked for the appropriate postexposure prophylaxis regimen for a health care worker who cared for a patient with pneumonic plague and a patient with anthrax, 83.4% of respondents recognized the importance of doxycycline prophylaxis for pneumonic plague, but most (90.3%) also favored ciprofloxacin prophylaxis for anthrax, which would not be indicated.
The ability to appropriately diagnose and care for patients in the clinical scenarios presented did not increase by level of residency training. Respondents in PGY-1 scored an average of 37.0%; those in PGY-2, 35.6%; and those in PGY-3, 38.4% (P = .54). However, average attending physician scores were higher than resident scores (50.0% vs 36.9%; P<.001). Pretest scoring also did not vary by geographic region. Physician performance at programs located in areas that have been affected by anthrax in the past (New York City, NY, and Washington, DC) was similar to that from those at other programs elsewhere in the United States (40.9% vs 36.1%; P = .64).
We demonstrated that physicians are undertrained in the diagnosis and management of infection caused by 4 pathogens identified as likely to be used in a potential bioterrorism attack. The ability of physicians to distinguish smallpox, anthrax, botulism, and plague from other, more common, disorders was poor, as was their ability to manage illness due to bioterrorism agents once a diagnosis had been made. Physicians were best able to distinguish anthrax from other community-acquired pneumonias, likely because of publicity associated with the anthrax attacks of 2001. The diagnosis of plague was poor, perhaps because of the inherent difficulty in diagnosing plague, which has significant clinical overlap with other serious respiratory tract infections. Only half of the health care workers tested recognized the classic presentations of botulism and smallpox. With the exception of botulism, physicians also had difficulty with the management of illness due to potential agents of bioterrorism. The high score on botulism management may represent better physician awareness that treatment requires antitoxin rather than antibiotics.
Physician education has been called the most important part of preparation for a bioterror attack because early recognition of illness due to a bioterrorism agent will decrease morbidity and mortality and improve containment of communicable agents, and because trained physicians will be better equipped to care for many ill patients.2,8,9 Nevertheless, one study10 of emergency medicine physicians found that only 53% of residency programs included formal training in bioterrorism and that more than 65% of physicians rated their abilities to diagnose and manage casualties of bioterrorism as less than adequate or very poor. Although this study was performed before the anthrax attacks of 2001, a second survey performed after the attacks reported that 68% of emergency physicians believed that they had limited knowledge of the concepts of bioterrorism.6 We showed that physicians had better baseline knowledge of diagnosis of illnesses due to bioterrorism agents compared with management; however, correct management is arguably of secondary importance when compared with correct diagnosis. Once the diagnosis is made, expert opinion can be obtained with regard to prophylaxis and management issues, but such response is dependent on early recognition. Nevertheless, all physicians should have some understanding of issues concerning infection control, prophylaxis, and first-line therapy.
In our study, attending physicians performed better than resident physicians, but scores did not improve among house staff and faculty based on year of training, which may indicate that education of physicians is not occurring during residency training. Attending physicians also may have scored higher than residents because of greater familiarity with the common disorders that have clinical overlap with illnesses due to bioterrorism agents rather than knowledge of bioterrorism syndromes. The significant improvement in posttest scores among respondents at all levels of training suggests that physicians can be trained using an online case-based format to learn how to diagnose and manage infection caused by category A bioterrorism agents. The lack of differences in scores from programs in the New York City and Washington, DC, areas compared with programs in other locations suggests that no additional emphasis is placed on bioterrorism training in areas that have previously been affected by terror and bioterror events.
Other studies have evaluated the role of Web-based bioterrorism training with varying results. One study6 found no difference in bioterrorism knowledge in a group of physicians randomized to receive case presentations by e-mail and to have access to an educational bioterrorism Web site when compared with a group of physicians receiving no educational materials. However, this intervention differs from our study in that our module requires active case-based learning. Another study evaluated the effectiveness of screen savers with informational text about potential agents of bioterror with links to Web-based learning modules and additional bioterrorism information in the emergency department. Medical students rotating in emergency medicine underwent a pretest before the start of clinical duties and a posttest after their rotation in the emergency department, and their test scores increased from 59.1% to 75.8% after exposure to the screen savers.11 The results of this study and our study suggest that Web-based case-oriented learning can be effective in the education of physicians about agents of bioterrorism.
There are several limitations to this study. The response rate to the survey at participating institutions was low. It is possible that physicians who completed the survey did so only because they believed their ability to diagnose and care for patients in the event of a bioterror attackwas poor. However, scores at residency training programs with low use were not lower than scores at residency training programs with high use (data not shown), making selection bias based on poor knowledge unlikely. Physicians who did complete the survey were aware that they were completing a module on bioterrorism, which may have had an impact on their differential diagnosis. However, physicians were informed that cases presented may or may not represent an illness due to a bioterrorism agent, and for some cases on diagnosis, the correct answer was a common illness. Results shown herein may more closely reflect performance after a known bioterror attack. Test questions in this module cannot fully replicate the clinical scenarios likely to be seen in the event of a bioterror attack. However, the superior performance of physicians on correctly diagnosing anthrax, and the superior performance by attending physicians (who are more likely to be familiar with the common disorders that may be confused with infection with bioterrorism agents), provides construct validity to the case questions. Finally, this study did not measure long-term knowledge retention, which is a significant issue in physician education. One attractive feature of this Internet-based module is that it can be administered frequently to bolster physician knowledge.
A recent analysis12 of US preparedness in the event of a bioterror attack found that hospitals would be overwhelmed with patients, government response would be confused, and technology to detect a biologic attack early is inadequate. This study demonstrates that physician knowledge about diagnosis and treatment of patients presenting with infection or exposure to a likely bioterrorism agent is poor, and adds physician education to the list of priorities the government should undertake to prepare for what many see as an inevitable event. The Internet can be used to distribute a curriculum to teach physicians to diagnose and care for patients exposed to or infected with a likely bioterrorism agent. Early recognition is essential to minimize the potentially devastating impact of a bioterror attack.
Correspondence: Stephen D. Sisson, MD, Division of General Internal Medicine, Department of Medicine, The Johns Hopkins University School of Medicine, 601 N Caroline St, Room 7150, Baltimore, MD 21287 (email@example.com).
Accepted for Publication: April 30, 2005.
Financial Disclosure: None.