Copyright 1999 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.1999
To develop and pilot test a telephone-based survey instrument that enables parents to identify and characterize the body region and severity of childhood injuries using the Abbreviated Injury Scale (AIS) scoring system.
A prospective cross-sectional survey.
The emergency department of an urban, tertiary care, pediatric trauma center.
One hundred forty-seven parents of children younger than 18 years and seen in the emergency department for acute treatment of an unintentional injury.
Main Outcome Measure
The degree of agreement, measured as sensitivity, specificity, and κ statistic, between medical record information and parents' responses to the telephone survey regarding the identification and characterization of clinically significant (AIS ≥2) injuries.
The survey, known as the Injury Severity Assessment Survey/Parent Report, was developed via a systematic review of the AIS 1990 manual. Answers to questions were developed in a way that enabled automated coding of responses into AIS scores or ranges of scores. The sensitivity of the survey (its ability to detect injuries scoring 2 or more on the AIS that were documented in the medical record) varied somewhat by the body region of injury, ranging from 88% for head, face, neck, and spine injuries to 95% for extremity injuries. Intermediate sensitivity (92%) was noted for the detection of significant chest and abdomen injuries. The specificity of the survey (its ability to rule out the presence of a significant injury when one was not documented in the medical record) was more than 95% in each of the 3 body region groups. The κ statistics for the 3 body region groups ranged from 0.89 to 0.92.
A new telephone-based survey has been developed that enables parents to characterize their child's injuries by body region and to differentiate between minor injuries and more significant injuries using a well-established injury classification system. This survey has a significant advantage over previous telephone-based or written surveys of childhood injuries and may be particularly valuable in population-based (eg, random-digit dial surveys) or multi-institutional studies of pediatric injuries.
PARENTS ARE frequently the subjects of interviews designed to gather information regarding injuries to children. Several investigators and governmental agencies have used telephone interviews with parents to track the incidence of nonfatal injuries to children as well as to assess various outcomes of childhood injuries.1- 6 Studies relying on parent telephone interview to determine the incidence of childhood injuries have typically assessed the occurrence of "medically attended injuries," those resulting in a physician's office or emergency department visit. While valuable in providing some insight into the burden of injury on children's health, these studies are limited by their failure to assess the location (ie, body region) and severity of childhood injuries, basic characteristics that are considered essential in determining effective countermeasures.
The Abbreviated Injury Scale (AIS) is the most widely recognized injury severity scoring system based on ana tomical descriptors.7 The AIS rates the severity of an anatomical injury from 1 (minor injury) to 6 (generally nonsurvivable injury) for each of 8 body regions: head and neck, face, abdomen, thorax, spine, upper extremity, lower extremity and pelvis, and external and burns. The AIS is a well-validated system for assessing severity of anatomical injury, including injuries to children.8 In its original form, the AIS score for an individual injury was determined following a detailed review of a patient's medical record. More recently, MacKenzie et al9 have devised a computerized system for converting International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes for individual injuries into AIS scores. The AIS is structured in such a way as to make clinically important general distinctions in injury severity possible without detailed medical record information, raising the possibility of developing a telephone interview tool based on the AIS. To our knowledge, no investigators thus far have attempted to classify injuries using the AIS system via a telephone interview. This study represents the first attempt to bridge methods used in telephone interviewing with those typically reserved for medical record abstraction. The development of a telephone interview tool that can describe injuries with respect to body region and severity according to the AIS system would be particularly valuable for use in population-based or multi-institutional studies where obtaining hospital or physician's office records might prove difficult and/or expensive. Therefore, the objective of this study was to develop and pilot test a telephone-based survey instrument that enables parents to identify and characterize the body region and severity of childhood injuries using the AIS scoring system, and to effectively differentiate between clinically significant (AIS ≥2) and minor (AIS 1) injuries.
This study was performed as pilot work for a motor vehicle safety research project known as Partners for Child Passenger Safety, a multiyear project being conducted at The Children's Hospital of Philadelphia, Philadelphia, Pa, and the University of Pennsylvania, Philadelphia, with funding from State Farm Automobile Insurance Company, Bloomington, Ill.10 This project aims to provide a comprehensive evaluation of the state of child occupant protection through the development of a child-specific automobile crash database. One of the principal sources of data collection for the project is telephone interviews with drivers of vehicles and parents of child occupants designed to ascertain information regarding the circumstances of the crash as well as injuries to the child occupants. To make results of this project compatible with previous motor vehicle safety research, we sought to describe injuries using the AIS system
The survey tool, known as the Injury Severity Assessment Survey/Parent Report, was developed via a systematic review by 2 physicians (D.R.D., F.K.W.) of the AIS 1990 manual,11 resulting in the identification of clinically related groups of injuries with similar severity scores for each of the 8 body regions identified by the AIS system. For each group of related diagnoses, questions were developed regarding either the treatment received by the injured child (eg, "Was a tube placed in your child's chest to drain air or blood from it?") or about specific types of injuries (eg, "Did your child fracture his or her skull?"). Both general (eg, "Did your child receive a blood transfusion?") as well as body region–specific questions were developed. The objective was to generate questions that could distinguish clinically significant injuries, those rated as having an AIS score of 2 or more, from minor injuries. Examples of minor injuries (those coded AIS 1) include most lacerations, abrasions, and contusions of the skin, minor sprains of the extremities, and fractures of the fingers and toes. Answers to each question were then constructed in a way that enabled automatic coding of responses into AIS body regions and severity scores or ranges of scores. For example, if the answer to the question, "Did your child fracture any bones in his or her arms?" was yes, a follow-up question was asked inquiring about the specific area of the arm (fingers, hand, wrist, forearm, elbow, or upper arm/shoulder) that was fractured. This enabled the coding of the response according to the specific part of the upper extremity fractured, as well as the assignment of a severity score. Some injuries could not be coded into exact severity scores without relying on detailed information typically available only in medical records. This resulted in the coding of some responses into ranges of AIS severity scores. For instance, according to the AIS 1990 manual, severity scores of 2 through 5 can be assigned to liver lacerations, depending on the extent of damage to the organ. It is not common for physicians to provide to parents a description of their child's injuries with enough detail necessary for a parent to distinguish these various increments in severity. Therefore, responses to questions regarding many internal organ injuries were often coded according to the body region of injury (eg, abdomen), specific organ injured (eg, liver), and a range of severity scores (eg, AIS ≥2). The survey tool is available for use by other investigators.
Once the survey tool was developed, a prospective cross-sectional survey was conducted to validate it. Validation consisted of comparing a parent's descriptions of injuries with information documented in medical records. Subjects eligible for the validation study included all children younger than 18 years presenting to the emergency department of The Children's Hospital of Philadelphia for treatment of an acute injury between June 2 and August 22, 1997. Children admitted to the hospital, as well as those discharged from the emergency department, were eligible for inclusion. Children with penetrating trauma or intentional injuries, including cases of suspected child abuse, were excluded to avoid any instances of interpersonal violence that might affect the accuracy of the parent report of injuries. Children with fatal injuries were also excluded to avoid interviewing parents soon after the death of their child.
Eligible cases were identified through an emergency department–based injury surveillance system, consisting of routine review of all emergency department records by research personnel, that had been previously developed. Relevant information including the patient's name, age, mechanism of injury, body region(s) injured, and specific injury diagnoses were initially abstracted from the emergency department record on all children treated for injuries. All eligible cases were then stratified according to their body region(s) of injury and severity of injury (AIS 1 vs AIS ≥2) by 1 of the investigators (D.R.D.). Stratified sampling was then performed to ensure an adequate representation of injuries to all body regions as well as a full spectrum of injury severity in the final study sample.
The parents or guardians of patients selected for the study were contacted by telephone by a research nurse within 4 weeks from the date of the emergency department visit. After obtaining verbal consent, the survey was conducted by the research nurse who was blinded to information contained in the medical record. All medical records (emergency department and inpatient records) were then abstracted by separate study personnel blinded to the results of the telephone interview. Traditional AIS coding of all documented injuries was performed by personnel experienced with the AIS classification system. Data from the telephone interview were then linked to those obtained from the medical record abstraction into a single database for analysis.
The principal analysis for the study was the calculation of the sensitivity and specificity, with associated 95% confidence intervals (CIs), of the survey to detect injuries scoring 2 or higher on the AIS that were documented in the medical record. In addition, the κ statistic was calculated to determine the "chance-corrected" degree of agreement between parental response and medical record information. We hypothesized that the accuracy of parental report of the severity of an injury might vary by the body region injured. For example, parents may be able to identify and characterize the severity of a fractured extremity more accurately than a liver laceration or intracerebral hemorrhage. Therefore, analyses were conducted separately for patients with injuries to each of 3 general body region subgroups: (1) head, neck, face, and spine; (2) thorax, abdomen, and pelvic contents; and (3) extremities and pelvis. The study protocol was approved by the institutional review board of The Children's Hospital of Philadelphia.
During the period of study, contact was made with 159 parents who were invited to participate in the study. Twelve parents refused to participate and 147 parents agreed to conduct the interview, for a consent rate of 92.5%. Median time to interview was 15 days from the date of the emergency department visit, with a range of 3 to 28 days. All interviews lasted less than 10 minutes. Interviewees included 113 mothers (77%), 21 fathers (14%), and 13 other relatives (9%). Fifty-one (35%) of the interviewees were present with the child at the time of the injury. Selected descriptive information regarding the children who were the subjects of the interviews is provided in Table 1. A good distribution of all relevant age subgroups within the pediatric age range was achieved, with 11% of patients younger than 2 years and 23% of patients between 11 and 17 years of age. Boys outnumbered girls by a nearly 2:1 ratio. Falls accounted for the largest proportion of injuries, followed by bicycle-related injuries, sports or contact injuries, pedestrian injuries, and motor vehicle crash injuries. A broad distribution of injury severity was achieved in the final study sample, with nearly two thirds of patients having injuries scoring 2 or higher on the AIS documented in their medical records, and nearly half of all patients admitted to the hospital for treatment of their injuries. According to the medical records, 42 patients had head, face, neck, or spine injuries scoring 2 or higher, 44 had extremity injuries scoring 2 or higher, and 12 had chest or abdomen injuries scoring 2 or higher.
The sensitivity of the survey (its ability to detect injuries scoring 2 or higher on the AIS that were documented in the medical record) ranged from 88% to 95%, varying somewhat by the body region of injury. Parents were better able to identify and characterize the severity of injuries to the extremities (sensitivity, 95% [95% CI, 83%-99%]) than injuries to the chest or abdomen (sensitivity, 92% [95% CI, 60%-99%]), and the head, face, neck, and spine (sensitivity, 88% [95% CI, 74%-95%]). The specificity of the survey (its ability to rule out the presence of a significant injury when one was not documented in the medical record) was more than 95% in each of the 3 body region groups, with 95% CIs above 90% for all 3 regions. The κ statistics for the 3 body region groups ranged from 0.89 to 0.92, indicating outstanding agreement between parental report and medical record documentation of injury.12
All cases of disagreement in injury severity between parental report and medical record documentation were investigated in detail to identify areas for improvement in the survey. Both cases in which the parent failed to identify an injury scoring 2 or higher on the AIS that was documented in the medical record (n = 8) as well as cases where the parent reported an injury scoring 2 or higher on the AIS that was not documented in the medical record (n = 5) occurred. Half (n = 4) of the cases in which a parent failed to identify a significant injury involved patients with head injuries resulting in alteration of consciousness without a clear loss of consciousness. The survey included a question inquiring about loss of consciousness, but did not have a question inquiring about varying levels of consciousness such as lethargy, stupor, or amnesia, all of which are classified as injuries scoring 2 or higher on the AIS. In addition to these cases, 1 parent each missed a skull fracture, a liver laceration, a minor (Salter 1) fracture of an extremity, and a fracture that was initially missed by the emergency department staff and later diagnosed by a radiologist.
The cases of disagreement involving a parent report of a significant injury that was not documented on the medical record included 1 child with a head injury whose parent reported a loss of consciousness that was not documented, a child with an abdominal injury and hematuria (documented) whose parent reported a kidney injury that was not documented, and 3 parents who reported extremity fractures that were not documented in available medical records. Similar to the discussion above, the disagreement in reported loss of consciousness may be due to a perceived alteration in consciousness that was not well-documented in the medical record. The child with hematuria was evaluated with a computed tomography scan of the abdomen that failed to reveal an abnormality. It is certainly possible that a small renal contusion was present (and the parent may have been informed that a kidney was injured) but could not be detected by the scan. The available medical records of the remaining 3 children whose parents reported an extremity fracture did not include official x-ray readings, and no fracture was documented in the medical record. It is possible that fractures were diagnosed but could not be detected with the available records.
Results of this study demonstrate that a telephone survey can be developed that enables the characterization of specific injuries using a modification of the AIS system. While providing less detail than previous methods of determining AIS scores, this survey does result in a significant increase in the level of detail in injury characterization over previous telephone-based surveys.2- 6
Several investigators have demonstrated that parents are capable of recalling accurate information regarding the occurrence of injuries to their children.1,2,4- 6,13 All of these studies have used either telephone-based or written questionnaires to ascertain the incidence of medically attended injuries in various population-based samples of children. Of note, Pless and Pless13 and others have suggested that, owing to the multiple potential sites of care to which parents may take their children following an injury (eg, hospital emergency department, physician's office), parent reports of injuries may, in fact, be preferable to medical records given that multiple records would need to be reviewed to provide an accurate number of medically attended injuries. Only 1 previous study has attempted to gather more specific information regarding the nature and/or severity of a parent-reported injury. In a study of British children cared for at a single health center, Agass and colleagues1 determined the annual incidence of "accidents meriting medical attention" via a written survey of parents. In addition, the survey asked parents to describe the type of injury (fracture vs laceration vs contusion/abrasion vs other) to each child. Parent responses were compared with medical records maintained by the health center, and were found to accurately describe the type of injury in 80 (98%) of 82 instances. It is not clear why other investigators have not attempted to ascertain more detailed information regarding the nature and severity of child injuries from parent surveys. This may be due, in part, to an assumption that parents are not capable of accurately providing this type of information. Results of our study suggest that, within a short (4 weeks) interval from the injury, parents are able to describe their child's injuries with sufficient detail to enable characterization of the body region and severity of injury based on a well-validated severity classification system.
Results of this study also demonstrated that the survey was able to function as an effective screening instrument for the presence of clinically significant (AIS ≥2) injuries. The survey demonstrated excellent specificity (the ability to rule out a significant injury if one was not present) across all 3 body region groups. As a screening instrument, we were particularly interested in ensuring maximal sensitivity of the instrument (the ability to detect a significant injury if one were present). We hypothesized that the sensitivity of the survey might vary by the body region of injury, due to the nature of what constitutes an injury scoring 2 or higher on the AIS in each body region. The survey had excellent sensitivity (95%) for detecting significant extremity injuries. Virtually all extremity fractures (except fractures of the fingers and toes) are injuries scoring 2 or higher. These injuries are likely easy to understand, relatively common, and readily apparent to parents, thus improving their ability to accurately identify and characterize them. Injuries to the chest and abdomen scoring 2 or higher most commonly involve an injury to an internal organ, which by its nature is less readily apparent, less common, and perhaps more difficult for parents to identify and characterize. The survey demonstrated very good sensitivity for detecting significant truncal injuries (92%), though the limited number of patients with these injuries resulted in a fairly wide CI around the point estimate. The sensitivity of the survey to detect significant head injuries was somewhat lower (88%) than for the other body regions. Many head injuries are classified as scoring 2 or higher owing to an alteration in consciousness, which is often readily apparent to parents. However, the version of the survey tested in this study did not include a question regarding alterations in consciousness (lethargy, obtundation, or amnesia) which accounted for nearly all (4/5) of the missed significant head injuries. Of note, according to the interviewer, all parents of children with altered consciousness volunteered this information during the survey. The survey has subsequently been revised to include a question regarding alteration in consciousness. Therefore, its sensitivity to detect significant head injuries is likely higher in its current version.
As an example of a potential use of the survey, in the Partners for Child Passenger Safety Project the survey will be used to screen for significant injuries, with subsequent confirmation by medical record information. Children who screen positive for the presence of an injury scoring 2 or higher on the AIS, as well as a sample of children who screen negative, will have ICD-9-CM diagnosis codes abstracted from all relevant medical records and bills contained in their insurance claims file. These diagnosis codes will then be converted to AIS scores using the mapping software developed by MacKenzie et al.9 Use of the survey as a screening tool will therefore result in a more efficient identification of claims files for abstraction. Because the survey was developed to perform this specific function in a larger research project, we did not evaluate its ability to distinguish higher levels (AIS ≥3) of injury severity. A larger sample of injured children would be required to generate sufficient numbers of more serious injuries to warrant this analysis. As noted previously, the information required to distinguish higher levels of injury severity may not routinely be given by medical staff to parents in the course of discussing a child's injuries. The level of detail provided by this survey is likely sufficient for studies that attempt to determine the incidence of clinically significant injuries by specific body regions.
The performance of the survey may be determined in large part by both parent factors (eg, level of education) and hospital/physician factors (eg, communication skills). The setting for this study was a tertiary care pediatric hospital that is designated a level I pediatric trauma center. The emergency department and trauma program nurses and physicians at the hospital are very experienced in talking with parents about their child's injuries. The interviewees for this study were a representative sample of the population of patients who come to the emergency department for the care of injuries. These included both patients from the local neighborhood, a predominantly poor, urban, minority population, as well as patients referred to the hospital from the surrounding area. This combination of factors, while not particularly unique among pediatric hospitals in the United States, may have generated results for this study that are not generalizable to other settings. The current version of the survey was specifically designed to characterize childhood injuries via a parent report. With modification, the survey could be used to characterize injuries to adults via either self report or through a proxy. Ongoing validation of the survey instrument throughout the Partners for Child Passenger Safety project will be conducted to ensure its validity in different populations of patients.
Accepted for publication August 25, 1998.
This study was supported in part by a grant from the State Farm Automobile Insurance Co, Bloomington, Ill.
Presented at the Annual Meeting of the Ambulatory Pediatric Association, New Orleans, La, May 3, 1998.
Corresponding author: Dennis Durbin, MD, MSCE, Center for Clinical Epidemiology and Biostatistics, Room 711, Blockley Hall, 423 Guardian Dr, Philadelphia, PA 19104 (e-mail: firstname.lastname@example.org).
Editor's Note: It's nice to see continued refinement of survey tools. If we could only refine our prevention tools . . .—Catherine D. DeAngelis, MD
Durbin DR, Winston FK, Applegate SM, Moll EK, Holmes JH. Development and Validation of the Injury Severity Assessment Survey/Parent ReportA New Injury Severity Assessment Survey. Arch Pediatr Adolesc Med. 1999;153(4):404-408. doi:10.1001/archpedi.153.4.404