External Validation of the Canadian CT Head Rule and the New Orleans Criteria for CT Scanning in Patients With Minor Head Injury

Context Two decision rules for indications of computed tomography (CT) in patients with minor head injury, the Canadian CT Head Rule (CCHR) and the New Orleans Criteria (NOC), suggest that CT scanning may be restricted to patients with certain risk factors, which would lead to important reductions in the use of CT scans.

Objective To validate and compare these 2 published decision rules in Dutch patients with head injuries.

Design, Setting, and Patients A prospective multicenter study conducted between February 11, 2002, and August 31, 2004, in 4 university hospitals in the Netherlands of 3181 consecutive adult patients with minor head injury who presented with a Glasgow Coma Scale (GCS) score of 13 to 14 or with a GCS score of 15 and at least 1 risk factor.

Main Outcome Measures Primary outcome was any neurocranial traumatic finding on CT scan. Secondary outcomes were neurosurgical intervention and clinically important CT findings. Sensitivity and specificity were estimated for each outcome for the CCHR and the NOC, using both rules as originally derived and also as adapted to apply to an expanded patient population.

Results Of 3181 patients with a GCS score of 13 to 15, neurosurgical intervention was performed in 17 patients (0.5%); neurocranial traumatic CT findings were present in 312 patients (9.8%). Sensitivity for neurosurgical intervention was 100% for both the CCHR and the NOC. The NOC had a higher sensitivity for neurocranial traumatic findings and for clinically important findings (97.7%-99.4%) than did the CCHR (83.4%-87.2%). Specificities were very low for the NOC (3.0%-5.6%) and higher for the CCHR (37.2%-39.7%). The estimated potential reduction in CT scans for patients with minor head injury would be 3.0% for the adapted NOC and 37.3% for the adapted CCHR.

Conclusions For patients with minor head injury and a GCS score of 13 to 15, the CCHR has a lower sensitivity than the NOC for neurocranial traumatic or clinically important CT findings, but would identify all cases requiring neurosurgical intervention, and has greater potential for reducing the use of CT scans.

Head injury is one of the most common injuries in the Western world with an estimated incidence of hospital-treated patients with minor head injury of 100 to 300 per 100 000 population.¹ Minor head injury is commonly defined as blunt trauma to the head, after which the patient has lost consciousness for less than 15 minutes or has a short posttraumatic amnesia of less than 1 hour, or both, as well as a normal or minimally altered mental status on presentation (a Glasgow Coma Scale [GCS] score of 13-15).^2,3

Intracranial complications of minor head injury are infrequent (6%-21%) but potentially life-threatening and may require neurosurgical intervention in a minority of cases (0.4%-1.0%).^3-8 Neurocranial injury that does not require neurosurgical intervention may still cause significant clinical problems; these patients will usually be kept under close clinical observation. Computed tomography (CT) of the head is the imaging modality of choice for diagnosing neurocranial traumatic lesions, such as skull fractures, epidural and subdural hematomas, and hemorrhagic contusion. In many Western countries, CT is therefore routinely used to evaluate patients with minor head injury for the presence of neurocranial complications.

A study by Haydel et al⁷ suggested that CT is only indicated in patients with minor head injury with 1 of 7 risk factors (the New Orleans criteria [NOC]) (Table 1). According to this decision rule, patients without any risk factors would not require CT scanning and implementation of this rule in the United States was estimated to reduce CT scans performed for minor head injury by 23%. A similar study by Stiell et al³ identified a different set of risk factors, the Canadian CT Head Rule (CCHR) (Table 1). The potential reduction in the number of CT scans by implementing this decision rule was estimated at 46%. Both decision rules had 100% sensitivity for identifying patients with traumatic brain injury, as is desirable according to a survey of emergency physicians, but both rules had low specificities.⁹

Proper external validation of these published decision rules is necessary before they can be implemented. Our goal was to externally validate these 2 decision rules, the NOC and the CCHR, in a large multicenter study in the Netherlands.

In our prospective multicenter study, data were collected on 3364 consecutively included patients between February 11, 2002, and August 31, 2004, in 4 Dutch university hospitals (Figure). Patients were included if they presented within 24 hours after blunt head injury, were older than 16 years, and had a GCS score of 13 to 14 or had a GCS score of 15 with 1 of the following risk factors: history of loss of consciousness, short-term memory deficit, amnesia for the traumatic event, posttraumatic seizure, vomiting, severe headache, clinical evidence of intoxication with alcohol or drugs, use of anticoagulants or history of coagulopathy, physical evidence of injury above the clavicles, and neurological deficit. Patients were excluded if a CT scan could not be performed due to concurrent injury or if there were contraindications to CT scanning.

After review of our study protocol, patient informed consent was waived by the institutional review board and medical ethical committee, because patients meeting our inclusion criteria routinely undergo a head CT scan according to most local hospital policies, as is recommended in the current Dutch guidelines.¹⁰

Patients were considered to have lost consciousness when reported by a witness or by the patient. Loss of consciousness was not considered an obligatory criterion for inclusion in study, as was the case in previously published studies, but rather as one of the risk factors for neurotraumatic findings. A deficit in short-term memory was defined as a persistent anterograde amnesia. If the patient could not recall the entire traumatic event, this was considered as amnesia for the traumatic event. Posttraumatic seizure was classified as either a witnessed or suspected seizure after the traumatic event. Vomiting included any emesis after the traumatic event. Headache included both diffuse and localized pain. No blood toxicology tests were performed to assess severity of intoxication; presence and severity of intoxication were evaluated clinically, evidenced by slurred speech, alcoholic fetor, or nystagmus. Anticoagulant treatment included only warfarin and not platelet aggregation inhibitors (eg, aspirin, clopidrogel). Presence of coagulopathy was assessed by patient history; no blood coagulation tests were performed. Physical evidence of injury was defined as clinically significant discontinuity of the skin or extensive bruising. Focal neurological deficit was defined as any abnormality on routine clinical neurological examination, indicating a focal cerebral lesion.

All patients were examined by a neurologist or a neurologist-in-training under the supervision of a neurologist. All included patients underwent head CT scanning following physical examination. CT scanning was performed according to a routine trauma protocol, which consisted of a maximum slice thickness of 5 mm infratentorially and 8 mm supratentorially, without intravenous contrast administration. All scans were interpreted by a neuroradiologist or a trauma radiologist in bone and brain window settings. The reading radiologist was not blinded to the patient’s clinical information, because all reading was performed in a clinical setting to evaluate the validity of the decision rules in daily practice.

Data were collected on patient and trauma characteristics (age, sex, time of injury and presentation, intoxication, anticoagulant treatment), accompanying symptoms (loss of consciousness, posttraumatic amnesia, posttraumatic seizure, short-term memory deficits, headache, vomiting), as well as on physical and neurological examination, CT findings, and the need for neurosurgical intervention. Selection of items was based on a literature review of published risk factors for intracranial complications after minor head injury. Data on patient history and examination were entered by the examining physician into a database¹¹ before the patient underwent CT, unless this interfered with the clinical work flow, in which case data were entered after the CT was performed.¹² CT findings were added separately by the reading radiologist (H.M.D., D.R.K., P.A.M.H., and H.L.J.T.). Data on neurosurgical intervention were collected by searching the included patients’ records in the hospital patient information system.

Our primary outcome measure was any traumatic finding of the neurocranium on the CT scan. Findings on the CT scan that led to neurosurgical intervention, although more important from a clinical point of view, were considered a secondary outcome, because of their low frequency and potential clinical variability across centers. However, because of their clinical significance, they will be reported first. A neurosurgical intervention was defined as any neurosurgical procedure (craniotomy, intracranial pressure monitoring, elevation of skull fracture, ventricular drainage) within 30 days after the traumatic event. Findings on the CT scan, which we considered to be important for clinical practice in that the patient would generally be admitted to hospital, were also considered a secondary outcome measure. These were defined as any intracranial traumatic finding on the CT scan, including depressed skull fractures.

To reliably validate the published decision models for predicting neurocranial traumatic findings on CT scan, a minimum of 100 events of our primary outcome were needed.^13,14 Given an incidence of traumatic findings on CT of 8% to 10%, at least 1250 patients who fulfilled the inclusion criteria of the original decision rules would need to be included.¹⁵

Patient data entered in the database were assessed by one of the authors (M.S.) for correct patient inclusion and for completeness of the data. Missing data of patients included in the analysis were assumed to be missing at random and imputed based on the available data means to avoid bias. The proportion of imputed missing data was 3.81%, which included both items documented as unknown and items that were not documented. We evaluated our patient population for demographic characteristics, mechanism of injury, traumatic findings, neurosurgical intervention, and occurrence of the risk factors of both decision rules.

We determined the sensitivity and specificity (and 95% confidence intervals [CIs]) for neurosurgical intervention, neurocranial traumatic findings on CT, and clinically important lesions on CT of both decision rules.¹⁶ The decision rule was considered positive when at least 1 of the risk factors was present. For the CCHR, in which a distinction is made between high-risk and medium-risk criteria, we chose not to use this distinction; therefore, all risk factors were considered equally important.

The published decision rules were designed for specific patient populations, which were more restricted than our patient population. We therefore first performed our validation analyses in the subgroup of patients for whom the decision rule was designed (Table 1); these decision rules are referred to as the original decision rules. We then adjusted the original decision rules for use in our entire study population, which also included patients without a history of loss of consciousness, by adding the exclusion criteria of the original rules as additional risk factors, which are referred to as the adapted decision rules. This means that the adapted NOC decision rule also included the risk factors neurological deficit and a GCS score of 13 or 14, and the adapted CCHR decision rule included the risk factors anticoagulation, posttraumatic seizure, and neurological deficit in addition to the original risk factors. The potential reduction in emergency CT scans was estimated by assuming that if the rule were to be adapted, then a positive result on the rule would be followed by a CT scan and a negative result on the rule would not.

Data were analyzed using SPSS version 12.0 software (SPSS Inc, Chicago, Ill); P<.05 was considered statistically significant.

The total number of patients presenting with head injury during our study at the 4 centers was estimated to be 6936 (Figure). A total of 3572 patients were not included because they did not meet the inclusion criteria. Of the 3364 patients originally included in the study, 112 did not meet the inclusion criteria on reassessment and were excluded from further analysis. One patient had a contraindication for CT, 1 was not seen by a neurologist, 16 patients did not have CT performed because of logistical reasons, 14 patients had no available data on patient history, and 39 patients had no available data on neurological examination. These patients were also excluded from further analysis, resulting in 3181 patients in the data analysis.

Patient characteristics are shown in Table 2. A total of 304 patients had neurological deficit, including 64 patients (21%) with lateralized motor weakness, 60 patients (20%) with lateralized sensory disturbances, and 251 patients (83%) with focal neurological deficits. Focal neurological deficits included, among other deficits, pathological reflexes (37%), nystagmus (19%), visual disturbances (9%), and pupil abnormalities (6%). In our study population, relatively few patients had posttraumatic seizure (0.7%), clinical signs of skull fracture (2.1%), or use of anticoagulation (6.9%).

Most patients presented with a normal GCS score of 15 (Table 3). Neurosurgical intervention was required in 17 patients (0.5%), which was performed for epidural hematoma in 8 cases, subdural hematoma in 3 cases, depressed skull fracture in 3 cases, and a combination of extra-axial hematoma and depressed skull fracture in the remaining 3 cases. Neurocranial traumatic lesions on the CT scan were found in 9.8% of the patients, with the highest proportion of traumatic findings in the category of patients with a GCS score of 13 (24.5%). The most common traumatic finding on the CT scan was a skull fracture (59.6%) (Table 4). Clinically important lesions were present in 243 patients (77.9%). Epidural hematoma was present in 11.2% of patients with traumatic findings; most of these hematomas were small with no or only localized mass displacement (25 of 35 cases) and were likely to be venous in origin in 4 cases. Subdural hematoma was present in 67 patients (21.5%) with traumatic findings on CT, and also was small in most cases with no (42 patients) or minimal (14 patients) mass displacement.

Notably, in 5 (29%) of 17 patients who underwent neurosurgical intervention, no history of loss of consciousness was present. A history of loss of consciousness was also absent in 85 (27%) of 312 patients with neurocranial traumatic CT findings and in 61 (25%) of 243 patients with clinically important CT findings.

For both the NOC and CCHR decision rules, both original and adapted, sensitivity for identifying patients who underwent neurosurgical intervention was 100% (Table 5). Sensitivity for neurocranial traumatic lesions on the CT scan, however, was not 100% for both rules. The adapted NOC reached the highest sensitivity for identifying patients with neurocranial traumatic findings on the CT scan (99.4%; 95% CI, 97.7%-99.8%); the original CCHR had the lowest sensitivity (83.4%; 95% CI, 77.7%-87.9%). Two patients with neurocranial traumatic CT findings were not identified using the adapted NOC rule. One of these patients with a nonhemorrhagic contusion would have been identified by the adapted CCHR because of the presence of prolonged (>30 minutes) posttraumatic amnesia. With the adapted CCHR, 47 patients with traumatic findings would have been missed; 46 of these patients would have been identified with the adapted NOC, because of the presence of external injury above the clavicles other than clinical signs of a skull fracture (41 patients) or headache (5 patients). Traumatic findings on the CT scan in these patients included skull fracture (n = 30), subdural (n = 5) and epidural (n = 2) hematoma, subarachnoid hemorrhage (n = 12), hemorrhagic (n = 11) and nonhemorrhagic (n = 1) contusion, and diffuse axonal injury (n = 2). One patient with diffuse cerebral swelling did not have any risk factors using either the CCHR or NOC decision rules. Sensitivity for clinically important traumatic CT findings was very similar to that for all neurocranial traumatic CT findings for both decision rules.

Specificity for neurosurgical intervention and neurocranial traumatic CT findings was very low for the adapted NOC decision rule but higher for the adapted CCHR decision rule (Table 5). Specificity for clinically important traumatic CT findings was almost identical to that for all neurocranial traumatic CT findings for both decision rules. The potential reduction in emergency CT scans by using these decision rules would have been higher with the adapted CCHR rule (37.3%; 95% CI, 35.6%-39.0%) than with the adapted NOC rule (3.0%; 95% CI, 2.4%-3.6%).

In this multicenter prospective validation study of 2 published decision rules for the use of CT scanning in patients with minor head injury, we found that both the NOC and the CCHR had 100% sensitivity for identifying patients who underwent neurosurgical intervention after minor head injury. This was true for both the original decision rules (when applied to the patient population these rules were designed for) and for the adapted rules applied to our entire study population. Sensitivity for neurocranial traumatic CT lesions or for clinically important lesions, however, was not 100% for both rules. The NOC decision rule had high sensitivity for neurocranial traumatic CT lesions, but the CCHR did not. The difference in sensitivities for neurocranial traumatic CT findings between the 2 decision rules seems to be mainly due to the more stringent use of the risk factor of external injury in the CCHR. In the NOC, this risk factor comprises all external injuries above the clavicles, whereas in the CCHR only external injury indicating a skull (base) fracture is considered a risk factor for neurocranial traumatic CT findings. Specificities for neurocranial traumatic CT findings and for neurosurgical intervention were low for the NOC decision rule, and higher for the CCHR but at the cost of a lower sensitivity for traumatic findings on the CT scan.

A single-center prospective study by Ibanez et al¹⁷ included 1101 patients and validated several guidelines and decision rules for minor head injury, including the CCHR and NOC. In this study, as well as in our study, none of the guidelines or decision rules reached a 100% sensitivity for traumatic lesions on the CT scan, with the NOC also reaching a higher sensitivity (95%) than the CCHR (86%). Unfortunately, Ibanez et al¹⁷ reported only sensitivities for relevant acute intracranial CT lesions and not the sensitivities for neurosurgical intervention. Furthermore, it is not clear from the article how the decision rules were validated, especially since that study population was not the same as the patient populations for which these guidelines and decision rules were designed.

The incidence of traumatic CT findings (9.8%) and of subsequent neurosurgical intervention (0.5%) in our study population correspond with findings in previous studies of patients with minor head injury (traumatic findings, 6.4%-21.0%; and neurosurgical intervention, 0.4%-1.0%).^3,7,8 Assault as the most common mechanism of injury is remarkable and may be due to the fact that 2 of our participating centers are in 2 large cities in the Netherlands, where crime rates are highest. Another reason may be that we included patients with minor head injury irrespective of loss of consciousness. Assault usually results in less severe injury than motor vehicle crashes or falls, which are the more common mechanisms of injury in similar studies.³

In our adaptation of the NOC and CCHR decision rules for use in our population, we did not include loss of consciousness as a risk factor, although loss of consciousness is generally regarded as a risk factor for complications after minor head injury. Loss of consciousness is often considered such a strong predictor that its presence usually is considered an obligatory part of the definition of minor head injury. The absence of loss of consciousness, however, does not exclude the possibility of neurocranial lesions, some of which may require neurosurgical intervention, although with a lower incidence than when the patient has lost consciousness.¹⁷ Our results emphasize this point, since almost 30% of patients requiring neurosurgical intervention had not lost consciousness. Clearly, certain patients without a history of loss of consciousness after minor head injury are also at risk of serious complications and therefore require a CT scan. By not including loss of consciousness as a risk factor in the adapted NOC and CCHR decision rules and validating them as such, the reported sensitivities and specificities of the adapted decision rules are valid for any patient with minor head injury, irrespective of whether loss of consciousness was present. Our simple adaptation to the 2 published decision rules may not be the optimal strategy, but it was beyond the scope of our study to design an entirely new decision rule. Designing a new decision rule, which is also applicable to patients with minor head injury but without loss of consciousness, may however be better than these simple adaptations to existing decision rules, and will require further research.

A limitation of our study is that we did not use the exact predictors that Stiell et al³ defined in the derivation of the CCHR. In that study, Stiell et al³ found that a GCS score of less than 15 at 2 hours after presentation to the emergency department was a risk factor for clinically important brain injury and neurosurgical intervention. In our participating centers, however, most patients underwent CT scanning within 2 hours of presentation. Therefore, we evaluated GCS at 1 hour after presentation instead of after 2 hours. Another predictor for clinically important brain injury according to the CCHR is amnesia preceding the traumatic event (retrograde amnesia) of more than 30 minutes. Our neurologists found it difficult to assess the duration of retrograde amnesia, which makes it an unreliable risk factor. Estimation of posttraumatic or anterograde amnesia is easier and seems to be more reliable. Since posttraumatic amnesia was also significantly associated with brain injury in the study by Stiell et al,³ we chose to use posttraumatic amnesia of more than 30 minutes as a risk factor instead of retrograde amnesia. In addition, the CCHR defined vomiting as involving more than 1 episode of emesis, whereas we defined vomiting as any period of emesis. It seems unlikely that the low sensitivity of the CCHR for traumatic lesions is due to our slight adjustments to the decision rule. If anything, the contrary is more likely since our adjusted predictors regarding the GCS score and vomiting are less restrictive than the CCHR original predictors.

Another limitation is that data on patient history and examination, although documented before the CT scan, were not always entered into the database before the CT scan was performed in those cases when data entry before the CT scan would have interfered with patient management. This may have caused some classification bias. In addition, we did not exclude patients who presented with minor head injury after a seizure and whose physical and neurological examination postictally is difficult to interpret. There is also a theoretical possibility that we may have missed patients undergoing neurosurgical intervention, who were initially discharged from 1 of the participating hospitals, and who later deteriorated and underwent emergency neurosurgical intervention in a different hospital. This is a highly unlikely event since the centers participating in our study are primary regional neurosurgical centers, which would mean that patients who would present to a different hospital than 1 of our participating centers would very likely be transferred to the participating center for neurosurgical intervention.

Despite these limitations, we were able to validate 2 published decision rules for the indications for CT scanning in patients with minor head injury in a large study population and multicenter setting. The NOC, both in its original form and adjusted for use in our entire patient population, reached the highest sensitivity for identifying patients with neurocranial traumatic lesions, but since its specificity was very low, the potential reduction in CT scans by implementing this rule would be much lower than estimated by Haydel et al (3% vs 22%).⁷ The potential reduction in CT scans by using the adapted CCHR decision rule in the entire patient population may be considerable (37%). However, the adapted CCHR attained a sensitivity for traumatic CT findings of only 83% to 87%, which may be too low for clinical practice, although the sensitivity for neurosurgical intervention was 100%.

A key clinical question is whether 100% sensitivity is needed for identifying patients with any neurocranial traumatic CT finding. The reason to perform an emergency CT scan in patients with minor head injury is to detect intracranial complications of minor head injury that require neurosurgical intervention or that cause significant clinical problems for which close clinical observation is needed. For this reason, we also validated the decision rules for clinically important lesions, which generally require clinical observation.

Still, the question remains whether all of the patients with a clinically important lesion on the CT scan really did require observation and, consequently, whether a CT scan was really indicated. A CT scan is certainly indicated in patients with traumatic lesions requiring neurosurgical intervention; however, we feel that focusing solely on neurosurgical intervention may be too restrictive. We therefore think it would also be important to assess the outcomes in terms of costs and effectiveness of head injury to identify those patients who may benefit from an emergency CT scan.¹⁸ In this respect, it is important to recognize that if the CT scan is negative and the patient may then be sent home without further clinical observation, this would result in both a gain in effectiveness (less days in hospital for the patient) as well as in cost-savings both for the health care system and society (a CT scan costs less than hospital admission).¹⁹ The magnitude of these effects will need to be ascertained, which can then be used to determine the optimal trade-off between sensitivity and specificity, and may help to formulate an evidence-based approach for reducing CT scans in patients with minor head injury.

In summary, based on application of the CCHR and the NOC in 3181 patients, the adapted NOC decision rule appears valid for use in all patients with minor head injury who are 16 years or older and have a GCS score of 13 to 15, irrespective of loss of consciousness. However, the potential reduction in CT scans performed for this indication is extremely low in the clinical context studied. The adapted CCHR has a lower sensitivity for traumatic findings on the CT scan, but would identify all patients requiring neurosurgical intervention. Further research is needed to identify patients with neurocranial injury who do not require neurosurgical intervention but may benefit from emergency CT scanning and to determine the optimal trade-off between sensitivity and specificity for a decision rule for CT scanning in patients with minor head injury based on cost and effectiveness outcomes.

Corresponding Author: M. G. Myriam Hunink, MD, PhD, Departments of Radiology and Epidemiology and Biostatistics, Erasmus MC–University Medical Centre, PO Box 1738, 3000 DR Rotterdam, the Netherlands (m.hunink@erasmusmc.nl).

Author Contributions: Dr Hunink had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Smits, Dippel, Hunink.

Acquisition of data: Smits, Dippel, Haan, Dekker, Vos, Kool, Nederkoorn, Hofman, Twijnstra, Tanghe.

Analysis and interpretation of data: Smits, Dippel, Hunink.

Drafting of the manuscript: Smits, Dippel, Hunink.

Critical revision of the manuscript for important intellectual content: Smits, Dippel, Haan, Dekker, Vos, Kool, Nederkoorn, Hofman, Twijnstra, Tanghe, Hunink.

Statistical analysis: Smits, Dippel.

Obtained funding: Hunink.

Administrative, technical, or material support: Smits, Dippel, Haan, Dekker, Vos, Kool, Nederkoorn, Hofman, Twijnstra, Tanghe, Hunink.

Study supervision: Dippel, Hunink.

Funding/Support: This study was supported by grant VAZ 01-104 from College voor Zorgverzekeringen.

Role of the Sponsor: The authors’ work was independent of College voor Zorgverzekeringen. College voor Zorgverzekeringen had no involvement in the study design, data collection and analysis, writing of the manuscript, or in the decision to submit this article for publication.

Acknowledgment: We thank Jolanda Brauer, RN, Department of Neurology at University Medical Center Nijmegen St. Radboud, as well as Wibeke J. van Leeuwen, RN, Caroline H. van Bavel-van Hamburg, RN, and Belinda Tara-Prins, RN, Department of Radiology at Erasmus Medical Center, for their invaluable contribution to patient data collection. Their work was funded solely by College voor Zorgverzekeringe and the medical centers involved in the study.