Customize your JAMA Network experience by selecting one or more topics from the list below.
Rossin EJ, Szypko C, Giese I, Hall N, Gardiner MF, Lorch A. Factors Associated With Increased Risk of Serious Ocular Injury in the Setting of Orbital Fracture. JAMA Ophthalmol. 2021;139(1):77–83. doi:10.1001/jamaophthalmol.2020.5108
Are there clinical and radiographic features that are associated with increased risk for substantial ocular injury in patients with orbital fracture?
In a case series of patients who sustained orbital fracture, there were 5 features that were significantly associated with increased risk of substantial ocular injury: blunt trauma with a foreign object, inability to count fingers, roof fracture, diplopia on primary gaze, and conjunctival hemorrhage or chemosis.
These findings may help nonophthalmologists triage patients with orbital fracture in terms of the need for urgent ophthalmic evaluation.
Orbital fractures are common in ocular trauma, and there is a need to develop predictive tools to estimate risk of concurrent ocular injury.
To identify clinical and radiographic features that are associated with increased risk of substantial ocular injury in the setting of orbital fracture.
Design, Setting, and Participants
Retrospective consecutive case series of patients who sustained orbital fractures between 2012 and 2018. Examinations were done at 1 of 2 level 1 trauma centers in the emergency or inpatient setting. A total of 430 consecutive patients (500 eyes) between 2012 and 2017 met inclusion criteria for the training sample. After building a predictive model, 88 additional consecutive patients (97 eyes) between 2017 and 2018 who met inclusion criteria were collected as a test sample.
Main Outcomes and Measures
The primary outcome measure was substantial ocular injury distinct from orbital fracture.
The mean age of our patient population was 53.5 years (range, 16-100 years). The overall rate of substantial ocular injury was 20.4%, and the rate of injury requiring immediate ophthalmic attention was 14.4%. Five variables were found to be associated with increased risk of substantial ocular injury: blunt trauma with a foreign object (odds ratio [OR], 19.4; 95% CI, 6.3-64.1; P < .001), inability to count fingers (OR, 10.1; 95% CI, 2.8-41.1; P = .002), roof fracture (OR, 9.1; 95% CI, 2.8-30.0; P = .002), diplopia on primary gaze (OR, 6.7; 95% CI, 1.7-25.1; P = .003), and conjunctival hemorrhage or chemosis (OR, 4.2; 95% CI, 2.2-8.5; P < .001). The results were translated into a bedside tool that was tested in an independent group of eyes (n = 97) and found to be associated with substantial ocular injury with a 95% sensitivity (95% CI, 77.2-99.9), 40% specificity (95% CI, 28.9-52.0), 31.8% positive predictive value (95% CI, 27.5-36.5), and 96.8% negative predictive value (95% CI, 81.3-99.5).
Conclusions and Relevance
A minority of patients with an orbital fracture had a substantial ocular injury. Certain radiographic and clinical findings were associated with substantial ocular injury. Testing of the algorithm in prospective longitudinal settings appears warranted.
The triage and treatment of orbital fractures is common practice in emergency medicine. It is estimated that 10% to 25% of facial trauma that presents to the emergency department involves an orbital fracture.1 Fortunately, the rate of ocular injury in the setting of ocular fractures overall is low, with estimates that 73% to 92.3% of patients with an orbital fracture have no associated ocular injury.1-6 In addition, large percentages of ocular findings with ocular fractures are benign (eg, subconjunctival hemorrhage or periorbital swelling), and the injury does not lead to any ophthalmic intervention or long-term ocular sequelae. However, the nonophthalmologist does not typically have the materials and experience to do a dilated eye examination to determine whether there is a substantial ocular injury. Owing to specialist unavailability in some areas, it can be difficult for emergency department or facial plastics physicians to readily obtain an experienced ophthalmic evaluation for all cases.
Although many have studied this topic, a sound method for predicting ocular injury that is accessible to the nonophthalmologist is lacking. Many large case series estimate the rate of ocular complication but do not offer a strategy for prediction.3-11 Some studies have suggested that subjective visual symptoms, objective visual acuity, or pupillary light reflex can be used to screen for ocular injury2,3,5,8,9,12; however, most studies are small, do not specifically measure the accuracy of their predictive factors, are based partially on outpatient follow-up, and the predictive factors are not easily collected in the acute emergency setting. A test that is specifically intended for the acute emergency setting at the bedside is still needed.
We therefore set out to perform a retrospective case series of all consults that were requested from the Massachusetts Eye and Ear (MEE) Department of Ophthalmology owing to orbital fracture from affiliated hospitals between 2013 and 2018, with the goal of devising an algorithm to identify risk factors associated with serious ocular injury (Table 1) based on variables that are readily assessed at the bedside without ophthalmic training and that is applicable to all patients, regardless of alertness.
The study was a retrospective case series. The study was approved by institutional review boards at MEE and Massachusetts General Hospital and complied with the Health Insurance Portability and Accountability Act of 1996 and adhered to the tenets of the Declaration of Helsinki. The study was a retrospective medical record review and as such was exempt from the need for informed consent.
Cases between December 2012 and June 2018 were identified through a central ophthalmology consult list at MEE. After June 2017, owing to a switch in how consults were tracked, cases were identified through the MEE Information Services Department by electronically searching for all ophthalmology encounters labeled and billed as a consult. The case collection was divided into 2 stages, one for model generation and one for validation. During the first stage, we reviewed 571 consecutive consults for orbital fracture across 2 affiliated hospitals (Massachusetts General Hospital and Brigham and Women’s Hospital). During the second stage, we collected 100 additional independent consecutive consults between June 2017 and August 2018, which compromised stage 2 data collection.
We excluded patients who presented directly to MEE via the ophthalmic emergency department because these patients undergo a detailed ocular workup automatically and such a workup is not available in other settings. We excluded any consult that was for a primary reason other than orbital fracture and any consult without computed tomography evidence of orbital fracture. We excluded patients 15 years or younger because typically this age group is less reliable in terms of reporting of symptoms and compliance with testing. We chose 15 years as the cutoff because teenagers at age 16 years are seen as reaching an age of being more responsible and reliable (for example, this is the age at which they are able to apply for a driver’s license). Any consult on a pediatric patient 15 years or younger who has sustained trauma needs to be evaluated with a full ophthalmic examination regardless of initial signs or symptoms of ocular injury. In addition, patients who underwent canthotomy and cantholysis prior to ophthalmology consultation were excluded because their ophthalmic examination was influenced by this intervention. We allowed drowsy and intubated patients to be included even if they could not provide the answers to the subjective variables. We only included patients for whom a consult was called immediately (within 24 hours), and all consults at MEE are completed within 24 hours from the initial call. We therefore only included patients who presented first to a general emergency department (Massachusetts General Hospital or Brigham and Women’s Hospital) and for whom ophthalmology was called and performed a complete ophthalmic examination including dilated fundus examination. After applying the inclusion criteria, we included 430 patients (500 eyes) in the first stage (training) pool and 88 patients (97 eyes) in the second stage (validation) pool.
For the patients included in the training stage, we collected demographic, symptomatologic, clinical, and radiographic data on each patient (Table 2 and Table 3). Visual acuity was converted into a binary variable based on ability to count fingers, as this is an easy bedside check for any health care clinician. If any symptomatologic or subjective measurement was unknown owing to decreased level of patient alertness, it was recorded as “unknown” and kept in the model. We excluded any variable for which greater than 25% of patients were missing data and required that all eyes had full examination data for the remainder of the measurements. We further filtered the variables to only those that a nonophthalmologist could easily assess at the bedside and these included location of the consult within the hospital, age, type of fracture, mechanism of injury, other bodily injuries, ocular history, alertness of patient, subjective complaint of double vision, pain with eye movement, flashes or floaters, visual acuity (defined as ability to count fingers from at least 1-2 ft), eyelid injury, whether the eye was swollen shut and any amount of conjunctival hemorrhage (either flat or bullous), or conjunctival chemosis (14 variables in total were analyzed after filtering for missing data).
The type of fracture was simplified into 4 categories and was based on the official computed tomography radiologist report. These categories were internal (floor, medial, or lateral wall), roof, zygomaticomaxillary complex, or panfacial fracture (complex or multiple facial fractures in addition to an orbital fracture, ie, naso-orbital-ethmoidal fractures and Le Fort fractures).
The primary outcome variable was substantial ocular injury (Table 3). In Table 3, we further divide substantial ocular injury into 2 groups, one requiring urgent ophthalmic attention and the other being suitable for outpatient follow-up; however, for the analysis we used the single definition of substantial ocular injury to be conservative. Insubstantial ocular injury was defined based on the lack of any findings considered urgent or requiring close follow-up. An eyelid laceration was not included in the definition of severity because it alone is an injury requiring repair by a facial plastics specialist or an ophthalmologist and therefore by definition warrants intervention; however, we did include eyelid laceration as a predictor variable to test for its ability to predict other types of ocular injury. The individuals collecting the input data (C.S. and I.G.) were independent from the individual defining severity of ocular injury (E.J.R.).
With the first-stage analysis, we used stepwise multivariate logistic regression with categorical and numeric data wrapped in a bootstrapping framework on the full training data set (500 eyes). The output variable was presence or absence of substantial ocular injury as defined in Table 3. Owing to a very low intraclass correlation coefficient between eyes in patients with bilateral fracture,13,14 each eye was analyzed as an independent unit. Because the results of stepwise logistic regression can be highly variable in the setting of rare events (for example, inability to count fingers is relatively rare), we used a bootstrapping approach whereby we reran the model on random subsets of data (sized at 67%) 500 times to estimate the frequency with which each predictor variable was highlighted as a substantial component of the model. All P values from the logistic regression were 2-sided. The bootstrapping approach was based on a published method in R (bootstepAIC, part of the R open source computing package15). All statistical analyses were performed using R, version 3.6.2 (the R Foundation).
We then devised a user-friendly prediction algorithm to use at the bedside. Variables nominated for the bedside algorithm had to meet 2 criteria: (1) they had to achieve a P value of .0035 or less in the final stepwise logistic regression model, which corresponds to a Bonnferoni-corrected P value of .05 in the setting of 14 independent tested variables, and (2) they had to confer increased risk for a severe outcome in at least 80% of the bootstrap runs. Patients earn 1 point for each of the risk variables that met the aforesaid criteria. We simulated the use of the bedside tool by collecting a series of 88 consecutive independent consults (after filtering based on our inclusion criteria) and only extracting the variables from their medical record to be used in algorithm. In parallel, a separate physician reviewed these medical record and decided whether the patient had a substantial ocular injury based on the criteria outlined in Table 1.
The clinical demographics of our population are summarized in Tables 1 and 2. Most consults (83.9%; 370 patients) were directly from the emergency department, and nearly half of the orbital fractures were from patients having fallen (52.6%; 226 patients). A total of 80 patients (18.8%) were either drowsy or intubated. The mean (SD) age was 52.9 (21.3) years. The most common type of fracture was an internal fracture (orbital floor and/or medial wall, collectively 45.8% or 229 eyes) and the most common examination findings were eyelid ecchymoses (65.4%; 327 eyes) and subconjunctival hemorrhage (61%; 304 eyes).
Overall, we found the rate of substantial ocular injury requiring urgent ophthalmic evaluation to be 14.4% (72 eyes of 500 with at least 1 injury). If we include less severe injuries for which routine outpatient follow-up would suffice, the rate of ocular injury was 20.2% (101 eyes of 500 with at least 1 injury). For 78.6% of eyes, an insubstantial or no ocular injury was identified.
Table 4 summarizes the findings for our logistic regression. We identified 5 variables associated with risk for substantial ocular injury, and these included blunt trauma with a foreign object (odds ratio [OR], 19.4; 95% CI, 6.3-64.1; P < .001), roof fracture (OR, 9.1; 95% CI, 2.8-30.0; P = .002), conjunctival hemorrhage or chemosis (OR, 4.2; 95% CI, 2.2-8.5; P < .001), inability to count fingers (OR, 10.1; 95% CI, 2.8-41.1; P = .002), and diplopia on primary gaze (OR, 6.7; 95% CI, 1.7-25.1; P = .003). We found that other facial fractures or injuries had a protective effect (OR, 0.5; 95% CI, 0.3-0.9, P = .03), but this was minimal. We found no association for any of the other covariates.
To convert the results of the statistical model into a bedside tool, we devised a system where patients who have any of the risk variables described above should receive an urgent ophthalmic examination (Table 5). Those without such findings may be appropriate for outpatient ophthalmic follow-up within a week, rather than urgent examination.
We collected 88 new consults (97 eyes) meeting the same inclusion criteria as the training set. We simulated the use of the bedside tool by extracting only the 5 predictor variables from their medical records. We found our bedside tool to be a 95.4% sensitivity (95% CI, 77.2-99.9) and 40% specific (95% CI, 28.9-52.0), with a 31.8% positive predictive value (95% CI, 27.5-36.5) and a 96.8% negative predictive value (95% CI, 81.3-99.5). There was one missed case of substantial ocular injury that corresponded to a patient with a retinal tear who was incorrectly screened out by the algorithm owing to lack of any risk variables.
Urgent ocular injury in the setting of orbital fracture is an uncommon event, but better tools are needed to triage and predict the likelihood of a serious ocular injury in the acute setting. We have identified 5 features that can be assessed quickly at the bedside without any special training or equipment, and the presence of these features helps triage patients to appropriate level of care. Importantly, our approach was further tested in a set of 97 independent eyes and shown to have 95.4% sensitivity, although this was at the cost of specificity (40%). Because no method will be 100% sensitive, we advise that this tool be used as a way of triaging patients rather than specifically screening out patients. Any patient who does not earn a high score should still be scheduled for close outpatient follow-up.
The need for nonophthalmologists to triage high-risk patients for a complete ophthalmic examination has been studied many times.2-7,10-12 An important aspect to any predictive tool for this purpose is that it can be used in the acute traumatic setting and that it can be used by a nonophthalmologist. There are 2 notable studies from 1991 and 2019 that have examined the question of ocular injury in orbital fracture patients with large sample sizes (al-Quirany et al,2 1991, and Richani et al,12 2019), and these provide important foundations for our work. However, these algorithms were reliant on Snellen visual acuity, the studies excluded intubated or drowsy patients, and the data were in part collected in the outpatient setting (with up to 1 week follow-up time). While we are encouraged that we find similar results in terms of prevalence of orbital injury, there are unique requirements in the emergency and inpatient setting that lead to inaccurate Snellen acuity. It is often not feasible for a nonophthalmologist without access to proper optical correction and/or pinhole occluder and good patient cooperation to obtain such accurate acuity. Moreover, in the trauma setting, the eyelids may be swollen shut and there can be substantial ocular and periorbital pain, further precluding refined measurement of vision. Our algorithm only relies on measures of vision that can be performed quickly and without tools at the bedside, and we include drowsy and intubated patients. We find this to be critical to any practical algorithm given that nearly 20% of consults from this study were intubated or drowsy. As a result of these changes, the algorithm works well in the bedside setting and is user friendly for nonophthalmologists, making it a useful triage tool.
A good screening tool is one that rarely misses disease, and therefore it has high sensitivity. The desired specificity depends on how many nondiseased individuals one wants to screen out, but it is less crucial than sensitivity for screening. In a good screening tool for ocular injury in the setting of fracture, we should aim to identify as many high-risk cases as possible, and therefore false positives are of secondary importance. Examples of widely used bedside algorithms in general medicine include the 2-step glucose challenge test in pregnancy,16 the Wells criteria for pulmonary embolism17 and the CHA2DS2-VASc18-20 score for thromboembolic risk. The accuracy of these tests range from sensitivities of 82% to 98% and the specificities range from 56% to 65%.16,20,21 The specificity of our algorithm (40%) is likewise low, but our sensitivity (95.4%) is quite high in comparison (at the cost of lower specificity).
The main limitation of this study is that both the training data set and the test data set were retrospectively collected. We have not truly tested the accuracy of this algorithm in the hands of the emergency department or inpatient physicians, but rather we report results based on retrospective review of ophthalmology examinations. To truly test this algorithm, we will need to use a prospective, longitudinal study of orbital fracture cases seen initially in the emergency setting by nonophthalmologists.
An important limitation to the bedside tool is that it is not 100% sensitive, and there was one patient with an orbital fracture who had a retinal tear but did not have any of the risk factors identified. Therefore, any patient with orbital fracture should have routine outpatient follow-up with an ophthalmologist if they do not get a bedside examination in the setting of their acute care.
In addition, the tool has low specificity (40%) and a low positive predictive value (31.8%). This means that many people who do not have serious ocular injury will nonetheless undergo an acute bedside ophthalmic examination. Unneeded consults can increase length of stay and expenditure of resources for the hospital. However, one might be willing to accept the low specificity if the consequence of a missed case is high, as it is with substantial ocular injury. Because delayed care of a substantial ocular injury is more harmful than low specificity, we feel a high sensitivity is most important.
It should be noted that despite a low specificity, it is high enough that the test reduces the number of patients referred for ophthalmic consultation by 31% (examination for 30 of 97 patients would have been safely averted). This allows hospital and physician resources to be diverted to the 69% of patients who are more likely to have substantial ocular injury.
Finally, its high sensitivity (95.4%) and subsequently its high negative predictive value (96.8%) mean that a patient who tests negative (ie, does not meet any of the 5 risk features) is very unlikely to have a serious ocular injury. This meets the definition of a good screening tool.
Here, we describe a bedside algorithm for evaluating substantial ocular injury in patients with facial trauma and orbital fractures. The 5 variables associated with increased risk for ocular injury were subconjunctival hemorrhage or chemosis, orbital roof fracture, diplopia on primary gaze, inability to count fingers at more than 1 to 2 feet, and blunt injury with a foreign object. These variables are associated with increased risk for substantial ocular injury with high sensitivity (95.4%) and high negative predictive value (96.8%), with a tradeoff of lower specificity (40%) and positive predictive value (31.8%). These findings, if confirmed in independent prospective longitudinal investigations, would support raising suspicion for high-risk patients among those with substantial ocular injuries based on bedside evaluations of facial trauma and orbital fractures.
Corresponding Author: Elizabeth Rossin, MD, PhD, Retina Department, 243 Charles St, 3rd Floor, Boston, MA 02114 (firstname.lastname@example.org).
Accepted for Publication: October 5, 2020.
Published Online: November 25, 2020. doi:10.1001/jamaophthalmol.2020.5108
Author Contributions: Dr Lorch had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Dr Rossin and Ms Szypko contributed equally to this work.
Concept and design: Rossin, Giese, Lorch.
Acquisition, analysis, or interpretation of data: Rossin, Szypko, Giese, Hall, Gardiner.
Drafting of the manuscript: Rossin, Szypko, Giese.
Critical revision of the manuscript for important intellectual content: Rossin, Szypko, Hall, Gardiner, Lorch.
Statistical analysis: Rossin, Hall.
Administrative, technical, or material support: Rossin, Szypko, Lorch.
Supervision: Rossin, Gardiner, Lorch.
Conflict of Interest Disclosures: None reported.