Figure 1. Photograph illustrating 7 relevant distances in facial paralysis. Lines represent (from top to bottom) top edge of eyebrow in midpupillary line (MPL), margin of upper eyelid in MPL, margin of lower eyelid in MPL, alar base, midupper lip position, oral commissure position, and midlower lip position. Horizontal black lines indicate height of these landmarks on the healthy side, and solid red lines indicate their position on the paralyzed side. The vertical lines represent facial midline (based on bisection of the interpupillary line) (black) and the actual center of the philtrum (red). A, Resting brow ptosis. B, Superior eyelid malposition. C, Inferior eyelid malposition. D, Nasal base ptosis. E, Midupper eyelip ptosis. F, Oral commissure malposition. G, Philtral deviation.
Figure 2. Graphic interface for the Facial Assessment by Computer Evaluation (FACE) program, which illustrates tabs to outline iris diameter and a pull-down menu specific to the parameters of interest.
Figure 3. Two patients with right facial paralysis performing the 7 standard expressions. Left: flaccid paralysis. Right: hypertonic paralysis. Note the difference in function based on the absence (left) or presence (right) of synkinesis.
Figure 4. Graph illustrating matched agreement between the 2 methods of determining oral commissure excursion with smiling. FACE indicates Facial Assessment by Computer Evaluation. Error bars indicate SD.
Figure 5. Graph illustrating the statistically significant difference in time it takes to analyze a resting photograph for 7 facial landmarks (P < .001). The Facial Assessment by Computer Evaluation (FACE) method took approximately one-tenth the time of the Photoshop measurements. Error bars indicate SD.
Figure 6. Graphs demonstrating the sensitivity of the Facial Assessment by Computer Evaluation (FACE) method in distinguishing between a diseased population and healthy volunteers. A, Brow ptosis at rest between facial paralysis patients and individuals with normal facial animation. B, Number of millimeters of commissure excursion with smiling. Note that the facial paralysis group shows essentially no commissure excursion, whereas individuals with normal facial animation show literature-matched values. Error bars indicate SD.
Hadlock TA, Urban LS. Toward a Universal, Automated Facial Measurement Tool in Facial Reanimation. Arch Facial Plast Surg. 2012;14(4):277-282. doi:10.1001/archfacial.2012.111
Author Affiliations: Department of Otolaryngology, Massachusetts Eye and Ear Infirmary and Harvard Medical School, Boston (Dr Hadlock); and Department of Computer Sciences, Massachusetts Institute of Technology, Cambridge (Mr Urban).
Objective To describe a highly quantitative facial function–measuring tool that yields accurate, objective measures of facial position in significantly less time than existing methods.
Methods Facial Assessment by Computer Evaluation (FACE) software was designed for facial analysis. Outputs report the static facial landmark positions and dynamic facial movements relevant in facial reanimation. Fifty individuals underwent facial movement analysis using Photoshop-based measurements and the new software; comparisons of agreement and efficiency were made. Comparisons were made between individuals with normal facial animation and patients with paralysis to gauge sensitivity to abnormal movements.
Results Facial measurements were matched using FACE software and Photoshop-based measures at rest and during expressions. The automated assessments required significantly less time than Photoshop-based assessments. FACE measurements easily revealed differences between individuals with normal facial animation and patients with facial paralysis.
Conclusions FACE software produces accurate measurements of facial landmarks and facial movements and is sensitive to paralysis. Given its efficiency, it serves as a useful tool in the clinical setting for zonal facial movement analysis in comprehensive facial nerve rehabilitation programs.
Facial paralysis arises from a broad array of pathologic conditions, ranging from congenital paralysis (in isolation or coupled with a syndrome) to acquired paralysis from infectious processes, trauma, benign or malignant tumors, chronic ear disease, neurosurgical conditions, and autoimmune disease. Other neurologic and neurodegenerative processes (most notably cortical and/or brainstem stroke, myasthenia gravis, tardive dyskinesia, and multiple sclerosis) likewise manifest quantifiable facial movement findings, whose fluctuations would be relevant to their management.
Although many surgeons have dedicated significant professional effort toward management of the paralyzed face and contributed in substantial ways to the facial nerve literature,1- 18 tailored management of facial nerve disorders is most effective when based on the specific examination of each facial zone, both in repose and with attempts at orchestrated facial movement. The change in resting positions and excursions over time yields prognostic information and information regarding the effectiveness of medical, surgical, or rehabilitative therapy. The lack of a precise, zonally based facial movement assessment tool that documents static facial landmarks at rest and excursion of relevant landmarks with involuntary expressions and voluntary attempts at movement has led to discordant management schemes. Thus, patients with the often devastating physiologic, functional, communication (verbal and nonverbal), and social consequences of facial paralysis are generally undertreated.
Currently, facial nerve assessments used in the clinical setting involve standard 2-dimensional photography and videography,19- 28 cumbersome distance measurements of points on the face in repose and during attempts at movement, subjective grading scales,29- 32 and survey gathering.33,34 Several centers have attempted to introduce more sophisticated algorithms by using 3-dimensional assessments, automation, and other technologies.35- 42 In response to the need for the development of a diagnostic assessment tool that could rapidly provide quantitative data regarding resting position and dynamic excursion of key facial structures, we developed a straightforward, Java-based software program, Facial Assessment by Computer Evaluation (FACE), that provides this information quickly and easily from standard patient photographs. In addition, we compared the accuracy and efficiency of our software with more traditional acquisitions of facial measurements using a ruler tool in Adobe Photoshop (Adobe Systems Inc).
In the current report, we describe normal resting facial distance relationships and excursion of relevant facial landmarks during expression using the FACE software to compare the efficiency of this software with standard photo-analysis methods and to demonstrate the sensitivity of the FACE software in discerning pathologic discordant movements in patients with facial paralysis. Our expectation is that this tool will permit rapid resolution of current controversies and the efficient, objective study of future questions in facial paralysis management.43
We built on a previously described Scaled Measure of Improvement in Lip Excursion (SMILE) program,44 which used a MATLAB-based image analysis software tool (Mathworks Inc) for quantification of oral commissure movement. The program was used as a template for building the more comprehensive FACE program. This expansion involved rewriting the program in Java format to make it universal to users without access to MATLAB and adding many other relevant facial landmark positions and movements, including brow height, upper and lower eyelid position in the midpupillary line, and midupper lip and midlower lip heights, to provide comprehensive output in facial paralysis management.
Figure 1 demonstrates 7 facial distances relevant in the paralyzed face at rest: brow ptosis, superior eyelid malposition, inferior eyelid malposition, nasal base ptosis, midupper lip malposition, oral commissure malposition, and philtral deviation toward the healthy side. The graphic interface included a pull-down menu (Figure 2) that was designed to introduce analysis features for each of these facial landmarks, all calculated from a single high-resolution frontal view of the patient's face in repose.
Of the nearly limitless ways in which humans may move the muscles of facial expression, we selected 5 movements important in human facial function and communication that receive attention by facial reanimation specialists: brow elevation with attempted brow raising, palpebral fissure narrowing with attempted eye closure, midupper lip excursion with attempted smiling, oral commissure excursion with attempted smiling, and midlower lip excursion with articulation of the sound “ee.” Figure 3 demonstrates these expressions in the flaccidly paralyzed face and in the hypertonic, frozen facial state often seen after Ramsay Hunt syndrome, Lyme-associated facial paralysis, and other severe but transient neural insults and highlights the vastly different problem areas according to the degree of flaccidity or hypertonicity.
Fifty healthy individuals underwent standard facial photography of the face with the head resting firmly against a head rest, both at rest and during 5 standard movements. Measurements were performed on both sides of the face. Straightforward verbal commands were used for brow elevation, eye closure, smiling, and saying the sound “ee” as in “cheese.” All photographs were analyzed using manual importing of photographs into Photoshop and scaling the photographs to the iris diameter (11.8 mm in humans) in the picture for normalization, as has been reported by others.45 A horizontal line was constructed through the pupils and a vertical line drawn to bisect the interpupillary line to facilitate measures using the built-in measuring tool. The time it took to perform this analysis for each photograph was recorded. The photographs were then analyzed using the automated FACE program, the time it took to perform this automated analysis for each patient was recorded, and the data were compiled for analysis. To test the sensitivity of the FACE software to discrete pathologic states (both static and dynamic), 2 common problems in facial paralysis were selected for comparisons with normative data: static resting brow ptosis and dynamic oral commissure excursion with smiling. Twenty patients with flaccid facial paralysis underwent FACE measurements of brow ptosis in preparation for brow ptosis correction. The degree of brow ptosis compared with the normal side was recorded and compared with our cohort of individuals with normal facial animation. Likewise, 20 patients with flaccid facial paralysis underwent calculations of oral commissure excursion and were compared with individuals with normal facial animation.
Each patient underwent pairwise assessment of Photoshop-based measures and automated FACE measures using 2-tailed t tests. P ≤ .05 was deemed significant. For comparisons among individuals with normal facial animation and the facial paralysis cohorts, the 2-tailed t test assuming unequal variances was applied, using P < .05 to determine statistical significance. Data for the time it took to perform analyses using the 2 methods were treated in the same way.
Of the photographs taken of the 50 individuals with normal facial animation, 46 sets of photographs were deemed acceptable for analysis by both Photoshop techniques and the FACE program. Four were eliminated based on minor head rotation from the plane of the camera, causing the automatically generated vertical line defining the midline in the FACE program to be placed inaccurately. Once recognized, the head rotation issue was resolved by the addition of padded sides to the headrest that ensured the perpendicularity of the camera to the facial plane.
For the face at rest, both programs predicted the same mean (SD) position of brow height, palpebral fissure width, and midupper and midlower lip position in the y-axis (Table). Interestingly, both programs identified modest brow height differentials between the right and left sides (1.2 [1.0] and 1.4 [1.2], respectively) in individuals with normal facial animation. Likewise, submillimeter differences were found from left to right in palpebral fissure width and lip heights as well, consistent with the findings of others that subtle facial asymmetries exist in most individuals with normal facial animation.
Analysis of facial measurements between rest and expression demonstrated consistency between both programs. For oral commissure excursion during smiling, the 2 programs agreed within 0.5 mm, with no significant difference between methods (Figure 4), and agreed with published normative smiling data. Midlower lip excursion differences between the right and left sides during the “ee” expression were also not statistically different using either method (1.1 [1.0] mm using Photoshop and 1.3 [0.7] mm using FACE). Palpebral fissure measurements during eye closure matched perfectly at 0.0 mm for all study participants using both methods because all study participants achieved normal, complete eye closure. For brow excursion analysis, Photoshop measurements revealed a mean brow excursion during maximal brow elevation of 7.6 (3.0) mm, whereas the FACE program yielded slightly higher excursion measurements of 9.0 (3.1) mm (P < .04).
For both methods, observers were familiarized with the steps necessary to make all relevant measurements and given an opportunity to repeat the analysis until a high comfort level was achieved. Once the observers thought they had minimized the length of time necessary to execute the measurements (ie, mastered the learning curve), analysis of the study data was begun in a timed manner. To complete the full set of measurements using the Photoshop method, a mean of approximately 14 minutes was necessary. However, using the FACE program, the mean time to complete a set of measurements was significantly shorter, at 1.3 minutes (Figure 5).
Statistically significant differences in resting brow position and dynamic commissure excursion were found when comparing healthy and paralyzed faces, demonstrating the sensitivity of the FACE to the diseased state (Figure 6).
Subjective and semiobjective facial grading scales have played an important role in studying and treating facial movement disorders, and many facial reanimation specialists have developed elegant and effective tools for determining facial function.19- 28,35- 42 Patient self-assessment and quality-of-life questionnaires have also contributed valuable information regarding the utility of various interventions in the paralyzed face.29- 34 Still, many physicians recognize that efficient, objective comparisons of outcomes after various medical, surgical, and rehabilitative procedures remain difficult. Although the technology to precisely quantify facial movement from still photographs and videographs is well developed, the software interfaces that make analysis efficient and accurate (and therefore routinely feasible) in the clinical setting have not previously been developed.
Recently, a smile analysis tool that permitted straightforward comparisons of oral commissure excursion between resting and smiling photographs was decribed.44 We began by performing analyses through importing photographs into Photoshop and using the measurement tool, scaled to iris diameter, and then constructed a simple MATLAB interface that permitted more rapid analysis than the Photoshop importing technique permitted. Subsequently, we discovered that, given the built-in iris diameter scale, even the MATLAB interface could be eliminated and that a universal, Java-based program could be designed to give not only commissure excursion information but also information regarding other relevant static and dynamic facial features.
In this report, we describe the use of this comprehensive facial analysis program and demonstrate that it generally predicts facial positions and distances as accurately as Photoshop importation. In healthy individuals, it predicts the same degree of commissure excursion as values reported in the literature. However, we demonstrate that the automated features, the user-friendly interface, and the pull-down menu features permit use of the FACE program in only one-tenth the time that more traditional techniques require. Moreover, we demonstrate its high sensitivity to the paralyzed state.
The range of error in reporting static and dynamic facial landmark positions with the FACE program was 0.4 to 3.5 mm, with the greatest error occurring while determining brow height during brow elevation and oral commissure excursion with smiling. This error range falls essentially within the parameters of facial asymmetry that have been shown to be easily overlooked by human observers naive to the presence of a facial difference.46 Moreover, when performing either static brow ptosis correction or dynamic smile reanimation, the expected facial landmark position changes before and after intervention (1.0-1.5 cm) far exceed the error range, indicating that the program would be likely to identify significant changes despite the measurement range.
To achieve the widespread use that would facilitate objective, multicenter participation required for maintenance of a nationwide or international database of patients with facial paralysis, it is essential that a facial assessment tool be both efficient and accurate. In addition, the tool is much more likely to enjoy widespread acceptance if it can be applied without relying on special apparatuses and lighting. We have established that the FACE program fulfills these 3 important criteria and, as such, may bring facial reanimation specialists closer to the widespread use of a single, universally acceptable facial landmark and excursion measuring tool that is desperately needed to make concerted progress in the challenging area of surgery for the paralyzed face.
Correspondence: Tessa A. Hadlock, MD, Department of Otolaryngology, Massachusetts Eye and Ear Infirmary, 243 Charles St, Boston, MA 02114 (firstname.lastname@example.org).
Accepted for Publication: January 24, 2012.
Published Online: April 16, 2012. doi:10.1001/archfacial.2012.111
Author Contributions:Study concept and design: Hadlock and Urban. Acquisition of data: Hadlock. Analysis and interpretation of data: Hadlock. Drafting of the manuscript: Hadlock. Critical revision of the manuscript for important intellectual content: Hadlock and Urban. Statistical analysis: Hadlock. Obtained funding: Hadlock. Administrative, technical, and material support: Hadlock and Urban. Study supervision: Hadlock.
Financial Disclosure: None reported.
Additional Contributions: Juan Malo, MD, provided technical assistance in this research.