Note that although the effect was not statistically significant, patients with a right-sided paralysis behaved similarly to healthy nonparalyzed persons4 in that they rated expressions starting on the left side of the stimulus face as being more authentic (higher score) than those starting on the right side. Patients with a left-sided paralysis (A), on the other hand, showed a more anomalous pattern of authenticity ratings than those with right-sided paralysis (B) across OAS-side.
A, Patients with left-sided facial paralysis made more errors for happy expressions starting on the left side of the stimulus face and for angry expressions starting on the right. B, In contrast, patients with right-sided facial paralysis were more accurate in detecting the onset of facial expressions of both happiness and anger starting on the left side of the stimulus face.
This interaction, though statistically insignificant (P = .07), suggests that patients with right-sided facial paralysis perceive changes between facial expressions more rapidly, both in angry-to-happy and happy-to-angry trials. In contrast, patients with left-sided paralysis perceive changes more slowly, specifically in angry-to-happy trials.
eMethods. OAS Task Instructions
eResults. Full description of results for OAS task
eFigure 1. Errors in Phase 2 of OAS task
eFigure 2. RTs in Phase 2 of OAS task
Customize your JAMA Network experience by selecting one or more topics from the list below.
Korb S, Wood A, Banks CA, Agoulnik D, Hadlock TA, Niedenthal PM. Asymmetry of Facial Mimicry and Emotion Perception in Patients With Unilateral Facial Paralysis. JAMA Facial Plast Surg. 2016;18(3):222–227. doi:10.1001/jamafacial.2015.2347
The ability of patients with unilateral facial paralysis to recognize and appropriately judge facial expressions remains underexplored.
To test the effects of unilateral facial paralysis on the recognition of and judgments about facial expressions of emotion and to evaluate the asymmetry of facial mimicry.
Design, Setting, and Participants
Patients with left or right unilateral facial paralysis at a university facial plastic surgery unit completed 2 computer tasks involving video facial expression recognition. Side of facial paralysis was used as a between-participant factor. Facial function and symmetry were verified electronically with the eFACE facial function scale
Across 2 tasks, short videos were shown on which facial expressions of happiness and anger unfolded earlier on one side of the face or morphed into each other. Patients indicated the moment or side of change between facial expressions and judged their authenticity.
Main Outcomes and Measures
Type, time, and accuracy of responses on a keyboard were analyzed.
A total of 57 participants (36 women and 21 men) aged 20 to 76 years (mean age, 50.2 years) and with mild left or right unilateral facial paralysis were included in the study. Patients with right facial paralysis were faster (by about 150 milliseconds) and more accurate (mean number of errors, 1.9 vs 2.5) to detect expression onsets on the left side of the stimulus face, suggesting anatomical asymmetry of facial mimicry. Patients with left paralysis, however, showed more anomalous responses, which partly differed by emotion.
Conclusions and Relevance
The findings favor the hypothesis of an anatomical asymmetry of facial mimicry and suggest that patients with a left hemiparalysis could be more at risk of developing a cluster of disabilities and psychological conditions including emotion-recognition impairments.
Level of Evidence
Facial expressions of emotion, signaling a person’s feelings and intentions,1 can be difficult to interpret at times, calling for a fast and powerful “interpreter” system. They are rarely symmetrical across both sides of the face, and this asymmetry may carry information about an expression’s authenticity. The left side of the face is believed to be more expressive than the right side.2
This asymmetry of facial expression manifests not only in overall amplitude but also in speed of onset, which appears to vary depending on whether the expression is produced voluntarily or involuntarily. Spontaneously produced facial expressions of emotion begin approximately 20 milliseconds earlier on the left side of the face than on the right, while posed (ie, voluntary) expressions begin earlier on the right side than on the left.3 Observers are sensitive to these asymmetries, and facial expressions of happiness and anger are detected more rapidly and reliably, and judged as more authentic, if they began on the left side of the stimulus face.4
People spontaneously imitate facial expressions.5 This facial mimicry, which is difficult to suppress voluntarily,6 facilitates the recognition of the expression through afferent feedback to the brain.7,8 For example, it influences judgments of the authenticity of smiles,9 and the blocking of facial mimicry reduces the speed and accuracy of recognizing emotional facial expressions10-14 and reduces neural activation in the amygdala in response to emotional facial expressions.15
It is unclear whether facial mimicry is asymmetric, and if so, whether this asymmetry corresponds to that of the perceived face. In one study, stronger facial mimicry of happiness and anger was found on the left than on the right side of the face; however, asymmetry of the stimuli was not verified or controlled for.16 Therefore, it remains unclear whether spontaneous facial mimicry occurs at the same speed and intensity on both sides of the face or whether it presents an asymmetry that possibly reflects the asymmetry of perceived faces.
Based on reports of greater emotional expressivity and facial mimicry on the left side of the face,2,3,16 3 competing hypotheses can be specified concerning the type of asymmetry facial mimicry might obey. The hypothesis of absolute asymmetry considers facial mimicry to be a spontaneous facial expression, which, regardless of the asymmetry in the stimulus, will always be more pronounced on the left side of the mimicker’s face. Two other hypotheses assume that the asymmetry of facial mimicry can vary depending on the asymmetry in the stimulus. If facial mimicry operates in an anatomical way, we mimic with our left hemiface (from our point of view) what we perceive in another person’s left hemiface (from their point of view), and vice versa for the right hemiface. If facial mimicry’s asymmetry is specular, it will reflect the asymmetries in the stimulus but from the observer’s point of view. Although the hypotheses of an absolute or anatomical asymmetry appear more likely, specular asymmetry cannot be ruled out without controlling for the asymmetry of the stimulus.
In the present study, the hypotheses of an absolute, anatomical, or specular asymmetry of facial mimicry were tested in patients with unilateral facial paralysis by experimentally manipulating the asymmetry of movement onset in the stimuli. Patients with unilateral facial paralysis, while paralyzed and insensitive on one side of the face, retain normal movement on the nonaffected side of the face, offering a unique opportunity to investigate the nature of the asymmetry of facial mimicry.
Participants completed 2 tasks in the present study that required the detection of either the onset or offset of dynamic facial expressions. Previous work has shown one of these tasks to be a powerful elicitor of facial mimicry and also found that performance of this task is altered when facial mimicry is blocked or inhibited.11,17 Importantly, in the other task,4 asymmetrical stimuli were used, such that these facial expressions started slightly earlier either on the left or the right side of the face.
Performance was predicted to rely more on left-sided facial mimicry, either in an absolute or an anatomical way.16 According to the hypothesis of absolute asymmetry, patients with a left- compared with right-sided paralysis were expected to be impaired in the speed and accuracy of the detection of facial expressions, regardless of the asymmetry in the stimuli. Following the hypothesis of an anatomical asymmetry, however, patients with left-sided paralysis should be impaired in response to expressions starting on the left, while they should be superior to those with right-side paralysis when observing facial expressions starting on the right. Finally, the predictions according to the hypothesis of a specular asymmetry were the reverse of those for the hypothesis of anatomical asymmetry.
A total of 57 patients of the Massachusetts Facial Nerve Center participated in the study according to a procedure approved by the Harvard ethical committee (36 women and 21 men, aged 20 to 76 years; mean age, 50.2 years). All patients provided written informed consent. They presented flaccid (n = 26), hypertonic (n = 26) or mixed paresis (n = 5), which could be acute (<6 weeks, n = 5), subacute (6 weeks to 4 months, n = 3), or chronic (>4 months, n = 49), of their right hemiface (n = 35), or their left hemiface (n = 22). Injections of botulinum toxin had been given to 21 patients.
For each patient, facial function and symmetry were verified electronically with the eFACE facial paralysis scale,18 which can range from 0 (extreme malfunction or asymmetry) to 100 (normal movement and symmetry). The mean (SD) eFACE score of the participants was 74 (13). Patients with more severe paralysis had also been paralyzed for the longest time, as shown by a significant negative correlation between time of paralysis in months and eFACE scores (ρ = −0.16, P = .02).
The onset asymmetry (OAS) task resembled the one used by Carr et al.4 Stimuli consisted of short videos (3 seconds at 50 frames per second) showing expressions of happiness and anger in 4 avatar faces (50% male), which started either 20 or 400 milliseconds earlier (factor OAS-time) on the right or left side (factor OAS-side, from the perspective of the stimulus). Videos were created in FACSGen software,19 based on the Facial Action Coding System.20
The offset task resembled the one used in Korb et al.17 Videos (5 seconds at 60 frames per second), constructed in morphing software (Morpheus Photo Morpher, version 3.17), showed a full-blown expression of happiness gradually morphing into anger, or the reverse.
The OAS and offset tasks were presented in counterbalanced order on a Dell laptop computer with a 15.5-inch screen, placed on a table at approximately 20 inches from the sitting participant. The OAS task was composed of 2 phases that were completed in the same order by all participants. In phase 1, all 32 stimuli were presented once in random order. Participants rated perceived authenticity of the facial expression from 1 (not at all authentic) to 7 (very authentic) using the number keys on the keyboard (see eMethods in the Supplement). Each video was preceded by a fixation cross (duration, 1500-2000 milliseconds) and followed by the rating screen. In phase 2, all stimuli were shown again in random order, and participants indicated with the left and right arrow keys (using index fingers) the side of the screen on which the expression began. Participants’ responses, which ended the trial, could be given during the 3 seconds of stimulus presentation or, failing that, during an unlimited time indicated through text on the screen.
In the offset task, participants reported at which moment they perceived the offset of the initial facial expression by pressing the spacebar on the keyboard. Response time was measured during 72 movie clips, shown for their entire length, and preceded by 2 practice trials.
For phase 1 of the OAS task, 101 trials (5% of the data) were rejected because response times exceeded the participant’s average response time by more than 2 SDs.
Statistical analyses were carried out in R (http://www.r-project.org) with linear mixed models (LMMs) using the lme4 package.21 Models for the OAS task included as fixed effects OAS-side (right, left, from the perspective of the stimulus), OAS-time (20 milliseconds, 400 milliseconds), emotion (happy, angry), and the between-participant factor paralysis-side (right, left, from the perspective of the patient). The LMM for the offset task included as fixed effects emotion (happy-to-angry, angry-to-happy), and paralysis-side. Intercepts for participant and stimulus were included as random effect in all models. F tests were computed, and degrees of freedom were estimated using the Satterthwaite approximation. The OAS results are reported focusing on effects including the factor paralysis-side (for a complete description see eResults in the Supplement).
For phase 1 of the OAS task, an LMM of the ratings of authenticity (Figure 1) showed a significant main effect of emotion (F1.00-1649.92 = 109.5, P < .001), with higher mean (SD) authenticity ratings for happy (5.3 [1.7]) than for angry faces (4.4 [1.9]). There were also findings, though statistically insignificant, for effects of OAS-time (F1.00-1649.89 = 3.0, P = .08), with higher ratings for expressions with a 20-millisecond asymmetry (4.9 [0.8]) than for those with a 400-millisecond asymmetry (4.8 [0.8]), and for an OAS-time × emotion interaction (F1.00-1650.28 = 3.6, P = .06), due to higher ratings for anger with a 20-millisecond (4.5 [1.4]) than a 400-millisecond (4.2 [1.3]) asymmetry, while happy faces were rated similarly across both OAS-times (5.3 [1.2] and 5.3 [1.1], respectively). No other effects were significant (F < 1.7 and P > .19 for all). When including eFACE scores and duration of paralysis as covariates, we found that the results remained unchanged (with exception of a statistically insignificant finding for an effect of OAS-time (F1.00-1619.84 = 2.3, P = .12), suggesting that severity of paralysis did not affect authenticity judgments.
In phase 2 of the OAS task, an average of 8.8 errors (27.5% of trials) was found. An LMM of the number of errors (Figure 2) showed significant effects of OAS-time (F1-1753 = 282.0, P < .001), OAS-side × emotion (F1-1753 = 4.0, P = .04), OAS-time × emotion (F1-1753 = 4.0, P = .04), OAS-side × paralysis-side (F1-1753 = 6.5, P = .01), and OAS-side × paralysis-side × emotion (F1-1753 = 6.0, P = .01). No other effects were significant (F < 1.6 and P > .20 for all).
The OAS-side × paralysis-side interaction was owing to patients with a right-sided paralysis making more errors in response to expressions starting on the right (mean [SD] rating, 4.9 [2.8]) compared with the left side of the stimulus (3.8 [1.7]), while patients with a left-sided paralysis made more errors for expressions starting on the left than on the right (4.7 [2.1] and (4.2 [2.4], respectively). Finally, the 3-way OAS-side × paralysis-side × emotion interaction reflected participants with right-sided paralysis making more errors for both happy (2.6 [0.4]) and angry expressions (2.4 [0.4]) starting on the right compared with the left side of the stimulus (1.9 [0.2]) and 1.9 [0.2] for happy and angry, respectively). Patients with a left-sided paralysis, on the other hand, made more errors when expressions of happiness started on the left (2.6 [0.3]) than on the right (1.7 [0.3]), and made more errors when expressions of anger started on the right (2.5 [0.4]) than on the left (2.0 [0.2]).
An LMM of the response times showed significant main effects of OAS-time (F1.00-1222.45 = 282.6, P < .001), and emotion (F1.00-1122.50 = 6.3, P = .01), and significant interactions of OAS-side × emotion (F1.00-1223.22 = 6.3, P = .01), OAS-time × paralysis-side (F1.00-1221.33 = 13.5, P < .001), and OAS-side × OAS-time × emotion × paralysis-side (F1.00-1123.12 = 3.8, P = .05). There was also a statistically insignificant finding for an OAS-time × emotion interaction (F1.00-1121.45 = 2.8, P = .09).
The OAS-time × paralysis-side interaction was owing to a bigger difference between stimuli with 20-millisecond and 400-millisecond asymmetry in patients with a right-sided paralysis (difference of means, 837.8) compared with those with a left-sided paralysis (difference of means, 546.3). The 4-way OAS-side × OAS-time × emotion × paralysis-side interaction (eFigure 2 in the Supplement) is difficult to interpret, but note that while patients with a right-sided paralysis tended to detect expression asymmetries of both emotions faster when the expression started on the left side of the stimulus, patients with a left-sided paralysis showed a more complex pattern of results. These patients showed either no effect of OAS-side (OAS-time of 20 milliseconds) or an effect that differed by the emotion of the stimulus (OAS-time of 400 milliseconds). Taking the mean, we find an overall difference of 150 milliseconds.
In summary, in phase 1 of the OAS task, no significant effects involving paralysis-side were found, but patients with a right-sided paralysis reacted similarly to healthy, nonparalyzed participants in an earlier study by our research group4 in that they rated expressions of both emotions starting on the left side of the stimulus as being more authentic than those starting on the right. Patients with a left-sided paralysis, on the other hand, showed a more anomalous pattern of authenticity ratings across emotions and OAS-side. In phase 2 of the OAS task, participants with right-sided paralysis were faster and more accurate (mean errors, 1.9 vs 2.5; Figure 2) in detecting the onset of both happy and angry expressions starting on the left side of the stimulus. In contrast, patients with left-sided paralysis were more accurate when expressions of happiness started on the right and expressions of anger started on the left.
Analyses of response times in the offset task (Figure 3) resulted in a statistically insignificant finding for a main effect of emotion (F1.0-3749.3 = 3.2, P = .07), reflecting longer mean (SD) response times for angry-to-happy (2859.3 [643.7] milliseconds) than for happy-to-angry trials (2831.6 [683.4] milliseconds), and in a statistically insignificant finding for an emotion × paralysis-side interaction (F1.0-3748.9 = 3.3, P = .07). The interaction was due to slower response times in angry-to-happy trials in patients with left-sided paralysis (2903.9 [608.2] milliseconds) than in patients with right-sided paralysis (2832.5 [671.3] milliseconds), while response times in happy-to-angry trials did not differ between groups (2827.9 [603.6] and 2833.8 [735.7] milliseconds, respectively).
In summary, no significant results emerged for the offset task. However, patients with paralysis of the left side of their face tended to be slower at detecting the onset of happiness expressions than patients with right-sided paralysis.
This research tested 3 competing hypotheses about the asymmetry of facial mimicry. Patients with unilateral left- or right-sided facial paralysis completed facial expression processing tasks that are known to rely, in part, on facial mimicry. Overall, the results support the conclusion that paralysis of the respective left and right sides of the face have diverging consequences for emotion recognition and provide support for the anatomical hypothesis of facial mimicry.
For the accuracy of responses in the OAS task, a significant OAS-side × paralysis-side interaction was found (Figure 2; eFigure 1 in the Supplement). The pattern found for patients with right-sided paralysis resembles accuracy scores for angry expressions in healthy participants.4 When trying to detect the onset of facial expressions, patients with right-sided paralysis committed more errors in response to expressions starting on the right side of the stimulus face, while the reverse was true for patients with left-sided paralysis. Patients paralyzed on the left detected the onset of happy expressions more accurately when the expressions started on the right, compared with the left, side of the stimulus face, suggesting anatomical asymmetry. Angry expressions, however, elicited the opposite pattern of greater accuracy when starting on the left.
Response times in the OAS task (eFigure 2 in the Supplement) revealed that, as for accuracy scores and similar to healthy controls,4 patients with right-sided paralysis detected expression asymmetries of both emotions faster when the expression started on the left. This difference was more pronounced in trials with 20-millisecond OAS-time, possibly because response times in trials with 400-millisecond OAS-time (which were easier to detect) reached a floor effect. In contrast, response times in patients with left-sided paralysis differed by emotional expression (especially with 400-millisecond OAS-time), with faster response times for happy expressions starting on the right and for angry expressions starting on the left.
Analysis of ratings of authenticity in the OAS task did not reveal statistically significant effects. However, patients with right-sided paralysis behaved similarly to healthy nonparalyzed persons,4 giving higher ratings to expressions starting on the left side of the stimulus (Figure 1). Patients with left-sided paralysis, on the other hand, rated happy faces as more authentic when the happy expression started on the right, but perceived anger authenticity similarly to how anger authenticity was perceived by healthy nonparalyzed persons4 and patients with right-sided paralysis.
In the offset task, asymmetries in the onset or overall amplitude of the dynamic facial expressions were not manipulated. This task can therefore only inform us about whether facial mimicry is asymmetrical but not about the type of asymmetry (anatomical or specular). The offset task uses pairs of facial expressions that are slowly morphing into each other and that induce mimicry of the second emotion of each pair.17 Response times in the offset task resulted in a statistically insignificant finding for an emotion × paralysis-side interaction. Patients with left-sided paralysis were slower at detecting changes in angry-to-happy than in happy-to-angry trials. Response times in patients with right-sided paralysis did not differ between stimulus types. Although not conclusive, results of the offset task suggest that mimicry of happiness, but not anger, is impaired in patients with left-sided paralysis. Since stimulus asymmetry was not controlled, but likely was left lateralized,16,17 results of the offset task suggest an absolute or anatomical left-sided asymmetry of facial mimicry of happiness.
Overall, the results speak for an asymmetry of facial mimicry that can be stronger on the right or left side of the face, depending anatomically on the asymmetry in the perceived facial expression. Thus, if the expression occurs more rapidly (and possibly more strongly) on the right side of the stimulus face, facial mimicry will be more pronounced on the right side of the perceiver’s face, and the reverse is true for expressions starting on the left. Patients in the present study had unilateral facial paralysis and were unable to mimic with and move the affected side of the face. Therefore, this population constitutes an ideal sample to test the hypothesis of an asymmetry in facial mimicry, and we were able to demonstrate a double dissociation concerning the detection of lateralized emotion onsets. Facial paralysis specifically led to impairment in the detection of facial expressions starting on the same anatomical side of the face in the stimulus. However, the occurrence of facial mimicry, which was not measured in the current study, should be assessed in future studies with electromyography on both sides of the face while controlling the asymmetry in the stimuli.
Of clinical relevance, the anatomical asymmetry of facial mimicry found in the present study, in combination with the previously demonstrated greater emotional expressivity on the left side of the face,2,3,16 suggests that patients who are paralyzed on their left side might be more greatly impaired than patients with right-sided paralysis. Although not universally accepted,22 we believe that besides aiding in emotion recognition, facial mimicry also improves the quality of social exchange by establishing trust and by increasing liking rapport between communication partners.23 An incapacity to mimic smiling, in particular, has been associated with an increase in the severity of depressive symptoms.24 Therefore, patients with left-sided paralysis can be expected to be more at risk for depressive symptoms—a hypothesis we suggest should be tested in future studies. Indeed, across several measures and 2 tasks, patients with left-sided facial paralysis were impaired in the recognition of changes from anger to happiness but not the reverse. Speculatively, this emotion-specific effect in patients with left-sided paralysis could be due to the effect of depressive states on the detection of positive emotion.25
To conclude, our results suggest that facial mimicry is asymmetrical in an anatomical way such that expressions perceived on the right side of the face are mimicked with one’s corresponding right-side muscles. Since the left side of the face is more expressive, future research should explore whether patients with left-sided unilateral facial paralysis are more likely to exhibit a cluster of debilitating symptoms, including impaired emotion recognition, poorer emotional expression, and more severe depressive symptoms.
Accepted for Publication: December 3, 2015.
Corresponding Author: Sebastian Korb, PhD, Neuroscience Area, International School for Advanced Studies (SISSA), Via Bonomea 265, 34136 Trieste, Italy (email@example.com).
Published Online: February 18, 2016. doi:10.1001/jamafacial.2015.2347.
Author Contributions: Dr Korb had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Korb, Niedenthal.
Acquisition, analysis, or interpretation of data: Korb, Wood, Banks, Agoulnik, Hadlock, Niedenthal.
Drafting of the manuscript: Korb, Agoulnik, Niedenthal.
Critical revision of the manuscript for important intellectual content: Wood, Banks, Hadlock, Niedenthal.
Statistical analysis: Korb.
Obtained funding: Hadlock.
Administrative, technical, or material support: Korb, Banks, Agoulnik, Hadlock.
Study supervision: Banks, Niedenthal.
Conflict of Interest Disclosures: None reported.
Funding/Support: Ms Wood was supported by a National Science Foundation (NSF) Graduate Research Fellowship (2012148522); Dr Niedenthal was supported by grant NSF grant BCS-1251101.
Role of the Funder/Sponsor: The National Science Foundation had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Create a personal account or sign in to: