File Name: the mechanism of human facial expression book.zip
Our daily lives are replete with a mixture of true and fake expressions. Nevertheless, are these fake expressions really effective at hiding our true emotions? Here we show that brief emotional changes in the eyes micro-expressions, thought to reflect true emotions can be successfully concealed by follow-up mouth movements e.
In the same manner as backward masking, mouth movements of a face inhibited conscious detection of all types of micro-expressions in that face, even when viewers paid full attention to the eye region.
This masking works only in a backward direction, however, because no disrupting effect was observed when the mouth change preceded the eye change.
These results provide scientific evidence for everyday behaviours like smiling to dissemble and further clarify a major reason for the difficulty we face in discriminating genuine from fake emotional expressions. To what degree is it possible to distinguish genuine from fake facial expressions?
Judging the authenticity of facial expressions is indeed an important cognitive skill in various situations, such as criminal forensic investigations 1 , 2 , airport security 3 , 4 , business 5 , 6 , health 7 , 8 , education 9 , 10 and politics Previous studies have mainly focused on how people discriminate genuine from fake smiles, showing that the major difference lies in muscular activity around eyes While both genuine and fake smiles share a contraction of the zygomaticus major muscle that pulls up the corners of the mouth, only the genuine smile recruits the orbicularis oculi muscle around the eyes Another cue to judge smile authenticity involves the dynamic features of facial expressions.
Because actions of several facial muscles, especially ones strongly related to emotions, cannot be controlled voluntarily 15 , deceptive expressions often involve a leakage of true emotions as a momentary change in facial parts Though it is still somewhat controversial 4 , several studies have suggested that micro-expressions may serve as cues for authenticity judgment In addition to micro-expressions, many static and dynamic cues have been studied for their ability to distinguish genuine from fake smiles, such as left-right asymmetry 20 , the velocity of facial movements 21 , 22 and rates of eye blinking It has also been shown, however, that the rich variety of potential cues may not be fully exploited in actual authenticity judgements.
This low capacity is not easily improved by learning or experience 1 , 30 , 31 and it remains unclear why we do not utilize various authenticity cues in actual situations Some studies explain the difficulty in authenticity judgements by pointing out the frequent errors people commit when detecting lies, such as the application of cognitive biases e. Other studies have focused on more perceptual or attentional factors that undermine the lie detection process 24 , Nevertheless, the validity of this account based on the limitations of perceptual and attentional mechanisms perceptual-attentional limitation theory remains a subject of debate 25 , 31 , Although many previous studies, especially those focusing on the perceptual-attentional factors mentioned above, have investigated cognitive processes of authenticity judgments using static single pictures of faces, facial expressions in real-world situations normally involve dynamic changes in different facial parts at different timings.
This seems especially likely, considering that the smiling mouth is the most salient feature of all facial expressions Thus, even when micro-expressions reflecting a true emotion are momentarily revealed by the eyes, the smiling mouth might conceal or neutralize these leakages, resulting in poor performance in authenticity judgements in the real world. Based on this idea, we examined whether the detection of micro-expressions in the eye region is actually prevented by mouth movements.
As shown in Fig. These dynamic changes were created via morphing still images in a facial expression database see Methods. We set four conditions in which an eye change micro-expressions and a mouth change from neutral into a smile were independently manipulated. Note that although faces in Figs 1 , 2 , 3 , 4 are depicted using line drawings, for copyright reasons, all experiments were actually performed using photographs of real faces taken from a database developed for academic researchers; see Methods.
If the mouth change disrupts the detection of the eye change, task accuracy might be lowered in a condition where both the eyes and mouth change simultaneously condition 4 in Fig. We independently manipulated changes in eye upper and mouth lower regions of a face across trials.
In this figure, an eye region briefly changes into a happy expression in conditions 2 and 4 micro-expression , while conditions 3 and 4 involve movements of a mouth. The hit rates were decreased for all micro-expressions in the MC condition red solid lines compared to the NC condition black solid lines , though FA rates were comparable between conditions dotted lines.
There was no main effect of the eye changes or an interaction. In these and subsequent figures, all error bars denote standard errors SEs across participants. Note that, although faces in Figs 1 , 2 , 3 , 4 are shown in line drawings for copyright reasons, all experiments were actually performed using photographs of real faces taken from a database developed for academic researches ATR DB See our previous study for sample images from this database Stereoscopic images when fused between two eyes are shown in the rightmost panels.
In the behind condition upper panels , upper and lower parts of a face were perceived to be stereoscopically behind a random-dot pattern, which enabled the participant to see a coherent image of a face behind an occluding pattern amodal completion.
This integrity perceptual continuity between the two parts was disconnected in the front condition lower panels where they were presented stereoscopically in front of the random-dot pattern. A background shading denotes SEs across participants in the NC condition. Perceptual bias induced by a smile 45 , Although the left and right faces share the same image of eyes with a neutral expression, the eyes on the right face are perceived to be happier than the left ones. Figure 1c shows hit solid lines and false-alarm FA, dotted lines rates for the detection task of micro-expressions of the eyes.
Trials of female and male faces were pooled. For all six types of micro-expressions, we found a decrease in the hit rates when an eye change was accompanied by a mouth change MC, mouth change condition, red lines compared to when not NC, no mouth change, black lines , although the FA rates were comparable between those two conditions. No main effect of the eye changes F 2. These results indicate that a dynamic change from a neutral to a smiling mouth disrupted the detection of all types of micro-expressions tested in the present study.
It remained to be elucidated in Experiment 1 why the task-irrelevant movements of the mouth prevented a conscious detection of micro-expressions in eyes. One possibility was that mouth movement distracted attention to the eye region. Another possibility involves the holistic or object-based processing of faces. In the high-level visual cortex along the ventral pathway, multiple features of visual stimuli are integrated into a representation of an object A face especially an upright face is a typical example of this object-based processing Many studies further emphasize the role of holistic face processing 38 whereby facial parts e.
In order to clarify these two possibilities, in Experiment 2 we employed a psychophysical paradigm called amodal completion Specifically, we created two conditions behind and front where an identical face image was perceived to be in different depth positions 3D layers by manipulating a disparity in binocular images. In the behind condition upper panels in Fig.
This facilitated a perceptual integration of the two parts, enabling the participants to see a coherent image of a face behind the occluding dot pattern amodal completion. Meanwhile, the integrity perceptual continuity between the two parts was broken in the front condition lower panels in Fig. We conducted a three-way ANOVA on the three types of micro-expressions, the presence MC or absence NC of mouth change and the depth of stimuli behind or front condition.
In Experiments 1 and 2, we showed that a detection of micro-expressions in the eye region was prevented by simultaneously presenting a mouth change. In Experiment 3, we further investigated whether this prevention would also be observed when eye and mouth changes occurred in separate time windows. Specifically, we tested a condition where mouth movement preceded eye change Exp. If reduced sensitivity to micro-expressions Figs 1d and 2b resulted from a temporal interaction between two visual events e.
Mouth change thus did not affect a perception of eye change when it preceded eye change. Figure 3b shows the results of Experiment 3b. Because we repeated t -tests four times, p -values were corrected according to the Bonferroni method. Significant differences were found in NC vs. These asymmetric results between Experiment 3a and 3b were not caused by a difference in task difficulty. Taken together, our results indicate that micro-expression perception is not disrupted by a mouth change preceding an eye change Exp.
The previous experiments used a smiling mouth as the facial expression displayed in the lower half of stimuli presentations. In Experiment 4, we explored whether a similar masking effect could be induced by various types of mouth changes other than a smile. Trials with six types of mouth changes MC trials; happiness, anger, disgust, sadness, fear and surprise were randomly intermixed with the NC no mouth change trials in each session of Experiment 4. Although perception of facial expressions has been traditionally studied using static images, recent studies emphasize the importance of dynamic aspects of expressive changes 40 , 41 , 42 , 43 , The present study provides an example of a dynamic interaction between the upper and lower halves of the face that can hamper our authenticity judgements in the real world.
Although the numbers of participants in our experiments were relatively small 10—19 , these experiments evince converging results that a dynamic change in the mouth region inhibits the detection of a brief change in the eye region. Why do mouth movements affect the perception of changes in a distant eye region? One key factor could be a perceptual bias caused by the smiling mouth 45 , It is known that a picture of a smiling mouth biases perceived expression in other facial parts e.
Interestingly, the reduced sensitivity to micro-expression was not observed when we disconnected perceptual continuity between upper and lower facial parts front condition in Exp. Therefore the failure to detect a change in the eye region was induced only when those two parts were recognized as a unified object in the brain. In the high level visual areas along the ventral pathway, multiple features of visual stimuli are integrated into a representation of an object An important role of perceptual continuity shown in Experiment 2 suggests that the masking effect in the present study emerges from object-based processing in high level visual regions.
One should note that the present masking effect is different from emotional neutralization or mixture between positive and negative expressions 47 , For example, participants in Calvo et al. On the other hand, our present effect represents a perceptual as opposed to emotional masking, a more radical type of prevention in which emotional signals in the eye region are made undetectable to observers. Contrary to the concept of neutralization, our smiling mouth equally disrupted the detection of eye changes Exp.
The fact that the masking effect was maximized when a mouth change followed an eye change Exp. Since the facial leakage of a true emotion including micro-expressions cannot be controlled voluntarily 15 , 16 , it is probable that such automatic facial changes arise quickly and reflectively in response to triggering events, followed by slow voluntary facial movements that act to hide them.
The backward nature of the present effect Exp. Thus the follow-up mouth movements might reflect a highly-adaptive strategy utilizing a neural mechanism that inhibits conscious perception of a preceding event.
In conclusion, the present study demonstrated a new type of dynamic masking between the upper and lower halves of the face. This masking effect was so strong that an insensitivity to micro-expressions unavoidably occurred even when full attention was paid to the eye region. Furthermore, this masking can be induced by a wide range of mouth movements Exp. Beyond their scientific implications, we hope our findings will provide practical insight for various fields concerned with decoding the truth of emotions, including but not limited to law enforcement 2 and international security 3 , 4.
We conducted five Exps.
The multitude of facial expressions that humans are capable of is particularly potent in capturing attention. Facial expressions provide crucial information regarding a person's internal state and intentions and therefore the rapid recognition of these expressions can facilitate efficient social interaction. This article reviews evidence from a number of domains and argues that common personality traits—that are distributed normally in the general population—can have a profound influence on the processing of facial expressions. It synthesizes data from behavioral and neuroimaging research to illustrate that these personality traits are an important determinant of emotion processing. The article focuses on the processing of facial emotional expressions paying particular attention to the influence of common personality traits in influencing emotion processing.
Body language refers to the nonverbal signals that we use to communicate. According to experts, these nonverbal signals make up a huge part of daily communication. Understanding body language is important, but it is also essential to pay attention to other cues such as context. In many cases, you should look at signals as a group rather than focusing on a single action. Here's what to look for when you're trying to interpret body language.
The Mechanism of Human Facial Expression. Search within full text Export citation; Buy the print book. Contents pp i-viii. Access. PDF; Export citation.
This important volume provides a holistic understanding of the cultural, psychological, neurological and biological elements involved in human facial expressions and of computational models in the analyses of expressions. It includes methodological and technical discussions by leading scholars across the world on the subject. Automated and manual analysis of facial expressions, involving cultural, gender, age and other variables, is a growing and important area of research with important implications for cross-cultural interaction and communication of emotion, including security and clinical studies.
This unique book provides an integrated view of human facial expressions based on contemporary knowledge about the evolution of signaling across the animal kingdom. Spanning fields that range from psychology and neurology to anthropology and linguistics, it reopens and discusses some of the classic questions in the field, including: What do facial expressions express? What are the relations between facial expressions and our motives and emotions? How did our facial expressions evolve? Are there really innate and universal facial expressions?
Advances in Human-Robot Interaction. With the remarkable advancements in the field of robotics, the application of robots is no longer restricted to industrial automation but has been extended to personal home services. Robots are built to interact with humans, since they have not been developed to function as automatic machines, but to coexist as in human society.
The purpose of this paper is to investigate the emotional contagion theory in print ads, and expand the literature of smiling to different type of smiles and gender congruency. Emotional contagion happens when an emotion is transferred from a sender to a receiver by the synchronization of emotions from the emitter. Drawing on emotional contagion theory, the authors expand this concept and propose that smiles in static facial expressions influence product evaluation. They suggest that false smiles do not have the same impact as genuine smiles on product evaluation, and the congruence between the model gender—product in a static ad and the gender of the viewer moderates the effects. In Experiment 1, subjects were randomly assigned to view one of the two ad treatments to guard against systematic error e.
in neurological research to the question of the mechanism of human facial expression. Duchenne has been little cited and little known in this century; his book.
A facial expression is one or more motions or positions of the muscles beneath the skin of the face. According to one set of controversial theories, these movements convey the emotional state of an individual to observers. Facial expressions are a form of nonverbal communication.
Do our background beliefs, desires, and mental images influence our perceptual experience of the emotions of others? In this paper, we will address the possibility of cognitive penetration CP of perceptual experience in the domain of social cognition. In particular, we focus on emotion recognition based on the visual experience of facial expressions. After introducing the current debate on CP, we review examples of perceptual adaptation for facial expressions of emotion.