Humans perceive facial expressions as conveying meaning, but where do they come from, and what exactly do they mean? Based on observations of facial expressions typically associated with emotions, some experts hypothesized that facial expressions are reliable indicators of emotion.
But, a new study contradicts the hypothesis, suggesting that facial expressions might not be reliable indicators of emotion. It might be more accurate to say we should never trust a person’s face.
Aleix Martinez, a professor of electrical and computer engineering at The Ohio State University, said, “The question we asked is: ‘Can we truly detect emotion from facial articulations? And the basic conclusion is, no, you can’t.”
For the study, scientists analyzed the kinetics of muscle movement in the human face and compared those muscle movements with an individual’s feelings. They found that endeavors to detect or define emotions based on a person’s facial expressions were quite often wrong.
Martinez said, “Everyone makes different facial expressions based on context and cultural background. And it’s important to realize that not everyone who smiles is happy. Not everyone who is happy smiles. I would even go to the extreme of saying most people who do not smile are not necessarily unhappy. And if you are happy for a whole day, you don’t go walking down the street with a smile on your face. You’re just happy.”
“It is also true that sometimes, people smile out of an obligation to the social norms. This would not inherently be a problem — people are certainly entitled to put on a smile for the rest of the world — but some companies have begun developing technology to recognize facial muscle movements and assign emotion or intent to those movements.”
“We also analyzed some of those technologies and largely found them lacking. Some claim they can detect whether someone is guilty of a crime or not, or whether a student is paying attention in class, or whether a customer is satisfied after purchase. What our research showed is that those claims are complete baloney. There’s no way you can determine those things. And worse, it can be dangerous.”
“The danger lies in the possibility of missing the real emotion or intent in another person, and then making decisions about that person’s future or abilities.”
“For example, consider a classroom environment, and a teacher who assumes that a student is not paying attention because of the expression on the student’s face. The teacher might expect the student to smile and nod along if the student is paying attention. But maybe that student, for reasons the teacher doesn’t understand — cultural reasons, perhaps, or contextual ones — is listening intently, but not smiling at all. It would be wrong for the teacher to dismiss that student because of the student’s facial expressions.”
Based on the data about facial expressions and emotion, scientists concluded that it takes more than expressions to detect emotion correctly.
Martinez said, “What we showed is that when you experience emotion, your brain releases peptides — mostly hormones — that change the blood flow and blood composition, and because the face is inundated with these peptides, it changes color.”
In one experiment, Martinez showed study participants a picture cropped to display just a man’s face. The man’s mouth is open in an apparent scream; his face is bright red.
Martinez said, “When people looked at it, they would think, wow, this guy is super annoyed, or mad at something, that he’s angry and shouting. But when participants saw the whole image, they saw that it was a soccer player who was celebrating a goal.”
In context, it’s clear the man is delighted. But isolate his face and he appears almost dangerous.
It means Cultural biases also play a vital role.
Martinez said, “the research group’s findings could indicate that people — from hiring managers to professors to criminal justice experts — should consider more than just a facial expression when they evaluate another person.”
And while Martinez said he is “a big believer” in developing computer algorithms that try to understand social cues and the intent of a person, he added that two things are essential to know about that technology.
“One is you are never going to get 100 percent accuracy,” he said. “And the second is that deciphering a person’s intent goes beyond their facial expression, and it’s important that people — and the computer algorithms they create — understand that.”
Scientists presented their findings at the annual meeting of the American Association for the Advancement of Science in Seattle.