New infrared-based system that reads body language

Infrared sensors and a marshmallow offer researchers a new way to monitor and assess social interaction.

Credit: Photo by Nik MacMillan on Unsplash
Credit: Photo by Nik MacMillan on Unsplash

Scientists at the University of Cambridge in collaboration with Dartmouth College, have developed a system called Protractor that uses infrared light tags to monitor face-to-face interactions. Scientists expect that the device could lead to a more exact understanding of how people interface in social settings and can build the adequacy of correspondences training.

The Protractor uses invisible light to record how people employ body language by measuring body angles and distances between individuals. Our body language influence many aspects of everyday life, Based on that the system includes a specific set of interaction details such as eye contact and hand gestures for which an accurate monitoring of distance and relative orientation is crucial.

Professor Cecilia Mascolo from Cambridge’s Department of Computer Science and Technology said, “The ability to use invisible light to determine someone’s role and attitude in social settings has powerful implications for individuals and organizations that are concerned about how they communicate.”

The protractor is a lightweight, wearable tag resembling an access badge worn with a lanyard or clip. It measures non-verbal conduct with fine granularity by utilizing close infrared light from photodiodes. The light innovation works at a wavelength generally utilized as a part of TV remote controls.

Before settling on infrared light for the unit, the exploration group likewise considered ultrasound and radio recurrence. Notwithstanding the general precision, infrared was ideal since light can’t enter human bodies, guaranteeing the exact detecting of eye to eye connection. Close infrared light is additionally subtle to human eyes and keeps the detecting unpretentious.

Although well-suited for measuring body language, the research team needed to correct for when a user’s hand or clothing could temporarily block the light channel. They did so by designing algorithms that exploit inertial sensors to work around the absence of light tracking results.

Co-author Xia Zhou from Dartmouth said, “Our system is a key departure from existing approaches. The ability to sense both body distance and relative angle with fine accuracy using only infrared light offers huge advantages and can deepen the understanding of how body language plays a role in social interactions.”

While demonstration, scientists had to devise a way for the sensors to accurately identify participants and to limit power consumption. To study its effectiveness,  used the Protractor tags to track non-verbal behaviors during a problem-solving group task known as “The Marshmallow Challenge.” In this task, teams of four members were given 18 minutes to build a structure that could support a marshmallow using tape, string and a handful of spaghetti.

Co-author Zhao Tian, a Ph.D. candidate at Dartmouth said, “By modulating the light from each Protractor tag to encode the tag ID, each tag can then figure out which individuals are participating. To increase energy efficiency, we also adapt the frequency of emitting light signals based on the specific context.”

Alessandro Montanari, a researcher at the University of Cambridge said, “Beyond simply observing body language with the tags, we identified the task role each group member was performing and delineated each stage in the building process through the recorded body angle and distance measurements.”

According to the research team, the system will not only support social research, but it can also potentially provide real-time feedback during interviews and other interactions. Trainers, supervisors and team facilitators can use these findings to better understand team dynamics and intervene during intense problem-focused discussions to achieve higher creativity.

A protractor can also help study the impact of culture on body language in light of research that shows that cultural backgrounds can impact the way people think, feel, and act while working with others – an important feature in today’s highly-internationalized workplaces.

Researchers at Maastricht University and the University of Nottingham also contributed to this study. The research was published in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.