Turn-taking dynamics of social interactions are important for speech and gesture synchronization, enabling conversations to proceed efficiently, according to a study published September 25, 2024, in the open-access journal PLOS ONE by Tifenn Fauviaux from the University of Montpellier, France, and colleagues.
Conversations encompass continuous exchanges of verbal and nonverbal information. Previous research has demonstrated that gestures and speech synchronize at the individual level. But few studies have investigated how this phenomenon may unfold between individuals.
To fill this knowledge gap, Fauviaux and colleagues used an online dataset consisting of 14 sessions of two people engaged in unstructured face-to-face conversations during which they were free to talk about specific topics. Each of these sessions contained between one and four discussions, and the conversations lasted from 7 to 15 minutes.
The researchers analyzed both audio and motion data, and measured speech and gesture synchronization at different timescales. Specifically, they focused on vocal properties through the speech amplitude envelope and movement properties through head and wrist gestures.
The results supported previous research on speech and gesture coordination at the individual level, revealing synchronization at all timescales of the conversation. That is, there was higher-than-chance synchronization between a given participant’s wrist and head movements, and similar synchronization between these movements and vocal properties.
Extending the literature, the researchers also found that gestures and speech synchronize between individuals. In other words, there was coordination between the voices and the bodies of the two speakers. Taken together, the findings suggest that this type of synchronization of verbal and nonverbal information likely depends on the turn-taking dynamics of conversations.
According to the authors, the study enriches our understanding of behavioral dynamics during social interactions at both the intrapersonal and interpersonal levels, and strengthens knowledge regarding the importance of synchrony between speech and gestures. Future research building on this study could shed light on prosocial behaviors and psychiatric conditions characterized by social deficits.
The authors add: “How do my speech and behaviors influence, or respond to, the speech and behaviors of the person I’m conversing with? This study answers this question by investigating the multimodal dynamic between speech and movements, both at the individual’s level and the dyadic level. Our findings confirm intrapersonal coordination between speech and gestures across all temporal scales. It also suggests that multimodal and interpersonal synchronization may be influenced by the speech channel, particularly the dynamics of turn-taking.”
Journal Reference:
- Fauviaux T, Marin L, Parisi M, Schmidt R, Mostafaoui G (2024) From unimodal to multimodal dynamics of verbal and nonverbal cues during unstructured conversation. PLoS ONE 19(9): e0309831. DOI: 10.1371/journal.pone.0309831
