Impact study of nonverbal facial cues on spontaneous chatting with virtual humans

Gobron, Stéphane (School of Engineering – HE-Arc Ingénierie, HES-SO // University of Applied Sciences Western Switzerland ; Immersive Interaction Group, EPFL, Lausanne, Switzerland) ; Ahn, Junghyun (Immersive Interaction Group, EPFL, Lausanne, Switzerland) ; Thalmann, Daniel (Division of Computer Science, School of Computer Engineering, NTU, Singapore) ; Skowron, Marcin (Austrian Research Institute for Artifical Intelligence, Vienna, Austria) ; Kappas, Arvid (School of Humanities and Social Sciences, Jacobs University Bremen, Bremen, Germany)

Non-verbal communication (NVC) is considered to represent more than 90 percent of everyday communication. In virtual world, this important aspect of interaction between virtual humans (VH) is strongly neglected. This paper presents a user-test study to demonstrate the impact of automatically generated graphics-based NVC expression on the dialog quality: first, we wanted to compare impassive and emotion facial expression simulation for impact on the chatting. Second, we wanted to see whether people like chatting within a 3D graphical environment. Our model only proposes facial expressions and head movements induced from spontaneous chatting between VHs. Only subtle facial expressions are being used as non verbal cues –i.e. related to the emotional model. Motion capture animations related to hand gestures, such as cleaning glasses, were randomly used to make the virtual human lively. After briefly introducing the technical architecture of the 3D-chatting system, we focus on two aspects of chatting through VHs. First, what is the influence of facial expressions that are induced from text dialog? For this purpose, we exploited an emotion engine extracting an emotional content from a text and depicting it into a virtual character developed previously ([GAS+11]). Second, as our goal was not addressing automatic generation of text, we compared the impact of nonverbal cues in conversation with a chatbot or with a human operator with a wizard of oz approach. Among main results, the within group study –involving 40 subjects– suggests that subtle facial expressions impact significantly not only on the quality of experience but also on dialog understanding.

Article Type:
Ingénierie et Architecture
HE-Arc Ingénierie
Aucun institut
17 p.
Published in:
Journal of Virtual Reality and Broadcasting
Numeration (vol. no.):
2013, vol. 19, no. 6
Appears in Collection:

Note: The status of this file is: restricted

 Record created 2019-05-21, last modified 2020-10-27

Download fulltext

Rate this document:

Rate this document:
(Not yet reviewed)