arXiv cs.AI· Parisa Ghanad Torshizi, Stacy Marsella·· 10 小时前AI 评分31
当言语与肢体不一致：LLM 如何实现言语与非言语行为的错位表达
When Words and Bodies Disagree: Realizing Verbal and Nonverbal Mismatch
AI 导读
研究提出强化、缓和与泄露三类言语—非言语错位分类法，并验证 LLM 在给定社交语境时能成功选择目标错位行为（如讽刺）。人类被试实验显示，这些非言语线索显著改变观察者对虚拟智能体态度与情绪效价的感知，且被试能区分不同错位功能，相较匹配基线出现明确的社会意义方向性偏移。
正文 · 原文
Abstract:Traditional nonverbal behavior generation systems for virtual agents typically take utterances as input and generate behaviors that match or illustrate the spoken words. However, human nonverbal behavior is shaped by more than speech content. It is influenced by speaker roles, interpersonal relationships, social context, and cognitive and emotional states of interactants. Therefore, the nonverbal channel carries information or emotions hidden in words. Modeling these complex verbal-nonverbal mismatches is critical for realistic virtual agents, particularly in training applications. We propose a taxonomy of mismatches, strengthening, mitigating, and leaking, and evaluate whether LLMs can select contextually appropriate mismatched behaviors. Through human-subject studies, we assess whether these context-driven behaviors, realized in a virtual human framework, produce the intended effects. Our results demonstrate that LLMs, when prompted with social context, successfully select targeted mismatched behaviors (e.g., sarcasm). Furthermore, the human-subject evaluation confirms that these nonverbal cues significantly shift observers' perceptions of the agent's attitude and emotional valence. Participants successfully distinguished between distinct mismatch functions, such as strengthening, mitigating, and leaking, perceiving clear directional shifts in social meaning compared to matching baselines.
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Robotics (cs.RO) |
| Cite as: | arXiv:2608.22731 [cs.AI] |
| (or arXiv:2608.22731v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.22731 arXiv-issued DOI via DataCite |
Submission history
From: Parisa Ghanad Torshizi [view email]
[v1]
Mon, 24 Aug 2026 02:38:55 UTC (1,994 KB)
[v2]
Tue, 10 Nov 2026 17:26:07 UTC (4,193 KB)
来源：arXiv cs.AI · arxiv.org