arXiv cs.CL· Haotian Qi, Gabriel Skantze·· 10 小时前AI 评分29
AVCC：用于轮次转换预测的音视频对话语料库与人类基准
AVCC: Audio-Visual Conversation Corpus for Turn-Taking Prediction with a Human Benchmark
AI 导读
AVCC 被构建为公开的多方下一说话人预测（NSP）基准，语料为 31 小时双人及三人线上对话，仅用单个固定摄像头和麦克风录制。新增说话人不重叠的 7 段视频测试集，在互静音间隙包含 865 个 hold 与 865 个 shift 事件，并提供全语料词级强制对齐转写及同事件上采集的人类参考基准。
正文 · 原文
Abstract:} Predicting who will speak next, and when, is a prerequisite for conversational agents and social robots that take turns without waiting for silence. Multiparty next-speaker prediction (NSP) has been studied on corpora with per-speaker sensors, each with its own event definitions. No public benchmark covers the single camera and microphone a robot has, and no model has been compared against how well people perform the same prediction. We turn AVCC, a 31-hour audio-visual corpus of two- and three-speaker online conversation recorded with a single static camera and microphone, into a public NSP benchmark. We add a speaker-disjoint seven-video test partition with 865 hold and 865 shift events at mutual-silence gaps, word-level forced-aligned transcripts for the whole corpus, and a human reference benchmark collected at the same events. We score an audio-only Role-Relative VAP model (RoleVAP) and the multimodal MuVAP model. Labels, transcripts, per-response human data, and the scripts that reproduce every number are released.
| Subjects: | Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Sound (cs.SD) |
| Cite as: | arXiv:2611.13957 [cs.CL] |
| (or arXiv:2611.13957v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13957 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Haotian Qi [view email]
[v1]
Tue, 10 Nov 2026 11:43:42 UTC (1,050 KB)
来源：arXiv cs.CL · arxiv.org