arXiv cs.CL· Jiapeng Shi, Qiuxia Hou, Jiaqi Leng, Junke Wang, Yanhao Zhang, Haonan Lu, Ziyi Ye, Zuxuan Wu·· 2 天前AI 评分23
EVA-Cap：通过事件中心对齐优化音视频字幕生成
EVA-Cap: Optimizing Audiovisual Video Captioning via Event-Centric Alignment
AI 导读
EVA-Cap 框架将音视频字幕分解为原子级视听事件，把非结构化文本监督转化为事件中心对齐，通过 SFT 和 GRPO 在自建数据集 EVA-Data 上训练。GRPO 阶段围绕全局事件完整性、事件内归因和事件间同步三个目标优化。在 ChronusAV 上平均提升 9.0%，WorldSense 上提升 3.3%，论文已被 NeurIPS 2026 接收。
正文 · 原文
Abstract:Recent advances in Omni Language Models (OLMs) provide a unified solution for audiovisual video captioning. Nevertheless, existing approaches predominantly treat captioning as an unstructured text generation process, neglecting the inherent semantic structure of videos and leading to suboptimal fine-grained audiovisual alignment. To bridge this gap, we propose EVA-Cap, a novel framework that decomposes captions into atomic audiovisual events and transforms unstructured text supervision into event-centric alignment. We train EVA-Cap via Supervised Fine-Tuning (SFT) and Group Relative Policy Optimization (GRPO) on a self-curated dataset, EVA-Data, which is built with event-centric quality control. During the GRPO phase, the model is explicitly optimized against three event-centric alignment targets: global event completeness, intra-event attribution, and inter-event synchronization. Extensive experiments demonstrate that EVA-Cap significantly outperforms existing methods in event-centric evaluation (e.g., +9.0% average gain on ChronusAV), while achieving competitive performance on general audiovisual captioning benchmarks (e.g., +3.3% on WorldSense).
| Comments: | Accepted at NeurIPS 2026. Code will be released at this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22186 [cs.CV] |
| (or arXiv:2610.22186v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22186 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiapeng Shi [view email]
[v1]
Thu, 15 Oct 2026 16:42:11 UTC (3,085 KB)
来源：arXiv cs.CL · arxiv.org