arXiv cs.AI· Junming Lin, Yuxuan Wang, Zhenxin Lei, Yuxin Liu, Ruixun Liu, Yinsong Yan, Ling Wang, Minghao Han, Yunfei Chu, Shun Lei, Xueyao Zhang, Qize Yang, Jin Xu, Yiwu Zhong·· 15 小时前AI 评分34
OmniReasoning：突破音视频联合推理的极限
OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
AI 导读
研究团队提出 OmniReasoning 方案，包含音视频证据缺一不可的 OmniReasoningBench 基准（1,150 道题）、自动构建训练数据的 OmniQA 数据引擎，以及按模态分解线索的 Modality-Factored Self-Distillation（MFSD）学习方法。
正文 · 原文
Authors:Junming Lin, Yuxuan Wang, Zhenxin Lei, Yuxin Liu, Ruixun Liu, Yinsong Yan, Ling Wang, Minghao Han, Yunfei Chu, Shun Lei, Xueyao Zhang, Qize Yang, Jin Xu, Yiwu Zhong
Abstract:Recent advances have enabled unified omni-modal models in understanding audio, vision, and language. However, existing benchmarks, training data, and learning methods largely treat the modalities independently, leaving the capability of audio-visual joint reasoning poorly evaluated and insufficiently elicited. We address this gap with a benchmark, data engine, and learning method. First, we introduce OmniReasoningBench, a benchmark where both audio and visual evidence are indispensable. It comprises 1,150 multiple-choice and open-ended questions across two tasks, reasoning over video and reasoning beyond video. Second, we develop a data engine OmniQA. It automatically constructs evidence-grounded QA pairs that explicitly necessitate audio-visual joint reasoning, together with time-stamped clue chains that guide the annotation of thinking process. Besides our benchmark, this engine produces training data OmniReasoning-SFT-112K and OmniReasoning-RL-19K. Finally, we propose an on-policy self-distillation method Modality-Factored Self-Distillation (MFSD). It evaluates each sampled response under modality-specific clue contexts, disentangling the contributions of individual clues and their cross-modal interactions for token-level credit assignment. With our training data and learning method, our model OmniReasoning-30B-A3B achieves 50.0% on OmniVideoBench and 42.5% on OmniReasoningBench, improving the base model Qwen3-Omni-30B-A3B-Thinking by 12.8 and 9.3 percentage points, respectively. Moreover, it delivers substantial gains on general and long-video benchmarks, including Video-MME-v2. We hope our work offers a solid step for facilitating future research in omni-modal joint reasoning.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.39490 [cs.CV] |
| (or arXiv:2609.39490v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2609.39490 arXiv-issued DOI via DataCite |
Submission history
From: Junming Lin [view email]
[v1]
Wed, 30 Sep 2026 10:51:24 UTC (22,317 KB)
[v2]
Mon, 26 Oct 2026 08:51:09 UTC (13,978 KB)
来源：arXiv cs.AI · arxiv.org