arXiv cs.AI· Yi Wang, Tianzi Wang, Peter Bell·· 14 小时前AI 评分34
精简工具集与模态整合：高效智能体音视频问答
Efficient Agentic Audio-Visual Question Answering through Toolset Pruning and Modality Consolidation
AI 导读
研究从专用工具 OmniAgent 出发，通过分阶段剪枝工具集并将模态专用工具逐步整合为全模态工具，构建出仅含全模态描述与全模态问答两个工具的紧凑智能体。在三个常用 AVQA 基准上，该紧凑智能体准确率与专用工具智能体相当或更优，同时推理成本降低约 70%。
正文 · 原文
Abstract:Fine-grained audio-visual question answering (AVQA) requires models to resolve complex cross-modal correspondences and temporally grounding from sparsely distributed evidence. Omni-modal agents, which autonomously orchestrate perception tools to actively collect evidence, achieve strong performance, but it remains unclear whether active perception from both audio and visual modalities depends on broad specialized toolsets, especially audio or visual modality-specific tools. Such systems may also introduce long execution trajectories and substantial inference cost. We investigate whether comparable performance can be retained with a compact workflow exposing only essential perceptual capabilities. Starting from the specialized-tool OmniAgent, we conduct staged controlled experiments that prune the toolset and progressively consolidate modality-specific tools into omni-modal tools, motivating a two-tool agent comprising only Omni-modal captioning and Omni-modal QA. Across three widely used AVQA benchmarks, our compact agent achieves comparable or better accuracy than the specialized-tool agent while reducing inference cost by around 70%.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.38960 [cs.AI] |
| (or arXiv:2610.38960v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38960 arXiv-issued DOI via DataCite |
Submission history
From: Yi Wang [view email]
[v1]
Wed, 28 Oct 2026 21:44:12 UTC (1,355 KB)
来源：arXiv cs.AI · arxiv.org