arXiv cs.CL· Shijue Huang, Hangyu Guo, Guanting Dong, Chenxin Li, Junting Lu, Xinyu Geng, Zhaochen Su, Zhenyu Li, Shuang Chen, Hongru Wang, Yi R. Fung·· 12 小时前AI 评分35
面向视觉原生多模态深度搜索智能体的同策略数据进化方法
Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents
AI 导读
研究者提出 On-policy Data Evolution（ODE），通过以图像库引用协议为核心的视觉原生智能体框架，让工具返回的中间图像可被后续工具复用，并基于当前策略的 rollout 逐轮迭代生成训练数据。
正文 · 原文
Authors:Shijue Huang, Hangyu Guo, Guanting Dong, Chenxin Li, Junting Lu, Xinyu Geng, Zhaochen Su, Zhenyu Li, Shuang Chen, Hongru Wang, Yi R. Fung
Abstract:Multimodal deep search requires an agent to solve open-world problems by chaining search, tool use, and visual reasoning over evolving textual and visual context. Two bottlenecks limit current systems. First, existing tool-use harnesses treat images returned by search, browsing, or transformation as transient outputs, so intermediate visual evidence cannot be re-consumed by later tools. Second, training data is usually built by fixed curation recipes that cannot track the target agent's evolving capability. To address these challenges, we first introduce a visual-native agent harness centered on an image bank reference protocol, which registers every tool-returned image as an addressable <image:N> reference and makes intermediate visual evidence reusable by later tools. On top of this harness, On-policy Data Evolution (ODE) runs a closed-loop data generator that refines itself across rounds from rollouts of the policy being trained. This per-round refinement makes each round's data target what the current policy still needs to learn. The same framework supports both diverse supervised fine-tuning data and policy-aware reinforcement learning data curation, covering the full training lifecycle of the target agent. Across 8 multimodal deep search benchmarks, ODE improves the Qwen3-VL-8B agent from 24.9% to 39.0% on average. At 30B, ODE raises the average score from 30.6% to 41.5%. Further analyses validate the effectiveness of image bank reuse, especially on complex tasks requiring iterative visual refinement, while rollout-feedback evolution yields more grounded SFT traces and better policy-matched RL tasks than static synthesis.
| Comments: | NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2605.10832 [cs.CL] |
| (or arXiv:2605.10832v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.10832 arXiv-issued DOI via DataCite |
Submission history
From: Shijue Huang [view email]
[v1]
Mon, 11 May 2026 16:49:36 UTC (1,845 KB)
[v2]
Fri, 5 Jun 2026 07:49:38 UTC (1,845 KB)
[v3]
Wed, 4 Nov 2026 16:30:19 UTC (2,013 KB)
来源：arXiv cs.CL · arxiv.org