arXiv cs.CL· Shengze Huang, Zihao Tang, Jinluan Yang, Xin Yu, Zhenghao Lin, Xiao Liu, Zengxuan Wen, Yeyun Gong, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Fei Wu, Kun Kuang·· 6 天前AI 评分31
SOAR：面向搜索智能体的来源感知强化学习与精确信用分配
SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents
AI 导读
针对搜索智能体绕过外部信息、编造错误答案的"证据捏造"问题，研究者提出 SOAR（来源感知强化学习），通过 Source-Aware Workflow 将外部信息与内部知识拆分为可观测的并行流，并用 Source-Consistent Reward 引导策略演化。在 3B 和 7B 骨干模型上，SOAR 分别比基线提升 7.3% 和 5.4%，同时将证据捏造减少 74.8%。
正文 · 原文
Authors:Shengze Huang, Zihao Tang, Jinluan Yang, Xin Yu, Zhenghao Lin, Xiao Liu, Zengxuan Wen, Yeyun Gong, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Fei Wu, Kun Kuang
Abstract:Reinforcement learning (RL) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information ($E_{ext}$) and fabricate an incorrect answer. We argue that this fabrication arises from an over-reliance on unreliable internal knowledge ($E_{int}$) as shortcuts due to the inherent opacity of reasoning trajectories. To address this, we propose SOAR (SOurce-Aware Reinforcement learning), a paradigm that transforms monolithic reasoning into source-explicit trajectories. SOAR (i) disentangles $E_{ext}$ and $E_{int}$ into parallel, observable streams via a Source-Aware Workflow, and (ii) incorporates Source-Consistent Reward to guide the policy evolution. By making information-source attribution explicit, SOAR enables precise credit assignment between different knowledge streams. Evaluations show that SOAR outperforms baselines on 3B and 7B backbones by 7.3% and 5.4%, while reducing evidence fabrication by 74.8%. These confirm that SOAR significantly improves performance on knowledge-intensive benchmarks while effectively mitigating evidence fabrication, fostering more faithful and grounded reasoning with enhanced robustness.
| Comments: | 17 pages, 3 figures, 16 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13157 [cs.CL] |
| (or arXiv:2610.13157v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13157 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shengze Huang [view email]
[v1]
Fri, 9 Oct 2026 07:45:11 UTC (376 KB)
来源：arXiv cs.CL · arxiv.org