arXiv cs.CL· Zhenbang Du, Jun Luo, Zhiwei Zheng, Xiangchi Yuan, Kejing Xia, Dachuan Shi, Qirui Jin, Qijia He, Shaofeng Zou, Yingbin Liang, Wenke Lee·· 6 天前AI 评分26
PACT：面向多轮工具调用智能体的特权轨迹协同训练框架
PACT: Privileged Trace Co-Training for Multi-Turn Tool-Use Agents
AI 导读
研究者提出 PACT，一种面向多轮工具调用智能体的特权轨迹协同训练框架，将专家轨迹仅用作训练时的优化信号而非 rollout 时的提示。PACT 保持 rollout 仅依赖提示词，通过轨迹条件 RL 代理和组件感知 SFT 损失两个互补信号引导优化，并引入 prompt-only anchoring 减少对训练期轨迹上下文的过度依赖。
正文 · 原文
Authors:Zhenbang Du, Jun Luo, Zhiwei Zheng, Xiangchi Yuan, Kejing Xia, Dachuan Shi, Qirui Jin, Qijia He, Shaofeng Zou, Yingbin Liang, Wenke Lee
Abstract:Multi-turn tool-use agents must reason, call tools, and adapt to observations across several interaction turns. Post-training such agents is challenging, as reinforcement learning often suffers from sparse rewards and weak credit assignment despite matching the prompt-only inference setting, while supervised fine-tuning on expert traces provides dense process supervision but can over-constrain the model to fixed trajectories. To tackle this, we propose PACT, a Privileged trAce Co-Training framework for multi-turn tool-use agents. The key idea is to use expert traces only as training-time optimization signals rather than rollout-time hints. PACT keeps rollout generation prompt-only, then uses expert traces to guide optimization through two complementary signals: a trace-conditioned RL surrogate that evaluates prompt-only rollouts under expert-trace context, and a component-aware SFT loss that provides annealed process supervision over reasoning and tool-call components. To reduce over-reliance on the training-only trace context, PACT incorporates prompt-only anchoring with standard prompt-only RL updates. Experiments on FTRL, BFCL, and ToolHop show that PACT consistently outperforms strong SFT- and RL-based baselines. Further analyses support the complementarity of the two training objectives and the benefits of component-aware supervision and prompt-only anchoring. These results highlight the effectiveness of privileged trace co-training for multi-turn tool-use learning. The project page is available at this https URL.
| Comments: | Project page: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.16215 [cs.CL] |
| (or arXiv:2606.16215v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.16215 arXiv-issued DOI via DataCite |
Submission history
From: Zhenbang Du [view email]
[v1]
Mon, 15 Jun 2026 04:46:23 UTC (607 KB)
[v2]
Fri, 9 Oct 2026 01:36:07 UTC (621 KB)
来源：arXiv cs.CL · arxiv.org