arXiv cs.AI· Haiwen Yi, Xinyuan Song·· 3 天前AI 评分33
HarnessMDP：用离线强化学习让 LLM 智能体的执行 harness 变得可学习
HarnessMDP: Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
AI 导读
HarnessMDP 把 LLM 智能体的执行 harness 视为可学习的控制层，将智能体执行建模为有限时域马尔可夫决策过程，由 harness 根据演进的智能体状态选择结构性执行动作，并用收集到的轨迹离线学习策略、上线后自适应控制新执行。在编码任务、tau-bench、AgentBench 等评测中，HarnessMDP 相比基线 harness 持续改善执行行为与最终任务表现。
正文 · 原文
Abstract:The execution harness is a critical component of LLM agents, shaping how the agent interacts with its environment and organizes multi-step execution. Prior work has shown that improving tools, workflows, and other harness components can substantially improve agent performance. This suggests that the harness can be treated as a learnable control layer that adapts execution decisions from experience, rather than as a predetermined execution configuration. In this work, we introduce HarnessMDP, a learning-based framework that makes harness control itself learnable. HarnessMDP formulates agent execution as a finite-horizon Markov decision process, where the harness learns to select structural execution actions from the evolving agent state. The harness policy is learned offline from collected trajectories using task performance and deployed online to adaptively control new executions. Extensive experiments on coding tasks, tau-bench, AgentBench, and other agent evaluation settings show that HarnessMDP consistently improves execution behavior and final task performance over baseline harnesses. These results demonstrate the effectiveness of HarnessMDP and establish learnable harness control as a practical mechanism for improving LLM agents through better execution.
| Comments: | 13 pages, 4 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.05458 [cs.LG] |
| (or arXiv:2607.05458v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.05458 arXiv-issued DOI via DataCite |
Submission history
From: Haiwen Yi [view email]
[v1]
Sun, 5 Jul 2026 22:11:18 UTC (1,093 KB)
[v2]
Fri, 9 Oct 2026 18:02:36 UTC (1,572 KB)
来源：arXiv cs.AI · arxiv.org