arXiv cs.CL· Xingzhi Qian, Sungho Park, Rongyuan Tan, Jianbo Yu, Jue Zhang, Lorenzo Cavallaro, Menglin Xia, Yanzhi Chen, Xinnuo Xu, Yu Kang, Xinran Zheng, Gagan Somashekar, Xuchao Zhang, Chetan Bansal, Qingwei Lin, Victor R\"uhle·· 12 小时前AI 评分38
ParetoSaddler：面向质量与成本的智能体 Harness 帕累托引导优化
ParetoSaddler: Pareto-Guided Optimization of Agent Harnesses for Quality and Cost
AI 导读
ParetoSaddler 将中间候选晋升与最终部署选择分离，用 Bayesian bootstrap 帕累托支持筛选在质量或成本上有可信进展的候选。在 ClawMark 上质量从 70.86% 提升至 81.35%，单任务成本从 $1.17 降至 $1.01；在 GAIA2 上质量从 56.22% 提升至 62.89%，成本从 $0.78 降至 $0.45。
正文 · 原文
Authors:Xingzhi Qian, Sungho Park, Rongyuan Tan, Jianbo Yu, Jue Zhang, Lorenzo Cavallaro, Menglin Xia, Yanzhi Chen, Xinnuo Xu, Yu Kang, Xinran Zheng, Gagan Somashekar, Xuchao Zhang, Chetan Bansal, Qingwei Lin, Victor Rühle
Abstract:Harness optimization can substantially improve LLM agents without modifying model weights, but existing methods primarily optimize response quality. Extending them to joint quality--cost optimization introduces a challenge: applying the deployment preference during intermediate updates can prematurely reject candidates that are valuable for later improvements. We introduce ParetoSaddler, which addresses this mismatch by separating intermediate candidate promotion from final deployment selection. At each iteration, an optimizer updates a parent harness to produce a candidate, and then compares the two over repeated executions and uses Bayesian bootstrap Pareto support to promote candidates showing credible progress in either quality or cost. Promoted candidates receive broader development evaluation and remain available as parents for later optimization steps. We evaluate ParetoSaddler on ClawMark and GAIA2 using domain-disjoint splits. On ClawMark, ParetoSaddler improves quality from 70.86\% to 81.35\% while reducing cost from \$1.17 to \$1.01 per task; on GAIA2, it improves quality from 56.22\% to 62.89\% while reducing cost from \$0.78 to \$0.45 per task. The Pareto-selected harnesses also achieve higher quality at lower cost than quality-only harness optimization. Analysis shows that Pareto-aware optimization retains intermediate candidates that would be discarded by deployment preference but can support later improvements, while Bayesian bootstrap screening filters out candidates whose gains are weakly supported across runs.
| Comments: | Project website and code will be available at this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2611.02079 [cs.AI] |
| (or arXiv:2611.02079v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02079 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xingzhi Qian [view email]
[v1]
Sun, 1 Nov 2026 17:36:05 UTC (589 KB)
来源：arXiv cs.CL · arxiv.org