arXiv cs.CL· Wei Guo, Yuanjian Xu, Jianing Hao, Yongyao Duan, Ran Chen, Zhong Li, Guang Zhang·· 13 小时前AI 评分38
如何弥合 LLM 基础能力与智能体能力之间的鸿沟
Bridging Foundational and Agentic Capabilities of LLMs
AI 导读
研究者提出一个操作级测量框架，用能力基准构建基础画像、用交互轨迹构建操作画像，并将"能力实现差距"定义为实际操作表现相对基础能力预测的偏离。在 τ²-Bench 上对 15 个模型服务的实验显示，基础画像只能部分预测智能体操作，画像相近的模型在执行上可能差异显著，且差距在不同任务子集上保持一致；操作画像能补充信息并显著提升任务得分预测。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly deployed as agents, yet it remains unclear how their foundational capabilities translate into agentic capabilities. End-to-end scores leave unresolved whether low benchmark capability or weak deployment drives a failure, limiting actionable diagnosis. We bridge foundational and agentic capabilities through an operation-level measurement framework. We represent each model with a foundational profile derived from capability benchmarks and an operation profile derived from interaction trajectories, and define capability realization gaps as deviations of observed operation performance from predictions based on foundational capabilities. Experiments on fifteen model services in $\tau^2$-Bench show that foundational profiles only partially predict agentic operations. Models with comparable profiles can differ substantially in execution, and these gaps remain consistent across task subsets. Operation profiles add information and substantially improve task-score prediction. The framework locates execution departures at the operation level to inform model development and agent evaluation. We will release our code and analysis artifacts.
| Subjects: | Computation and Language (cs.CL); Software Engineering (cs.SE) |
| Cite as: | arXiv:2610.29751 [cs.CL] |
| (or arXiv:2610.29751v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29751 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei Guo [view email]
[v1]
Wed, 21 Oct 2026 13:50:43 UTC (783 KB)
来源：arXiv cs.CL · arxiv.org