AgentEval：面向有状态智能体的证据受限评估系统

Authors:Ruiqi Gao (Alibaba Cloud, Alibaba Group)

Abstract:Task completion alone does not determine whether a stateful agent satisfied its obligations, and a recorded judgment does not guarantee that its supporting evidence remains valid. We present AgentEval, a system that carries a contract-bound assessment record through tool execution, state and event capture, report qualification, and case maintenance. It separates what the agent did, what the evidence establishes, and whether the conclusion is supported for present use. On 32 tool-world administrations, independent checks agree on all 30 complete episodes, distinguishing satisfied, violated, and unresolved obligations; two interrupted administrations are retained. Controlled interventions on 48 evidence views expose 36 binding faults missed by hash-only checks without changing agent behavior. Scenario Studio connects reviewed cases to 12 two-turn reports with 24 independently checked answers. A separate public-benchmark study on 192 held-out executions reveals ten completed return subgoals within broader task failures and isolates the effect of missing evidence. These studies show how an executable assessment record makes scope, evidential support, and report reuse inspectable across an agent evaluation workflow.

Comments:	23 pages, 6 figures
Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.21177 [cs.AI]
	(or arXiv:2610.21177v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.21177 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ruiqi Gao [view email]
[v1] Thu, 15 Oct 2026 03:32:39 UTC (3,570 KB)