arXiv cs.AI· Wenshuo Zhang, Yuquan Lin, Yushi Sun, Yishi Lin, Rui Sheng, Peng He, Han Gao, Wang Jikai, Yifeng Rong·· 12 小时前AI 评分38
RITE：评估电商智能体系统对静默工具错误的端到端韧性
RITE: Evaluating End-to-End Resilience to Silent Tool Errors in E-Commerce Agentic Systems
AI 导读
研究者提出 RITE 基准，用于评估工具调用型 LLM 智能体在静默工具错误污染中间状态后能否完成原始查询。该基准基于人工编写的工业工作流和生产日志输入，覆盖 9 个电商工作流、167 个匹配单元，共 501 个静默错误、正确值与显式失败实例。在 10 个近期 LLM 上，静默错误使任务成功率大致减半，且始终比显式失败更难，识别错误、使用可用验证路径与整合证据构成连续瓶颈。
正文 · 原文
Abstract:Tool-using LLM agents increasingly answer e-commerce queries through multi-step tool interactions. The expected output of a tool call can shift between agent development and execution, allowing a plausible but incorrect value to be returned without an explicit failure signal. These silent errors may propagate through subsequent decisions into final answers. To study agents' resilience across this end-to-end process, we formulate Silent-error Resilience Task, an interactive task that evaluates whether they complete the original query after a silent error corrupts an intermediate state. We introduce RITE, a benchmark built from human-authored industrial workflows and production-log-derived inputs. To enable controlled and reproducible evaluation, we use a semantic error taxonomy to inject errors into fixed workflow prefixes, freeze recorded tool behavior, and let agents continue freely. It contains 167 matched units across nine e-commerce workflows, yielding 501 silent-error, correct-value, and explicit-failure instances. Across 10 recent LLMs, silent errors roughly halve task success and are consistently harder than explicit failures. Recognition, use of an available verification path, and integration of the resulting evidence form successive bottlenecks. We also propose a light State Checker as an intervention to test whether resilience can be improved.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.27777 [cs.AI] |
| (or arXiv:2610.27777v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.27777 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenshuo Zhang [view email]
[v1]
Tue, 20 Oct 2026 07:42:11 UTC (1,292 KB)
来源：arXiv cs.AI · arxiv.org