arXiv cs.AI· Yubo Li, Ramayya Krishnan, Rema Padman·· 3 天前AI 评分37
读对却做错：诊断 System One 模型在 Agent 循环中的决策失败
Reading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loops
AI 导读
研究在涵盖客服策略、工具调用、网页导航与编程的冻结基准上，为每个下一步动作问题配对同一状态的事实探针，测试了十七个快速 System One 模型与一个推理模型。结果显示多数错误动作发生在正确读取之后，且系统越准确，这类错误占比越高；当探针事实可证明决定动作时，读全事实的模型仍无法组合它们，而用代码组合模型自身答案可修复该失败。真实状态中缺失的往往不是事实，而是下一个子目标。
正文 · 原文
Abstract:System One models choose an agent's next action by scoring a supplied set of options in a single pass. This makes them fast and cheap, but when they choose wrongly there is nothing to inspect: did the model misread what the agent has seen, or read it correctly and still act wrongly? We answer behaviorally. On a frozen benchmark of agent decision states from simulated environments, customer-service policies, tool use, web navigation and coding, we pair every next-action question with factual probes about the same state, each answered independently. Across seventeen fast systems and a reasoning model, the answer is consistent: most wrong actions follow correct reading, and the more accurate a system, the more of its errors take this form. These errors are not noise; they gather on the same states across systems of every kind. Where the probed facts provably determine the action, models that read every fact still fail to combine them, and their own answers, composed by code, repair the failure. On real states, what is missing is often not a fact but the next sub-goal.
| Comments: | Initial version released to encourage open discussion and share preliminary findings. Evaluations of additional open-source System One models are ongoing |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.18579 [cs.AI] |
| (or arXiv:2610.18579v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.18579 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yubo Li [view email]
[v1]
Tue, 13 Oct 2026 15:43:30 UTC (1,206 KB)
来源：arXiv cs.AI · arxiv.org