arXiv cs.AI· Yue Zhao, Zixiang Xu, Angela Huyue Zhang, S. Alex Yang, Xiyang Hu·· 3 天前AI 评分27
agent-risk：面向 AI 智能体变更的风险评估方法
agent-risk: Risk Assessment for Changes to AI Agents
AI 导读
研究者提出 agent-risk，用于在修改已部署 AI 智能体前基于现有日志评估变更风险，可重放日志片段并假设其他条件不变、智能体不预知变更。在 bash 沙箱中，仅凭 7,200 条 episode 日志无法判定两个重试门控，而从保存状态仅重跑被阻断 episode 即达到与实时测试相同的决策，智能体调用量减半；缺少逐步危害记录时，停止门控的认证危害上限最高上升 2.5 倍。
正文 · 原文
Abstract:Before changing a deployed AI agent (a stricter safety gate, a new model), organizations must approve the change, starting from existing logs. Extending classical risk analysis to partial records and partial changes to running systems, agent-risk replays logged episodes, assuming nothing else changes and the agent does not anticipate the change. The log settles episodes acting as before, a declared stop value settles stopped ones, and the rest need logged support, a saved-state rerun, or a bound. We prove what logs must record for exact replay and which saved states serve a gate family. For independent episodes, we bound the reruns a decision needs and tolerable drift from live behavior. In a bash sandbox, a 7,200-episode log alone decided neither of two retry gates, which let the agent act after refusals the log never shows. Rerunning only each gate's blocked episodes from saved states reached both live decisions at half the agent calls of a live test sized post hoc for them, given the log. Without per-step harm, a stopping gate's certified harm ceiling rose up to 2.5-fold, since the log hides whether harm preceded the stop. Approval thus needs evidence designed in advance: log fields, saved states, targeted reruns.
| Comments: | 43 pages, 5 figures, 7 tables. Code, registrations, and logs: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.16280 [cs.AI] |
| (or arXiv:2610.16280v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16280 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yue Zhao [view email]
[v1]
Mon, 12 Oct 2026 07:05:34 UTC (347 KB)
来源：arXiv cs.AI · arxiv.org