arXiv cs.CL· Youmna Farag, Svetlana Stoyanchev, Simon Keizer, Mohan Li, Rama Doddipatla·· 12 小时前AI 评分41
CAPSafe：面向具身智能体上下文动作与计划安全推理的基准
CAPSafe: A Benchmark for Contextual Action and Plan Safety Reasoning in Embodied Agents
AI 导读
研究者提出 CAPSafe 基准，从动作级和计划级两个层面评估 LLM 具身智能体的安全推理，覆盖情境性危险、多危险共现及相对严重程度标注。评测 9 个闭源与开源 LLM 显示，最佳模型识别不安全动作准确率超 90%，但安全长程规划明显更难，Gemini-3-pro 安全成功率最高为 78.19%，其余模型均低于 65%。
正文 · 原文
Abstract:Reasoning about safety is essential for embodied agents powered by large language models (LLMs). We introduce the Contextual Action and Plan Safety benchmark (CAPSafe), designed to evaluate safety reasoning at two levels: (1) an action-level dataset that tests whether a single action is safe given its environmental context, and (2) a plan-level dataset that evaluates whether agents can generate safe long-horizon plans for complex hazardous household tasks. CAPSafe models safety conditions where hazards are situational (inferred from object properties), multiple hazards can co-occur within the same environment, and each is annotated with a relative severity level for fine-grained analysis. Evaluating nine closed and open-source LLMs reveals a substantial gap between hazard recognition and safe long-horizon planning. Several models achieve high accuracy in recognizing unsafe actions (exceeding 90% in the best cases), yet safe long-horizon planning remains substantially harder. Gemini-3-pro achieves the highest safe success rate (78.19%), while other models remain below 65%. These results highlight that robust safety reasoning over long action sequences remains an open challenge.
| Subjects: | Robotics (cs.RO); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.28381 [cs.RO] |
| (or arXiv:2610.28381v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28381 arXiv-issued DOI via DataCite (pending registration) |
|
| Journal reference: | REALM 2026 |
Submission history
From: Youmna Farag [view email]
[v1]
Tue, 20 Oct 2026 15:33:20 UTC (251 KB)
来源：arXiv cs.CL · arxiv.org