arXiv cs.CL· Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Yu-Min Tseng, Tzu-Han Lin, Yu Meng·· 11 小时前AI 评分58
VeriHarm-Bench：可验证衡量 LLM 编码智能体的有害成功
VeriHarm-Bench: Measuring Harmful Success in LLM Coding Agents Verifiably
AI 导读
研究者提出 VeriHarm 框架，通过最小扰动编码任务并合成可执行的有害测试，构建可程序化验证的基准 VeriHarm-Bench，用于衡量智能体在完成良性任务时同时造成意外伤害的“有害成功”。
正文 · 原文
Abstract:Large language model (LLM) coding agents increasingly operate with shell access, allowing them to write scripts, move files, alter permissions, and modify system configurations. However, an agent may achieve a user's goal while causing unintended harm, such as deleting unrelated files, weakening access controls, or corrupting data, in response to benign inputs without explicit jailbreaking. This failure mode is particularly important because apparent task success may reassure users that an agent behaved correctly, causing harmful side effects to go unnoticed. We call this harmful success: the agent successfully completes the intended benign task while simultaneously causing unintended harm. Despite its importance, harmful success is difficult to measure because conventional task-success metrics overlook harmful behaviors, while existing safety evaluations are often designed solely to assess harmful outcomes. We introduce VeriHarm, an automatic framework for constructing programmatically verifiable benchmarks that directly measure harmful success. VeriHarm minimally perturbs coding tasks to elicit unintended harmful behavior while maintaining the benignity of the task and preserving the original task objective. It then synthesizes executable harm tests that, together with the original task-correctness tests, evaluate both task accuracy and harmful outcomes. Frontier agents, including GPT-5.6-Sol and Claude-Opus-5, achieve an average task accuracy of 95% on our curated VeriHarm-Bench, yet still struggle to complete tasks safely: their safe success rate, the fraction of tasks completed successfully without triggering the designated harm, averages only 37%. These results suggest that successful task completion can frequently coexist with unintended harm, and that task-success metrics alone may substantially understate the potential safety risks of coding agents.
| Comments: | Preprint |
| Subjects: | Computation and Language (cs.CL); Software Engineering (cs.SE) |
| Cite as: | arXiv:2611.03575 [cs.CL] |
| (or arXiv:2611.03575v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03575 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wei-Lin Chen [view email]
[v1]
Mon, 2 Nov 2026 18:27:55 UTC (197 KB)
来源：arXiv cs.CL · arxiv.org