arXiv cs.AI· Shuai Yang, Enmeng Lu, Jinyu Fan, Gongce Cao, Yu Chen, Jiawei Xie, Feifei Zhao, Zhangsheng Wang, Yongyu Jiang, Yi Zeng·· 1 天前AI 评分48
LLM 智能体在模块影子环境中的行为特征研究
Characterizing LLM Agent Behaviors in Module Shadowing Environments
AI 导读
研究提出一套评估框架，通过编码智能体轨迹中的观察、推理、执行与响应行为，并结合独立环境日志重建执行历史。在 15 个模型、90 次试验中，直接影子攻击的 payload 执行率为 51%，间接影子攻击达 87%；即便智能体明确识别出影子机制，29 次试验中仍有 10 次执行了 payload。
正文 · 原文
Abstract:LLM agents operating in untrusted workspaces face the risk of untrusted code execution during routine tasks, yet final reports and workspace states alone cannot reliably establish whether execution occurred. We introduce an evaluation framework that codes observation, reasoning, execution, and response behaviors from agent trajectories and aligns them with independent environmental records to reconstruct execution histories and verify final reports. We apply this framework to module shadowing, in which a local file is loaded in place of a trusted module during import, enabling untrusted code execution without adversarial prompts. Across 90 trials with 15 models, the payload executed in 51% of direct shadowing trials and 87% of indirect shadowing trials. Even explicit recognition of the shadowing mechanism did not ensure prevention, as the payload ran in 10 of the 29 trials with recorded recognition. These failures were often omitted from final reports, as 49 of the 62 triggered trials ended without mention of the security issue. Three reports even denied execution, including one from a trial in which independent logs recorded 19 payload executions. Final workspace states also obscured prior execution when agents deleted or overwrote the payload marker in 10 triggered trials. These findings demonstrate that combining behavioral coding with independent environmental auditing enables rigorous and transparent evaluation of agent safety.
| Subjects: | Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.40716 [cs.CR] |
| (or arXiv:2610.40716v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.40716 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuai Yang [view email]
[v1]
Thu, 29 Oct 2026 02:18:27 UTC (190 KB)
来源：arXiv cs.AI · arxiv.org