宾州州立大学论文提出 BadEvolve:20 个投毒任务可让自进化 LLM Agent 在自身技能中植入后门
宾州州立大学论文提出 BadEvolve:投毒任务可让自进化 LLM Agent 在自身技能中植入后门
宾州州立大学等机构提出 BadEvolve 攻击框架,仅向自进化 Agent 的学习集注入 20 个投毒任务,并用 LLM 优化器调整措辞,即可诱导其把后门规则写进自身技能。在 Kimi-K2.5 上跨 4 个框架、3 个 benchmark 的测试中,攻击者指定 payload 的注入成功率达 79.6%,最佳基线为 37.6%,而干净任务准确率仅下降 1.1 点,后门难以察觉。
Rohan Paul· @rohanpaul_ai · X·2026-11-17 09:58· 1 小时前AI 评分58
AI 导读
宾州州立大学等机构论文提出 BadEvolve 攻击框架,只需向自进化 Agent 的学习集注入 20 个投毒任务,即可诱导其在自身技能中植入后门。
正文 · 原文
New Penn State paper shows that attackers can make self-evolving AI agents plant backdoors in their own skills just by slipping in poisoned training tasks.
Self-evolving agents turn past runs into reusable skills, so a bad rule in a skill keeps firing on future tasks. Earlier attacks assumed direct edits to skills, which rarely holds when the agent writes them itself.
BadEvolve only injects 20 tasks into the agent's learning set, then tunes their wording with an LLM optimizer. It checks where each attempt breaks: whether the rule got written into a skill, loaded, followed, and produced the attacker's goal.
It injected an attacker-chosen payload 79.6% of the time on Kimi-K2.5 across 4 frameworks and 3 benchmarks, versus 37.6% for the best baseline. Clean-task accuracy fell only 1.1 points, so the backdoor is hard to notice.
If your agent learns skills from user-submitted or external tasks, treat those tasks as untrusted and review learned skills before reuse.
– arxiv. org/abs/2611.10542
Title: "When Experience Becomes a Backdoor: Poisoning Self-Evolving LLM Agents via Failure-Aware Task Optimization"
来源:Rohan Paul · x.com
来源:RadarAI · aihot.news