arXiv cs.CL· Ehsan Saleh, Roozbeh Nahavandi, Sama Salarian, Raef Bassily, Andrew Perrault·· 13 小时前AI 评分47
AfterDeleteBench：审计 LLM 智能体删除后信息残留
AfterDeleteBench: Auditing Post-Deletion Information Persistence in LLM Agents
AI 导读
研究者提出 AfterDeleteBench，用于行为级审计固定权重 LLM 智能体在记忆删除后的信息残留：Qwen2.5-7B-Instruct 在单次查询下约三分之二的案例可恢复已删除秘密，而从未接触该秘密的对照组恢复率为零。
正文 · 原文
Abstract:Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not remove information carried by its descendants. We introduce AfterDeleteBench, a controlled behavioral audit for this failure mode in fixed-weight LLM agents. For each synthetic secret, we compare an exposed-then-deleted agent with an otherwise matched agent that was never exposed to the secret, allowing post-deletion recovery to be attributed to the original exposure. In our primary setting, a single query recovers the deleted secret from Qwen2.5-7B-Instruct in roughly two thirds of cases, while matched never-exposed controls have zero recovery. Crucially, recovery does not require a surviving verbatim copy: when the complete secret string is absent and only separately stored components remain under a public composition rule, exact reconstruction succeeds in 62-99% of cases across Qwen, Mistral, and Phi-3. We also show that the response interface is part of the measurement system: target-bearing replies can be scored as failures when the requested format and parser are misaligned. Finally, we separate reconstruction from write-back and fresh-session recall. Provenance-aware closure eliminates observed recovery in the tested positive-baseline settings, whereas an exact-literal write filter blocks reinsertion but cannot erase information already present. Together, these results show that source deletion should be evaluated behaviorally at the level of the agent and its memory system, not only as a storage operation.
| Comments: | 25 pages, 7 figures, 12 tables |
| Subjects: | Cryptography and Security (cs.CR); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2.7; K.4.1; D.4.6 |
| Cite as: | arXiv:2610.31366 [cs.CR] |
| (or arXiv:2610.31366v1 [cs.CR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31366 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ehsan Saleh [view email]
[v1]
Thu, 22 Oct 2026 14:41:36 UTC (1,153 KB)
来源：arXiv cs.CL · arxiv.org