arXiv cs.CL· Xunlei Chen, Qinghui Gong, Ruini Xue, Yaodong Hu, Tian Lan, Wenhong Tian·· 10 小时前AI 评分31
ST²U：通过受限知识边界控制实现有状态测试时遗忘
ST$^2$U: Stateful Test-Time Unlearning via Restricted Knowledge Boundary Control
AI 导读
ST²U 将测试时遗忘建模为轨迹级边界控制，在低维可逆坐标中刻画受限知识边界，并在推理时沿轨迹监测风险、施加最小边界修正，同时跨 token 传播历史修正状态以抑制知识回流。在三个 benchmark 和三个模型家族上，ST²U 取得最佳或次佳的知识保留与遗忘平衡，受限知识回流率为 13.76%–19.84%，显著低于测试时基线的 30.44%–59.10%。
正文 · 原文
Abstract:Controlling restricted knowledge in large language models is essential for model alignment and safe deployment. Test-time unlearning avoids costly retraining and parameter updates by intervening only during inference. However, existing activation-editing methods apply isolated pointwise corrections, overlooking how autoregressive generation continually reconstructs hidden states from the prompt, cache, and generated prefix. Consequently, later states may return to restricted knowledge regions after a locally successful correction, causing restricted knowledge re-entry. In this work, we propose Stateful Test-Time Unlearning via restricted knowledge boundary control (ST$^2$U), which formulates test-time unlearning as trajectory-wide boundary control. ST$^2$U first models restricted knowledge boundaries in low-dimensional invertible coordinates while leaving orthogonal non-target components unchanged. During inference, ST$^2$U monitors risk along the trajectory, applies minimal boundary corrections with contextual anchoring, and propagates historical correction states across tokens to mitigate knowledge re-entry. This trajectory-wide control enables more persistent forgetting while preserving non-target capabilities and limiting inference overhead. Across three benchmarks and three model families, ST$^2$U delivers the strongest overall balance, combining best or second-best retention with competitive forgetting and substantially less restricted knowledge re-entry than test-time baselines (13.76--19.84% versus 30.44--59.10%).
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.23034 [cs.LG] |
| (or arXiv:2608.23034v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2608.23034 arXiv-issued DOI via DataCite |
Submission history
From: Xunlei Chen [view email]
[v1]
Mon, 24 Aug 2026 09:37:51 UTC (4,324 KB)
[v2]
Tue, 10 Nov 2026 14:32:48 UTC (5,308 KB)
来源：arXiv cs.CL · arxiv.org