arXiv cs.CL· Hirofumi Morise·· 11 小时前AI 评分26
基于稀疏自编码器的激活引导：信息论设计框架
Information-Theoretic Design of Sparse Autoencoder-Based Steering
AI 导读
研究提出一个信息论框架，通过经验特征激活分布分析稀疏自编码器（SAE）激活，以 Jensen-Shannon 散度（JSD）量化生成或干预状态偏离数据集目标分布的程度。基于虚拟偏移状态的解析散度响应，该方法构造出指向目标激活分布的引导向量，并在问题模式生成任务中验证了有效性。教师强制脉冲实验显示局部干预会在后续层引发结构化的下游 JSD 响应，该框架还被用于安全相关的目标—对比响应特征分析。
正文 · 原文
Abstract:Sparse autoencoder (SAE)-based activation steering is a lightweight approach for modifying the behavior of large language models (LLMs) by adding small vectors to hidden states during inference. However, designing effective steering interventions remains challenging. In this work, we develop an information-theoretic framework for analyzing SAE activations through empirical feature-firing distributions. The central object of the framework is the Jensen--Shannon divergence (JSD) between such firing distributions, which quantifies how far a generated or intervened state deviates from a dataset-induced target distribution. Based on an analytic divergence response for virtually shifted states, we construct steering vectors toward the target firing distribution. The same divergence can also be used to monitor deviation from another empirical distribution, such as a general valid-language distribution. We demonstrate the effectiveness of the steering in a question-mode generation task study. We further use teacher-forced pulse experiments to show that local interventions induce structured downstream JSD responses in later layers, and apply the framework to a safety-related target--contrast response setting for feature analysis.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.01378 [cs.CL] |
| (or arXiv:2611.01378v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01378 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hirofumi Morise [view email]
[v1]
Sun, 1 Nov 2026 00:33:01 UTC (439 KB)
来源：arXiv cs.CL · arxiv.org