arXiv cs.AI· Ruikang Zhang, Shuo Wang, Qi Su·· 3 天前AI 评分10
基于稀疏自编码器检索与调控 LLM 高阶语义特征的研究被撤回
Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders
AI 导读
该论文已被作者 Ruikang Zhang 撤回，原因是论文经过大幅重构，新版本已提交为 arXiv:2607.26853。原研究提出基于稀疏自编码器的框架，通过受控语义对立的对比特征检索流程，从稀疏激活空间中提取单义功能特征，并以大五人格特质为案例实现模型行为的双向精准调控，同时提出"功能忠实性"效应。
正文 · 原文
This paper has been withdrawn by Ruikang Zhang
No PDF available, click to view other formats
Abstract:Recent work in Mechanistic Interpretability (MI) has enabled the identification and intervention of internal features in Large Language Models (LLMs). However, a persistent challenge lies in linking such internal features to the reliable control of complex, behavior-level semantic attributes in language generation. In this paper, we propose a Sparse Autoencoder-based framework for retrieving and steering semantically interpretable internal features associated with high-level linguistic behaviors. Our method employs a contrastive feature retrieval pipeline based on controlled semantic oppositions, combing statistical activation analysis and generation-based validation to distill monosemantic functional features from sparse activation spaces. Using the Big Five personality traits as a case study, we demonstrate that our method enables precise, bidirectional steering of model behavior while maintaining superior stability and performance compared to existing activation steering methods like Contrastive Activation Addition (CAA). We further identify an empirical effect, which we term Functional Faithfulness, whereby intervening on a specific internal feature induces coherent and predictable shifts across multiple linguistic dimensions aligned with the target semantic attribute. Our findings suggest that LLMs internalize deeply integrated representations of high-order concepts, and provide a novel, robust mechanistic path for the regulation of complex AI behaviors.
| Comments: | The paper has been substantially restructured, and the new version has been submitted as arXiv:2607.26853 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.02978 [cs.CL] |
| (or arXiv:2601.02978v3 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.02978 arXiv-issued DOI via DataCite |
Submission history
From: Ruikang Zhang [view email]
[v1]
Tue, 6 Jan 2026 12:40:37 UTC (621 KB)
[v2]
Tue, 7 Apr 2026 02:46:06 UTC (626 KB)
[v3]
Tue, 13 Oct 2026 12:15:54 UTC (1 KB) (withdrawn)
来源：arXiv cs.AI · arxiv.org