arXiv cs.CL· Brandon Hsu, Daniel Beaglehole, Adityanarayanan Radhakrishnan, Mikhail Belkin·· 12 小时前AI 评分34
CLAS：面向语言模型的情境化线性激活引导方法
Contextual Linear Activation Steering of Language Models
AI 导读
研究者提出情境化线性激活引导（CLAS），将激活引导分解为固定引导方向与随上下文变化的引导系数，从而在保持方向可解释性的同时让引导强度随输入变化。CLAS 还能在单次生成的不同片段上引导多种行为，这是固定强度方法无法做到的。在多项引导基准和语言模型上，CLAS 稳定优于标准线性激活引导，在标注数据有限时达到或超过 ReFT 与 LoRA，且所用方向可作为可靠的概念监测器。
正文 · 原文
Abstract:Linear activation steering is a powerful approach for controlling and specializing large language models by adding linear representations of concepts to their activations at inference. While effective, existing methods often steer with a fixed strength, independent of how much steering any given input requires. In this work, we introduce Contextual Linear Activation Steering (CLAS), which decomposes activation steering into a fixed steering direction and a context-dependent steering coefficient. This decomposition preserves the interpretability of the steering direction while enabling the steering strength to vary with context. It also enables CLAS to perform selective steering: steering multiple behaviors across different portions of a single generation, which cannot be achieved using a fixed steering strength. In addition, across a wide range of steering benchmarks and language models, CLAS consistently outperforms standard linear activation steering and matches or exceeds ReFT and LoRA in settings with limited labeled data. Notably, CLAS achieves this using directions that also serve as reliable concept monitors, a property that directions learned via fine-tuning lack. We therefore propose CLAS as an interpretable and accurate method for controlling and specializing large language models.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2604.24693 [cs.CL] |
| (or arXiv:2604.24693v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2604.24693 arXiv-issued DOI via DataCite |
Submission history
From: Brandon Hsu [view email]
[v1]
Mon, 27 Apr 2026 16:54:16 UTC (99 KB)
[v2]
Fri, 30 Oct 2026 23:19:16 UTC (326 KB)
来源：arXiv cs.CL · arxiv.org