面向 LLM 的正交与参数化多属性引导方法

Orthogonal and Parametrized Multi-Attribute Steering of LLMs

AI 导读

研究提出参数化引导方法，不再沿方向推动激活，而是将激活直接设定到目标特质强度，并先为每个引导向量识别最可引导区间、按此重缩放引导范围，使引导强度具有统一可解释含义。针对多特质引导中朴素组合向量会相互干扰的问题，作者开发了向量正交化方法，解耦各特质，使模型可同时引导多个行为而不互相诱发。

正文 · 原文

View PDF HTML (experimental)

Abstract:Activation steering offers an effective and efficient way to modify an LLM's behavior at inference time. However, it remains unclear how to apply steering such that the signal is consistently strong and the steering strength has a universally interpretable meaning. We address this first by identifying the most steerable interval for each steering vector and rescaling the steering range accordingly. Building on this, we introduce a more interpretable method for steering, called parameterized steering which instead of pushing activations in a direction, it sets them at the desired trait strength. Next, with multi-trait steering, we find that naively combining vectors causes them to interfere with one another. In response, we develop a method to orthogonalize the vectors; it decouples these traits, allowing a model to exhibit one behavior without inducing the others, thus enabling multiple traits to be steered simultaneously.

Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.37517 [cs.LG]
	(or arXiv:2610.37517v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2610.37517 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Remy Ogasawara [view email]
[v1] Tue, 27 Oct 2026 16:39:53 UTC (1,326 KB)

来源：arXiv cs.AI · arxiv.org