arXiv cs.CL· Enrico Liscio, Anne Arzberger, Aditya Shankar, Pradeep K. Murukannaiah·· 3 天前AI 评分38
模型 vs. 用户：谁的价值观说了算？多轮对话中的推理时价值对齐
Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations
AI 导读
研究评估了四种基于提示词和激活的引导方法在多轮对话中的价值对齐效果，用户与助手模型会朝相同或相反但同等可取的价值方向施压。通过线性探针追踪三个模型和两组价值权衡发现：当助手与用户方向相反时，与助手基线偏好一致的一方往往占上风，早期看似成功的引导效果常在数轮后消退甚至反转。
正文 · 原文
Abstract:Human values shape how we expect language models to behave. Dominant alignment approaches treat values as global objectives, although the appropriate value orientation is personal and contextual. Training-free alignment can steer models toward target values at inference time, yet has been largely evaluated in single-turn settings and with a single steering direction (e.g., toward safety). We evaluate four prompt- and activation-based steering methods in multi-turn conversations, where user and assistant models push toward the same or opposing, equally desirable values. We track alignment with a linear probe across three models and two value trade-offs. When assistant and user pull in opposite directions, the side matching the assistant's baseline bias often prevails, and effects that look successful early often fade or reverse after a few turns. Steering should therefore be evaluated over full conversations and in both directions of a value trade-off.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.19817 [cs.CL] |
| (or arXiv:2610.19817v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.19817 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Enrico Liscio [view email]
[v1]
Wed, 14 Oct 2026 08:58:17 UTC (426 KB)
来源：arXiv cs.CL · arxiv.org