arXiv cs.CL· Siddhesh Pawar, Chen Cecilia Liu, Anna Korhonen, Isabelle Augenstein·· 13 小时前AI 评分47
研究揭示 LLM 价值微调的意外副作用：未训练价值、风险偏好与安全特质均受波及
Unintended Side Effects of Value Training in LLMs
AI 导读
研究对四个指令微调 LLM 按 Hofstede 各价值维度分别微调后发现，针对特定价值的训练会对未训练价值、风险偏好、社会认知和安全特质产生意外副作用。作者据此建议，后训练阶段的价值对齐应纳入更广泛的行为评估，并需进一步研究额外价值训练在流程中的最佳位置。
正文 · 原文
Abstract:As LLMs are increasingly deployed in value-sensitive contexts, aligning them with human values has emerged as a post-training objective in NLP, aimed at instilling additional values and guiding model behaviour. However, current evaluations remain narrow. Because values are interconnected, it is unclear how fine-tuning for specific values alters unrelated capabilities and behaviours. In this work, we adapt four instruction-tuned LLMs across individual Hofstede's value dimensions and evaluate them on both cross-value transfer and broader behaviours. We find that value-specific training produces unintended side effects on untrained values, risk preferences, social cognition, and safety traits. Consequently, we suggest that value alignment after post-training should incorporate broader behavioral evaluations, and we highlight the need to investigate where additional value training might best fit into the pipeline.
| Comments: | 37 pages, 31 figures, 26 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| MSC classes: | 68T50 |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.29922 [cs.CL] |
| (or arXiv:2610.29922v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29922 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Siddhesh Pawar [view email]
[v1]
Wed, 21 Oct 2026 15:58:53 UTC (1,600 KB)
来源：arXiv cs.CL · arxiv.org