跳到正文
arXiv cs.AI· Davide Liga, Liuwen Yu, Matteo Magnini·· 4 小时前AI 评分32

语言模型中的道德回路与道德特征辨析:道德内容、规范判断还是提示词?基于 14 个模型的证据

Disambiguating Moral Circuits and Moral Features in Language Models: Moral content, normative judgment or cue words? Evidence from 14 models

AI 导读

研究用 227 组经五名标注者验证的最小短语对测试 14 个开源模型(Llama、Gemma、Qwen,1B 至 14B,含基座与指令微调版),发现改变道德答案的注意力头属于模型处理所有规范判断共用的评估回路,道德对与其他规范对的效应份额合并比值为 1.00(含 "morally" 一词,97.5% 置信区间 [0.92, 1.07]),无该词时为 0.90。

正文 · 原文

View PDF HTML (experimental)

Abstract:Interpretability methods can locate the attention heads and sparse autoencoder (SAE) features that change the answer a language model gives to a moral prompt, e.g., whether "Stealing medicine for a dying child is morally" continues with "justified" or "wrong". Such a component could respond to moral content, to the judgment of conduct by any norm, or to the word "morally" in the prompt. We built 227 minimal pairs of short phrases, validated by five annotators, whose two versions differ in a moral standard, in another kind of norm (prudence, etiquette, aesthetics or evidence) or in a descriptive quality, and showed them to 14 open-weight models (Llama, Gemma and Qwen, 1B to 14B, base and instruction-tuned), with and without the word "morally". In every model, the heads that change the moral answer belong to an evaluation circuit the models use for every kind of normative judgment we tested: these heads carry the same share of the effect for moral as for other normative pairs (pooled ratio 1.00, 97.5% confidence interval [0.92, 1.07], with the word; 0.90, [0.84, 0.96], without it), and the models need them as much for both. The models need these heads also when nothing in the prompt asks for a judgment, and on 400 scenarios of the ETHICS benchmark, where removing them takes away 37 to 39% of the difference between acceptable and unacceptable scenarios, against 1 to 3% for ten random heads. The SAE features we located respond mostly to the word "morally" rather than to what the phrase is about. The models represent the category of a pair (the standard the annotators assigned to it): it can be read from their hidden states, and, in the eight models we tested, removing the direction that carries it weakens the moral verdict by a few percent, and more than the other verdicts in seven of them. We release the pairs, the ratings, the code, the trained SAE dictionaries and the result files.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.21705 [cs.CL]
  (or arXiv:2611.21705v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21705

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Davide Liga [view email]
[v1] Mon, 16 Nov 2026 16:12:45 UTC (394 KB)

来源:arXiv cs.AI · arxiv.org