DeepMind 论文：强化学习训练 Gemma-4 4B 写道德论证，答案更稳定但判断力未提升
DeepMind 论文：训练模型写道德论证更果断但判断力未提升
Google DeepMind 新论文用强化学习训练 Gemma-4 4B 写道德论证，模型在选项互换时的答案翻转率从 14.3% 降至 6.6%，但附和用户的比例仅从 45.3% 变为 44.0%，道德寓言准确率反而从 56.5% 降到 53.4%。论文据此建议，评估模型的建议或判断任务时应分别测试一致性与正确性，因为更稳定的模型仍可能出错。
Rohan Paul· @rohanpaul_ai · X·2026-10-30 00:23· 35 分钟前AI 评分49
AI 导读
Google DeepMind 新论文发现，用强化学习训练 Gemma-4 4B 写道德论证后，模型在选项互换时的答案翻转率从 14.3% 降至 6.6%，但附和用户的比例仅从 45.3% 变为 44.0%，道德寓言准确率反而从 56.5% 降到 53.4%。论文据此建议，评估模型的建议或判断任务时应分别测试一致性与正确性，因为更稳定的模型仍可能出错。
正文 · 原文
New Google DeepMind paper finds that training a model to write better moral arguments makes it more decisive, but no better at moral judgment.
LLMs are more and more often asked for moral advice. A model that keeps its answer when the options are reordered is often taken as a sign of sound judgment.
They trained Gemma-4 4B with reinforcement learning to argue dilemmas taken from classic novels. The model judged its own arguments in pairs, and the winners set the reward.
Answer flips when options were swapped fell from 14.3% to 6.6% on ambiguous MoralChoice cases. Yet siding with the user barely moved, from 45.3% to 44.0%, and accuracy on moral fables dropped from 56.5% to 53.4%.
If you evaluate models for advice or judgment tasks, test consistency and correctness separately, since a steadier model can still be wrong.
来源：Rohan Paul · x.com
来源：RadarAI · aihot.news