arXiv cs.CL· Anxin Guo, Chirag Pabbaraju, Sara Ahmadian, Kamesh Munagala, Aravindan Vijayaraghavan·· 12 小时前AI 评分39
推理语言模型有多信任一只狗？量化 LLM 的提示跟随行为
How Much Do You Trust A Dog? Quantifying Hint-Following in Reasoning Language Models
AI 导读
研究在 MMLU 题目上以「[来源]认为答案是 X」的提示测试 Qwen、Gemma、SmolLM 三个开源模型族，发现提示引发的答案偏移呈指数倾斜结构，即对被提示答案的 log-odds 产生近似恒定的均值偏移。这种偏移在提示来源被声明不准确或附带明确警告时依然存在，说明模型并未按来源可靠性校准信任。
正文 · 原文
Abstract:Large language models (LLMs) often exhibit sycophancy, altering their answers to align with user suggestions even on objective factual and reasoning tasks. Yet the quantitative form of this influence--- and whether it is calibrated to source reliability--- remains poorly understood. In this work, we investigate this behavior through hint-induced answer shifts on MMLU questions, evaluating prompts of the form ``[Source] thinks the answer is (X)''. We systematically vary the source to which the hint is attributed (a professor, a friend, or a dog) across three open-source model families, Qwen, Gemma, and SmolLM. We show that the change induced by the hint follows a predictable structure well-characterized by an exponential tilt: the hint induces an approximately constant mean shift in the hinted answer's log-odds, with question--target susceptibility largely shared across sources. To explore how this pattern can arise, we introduce a formal decomposition of the hint response and illustrate in controlled synthetic experiments how KL-regularized reward-based training reshapes both hinted and unhinted answer distributions. Finally, we evaluate whether this shift is calibrated to source reliability, finding that the log-odds shift persists even when the hint-giver is declared inaccurate or accompanied by explicit warnings not to trust the source, highlighting the vulnerability of LLMs to adversarial steering.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.32055 [cs.CL] |
| (or arXiv:2610.32055v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32055 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Anxin Guo [view email]
[v1]
Fri, 23 Oct 2026 01:51:31 UTC (1,426 KB)
来源：arXiv cs.CL · arxiv.org