arXiv cs.CL· Guang Yang, Homa Hosseinmardi, Fengchen Liu, Amir Ghasemian·· 10 天前AI 评分40
超越谄媚分数：任务、模型与压力如何影响 LLM 让步行为
Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding
AI 导读
一项基于 103,939 条评分回复的研究发现，LLM 谄媚让步的主导因素是验证用户主张的成本，以及是否有训练过的护栏覆盖该场景，而非压力话术。从逻辑模型中移除任务因素会损失 0.485 的 McFadden R²，模型家族为 0.139，压力策略仅 0.009。
正文 · 原文
Abstract:Large language models (LLMs) often abandon a correct answer, or endorse a user's position, once the user pushes back. This behavior, called sycophancy, is usually reported as a single rate per model, which says little about when it happens or how a user can avoid it. We study the conditions that produce it with 103,939 graded replies from ten configurations: eight LLMs with reasoning disabled, and two of them again with maximum reasoning, all facing the same 200 items, 13 pressure conditions, and four-turn conversations, with every reply labeled by two independent LLM judges. We find that the dominant factors are how costly it is for the model to verify the user's claim, and whether a trained guardrail covers it. Removing this task factor from a logistic model costs 0.485 of McFadden $R^2$, against 0.139 for model family and 0.009 for pressure tactic. Anchored facts are almost never conceded (1.3%), while adoption on logic puzzles rises with the number of clues needed to refute the pushed answer. Personal choices are endorsed in 77.0% of conversations. Most concessions on hard items come from models that cannot reliably solve them; models that can solve them rarely give the answer up. For both models tested, maximum reasoning removes these concessions completely: adoption on deep puzzles falls from 19.2% and 12.5% to 0%. Fallacious or emotional framing adds nothing beyond plain repetition. Three human annotators agree with the judges' consensus on 118/120 calibration items. These results give practical rules for reliable use: simplify hard-to-verify problems and reason deeply, state the question rather than one's preferred answer, ask for evidence on open questions, and choose models by their measured guardrail profile.
| Comments: | Preprint. 27 pages |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.08840 [cs.CL] |
| (or arXiv:2610.08840v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.08840 arXiv-issued DOI via DataCite |
Submission history
From: Guang Yang [view email]
[v1]
Wed, 30 Sep 2026 05:27:34 UTC (360 KB)
来源：arXiv cs.CL · arxiv.org