arXiv cs.AI· Himaghna Bhattacharjee, Yasha Sheynin, Shivam Singhal, Arjun Subramonian, Ankur Samanta, Julian Michael, Smitha Milli, Maximilian Nickel, Claudia Shi·· 4 小时前AI 评分34
从偏好数据到个性化:追踪大语言模型中的谄媚行为
From Preference Data to Personalization: Tracing Sycophancy in Large Language Models
AI 导读
研究将大语言模型谄媚重新定义为"不当认同",即上下文需要纠正或质疑时模型仍认可用户主张,并提出有效性感知评分器与含 3,298 条有效性标注探针的 SycoDrift 评测集。分析发现人类偏好数据中的认同倾向较弱且集中于认同本属合理的场景,而推理时个性化会加剧不当认同,其中紧邻查询放置匹配用户画像的影响最大。
正文 · 原文
Abstract:Sycophancy in large language models is often measured as agreement with the user, yet agreement can be appropriate when the user's claim is valid, their preference is genuine, or their experience warrants acknowledgment. We study sycophancy as inappropriate agreement: validating a claim, preference, or self-assessment when the context calls for correction, qualification, or resistance. We introduce a validity-aware sycophancy grader and SycoDrift, a persona-targeted evaluation set of 3,298 validity-annotated probes with controlled annotations for user openness and stakes. We use this framework to analyze three sources of agreement pressure: human preference data, automated judge specification, and in-context personalization. First, preference for validating responses in human datasets (Chatbot Arena, HH-RLHF, Community Alignment) is weak and concentrated where agreement is often appropriate. Second, in a selected-value, single-model experiment, emphasized judge rubrics produce consistent but modest increases relative to paired suppressed rubrics, demonstrating a plausible mechanism for training-time amplification. Third, in off-policy evaluations across 14 models, inference-time personalization increases inappropriate agreement, with the largest effect from matched user profiles placed directly before the query.
| Comments: | 66 pages, 19 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.21650 [cs.CL] |
| (or arXiv:2611.21650v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21650 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Himaghna Bhattacharjee [view email]
[v1]
Thu, 15 Oct 2026 04:53:04 UTC (463 KB)
来源:arXiv cs.AI · arxiv.org