arXiv cs.CL· Tapan Parikh·· 12 小时前AI 评分46
语言模型如何学会品味：LLM 中的品牌偏好
How Language Models Learn Taste: Brand Preference in LLMs
AI 导读
研究覆盖 102 个模型和 42 个品牌类别发现，模型被要求说出跑鞋品牌时答 Nike，被要求推荐时却选 Brooks，命名品牌与推荐品牌在 26 个类别中不一致。三条开源训练流程显示，预训练已把推荐品牌与建议绑定，监督微调基本不改变选择，而 DPO 让选择在三条流程中移动 11 至 24 个百分点，且偏好数据几乎不含品牌。作者怀疑这种偏好来自偏好训练中判定答案优劣的 judge 模型。
正文 · 原文
Abstract:Asked to name a running shoe brand, language models say Nike. Asked to recommend one, they say Brooks. This paper asks where models learn that taste. Across 102 models and 42 brand categories, the brand a model picks when recommending differs from the brand it names in 26. The field moves together to a shared recommendation. The Named brands are market leaders, and the Picked brands are mostly niche and specialist. Three open training pipelines show where this forms. Pretraining already attaches the recommended brand to advice. Supervised fine-tuning teaches the model to answer and mostly leaves the pick unchanged. Preference optimization (DPO) teaches the model which answer is better, and the pick moves in all three pipelines, by 11 to 24 points. The preference data contains almost no brands. DPO moves the model's judgments toward a second set of associations even when the preferred brand is not in the preference data, and the effect reaches beyond brands. Asked to choose a sauce, models switch from ketchup to pesto. We do not know exactly how these preferences arise. We suspect they come from the judge models that decide which answers count as better during preference training. For brands, a recommendation depends less on what a model knows about them than on whether they fit this taste.
| Comments: | 11 pages |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.28272 [cs.CL] |
| (or arXiv:2610.28272v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28272 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tapan Parikh [view email]
[v1]
Tue, 20 Oct 2026 14:26:19 UTC (13 KB)
来源：arXiv cs.CL · arxiv.org