arXiv cs.AI· Carlos Pel\'aez-Gonz\'alez, Francisco Herrera, Andr\'es Herrera-Poyatos·· 14 小时前AI 评分34
LLM 推荐系统中的位置偏差可被统计检测：POB-ICM 与 POB-UNBP 方法及实证
Statistical Detection of Positional Bias in LLM-Based Recommender Systems: Methodologies and Empirical Evidence
AI 导读
研究提出 POB-ICM 与 POB-UNBP 两种方法检测 LLM 推荐系统中的位置偏差，即推荐结果依赖候选商品在提示词中的排列顺序。基于真实商品目录、固定提示模板和 2000 条查询各 20 种排列（共 40,000 个实例）的 RAG 测试台，所有受测 SOTA 模型均拒绝位置独立性与均匀性原假设。
正文 · 原文
Abstract:Large Language Models (LLMs) are increasingly used as recommendation generators in conversational recommender systems (RS) via retrieval-augmented generation (RAG). This flexible design introduces new sources of structural bias. This paper studies positional bias in top-1 and single-turn LLM-based RS: systematic dependence of the recommended item on the order in which candidates are presented in the prompt. We propose two complementary detection methodologies: (i) POB-ICM, based on contingency matrix analysis and Chi-square independence testing, and (ii) POB-UNBP, a sample-efficient approach derived from a uniformity null model under permutation-based evaluation, enabling stochastic decoding with formal statistical guarantees. Using a controlled RAG testbed for product recommendation, built on a real product catalog, a fixed prompt template and 2000 template-generated user queries with 20 permutations each (40,000 instances), both methodologies reject the null hypothesis of positional independence and uniformity in all evaluated state-of-the-art models. We introduce three complementary metrics -- permutation sensitivity, positional bias, and strong positional bias -- to quantify this effect, the latter targeting cases where the position displaces the recommendation towards clearly less relevant items rather than only choosing among comparably relevant ones. A robustness analysis across candidate-set size and decoding temperature shows that the bias is structural and model-dependent, not an artifact of deterministic decoding or candidate-set size. These results establish positional bias as a measurable, reproducible property of prompt-mediated recommendation pipelines in the evaluated setting and motivate routine auditing and governance controls (e.g., order randomization, standardized prompting) for deployed LLM-based RS. Source code is publicly released.
| Comments: | 34 pages, 8 figures, 18 tables |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI) |
| ACM classes: | H.3.3; I.2.7; G.3 |
| Cite as: | arXiv:2610.37962 [cs.IR] |
| (or arXiv:2610.37962v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37962 arXiv-issued DOI via DataCite |
Submission history
From: Carlos Peláez-González [view email]
[v1]
Tue, 27 Oct 2026 20:48:30 UTC (963 KB)
来源：arXiv cs.AI · arxiv.org