arXiv cs.AI· Jaeyoung Park, Hantian Zhang, Kexin Rong, Steven Euijong Whang·· 12 小时前AI 评分31
RAG 系统的话题级公平性研究
Topic-Level Fairness in Retrieval-Augmented Generation
AI 导读
针对 RAG 系统聚合指标掩盖话题级群体差异的问题，研究者提出按话题分别用 Bradley-Terry 目标微调重排序器，偏好对直接由生成器自身输出构建。仅用 RoMQA 数据构建偏好对时，该方法在 RoMQA 上将各话题平均绝对差异降低 51%，在 TREC Fair Ranking 上降低 40%，同时保持准确率。所有基线方法及单一重排序器方案均未达到同等降幅。
正文
Abstract:Group fairness in retrieval-augmented generation (RAG) systems is typically evaluated using metrics aggregated across queries. However, such metrics can conceal topic-level disparities: a system may favor women on questions about human rights and men on questions about actors and filmmakers, appearing nearly neutral overall even though neither topic is fair. We therefore study topic-level fairness, which requires group fairness to hold within each topic rather than only in aggregate. To reduce the topic-level disparity, we adapt the reranker separately for each topic with a Bradley-Terry objective under direct supervision from the generated output: preference pairs built from the generator's own output carry the direction that topic requires. With preference pairs built from RoMQA alone, the adaptation reduces the mean absolute per-topic disparity by 51% on RoMQA and by 40% on TREC Fair Ranking while retaining accuracy. No baseline reaches the reduction, even when modified to treat each topic separately rather than apply one correction to every topic. Neither does our method with one reranker for all topics, which follows the direction that most topics require.
| Comments: | 21 pages, 8 figures, 16 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.34683 [cs.AI] |
| (or arXiv:2610.34683v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34683 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaeyoung Park [view email]
[v1]
Mon, 26 Oct 2026 01:19:50 UTC (484 KB)
来源：arXiv cs.AI · arxiv.org