arXiv cs.CL· Ruiyi Yan, Zhuoyuan Mao, Yugo Murawaki, Yiwen Guo·· 11 小时前AI 评分26
Re³Rank：用推理条件检索头做列表式重排序
Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval Heads
AI 导读
Re³Rank 是一种基于推理条件检索头（RRHeads）的列表式重排序器，先由生成器针对问题和候选写出推理轨迹，再由打分器将轨迹 token 作为注意力查询读取相关性分数。团队提出直通估计器让排序损失穿过离散轨迹回传，使生成器与打分器可联合训练。在 9 个基准上平均，4B 版本比最强基线高 3.11 NDCG@10，0.6B 版本也超过参数量达 32B 的重排序器。
正文 · 原文
Abstract:Attention-based rerankers score each candidate by the attention mass that retrieval heads route to it from the question, but the question, serving as a fixed query, carries limited information for locating relevant evidence. We propose Re$^3$Rank, a listwise reranker built on reasoning-conditioned retrieval heads (RRHeads). A generator first writes a reasoning trace over the question and the candidates, and a scorer then uses the trace tokens as the attention query of a few selected RRHeads, reading relevance as continuous scores. We present a straight-through estimator to propagate the ranking loss through the discrete trace, so that the generator and the scorer can be trained jointly for ranking. Averaged over 9 benchmarks, Re$^3$Rank outperforms 11 baseline rerankers: our 4B variant exceeds the strongest baseline by 3.11 NDCG@10 points on average, and even our 0.6B variant surpasses rerankers with up to 32B parameters.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.02550 [cs.CL] |
| (or arXiv:2611.02550v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02550 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ruiyi Yan [view email]
[v1]
Mon, 2 Nov 2026 05:24:12 UTC (590 KB)
来源：arXiv cs.CL · arxiv.org