arXiv cs.CL· Yibo Wang, Jisheng Dang, Bimei Wang, Jiongpei Hou, Jingwen Zhao, Wenjun Ma, Fei Shen, Juncheng Li, Meng Wang, Qi Tian, Bin Hu, Tat-Seng Chua·· 11 小时前AI 评分31
EviCast：通过共享响应校准预测检索增益
EviCast: Forecasting Retrieval Gains Through Shared Response Calibration
AI 导读
EviCast 框架通过干预式证据状态分解预测检索价值，只需测量一次生成器对受控证据状态的响应，再结合各候选检索器的离线证据状态分布，即可在不执行全部 RAG 配置的情况下得出端到端净效用及其 95% 置信下界。
正文 · 原文
Authors:Yibo Wang, Jisheng Dang, Bimei Wang, Jiongpei Hou, Jingwen Zhao, Wenjun Ma, Fei Shen, Juncheng Li, Meng Wang, Qi Tian, Bin Hu, Tat-Seng Chua
Abstract:Retrieval-augmented generation (RAG) extends large language models with external evidence, yet relevant evidence does not ensure that a generator can use it. Indiscriminate retrieval can degrade task performance while adding retrieval and long-context inference costs. Existing methods neither determine before large-scale execution whether a generator, task distribution, and retriever configuration is worth deploying nor diagnose multiple candidate retrievers under a shared budget. We introduce EviCast, a framework that forecasts retrieval value through interventional evidence-state decomposition. EviCast measures a generator's response to controlled evidence states once and combines this shared profile with the offline evidence-state distribution of each candidate retriever, yielding the end-to-end net utility and its 95% lower confidence bound without executing every RAG configuration. A state-refinement analysis characterizes when its binary and topology-aware instantiations are exact and when shared calibration is statistically more efficient than per-retriever calibration. In frozen evaluations, EviCast correctly decides all 14 HotpotQA retrieval configurations with a forecast error of 2.75 F1 points while reducing diagnostic generation calls by 80%, and it attains the lowest forecast error among seven predictors in the low-budget regime. Its topology-aware instantiation, EviCast-T, captures the complementarity between terminal and bridge evidence, correctly deciding all 16 configurations on MuSiQue and HotpotQA and 23 of 24 after prospective transfer to a third dataset. These results show that retrieval deployment depends on both evidence availability and the generator's ability to convert evidence structure into stable net utility. Code is available at this https URL.
| Comments: | 13 pages, 6 figures, 8 tables |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.02387 [cs.CL] |
| (or arXiv:2611.02387v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.02387 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yibo Wang [view email]
[v1]
Mon, 2 Nov 2026 01:55:58 UTC (5,541 KB)
来源：arXiv cs.CL · arxiv.org