arXiv cs.CL· Mykhailo Poliakov, Nadiya Shvai·· 12 小时前AI 评分29
用隐藏状态探针替代 GPT-3.5：多源 RAG 元数据过滤新方法
Probe, Don't Prompt: A Hidden-State Probe for Metadata Filtering in Multi-Meta-RAG
AI 导读
研究提出用小型开源模型隐藏状态上训练的探针替代 GPT-3.5 完成 Multi-Meta-RAG 的新闻来源过滤，在全部 2556 条 MultiHop-RAG 查询上取得 90.9% 的集合完全准确率，高于子串基线的 88.0% 和 GPT-3.5 的 80.9%。
正文 · 原文
Abstract:Multi-Meta-RAG improves retrieval for multi-hop question answering by filtering a vector store on metadata, specifically the news source and the publication date, which it extracts from each query using GPT-3.5. This extractor requires a proprietary API call for every query, and its output is not restricted to the allowed sources: even though the prompt specifies 49 sources, GPT-3.5 generates 128 distinct source names, 79 of which do not appear on the list. We show that the source part of this extractor can be replaced by a local, deterministic probe trained on the hidden states of a small open-source language model. The probe is a multi-label logistic classifier on the mean-pooled hidden states of one layer, trained with a class-balanced loss on the gold evidence sources. Using 5-fold cross-validation to select the layer, we evaluate four models ranging from 135M to 1.5B parameters out-of-fold across all 2556 MultiHop-RAG queries. The probe reaches a set-exact accuracy of 90.9%, compared to 88.0% for a substring baseline and 80.9% for GPT-3.5. The difference comes from null queries, for which GPT-3.5 never predicts an empty source set. Across the remaining queries, all three methods achieve set-exact accuracies within 1.1 percentage points of each other. Unlike earlier probing work, we use the probe as a component of a retrieval pipeline rather than for interpretation. For all four models, the cross-validation selects one of the first layers, in contrast to the middle layers reported for retrieval tasks. We attribute this to the source name appearing verbatim in 95.4% of gold query-source pairs. The 135M model is 1.5 percentage points less accurate than the 1.5B model, so the filter requires only a partial forward pass through a small model and one linear layer. The code is available at this https URL.
| Comments: | Accepted to Computer Science and Applied Mathematics |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.03929 [cs.CL] |
| (or arXiv:2607.03929v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.03929 arXiv-issued DOI via DataCite |
Submission history
From: Mykhailo Poliakov [view email]
[v1]
Sat, 4 Jul 2026 15:47:38 UTC (85 KB)
[v2]
Tue, 20 Oct 2026 10:26:00 UTC (88 KB)
来源：arXiv cs.CL · arxiv.org