arXiv cs.CL· Esther Xin·· 2 天前AI 评分24
RLVR 数学语料去污染：语义阈值清洗的可靠性究竟如何？
What Should We Remove? Evaluating the Reliability and Costs of RLVR Data Decontamination
AI 导读
研究审计了 16 个开源 RLVR 数学语料（1,673,514 条训练数据）对 6 个基准（1,949 道题）的去污染策略，发现余弦阈值 0.80 下 59 个抽样删除中有 52 个（88%）被确认为误删。
正文 · 原文
Abstract:Reinforcement learning with verifiable rewards (RLVR) math corpora are routinely screened for benchmark overlap, but the reliability and data-loss costs of cleaning policies remain underexamined. We audit sixteen open RLVR math corpora comprising 1,673,514 corpus-specific training items against six benchmarks containing 1,949 questions. We evaluate eight policies based on exact, lexical, and semantic matching using 594 independently and blindly dual-reviewed training-benchmark pairs. Calibration and held-out sets have disjoint benchmark questions and training-example families. A separate review of all 83 candidate matches for 80 sampled training items measures item-level false removal. At cosine threshold 0.80, 52 of 59 sampled removals (88%) were confirmed false removals; this is not a demonstrated population rate. No semantic threshold in our preregistered grid (0.80-0.98) meets the 90% conservative-precision target on calibration. The existing judge-adjudicated policy achieves 98.1% resolved precision and 98.2% candidate-frame recall on held-out corpora. It has zero confirmed false removals among six sampled removals, with one unresolved case; this does not establish zero risk. Within the evaluated candidate frame, this policy offers a more favorable precision-recall balance than the semantic-threshold alternatives tested. Recall does not measure contamination outside the candidate frame. We release reviewed labels, per-policy decisions, and coverage accounting to support evaluation of cleaning reliability and cost rather than overlap counts alone.
| Comments: | 13 pages, 4 figures, 4 tables,code and data:this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.21229 [cs.CL] |
| (or arXiv:2610.21229v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21229 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Esther Xin [view email]
[v1]
Thu, 15 Oct 2026 04:19:03 UTC (45 KB)
来源：arXiv cs.CL · arxiv.org