arXiv cs.CL· Marek \v{S}uppa, Ivan Vykopal, Jaroslav Kop\v{c}an·· 13 小时前AI 评分34
智能体能否调试网络语料库？对斯洛伐克语 FineWeb2 的智能体审计
Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
AI 导读
研究将 LLM 用作语料库调试器而非分类器，通过只读、有预算限制的 API 让工具型智能体探索 FineWeb2 的 2650 万篇斯洛伐克语文档，三次运行各花费 \$2.21-\$2.84，共发现四类现象下的七个问题。
正文 · 原文
Abstract:Multilingual pipelines such as FineWeb2 apply one recipe across a thousand languages, leaving the failure modes of individual languages unanticipated. We use an LLM once, as a corpus debugger rather than a classifier: a tool-using agent explores the corpus through a read-only, budgeted API and compiles each finding into an executable test, which a native speaker then validates. On the 26.5M-document Slovak portion of FineWeb2, three runs costing \$2.21-\$2.84 each found seven findings over four phenomena. The largest is a machine-generated content farm of 1,408,437 documents (5.32%) that passed the pipeline's filters, 86.6% of it passing even stricter thresholds. Blinded labels support 6 of the 7 findings (defect rates .53-1.00 against .08 in a random control) and do not support one. The compiled suite scans the full corpus in 0.7 s of wall-clock time without a GPU, against \$12k-\$90k per LLM classification pass. The best static sampling baseline validates more selectors than any agent run, at under half the cost. The agent contributes exact prevalence, a precision estimate checked against sampled documents, an account of why the pipeline missed each slice, and a reusable test. We release the framework, Slovak test suite, validation data and agent trajectories.
| Comments: | Accepted to WaC Workshop at EMNLP 2026 |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.31636 [cs.CL] |
| (or arXiv:2610.31636v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31636 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ivan Vykopal [view email]
[v1]
Thu, 22 Oct 2026 17:52:22 UTC (156 KB)
来源：arXiv cs.CL · arxiv.org