arXiv cs.CL· Stefan Larson, Kevin Leach·· 12 小时前AI 评分39
RVL-CDIP 数据质量系统审计：约 10.5% 文档标签错误
Data Quality in RVL-CDIP: A Systematic Audit
AI 导读
对文档分类基准 RVL-CDIP 的系统审计发现，其 40 万份文档中约 10.5% 标签错误，测试集 35.2% 存在重复或近重复样本。仅凭页面识别码即可达到 69% 分类准确率，SOTA 模型去掉识别码后下降 11 个点。修复全部缺陷后重新基准测试 25 个模型，重训的 13 个模型准确率提升 0.9 至 2.9 个点，其中 LayoutLMv3 达 96.5%。
正文 · 原文
Abstract:RVL-CDIP is the de facto standard benchmark for document classification, and reported accuracy on it is now 98.8%. In this paper, we audit RVL-CDIP for data quality issues and find that approximately 10.5% of its 400,000 documents have erroneous labels and that 35.2% of the test set has a duplicate or near-duplicate in the training set. Identification codes on the pages are sufficient to classify at 69% accuracy, and a state-of-the-art model loses 11 points without them. Image-only models trained on RVL-CDIP lose 16 to 28 points on documents from other sources, and the choice of OCR engine changes accuracy by 4.1 to 8.3 points. We find that nearly 16,000 RVL-CDIP documents contain sensitive personal data, and produce de-identified versions of these documents. We re-benchmark 25 models, and with all defects remediated, the 13 that we retrain are 0.9 to 2.9 points more accurate than in the original condition; the most accurate of them, LayoutLMv3, is 96.5% accurate. We release the corrected corpus, the annotations and the out-of-distribution data.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.32126 [cs.CV] |
| (or arXiv:2610.32126v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32126 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Stefan Larson [view email]
[v1]
Fri, 23 Oct 2026 03:52:03 UTC (12,996 KB)
来源：arXiv cs.CL · arxiv.org