arXiv cs.CL· Amr Sobhy·· 12 小时前AI 评分18
FrenchNews-7：跨出版商法语新闻编辑部版面分类基准
FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification
AI 导读
FrenchNews-7 是一个跨出版商法语新闻编辑部版面分类基准，结合多媒体语料、基于 URL 的七类分类体系和微调后的 CamemBERT 分类器。在留出出版商设置下，基于全文的 CamemBERT-base 整体召回率达 0.799，超过 GPT-OSS-120B、Mistral Small 3.2、Llama-3.3-70B 三个零样本 LLM 基线。
正文 · 原文
Abstract:We present FrenchNews-7, a cross-publisher France-based French-language news editorial desk classification benchmark combining a large multi-outlet corpus, a URL-derived seven-class taxonomy, and a fine-tuned CamemBERT classifier. Labels are assigned via a hybrid pipeline combining publisher URL slugs with LLM annotation for structurally ambiguous cases, audited through an inter-rater study (2 humans + 2 LLMs; pairwise $\kappa \geq 0.766$, human--human $\kappa = 0.806$). We evaluate lexical, multilingual, and French-specific trained classifiers under both in-distribution and held-out-publisher settings, with additional comparison against zero-shot LLM baselines (GPT-OSS-120B, Mistral Small 3.2, Llama-3.3-70B) on the held-out pool. The strongest model, CamemBERT-base on full article text, outperforms headline-only input, generalizes to unseen outlets, and exceeds all three zero-shot LLM baselines on overall recall (0.799), with the gap concentrated in the ambiguous editorial-boundary categories Economie and Societe. Cross-publisher evaluation reveals uneven boundary stability: Sport, Culture & Loisirs, and International transfer cleanly, while Economie (recall = 0.517) is statistically indistinguishable from blinded human agreement on the same articles (0.55, $n = 40$), and Societe (precision = 0.577) absorbs boundary ambiguity, both suggesting editorial conventions rather than recoverable classifier headroom. The fine-tuned CamemBERT-base model, labeled manifest, reference collection scripts, and a reliability-tier guidance table are available at this https URL (model) and this https URL (dataset).
| Comments: | 16 pages, 5 figures. Accepted at AACL-IJCNLP 2026 (Main Conference) |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2608.18097 [cs.CL] |
| (or arXiv:2608.18097v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.18097 arXiv-issued DOI via DataCite |
Submission history
From: Amr Sobhy [view email]
[v1]
Mon, 8 Jun 2026 16:54:07 UTC (140 KB)
[v2]
Thu, 15 Oct 2026 19:55:54 UTC (231 KB)
来源：arXiv cs.CL · arxiv.org