arXiv cs.CL· Alexander Nemecek, Osama Zafar, Debargha Ganguly, Vikash Singh, Vipin Chaudhary, Erman Ayday·· 10 小时前AI 评分31
语言模型水印的跨语言公平性审计
Auditing Cross-Lingual Fairness in Language Model Watermarking
AI 导读
研究提出一套评估框架，用于审计大语言模型水印方案在跨语言场景下的公平性，包含按部署环境经验校准的检测阈值、区分校准失败与检测失败的阈值无关指标、三种互不重叠的质量测量范式，以及基于类型学语系划分的广义熵分解。
正文 · 原文
Abstract:Watermarking schemes for large language model output are evaluated almost exclusively on English text using each scheme's detection threshold and a narrow set of quality measurements. Multilingual deployment exposes evaluation-design choices that are inconsequential on English but determine conclusions cross-lingually. We propose an evaluation framework with four components: detection thresholds calibrated empirically per deployment context, a threshold-independent companion measurement that distinguishes calibration failures from detection failures, three disjoint quality measurement paradigms (distributional, paired-semantic, and reference-perplexity), and a generalized-entropy decomposition of cross-language disparity over a typological family partition. Applied to six watermarking schemes, three open-weight generators, eleven languages spanning four scripts and eight typological families, and both base and instruction-tuned regimes, the framework reveals failure modes that single-language, single-paradigm evaluation cannot surface. Against a shape-matched null, detection disparity is predominantly between-family on the typological partition, indicating that detection gaps are structural to language properties rather than idiosyncratic to particular languages; quality disparity is large but not reliably family-structured.
| Comments: | Accepted at NeurIPS 2026 Evaluations & Datasets Track |
| Subjects: | Computation and Language (cs.CL); Cryptography and Security (cs.CR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2608.20047 [cs.CL] |
| (or arXiv:2608.20047v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.20047 arXiv-issued DOI via DataCite |
Submission history
From: Alexander Nemecek [view email]
[v1]
Thu, 20 Aug 2026 13:48:12 UTC (308 KB)
[v2]
Tue, 10 Nov 2026 06:11:46 UTC (320 KB)
来源：arXiv cs.CL · arxiv.org