arXiv cs.CL· Arzu Burcu G\"uven, Anna Rogers, Leonie Weissweiler, Joakim Nivre, Rob van der Goot·· 6 天前AI 评分20
BLiMPPI：覆盖 33 种语言的极性词语言最小对基准
BLiMPPI: Benchmark of Linguistic Minimal Pairs for Polarity Items
AI 导读
研究者推出 BLiMPPI，一个覆盖 33 种语言、超过 12,000 组 NPI 最小对的多语言评测套件，通过从 Universal Dependencies 树库中发现极性词（NPI）、标注其允准环境并扰动生成最小对构建而成。
正文 · 原文
Abstract:Negative polarity items (NPIs), such as "anymore" and "ever", are grammatical only in select environments, predominantly in the context of negation. Because their grammaticality depends on both syntax and semantics, they have a rich, multilingual literature in theoretical linguistics, yet work on NPIs in language models is almost entirely confined to English. We introduce BLiMPPI, a multilingual evaluation suite of over 12,000 NPI minimal pairs in 33 languages, built by a pipeline that discovers NPIs in Universal Dependencies (UD) treebanks, labels their licensing environments, and perturbs these into minimal pairs. Evaluating 12 multilingual and 2 monolingual models alongside n-gram baselines, we find that multilingual models struggle most with lower-resource languages and with pairs that are not n-gram solvable. We further examine the effects of NPI typology, language family, licensing variability, and NPI-licenser distance. Data: this https URL
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.13607 [cs.CL] |
| (or arXiv:2610.13607v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.13607 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arzu Burcu Güven [view email]
[v1]
Fri, 9 Oct 2026 13:55:00 UTC (286 KB)
来源：arXiv cs.CL · arxiv.org