arXiv cs.CL· Noor Abo Mokh, Teresa Lynn, Nizar Habash, Ahmed Elshabrawy, Sara Alhajeri, Alaa Elsetohy, Preslav Nakov, Alham Fikri Aji·· 12 小时前AI 评分31
DAR-MGSM：面向方言阿拉伯语的数学推理基准发布
Benchmarking Arabic Dialects for Math Problems
AI 导读
研究者推出 DAR-MGSM，首个针对方言阿拉伯语数学推理的人工构建基准，覆盖五种主要阿拉伯方言及现代标准阿拉伯语，含纯翻译与翻译加文化本地化两个平行版本。在 11 个阿拉伯语及多语言大语言模型上评测显示，本地化版本性能普遍下降，且阿拉伯语中心模型对本地化的敏感度各不相同。机制可解释性分析发现，同一题目的翻译版与本地化版常引发不同的内部推理轨迹，即使总体准确率相近。
正文 · 原文
Abstract:Mathematical reasoning remains underexplored for low-resource languages, including Dialectal Arabic. To bridge this gap, we introduce the DAR-MGSM (Dialectal Arabic Multilingual Grade School Math) Benchmark, the first manually curated benchmark for evaluating mathematical reasoning in Dialectal Arabic. DAR-MGSM consists of math word problems (MWPs) requiring multi-step reasoning and covers five major Arabic dialects alongside Modern Standard Arabic (MSA). The benchmark comprises two parallel versions: translated-only and translated & culturally localized. The latter version adapts entities and references to regional varieties. We evaluate eleven Arabic-centric and multilingual large language models on DAR-MGSM. The results show that the performance generally declines on the localized version, highlighting the challenges introduced by culturally grounded adaptations. However, the magnitude of this effect varies across models, with Arabic-centric models exhibiting different levels of sensitivity to localization. A mechanistic interpretability analysis shows that translated and localized variants of the same MWPs often induce distinct internal reasoning trajectories, even when aggregate accuracy remains similar. Overall, our results show that mathematical reasoning performance depends on both linguistic variety and cultural context.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.28607 [cs.CL] |
| (or arXiv:2610.28607v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.28607 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Noor Abo Mokh [view email]
[v1]
Tue, 20 Oct 2026 17:50:13 UTC (843 KB)
来源：arXiv cs.CL · arxiv.org