arXiv cs.CL· Rui Sun, Peizhao Mei, Xiewei Cheng, Kaizhuo Chen, Taoran Li, Zhiyuan Yu, Varun Chandrasekaran·· 13 小时前AI 评分31
同一答案，不同理由：多语言大模型的推理轨迹级分歧
Same Answer, Different Reasons: Trace-Level Divergence in Multilingual Large Language Models
AI 导读
研究提出轨迹级分歧框架，将多语言大模型的可见推理轨迹拆解为结构相似度、顺序一致性和整体相似度三项指标。在三个推理基准、七种语言和六个前沿 LLM 上，整体嵌入相似度会系统性高估一致性：整体相似度前 25% 的样本中，有 52.1%–67.3% 并不在结构相似度前 25%。双语盲标注显示人类判断更贴近结构相似度，且同语言对照表明分歧程度具有语言特异性。
正文 · 原文
Abstract:Multilingual large language models are increasingly deployed in settings where visible reasoning traces support verification, education, and safety auditing, yet standard evaluations typically measure only final-answer agreement or holistic response similarity. As a result, they establish outcome consistency without determining whether users across languages receive comparable supporting evidence. We introduce a trace-level divergence framework that compares visible reasoning traces as structured sequences, decomposing agreement into structural similarity, order consistency, and holistic similarity, with pair-specific calibration to enable comparison across languages. Across three reasoning benchmarks, seven languages, and six frontier LLMs, the natural baseline---embedding each whole response and comparing the two by cosine---systematically overstates agreement: $52.1\%$--$67.3\%$ of examples in the top quartile of holistic similarity are not in the top quartile of structural similarity. This dissociation is stable across sentence encoders and step segmentations, and blind bilingual annotation studies in two English-anchored language pairs show that human judgments align with structural similarity rather than with the holistic baseline. A same-language control shows that the level of divergence is language-specific: two English traces for paraphrases of the same question agree more closely than an English trace and its counterparts in other languages. Our results show that answer-level and holistic evaluations can substantially overestimate the consistency of multilingual explanatory artifacts, motivating trace-level evaluation for reliable multilingual deployment.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.29092 [cs.AI] |
| (or arXiv:2610.29092v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29092 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rui Sun [view email]
[v1]
Wed, 21 Oct 2026 02:35:06 UTC (2,378 KB)
来源：arXiv cs.CL · arxiv.org