arXiv cs.CL· Anna Mosolova, Djam\'e Seddah·· 10 天前AI 评分27
多语言 LLM 的日常知识短板：TriviaRoomQA 基准揭示流行文化盲区
When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs
AI 导读
研究者提出多语言基准 TriviaRoomQA，用 288 个主题的 3,300 道平行多选题（六种欧洲语言）加 5,340 道法语题，评测了 30 个 7B 至 70B 参数的开源权重 LLM。结果显示模型在历史、地理、数学等知识密集型主题上表现强劲，但在名人、音乐、电影、新闻等日常流行文化主题上明显偏弱，且同一问题在不同语言下表现不一，说明事实知识获取并非总是与语言无关。
正文 · 原文
Abstract:Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics. We introduce TriviaRoomQA, a multilingual benchmark designed to evaluate everyday, culturally grounded, and long-tail knowledge across 288 topics. The benchmark contains 3,300 parallel multiple-choice questions in six European languages and additional 5,340 French-only questions for a more fine-grained case study. We evaluate 30 open-weight LLMs from European, Asian, and North American providers, covering models from 7 to 70B parameters. We find that models are strong on knowledge-intensive topics such as history, geography, and mathematics, but substantially weaker on everyday popular-culture topics such as celebrities, music, movies, and news. Moreover, model performance varies across languages even for the same underlying questions, suggesting that access to factual knowledge is not always language-independent. In sum, our dataset and experiments demonstrate an important knowledge gap which is not captured by existing academic-based saturated benchmarks.
| Comments: | EMNLP 2026 Findings |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.21445 [cs.CL] |
| (or arXiv:2607.21445v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.21445 arXiv-issued DOI via DataCite |
Submission history
From: Anna Mosolova [view email]
[v1]
Thu, 23 Jul 2026 15:52:00 UTC (41,704 KB)
[v2]
Wed, 7 Oct 2026 16:43:46 UTC (14,228 KB)
来源：arXiv cs.CL · arxiv.org