arXiv cs.CL· Yeonkyoung So, Jongmin Kim, Sungmok Jung, Gyuseong Lee, Sangho Kim, Jongyeon Park, Joonhak Lee, Seho Pyo, Gyeongje Cho, Seorin Kim, Jisoo Kim, Suyoung Park, Hyunji M. Park, Yelim Ahn, Yeongho Seo, Jaejin Lee·· 2 天前AI 评分22
KBS：面向基础模型评估的韩语基准套件
KBS: A Korean Benchmark Suite for Foundation Model Evaluation
AI 导读
研究者发布韩语基准套件 KBS，包含 Ko-ARC、Ko-GSM8K、Ko-EQ-Bench、Ko-WinoGrande、Ko-LAMBADA 和 Ko-IFEval 六项任务，共 9,396 道题，覆盖六种能力，采用审校、本地化、直接构建与重新设计而非纯翻译的方式适配韩语。
正文 · 原文
Authors:Yeonkyoung So, Jongmin Kim, Sungmok Jung, Gyuseong Lee, Sangho Kim, Jongyeon Park, Joonhak Lee, Seho Pyo, Gyeongje Cho, Seorin Kim, Jisoo Kim, Suyoung Park, Hyunji M. Park, Yelim Ahn, Yeongho Seo, Jaejin Lee
Abstract:We present a Korean Benchmark Suite (KBS) comprising Ko-ARC, Ko-GSM8K, Ko-EQ-Bench, Ko-WinoGrande, Ko-LAMBADA, and Ko-IFEval, covering six capabilities across 9,396 items. The suite combines review, localization, direct Korean construction, and redesign for Korean, avoiding a translation-only approach to benchmark adaptation. We evaluate Korean-developed models, multilingual open models, and closed API models under protocols that account for model access, prompt format, and scoring differences. Results show that Korean benchmark scores depend on task construction, scoring rules, prompting, and model post-training. These findings suggest that Korean evaluation should report both final scores and the settings used to obtain them.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.21227 [cs.CL] |
| (or arXiv:2610.21227v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21227 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yeonkyoung So [view email]
[v1]
Thu, 15 Oct 2026 04:18:09 UTC (437 KB)
来源：arXiv cs.CL · arxiv.org