arXiv cs.CL· Youngkyung Seo, MiHyeon Kim, Jisoo Lee, Hwijung Ryu·· 11 小时前AI 评分31
SERIAL-7：面向七种表示的可执行验证基准，覆盖机器可验证输出的推导、实现与保持
SERIAL-7: Deriving, Realizing, and Preserving Machine-Verifiable Outputs Across Representations
AI 导读
SERIAL-7 是一个韩英双语基准，用可执行验证框架检验大语言模型在七种表示上的机器可验证生成，包含 177 个经校验的规范产物和 43 条原子表示约束。基准将任务拆为语义推导（Derive）、受限产物生成（Realize）与跨表示保持（Preserve），并支持从表示合规到完整产物有效性的分级验证。
正文 · 原文
Abstract:Large language models increasingly generate structured outputs consumed by programs and downstream systems, where semantic correctness alone does not guarantee reliable execution. We introduce SERIAL-7, a Korean-English benchmark built around an executable verification framework for machine-verifiable generation across seven representations. Using 177 validated canonical artifacts, SERIAL-7 separates semantic derivation (Derive), constrained artifact generation (Realize), and cross-representation preservation (Preserve). Its 43 atomic representation constraints are paired with executable checks over format, structure, and task-specific semantics, enabling progressively stricter verification from representation compliance to full artifact validity. Across ten models, REALIZE-L3 HSR ranges from 42.0% to 93.9%, while Full Compliance exposes additional artifact-level failures beyond correct final answers and valid representations. These results show that reliable structured generation requires explicit, executable verification of how artifacts are realized and preserved across representations.
| Comments: | NeurIPS 2026 Workshop on AI for Verifiable Coding |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41308 [cs.AI] |
| (or arXiv:2610.41308v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41308 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mihyeon Kim [view email]
[v1]
Fri, 30 Oct 2026 06:15:59 UTC (4,645 KB)
来源：arXiv cs.CL · arxiv.org