arXiv cs.CL· Xiaojun Wu, Cehao Yang, Xuhui Jiang, Chengjin Xu, Jia Li, Jian Guo·· 11 小时前AI 评分34
TB-MathModel：首个面向数学建模的智能体化可验证基准
Terminal-style Mathematical Modeling Bench: The First Agentic and Verifiable Benchmark for Mathematical Modeling
AI 导读
TB-MathModel 是首个同时做到智能体化、可验证、终端化的数学建模基准，覆盖 2023–2025 年 18 道真实 CUMCM 与 MCM 赛题。智能体在容器化终端中返回单个 JSON 产物，按最终问题数值核心对照特等奖论文复现结果评分，并采用方向感知单侧评分与两层可行性验证（六类可判定不变量程序门 + 624 条 Lean 4 义务）。
正文 · 原文
Abstract:Mathematical modeling turns an open-ended real-world problem into assumptions, a model, a solution, and a defended recommendation, and its answers matter only if they are both close to a credible reference and physically feasible; existing benchmarks either score solver fragments or grade reports with an LLM judge, leaving those numbers unchecked. We introduce TB-MathModel (the Terminal-style Mathematical Modeling Bench), the first benchmark to make complete contest-style modeling problems agentic, verifiable, and terminal at once, over 18 real CUMCM and MCM problems (2023--2025). Each task ships the verbatim statement, official attachments, and figures in a containerized terminal; the agent returns one JSON artifact, scored on the numeric core of the final question against a reproduction of the Outstanding-award paper. Two design choices distinguish it: direction-aware, one-sided scoring, which never penalises beating the anchor, and two-layer feasibility verification---a programmatic gate over six decidable invariant forms plus 624 Lean~4 obligations---with a hard gate that zeroes infeasible answers. Each task is scored against a panel of human outstanding papers rather than one, so a model must be close to many accepted answers; across seven multimodal entries (three seeds each) the strongest reach a multi-reference O-Eval of 64.58\%, while the \$2.54 open model holds 59.81\% at $1/100$ the cost. TB-MathModel turns the numeric core of modeling into a reproducible, machine-checkable signal for measuring long-horizon modeling competence and for collecting agent trajectories. The source code is available at this https URL.
| Comments: | 20 pages, 4 figures, 14 tables |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.08237 [cs.CL] |
| (or arXiv:2611.08237v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08237 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaojun Wu [view email]
[v1]
Thu, 5 Nov 2026 16:18:35 UTC (902 KB)
来源：arXiv cs.CL · arxiv.org