arXiv cs.CL· Lijia Liu, Kyohei Atarashi, Jiyi Li, Koh Takeuchi, Shunji Umetani, Hisashi Kashima·· 12 小时前AI 评分29
NLData2Opt：评估 LLM 从自然语言与数据建模优化问题的基准
NLData2Opt: Benchmarking Parameter Derivation and Prediction for Modeling Optimization Problems from Natural Language and Data
AI 导读
研究者提出 NLData2Opt 基准，用于评估 LLM 在真实优化建模流程中的多阶段推理能力，要求模型从含误导属性的数据中提取参数、通过变换已有属性计算缺失参数，并从历史观测预测未来值。结果显示，模型在该基准上的表现较现有基准大幅下降，说明基准分数的提升并不能直接转化为实际优化工作流中的建模准确率。该基准已公开。
正文 · 原文
Abstract:Formulating optimization problems from raw inputs of problem descriptions and data is time-consuming and costly. While Large Language Models (LLMs) offer a promising way to automate this process, existing benchmarks overestimate their capabilities by relying on idealized inputs where all data is relevant and complete. This diverges from reality, where data-centric reasoning is required to extract, compute, and predict parameters from noisy sources. To address this, we introduce NLData2Opt, a benchmark to evaluate LLMs on the multi-stage reasoning inherent in practical optimization workflows. NLData2Opt decomposes problems into problem descriptions and data, and requires models to (1) extract relevant parameters from data containing misleading attributes, (2) compute missing parameters derived by transforming existing data attributes, and (3) predict unavailable future values from historical observations. Evaluation results show substantial performance decline relative to existing benchmarks, indicating that benchmark progress does not directly translate to formulation accuracy in practical optimization workflows. NLData2Opt is available at this https URL.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.22889 [cs.CL] |
| (or arXiv:2610.22889v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.22889 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lijia Liu [view email]
[v1]
Fri, 16 Oct 2026 03:43:00 UTC (1,786 KB)
来源：arXiv cs.CL · arxiv.org