arXiv cs.AI· Xianliang Yang, Yapu Zhang, Li Zhao·· 6 天前AI 评分22
GraphDecisionBench：在图表任务上评测 System One 模型
GraphDecisionBench: Benchmarking System One Models on Graph Tasks
AI 导读
研究者提出 GraphDecide，一个与模型无关的基准，通过结构化任务画像、匹配的图-文输入对比和启发式提案控制来诊断图决策性能，并评测了 Jev 等基于选择的模型与语言模型基线，覆盖十四种模型-接口配置。Jev 的结果显示：准确的邻接识别并不保证更广泛的结构正确性，图-文联合输入并不总能提升预测，可行构造也不代表高解质量。代码与汇总结果已公开。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly explored for graph understanding and decision-making, while System One models such as Jev select directly from supplied options. However, the capabilities of System One models on graph-related tasks remain unclear. We introduce GraphDecide, a model-independent benchmark that combines structural task profiles, matched graph-text input contrasts and heuristic-proposal controls to diagnose graph decision performance. We evaluate Jev and related choice-based models alongside language-model baselines, covering fourteen model-interface configurations. Jev's results illustrate the benchmark's central distinctions: accurate adjacency recognition does not guarantee broader structural correctness, joint graph-text input does not consistently improve prediction, and feasible construction does not establish high solution quality. Its task contracts, candidate interfaces and scoring rules support comparison across native selectors and language-model adapters. Code and aggregate results are available at this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.06354 [cs.AI] |
| (or arXiv:2610.06354v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.06354 arXiv-issued DOI via DataCite |
Submission history
From: Xianliang Yang [view email]
[v1]
Mon, 5 Oct 2026 13:54:03 UTC (889 KB)
[v2]
Fri, 9 Oct 2026 11:26:31 UTC (867 KB)
来源：arXiv cs.AI · arxiv.org