arXiv cs.AI· Pankaj Chouhan, Rajarshi Das, Aditya Dubey, Manminder Tomar, Sreekar Reddy, Samta Sharma, Satyapragnya Kar, PR Panigrahi·· 2 天前AI 评分31
DCBench-440：AI 数据中心建设反事实推理基准
Watts If? A Benchmark for Counterfactual Reasoning About the AI Data Center Buildout
AI 导读
研究者提出 DCBench-440，用于测试 LLM 对 AI 数据中心建设承诺的反事实推理能力，包含 200 道基于 STRIPS 世界的生成题和 240 道基于站点记录与排放公式的选择题。开放权重模型生成题准确率 12.5-71.5%，选择题 58.3-92.5%；在列出决策节点所有可选动作时，无模型精确匹配超过 4.0%，最强模型仅列出 72-85% 会改变项目状态的动作。
正文 · 原文
Abstract:An AI data center is committed years before it draws a watt: a few discrete choices (grid or on-site gas, evaporative or dry cooling, which county to file in) fix its capital, water draw, and carbon path, and LLMs are now asked to help screen them with no test of whether they track what each commitment rules out. We introduce DCBench-440, with two arms reported separately: 200 generative questions over STRIPS worlds for five US deal archetypes, with search-verified gold answers and no tested shortcut above 12% on any question type, and 240 multiple-choice questions grounded in quoted site records, emission equations, and stated scenarios. Across open-weight deployments, accuracy spans 12.5-71.5% (generative) and 58.3-92.5% (multiple choice). One skill fails everywhere: asked to list every action available at a decision gate, no model exceeds 4.0% exact match. The strongest list 72-85% of the actions that would change the project's state but omit most permitted actions that change nothing; with those excused, the best exact match is still 24%. Code, data, and other materials are available upon request.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.21397 [cs.AI] |
| (or arXiv:2610.21397v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21397 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Rajarshi Das [view email]
[v1]
Thu, 15 Oct 2026 07:16:25 UTC (54 KB)
来源：arXiv cs.AI · arxiv.org