arXiv cs.AI· James Sadler, Rotimi Alabi, Kotub Uddin·· 4 小时前AI 评分31
EJBench:基于共轭传热设计活动的工程判断力基准
Bookkeeping Is Not Judgement: EJBench, an Engineering Judgement Benchmark Built from a Conjugate Heat Transfer Design Campaign
AI 导读
研究者发布 EJBench,一个与软件无关的工程判断力基准,考察模型能否从少量已有结果预测设计在新工况下的表现并做出决策。任务源自电池冷板的共轭传热设计活动,答案均取自收敛的仿真结果而非 LLM 评判;十个前沿模型中表现最好的 GPT-6 Astra 得分为 0.71,且模型在答错任务上的置信度与答对时几乎相同。
正文 · 原文
Abstract:We introduce EJBench, a benchmark of engineering judgement: whether a model can predict how a design behaves at new conditions from a few earlier results, and decide on it. Unlike engineering benchmarks that test whether an agent can operate simulation or CAD software, EJBench is software-agnostic and asks what an engineer would conclude without running a simulation. Its tasks come from a conjugate heat transfer design campaign for battery cold plates, in three families: matching unlabelled test results to their operating points, choosing which stored runs to reveal before signing off a design, and ranking new duties by peak cell temperature. Every answer is taken from a converged simulation, not from an LLM judge. The best of ten frontier models, GPT-6 Astra, scores 0.71, and models state almost the same confidence on tasks they get wrong as on tasks they get right. We also post-train Qwen3.5-9B with LoRA and reinforcement learning, rewarded by the share of answer criteria the simulation results confirm; on the held-out test split its score rises from 0.071 to 0.106. We release the train, validation and test splits, the evaluation harness and the leaderboard.
| Comments: | 23 pages |
| Subjects: | Computational Engineering, Finance, and Science (cs.CE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.21707 [cs.CE] |
| (or arXiv:2611.21707v1 [cs.CE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21707 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: James Sadler [view email]
[v1]
Mon, 16 Nov 2026 16:41:07 UTC (2,594 KB)
来源:arXiv cs.AI · arxiv.org