arXiv cs.AI· Chenhao Hong, Jiakai Xu, Kaushiki Singh, Yusen Zhang, Eugene Wu·· 1 天前AI 评分29
CIRCE 利用约束结构预测 LLM 任务复杂度
CIRCE Divines LLM Task Complexity from Constraint Structure
AI 导读
CIRCE 通过廉价 LLM 构建约束中间表示并确定性编译为约束图，从问题约束结构预测 LLM 能否答对。在七个推理基准（含 SAT、逻辑与数学）和四个 LLM 上，随机初始化的 GNN 编码器即把负对数似然从 0.465 降至 0.423，AUROC 从 0.691 提升至 0.727。该估计可在模型尝试前指导模型路由与测试时算力分配。
正文 · 原文
Abstract:Estimating whether a problem is too complex for a large language model (LLM) before the model attempts it can guide model routing, how to allocate test-time compute, and suggest the need for new tools to aid the agent. While existing estimators rely on the problem text, LLM-annotated task difficulty labels, or signals obtained after a model attempts to solve the problem, such as its hidden states, none explicitly leverage the constraint structure inherent in the problem: the variables to assign and how the constraints over them interact.
We introduce CIRCE, which extracts a problem's structure by using a cheap LLM to construct a Constraint Intermediate Representation that deterministically compiles into the constraint graph. Across seven reasoning benchmarks (including SAT, logic, and math) and four LLMs, even a randomly initialized GNN encoder improves predictions of LLM response correctness, reducing negative log-likelihood from 0.465 to 0.423 and increasing AUROC from 0.691 to 0.727 relative to the strongest baseline for each metric.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.41089 [cs.AI] |
| (or arXiv:2610.41089v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41089 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chenhao Hong [view email]
[v1]
Fri, 30 Oct 2026 00:08:01 UTC (139 KB)
来源：arXiv cs.AI · arxiv.org