arXiv cs.CL· Emma Graham, Richard Granger·· 11 小时前AI 评分38
用形式语法基准区分 GPT-5.5 Pro 各推理等级
Distinguishing among reasoning levels of GPT 5.5 Pro via a grammar lens benchmark
AI 导读
研究者用源自计算理论的形式语法基准测试 GPT-5.5 及 GPT-5.5-Pro-Medium、-High、-XHigh，四者对测试数据的接受率依次为 16%、40%、43%、52%，显示该基准能分辨模型测试时推理等级的细微差异。随着测试时计算投入增加，模型回答与人类回答的分歧也在扩大，作者认为这种分歧可能成为更强语言模型的普遍特征，并带来能力与对齐之间的潜在冲突。
正文 · 原文
Abstract:Does the level of test-time processing effort affect a language model's computational capability? A formal grammar-lens benchmark method, derived from the theory of computation, generates novel data at specified levels, each successively more computationally powerful than the levels below. Using this data, we tested GPT-5.5, and GPT-5.5-Pro-Medium, -High, and -XHigh, showing incrementally increased acceptance of test data: 16, 40, 43, and 52\%, respectively, illustrating the benchmark's ability to discern fine-grained differences among models' added test-time reasoning levels. Additionally, as test-time compute effort increases, model responses diverge from human responses to the same data. This increasing disagreement may turn out to be broadly characteristic of more capable language models, introducing the potential of a general capability-versus-alignment conflict.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.00555 [cs.CL] |
| (or arXiv:2611.00555v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00555 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Richard Granger [view email]
[v1]
Sat, 31 Oct 2026 02:28:02 UTC (703 KB)
来源：arXiv cs.CL · arxiv.org