arXiv cs.CL· I\c{s}{\i}l \"Ozg\"u, Yaoxuan Wu, Guy Van den Broeck, Miryung Kim·· 10 天前AI 评分22
SHIM：利用解析器状态实现轻量级约束解码偏差校正
The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding
AI 导读
研究提出 SHIM，一种离线训练的轻量级校正方法，利用 GCD 工具已维护的解析器和词法状态来修正语言模型下一 token 概率，无需改动模型本身。在 bit-vector 和 text-to-SQL 语法上，SHIM 相比掩码和在线采样显著缩小了与模型语法条件分布的差距，速度接近掩码。仅看下一 token 的变体也能优于两种基线，可兼容不暴露解析器状态的 GCD 工具。
正文 · 原文
Abstract:Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, because masking only checks whether each token is valid so far, the resulting distribution over complete outputs diverges from the LM's own distribution conditioned on the grammar, biasing generation toward valid but suboptimal outputs. Online sampling can restore this distribution, but only through costly iterative resampling. Our key insight is that the parser and lexer states that GCD tools already maintain carry a strong signal about future grammatical validity. We introduce SHIM, a lightweight, offline-trained correction of the LM's next-token probabilities, conditioned on this syntactic and lexical state together with candidate next tokens. Since GCD tools already compute these states, SHIM leaves the LM itself untouched. Across bit-vector and text-to-SQL grammars, this correction substantially narrows the gap to the LM's grammar-conditioned distribution compared to masking and online sampling, while running at nearly masking's speed. Even a variant that sees only the next token can improve on both baselines, making SHIM usable with GCD tools that do not expose their parser state.
| Comments: | 10 pages, 5 figures |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2.7; F.4.2 |
| Cite as: | arXiv:2608.10137 [cs.CL] |
| (or arXiv:2608.10137v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2608.10137 arXiv-issued DOI via DataCite |
Submission history
From: Işıl Özgü [view email]
[v1]
Mon, 10 Aug 2026 18:52:43 UTC (233 KB)
[v2]
Wed, 7 Oct 2026 00:39:27 UTC (235 KB)
来源：arXiv cs.CL · arxiv.org