arXiv cs.CL· Marco De Negri, Luca Sartori, Lorenzo Baggi, Moritz Reihs, Eduard Frank \v{D}urech, Mennatallah El-Assady, Valentina Pyatkin, Anna Hedstr\"om·· 2 天前AI 评分31
检测不等于预警:评估语言模型失败上的 token 级警报
Detection Is Not Early Warning: Evaluating Token-Level Alarms on Language Model Failures
AI 导读
研究者提出 PREWARN 评估协议,针对带标注起始点(onset)的模型失败,按应用选定错误率校准各监控器阈值,并相对 onset 对预警、检测、分类三项任务打分。
正文 · 原文
Abstract:Many failures of language models begin at a specific token, the onset. A monitor that reads the model during generation can classify the finished response, detect the failure once it is under way, or warn before it begins, and only a warning leaves time to stop or steer the generation. Monitors are usually evaluated with response-level metrics such as AUROC, which ignore when the monitor fires. We introduce PREWARN, an evaluation protocol for any failure with an annotated onset. It calibrates each monitor's threshold on validation data to an error rate chosen for the application, runs the monitor, and scores its alarms relative to the onset on all three tasks. As a case study we apply it to repetitive degeneration in three open language models, with early warning as the main task, using linear probes on activations and six untrained baselines. Response-level metrics saturate on this failure, and a rule that reads only the response length scores almost perfectly, while probes that AUROC ranks as equal warn very differently under PREWARN. Probes warn before the onset on most degenerate responses of two of the three models, low-rank adapters below the probe more than double the share of tokens just before the onset that it flags, largely by flagging degenerate responses long before the onset, and no untrained baseline warns much more often than random alarms. We release the probes with calibrated thresholds at this https URL
| Comments: | Accepted at the NeurIPS 2026 Workshop on Interpretability as a Science. 30 pages (10 main text), 6 figures, 16 tables. Code: this https URL |
| Subjects: | Machine Learning (cs.LG); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.16803 [cs.LG] |
| (or arXiv:2611.16803v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16803 arXiv-issued DOI via DataCite |
Submission history
From: Marco De Negri [view email]
[v1]
Thu, 12 Nov 2026 13:43:19 UTC (454 KB)
[v2]
Fri, 13 Nov 2026 07:07:45 UTC (454 KB)
来源:arXiv cs.CL · arxiv.org