arXiv cs.CL· Yuchuan Weng, Chi Liu, Xin Chen, Srinivasan Manoharan·· 13 小时前AI 评分52
自研 Jev 式类型化判定：单次前向传播，从内部判官到开源 LoRA 模型
Our Own Jev: Typed Decisions in One Forward Pass, from an In-House Judge to Open LoRA Models
AI 导读
论文报告了 Jev 式类型化判定的两种用法：一次前向传播读取模型 logits 或隐状态上的小型预测头，直接为每个选项输出概率，无需生成并解析文本。
正文 · 原文
Abstract:A typed decision maps evidence, a question and a fixed set of options to a probability for every option, computed in one forward pass, by reading a language model's logits or a small head over its hidden states, instead of generating and parsing text. The commercial system Jev popularised this System~1 interface, and open reproductions have followed. We report two uses of the pattern. \emph{Part I} describes an in-house System~1 judge that triages about 148K agentic coding session records for a training corpus. It replaces a frontier LLM judge that cost on the order of \$1,000 per pass and whose stored labels matched a fresh two-judge consensus on only 26.9\% of the sessions where the two judges agreed; a self-hosted frozen backbone with a rank-16 LoRA adapter and a pointer head answers three typed questions in one forward pass, judges 1,000 sessions in about three minutes of one GPU, about \$0.06 at a reference rate and roughly 1\% of the frontier judge's cost, and keeps every session inside the organisation. \emph{Part II} asks whether a readout that adds no parameters can be competitive and whether training one helps. Applying the public Eikos recipe and corpus of about 25K decisions to three further backbones (1B, 2B and 35B), we read the frozen output layer of a LoRA-adapted decoder. On the 111 public hard items of JevBench our single-seed 35B, developed with those items in view, scores 79.3\%, not significantly different from the commercial reference (73.0\%, unadjusted $p=0.12$) and ahead of five open systems after Holm--Bonferroni correction; our 2B is ahead of all four kev checkpoints on the leaderboard. A single-seed ablation finds no detectable benefit from training the 26 letter rows of the output layer, and a retrain on a later corpus release moves the 2B's hard-tier accuracy by 11.7 points, which shows how little small evaluations resolve.
| Comments: | 22 pages |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.29052 [cs.CL] |
| (or arXiv:2610.29052v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.29052 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chi Liu [view email]
[v1]
Wed, 21 Oct 2026 01:28:58 UTC (482 KB)
来源：arXiv cs.CL · arxiv.org