arXiv cs.CL· Chung-Wei Victor Yuan·· 11 小时前AI 评分26
Embedding Consistency Regulation：紧凑语言模型中的持久语义路由
Embedding Consistency Regulation for Persistent Semantic Routing in Compact Language Models
AI 导读
研究提出 Embedding Consistency Regulation（ECR），用冻结嵌入模型为每个输入生成短离散前缀，并在训练与推理阶段采用相同前缀构造，在受控前缀分配实验中实现 99.93%±0.12% 的语言路由准确率，而随机、打乱和 dummy 对照组仅接近随机水平（34–36%）。
正文 · 原文
Abstract:Language models are increasingly deployed under tight memory and compute constraints, where semantic routes (e.g., language, tone, or intent) may be assigned before decoding. However, these routes may not remain reflected in generated responses. We study this phenomenon as \emph{persistent semantic routing}. We investigate this property using Embedding Consistency Regulation (ECR) as a minimal probe. ECR derives a short discrete prefix for each input using a frozen embedding model and applies the same prefix construction during both training and inference. In controlled prefix-assignment experiments, ECR achieves 99.93\% $\pm$ 0.12\% language routing accuracy, while random, shuffled, and dummy controls remain near chance (34--36\%). This gap is not reliably captured by likelihood: some controls achieve competitive NLL despite losing routing consistency. We introduce Response Routing Recoverability (RRR), an encoder-independent diagnostic that tests whether assigned routes remain observable after generation. Routing recoverability persists across multilingual settings, unsupervised partitions, reduced precision, and transfer tasks. These findings establish routing persistence as a distinct and measurable property of language models, complementary to task performance and likelihood.
| Comments: | Preprint 19 pages, 6 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2601.00543 [cs.CL] |
| (or arXiv:2601.00543v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2601.00543 arXiv-issued DOI via DataCite |
Submission history
From: Chung-Wei Victor Yuan [view email]
[v1]
Fri, 2 Jan 2026 03:16:24 UTC (10,164 KB)
[v2]
Sat, 31 Oct 2026 16:34:24 UTC (4,573 KB)
来源：arXiv cs.CL · arxiv.org