arXiv cs.CL· Boris Malashenko, Ivan Jarsky, Egor Soldatov, Vladislav Luzhnov, Sviatoslav Stumpf, Vladislav Shmatkov, Viacheslav Shalamov, Andrey Stankevich, Valeria Efimova·· 10 小时前AI 评分26
StaRSE：用符号编码静态编码器实现紧凑俄语句子嵌入
StaRSE: Compact Russian Sentence Embeddings with a Sign-Coded Static Encoder
AI 导读
StaRSE 是一款紧凑的俄语静态句子编码器，部署时无需 Transformer 推理：token 方向以符号位存储、token 范数单独保存，句子通过对范数缩放的符号方向做均值池化编码。
正文 · 原文
Abstract:Transformer sentence encoders provide strong retrieval quality, but their inference cost becomes a bottleneck when embeddings must be produced on CPUs or refreshed frequently. We introduce StaRSE, a compact Russian static sentence encoder that removes Transformer inference at deployment time: token directions are stored as sign bits, token norms are stored separately, and sentences are encoded by mean pooling over norm-scaled sign directions. StaRSE is initialized from ruBERT-base, fine-tuned on Russian adjacent-sentence and task-shaped pairs, and post-trained with a contrastive stage that directly optimizes the sign-projected token table. On RuMTEB, StaRSE reaches a task mean of $51.16$, $7.98$ points above the strongest included static baseline. The packaged model occupies $11.3$ MiB and has a mean CPU latency of $0.08$ ms per sentence, compared with an estimated $55.7$ MiB 16-bit parameter footprint and $1.43$ ms mean CPU latency for the closest-quality contextual encoder. Ablations examine adjacent-sentence supervision, token norms, and direct sign-projected post-training.
| Comments: | Accepted to ReNeuIR @ SIGIR 2026. 16 pages |
| Subjects: | Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2611.14236 [cs.CL] |
| (or arXiv:2611.14236v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14236 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ivan Jarsky [view email]
[v1]
Tue, 10 Nov 2026 15:47:46 UTC (114 KB)
来源：arXiv cs.CL · arxiv.org