arXiv cs.CL· Mahdi Naser Moghadasi·· 12 小时前AI 评分42
现代文本嵌入模型的几何盲点：语义丰富却对情感度量失明
Semantically Rich, Metrically Affect-Blind: A Geometric Blind Spot in Modern Text Embedders
AI 导读
研究者在 2021 至 2026 年的 11 个冻结编码器（含 4B 参数的 Qwen3-Embedding-4B）上发现系统性分离：线性探针最高以 91.3% 准确率还原句子极性，但余弦相似度对极性的区分边际仅 0.003-0.064。
正文 · 原文
Abstract:We report a systematic dissociation in modern text embedders: sentiment is almost perfectly decodable from their embeddings, yet almost entirely absent from their metric. Across eleven frozen encoders spanning 2021 to 2026, including 2026-generation models and the 4B-parameter LLM embedder Qwen3-Embedding-4B, a linear probe recovers sentence polarity at up to 91.3% accuracy while cosine similarity separates polarity by a margin of only 0.003-0.064. Qwen3-Embedding-4B, the strongest probe in the panel, reaches 91.3% probe accuracy but only 64.3 retrieval MAP. Because retrieval, clustering, deduplication, and retrieval-augmented generation operate on the metric rather than on a trained probe, these systems are effectively blind to information their encoder demonstrably contains. A task-conditioning control suggests this reflects the pretraining objective rather than a prompting artifact: conditioning Jina-v5 on its classification task rather than its retrieval task recovers only about half of the MAP gain that injection achieves on the same weights (63.2 to 73.8, versus 85.1), and leaves the alignment gap essentially unchanged. Sentiment-Injected Sentence Embeddings (SISE) train a small (0.13M-0.69M-parameter) supervised-contrastive head on a frozen encoder in seconds on a CPU, using the labelled training set. The head improves every encoder released since 2022 on full-data probe accuracy (up to +5.8 points), retrieval MAP (up to +26.0) and polarity margin (7-61x). This is a specialization, not a free gain: the compact 128-d head loses 38-41 Spearman points on STS-B, while a fused variant that concatenates it with the base embedding loses 2.4-12.0. With the head, a 0.6B encoder at 128 dimensions matches a 4B encoder given the same head (91.8 vs. 90.3 MAP). We release a CPU-reproducible pipeline and a reusable diagnostic, the Attribute Alignment Gap.
| Comments: | 6 pages, 2 figures, 3 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.32122 [cs.CL] |
| (or arXiv:2610.32122v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32122 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mahdi Naser Moghadasi [view email]
[v1]
Fri, 23 Oct 2026 03:50:48 UTC (113 KB)
来源：arXiv cs.CL · arxiv.org