arXiv cs.CL· Andr\'es Daniel Godoy Ortiz·· 13 小时前AI 评分31
论文提出「意义契约」：大语言模型中的局部语义几何
Meaning Contracts: Local Semantic Geometry in Large Language Models
AI 导读
一篇 arXiv 论文提出「意义契约」框架，用跨语言探针和细化探针刻画大语言模型概念内部的局部语义几何，在 Gemma 4、Llama 3.2、Qwen 2.5、Pythia 四个架构各异的 decoder-only 模型上验证。研究发现，均值池化下移除主方差方向后，句对可按语义相关度分层排序且层带间无重叠；语义因子围绕不变核心组织，细化会使表征定向位移进入更紧致的邻域。
正文 · 原文
Abstract:This paper studies the local geometry of semantic meaning in large language models. Current representation analyses ask whether meanings are close in embedding space, or whether linear probes separate semantic classes, but they do not characterize the within-concept local geometry: what is invariant across realizations of a single semantic factor, what varies, and what changes when meaning is refined. This paper characterizes that geometry through two controlled probes evaluated on four architecturally diverse decoder-only LLMs (Gemma 4, Llama 3.2, Qwen 2.5, Pythia). A cross-lingual probe uses language as a perturbation to separate exact meaning matches, same-factor realizations, nearby concepts, and out-of-domain contrasts. A refinement probe separates specificity-bearing semantic narrowing from high-probability attributive elaboration and adversarial controls; both probes are validated by blind human judgments. Four findings recur across all models: under mean pooling, once dominant variance directions are removed, sentence pairs order by semantic relatedness (same factor, then same domain, then different domain) without overlap over a band of layers; semantic factors organize around an invariant core surrounded by structured realization-specific variation; refinement produces directed displacement into tighter neighborhoods, including against controls built on the same base noun; and sentence-level and token-local readouts expose complementary geometry (volume contraction at the sentence level, sharper anisotropic endpoints at token-local readout). We interpret these results as evidence that LLM semantics are organized as local anisotropic neighborhoods in which meaning contracts under refinement.
| Comments: | 24 pages, 8 figures. Code and data: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7; I.2.4 |
| Cite as: | arXiv:2610.30715 [cs.CL] |
| (or arXiv:2610.30715v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.30715 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Andres Godoy Ortiz [view email]
[v1]
Thu, 22 Oct 2026 04:32:30 UTC (8,880 KB)
来源：arXiv cs.CL · arxiv.org