arXiv cs.CL· Adam Hollowell·· 14 小时前AI 评分31
Gemma 2 情绪表征几何研究：跨规模、深度与指令微调的测量
The Geometry of Emotion Representations Across Scale, Depth, and Instruction Tuning
AI 导读
研究测量了 Gemma 2 家族六个 LLM（2B、9B、27B 的 base 与指令微调版）的情绪几何，发现所有模型单元都具备核心情绪几何结构，模型间线性映射在十五对模型中仅失败一次、中位排名均为 1。
正文 · 原文
Abstract:We measure the emotion geometry of six LLMs spanning Google's Gemma 2 family. Using an extraction pipeline established by Sofroniew et al. (2026) on Claude Sonnet 4.5 and replicated by Hollowell (2026) on Gemma 2 27B base, we compare emotion geometry across base and instruction-tuned variants of Gemma 2 2B, 9B, and 27B. First, all six model cells exhibit the core emotion geometry. A linear map fit between two cells places held-out emotions nearest their own vectors in the target model. Across all fifteen model pairs, fit in one direction each, this failed exactly once, and the median rank was 1 in every pair. Linear maps refit on cleaned embedding-table vectors recover every held-out emotion at rank 1 in all thirty ordered pairs. Second, scale spreads the geometry's variance across more directions without weakening its tie to valence. The leading component's share of variance falls from 31.6 percent (2B) to 29.8 (9B) to 26.7 (27B) in the base models and from 31.9 (2B) to 29.2 (9B) to 27.3 (27B) in the tuned ones, while its correlation with human valence ratings stays between .71 and .78 in all six models. Third, we project each emotion vector onto the output vocabulary and ask at what depth the vector's own emotion word enters the top 10,000 of the 256,000 tokens. In the 2B and 9B models the median emotion word is there from layer 0, while in the 27B models it begins near chance, stays outside the top 10,000 for half the network, and enters at layer 23 in one step. Fourth, tuning leaves every registered analysis-layer measure within its floor. At every size, valence correlation rises and arousal correlation falls by amounts within their floors, the latter against prediction. PC2's variance share also falls, with a pooled decrease significant after correction. Mid-late layer stability exceeds its floor and traces to base 27B's final layer. Code, data, and audit reports are released.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.35411 [cs.CL] |
| (or arXiv:2610.35411v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35411 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Adam Hollowell [view email]
[v1]
Mon, 26 Oct 2026 13:23:07 UTC (350 KB)
来源：arXiv cs.CL · arxiv.org