arXiv cs.CL· Chenhui Hu, Pengfei Cao, Kang Liu, Jun Zhao·· 10 小时前AI 评分31
神经网络中的表征原子：用 TSAE 识别基本表征单元
Representational Atoms in Neural Networks
AI 导读
研究正式定义了神经网络的基本表征单元（FRU），并提出保真度与稳定性两项评价标准，证明阈值激活稀疏自编码器（TSAE）在适当条件下能高保真、高稳定地识别 FRU，并通过大规模实验验证。稳定性更高的 FRU 单义性更强，且可跨层组合还原出对应人类概念结构的语义组织。
正文 · 原文
Abstract:Understanding the internal representations of neural networks and characterizing their relationship with human conceptual structure are two critical questions for elucidating model mechanisms. The former reveals how learned knowledge is organized in internal representations, whereas the latter provides a basis for interpreting these representations through human conceptual structure. Here, we formally define fundamental representational units (FRUs) and establish two evaluation criteria: fidelity and stability. We further prove that threshold-activated sparse autoencoders (TSAEs) can identify FRUs with high fidelity and stability under appropriate conditions, and validate this result through large-scale experiments. Further analyses reveal that FRUs with higher stability exhibit greater monosemanticity and can be composed across layers to recover semantic organization corresponding to human conceptual structure. These results provide a unified framework for defining, evaluating and identifying FRUs in neural networks, and establish a foundation for characterizing the relationship between neural network representations and human conceptual structure.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14168 [cs.CL] |
| (or arXiv:2611.14168v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14168 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chenhui Hu [view email]
[v1]
Tue, 10 Nov 2026 14:56:23 UTC (27,134 KB)
来源：arXiv cs.CL · arxiv.org