arXiv cs.CL· Eunchan Kim, Inwhee Joe·· 11 小时前AI 评分31
PEEL：从检索嵌入中提取本体结构，利用 Matryoshka 前缀粒度对齐
Peeling the Matryoshka: Extracting Ontological Structure from Retrieval Embeddings
AI 导读
研究者提出 PEEL，一种在冻结的 Matryoshka 表示学习（MRL）嵌入上训练的最小探针，用序嵌入目标将层级表示为向量包含关系，再通过最小生成树解码分类 DAG。
正文 · 原文
Abstract:Taxonomy construction is essential for knowledge-intensive NLP yet remains costly: clustering-based methods (i) need careful per-corpus tuning and yield unstable hierarchies, while LLM-based pipelines (ii) depend on expensive task-specific training that does not transfer across domains. We report an empirical finding that bears on both: in Matryoshka Representation Learning (MRL) embeddings, where each leading prefix is itself a usable, coarser vector, prefix length aligns with the coarse-to-fine granularity of gold taxonomies; shuffled- and reverse-dimension controls isolate the alignment to the learned early prefix dimensions. The pattern holds across three architecturally distinct MRL models. To test whether this regularity is usable, we introduce PEEL (Peeling Embeddings to Extract Latent ontologies): a minimal probe over these frozen embeddings, trained once with an order-embedding objective that represents hierarchy as vector containment; a minimum spanning arborescence then decodes taxonomic DAGs. Using only a concept vocabulary (no documents), PEEL recovers taxonomic DAGs with high structural fidelity on two OLLM benchmarks, transferring to a new domain with no target-edge supervision. The end-to-end OLLM pipeline, which jointly discovers concepts and induces structure from document corpora, is reported as a cross-task reference. Pre-trained retrieval embeddings encode a decodable granularity gradient; a minimal probe renders it as explicit parent-child (is-a) structure.
| Comments: | Accepted to EMNLP 2026 (Main Conference). 13 pages, 6 figures, 7 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Information Retrieval (cs.IR); Machine Learning (cs.LG) |
| ACM classes: | I.2.7 |
| Cite as: | arXiv:2610.41785 [cs.CL] |
| (or arXiv:2610.41785v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41785 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Eunchan Kim [view email]
[v1]
Fri, 30 Oct 2026 14:19:53 UTC (1,751 KB)
来源：arXiv cs.CL · arxiv.org