arXiv cs.AI· Kriste Krstovski, Sharyn O'Halloran, Kshitij Chandna·· 4 小时前AI 评分20
hETM:在嵌入空间建模文档与章节层级的层次化主题模型
Modeling Document and Section Hierarchies in Embedding Space
AI 导读
研究者提出层次化嵌入主题模型 hETM,通过嵌入空间中的层次先验,将文档的主题分布与其各章节的主题分布耦合起来,并推导了摊销变分推断流程。在多个领域的结构化文档集合上,hETM 的留出困惑度在扁平与层次主题模型基线中达到最低或接近最低,主题质量相当或更高,并在多数集合的下游聚类任务中取得最佳表现。
正文 · 原文
Abstract:Embedded topic models integrate word embeddings into the topic generation process, producing coherent topics and robust performance on large vocabularies. However, each document is represented by a single topic distribution, ignoring its internal structure. Many real world collections, including legal documents, company and government reports, and online discussion forums, exhibit hierarchical organization in which documents consist of multiple sections that often address distinct subtopics. We introduce the Hierarchical Embedded Topic Model (hETM), a structured topic model that couples the topic distribution of a document and those of its sections through a hierarchical prior in the embedding space. We derive an amortized variational inference procedure for hETM and evaluate it on structured collections from each of these domains. Experimental results show that hETM achieves the lowest or near-lowest held-out perplexity among flat and hierarchical topic model baselines, along with comparable or higher topic quality. It also performs strongly on a downstream clustering task, achieving the best clustering performance on most of the collections.
| Subjects: | Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.21980 [stat.ML] |
| (or arXiv:2611.21980v1 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Kriste Krstovski [view email]
[v1]
Mon, 16 Nov 2026 23:09:30 UTC (108 KB)
来源:arXiv cs.AI · arxiv.org