arXiv cs.CL· Federica Granese, Serena Villata, Charles Bouveyron·· 13 小时前AI 评分22
SentenceETM：结合词汇与上下文证据的句子级主题建模
Sentence-Level Topic Modeling with Lexical and Contextual Evidence
AI 导读
研究者提出神经主题模型 SentenceETM，为每个句子分配主题，同时利用词汇与上下文两类证据：主题既由词嵌入定义的词表分布描述，也由集中在主题特定低维子空间的句子嵌入高斯分布描述。句子主题同时生成其词与嵌入并被精确边缘化，文档主题比例用摊销变分推断，高斯参数以闭式解重估。实验显示其句子级主题分配持续优于现有方法，总统辩论案例研究展示了候选人如何处理相同话题。
正文 · 原文
Abstract:Topic modeling faces a fundamental granularity dilemma: probabilistic topic models assign topics at the word level, whereas transformer-based methods operate at the document level. The latter overlooks that real-world documents typically cover multiple topics, whereas the former relies on individual words, which rarely provide sufficient context to identify them. We introduce SentenceETM, a neural topic model that assigns a topic to each sentence by jointly using lexical and contextual evidence. Each topic is described both by a distribution over the vocabulary, defined through word embeddings, and by a Gaussian over sentence embeddings concentrated on a topic-specific low-dimensional subspace. The topic of a sentence generates both its words and its embedding and is marginalized exactly, so that its posterior combines the two sources of evidence. Document proportions are inferred with amortized variational inference, while the Gaussian parameters are re-estimated in closed form. Experiments show that SentenceETM consistently improves sentence-level topic assignment over state-of-the-art approaches, and a case study of a presidential debate shows how sentence-level topics reveal how candidates handle the same subjects.
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.23388 [cs.CL] |
| (or arXiv:2610.23388v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.23388 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Federica Granese [view email]
[v1]
Fri, 16 Oct 2026 12:48:59 UTC (1,051 KB)
来源：arXiv cs.CL · arxiv.org