跳到正文
arXiv cs.CL· Th{\'e}otime de la Selle (ISC, HiSoMA, CNRS)·· 10 小时前AI 评分22

从转录到语义语料分析:古语言句子表示的无监督学习

From transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languages

AI 导读

研究用TSDAE和无监督SimCSE两种全无监督策略,将专用token级语言模型适配为语料专属句子编码器,仅需原始句子。在拉丁语和古希腊教父文献的圣经引用识别任务上(2,936句专家核验引用),适配编码器在检测与检索两项任务上均领先多语言、专用、蒸馏及监督微调基线;CSE仅需4–8k句域内原始句子即可在笔记本GPU上数十秒训练达到最优,并能跨作品、跨作者迁移。

正文 · 原文

View PDF

Abstract:Automatic Text Recognition (ATR) now supplies digital humanities with large volumes of unstruc- tured, heterogeneous, and often noisy text in ancient languages. Downstream semantic analyses- text reuse identification, alignment, and semantic search-rely on sentence embeddings, yet ex- isting methods transfer poorly to ancient languages: generic multilingual encoders underperform, specialized language models yield anisotropic representation spaces, and labeled similarity data is scarce. We study two fully unsupervised strategies-TSDAE and unsupervised SimCSE (hence- forth CSE)-that adapt a specialized token-level language model into a corpus-specific sentence encoder using only raw sentences. We validate them on biblical reuse in patristic literature in Latin and Ancient Greek: 2,936 sentences carrying expert-verified reuses in works of Augustine, Jerome, and Athanasius, matched against the Vulgate, the Septuagint, and the Nestle--Aland New Testament. We decompose reuse identification into binary detection and correspondence retrieval, evaluate both tasks separately, and benchmark the adapted encoders against multilingual, specialized, distilled, and supervised fine-tuned baselines, on clean text and on text perturbed by simulated HTR errors and scribal abbreviations. On these test sets, the adapted encoders lead all baselines on both tasks, with complementary profiles: TSDAE leads detection given a large in-domain corpus, while CSE leads retrieval, reaches its optimum with as few as 4--8k raw in-domain sentences-a few tens of sec- onds of training on a laptop GPU-and transfers across works and authors. Under simulated noise, retraining CSE on the perturbed text itself outperforms a clean-trained encoder. Measurements of alignment, uniformity, and anisotropy relate the geometric effect of each strategy to the measured gains. The full pipeline-segmentation, fine-tuning, cross-corpus semantic search-is available to non-specialists through the online tool Paraphrasis.
Subjects: Computation and Language (cs.CL); Digital Libraries (cs.DL); Information Retrieval (cs.IR)
Cite as: arXiv:2607.24542 [cs.CL]
  (or arXiv:2607.24542v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2607.24542

arXiv-issued DOI via DataCite

Submission history

From: Theotime de la Selle [view email] [via CCSD proxy]
[v1] Mon, 27 Jul 2026 15:20:13 UTC (6,202 KB)
[v2] Thu, 19 Nov 2026 13:27:36 UTC (6,763 KB)

来源:arXiv cs.CL · arxiv.org