跳到正文
arXiv cs.AI· Guillem Ram\'irez·· 10 小时前AI 评分26

SALT:用片段级监督改进跨语言 Token 表示

Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

AI 导读

研究者提出轻量级后训练方法 SALT,通过向现有句子编码器注入片段级监督来改进 Token 表示。在五个多语言 Token 级基准上,SALT 在其中四个取得最佳整体结果,优于其他微调策略和竞争编码器。该方法还提升了跨语言检索与分类的句子级表现,说明片段级监督能同时改善 Token 与句子表示。

正文 · 原文

View PDF HTML (experimental)

Abstract:Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2609.09953 [cs.CL]
  (or arXiv:2609.09953v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.09953

arXiv-issued DOI via DataCite

Submission history

From: Guillem Ramírez [view email]
[v1] Wed, 9 Sep 2026 09:43:41 UTC (487 KB)
[v2] Wed, 18 Nov 2026 17:40:23 UTC (2,141 KB)

来源:arXiv cs.AI · arxiv.org