arXiv cs.AI· Guillem Ram\'irez·· 10 小时前AI 评分26
SALT:用片段级监督改进跨语言 Token 表示
Improving Cross-Lingual Token Representations by Adding a Pinch of SALT
AI 导读
研究者提出轻量级后训练方法 SALT,通过向现有句子编码器注入片段级监督来改进 Token 表示。在五个多语言 Token 级基准上,SALT 在其中四个取得最佳整体结果,优于其他微调策略和竞争编码器。该方法还提升了跨语言检索与分类的句子级表现,说明片段级监督能同时改善 Token 与句子表示。
正文 · 原文
Abstract:Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.09953 [cs.CL] |
| (or arXiv:2609.09953v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.09953 arXiv-issued DOI via DataCite |
Submission history
From: Guillem Ramírez [view email]
[v1]
Wed, 9 Sep 2026 09:43:41 UTC (487 KB)
[v2]
Wed, 18 Nov 2026 17:40:23 UTC (2,141 KB)
来源:arXiv cs.AI · arxiv.org