arXiv cs.AI· Junwei Chen, Manjiang Yu, Pengpeng Qiao, Yang Cao·· 14 小时前AI 评分22
面向 LLM 对齐的排序感知列表偏好差分隐私发布方法
Rank-Aware Differentially Private Release of Listwise Preferences for LLM Alignment
AI 导读
研究者提出一种排序感知指数机制，将列表偏好数据在本地差分隐私（LDP）下转化为低敏感度的固定大小二元划分，使噪声集中在边界条目而非像随机化响应那样均匀扰动所有条目。在下游 LLM 对齐任务中，该机制在相同隐私预算下持续优于现有 LDP 基线。代码已公开。
正文 · 原文
Abstract:Listwise preference feedback offers richer supervision for large language model (LLMs) alignment than pairwise comparisons, but a full ranking also reveals more sensitive user preferences. Existing privacy-preserving alignment methods focus mainly on pairwise feedback, while listwise alignment under local differential privacy (LDP) remains unexplored. Extending LDP to this listwise domain raises a critical challenge: the high sensitivity of the full ranking space demands excessive noise, neutralizing the benefits of listwise supervision. To address this challenge, we analyze the interaction between supervision granularity and privacy noise in downstream training, and introduce a rank-aware exponential mechanism that privatizes listwise preference data into a low-sensitivity granularity suitable for downstream alignment. The mechanism leverages ranking information to sample a fixed-size binary partition, concentrating the noise near borderline items instead of perturbing all items uniformly as randomized response (RR) does. Empirical evaluations on downstream LLM alignment tasks show that our mechanism consistently outperforms existing LDP baselines at matched privacy budgets. The code is available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37682 [cs.CL] |
| (or arXiv:2610.37682v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37682 arXiv-issued DOI via DataCite |
Submission history
From: Junwei Chen [view email]
[v1]
Tue, 13 Oct 2026 04:17:15 UTC (508 KB)
来源：arXiv cs.AI · arxiv.org