arXiv cs.AI· Aizan Zafar, Sreekantam Sai Venkata, Gathik Jindal, Ritu Raj Sharma, Zishan Ahmad, Tulika Saha, Srinath Srinivasa·· 10 小时前AI 评分38
TrustMod-SM：评估 LLM 社交媒体内容审核可信度的多维度基准
TrustMod-SM: A Multi-Axis Benchmark for Evaluating Trustworthiness of LLMs in Social Media Content Moderation
AI 导读
研究者推出 TrustMod-SM 基准，覆盖 8 个数据集约 29,000 条样本，从真实性、公平性、安全性、鲁棒性和上下文完整性五个维度评估 LLM 内容审核可信度。
正文 · 原文
Abstract:Large Language Models (LLMs) show strong potential for social media content moderation, but their trustworthiness beyond overall accuracy remains underexplored. We introduce TrustMod-SM, a benchmark of roughly 29,000 instances across eight datasets, to evaluate LLM moderators on five dimensions: trustfulness, fairness, safety, robustness, and context integrity. Evaluating 16 open-weight models (0.5B-284B parameters, spanning text-only and vision-language architectures) and one closed frontier model, we find persistent trustworthiness gaps. Higher detection accuracy often coincides with greater demographic disparity, and models frequently comply with concealment or exaggeration prompts, assign high confidence to wrong predictions, rarely escalate ambiguous cases, and struggle to separate hate speech from counter-speech or reclaimed language (especially below 8B parameters). Scaling does not uniformly resolve these issues, and no single model excels across all five dimensions.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.12970 [cs.AI] |
| (or arXiv:2611.12970v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12970 arXiv-issued DOI via DataCite |
Submission history
From: Zishan Ahmad [view email]
[v1]
Sat, 7 Nov 2026 16:37:46 UTC (2,016 KB)
来源：arXiv cs.AI · arxiv.org