arXiv cs.AI· Spandan Roy, Subhabrata Majumdar, Soudeep Deb·· 4 小时前AI 评分38
AI 能评判你吗?LLM 辅助招聘系统部署就绪度的实证审计
Can AI Judge You? An Empirical Audit of Deployment Readiness in LLM-assisted Hiring Systems
AI 导读
一项实证审计构建了分离候选人证据、评估、权限控制与评估记录的系统架构,在 210 份简历、14 个岗位和 6 种决策界面上测试了 4 个 LLM。结果显示标量评分比简单通过/不通过更能区分资质变化,比较式判断则对呈现顺序敏感;显式权限规则与简历内容/指令分离可降低未授权遵从,但效果因 LLM 和使用场景而异。研究还发现 LLM 评估者的决策与给出的理由常不一致。
正文 · 原文
Abstract:Delegating resume screening to large language models (LLMs) requires organizations to establish whether they can distinguish candidates by relevant qualifications while allowing only authorized users to change screening rules or approve exceptions. To this end, we develop a system architecture that separates candidate evidence, evaluation, permission controls, and assessment record, and empirically examine two complementary dimensions of its deployment readiness. First, we assess its operational readiness on four LLMs using a controlled experiment on 210 resumes across 14 job roles and six decision interfaces. Second, we assess its compliance readiness by testing how it handles unauthorized user requests and instructions embedded in resumes. We find that operational performance depends on the choice of evaluation interface: scalar scores generally distinguish qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to presentation order. Explicit permission rules and separation of resume content from instructions reduce unauthorized compliance, to a varying degree of effectiveness across LLMs and usage contexts. Finally, we find that LLM evaluators often diverge between their decisions and offered justifications. Going beyond simply evaluating LLMs for hiring tasks, our framework supports an audit of a fully configured resume evaluation system to enable its deployment decision.
| Comments: | 46 pages, 11 figures |
| Subjects: | Computers and Society (cs.CY); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.20140 [cs.CY] |
| (or arXiv:2611.20140v2 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2611.20140 arXiv-issued DOI via DataCite |
Submission history
From: Spandan Roy [view email]
[v1]
Sun, 15 Nov 2026 17:16:09 UTC (5,334 KB)
[v2]
Tue, 17 Nov 2026 04:52:31 UTC (5,334 KB)
来源:arXiv cs.AI · arxiv.org