arXiv cs.CL· Devon Yanitski, Chris Westbury·· 13 小时前AI 评分18
semanticfa：用于心理测量量表无响应语义因子分析的 R 包
semanticfa: An R Package for Response-Free Semantic Factor Analysis of Psychometric Scales
AI 导读
研究者发布 R 包 semanticfa，将量表题目转为嵌入向量后做语义因子分析，可解释因子、标记冗余题目、构建简版量表并筛选新题。示例用 Qwen3-Embedding-8B 嵌入 50 题 IPIP 大五人格量表，与 874,434 名受访者的因子分析对比，各因子与最佳匹配人类因子的同余系数从 Openness 的 .92 到 Agreeableness 的 .52。
正文 · 原文
Abstract:The items of a reflective psychometric scale can be read as a proposal about a construct's structure. Language models make that proposal measurable by converting each item into an embedding, a list of numbers, similar for similar meanings. We introduce semanticfa, an R package for semantic factor analysis, which factors item-embedding similarities. It also interprets factors, flags redundant items, builds short forms, and screens new items. A worked example analyzes the 50-item International Personality Item Pool Big-Five Factor Markers, embedded with Qwen3-Embedding-8B, and compares the results with a factor analysis of 874,434 respondents. Retention criteria tied between five and six factors, and the tie rule kept five. Three corresponded mainly to single domains, and two blended most Extraversion items with five Agreeableness items. Each factor's congruence with its best-matching human factor (1 = identical) ranged from .92 for Openness to .52 for Agreeableness. Item pairs more similar in meaning tended to be answered more alike (r = .46 with raw responses), but same-domain reverse-keyed opposites, answered oppositely, were about as similar as cross-domain pairs (.58 vs. .57). Sign-flipping reverse-keyed items' embeddings did not improve agreement with raw responses and worsened it with reverse-scored ones. With five factors, a 25-item short form matched the domains better than all 200 random five-per-domain sets, but parallel analysis kept only three. We propose semantic-behavioral coherence, the agreement between semantic and response structures, as a measurable property of a scale, given an embedding model. Semantic factor analysis complements, and does not replace, response-based factor analysis.
| Comments: | 96 pages, 14 figures. Submitted to Behavior Research Methods. Data and code: this https URL |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.37494 [cs.CL] |
| (or arXiv:2610.37494v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37494 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Devon Yanitski [view email]
[v1]
Tue, 27 Oct 2026 16:18:44 UTC (342 KB)
来源：arXiv cs.CL · arxiv.org