arXiv cs.CY· Brett Puppart, Kristjan-Julius Laak, Jaan Aru·· 12 小时前AI 评分26
GenAIT：面向高中生的生成式 AI 素养客观测试开发与验证
GenAIT: Development and Validation of an Objective Generative AI Literacy Test for High School Students
AI 导读
研究者开发并验证了 GenAIT——一套 18 道选择题的生成式 AI 素养测试，覆盖技术、实践与人类影响三个领域，用于测量高中生的 GenAI 概念知识。基于 7432 名爱沙尼亚高中生的调查，验证性因子分析、经典测量理论与项目反应理论支持其近似单维性，边际信度 .72、KR-20 = .69，3PL 模型拟合良好（RMSEA = .013、CFI = .990）。
正文 · 原文
Abstract:There is growing international interest in generative AI (GenAI) literacy and its assessment among high school students, but objective assessment in this population remains underdeveloped. This article reports the iterative development and validation of the GenAI Literacy Test (GenAIT), an 18-item multiple-choice test measuring high school students' conceptual knowledge about GenAI, with content spanning technical, practical, and human-impact domains. Expert review of relevance, clarity, and comprehensiveness provided evidence of content validity. In a large-scale survey of 7432 Estonian high school students, we evaluated the psychometric functioning of the Estonian-language GenAIT using confirmatory factor analysis, classical test theory, and item response theory. Results supported approximate unidimensionality, broadly adequate reliability for group-level research (marginal reliability = .72, KR-20 = .69), and good fit of a three-parameter logistic model (RMSEA = .013, TLI = .987, CFI = .990, SRMSR = .021). Measurement precision was sufficient for the majority of students but varied substantially across the latent trait, with lower precision for lower scoring students. GenAIT is therefore more suitable for group-level research than high-stakes individual classification. GenAIT scores were unrelated to perceived usefulness and perceived ease of use, and negatively associated with LLM use frequency, suggesting that frequent use and favorable perceptions of AI should not be treated as proxies for conceptual understanding.
| Subjects: | Computers and Society (cs.CY); Neurons and Cognition (q-bio.NC) |
| Cite as: | arXiv:2608.25815 [cs.CY] |
| (or arXiv:2608.25815v2 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2608.25815 arXiv-issued DOI via DataCite |
Submission history
From: Jaan Aru [view email]
[v1]
Wed, 26 Aug 2026 13:58:56 UTC (756 KB)
[v2]
Fri, 23 Oct 2026 06:31:48 UTC (670 KB)
来源：arXiv cs.CY · arxiv.org