arXiv cs.CL· Shengran Guo, Min Chi, Feifei Qiao, Varad Patwardhan·· 11 小时前AI 评分27
AICG：面向数学简答题自动评分的 AI 感知认知监督框架
Beyond Response Scoring: An AI-aware Cognitive Supervision Framework for Automated Mathematical Short Answer Grading
AI 导读
研究者提出 AICG 双分支多任务框架，用于数学简答题自动评分，在 49,402 条教师评分作答上相较 RoBERTa 回归将 MAE 降低 50.3%，QWK 从 0.798 提升至 0.927。
正文 · 原文
Abstract:Transformer-based language models have achieved remarkable success in automated short answer grading (ASAG). However, current methods in ASAG struggle to comprehend, reason about and evaluate complex mathematical answers involving symbols and formulas. These score-only methods fail to generalize across diverse responses and skill states associated with identical scores. Moreover, they ignore the potential upward bias introduced by generative Artificial Intelligence (AI) usage in assessing students' performance and knowledge mastery. To address these limitations, we propose AICG (AI-aware Cognitive-supervised Grading), a two-branch multi-task framework. The first branch combines RoBERTa-based regression, ordinal, and classification heads for individual-answer grading. The second branch uses an expert-designed Q-matrix to associate responses with assessed skills and jointly predicts G-DINA-derived skill mastery and learning gain. Through the shared encoder, these skill-level targets provide cognitive supervision for answer grading beyond score labels. AICG further uses a Gen-AI detector, trained through supervised contrastive pretraining followed by linear classification, to estimate assistance probabilities and convert them into reliability weights, attenuating unreliable cognitive evidence during learning assessment. We conduct experiments on 49,402 instructor-graded responses spanning 19 academic semesters. Relative to RoBERTa regression, AICG reduces mean absolute error (MAE) by 50.3% and improves quadratic weighted kappa (QWK) from 0.798 to 0.927. Ablations identify Q-matrix-guided cognitive supervision as the primary source of improvement, while AI-aware weighting further reduces overall MAE by 4.2% over the unweighted cognitive model and consistently improves MAE and QWK across the four most recent post-Gen-AI semesters.
| Comments: | 12 pages, including references and appendix |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41215 [cs.CL] |
| (or arXiv:2610.41215v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41215 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shengran Guo [view email]
[v1]
Fri, 30 Oct 2026 03:52:32 UTC (548 KB)
来源：arXiv cs.CL · arxiv.org