arXiv cs.CY· Zeyu Dong, Mikael Kwame Kuada, Asa Holley Buchakjian-Tweedy, Ibrahem Aljabea·· 12 小时前AI 评分19
LLM 辅导交互对学生成绩的增量预测价值研究
Incremental Predictive Value of LLM Tutoring Interactions for Student Performance
AI 导读
一项基于 StudyChat 的研究评估 LLM 辅导交互特征能否在恒定基线之外提升作业成绩预测，数据含两学期本科 AI 课程的 16,851 条学生-助手对话，匹配 175 份学生学期记录。
正文
Abstract:We evaluate whether semester-level interaction features from a large language model tutor improve assignment-score prediction beyond constant baselines. StudyChat provides 16,851 student-assistant exchanges from two offerings of an undergraduate artificial intelligence course, with outcomes matched to 175 of 203 student-semester records. Six fitted estimators and training-fold mean and median baselines share repeated nested cross-validation. After averaging each student's held-out predictions across ten repeats, Gradient Boosting achieves mean absolute error of 3.023 grade points, versus 3.138 for the median baseline. The paired difference is -0.114, with a conditional 95% bootstrap interval of [-0.245,0.013]. Representation changes and a semester-specific median yield similarly small gains. In the lowest observed-score quartile, however, boosting's error is 7.30 points, versus 8.07 for the median and 6.67 for the mean; Ridge has the lowest fitted-model error at 6.37 despite its worse overall error of 3.331. Scores concentrate near the ceiling: 160 students (91.4%) score above 90, and the lowest quartile contributes 62.1% of boosting's total absolute error. This retrospective comparison shows how baseline choice and error distribution affect predictive value, informing evaluation of student-support tools without establishing a learning effect.
| Comments: | 8 pages, 3 figures |
| Subjects: | Computation and Language (cs.CL); Computers and Society (cs.CY) |
| ACM classes: | K.3.1; I.2.6 |
| Cite as: | arXiv:2610.33488 [cs.CL] |
| (or arXiv:2610.33488v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33488 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zeyu Dong [view email]
[v1]
Sat, 24 Oct 2026 06:34:50 UTC (564 KB)
来源：arXiv cs.CY · arxiv.org