arXiv cs.CL· Sandesh Kumar Kafle, Jan Vykopal, Valdemar \v{S}v\'abensk\'y·· 12 小时前AI 评分31
基于评分标准的小语言模型学生报告评估方案
Rubric-Based Assessment of Student Reports Using Small Language Models
AI 导读
一项研究提出基于小语言模型的自动化评估流水线，用于批改网络安全课程学生报告，纯文本评估准确率最高达 97%，富文本评估达 94%，可媲美甚至超越大模型。该方案整合轻量语言模型与视觉语言模型，支持本地部署、保护隐私，在低成本硬件上运行，已开源。研究基于两个学期 105 名学生的数据，论文被 SIGCSE TS 2027 接收。
正文 · 原文
Abstract:Language models can analyze and assess student texts using predefined rubrics and deliver structured feedback, reducing instructors' workload. However, existing solutions often rely on external services that are resource-intensive, costly, and raise privacy concerns, making them unsuitable for constrained environments. This study evaluates an automated assessment pipeline with three key aims: design for low-resource environments, privacy, and focus on a novel context in cybersecurity education. The pipeline integrates lightweight language models for textual evaluation and a vision-language model for image interpretation. Responses from 105 students were collected over two semesters of an undergraduate course during a cybersecurity exercise on incident response. Results across different hardware configurations show that small models in low-resource environments can achieve up to 97% accuracy in text-only and up to 94% in rich-text assessments. This rivals or even outperforms a large model while adding the benefits of locally hosted assessment. Our solution can help educators and practitioners who need grading assistance without replacing human oversight, especially in environments where cloud-based solutions may be impractical. This study contributes an open-source, locally deployable AI-assisted scoring pipeline that runs on affordable hardware and can be extended to various types of student reports in computing education. As a result, it makes automated assessment accessible to a broad range of educators and institutions.
| Comments: | Full paper accepted for the 58th ACM Technical Symposium on Computer Science (SIGCSE TS 2027) |
| Subjects: | Computers and Society (cs.CY); Computation and Language (cs.CL); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2611.01785 [cs.CY] |
| (or arXiv:2611.01785v1 [cs.CY] for this version) | |
| https://doi.org/10.48550/arXiv.2611.01785 arXiv-issued DOI via DataCite (pending registration) |
|
| Related DOI: | https://doi.org/10.1145/3857222.3860169
DOI(s) linking to related resources |
Submission history
From: Sandesh Kumar Kafle [view email]
[v1]
Sun, 1 Nov 2026 10:46:27 UTC (818 KB)
来源：arXiv cs.CL · arxiv.org