arXiv cs.CL· Samra Zafar, Shifa Yousaf·· 11 小时前AI 评分29
学习者如何看待过程感知型 LLM 写作反馈：一项混合方法研究
"I Wrote, I Paused, I Rewrote": A Mixed-Methods Study of Learner Perceptions of Process-Aware LLM Feedback
AI 导读
一项针对 43 名本科英语学习者的混合方法评估显示，基于最终作文、中间快照和击键记录生成的过程感知型 LLM 写作反馈获得较高评价，六个五点 Likert 题项中选择 4 或 5 分的比例在 74.4% 至 95.3% 之间，且经 Holm 校正后六项单侧精确符号检验均显著。
正文 · 原文
Abstract:Large language models increasingly provide formative writing feedback, but most systems evaluate only the submitted text. Process-aware systems can additionally incorporate intermediate drafts and interaction records, yet access to these traces does not establish whether learners perceive the resulting feedback as useful, fair, or aligned with their own revision experience. We conducted a convergent mixed-method post-task evaluation with 43 undergraduate English-language learners. Participants completed a timed essay, received LLM-generated feedback based on the final essay, intermediate snapshots, and keystroke-derived records, and then answered six five-point Likert items and four open-ended questions.
Ratings were concentrated above the neutral midpoint across all six items. Between 74.4% and 95.3% of participants selected ratings of 4 or 5, and all six one-sided exact sign tests remained significant after Holm correction. Perceived correspondence between the revision analysis and the participant's reported thought process was positively associated with overall perceived usefulness, Spearman's $\rho_s=.327$, 95% bootstrap CI $[.009,.609]$, $p=.032$, $n=43$.
Qualitative responses were analysed through an LLM-assisted codebook development procedure followed by independent human coding of 14 binary codes. Participants frequently described the product feedback and revision rationale as accurate, while a smaller set reported misinterpretation of surface corrections, time constraints, or rhetorical intention. The findings distinguish perceived detection of revision activity from perceived interpretation of why revision occurred.
| Comments: | 8 pages, 1 figure, 6 tables |
| Subjects: | Human-Computer Interaction (cs.HC); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.00009 [cs.HC] |
| (or arXiv:2611.00009v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2611.00009 arXiv-issued DOI via DataCite |
Submission history
From: Shifa Yousaf [view email]
[v1]
Fri, 4 Sep 2026 21:31:04 UTC (93 KB)
来源：arXiv cs.CL · arxiv.org