arXiv cs.CL· Michael C. Frank, Alvin W. M. Tan, Khai Loong Aw, Robert Z. Sparks, Victor Moreno, Grace Keene-Lee, Meesha R. Ryan, Tarun Sepuri, Jane Yang, Virginia A. Marchman, Daniel L. K. Yamins, Bria Long·· 12 小时前AI 评分31
在儿童真实生活视频上扩展跨情境词汇学习
Scaling cross-situational word learning on children's naturalistic experiences
AI 导读
研究团队用包含 48 名儿童、2441 小时纵向第一视角视频的 BabyView 语料训练跨情境词汇学习模型，模型能学到物体与标签的映射。使用互联网预训练的视觉表征时表现远高于仅从发育视频数据学习视觉表征，且大部分学习信号来自少数描述可见指称对象的语句。模型习得的词汇表征呈现可识别的人类式语义结构，但训练过程中的表现轨迹与儿童差异明显。
正文 · 原文
Authors:Michael C. Frank, Alvin W. M. Tan, Khai Loong Aw, Robert Z. Sparks, Victor Moreno, Grace Keene-Lee, Meesha R. Ryan, Tarun Sepuri, Jane Yang, Virginia A. Marchman, Daniel L. K. Yamins, Bria Long
Abstract:How do children learn the meanings of the simple, concrete words that form a majority of their early vocabulary? Statistical information about many word meanings can be recovered from the gradual accumulation of information about the co-occurrence of words with their referents, a proposal known as ``cross-situational learning''. Contrastive vision--language models allow this learning strategy to be tested on egocentric video data capturing children's early real-life experiences with visual and linguistic inputs. Initial successes of this method on small datasets provide a proof-of-concept, but vision models struggle to learn robust visual representations when trained on these same egocentric videos. How dependent is cross-situational word learning performance on the quality of the model's visual representations? And can cross-situational models scale to larger and more diverse datasets? To address these questions, we trained models on the BabyView corpus, including 2,441 hours of longitudinal egocentric video from 48 children. Models trained on this dataset learned object--label mappings, but performance was far higher when they used internet-pretrained visual representations as opposed to when visual representations were learned solely from the developmental video data. Further, the majority of the learning signal was carried by a small proportion of utterances that described a visible referent. Models learned lexical representations that showed recognizable human-like semantic structure, but model performance showed substantially different trajectories over training experiences than those observed in children. Cross-situational learning is a viable word learning strategy for early noun vocabulary, but constructing developmentally plausible models will require understanding how to learn high-quality visual representations from naturalistic data.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.31842 [cs.CL] |
| (or arXiv:2610.31842v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31842 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Michael Frank [view email]
[v1]
Thu, 22 Oct 2026 19:24:51 UTC (2,197 KB)
来源：arXiv cs.CL · arxiv.org