跳到正文
arXiv cs.AI· Renhe Sun, Yueying Feng, Bohan Yu, Tong Niu, Yunqi Liu, Zitong Wang, Haolin He, Kexin Shi, Hualei Wang, Qiuqiang Kong, Chang Yao, Yuge Huang, Weiqiang Wang, Jingyuan Chen, Jian Liu·· 4 小时前AI 评分26

EgoLink:ACM MM 2026 第一人称语言-视觉交互网络知识挑战赛总结

EgoLink: Egocentric Language-Vision Interactive Network Knowledge Challenge Summary

AI 导读

EgoLink 挑战赛在 ACM Multimedia 2026 上评测第一人称多模态智能,覆盖自我中心视频社会推理与通过对话、工具调用完成服务任务的交互智能体两条赛道。

正文 · 原文

Authors:Renhe Sun, Yueying Feng, Bohan Yu, Tong Niu, Yunqi Liu, Zitong Wang, Haolin He, Kexin Shi, Hualei Wang, Qiuqiang Kong, Chang Yao, Yuge Huang, Weiqiang Wang, Jingyuan Chen, Jian Liu

View PDF HTML (experimental)

Abstract:The EgoLink Challenge at ACM Multimedia 2026 benchmarks egocentric multimodal intelligence linking first-person perception, social reasoning, and interactive action. Unlike benchmarks centered on visible objects and activities, EgoLink evaluates whether models can interpret emotional and social cues, resolve visually grounded references, and convert user requests into verified tool-mediated actions. Its two tracks cover social reasoning in egocentric video and interactive agents that complete service tasks through dialogue, tool use, and closed-loop execution. Together, they test whether perception, reasoning, and decision-making remain coupled under both offline question answering and state-changing interaction. We highlight a multi-stage Track 1 data pipeline for multiple-choice question (MCQ) generation, chain-of-thought generation, automatic quality grading, and video-contribution filtering. We summarize the task design, evaluation metrics, rules, leaderboards, and representative solutions. The benchmark combines objective answer scoring with process- and result-based evaluation, enabling analysis of both final correctness and execution quality. The best Track 1 system achieved 0.9263 accuracy, while the best Track 2 system achieved 0.844 joint success and 0.916 micro accuracy. Participant reports reveal a shared principle: strong systems make intermediate evidence explicit through candidate verification, option-robust reasoning, scenario-aware planning, tool canonicalization, and state-aware correction before answering or acting. These findings highlight the importance of process verification and evidence-grounded reasoning for reliable egocentric multimodal intelligence.
Comments: Published in ACM Multimedia 2026 (MM '26)
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.21690 [cs.CV]
  (or arXiv:2611.21690v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.21690

arXiv-issued DOI via DataCite (pending registration)

Journal reference: Proceedings of the 34th ACM International Conference on Multimedia (MM '26), pp. 17505-17509, 2026
Related DOI: https://doi.org/10.1145/3767308.3837706

DOI(s) linking to related resources

Submission history

From: Roy Sun [view email]
[v1] Mon, 16 Nov 2026 07:43:02 UTC (12,040 KB)

来源:arXiv cs.AI · arxiv.org