跳到正文
arXiv cs.CY· Huy Nguyen, Don\'at \'Akos K\"oller, Levente Murg\'as, J\'ozsef Pint\'er, Marcell Nagy, Kate Barnes, Roland Molontay·· 23 小时前AI 评分24

表情包模板识别研究:半开放集识别方法对比

Meme Template Identification in the Wild: Comparing Methods for Semi-Open-Set Recognition

AI 导读

研究将表情包模板识别形式化为半开放集识别问题,要求方法既能分类已知模板,又能拒识无模板表情包和非表情包内容。在真实社交媒体样本中,仅21%图像使用已知模板,58%无模板,21%并非表情包,最佳MCC从受控设置的0.974降至0.684,性能损失主要来自判断图像是否为模板表情包。

正文 · 原文

View PDF HTML (experimental)

Abstract:Image-with-text memes are a dominant form of online communication, and much of their spread happens through meme templates which are recurring visual formats that users adapt with new text or imagery. Most prior work scores memes individually for engagement or harmful content, an approach that is structurally blind to templates. Templates might amplify these patterns and enable coordinated harassment, which becomes visible only when memes are grouped by shared template. We formalize meme template identification as a semi-open-set recognition problem, requiring methods to both classify known templates and reject template-free memes and non-meme content. We introduce an evaluation framework spanning a controlled setting (1,704 ImgFlip templates) and a heterogeneous real-world social media sample, comparing supervised CNN- and distance-based methods, unsupervised density-based clustering, a novel fused SigLIP2+DINOv2 representation, and a retrieval-augmented LLM pipeline. In real-world social media sources dedicated to meme sharing, we find that only 21% of images use a known template, while 58% are template-free and 21% are not memes at all; identification performance also drops sharply from the controlled setting (best MCC 0.974 to 0.684). The loss comes mainly from deciding whether an image is a template-based meme at all, rather than identifying the template. Our dataset of 1.1M soft-labeled social media images, together with code for reproducibility, is available upon request.
Comments: 18 pages, 4 figures
Subjects: Computers and Society (cs.CY)
ACM classes: I.4.0; I.5.3; J.4; K.4.2
Cite as: arXiv:2408.08126 [cs.CY]
  (or arXiv:2408.08126v3 [cs.CY] for this version)
  https://doi.org/10.48550/arXiv.2408.08126

arXiv-issued DOI via DataCite

Submission history

From: Roland Molontay [view email]
[v1] Thu, 15 Aug 2024 12:52:06 UTC (5,678 KB)
[v2] Mon, 28 Sep 2026 10:00:17 UTC (2,684 KB)
[v3] Wed, 18 Nov 2026 16:52:55 UTC (2,684 KB)

来源:arXiv cs.CY · arxiv.org