arXiv cs.AI· Qingyun Liu, Bo Yan, Yang Liu, Yuji Roh, Ekansh Sharma, Likang Yin, Emma Olowo, Yuxuan Li, Yikun Jiang, Diego Uribe, Saksham Aggarwal, Siqi Wu, Yuan Hao, Vikas Kedigehalli, Min-hsuan Tsai, Lukasz Heldt, Lichan Hong, Li Wei, Xinyang Yi·· 2 天前AI 评分50
TokenMinds：用预训练用户 Token 与嵌入实现大规模推荐系统的用户理解
TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems
AI 导读
TokenMinds 是面向工业推荐系统的生成式用户建模系统，将 PLUM 框架从基于 SID 的物料检索扩展到用户序列建模，采用改编自预训练 LLM 的编码器-解码器双输出设计：编码器输出稠密用户嵌入，解码器生成基于 SID 的离散用户 token 表示多面向未来兴趣。
正文 · 原文
Authors:Qingyun Liu, Bo Yan, Yang Liu, Yuji Roh, Ekansh Sharma, Likang Yin, Emma Olowo, Yuxuan Li, Yikun Jiang, Diego Uribe, Saksham Aggarwal, Siqi Wu, Yuan Hao, Vikas Kedigehalli, Min-hsuan Tsai, Lukasz Heldt, Lichan Hong, Li Wei, Xinyang Yi
Abstract:User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints due to fixed-dimensional vectors. As an emerging alternative for discrete user representation, LLM-generated natural-language user profiles often capture language semantics rather than sequential patterns in user behaviors. Meanwhile, Semantic ID (SID) is effective for item representation, yet existing discrete user representations remain largely confined to quantizing already-pooled user vectors, clustering historical watches, or predicting coarse topic clusters solely for candidate retrieval. We propose TokenMinds, an industrial system that extends the PLUM framework from SID-based item retrieval to generative sequential user modeling. Built on an encoder-decoder architecture adapted from pretrained LLMs, TokenMinds introduces a dual-output design: the encoder outputs a dense user embedding capturing a holistic user state, while the decoder generates discrete SID-based user tokens representing multi-faceted future interests. Grounded in item semantics, these tokens preserve distinct interests that a single pooled vector would average away. Additionally, a shared SID vocabulary across long- and short-form videos unifies cross-scenario behaviors into a single model, cutting upstream training and serving compute by 50% and 31%. Furthermore, leveraging the text vocabulary inherited from the pretrained LLM, TokenMinds interleaves textual search queries directly with watch histories to capture explicit user intent. We validate TokenMinds through offline evaluations and live A/B experiments across YouTube's primary ranking surfaces, serving billions of users asynchronously: SID-based user tokens are effective on their own, yield additive gains alongside dense embeddings, and together outperform the previous-generation, non-LLM user model in production.
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.25147 [cs.IR] |
| (or arXiv:2606.25147v2 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2606.25147 arXiv-issued DOI via DataCite |
Submission history
From: Qingyun Liu [view email]
[v1]
Tue, 23 Jun 2026 20:20:37 UTC (1,384 KB)
[v2]
Mon, 26 Oct 2026 22:51:11 UTC (1,130 KB)
来源：arXiv cs.AI · arxiv.org