SP
BravenNow
Diffusion Generative Recommendation with Continuous Tokens
| USA | technology | ✓ Verified - arxiv.org

Diffusion Generative Recommendation with Continuous Tokens

#Continuous tokens #Recommender systems #Large language models #Diffusion models #Information retrieval #ACM Web Conference #ContRec #Vector quantization

📌 Key Takeaways

  • Researchers developed ContRec framework integrating continuous tokens into LLM-based recommender systems
  • ContRec consists of sigma-VAE Tokenizer and Dispersive Diffusion module components
  • The framework outperforms traditional and state-of-the-art LLM-based recommender systems
  • Continuous tokenization addresses limitations of quantization methods in existing approaches

📖 Full Retelling

Researchers Haohao Qu, Shanru Lin, Yujuan Ding, Yiqi Wang, and Wenqi Fan developed ContRec, a novel framework that integrates continuous tokens into large language model-based recommender systems in a paper submitted on April 16, 2025 and accepted by The ACM Web Conference (WWW 2026), addressing limitations in existing approaches that use discrete vector-quantized tokenizers which result in lossy tokenization and suboptimal learning due to inaccurate gradient propagation. The ContRec framework consists of two key components: a sigma-VAE Tokenizer that encodes users and items with continuous tokens, and a Dispersive Diffusion module that captures implicit user preferences through a conditional diffusion process with a novel Dispersive Loss. By conditioning on previously generated tokens of the LLM backbone during user modeling, the system enables high-quality user preference generation through next-token diffusion while avoiding representation collapse through three effective techniques. The researchers demonstrated that ContRec consistently outperforms both traditional and state-of-the-art LLM-based recommender systems through extensive experiments on four datasets, delivering comprehensive recommendation results by leveraging both textual reasoning from the LLM and latent representations from the diffusion model for Top-K item retrieval.

🏷️ Themes

Artificial Intelligence, Information Retrieval, Recommender Systems

📚 Related People & Topics

Recommender system

System to predict users' preferences

A recommender system, also called a recommendation algorithm, recommendation engine, or recommendation platform, is a type of information filtering system that suggests items most relevant to a particular user. The value of these systems becomes particularly evident in scenarios where users must sel...

View Profile → Wikipedia ↗

Information retrieval

Finding information for an information need

Information retrieval (IR) in computing and information science is the task of identifying and retrieving information system resources that are relevant to an information need. The information need can be specified in the form of a search query. In the case of document retrieval, queries can be base...

View Profile → Wikipedia ↗

Diffusion model

Technique for the generative modeling of a continuous probability distribution

In machine learning, diffusion models, also known as diffusion-based generative models or score-based generative models, are a class of latent variable generative models. A diffusion model consists of two major components: the forward diffusion process, and the reverse sampling process. The goal of ...

View Profile → Wikipedia ↗

Large language model

Type of machine learning model

A large language model (LLM) is a language model trained with self-supervised machine learning on a vast amount of text, designed for natural language processing tasks, especially language generation. The largest and most capable LLMs are generative pre-trained transformers (GPTs) that provide the c...

View Profile → Wikipedia ↗

Entity Intersection Graph

Connections for Recommender system:

🌐 Machine learning 1 shared
🌐 Graph neural network 1 shared
🌐 Information retrieval 1 shared
🌐 Artificial intelligence 1 shared
View full profile
Original Source
--> Computer Science > Information Retrieval arXiv:2504.12007 [Submitted on 16 Apr 2025 ( v1 ), last revised 24 Feb 2026 (this version, v5)] Title: Diffusion Generative Recommendation with Continuous Tokens Authors: Haohao Qu , Shanru Lin , Yujuan Ding , Yiqi Wang , Wenqi Fan View a PDF of the paper titled Diffusion Generative Recommendation with Continuous Tokens, by Haohao Qu and 4 other authors View PDF HTML Abstract: Recent advances in generative artificial intelligence, particularly large language models , have opened new opportunities for enhancing recommender systems . Most existing LLM-based RecSys approaches operate in a discrete space, using vector-quantized tokenizers to align with the inherent discrete nature of language models. However, these quantization methods often result in lossy tokenization and suboptimal learning, primarily due to inaccurate gradient propagation caused by the non-differentiable argmin operation in standard vector quantization. Inspired by the emerging trend of embracing continuous tokens in language models, we propose ContRec, a novel framework that seamlessly integrates continuous tokens into LLM-based RecSys. Specifically, ContRec consists of two key modules: a sigma-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference. The tokenizer is trained with a continuous Variational Auto-Encoder objective, where three effective techniques are adopted to avoid representation collapse. By conditioning on the previously generated tokens of the LLM backbone during user modeling, the Dispersive Diffusion module performs a conditional diffusion process with a novel Dispersive Loss, enabling high-quality user preference generation through next-token diffusion. Finally, ContRec leverages both the textual reasoning output from the LLM and the latent representations produced by the diffusion model for Top-K item retrieval, thereby delivering comprehensive reco...
Read full article at source

Source

arxiv.org

More from USA

News from Other Countries

🇬🇧 United Kingdom

🇺🇦 Ukraine