REGER adds inter-item relations to generative recommendation with semantic IDs
Original title: REGER: Modeling inter-item Relations with Semantic IDs for Generative Recommendation
The method pretrains by generating semantic neighbors' IDs and distills a teacher's full next-token distributions along the target path.
REGER is a generative recommendation framework that puts inter-item relations into semantic pretraining, history encoding, and semantic-ID generation. The model is pretrained with generative semantic retrieval so it produces neighbors' semantic IDs, and it mixes global attention with local attention aligned to ID boundaries. A hierarchical preference factorization over the semantic-ID prefix tree supports distillation of a discriminative teacher's full conditional distributions, jointly optimized with cross-entropy. The authors report consistent gains over existing generative recommendation baselines on real-world datasets.
- Random SID embeddings and scratch training do not inherit item semantic relations.
- Pretraining generates semantic neighbors' SIDs to capture similarity.
- Local attention is aligned with semantic-ID boundaries for adjacent transitions.
- Preference distillation covers all legal next tokens at ground-truth prefixes.
- Experiments claim consistent improvements over existing GR baselines.
From the original article
1 min read at arxiv.orgPersonalized recommendation depends on learning semantic and collaborative relations among items from online content and user interactions. Generative recommendation (GR) offers a promising paradigm by autoregressively generating discrete semantic IDs (SIDs) of items.
By Yuxi Huang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.