Distillation aligns item and code spaces for generative recommenders
Original title: Knowledge Distillation across Item and Code Spaces for Parallel Generative Recommendation
Item Distribution Matching lifts average Recall@10 by 15.39%, and Code Marginal Matching lifts NDCG@10 by 8.28%.
The paper targets parallel semantic-ID recommenders that predict every code position at once, without autoregressive prefixes. Teachers supply item-level soft distributions while students generate discrete codes. Item Distribution Matching rebuilds a catalogue item distribution from student code predictions and matches the teacher. Code Marginal Matching projects teacher item probabilities onto each code position and matches those marginals to student heads. Both beat prior methods on the reported benchmarks while keeping parallel decoding.
Unconfirmed: The abstract does not name the benchmark datasets behind the reported gains.
- Existing semantic-ID models are usually trained with hard labels for the next item.
- Both objectives transfer soft teacher knowledge without changing parallel prediction.
- The abstract reports benchmark gains but does not name the datasets.
From the original article
1 min read at arxiv.orgGenerative recommendation with semantic IDs represents each item as a sequence of discrete codes derived from its semantic representation and predicts these codes to generate recommendations. Existing semantic-ID recommenders are typically trained with hard labels for the observed next item.
By Yixuan Xu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.