Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation
A distillation framework compresses LLM reasoning into a 15.5M-parameter trade-up recommendation model reaching AUC 0.941 with product-type test-time training.
The paper targets trade-up recommendation, which identifies higher-quality alternatives that preserve customer purchase intent. A retrieval-augmented few-shot LLM teacher generates labels and rationales that supervise a compact embedding-pair classifier; at inference the 15.5M-parameter student uses only two precomputed 768-dimensional embeddings with no LLM calls. On 8,352 annotated pairs, label-only training scored AUC 0.912, reasoning distillation reached 0.924, and product-type test-time training lifted it to 0.941 with average precision 0.940. The distilled student is roughly 5,000x faster and 10,000x cheaper than direct LLM inference on a 100K-pair proxy catalog.
- Two-level framework distills LLM teacher rationales into a non-generative student classifier.
- Reasoning distillation lifts AUC from 0.912 to 0.924 on 8,352 annotated pairs.
- Product-type test-time training raises AUC to 0.941 and average precision to 0.940.
- Student runs about 5,000x faster and 10,000x cheaper than direct LLM inference.
From the original article
1 min read at arxiv.orgTrade-up recommendation identifies higher-quality alternatives that preserve a customer's purchase intent while offering upgraded benefits. Large language models (LLMs) can reason about such distinctions, but applying them directly to hundreds of millions of product pairs is operationally impractical.
By Siliang Liu · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.