Distill Globally, Adapt Locally: Reasoning Distillation and Product-Type Test-Time Training for Scalable Trade-Up Recommendation
A distillation framework compresses LLM reasoning into a 15.5M-parameter trade-up recommendation model reaching AUC 0.941 with product-type test-time training.
The paper targets trade-up recommendation, which identifies higher-quality alternatives that preserve customer purchase intent. A retrieval-augmented few-shot LLM teacher generates labels and rationales that supervise a compact embedding-pair classifier; at inference the 15.5M-parameter student uses only two precomputed 768-dimensional embeddings with no LLM calls. On 8,352 annotated pairs, label-only training scored AUC 0.912, reasoning distillation reached 0.924, and product-type test-time training lifted it to 0.941 with average precision 0.940. The distilled student is roughly 5,000x faster and 10,000x cheaper than direct LLM inference on a 100K-pair proxy catalog.
- Two-level framework distills LLM teacher rationales into a non-generative student classifier.
- Reasoning distillation lifts AUC from 0.912 to 0.924 on 8,352 annotated pairs.
- Product-type test-time training raises AUC to 0.941 and average precision to 0.940.
- Student runs about 5,000x faster and 10,000x cheaper than direct LLM inference.
Full article194 words · extracted from arxiv.org · click to collapse
Trade-up recommendation identifies higher-quality alternatives that preserve a customer's purchase intent while offering upgraded benefits. Large language models (LLMs) can reason about such distinctions, but applying them directly to hundreds of millions of product pairs is operationally impractical. We introduce a two-level framework that distills LLM reasoning into an efficient non-generative student and adapts its decision boundary to product-type-specific trade-up criteria. At Level 1, a retrieval-augmented few-shot LLM teacher generates structured relation labels and natural-language rationales. These rationales supervise a compact embedding-pair classifier through alignment and contrastive objectives; at inference, the student uses only two precomputed 768-dimensional product embeddings, with no LLM calls or text generation. On a fixed human-annotated benchmark of 8,352 pairs, a 15.5M-parameter four-class reasoning-distilled student achieves AUC 0.924 (95% CI [0.918, 0.929]), compared with 0.912 for the four-class label-only student. At Level 2, product-type test-time training (PT-TTT) uses few-shot demonstrations to optimize lightweight category-specific adapters over the frozen student. PT-TTT improves AUC from 0.924 to 0.941 and average precision from 0.920 to 0.940. On a 100K-pair proxy catalog, the distilled student on a single eight-GPU machine is approximately 5,000x faster and 10,000x lower in estimated cost than direct LLM inference.
Text extracted automatically; images, tables and formatting may be missing. Original: https://arxiv.org/abs/2609.05363