Paper learns sequence compression via test-time regression
First merged summary.
TTR-Compressor learns sequence compression as end-to-end regression, matching merger quality with 55.6% fewer visual tokens.
Sequence compression matters for efficient language and multimodal models, but pooling and projections underperform. TTR-Compressor treats input tokens as a regression problem learned end-to-end by backpropagation, while coefficients are computed in closed form on the forward pass. Used for block-wise sparse attention and as a visual-token merger in multimodal LLMs, it improves RULER performance over attention-based compressors. It matches MLP merger performance with 55.6% fewer visual tokens and is competitive on GSM8K, MATH-500, and HumanEval.
- Regression coefficients are computed in closed form during the forward pass.
- The regression is learned end-to-end with backpropagation.
- It uses 55.6% fewer visual tokens while matching MLP merger performance.
- It improves RULER results over attention-based compressors.
- Evaluated for sparse attention and multimodal visual-token merging.
Timeline
dated events in the reporting · oldest first- Oct 22, 2026TTR-Compressor paper describes test-time regression compression. · arXiv
What each outlet adds
beyond the shared facts- arXivClosed-form regression compressor and 55.6% visual-token reduction.
- AI models
- LLaDA-8B-InstructDream-7B-Instruct
Coverage
oldest first · one row per article · UTC- · 20d ago
arXiv cs.AI / cs.LG / cs.CLFirst report
Test-time Regression Improves Sequence Compression
TTR-Compressor uses end-to-end learned regression for sequence compression, outperforming pooling and projections in language and multimodal models.
- · 20d ago
Learned Unmasking Policies Improve Diffusion Language Model Efficiency
A 856K-parameter policy trained on trajectory-contraction targets achieves best accuracy-NFE frontier on LLaDA-8B-Instruct.