UniCache: Task- and Type-Aware KV Cache Compression for Unified Multimodal Models
UniCache compresses multimodal KV caches up to 5× with little quality loss and higher throughput.
UniCache is a training-free KV cache compression method for unified multimodal models that handle understanding, generation, and editing. Offline calibration selects cache segments and compression policies per task and cache type, then allocates a shared budget with attention-guided and temporal scheduling. The authors report 5× compression for understanding and editing and 2.5× for generation with negligible quality loss, plus up to 1.78× higher throughput in long-context settings.
- Training-free UniCache assigns task- and type-specific KV compression policies.
- Reaches 5× compression for understanding and editing, and 2.5× for generation.
- Long-context throughput rises by up to 1.78× with negligible quality loss.
From the original article
1 min read at arxiv.orgUnified multimodal models combine understanding, generation, and editing within a single network, offering a promising foundation for versatile multimodal applications. However, growing multimodal contexts make KV cache storage and access increasingly costly.
By Wanqi Yang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.