Fathom: Per-Query Read Depth for Sparse Decoding over Offloaded KV Caches
Fathom introduces per-query bit-depth scanning of offloaded 4-bit KV caches, making long-context decoding 1.67x faster on Qwen3-8B at one million tokens.
1 stories, newest first · one card per story
Fathom introduces per-query bit-depth scanning of offloaded 4-bit KV caches, making long-context decoding 1.67x faster on Qwen3-8B at one million tokens.
Today 1.6K readers · 2.4K views · 775 reads · all time 189K views