CECAR cache-aware routing speeds on-device Qwen3-30B-A3B inference
Authors report 12.5 tokens/s on a consumer GPU with 16 GB VRAM, a 4.5x speedup, with non-resident experts fetched from SSD.
1 stories, newest first · one card per story
Authors report 12.5 tokens/s on a consumer GPU with 16 GB VRAM, a 4.5x speedup, with non-resident experts fetched from SSD.
Today 2.4K readers · 2.6K views · 140 reads · all time 206K views