GAAC losslessly compresses BF16 KV caches about 1.41×
On NVIDIA DGX Spark, 64K-token decode attention sped up 1.19–1.27× and resident KV memory fell about 26%.
50
On NVIDIA DGX Spark, 64K-token decode attention sped up 1.19–1.27× and resident KV memory fell about 26%.
Today 349 readers · 375 views · 292 reads · all time 158K views