LowRAM compresses KV cache with low-rank keys and amortised values
Test-time value predictors plus cross-layer key compression give about 4× compression and up to 5× decode throughput.
Stories per day · 30 days
6 stories, newest first · one card per story
Stories tagged #low-rank
LoGRA: Scaling LLM Reinforcement Learning with Low-Rank Gradient Sketches
LoGRA cuts LLM reinforcement-learning memory by up to 45.7% using low-rank gradient sketches.
Learning Functional Subspaces for Neural Network Compression
Learnable Subspace Projections compress LLMs end-to-end, beating baselines at 70% compression on Llama-2-7B.
The Low-Rank Structure of VLA Reinforcement Learning
RL post-training of flow-based VLA models produces low-rank updates concentrated in action-expert Timestep Modules.
Not All Objectives Are Born Equal: Priority-Constrained Descent for Hierarchical Multi-Objective Optimization
Priority-Constrained Descent optimizes hierarchical objectives while limiting distortion of the primary gradient.
On the Regularization Landscape for the Linear Recommendation Models
Study shows leading linear recommendation models reduce to nuclear-norm or Frobenius-norm regularization, with two new closed-form low-rank solutions proposed.