STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
STEPQuant matches FP32 recurrent-state accuracy at about 6 bits while cutting serving memory.
Stories per day · 30 days
2 RSS
3 stories, newest first · one card per story
Stories tagged #delta-rule
STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization
STEPQuant quantizes delta-rule recurrent states by error lifetime and location, matching FP32 accuracy at 6 bits.
Complex KDA: Understanding and Enhancing the Expressivity of Kimi Delta Attention
Complex KDA extends Kimi Delta Attention so one transition can represent 2D rotations and stronger state tracking.