MicroQonv: Reshaping Convolution Tensors for Efficient Microscaling in Training and Inference
MicroQonv reduces convolutional microscaling cost by quantizing tensors once before a modified im2col.
MicroQonv adapts microscaling quantization to convolutional forward and backward passes by quantizing each tensor only once and quantizing activations before a channel-batch-first im2col. It cuts quantization cost by 2x for weights and gradients and by up to 9x for activations, with negligible accuracy loss, and reduces memory movement and storage by up to 7.53x versus full precision. Reported activation-memory reductions are 3.5x for YOLOv8-nano and 2.2x for YOLOv26-nano. Four-bit microscaling in quantized latent replay for edge continual learning improved accuracy by 5.7 to 11 percentage points.
- Quantizes each tensor once and activations before channel-batch-first im2col
- Cuts quantization cost by 2x for weights and gradients and up to 9x for activations
- Up to 7.53x less memory movement versus full precision
- Activation movement falls 3.5x on YOLOv8-nano and 2.2x on YOLOv26-nano
- 4-bit latent replay improves continual-learning accuracy by 5.7 to 11 points
- AI models
- YOLOv8-nanoYOLOv26-nano
From the original article
1 min read at arxiv.orgMicroscaling quantization techniques are increasingly used to represent neural network parameters with 8 bits or fewer while preserving near-full precision accuracy. However, applying these methods efficiently in convolutional layers is not straightforward.
By Romain Facq · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.