FP4 pretraining needs more exposures to learn rare facts
NVFP4 needed 1.32× and 1.52× as many exposures as BF16 to match recall while loss differed by 0.5% and 2%.
Stories per day · 30 days
2 RSS
6 stories, newest first · one card per story
Stories tagged #fp4
BitDecoding-FP4 enables accurate FP4 KV cache decoding
Algorithm-system co-design achieves near-BF16 accuracy with 2.4-3x higher throughput for LLM serving.
Unofficial Qwen3.8-Flash-Next E384 FP4 model trends on Hugging Face
The NAS build keeps 384 of 512 experts, cuts weights about 16%, and was merged after GRPO-style LoRA.
nvidia/Qwen3.8-Flash-Next-FP4-Local — new model trending #30 on Hugging Face
NVIDIA published an FP4-quantized Qwen3.8-Flash-Next checkpoint for local Blackwell inference, occupying about 96 GiB.
TRACE: Rollout-Guided Quantization-Aware Training for FP4 Reinforcement Learning of MoE Language Models
TRACE aligns FP4 train and rollout paths for MoE RL, matching BF16 quality with up to 5.4x rollout speedup.
DeepSeek-v4.1 Flash: Pushing the Limits of KV Cache Compression
DeepSeek-V4.1 Flash is a 552B-parameter multimodal MoE model with 1M-token context achieving 4x KV cache compression for long-horizon agent workloads.
11