Looping cuts training FLOPs for uniform diffusion language models
Looped UDLMs match a roughly 0.5-trillion-token baseline with up to 2.8x fewer FLOPs on OpenWebText and 3.3x fewer on FineWeb-Edu.
Stories per day · 30 days
2 RSS
Relateddiffusion-language-models2looping2diffusion-models1dpo1fineweb-edu1generative-models1imagenet1language-models1llm-training1openwebtext1preference-optimization1udlm1weak-to-strong1
9 stories, newest first · one card per story
Stories tagged #training-efficiency
Stochastic DPO graph pruning speeds training about 1.4x
Branch pruning cut training time about 30% versus stock DPO, with slightly lower mean accuracy.
Looped uniform diffusion language models cut training FLOPs
Selective looping matches a non-looped UDLM's validation NLL with up to 3.3x fewer training FLOPs.
Swift trains stronger diffusion models with a frozen weak denoiser
On one ImageNet-256 setup it beat the baseline's best FID with about half the training compute, including guide overhead.
KV4-RL: Unlocking Faster Asynchronous Rollouts for RLVR with INT4 KV Cache Quantization
KV4-RL uses selective INT4 KV quantization to speed Qwen3 GRPO rollouts nearly 2x while keeping benchmark accuracy.
Prism: Dynamic Sparse Attention for Native 2K Joint Video-Audio Generation Model Training
Prism uses dynamic sparse attention to train joint 2K video-audio models 2.5× faster than full attention.
KV-streams for Efficient Compaction in Agentic Reinforcement Learning
KV-streams stream the KV cache through compaction, speeding agentic RL training 2.6 to 5 times without hurting performance.
No Free Efficiency: Revisiting the Trade-off Between Training Efficiency and Model Vulnerability
Efficient training recipes make vision and language models easier to attack and less robust.
ThinkPrior: Zero-Rollout Difficulty Priors for Cold-Start Prompt Selection in RLVR
ThinkPrior builds zero-rollout difficulty priors via an offline verifier-anchored pass, halving silent groups in RLVR and cutting wasted rollouts on Qwen2.5-Math-7B.