BYOD converts AR models to diffusion LLMs via LoRA
BYOD enables parameter-efficient conversion of Gemma 2 9B, Llama 3.1 8B, Qwen2.5 7B and Ministral 8B in 3.1-4.0 hours using 4.1-5.8% LoRA adapters.
Stories per day · 30 days
2 stories, newest first · one card per story
Stories tagged #efficient-training
Metis-1.6 models reach math benchmarks on low data
Praxis and Logos achieve strong math scores with hybrid Mamba-2 architecture using fraction of training data of comparators.