Near-zero embeddings stall gated model training
Untied gated checkpoints contain near-zero embeddings that amplify gradients 30+ orders of magnitude and trigger FP32 clipping, freezing rows in 20% of Telugu pretraining steps.
Stories per day · 30 days
1 stories, newest first · one card per story
Stories tagged #unlocking