Scaling law favors full BPTT over truncated looped transformers
Two fully trained loops beat the compute-optimal non-looped model with about 27% fewer non-embedding parameters.
Stories per day · 30 days
1 stories, newest first · one card per story
Stories tagged #training-compute