How Model Growth, Recursion, and Boundary Operators Influence Scaling Exponents
Researchers show model growth via looped transformers improves scaling exponents; a 7.4B architecture matches GPT-3 13B with roughly 20x less compute.
1 stories, newest first · one card per story
Researchers show model growth via looped transformers improves scaling exponents; a 7.4B architecture matches GPT-3 13B with roughly 20x less compute.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views