Library of Layers dynamically orders transformer layers at runtime
LoL uses a single hyperparameter for its global layer library to generalize dense, recurrent, and mixture-of-experts architectures with distinct ordering for different tasks.
1 stories, newest first · one card per story
LoL uses a single hyperparameter for its global layer library to generalize dense, recurrent, and mixture-of-experts architectures with distinct ordering for different tasks.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views