Pretraining Latent Information Feedback Transformers with Teacher Supervision
LIFT pretrains Transformers to recycle latent state and beats token-matched baselines from 135M to 1B.
1 stories, newest first · one card per story
LIFT pretrains Transformers to recycle latent state and beats token-matched baselines from 135M to 1B.
Today 1.6K readers · 2.5K views · 779 reads · all time 189K views