LEV deepens decision models by looping layers instead of tokens
Original title: LEV: Thinking Fast and Deep with Looped Decision Models
A 2.6B LEV rose from 63.2% to 76.2% on JevBench between depth one and depth four.
LEV is a decision model that answers typed questions by rerunning pretrained looped layers rather than generating tokens. One checkpoint can answer at any depth, and accuracy rises with depth on the same weights, mostly on hard questions. A small stopper can set depth per question, cutting 15% to 24% of computation for under one accuracy point. On a locked test set, 1.4B and 3B LEV models beat same-latency dense models and cascades by 5.4 to 8.8 points below 50 ms.
- The 2.6B model scored 63.2% at depth one and 76.2% at depth four.
- On-demand depth saved 15% to 24% of computation.
- That saving cost under one accuracy point.
- 1.4B and 3B models gained 5.4 to 8.8 points versus dense models.
- Comparisons used latency below 50 ms.
From the original article
1 min read at huggingface.co
Decision models answer typed questions about an input with one probability per option, computed in a single forward pass without generating text. Like the fast System 1 of dual-process theory, they answer at once, but their depth is fixed at training time, so they spend the same computation on easy and hard questions.
By Dingjie Song, Zhiling Yan, Yuxuan Zhang · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.