FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation
FLEET matches repeated sampling accuracy at 3x speed and raises LiveCodeBench Pass@32 from 59.9% to 66.2%.
FLEET replaces memoryless temperature sampling with a memory of sparse trajectories through high-entropy generation states, then infers per-token utility scores that adjust logits. It matches the accuracy of a repeated-sampling baseline with a 3x speedup. Under the same compute budget, LiveCodeBench Pass@32 rises from 59.9% to 66.2% on complex coding tasks. In the evaluated greedy configuration it is deterministic and needs only a single calibration pass.
- FLEET stores sparse high-entropy generation trajectories as memory.
- Trajectories yield per-token utility scores that adjust logits.
- It matches repeated-sampling accuracy with a 3x speedup.
- LiveCodeBench Pass@32 rises from 59.9% to 66.2% under the same budget.
- Greedy mode is deterministic after one calibration pass.
From the original article
1 min read at huggingface.co
Solutions based on large language models (LLMs) often rely on temperature sampling to improve accuracy and stability by aggregating multiple samples from the completion distribution.
By Oleksii Streltsov, Oleksandra Vitko · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.