FLEET: From Logits Entropy to Enhanced Trajectories in Text Generation
FLEET matches repeated sampling accuracy at 3x speed and raises LiveCodeBench Pass@32 from 59.9% to 66.2%.
FLEET replaces memoryless temperature sampling with a memory of sparse trajectories through high-entropy generation states, then infers per-token utility scores that adjust logits. It matches the accuracy of a repeated-sampling baseline with a 3x speedup. Under the same compute budget, LiveCodeBench Pass@32 rises from 59.9% to 66.2% on complex coding tasks. In the evaluated greedy configuration it is deterministic and needs only a single calibration pass.
- FLEET stores sparse high-entropy generation trajectories as memory.
- Trajectories yield per-token utility scores that adjust logits.
- It matches repeated-sampling accuracy with a 3x speedup.
- LiveCodeBench Pass@32 rises from 59.9% to 66.2% under the same budget.
- Greedy mode is deterministic after one calibration pass.
Full article175 words · extracted from huggingface.co · click to collapse
Solutions based on large language models (LLMs) often rely on temperature sampling to improve accuracy and stability by aggregating multiple samples from the completion distribution. However, this memoryless approach is inherently suboptimal: because it lacks awareness of prior generations and their evaluations, it produces an increasing proportion of semantically duplicate answers as more samples are drawn, leading to diminishing returns. To address this limitation, we introduce FLEET, a novel method that integrates a memory mechanism into the generation process. FLEET represents each generation as a sparse trajectory through states whose entropy exceeds a predefined threshold and uses these trajectories to infer per-token utility scores that adjust the logits. Benchmark evaluations demonstrate that FLEET achieves the same accuracy as the repeated sampling baseline, with a 3x speedup, and substantially improves accuracy on complex coding tasks (LiveCodeBench Pass@32 increases from 59.9% to 66.2%) under the same budget. Furthermore, in the greedy-decoding configuration evaluated here, the approach is deterministic and uses a single calibration pass to derive its principal hyperparameters, requiring only minimal modifications to existing LLM pipelines.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.27657