PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
PreSTO speeds PowerMH LLM sampling by prefetching proposal subtrees without changing the transition kernel.
PreSTO accelerates PowerMH test-time power sampling by batching future Metropolis-Hastings proposals as a prefetched accept/reject subtree, then following the realized path with the original acceptance rule. The MH transition kernel is unchanged. Across seven datasets and five base LLMs, median speedups are 1.71x on PowerMH, 1.91x on EntropyCut, and 2.33x on MultiTryMH. On LCB V6 with Qwen3.5-9B, a predictive traversal rule is a further 1.25x-1.37x faster than breadth-first prefetching.
- PreSTO prefetches Metropolis-Hastings proposal subtrees without changing the kernel.
- Median speedups are 1.71x, 1.91x, and 2.33x versus PowerMH, EntropyCut, and MultiTryMH.
- PowerMH and EntropyCut need a median 3.9x fewer calls, each about 2.1x longer.
- Predictive traversal beats breadth-first by 1.25x-1.37x on LCB V6 with Qwen3.5-9B.
From the original article
1 min read at huggingface.co
Power sampling improves large language model (LLM) reasoning at test time by sampling from a sharpened version of the base model's distribution. Its sampler, PowerMH, is slow because its Metropolis--Hastings (MH) updates are sequential: each accept/reject decision sets the state from which the next proposal is generated. Prior work reduces this cost by changing the sampler or the model (e.
By Nan Jiang, Panagiotis Theodoropoulos, Wei Deng · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.