Holdout Best-of-N: Unbiased Evaluation and Its Cost
Holdout Best-of-N unbiasedly estimates selection reward only when selector depth is below the score count.
The paper studies unbiased evaluation of Best-of-N selection from a fixed matrix of K independent scores per candidate. A matrix-only estimator is exactly unbiased for every stable candidate-specific score law if and only if the selector uses J fresh scores with J less than K. For independent Gaussian scores, unbiased minimax risk is order sigma squared over square root of K and is attained by Holdout, while allowing bias improves the rate to sigma squared over K. For two candidates, Holdout attains the sharp asymptotic constant 1 over pi times square root of 2 without knowing the variance.
- Reusing selection scores overstates expected Best-of-N reward.
- A fixed score matrix is unbiased only when selector depth J is below K.
- For Gaussian scores, unbiased minimax risk is order sigma squared over square root of K.
- One extra fresh winner score permits unbiased evaluation of the all-K policy.
From the original article
1 min read at arxiv.orgReusing the scores that select a Best-of-$N$ winner can overstate its expected reward. We study evaluation from a fixed matrix of $K$ independent scores per candidate for a policy that selects using $J$ fresh scores.
By Shrey Shah · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.