The Optimal Ranking Tradeoffs Between Two Ranking Scores in Two-Class Classification
Paper derives an exact optimal ranking tradeoff between pairs of two-class classification scores.
The paper treats accuracy, class-wise precision and recall, F-scores, Cohen's kappa, balanced accuracy, and related measures as special cases of a parameterized family of ranking scores. It defines an optimal tradeoff between two scores as the ranking that minimizes Fréchet variance under Kendall distance, located at the center of the shortest path on the induced ranking manifold. For equal class priors, it gives an exact analytical ranking score for any pair of initial scores. Averaging the corresponding importance values in parameter space rarely yields that optimum.
- Accuracy, F-scores, kappa, and related metrics are special cases of ranking scores.
- The optimal tradeoff minimizes Fréchet variance under Kendall ranking distance.
- That ranking sits at the center of the shortest path on the ranking manifold.
- An exact solution is given when class priors are the same across performances.
- Averaging the scores' importance parameters rarely produces that optimum.
From the original article
1 min read at arxiv.orgThe accuracy, true negative rate (negative class recall), true positive rate (positive class recall), negative predictive value (negative class precision), positive predictive value (positive class precision), class-specific intersections over unions, class-specific F-scores, Cohen's kappa, and balanced accuracy are scores that one intuitively wants to maximize when developing machine learning methods.
By Sébastien Piérard · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.