Conformal prediction cannot guarantee coverage at performative equilibrium for L >=1
Original title: Conformal Prediction at Performative Equilibrium: The Price of Performativity
Under total-variation L-sensitivity no finite-sample procedure guarantees both equilibrium-valid coverage and nontrivial prediction sets when L >=1.
Conformal prediction assumes exchangeability but fails when deployed predictor affects data distribution. We study conformal prediction in performative settings seeking prediction sets valid at performative equilibrium. We prove a phase transition at L=1. If L >=1 no procedure can guarantee both. If L<1 minimax excess miscoverage is Omega(1/((1-L)sqrt(n))). We provide transfer theorem and instantiate with retraining algorithms. Experiments on semi-synthetic credit scoring show scaling laws.
- Phase transition at L=1 for total-variation sensitivity
- For L >=1 no finite-sample procedure can guarantee both coverage and nontrivial sets
- Minimax excess miscoverage for L<1 is Omega(1/((1-L)sqrt(n)))
- Transfer theorem: predictor within epsilon of equilibrium yields coverage up to additive L epsilon
- Experiments on semi-synthetic credit scoring show scaling laws
From the original article
1 min read at arxiv.orgConformal prediction gives finite-sample coverage when calibration and test points are exchangeable, but exchangeability can fail when deployment changes the data distribution.
By Ahmed Mahrous · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.