PORTAL chooses when a test-time model intervention is worth using
Original title: When Should We Intervene at Test Time?
The controller selects which batches to adapt, which inputs to label, and when querying should stop.
PORTAL is a test-time controller that decides whether to deploy an intervention, which inputs to label, and when more labels cannot change the choice or justify their cost. Each candidate prediction rule is compared on the same labeled inputs before new feedback arrives. Experiments show selective deployment can make an intervention that loses accuracy overall useful on chosen batches, while avoiding severe failures from arrival order and corruption streams.
- Analysis ties required feedback to accuracy gaps and shared mistakes.
- Periodic label checks refresh evidence as conditions change.
- Collected labels also train stronger candidates for later batches.
- A rule that already works well needs little additional feedback.
From the original article
1 min read at arxiv.orgTest-time adaptation helps a deployed model respond to changing inputs, but the same intervention can rescue one batch and damage the next. Updating normalization statistics, for example, can improve corrupted images yet fail when a batch contains mostly one class. Average accuracy hides these failures and cannot tell a model when to use an intervention.
By Jiahao Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.