Label-free SSL signals miss known encoder decline
Original title: Improving Signals, Declining Encoder: A Known-Answer Test of Label-Free Monitoring in Self-Supervised Learning
In nine of ten LeJEPA seeds, factor readability dropped while loss and most of eleven signals kept improving.
Researchers trained LeJEPA encoders on synthetic images with known generating factors so they could test whether label-free training monitors notice a real loss of learned content. In nine of ten seeds, readability of those factors fell abruptly and stayed down, nearly always in the same direction in factor space. The loss kept falling, and most of eleven label-free signals kept improving and rated declining runs better than runs that did not decline. Standard change-point detection rarely flagged the decline near onset, and checkpoint selection by these signals mostly chose points after it.
- Nine of ten seeds showed an abrupt, persistent drop in factor readability.
- The decline nearly always followed the same direction in factor space.
- Most of the eleven tracked signals rated declining runs as better.
- Change-point detection rarely marked the decline near its start.
- Signal-based checkpoint selection mostly picked checkpoints after the decline.
From the original article
1 min read at arxiv.orgSelf-supervised training is monitored without labels, through the loss and statistics of the learned features. We report a case where we know the encoders lost part of what they had learned, so we can check whether these signals noticed. We trained LeJEPA encoders on synthetic images with known generating factors.
By Nilotpal Kakati · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.