Paper maps phase transitions in max-pooled multiple instance learning
Original title: Phase Transitions in Learning Driven by Extremal Data
A minimal model shows freezing and retrieval transitions as softmax sharpness rises, and simulations match the predicted flow.
The paper studies extremal regression, a multiple instance learning setting where hidden instance labels are max-pooled into a bag label. A softmax approximates the max using a learnable sharpness parameter, and increasing it during gradient descent can enter regions with vanishing gradients. In a joint large-bag, high-dimension limit, the authors derive freezing and retrieval transitions and say simulations confirm the phase diagram.
- The sharpness of the softmax approximation is a learnable parameter lambda.
- The analysis fixes alpha as the natural log of n over d.
- One transition is linked to the random energy model.
From the original article
1 min read at arxiv.orgWe study an extremal regression problem, an example of multiple instance learning (MIL), where the predictors are given as multisets, called bags. The bag elements (called instances) possess hidden labels which are pooled to produce an observed bag-level label. Max pooling is common in MIL applications, but gives rise to vanishing gradients.
By Ethan Levien · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.