TEA-Prompt anchors attention for rehearsal-free continual learning
Original title: Anchor Behavior, Not Parameters: Rehearsal-Free Continual Learning with a Single Shared Prompt
The method uses 0.38M parameters and stores Gaussian query statistics instead of old images.
TEA-Prompt is a one-prompt continual learning method for a frozen pretrained vision transformer. A token evaluation module scores each prompt token and injects that score into attention logits, while an anchoring loss keeps each token's attention output near class-specific query regions. Anchoring uses compact Gaussian query statistics rather than stored old images, so parameters can keep updating if their interaction stays stable. The authors report state-of-the-art accuracy and lower forgetting on CIFAR-100, ImageNet-R, and CUB-200.
Unconfirmed: State-of-the-art accuracy and forgetting comparisons are the authors' reported results.
- Only 0.38M parameters are used.
- No old images are stored; compact Gaussian query statistics are used instead.
- Evaluated on CIFAR-100, ImageNet-R, and CUB-200.
- Code is published at the authors' GitHub repository.
From the original article
1 min read at arxiv.orgPrompt-based continual learning adapts a frozen pre-trained vision transformer by tuning a small set of prompt tokens. Recently, one-prompt methods which share one single prompt across all tasks make this paradigm maximally parameter efficient. The open question is how a shared prompt can keep the knowledge of old classes while being updated for new ones.
By Duy-Cuong Bui · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.