ViT-HarDis conditions attention on hard samples for face recognition
Original title: Hard Sample-Conditioned Decorrelated Attention for Robust Face Recognition
ViT-HarDis applies stronger head decorrelation to ambiguous samples, outperforming uniform regularization on IJB-B and TinyFace.
Multi-head attention in Vision Transformers often learns redundant patterns. ViT-HarDis conditions decorrelation penalties on sample difficulty using cosine similarity to negative class centers. This targets specialization where identity ambiguity is highest. Experiments demonstrate reduced hard-sample similarity to competing centers and best SOTA results among ViT methods on multiple benchmarks.
- Conditions disentanglement on sample difficulty
- Improves over uniform regularization on most benchmarks
- Reduces inter-head attention similarity
- Best IJB-B and TinyFace results with MS1MV3 training
- No added inference cost
From the original article
1 min read at huggingface.co
Multi-head attention in Vision Transformers (ViTs) allows independent attention patterns across heads, but without explicit regularization, heads may learn redundant patterns rather than complementary roles. Correlation-based penalties encourage head disentanglement, yet apply uniform regularization across all training samples.
By Tahar Chettaoui, Guray Ozgur, Eduarda Caldeira · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.