Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition
LGKD uses ground-truth labels to guide feature distillation for 3D-CNNs, combining sample-wise and class-wise distillation for action recognition.
The paper proposes Label-Guided Knowledge Distillation (LGKD) for 3D-CNNs, noting that most video feature distillation methods are simple adaptations of image techniques that neglect temporal-dimension differences. LGKD combines sample-wise distillation, which uses label information and the teacher's probability distribution to guide features impacting temporal accuracy, with class-wise distillation employing a prototype network to capture relational knowledge among same-category samples. Experiments on the UCF101 and HMDB51 action recognition benchmarks achieve competitive results.
- Addresses temporal-dimension gaps in image-derived video distillation methods
- Sample-wise distillation uses labels and teacher probabilities to guide features
- Class-wise distillation uses a prototype network for relational knowledge
- Competitive results on UCF101 and HMDB51 benchmarks
From the original article
1 min read at arxiv.orgAs a key model compression technique, knowledge distillation aims to transfer knowledge from a high-capacity teacher model to a lightweight student model for enhancing the latter's performance.
By Yanjiang Shi · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.