SimOPD: Relational Knowledge Transfer for On-Policy Distillation
SimOPD distills teacher knowledge through response similarities, gaining up to 3.1 points cross-architecture.
Stories per day · 30 days
Relatedcontrastive-learning1knowledge-distillation1llm1on-policy-distillation1representation-learning1
1 stories, newest first · one card per story