ColNanoVDR: Document-Free Query Distillation for Multi-Vector Visual Document Retrieval via Optimal Transport
ColNanoVDR distills multi-vector document retrievers into 149M students that keep 95% NDCG@5 and run up to 26x faster.
ColNanoVDR distills multi-vector visual document retrievers into 149M text-only students without encoding training pages. Its OTW objective aligns student and teacher query tokens using entropic optimal transport and learned token weights, and the alignment cost is proven to bound MaxSim differences on every page. Students distilled from five state-of-the-art teachers retain about 95% of teacher NDCG@5 on ViDoRe v1-v3 while encoding queries up to 26x faster. Under identical training, OTW matches score distillation while reading 12.6x less cached teacher data.
- OTW aligns query tokens by entropic optimal transport without page encodings.
- 149M students retain about 95% of teacher NDCG@5 on ViDoRe v1-v3.
- Query encoding is up to 26 times faster than multi-billion-parameter teachers.
- OTW reads 12.6 times less cached teacher data than score distillation.
Full article192 words · extracted from huggingface.co · click to collapse
Multi-vector retrievers built on vision-language models lead visual document retrieval (VDR), but they run a multi-billion-parameter query encoder on every search. Distilling this encoder into a small student that queries the teacher's existing index would remove the bottleneck. The standard recipe, however, matches the teacher's MaxSim scores and so requires encoding and caching every training page, which can reach terabytes of page tokens. NanoVDR avoids pages entirely by training on the teacher's query embeddings alone, but only for single-vector retrievers. We present ColNanoVDR, to our knowledge the first framework to bring this document-free distillation to multi-vector VDR. Its objective, OTW (Optimal Transport with Learned Weights), aligns the student's query tokens with the teacher's by entropic optimal transport, with a learned weight for each student token, and needs no correspondence between the two tokenizations. We prove that the resulting alignment cost bounds the MaxSim score difference on every page. Distilled from five state-of-the-art teachers, the 149M text-only students retain about 95% of their teachers' NDCG@5 on ViDoRe v1-v3 while encoding queries up to 26x faster. Under identical training, OTW matches score distillation while encoding no page and reading 12.6x less cached teacher data.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.34899