Generative Late-Interaction Embeddings For Visual Document Retrieval
GLIE compresses visual document retrieval embeddings to four vectors per page while retaining nearly 80% of uncompressed nDCG@5 accuracy.
Researchers analyzing late-interaction retrieval embeddings found they lie exactly on the unit sphere and concentrate near a manifold of intrinsic dimension five to six. GLIE learns a few k vectors per page that serve as a lightweight index and a basis to regenerate the full embedding set for exact rescoring of top candidates at query time. On ViDoRe v1 with four vectors per page, GLIE retains nearly 80% of uncompressed nDCG@5 versus 70% for the best prior post-hoc method, using a 415K-parameter network trained in under three GPU-minutes on 1,000 pages.
- Late-interaction vectors lie on the unit sphere near a manifold of intrinsic dimension five to six.
- Normalizing k-means centroids to the sphere surface adds up to +0.093 nDCG@5 for free.
- GLIE searches on k vectors and regenerates full embeddings only for top candidates to rescore.
- Four vectors per page retain nearly 80% nDCG@5 versus 70% for the best prior method.
- Decoder network has 415K parameters and trains in under three GPU-minutes on 1,000 pages.
From the original article
1 min read at huggingface.co
Late-interaction retrieval is the state-of-the-art for visual document search, but it pays for its accuracy in storage. Existing compression methods retain a subset or local average of the N~1,000 vectors per page. Under aggressive storage budgets, however, these methods degrade sharply, and alternatives require retraining the encoder.
By Mohamed Eltahir, Talal Aloushan, Rose Khairoalsendi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.