Ovis-Embedding: Pushing the Frontiers of Universal Omni-Modal Embeddings
Ovis-Embedding reports state-of-the-art omni-modal retrieval across text, image, video, and audio.
Ovis-Embedding is an omni-modal embedding family that encodes text, image, video, and audio with a shared backbone initialized from a pretrained Qwen-omni model and adapted through contrastive low-rank training. Training uses a broad multimodal corpus, homogeneous-source sampling for informative in-batch negatives, focal loss, and similarity-based embedding distillation. Low-rank feature decomposition produces compact embeddings with flexible dimensionality and limited quality loss. The family reports state-of-the-art results on MMEB-v3, MMEB-v2, MVEB, MAEB, and RTEB.
- Shared backbone is initialized from a pretrained Qwen-omni model.
- Training uses focal loss and similarity-based embedding distillation.
- Inference supports compact embeddings with flexible dimensionality.
- State of the art reported on MMEB, MVEB, MAEB, and RTEB.
- AI models
- Ovis-EmbeddingQwen-Omni
From the original article
1 min read at huggingface.co
In this report, we introduce Ovis-Embedding, a state-of-the-art omni-modal embedding family built on native integration of text, image, video, and audio. Instead of assembling separate modality towers, Ovis-Embedding uses a shared multimodal backbone to encode different modalities in a common representation space.
By Embedding Team · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.