ENEAS: Embedding-guided Neural Ensemble for Adaptive Segmentation
ENEAS adds text prompting and semantic verification to video segmentation to keep tracking targets through occlusion and reject lookalike distractors.
ENEAS is a unified text-promptable method for instance tracking and open-concept semantic discovery in video, designed to fix temporal hallucinations, spatial fragmentation, and semantic misclassification seen in SAM 3-class foundation models. It extends the geometrically robust SeC architecture with a text-prompting adapter and temporal memory, and uses a verification layer combining fast visual embedding matching with conditional VLM refinement for ambiguous candidates. It targets 3D reconstruction pipelines where a single misclassified distractor corrupts the asset. Code and models are open-sourced.
- Fixes SAM 3-class failures: target absence, close-up fragmentation, statue/reflection confusion
- Extends SeC with a text-prompting adapter and temporal memory
- Verification layer reserves VLM reasoning for ambiguous candidates, keeping latency low
- Designed for 3D reconstruction asset pipelines
- Code and models released on GitHub
From the original article
1 min read at huggingface.co
We present ENEAS, a unified, text-promptable method for instance tracking and semantic discovery.
By Javier del Pino, Salvador Rodríguez, Alejandro Garabito · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.