Youtu-Parsing-Omni parses documents, audio, and video
Original title: Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities
A single 5B checkpoint is reported at 96.96 on OmniDocBench v1.6 and 75.08 average on OmniParsingBench.
Youtu-Parsing-Omni is an omni-modal parser that turns documents, images, audio, and video into structured records with space and time grounding. One shared encoder, initialized from a pretrained language model, and one OmniSchema cover ten parsing tasks, with a few fusion layers so video frames can attend to concurrent audio. Training uses OmniSchema-Aware On-Policy Distillation, where the model is trained on its own rollouts against a reference copy that is then refreshed. With one 5B-parameter checkpoint the authors report state-of-the-art OmniDocBench v1.6 (96.96 overall), the lead among open-weight models on OmniParsingBench (75.08 average), a 95.02 chart score, and the highest exact chemical-fingerprint agreement among systems compared. Code and weights are public.
Unconfirmed: Benchmark leadership and scores are the authors' comparisons, not an independent evaluation.
- One 5B-parameter checkpoint covers ten parsing tasks.
- OmniDocBench v1.6 overall score reported as 96.96.
- OmniParsingBench average reported as 75.08, strongest on audio and video.
- Chart-parsing score reported as 95.02.
- Matches a dedicated optical-music model on typeset sheet music, authors say.
From the original article
1 min read at huggingface.co
We present Youtu-Parsing-Omni, an omni-modal parser that converts documents, images, audio, and video into structured records grounded in space and time, providing a perception layer for agents that read, locate, and cite evidence across modalities. Its design rests on one encoder for every modality and one schema for every task.
By Yunhang Shen, Zhangquan Chen, Xiaotian Li · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.