MJev-Spatial: System One Spatial Intelligence via Native Attention Matching
MJev-Spatial classifies spatial answers on Qwen3.5-4B without decoding tokens, topping twelve benchmarks and running much faster.
MJev-Spatial is a System 1 multimodal model that answers spatial choice, scalar, and point tasks by classification rather than free-form text generation. Built on Qwen3.5-4B and trained on 12.3M converted spatial pairs, it scores candidates using pretrained query and key projections without new parameters. It posts the highest overall score among eight models at 65.64%, ranks first on seven of twelve benchmarks, and beats its unmodified autoregressive base on all twelve. Inference uses one forward pass and no answer tokens, running 2.5 times faster than the fastest autoregressive baseline.
- Scores spatial candidates with pretrained query and key projections and no added parameters.
- Trained on 12.3M converted spatial question-answer pairs from existing datasets.
- Highest overall score of 65.64% and first on seven of twelve benchmarks.
- Runs 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.
- Length-aware batching and load balancing raised training throughput by 30.8%.
Full article262 words · extracted from huggingface.co · click to collapse
Spatial intelligence lets an agent perceive and act in the physical world, and many of its judgments must be made at interaction rates: which spatial relation holds, how far an object is, or where to place it. Vision-language models (VLMs) produce these answers by free-form text generation, paying for autoregressive decoding and answer parsing even though the output space is compact: one of a few options, a scalar, or a 2D point. In light of this, we introduce MJev-Spatial, a System 1 multimodal decision model that tackles all three types of spatial understanding tasks within a unified classification framework. Specifically, MJev-Spatial treats each candidate as a class and scores it using the backbone's pretrained query and key projections, without adding parameters. The candidates are the given options for choice tasks, or value bins and image regions refined through coarse-to-fine classification for numerical and point prediction, respectively. At inference, each classification decision takes a single forward pass and generates no answer tokens. To train on heterogeneous spatial data efficiently, we supervise all coarse-to-fine stages jointly with teacher forcing in one forward pass, and combine this with length-aware global batching and cross-GPU load balancing, raising training throughput by 30.8% over the original batching strategy. Built on Qwen3.5-4B and trained on 12.3M question-answer pairs converted from existing spatial datasets, MJev-Spatial achieves the highest overall score (65.64%) among eight tested models, ranks first on seven of twelve benchmarks, and outperforms its unmodified autoregressive base model on all twelve, while running 2.5 times faster than the fastest autoregressive baseline and 5.6 times faster than the base model.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2610.22276