akhilaaa3/Jev-Omni — new model trending #30 on Hugging Face
Jev-Omni, a 12B Gemma 4 multimodal decision classifier, is trending on Hugging Face.
Independent developer akhilaaa3 released Jev-Omni, an Apache-2.0 multimodal decision classifier fine-tuned from Gemma 4 12B IT on 30,000 questions. It returns calibrated probabilities for text, image, audio and video choices rather than generated explanations. Reported scores include 87.57% on DecisionBench Medium, 63.10% micro accuracy on 1,000 MMAU questions and 53.10% on MVBench, trailing larger references such as Inkling and Qwen3.5-397B-A17B. FP32 weights are about 50 GB; inference uses BF16 on CUDA.
- Fine-tuned from Gemma 4 12B IT on 30,000 questions.
- DecisionBench Medium accuracy is 87.57%; MMAU micro accuracy is 63.10%.
- Covers text, image, audio and video; quality past 20 options is unproven.
- Apache-2.0; author says it is unaffiliated with TypeSafe AI's Jev.
Full article543 words · extracted from huggingface.co · click to collapse
# Jev-Omni
A multimodal decision classifier for **text, images, audio and video**. Supply a question and options; receive a probability for each option—not a generated explanation.
Built on Gemma 4 12B IT, with a 30,000-question fine-tuning run.
## Results
| Benchmark | Accuracy¹ | Micro accuracy |
|---|---:|---:|
| DecisionBench Medium · 80 scenarios / 293 questions | **87.57%** | 86.01% |
| JevBench · matched 195 groups / 231 decisions | **86.15%** | 87.45% |
| MMAU · 1,000 questions | — | **63.10%** |
| MVBench · 14 evaluated tasks / 2,786 questions | **53.10%** | 53.09% |
Merged-model results. ¹Equal-weight scenario/group average.
### Open-weight comparison
| Model | Parameters | MMAU | MVBench | Modalities |
|---|---:|---:|---:|---|
| **Jev-Omni** | 12B | **63.10%** | **53.10%** | Text, image, audio, video |
| [Inkling](https://huggingface.co/thinkingmachines/Inkling) | 975B total / 41B active | 77.20% | — | Text, image, audio |
| [Qwen3.5-397B-A17B](https://huggingface.co/Qwen/Qwen3.5-397B-A17B) | 397B total / 17B active | — | 77.60% | Text, image, video |
Reference scores are officially reported by their developers and may use different evaluation protocols.
Jev-Omni cost uses its recorded input tokens at OpenRouter's Gemma 3 12B input rate ($0.05/M); its classifier generates no output tokens.
Both classifiers are priced **one call per question**: a classifier answers one question at a time, so the state is re-sent for each of them and there is no discount for asking several at once. The three chat models are priced one call per state, with all of that state's questions together, which is their own cheapest shape. Splitting a state into per-question calls multiplies input tokens by 2.82x on this set, so pricing the chat models the same way would move all three marks right by roughly that factor without changing the order.
[DecisionBench dataset](https://huggingface.co/datasets/akhilaaa3/decision-bench)
## Quick start
CUDA GPU required. FP32 weights use about 50 GB before runtime overhead; inference uses BF16 autocast.
```bash
pip install -r https://huggingface.co/akhilaaa3/Jev-Omni/resolve/main/requirements.txt
# ffmpeg is also required for audio input
```
File: [`requirements.txt`](./requirements.txt)
```python
from huggingface_hub import snapshot_download
Hacker News · AI