openbmb/MiniCPM-V-4.7-35B-A3B — new model trending #29 on Hugging Face
OpenBMB released MiniCPM-V 4.7 35B-A3B, its first on-device mixture-of-experts vision-language model.
OpenBMB published MiniCPM-V 4.7 in a 35B-A3B mixture-of-experts variant, the first MoE model in the MiniCPM-V series, with weights on Hugging Face. It covers single-image, multi-image, and video understanding, and the card claims on-device decoding above 13 tokens per second in about 3 GB on an iPhone 17 Pro Max using Lookahead Expert Scheduling, layer-wise quantization, and speculative sampling. In Instruct mode, 35B-A3B is said to beat Qwen3.6 35B-A3B on overall score in less inference time. The release supports iOS, Android, and HarmonyOS, GGUF, AWQ, GPTQ, and FP8 checkpoints, and inference through SGLang, vLLM, llama.cpp, and Ollama.
- First MiniCPM-V mixture-of-experts model, sized 35B-A3B.
- Claims 13+ tokens/s on iPhone 17 Pro Max in 3 GB.
- Reports a better efficiency tradeoff than Qwen3.6 35B-A3B.
- Adds FP8 plus GGUF, AWQ, GPTQ, and BNB weights.
- Ships day-0 support for Rockchip, Houmo, Intel, AMD, and Arm.
Full article1,918 words · extracted from huggingface.co · click to collapse
# MiniCPM-V 4.7
[On-device results](#on-device-capability-and-efficiency) · [Evaluation](#evaluation) · [Examples](#examples) · [Usage](#usages) · [Model weights](#model-weights)
**MiniCPM-V 4.7** is the latest model in the MiniCPM-V series, available in a **35B-A3B** variant, with **35B-A3B being the first MoE model in the MiniCPM-V series**. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include:
- 📱 **The First Open-Source MoE Model with Lossless, Smooth On-Device Inference.**
With our [**Lookahead Expert Scheduling (LES)**](https://github.com/tc-mb/llama.cpp-omni/blob/master/docs/les/lookahead-expert-scheduling.md) technique introduced alongside MiniCPM-V 4.7, combined with fine-grained layer-wise quantization and DSpark speculative sampling integration, MiniCPM-V 4.7 35B-A3B becomes the **first open-source MoE model to run smoothly on-device without compromising performance**, achieving a decoding speed of **13+ tokens/s with just 3 GB of memory on iPhone 17 Pro Max**.
- 🔥 **A New Pareto Frontier in Performance and Efficiency.**
As model capabilities continue to advance, inference efficiency has become an important measure of a model's value. On resource-constrained devices, delivering the strongest possible capabilities within a limited resource budget calls for higher capability density. The **35B-A3B** variant of MiniCPM-V 4.7 advances the performance–efficiency Pareto frontier for MLLMs. In Instruct mode, 35B-A3B **achieves a higher overall score in less inference time than Qwen3.6 35B-A3B**.
- 🚀 **Efficient Architecture.**
Based on the intra-ViT early compression technique from [LLaVA-UHD v4](https://github.com/THUMAI-Lab/LLaVA-UHD-v4), MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports **mixed 4x/16x visual token compression**, allowing developers to balance visual detail and inference cost across tasks.
- 💻 **Broad Platform and Chip Support.**
MiniCPM-V 4.7 supports **iOS, Android, and HarmonyOS**, together with open-source on-device deployment support. It also provides **day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm**, extending deployment options across mobile and local computing devices.
- 🛠️ **Developer Friendly.**
MiniCPM-V 4.7 supports [inference frameworks](https://github.com/OpenBMB/MiniCPM-o/blob/main/docs/deployment_en.md#supported-inference-and-training-frameworks-and-api) including SGLang, vLLM, llama.cpp, and Ollama, along with [fine-tuning frameworks](https://github.com/OpenBMB/MiniCPM-o/blob/main/docs/deployment_en.md#supported-inference-and-training-frameworks-and-api) including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds **FP8 quantization support**. We also release **Self-Replay Fine-Tuning**, a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier.