IQuest-Q1 open MoE model, 320B total and 15B active, is released
Original title: IQuest-Q1: Advancing Foundation Capabilities for Agentic CLI Systems
The authors describe a three-stage training pipeline and model merging, and release the weights for research.
IQuest-Q1 is a sparse mixture-of-experts foundation model with 320B total parameters and 15B activated, aimed at agentic command-line coding. Training has a stable pre-training stage with a coding-heavier second curriculum, mid-training that widens domains and context, and post-training across scaffolds using SFT, domain-specialized RL, and MOPD. Model merging combines experts and checkpoints. The authors release it as open source.
Unconfirmed: Coding and general-task capability claims are reported by the authors and not independently verified here.
- Pre-training uses Stable SplitNorm, Muon, and muP.
- The second pre-training stage increases the coding-data mixture.
- Post-training spans multiple scaffolds and model versions.
- Merging is used to limit conflicts among parameter updates.
From the original article
1 min read at huggingface.co
We present IQuest-Q1, a sparse Mixture-of-Experts (MoE) agentic foundation model with 320B total parameters (15B activated).
By A. Jian Yang, An Huang, Bo Dan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.