Skip to content
NewA calmer, faster ZeroHour

K2-Horizon open-sources six language models up to 375B-A23B

Original title: K2-Horizon: A Frontier Fleet of Open Source Language Models

Hugging Face daily papersby K2 Team, Zhengzhong Liu, Haonan Li ()1 min read
AI summary · x-ai/grok-4.7Summaries by Vani AI

The release includes stage checkpoints, pre-training data, logs, and training code, plus about 10 trillion synthetic pre-training tokens.

K2-Horizon is a fleet of six fully open language models, from a 0.9B dense model to a 375B-A23B mixture-of-experts, aimed at reasoning and agentic work. The authors introduce Mixture-of-Value Attention, softplus attention gating, and Uno lossless diffusion adapters that they say speed decoding by up to 2.2x. A 36B-A4B model using both attention changes nearly matches a 32B dense model trained on the same data with about one-seventh the active parameters. Four models share a 22T-token corpus, and artifacts are posted at ifm.ai/k2 under the LLM360 openness definition.

Unconfirmed: Benchmark leadership and the 2.2x decoding speedup are author-reported.

Key facts
  • Sizes run from 0.9B dense to 375B-A23B MoE.
  • Uno adapters need no separate draft model.
  • Pipelines produced roughly 10T synthetic pre-training tokens.
  • The 36B-A4B model uses MoVA and softplus attention gating.
  • Intermediate checkpoints, data, logs, and infrastructure code are released.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

We present K2-Horizon, a fleet of six fully open language models, from a 0. 9B dense model to a 375B-A23B MoE, built for general-purpose reasoning and agentic work across math and science, software engineering, deep research, office automation, and beyond. The models lead or are competitive with strong open models of their size on agentic and coding benchmarks.

Hugging Face daily papers

By K2 Team, Zhengzhong Liu, Haonan Li · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.