Astra reads and writes neural network weights
Original title: Astra can read and write weights to improve Qwen
Frontier LLM Astra identifies trained behaviors from raw weights and edits Qwen3.5-0.8B weights to improve math performance without gradients.
Frontier LLMs like Astra are surprisingly good at reading and writing neural network weights directly in natural language. Astra can identify what game a network was trained on from weights alone and fix issues in Qwen3.5-0.8B by editing 0.016% of its weights, raising MATH-500 accuracy from 7.9% to 48.1%. This enables weight-space programming where LLMs reason about weights to optimize networks without training loops.
- Astra correctly identifies game from weights for Breakout, Pong, Pac-Man, Freeway
- Edits 3.5% of weights for car control task achieving 100% success
- Improves Qwen3.5-0.8B math by editing 0.016% weights
- Generalizes to instruction following
- Writes small networks outperforming PPO trained ones
- AI models
- AstraQwen3.5-0.8B
From the original article
15 min read at rikiyat.github.io
We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights , well past what we’d expect from humans or conventional algorithms. This might be a big deal for both optimization and interpretability. Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do .
By dbreunig · rikiyat.github.io
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.