WeightBench scores LLMs on reading, fixing and writing weights
Original title: WeightBench: Evaluating Language Models as Weight-Space Programmers
Opus 5.5 scores 0.535 on all 42 tasks; models can write small policies but cannot patch language models.
Researchers introduce WeightBench, a benchmark of 42 weight-space programming tasks across five tracks: reading, fixing and writing weights, recognizing impossible edits, and data-free compression. Networks range from 2K-parameter control policies to 7B-parameter language models. Frontier agents can read small networks and write working control policies without running them, but cannot yet patch language models or compress them without data. Code, a sandbox harness and a leaderboard are public on GitHub.
Unconfirmed: Scores are the authors' own benchmark runs, not an independent evaluation.
- Claude Opus 5.5 scores 0.535; Sonnet 5.5 0.466; Haiku 5.5 0.402.
- DeepSeek v4-flash scores 0.215 on all 42 tasks.
- GPT-6.1-sol scores 0.568 on 38 tasks, versus Opus 0.546.
- Networks span 2K-parameter control policies to 7B-parameter language models.
- Agents cannot yet tell an impossible edit from a hard one.
- Products
- WeightBench
From the original article
1 min read at huggingface.coRecent work shows that large language models (LLMs) can operate directly on the parameters of other neural networks. They can read what a network was trained to do, fix a network that fails, and write working networks from scratch, all without gradient descent. This capability has been termed Weight-Space Programming (WSP).
By Aayush Marasini, Barsat Khadka, Nick Rahimi · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.