Skip to content
NewA calmer, faster ZeroHour

WeightBench scores LLMs on reading, fixing and writing weights

Original title: WeightBench: Evaluating Language Models as Weight-Space Programmers

Hugging Face daily papersby Aayush Marasini, Barsat Khadka, Nick Rahimi ()1 min read1
AI summary · x-ai/grok-4.7Summaries by Vani AI

Opus 5.5 scores 0.535 on all 42 tasks; models can write small policies but cannot patch language models.

Researchers introduce WeightBench, a benchmark of 42 weight-space programming tasks across five tracks: reading, fixing and writing weights, recognizing impossible edits, and data-free compression. Networks range from 2K-parameter control policies to 7B-parameter language models. Frontier agents can read small networks and write working control policies without running them, but cannot yet patch language models or compress them without data. Code, a sandbox harness and a leaderboard are public on GitHub.

Unconfirmed: Scores are the authors' own benchmark runs, not an independent evaluation.

Key facts
  • Claude Opus 5.5 scores 0.535; Sonnet 5.5 0.466; Haiku 5.5 0.402.
  • DeepSeek v4-flash scores 0.215 on all 42 tasks.
  • GPT-6.1-sol scores 0.568 on 38 tasks, versus Opus 0.546.
  • Networks span 2K-parameter control policies to 7B-parameter language models.
  • Agents cannot yet tell an impossible edit from a hard one.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at huggingface.co

Recent work shows that large language models (LLMs) can operate directly on the parameters of other neural networks. They can read what a network was trained to do, fix a network that fails, and write working networks from scratch, all without gradient descent. This capability has been termed Weight-Space Programming (WSP).

Hugging Face daily papers

By Aayush Marasini, Barsat Khadka, Nick Rahimi · huggingface.co

Read the full article at Hugging Face daily papers

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.