LLMs are not superhuman at reading and writing neural network weights

A recent paper claims frontier LLMs can work with neural network weights directly, but this is not superhuman.

A recent paper argues that frontier LLMs can work with neural network weights directly. This seems superhuman, but it isn’t. The paper shows that GPT-6 Astra can take a small neural network and interact with its weights directly, “well past what we’d expect from humans or conventional algorithms”. Specifically, Astra was able to identify what games a small network 1 was trained to play just by looking at the weights, and even tweak the weights itself. The most impressive result is Astra significantly improving a 0.8B Qwen LLM’s ability on mathematics by natively editing its weights. The paper demonstrates that such capabilities can be replicated by competent human researchers using methods like steering and targeted weight adjustments.