LLMs program tabular foundation model weights and KV caches
Original title: Weights & Caches: LLMs Can Program Tabular Foundation Models
GPT-6 Astra programmed zero-initialized nanoTabPFN to a mean ROC AUC of 0.849 on 20 TabArena datasets.
The paper reports that large language models can program statistical computations in tabular foundation models without gradient-based optimization, using GPT-6 Astra. Astra initialized nanoTabPFN from zero weights and repaired a signed-product regression failure in pretrained TabPFN-3.5. It also wrote KV-cache edits that changed predictions on unseen synthetic data without changing weights.
Unconfirmed: Metrics are author-reported; the paper does not describe GPT-6 Astra beyond these experiments.
- Zero-initialized nanoTabPFN reached mean ROC AUC 0.849 versus 0.876 pretrained.
- The TabPFN-3.5 repair added 0.08% average relative error on TabArena-Lite.
- Cache edits reversed linear trends and turned sine waves into approximate square waves.
- All three settings avoided gradient-based optimization.
- Products
- nanoTabPFNTabPFN-3.5
- AI models
- GPT-6 Astra
From the original article
1 min read at arxiv.orgNeural networks are typically programmed through optimization, rather than by explicitly specifying their computations. Recent work on weight-space programming suggests that large language models (LLMs) can instead reason about and manipulate neural-network weights directly.
By Shi Bin Hoo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.