Skip to content
NewA calmer, faster ZeroHour

LLMs program tabular foundation model weights and KV caches

Original title: Weights & Caches: LLMs Can Program Tabular Foundation Models

arXiv cs.AI / cs.LG / cs.CLby Shi Bin Hoo ()1 min read1 · 1 read
AI summary · x-ai/grok-4.7Summaries by Vani AI

GPT-6 Astra programmed zero-initialized nanoTabPFN to a mean ROC AUC of 0.849 on 20 TabArena datasets.

The paper reports that large language models can program statistical computations in tabular foundation models without gradient-based optimization, using GPT-6 Astra. Astra initialized nanoTabPFN from zero weights and repaired a signed-product regression failure in pretrained TabPFN-3.5. It also wrote KV-cache edits that changed predictions on unseen synthetic data without changing weights.

Unconfirmed: Metrics are author-reported; the paper does not describe GPT-6 Astra beyond these experiments.

Key facts
  • Zero-initialized nanoTabPFN reached mean ROC AUC 0.849 versus 0.876 pretrained.
  • The TabPFN-3.5 repair added 0.08% average relative error on TabArena-Lite.
  • Cache edits reversed linear trends and turned sine waves into approximate square waves.
  • All three settings avoided gradient-based optimization.
Explain this AI news with Vani AIWhat was announced, what is new and why it matters, in a new chat

From the original article

1 min read at arxiv.org

Neural networks are typically programmed through optimization, rather than by explicitly specifying their computations. Recent work on weight-space programming suggests that large language models (LLMs) can instead reason about and manipulate neural-network weights directly.

arXiv cs.AI / cs.LG / cs.CL

By Shi Bin Hoo · arxiv.org

Read the full article at arXiv cs.AI / cs.LG / cs.CL

A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.