TabFM: A Zero-Shot Foundation Model for Tabular Data
TabFM, a 400M tabular foundation model, delivers calibrated zero-shot predictions that beat tuned AutoML.
TabFM is a 400-million-parameter foundation model that treats supervised tabular prediction as in-context learning and outputs calibrated zero-shot predictions in one forward pass. It is trained only on synthetic tables generated from structural causal models. Across all 51 TabArena datasets, 38 classification and 13 regression, zero-shot TabFM ranks first among default tabular foundation models and beats tuned AutoML pipelines. Frozen-weight extensions TabFM+ and TabFM-Auto further improve both tracks.
- 400M-parameter model trained only on synthetic causal tables
- Zero-shot predictions in one forward pass, with no task tuning
- Ranks first among default tabular models on all 51 TabArena sets
- TabFM+ and TabFM-Auto improve results on frozen base weights
Full article127 words · extracted from huggingface.co · click to collapse
Tabular machine learning typically relies on per-dataset workflows, fitting tree ensembles or running AutoML searches from scratch for every task. We present TabFM, a 400M-parameter tabular foundation model that formulates supervised tabular prediction as in-context learning. TabFM produces calibrated zero-shot predictions in a single forward pass without task-specific tuning. Trained entirely on synthetic tables generated from structural causal models, TabFM learns general tabular representations that transfer zero-shot to real-world tasks. Across all 51 benchmark datasets in TabArena (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and outperforms tuned AutoML pipelines. Two extensions over the same frozen weights improve performance further on both tracks: multi-view feature expansion with ensembling and post-hoc calibration (TabFM+), and LLM-guided, dataset-specific data processing and feature engineering (TabFM-Auto).
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.37959