TabFM and TabFM-Auto Lead Tabular Benchmarks

A 400M tabular model, TabFM, beats tuned AutoML zero-shot, and agent-driven TabFM-Auto further leads TabArena and MLE-Bench.

TabFM is a 400-million-parameter foundation model trained only on synthetic tables from structural causal models. It treats supervised tabular prediction as in-context learning and produces calibrated zero-shot predictions in one forward pass. Across all 51 TabArena datasets—38 classification and 13 regression—zero-shot TabFM ranks first among default tabular foundation models and beats tuned AutoML pipelines, while frozen-weight extensions TabFM+ and TabFM-Auto further improve both tracks. TabFM-Auto pairs TabFM with a language-model agent that iteratively refines cleaning, feature engineering, context selection, and post-processing using metadata and validation feedback. Five configurations take the top five overall TabArena ranks, and the best lifts TabFM from 1785 to 2013 Elo; transferred pipelines add +69 to +143 Elo on other frozen tabular models, and TabFM-Auto ranks first among MLE agents on eight MLE-Bench tabular contests. The two reports do not disagree; the second supplies Elo, transfer, and MLE-Bench detail for TabFM-Auto.

• TabFM is a 400-million-parameter foundation model trained only on synthetic tables generated from structural causal models.
• It treats supervised tabular prediction as in-context learning and outputs calibrated zero-shot predictions in one forward pass, with no task tuning.
• On all 51 TabArena datasets (38 classification and 13 regression), zero-shot TabFM ranks first among default tabular foundation models and beats tuned AutoML pipelines.
• Frozen-weight extensions TabFM+ and TabFM-Auto further improve both classification and regression tracks.
• TabFM-Auto uses a language-model agent to iteratively improve data cleaning, feature engineering, context selection, and post-processing from dataset metadata and validation feedback.
• Five TabFM-Auto configurations occupy the top five overall ranks on all 51 TabArena datasets; the best raises TabFM from 1785 to 2013 Elo.
• Discovered pipelines transfer to other frozen tabular foundation models for gains of +69 to +143 Elo without further search.
• On eight tabular competitions in MLE-Bench, TabFM-Auto ranks first among MLE agents.