TabFM-Auto: Self-Evolving Pipelines for Tabular Foundation Models
TabFM-Auto uses an agent to evolve data pipelines and leads TabArena and MLE-Bench tabular results.
TabFM-Auto pairs the tabular foundation model TabFM with a language-model agent that iteratively improves data cleaning, feature engineering, context selection, and post-processing using dataset metadata and validation feedback. Across all 51 TabArena datasets, five configurations occupy the top five overall ranks, and the best raises TabFM from 1785 to 2013 Elo. Discovered pipelines transfer to other frozen tabular foundation models for gains of +69 to +143 Elo without further search. On the eight tabular competitions in MLE-Bench, TabFM-Auto ranks first among MLE agents.
- A language-model agent evolves cleaning, features, context, and post-processing.
- Five TabFM-Auto setups take the top five spots on all 51 TabArena datasets.
- Best configuration lifts TabFM from 1785 to 2013 Elo.
- Pipelines transfer to other frozen tabular models for +69 to +143 Elo.
- Ranks first overall among MLE agents on eight MLE-Bench tabular contests.
Full article168 words · extracted from huggingface.co · click to collapse
Tabular foundation models achieve strong zero-shot accuracy on structured data by pretraining on synthetic tables, but they ignore the column names, task descriptions, and auxiliary files that carry dataset semantics. Meanwhile, self-evolving machine learning engineering (MLE) agents train models from scratch on each dataset, yet jointly searching over features, architectures, and hyperparameters is noisy and prone to overfitting. We introduce TabFM-Auto, which pairs a tabular foundation model, TabFM, with a language model agent that evolves the data pipeline around it. Guided by dataset metadata and validation feedback, TabFM-Auto iteratively refines data cleaning, feature engineering, context selection, and post-processing to reduce TabFM's error. Across all 51 datasets of the TabArena benchmark, five TabFM-Auto configurations with different agents and language models take the top five overall positions, and the best raises TabFM from 1785 to 2013 Elo. The discovered pipelines also transfer to other frozen tabular foundation models (+69 to +143 Elo) with no further search. On the 8 tabular competitions of MLE-Bench, TabFM-Auto ranks first overall among MLE agents.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.37989