LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence
LimiX-2, a tabular foundation model built on Contextual Mechanism Networks, outperforms existing tabular models on TabArena, TALENT, and BCCO.
Researchers introduced LimiX-2, a new model in the LimiX family that adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models. Unlike tabular PFNs centered on p(y | x, D_context), CMNs learn mechanism-oriented joint modeling of p(x, y | D_context). Evaluations on TabArena, TALENT, and BCCO show LimiX-2 outperforms current dataset-specific models and tabular foundation models. Its feature attention also encodes direct causal relationships, enabling accurate causal skeleton recovery.
- Contextual Mechanism Networks shift in-context learning from target-centric prediction to mechanism-oriented joint modeling.
- Pretraining uses synthetic datasets generated by structural causal models spanning diverse graph structures and observation processes.
- Outperforms dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO.
- Feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
- Model and data scaling guided by previously established scaling laws.
- Tags
- causal-inferencefoundation-modelsin-context-learningpretrainingscaling-lawsstructured-datatabular
- AI models
- LimiX-2
From the original article
1 min read at huggingface.co
We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling.
By Xingxuan Zhang, Gang Ren, Hao Yuan · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.