LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence
LimiX-2, a tabular foundation model built on Contextual Mechanism Networks, outperforms existing tabular models on TabArena, TALENT, and BCCO.
Researchers introduced LimiX-2, a new model in the LimiX family that adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models. Unlike tabular PFNs centered on p(y | x, D_context), CMNs learn mechanism-oriented joint modeling of p(x, y | D_context). Evaluations on TabArena, TALENT, and BCCO show LimiX-2 outperforms current dataset-specific models and tabular foundation models. Its feature attention also encodes direct causal relationships, enabling accurate causal skeleton recovery.
- Contextual Mechanism Networks shift in-context learning from target-centric prediction to mechanism-oriented joint modeling.
- Pretraining uses synthetic datasets generated by structural causal models spanning diverse graph structures and observation processes.
- Outperforms dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO.
- Feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
- Model and data scaling guided by previously established scaling laws.
Full article149 words · extracted from huggingface.co · click to collapse
We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the p(y mid x, D_{context}) objective of conventional tabular PFNs, it is designed around learning p(x, y mid D_{context}), a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.17488