LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence
LimiX-2 scales Contextual Mechanism Networks pretrained via context-conditional masked modeling, beating tabular foundation models on TabArena, TALENT, and BCCO benchmarks.
LimiX-2 is a new tabular model in the LimiX family, developed through model and data scaling guided by previously established scaling laws under the Contextual Mechanism Networks (CMNs) paradigm. It is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models spanning diverse graph structures, functional mechanisms, and observation processes. It outperforms dataset-specific models and tabular foundation models on TabArena, TALENT, and BCCO, and its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.
- New tabular model built on the Contextual Mechanism Networks paradigm
- Pretrained with Context-Conditional Masked Modeling on synthetic SCM data
- Outperforms dataset-specific and tabular foundation models on three benchmarks
- Feature attention encodes direct causal relationships for skeleton recovery
From the original article
1 min read at arxiv.orgWe introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM). CMNs shifts the organizing principle of in-context learning from target-centric prediction to mechanism-oriented joint modeling.
By Xingxuan Zhang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.