Causal Foundation Models
A paper introduces causal foundation models (CFMs): pretrained networks that estimate treatment effects on new datasets via in-context learning without fine-tuning.
Causal foundation models (CFMs) apply the foundation-model paradigm to causal inference, replacing bespoke per-problem estimator pipelines with networks pretrained once at scale. CFMs estimate causal quantities such as the average treatment effect on entirely new datasets through in-context learning, without model updates. The work serves as a practical introduction to the emerging area, covering background in causal inference and machine learning and including example code and Jupyter notebooks.
- CFMs estimate causal quantities like average treatment effect without per-problem pipelines
- Positioned as an emerging paradigm applying foundation models to causal inference
Full article137 words · extracted from huggingface.co · click to collapse
Causal inference is the practice of estimating the effect of a treatment or intervention from data. It traditionally requires a bespoke pipeline for every new problem: first proposing a causal mechanism, selecting a compatible estimator, and finally training it. Meanwhile, across diverse settings and modalities, much of machine learning has shifted to the paradigm of foundation models: networks pretrained once at scale and applied to new tasks without fine-tuning. Causal foundation models (CFMs) bring this paradigm to causal inference. CFMs are pretrained neural networks that estimate causal quantities, such as the average treatment effect, on entirely new datasets using in-context learning without requiring model updates. This work provides a practical introduction to this emerging area. We summarize the necessary background in causal inference and machine learning before discussing CFMs. Throughout, we include example code and Jupyter notebooks.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.03003