In-Context Learning for System Identification: Probabilistic Formulation, Long-Sequence Handling, and Benchmark Validation
A probabilistic Transformer meta-model performs zero-shot system identification on long sequences with calibrated uncertainty.
The paper advances in-context learning for system identification with a probabilistic Transformer encoder-decoder that maps training data and query inputs directly to outputs. Tailored mechanisms handle long training sequences and produce calibrated predictions over long horizons. A pretrained meta-model can also generate uncertainty-aware synthetic data that regularizes classical parametric models when real data are scarce. On standard nonlinear benchmarks and a challenging out-of-distribution case, it reports competitive accuracy and strong generalization, including input-only adaptation.
- A Transformer meta-model performs zero-shot system identification.
- Long-sequence mechanisms produce calibrated long-horizon predictions.
- The pretrained model can synthesize data when real samples are scarce.
- Tests show competitive accuracy and strong out-of-distribution generalization.
From the original article
1 min read at arxiv.orgTraditional system identification typically involves two steps: initially, training data is processed to estimate a model of the system of interest; then, the model is used to predict outputs for new input queries. In contrast, recent in-context learning approaches learn a meta model that maps directly from training data and query inputs to outputs in a zero-shot fashion, bypassing explicit model estimation.
By Matteo Rufolo · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.