CoT reasoning generalizes via shared predictors in LLMs
CoT generalizes OOD by composing reusable ID transitions with risk bounded by O(k) coefficient.
Stories per day · 30 days
4 RSS
Relatedfine-tuning2out-of-distribution2adversarial-attacks1agentic-rl1attention1benchmarks1bottleneck-rank1chain-of-thought1compositionality1concentration-inequalities1deep-learning1distillation1harness-scaling1knowledge1language-models1learning-theory1linear-probes1llm-reasoning1
29 stories, newest first · one card per story
Stories tagged #generalization
On-Policy Distillation Lacks Generalization Beyond Supervised Data
MOPD does not fully transfer teachers' out-of-distribution capabilities from multiple teachers.
RLVR models abandon reasoning strategies under OOD shifts
Co-training with bijective inversion tasks restores derivation and OOD accuracy.
Dissertation explores bad minima in transfer learning via adversarial attacks
Adversarial attacks on transfer learning reveal a strong link between source generalization and successful transfer.
Sim-trained ZAP robot policy hits 96.7% on real tasks
A 27B vision-language model fine-tuned only in simulation matches GPT-6 Astra and beats MolmoAct2's 33.3% in the same harness.
Bernstein bounds for regenerative data yield non-i.i.d. generalization guarantees
The bounds support agnostic classifiers from a first-order term and are minimax-optimal in sample size and expected regeneration time.
Harness Scaling trains agentic RL across diverse harnesses
A factorized training subset improved transfer to unseen white-box setups and external black-box harnesses.
AnyGoal tests whether world models generalize past expert rollouts
The benchmark recombines in-support states and actions that expert trajectories rarely pair, and current models fail those cases.
How Finite-Rank Bottlenecks Shape the Low-Rank Adaptation Landscape
A theory paper shows frozen-activation bottleneck rank, not sample size, controls why LoRA updates stay low-rank.
Knowledge That Is Linear Before Fine-Tuning Generalizes After It
How linearly readable a pretrained fact is strongly predicts whether fine-tuning on it generalizes.
PINT: A Unified Model with Shared Predictive Intent for Robot Manipulation
PINT coordinates vision-language, video, and action models via shared Predictive Intent for stronger robot manipulation.
Learning by Compression: Risk Estimates Attain the Law-of-Large-Numbers Rate
Compression-based learning risk bounds match the law-of-large-numbers rate without a logarithmic penalty.
Revisiting Accent Classification with DistilHuBERT: Speaker-Disjoint Evaluation and Cross-Corpus Generalization
DistilHuBERT accent classifiers look strong in-corpus but fail speaker-disjoint and cross-corpus tests.
Why Linear Models Can Still Work in a Complex and Changing World
Linear models can beat nonlinear ones when unstable curvature fails to transfer across environments.
Toward Joint Optimization of Circuit Depth and Training Data Size in Adaptively Grown Quantum Classifiers
Reimplemented Q-FLAIR finds no reliable link between MNIST training-set size and grown quantum circuit depth.
Prospective Prediction of OOD Degradation from Source-Side Training Dynamics
Source-side training dynamics can prospectively predict later out-of-distribution model failure.
Many Ways to Succeed: Diversity-Driven RL Fine-Tuning for VLA Generalization
DRIVE, a diversity-based RL fine-tune, lifts vision-language-action out-of-domain success, including +9.2 points on real dual-arm robots.
Outperformance Inverse Optimization: Learning Objective Functions that Outperform Agent Decisions
Outperformance inverse optimization learns MILP objective weights that beat observed decisions on every component.
Generalization Is Stability, Not Accuracy: Multi-Axis Evaluation of LLMs
SAGO scores LLM generalization as stability across input variants, not single-format accuracy.
Do Self-Evolving Skills Generalize to Held-Out Tasks?
Of 21 self-evolving skills that improve in training, only five fully transfer their gains to held-out tasks.
On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training
Paper shows constraining SFT parameter updates to on-policy update directions (OPSFT) transfers on-policy generalization gains to supervised fine-tuning.
Small transformers track Bayesian evidence for latent common causes via a context-invariant mechanism
Small transformers accumulate Bayesian evidence for latent common causes through a context-invariant, generalizing mechanism.
X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization
X-Tree tokenizes reusable agent skills and raises success on WebArena, ScienceWorld, and WebShop.
Generalization behavior of OPTQ and the role of regularization
New bounds show OPTQ generalization depends on regularization lambda, and a revised lambda choice beats prior recommendations.