Smart search ranks by meaning as well as keywords (one row per story, last 45 days).
Paper generalizes Amari's Bayesian duality by connecting it to a convex duality of Bayes' rule.
The authors revisit Amari's less-known work on Bayesian duality from information geometry. They connect Bayesian duality to a convex duality formulation of Bayes' rule and present a generalization of it. The paper is purely theoretical and discusses relevance for modern AI, with no experiments or model releases.
Review connects control theory, optimal transport, probabilistic inference, thermodynamics, and machine learning via free-energy optimization under constraints.
The review unifies five fields: control theory, optimal transport, probabilistic inference, non-equilibrium thermodynamics, and machine learning. The common conceptual thread is optimization of free-energy-like functionals under dynamical or statistical constraints. Selected applications are presented in reinforcement learning, variational inference, and generative modeling. The tutorial-style text assumes no prior familiarity and begins from physics principles.
Probing six open-source LLMs on HMM-generated data shows belief states are linearly decodable (R² 0.83–0.99), suggesting in-context learning approximates Bayesian prediction.
Researchers prompted six open-source LLMs with data from 40 hidden Markov models selected for non-trivial belief structure and probed residual-stream activations for belief states (posteriors over hidden states). Belief states were linearly decodable with peak R² values of 0.83–0.99 across HMM/LLM combinations, spanning early to late layers. Patching and steering the probe-identified subspace preserved downstream prediction quality while control interventions degraded performance substantially, establishing functional relevance. The results provide representation-level evidence that in-context learning approximates optimal Bayesian prediction over a context-inferred generative model.
Researchers decompose normalizing flows to derive near-pivotal statistics for likelihood-free inference with nuisance parameters, recovering the t-test and beating Welch limits.
A new paper decomposes neural-network normalizing flows to uncover pivotal statistics in the presence of nuisance parameters using only a sample generator from the distribution of interest. The statistic is near-pivotal in the sense of minimum average KL-divergence of its p-values and can incorporate prior knowledge of group invariances such as translation and scale. Experiments show it recovers the one-sample t-test almost exactly, outperforms the Welch test on worst-case size over a constrained variance-ratio range, and delivers higher power and much faster runtime than profile likelihood-ratio techniques on small-to-moderate samples.
RS-MFBO couples global sensitivity analysis with fidelity-augmented Gaussian processes to slash costly high-fidelity simulation runs in industrial flowsheet optimization.
The paper presents RS-MFBO, a reduced-space multi-fidelity Bayesian optimization framework for high-dimensional, expensive black-box functions. It integrates Global Sensitivity Analysis for dimensionality reduction with a fidelity-augmented Gaussian process and a cost-aware acquisition strategy featuring cooldown and promotion mechanisms. Validation on a plasmid DNA bioprocess (SuperPro Designer) and a green fuel synthesis plant (Aspen HYSYS) shows substantial reductions in high-fidelity evaluations while remaining competitive with single-fidelity baselines.
Paper proposes γ*-concept shifts via entropic optimal transport, deriving estimable generalization bounds unifying covariate and concept shift under distribution shift.
The authors show existing definitions of concept shift break when source and target supports mismatch and propose γ*-concept shifts grounded in entropic optimal transport. They derive a general error bound covering broad loss functions, label spaces and stochastic labeling, plus estimators with concentration guarantees. The resulting DataShifts algorithm quantifies distribution shifts and estimates the error bound in most applications, addressing learning bounds that were previously non-estimable from samples.
Theorists prove multi-step lookahead RL planning is NP-hard for every fixed rational discount factor yet give a randomized polynomial-time approximation scheme.
The paper resolves open questions about reinforcement learning with multi-step transition lookahead. It shows exact planning remains NP-hard for every fixed rational discount factor in (0,1), not just discounts arbitrarily close to one, and introduces a randomized polynomial-time approximation scheme for every fixed lookahead depth. Extending to unknown transitions and stochastic rewards via optimism and variance-adaptive confidence bounds, the algorithm achieves cumulative regret matching classical tabular discounted RL up to logarithmic factors.
Researchers derive sufficient sample-size conditions for recovering sparse binary signals from sparse Gaussian measurements, quantifying an information-theoretic threshold of order slog(p/s)/log(ds/p).
The paper studies support recovery of sparse binary signals from noisy linear measurements. For sparse Gaussian designs, the authors identify sufficient minimal sample sizes for maximum-likelihood recovery in the high-SNR regime d*s/p -> infinity, yielding an information-theoretic threshold of order slog(p/s)/log(ds/p) that makes the price of measurement sparsity explicit. They also show a regime where the sample-complexity loss from sparsity is only logarithmic while computational gains are nearly linear, and prove that for independently sparsified dense Gaussian designs a sample size of order p/ψ² suffices for support recovery at any fixed error level.
Drift-Constrained Optimization reformulates fine-tuning as update-direction selection, letting Qwen3 models improve target tasks within a behavioral drift budget.
The paper specifies a behavioral drift budget before optimization and shows that update direction is the remaining degree of freedom, reformulating fine-tuning as a direction-selection problem. In a stringent QA-only setting where instruct models must still generate multi-step reasoning at inference, a coarse layer-selective probe reverses the failure of QA-only fine-tuning. Across Qwen3-8B and Qwen3-14B, these directions substantially improve scientific reasoning and multilingual translation, matching or outperforming dedicated translation systems over 100+ languages and giving stronger initialization for reinforcement learning.
Thin-shell stability proof along the Gaussian cooling path improves cold-start logconcave sampling complexity to near n^2.5 from n^2.75.
The authors prove that logconcave probability measures along the Gaussian cooling path have thin-shell stability, generalizing the thin-shell theorem. This yields improved complexity for sampling an arbitrary logconcave distribution from a cold start. For (near-)isotropic logconcave distributions the complexity is nearly n^2.5, improving the previous n^2.75 bound and matching the abstract Speedy walk.
Researchers introduce model-aware diffusion schedules via fiberwise optimal transport, cutting flow-matching FID on CIFAR-10 by 38.6% at 16 function evaluations.
The paper proposes constructing diffusion and flow-matching sampling schedules from a fiberwise prediction risk defined via optimal transport, combined with coefficient-path kinetic action, yielding a closed-form time allocation. Across DDPM and flow-matching experiments spanning targets, datasets, and architectures, the schedules beat model-agnostic baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Normalized fiberwise-risk profiles from independently trained models align closely, suggesting empirical universality, and a frozen analytic allocation template retains most of the gains.
SQS unifies weight pruning and low-bit quantization via Bayesian variational learning, compressing Llama3.2 and Qwen2.5 at higher rates with comparable accuracy.
SQS introduces a unified Bayesian variational framework performing simultaneous pruning and low-bit quantization, using a spike-and-slab prior for sparsity and Gaussian Mixture Models to model quantized weights. The authors derive an efficient approximation for the intractable objective and provide a consistency result for the variational approach. Experiments on ResNet, BERT-base, Llama3.2, and Qwen2.5 show higher compression rates than prior baselines with comparable performance drops.
New proof resolves the gap-entropy conjecture for Gaussian bandits, bounding optimal best-arm identification samples by H(log(1/delta)+Ent(I)) up to constants.
A paper proves the gap-entropy conjecture for fixed-confidence best-arm identification with independent unit-variance Gaussian arms, means in [0,1], and a unique optimal arm. It shows the optimal expected sample count, averaged over arm-label permutations, is within absolute constant factors of H(log(1/delta)+Ent(I)), where H sums squared gaps and Ent(I) is the instance's gap-entropy. It also gives an instance-independent algorithm bounded by a constant multiple of this quantity plus a g^-2 loglog(e^e/g) term for the smallest gap g.
Quenched Ensemble Sampling generalizes nested sampling's hard energy constraint to repulsive potentials, traversing first-order phase transitions where tempering fails.
Quenched Ensemble Sampling generalizes nested sampling's hard energy constraint into a family of repulsive potentials at the energy boundary, preserving monotone energy descent while making the constrained target amenable to scalable gradient-based kernels. On synthetic phase-transition models it estimates marginal likelihood and draws posterior samples across first-order transitions where popular alternatives such as tempering fail. Applications include marginal likelihood estimation for Bayesian neural network architecture comparison and partition function estimation in a high-dimensional continuous lattice field theory.
Bellman Policy Optimization, a critic-free RLVR method derived from Policy Mirror Descent, improves LLM mathematical reasoning without intermediate state-value estimation.
The paper introduces Bellman Policy Optimization (BPO), a critic-free reinforcement learning method for LLMs with verifiable rewards, derived from Policy Mirror Descent. BPO uses the Bellman equations to reformulate PMD as a trajectory-level objective for autoregressive generation with terminal rewards, avoiding state-value estimation at intermediate states. The authors prove BPO shares the same unique optimal solution as PMD and validate it on mathematical reasoning benchmarks.
DynSHAP extends SHAP explainability to dynamic survival analysis, treating time-feature pairs as Shapley players for longitudinal clinical predictions.
DynSHAP adapts marginal SHAP estimators to dynamic survival analysis by treating time-feature pairs as players in the Shapley game, handling longitudinal irregular inputs and functional survival outputs. Temporal DynSHAP learns linear feature dependencies over time and addresses them with conditional sampling. On synthetic data with ground-truth attributions it recovers temporally dependent features more accurately than marginal estimators, and it produces faithful attributions on two real-world clinical datasets across two DSA architectures.
OptiFlow learns one-step multimodal flow policies for offline RL via state-wise entropic optimal transport, avoiding critic overestimation and mode collapse.
The paper introduces OptiFlow, a framework that frames one-step flow policy learning as a structured sample-allocation problem in offline reinforcement learning. It jointly trains a value-aware reference flow policy and a one-step policy, coupling action samples through state-wise entropic optimal transport where critic values set distillation priority and action-distance cost ensures geometrically compatible pairings. By avoiding direct critic maximization, it anchors the policy to high-value dataset-supported modes without out-of-distribution divergence. Code is released on GitHub and the method performs strongly across diverse offline RL benchmarks.
Researchers release OR-Clarify, a benchmark testing whether LLM agents ask clarifying questions before formulating optimization models from incomplete requests.
OR-Clarify evaluates pre-formulation clarification in operations research: each task gives a partial problem description, withholds structured hidden slots, and scores agents via bounded interaction with a simulated user, measuring slot recovery, stopping behavior, silent assumptions, and interaction cost. The authors also propose InterOPT, a two-stage framework that identifies formulation-critical gaps to decide when to ask or stop. In choice-based experiments InterOPT substantially outperforms all baselines in exact slot recovery and remains competitive in the open-ended setting.
PLC-DPO corrects noisy preference labels during DPO by routing each pair as clean, flip, or tie using calibrated policy-reference margins.
PLC-DPO addresses reversed, weak, or ambiguous preference labels in Direct Preference Optimization by using the calibrated policy-reference margin as online evidence to route each pair's training signal as clean, flip, or tie. This actively corrects supervision direction and strength rather than only filtering suspicious examples. Across 57 dataset-model-benchmark cells, PLC-DPO achieves the best mean win rate of 60.5 versus 55.5 for the next-best method. Injected-noise and tie stress tests plus human disagreement analysis show the routing stays stable and distinguishes flipped from weakly directional pairs.
Researchers derive an exact law linking learning-rate schedules and weight decay in normalized networks, pinpointing when scale-invariant optimization destabilizes.
The paper shows that normalization makes large parts of neural networks scale-invariant, creating a hidden feedback loop where learning-rate schedules and weight decay interact through the parameter norm to control the effective optimizer step. An exact discrete-time law with a single scalar quantity separates contraction- and expansion-dominated effective learning-rate regimes, and the balance point is intrinsically unstable, so constant learning rate with weight decay produces recurrent behavior instead of a stable equilibrium. A unified homogeneous-optimizer framework explains why adaptive methods stabilize more weakly under normalization. The law is validated with high precision on MLPs, CNNs, and GPT-2 across MNIST, CIFAR, WikiText, and OpenWebText, with code released on GitHub.
Safe meta-RL framework reasons about safety in information space, learning a safety value function used for safety filtering and constrained policy optimization.
The paper proposes safe meta-RL that reasons about safety in information space, capturing both physical state and the agent's belief over the underlying task. A safety value function measures the probability of avoiding unsafe regions indefinitely and satisfies a self-consistency condition and Bellman equation, making it learnable via meta-RL. The resulting algorithm uses the learned function for safety filtering and constrained policy optimization, with effectiveness demonstrated on meta-RL benchmarks.
Quantile-k-Loss SGD filters corrupted component losses by quantile sampling, proving linear convergence while outperforming standard and min-k-loss SGD.
The paper proposes Quantile-k-Loss SGD (Q(k)L-SGD), a loss-filtering framework for finite-sum optimization with corrupted components that samples k losses per iteration and updates using an index from the lower empirical q-quantile. The authors prove linear convergence under standard convexity, requiring sample size to scale with the number of corruptions, plus a complementary small-sample probabilistic analysis. Experiments on polynomial regression, regularized logistic regression, and hinge loss show intermediate quantiles often outperform both standard SGD and min-k-loss SGD.
Paper argues diffusion models implicitly form Cobweb-like concept hierarchies, with a basic level emerging at intermediate noise levels.
The authors draw a formal correspondence between diffusion models and Cobweb, a classic incremental concept-hierarchy learner, noting both are hierarchical Bayesian density models with Gaussian prototypes. Modes of the diffusion model's noisy marginals form a hierarchy whose basic level sits at intermediate noise, where class identity commits. The correspondence is tested on MNIST and Fashion-MNIST via mode-finding. Diffusion is reframed as a cognitive model of concept formation.
Lower bounds show heterogeneous asynchronous optimization cannot match homogeneous rates under standard similarity assumptions; strong interpolation plus local PL condition closes the gap.
The paper examines whether pessimistic optimal time complexities for asynchronous distributed optimization with heterogeneous workers (different data distributions) can be overcome. It proves improvement is provably impossible under widely used first- and second-order similarity assumptions for any randomized algorithm, and that the weak interpolation assumption alone is also insufficient. Combining strong interpolation with the local Polyak-Lojasiewicz condition yields a new time complexity bound matching the best-known homogeneous dependence on worker computation times without requiring identical data distributions.
Theoretical work derives a general framework quantifying stability guarantees for averaging-based ensembles under arbitrary data perturbations via covariance operator norms.
The paper develops a framework for quantifying algorithmic stability of ensembling strategies defined via averaging, for varied types of data perturbation. The main result bounds the stability of the ensembled algorithm in terms of the norm of a covariance operator describing the ensembling process. The framework yields interpretable insights across practical perturbation examples and provides sharper guarantees than those derived from differential privacy considerations.
Researchers apply reinforcement learning to identify SMEFT operators explaining particle physics anomalies, reproducing and improving known CDF W-mass results.
The paper introduces a reinforcement learning method to search the large Standard Model Effective Field Theory (SMEFT) operator space for explanations of measurement anomalies. It was validated on the CDF W-mass anomaly, reproducing and improving known results, then applied to a harder multi-anomaly scenario. RL efficiently navigates complex loop-level operator correlations that bias human-driven phenomenological analysis.
Signed Rescue Routing improves LLM cascade efficiency by predicting when a larger model actually corrects a smaller one rather than uncertainty.
Signed Rescue Routing (SRR) is a budgeted cascade method that separately predicts rescues and regressions when escalating from a small to a large model, ranking requests by their signed difference. The authors prove this signed conditional gain is Bayes-optimal under a fixed escalation budget and add only a lightweight two-head router needing small-model output statistics at deployment. Evaluation with Qwen3-4B and Qwen3-8B on MMLU, HellaSwag, and ARC-Challenge shows better accuracy-compute tradeoffs than entropy routing and learned error predictors.
New optimization theory paper proves near-optimal lower bounds for gradient descent with predetermined stepsizes, confirming silver-schedule optimality.
The paper studies the limits of accelerating gradient descent using predetermined nonnegative stepsizes in smooth convex optimization, with the key constant p_sil = log2(1 + sqrt(2)). It proves a non-anytime lower bound of Omega(n^(-p_sil - O(sqrt(log log n / log n)))) on the error achievable by any such stepsize schedule. In the anytime setting, it shows every infinite nonnegative schedule must incur error Omega(n^(-2*p_sil/(1+p_sil) - O(sqrt(log log n / log n)))) at infinitely many horizons. Combined with the silver-schedule upper bound of Altschuler and Parrilo (2025) and the anytime upper bound of Zhang et al. (2025), these results determine the optimal polynomial convergence exponents in both settings.
Study shows video models often learn correct physics but fail to use it; low-dimensional 'causal writability' edits can restore correct motion.
The paper demonstrates 'causal writability' in video generation models: physically correct motion remains available inside the model even when the model outputs incorrect motion. In a red/blue mass oscillation setup, a low-dimensional edit predicted from simple physical variables restores correct fast motion, with a sharp depth boundary marking commitment. Early causal writability predicts which training errors later get corrected, and both writability and closure reproduce in a pretrained 1.3B video model.
Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.
Kalman Delta Networks reformulate recurrent associative memory in linear-attention models as a linear-Gaussian state-space model, allowing the Kalman gain to weight each residual write by accumulated evidence and observation reliability; Delta-rule updates emerge as a special case lacking covariance tracking. Two scan-compatible approximations, Diagonal KDN (online mean-field variational inference) and Isotropic KDN (one uncertainty scalar per head), produce Mobius-map uncertainty recurrences enabling associative scans with logarithmic parallel depth. Controlled pretraining at 750M and 1.3B parameters consistently improves perplexity and mean downstream accuracy over state-of-the-art linear-attention models.