Search: “phase-transitions”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Quenched Ensemble Sampling

Quenched Ensemble Sampling generalizes nested sampling's hard energy constraint to repulsive potentials, traversing first-order phase transitions where tempering fails.

Quenched Ensemble Sampling generalizes nested sampling's hard energy constraint into a family of repulsive potentials at the energy boundary, preserving monotone energy descent while making the constrained target amenable to scalable gradient-based kernels. On synthetic phase-transition models it estimates marginal likelihood and draws posterior samples across first-order transitions where popular alternatives such as tempering fail. Applications include marginal likelihood estimation for Bayesian neural network architecture comparison and partition function estimation in a high-dimensional continuous lattice field theory.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:16 UTC · 2d agoAI research1

15

Thin-shell stability of Gaussian cooling: logconcave sampling with sesteric complexity from a cold start

Thin-shell stability proof along the Gaussian cooling path improves cold-start logconcave sampling complexity to near n^2.5 from n^2.75.

The authors prove that logconcave probability measures along the Gaussian cooling path have thin-shell stability, generalizing the thin-shell theorem. This yields improved complexity for sampling an arbitrary logconcave distribution from a cold start. For (near-)isotropic logconcave distributions the complexity is nearly n^2.5, improving the previous n^2.75 bound and matching the abstract Speedy walk.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:10 UTC · 2d agoAI research

10

Searching for New Physics with Reinforcement Learning

Researchers apply reinforcement learning to identify SMEFT operators explaining particle physics anomalies, reproducing and improving known CDF W-mass results.

The paper introduces a reinforcement learning method to search the large Standard Model Effective Field Theory (SMEFT) operator space for explanations of measurement anomalies. It was validated on the CDF W-mass anomaly, reproducing and improving known results, then applied to a harder multi-anomaly scenario. RL efficiently navigates complex loop-level operator correlations that bias human-driven phenomenological analysis.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:07 UTC · 7d agoAI research

22

Bridging Control, Inference, Transport, and Thermodynamics: From Theory to Applications in Learning

Review connects control theory, optimal transport, probabilistic inference, thermodynamics, and machine learning via free-energy optimization under constraints.

The review unifies five fields: control theory, optimal transport, probabilistic inference, non-equilibrium thermodynamics, and machine learning. The common conceptual thread is optimization of free-energy-like functionals under dynamical or statistical constraints. Selected applications are presented in reinforcement learning, variational inference, and generative modeling. The tutorial-style text assumes no prior familiarity and begins from physics principles.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:16 UTC · 2d agoAI research

15

When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

Researchers derive an exact law linking learning-rate schedules and weight decay in normalized networks, pinpointing when scale-invariant optimization destabilizes.

The paper shows that normalization makes large parts of neural networks scale-invariant, creating a hidden feedback loop where learning-rate schedules and weight decay interact through the parameter norm to control the effective optimizer step. An exact discrete-time law with a single scalar quantity separates contraction- and expansion-dominated effective learning-rate regimes, and the balance point is intrinsically unstable, so constant learning rate with weight decay produces recurrent behavior instead of a stable equilibrium. A unified homogeneous-optimizer framework explains why adaptive methods stabilize more weakly under normalization. The law is validated with high precision on MLPs, CNNs, and GPT-2 across MNIST, CIFAR, WikiText, and OpenWebText, with code released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 7d agoAI research

30

StudyBench: Can Self-Evolution Squeeze Textbooks for Olympiad Capability?

Researchers introduce StudyBench, a physics benchmark showing self-evolution gains on textbook problems rarely transfer to olympiad-level questions.

StudyBench is a controlled physics benchmark splitting test data into an Application Set of difficult textbook problems and a Transfer Set of olympiad-level problems. Across three base models, representative self-evolution methods improved on the Application Set but rarely transferred to the harder Transfer Set. A guidance ablation reveals a Guidance Gap, and every method hits a Compute Plateau, indicating the remaining limits are method problems rather than data or compute problems.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research

28

Training a coding model to paint watercolours with TRL and OpenEnv

Hugging Face tutorial trains a coding model with TRL and OpenEnv to paint watercolours through generated code.

A Hugging Face blog walkthrough uses the TRL reinforcement learning library and the OpenEnv environment framework to train a coding model. The target task is generating code that produces watercolour-style drawings, serving as a hands-on reinforcement learning training example. No article body was available in the feed, so specifics are limited to the title.

Hugging Face Blog·Sep 3, 00:00 UTC · 13d agoAI tools & infra1

18

One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

A review paper frames on-policy self-distillation collapse as governed by three levers: token weighting, privileged information, and guidance decay.

The paper critically reviews On-Policy Self-Distillation (OPSD), where a language model trains on its own generations scored token-by-token by a teacher conditioned on privileged information such as reference solutions or environment feedback. It identifies collapse, the progressive narrowing of producible reasoning paths, as the dominant failure mode and analyzes it through three levers: signal weighting, the nature of privileged information, and teacher dynamics. The review is restricted to mathematical reasoning, reports no new experiments, and offers a shared vocabulary separating settled findings from disputed ones.

Hugging Face daily papers·Aug 25, 20:00 UTC · 21d agoAI research

45

Characterizing Language Generation in the Limit: Finite Witnesses and a Separation-Width Hierarch

New work characterizes language generation in the limit via finite witnesses, proves a full separation-width hierarchy, and formalizes all results in Lean.

The paper fully characterizes when language generation in the limit is possible for arbitrary families over a countable universe: each target must admit a finite positive witness such that targets activated by any finite sample share an infinite common intersection. It defines positive separation width and proves every level of the resulting hierarchy occurs, with countable families admitting singleton witnesses and unions of families with infinite common cores requiring unbounded finite witnesses. The characterization, a universal normalization, and a diagonal capture lemma are machine-checked in the Lean proof assistant, with the development maintained on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 17:54 UTC · 6d agoAI research1

18

Convergent Emergence of In-Context Learning Across Modalities

Controlled experiments show few-shot in-context learning emerges across six modalities including language, genomes, images, and proteins, partially supporting a convergence hypothesis.

The paper tests the Convergent Emergence Hypothesis: that few-shot in-context learning, when it emerges, shares a common cross-modality difficulty profile. A controlled framework instantiated the same task suite across six modalities: language, genome, integer sequences, time series, images, and proteins. Paired-mapping ICL emerged in all six modalities, surpassed controlled baselines, and showed correlated per-task effects in five of them, providing partial support for the hypothesis.

Hugging Face daily papers·Sep 11, 20:00 UTC · 4d agoAI research

30

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.

Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.

MarkTechPost·Sep 13, 17:02 UTC · 3d agoAI research1

40

Show HN: LLM Attention Visualization

A developer released a browser-based tool that visualizes which past tokens influence each LLM output token using aggregated, value-weighted attention scores.

A Show HN project presents a React application built on Transformers.js that renders per-token attention influence by aggregating attention weights scaled by value-vector magnitudes across all attention heads and layers. To expose internal tensors, the author instrumented the ONNX computation graph, hosted a modified model on Hugging Face, and pre-generated prompts to avoid long model downloads in the browser. Demos with a 600-million-parameter model show how verbatim copying draws heavily on source tokens and how single outputs blend information from multiple phrases.

Hacker News · AI·Sep 8, 16:59 UTC · 8d agoAI tools & infraHN 41↑ · 9 comments

28

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

Researchers introduce model-aware diffusion schedules via fiberwise optimal transport, cutting flow-matching FID on CIFAR-10 by 38.6% at 16 function evaluations.

The paper proposes constructing diffusion and flow-matching sampling schedules from a fiberwise prediction risk defined via optimal transport, combined with coefficient-path kinetic action, yielding a closed-form time allocation. Across DDPM and flow-matching experiments spanning targets, datasets, and architectures, the schedules beat model-agnostic baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Normalized fiberwise-risk profiles from independently trained models align closely, suggesting empirical universality, and a frozen analytic allocation template retains most of the gains.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:30 UTC · 6d agoAI research

32

Show HN: MultiMatte, a Promptable Image Background Removal Model

Feyn releases MultiMatte, a promptable background-removal model fine-tuned from Meta's SAM 3 via LoRA, outputting alpha mattes that beat SAM 3 on segmentation benchmarks.

Feyn introduced MultiMatte, a promptable image background-removal model built on Meta's SAM 3 (860M parameters). It modifies only 19.49M parameters (2.27%) using a rank-16 LoRA adapter and replaces binary masks with alpha mattes to handle fuzzy boundaries like hair. On the DIS-VD benchmark it scores 0.901 S-measure versus SAM 3's 0.667, and it improves on SAM 3 across all twelve evaluated splits. Training used 19,953 images for 14,000 steps with focal and Dice loss, and the merged weights are available via the nobg library and a web demo.

Hacker News · security·Sep 10, 15:50 UTC · 6d agoModel releaseHN 21↑ · 4 comments1

45

A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models

Study shows video models often learn correct physics but fail to use it; low-dimensional 'causal writability' edits can restore correct motion.

The paper demonstrates 'causal writability' in video generation models: physically correct motion remains available inside the model even when the model outputs incorrect motion. In a red/blue mass oscillation setup, a low-dimensional edit predicted from simple physical variables restores correct fast motion, with a sharp depth boundary marking commitment. Early causal writability predicts which training errors later get corrected, and both writability and closure reproduce in a pretrained 1.3B video model.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:58 UTC · 1d agoAI research

33

TempCloze: Can Video-LLMs Identify the Missing Middle?

TempCloze benchmark tests Video-LLMs' temporal reasoning with 1,521 videos, finding temporal alignment is the primary failure mode across 31 models.

TempCloze is a video cloze benchmark in which models must identify the true missing middle clip given the beginning and ending clips, using 1,521 carefully filtered videos from seven sources, mostly long-take and egocentric footage. Distractors are constructed along three dimensions: Semantic, Alignment and Progression, with shared scenes and objects to reduce appearance cues. Evaluation of 10 proprietary and 21 open-source Video-LLMs found Alignment is the primary bottleneck, with models often recognizing plausible semantics and local event progression but struggling with temporal alignment.

Hugging Face daily papers·Aug 31, 20:00 UTC · 15d agoAI research

25

🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

Caltech professor Anima Anandkumar discusses Neural Operators and FourCastNet for physics modeling, arguing inductive biases beat pure token scaling.

Anima Anandkumar, Bren Professor at Caltech and co-founder of Accelerated Understanding, describes Fourier Neural Operators that learn in frequency and spherical-harmonic domains to model weather, fusion, and fluid or heat flow. Her team built FourCastNet 3, a global weather model competitive with physics-based simulations that runs on consumer-grade GPUs. She also introduced TorchLean, a framework for writing PyTorch-style networks inside the Lean proof assistant for formal verification, and was appointed to the United Nations Scientific Advisory Board. She argues physical domains resist scaling due to tiny datasets and context lengths in the hundreds of billions, so progress comes from built-in structure and physical priors.

Latent Space·Aug 26, 15:15 UTC · 21d agoAI research1

42

Diffusion Models and Concept Formation

Paper argues diffusion models implicitly form Cobweb-like concept hierarchies, with a basic level emerging at intermediate noise levels.

The authors draw a formal correspondence between diffusion models and Cobweb, a classic incremental concept-hierarchy learner, noting both are hierarchical Bayesian density models with Gaussian prototypes. Modes of the diffusion model's noisy marginals form a hierarchy whose basic level sits at intermediate noise, where class identity commits. The correspondence is tested on MNIST and Fashion-MNIST via mode-finding. Diffusion is reframed as a cognitive model of concept formation.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:43 UTC · 5d agoAI research

15

CodeTD: Topology of Attention Detects Hallucinations in Code LLMs

CodeTD detects hallucinations in code LLMs before execution by analyzing topological patterns of attention maps, outperforming recent baselines.

CodeTD applies topological data analysis (TDA) to code LLM attention maps to quantify prompt-generation mismatch as a pre-execution correctness signal. Experiments cover HumanEval, MBPP, BigCodeBench, and MultiPL-E across 5 programming languages and 10 code LLMs up to 34B parameters. The method outperforms recent baselines and transfers between coding benchmarks, helping catch code that fails the task or embeds security vulnerabilities.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:26 UTC · 9d agoAI research1

24

Embedded Graph Flows for Categorical Graph Generation

Researchers propose Embedded Graph Flows, a generative model with learned categorical embeddings that beats DiGress and GruM on molecular graph benchmarks.

Embedded Graph Flows (EGF) learns continuous embeddings for node and unordered-edge categories and transports Gaussian noise toward these endpoints using a permutation-equivariant graph transformer. On QM9 it achieves the best result on all four reported metrics, with a Fréchet ChemNet Distance of 0.150 versus 0.717 for DiGress and 0.812 for GruM. On ZINC250k it retains the lowest NSPDK MMD, indicating close agreement with local substructures of reference molecules. Code is released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:26 UTC · 12d agoAI research1

22

Opaque recurrence, and other AI terms that you should probably know

TechCrunch updates its plain-English glossary defining common AI terms from AGI and agents to chain-of-thought reasoning.

TechCrunch maintains a regularly updated glossary of AI terminology, defining terms such as AGI, AI agents, API endpoints, chain of thought, coding agents, compute, deep learning, and diffusion. It highlights 'opaque recurrence', the reasoning technique in OpenAI's new Astra model that has drawn attention from AI safety researchers. The piece is an educational living document rather than new research or a product announcement.

TechCrunch · AI·Sep 7, 19:24 UTC · 8d agoAI industry1

18

Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout

Researchers propose Mask Forcing, a dual-noise masking rollout that mitigates mode collapse in autoregressive video diffusion distillation.

The paper targets over-saturation and over-smoothing in autoregressive video diffusion models distilled via Distribution Matching Distillation, attributed to reverse-KL mode-seeking behavior. Mask Forcing perturbs the student self-rollout with random masks along spatial and temporal axes, injecting cleaner tokens that act as denoising guidance for noisier tokens. Experiments show improved visual quality across multiple distillation methods without using real video data or extra post-training stages.

Hugging Face daily papers·Sep 7, 20:00 UTC · 8d agoAI research

22

It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention

Study shows attention sinks and massive activations stem from causal-mask self-concentration and value-non-mixing rather than RoPE, informing quantization work.

The paper analyzes why attention sinks and massive activations emerge at initial sequence positions regardless of which token occupies them. Experiments attribute both phenomena to self-concentration of attention induced by the causal mask and the subsequent value-non-mixing in attention outputs. The findings provide empirical evidence on LLM internal dynamics and may inform low-bit quantization strategies, which massive activations currently complicate.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:32 UTC · 8d agoAI research

32

Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead

Theorists prove multi-step lookahead RL planning is NP-hard for every fixed rational discount factor yet give a randomized polynomial-time approximation scheme.

The paper resolves open questions about reinforcement learning with multi-step transition lookahead. It shows exact planning remains NP-hard for every fixed rational discount factor in (0,1), not just discounts arbitrarily close to one, and introduces a randomized polynomial-time approximation scheme for every fixed lookahead depth. Extending to unknown transitions and stochastic rewards via optimism and variance-adaptive confidence bounds, the algorithm achieves cumulative regret matching classical tabular discounted RL up to logarithmic factors.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 16:58 UTC · 6d agoAI research1

22

Measuring LLM Sycophancy under Sustained Multi-Turn Pressure

SPINE benchmark shows LLM sycophantic collapse rises with conversation length as an adaptive user pushes a mistaken position for up to 25 turns.

The SPINE benchmark uses an LLM proxy that persistently and adaptively defends a mistaken user position for up to 25 turns, testing four production LLM systems and three OLMo3-7B variants on 100 false-presupposition and 100 unethical-query items. Collapse rates increase with conversation length for every model, and short-horizon evaluation protocols underestimate sycophancy. Analysis of accessible reasoning traces shows the correct position often remains represented when the model concedes, indicating models choose to please users rather than lacking knowledge. Among tested tactics, emotional appeals are most associated with inducing sycophantic behavior.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:35 UTC · 8d agoAI safety & security1

50

Closed-Loop Cooling Explained: The Plumbing Behind Meta’s AI

Meta engineer Tom Shaw explains the closed-loop liquid cooling systems that power Meta's AI data centers more efficiently.

Meta published an explainer describing its use of closed-loop liquid cooling to support AI workloads. The post, authored by Tom Shaw, frames the plumbing and thermal design as key to running AI infrastructure efficiently. The content is primarily corporate/infrastructure marketing rather than a security or product announcement.

Meta Newsroom·Aug 27, 21:44 UTC · 19d agoAI industry

15

General Quantification of Covariate and Concept Shifts

Paper proposes γ*-concept shifts via entropic optimal transport, deriving estimable generalization bounds unifying covariate and concept shift under distribution shift.

The authors show existing definitions of concept shift break when source and target supports mismatch and propose γ*-concept shifts grounded in entropic optimal transport. They derive a general error bound covering broad loss functions, label spaces and stochastic labeling, plus estimators with concentration guarantees. The resulting DataShifts algorithm quantifies distribution shifts and estimates the error bound in most applications, addressing learning bounds that were previously non-estimable from samples.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:57 UTC · 5d agoAI research1

28

Decomposition-Guided Diffusion Language Models for Inertial Confinement Fusion Prediction

ICF-DLM, the first language-model-based inertial confinement fusion predictor, cuts peak-timing error from 11.6 to 9.2 steps versus LLaMA-3-8B.

Each National Ignition Facility shot costs roughly one million dollars, motivating accurate AI surrogates for predicting 512-step neutron-rate waveforms from laser pulses and target parameters. ICF-DLM combines physics-typed decomposition into yield, peak timing, and local waveform; bidirectional denoising that defers commitment to peak location; and a physics-driven PPO reward. On ICFBench (50,000 simulations plus 232 experimental shots) it outperforms a matched autoregressive LLaMA-3-8B, classical sequence models, and LLM-based time-series predictors.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:00 UTC · 9d agoAI research

16

Bridging the Confidence Gap: Temperature Scaling for Calibrating Test-Time Prompt Tuning

CoTS temperature scaling cuts test-time prompt tuning's expected calibration error from 11.90% to 5.38% on ImageNet variants while raising accuracy.

The paper proposes CoTS, a post-hoc calibration method that applies temperature scaling to minimize the confidence gap between test-time-adapted and zero-shot predictions. A weak-strong ensemble variant, E-CoTS, further exploits multiple test-time augmentations to boost accuracy while maintaining calibration. E-CoTS reduces average expected calibration error from 11.90% to 5.38% on ImageNet variants while increasing accuracy from 60.74% to 62.95%.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 16:25 UTC · 1d agoAI research

20

Molecular Déjà Vu: Digit-Level Retrieval of Published Values in Frontier Language Models

Audit of 22 frontier models finds widespread verbatim retrieval of published molecular property values, with higher reasoning increasing recall of memorized numbers.

An arXiv audit tests 22 frontier LLMs across 12 molecular regression benchmarks for verbatim retrieval of published values. More than 50% of the LLMs show verbatim retrieval on five datasets, and identical experiments are flagged 89% more often at a high reasoning level than at the lowest one. Suppressing retrieval moves model prediction errors closer together in relative terms, suggesting predictive capability is not determined solely by memorized values.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 17:32 UTC · 12d agoAI research1

42