ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “pretraining”

56 stories

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllAI research 61Model release 16AI safety & security 11AI industry 7Vulnerability 4Research 3Threat actor 2Industry 2Malware 2Policy & legal 2Ransomware 2Data breach 1Exploit / PoC 1
SeverityAllcritical 5high 5medium 5low 5info 94

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

OpenWAM releases an open modular stack for world-action model pretraining, plus OpenWAM-alpha trained on about 6,400 hours of egocentric and robot data.

OpenWAM is an open research stack that factorizes World-Action Model pretraining into composable infrastructure, study, and model components with unified training, inference, and evaluation. Controlled experiments distill three principles on knowledge inheritance, world-action synergy, and out-of-domain generalization gains from embodied co-training. The resulting OpenWAM-alpha, pretrained on roughly 6,400 hours of egocentric human and robot data, achieves top-tier results across eight simulation benchmarks and real-robot tests spanning single-arm, bimanual, and dexterous embodiments. The full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, is released openly.

Hugging Face daily papers · 10d agoAI research

GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation

GE-Act 2.0 is a from-scratch pretrained world-action model for robotic manipulation, with success rising from 17.1% to 44.1% as co-training data scales to 30,000 hours.

Genie Envisioner Act 2.0 (GE-Act 2.0) is a world-action model whose generative and action components are all initialized from scratch on manipulation data, combining a control-oriented autoencoder (CoAE), single-step visual planner (SVP), and inverse dynamics model (IDM) trained jointly via knowledge-aligned selective optimization (KASO). Scaling co-training data from 300 to 30,000 hours raises zero-shot success from 17.1% to 44.1% on G1-OP and 13.4% to 31.1% on G2-90D, despite the latter comprising under 2% of data, suggesting cross-embodiment transfer. Gains span 19/20 and 18/20 skill groups, and skill-specific coverage correlates with zero-shot OOD success (Pearson r=0.80).

Hugging Face daily papers · 13d agoAI research

VidaForge: Open Research Infrastructure for Video Pretraining Data Recipes

VidaForge releases open infrastructure and VIDAFORGE-3M (3.14M clips, 6,475 hours) linking video pretraining data recipes to downstream model performance.

VidaForge is an open research infrastructure that represents a video pretraining data recipe as an executable five-stage workflow from raw videos to training datasets. The team compares data recipes with different coverage and quality during early from-scratch pretraining of Wan 2.1 and V-JEPA 2.1, finding that broader-coverage recipes achieve the highest downstream benchmark scores while loss-based evaluation favors different recipes. They also release VIDAFORGE-3M, containing 3.14 million scene-level clips totaling 6,475 hours with fine-grained annotations and curation signals for video data-recipe research.

Hugging Face daily papers · 11d agoAI research

Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining

Climate-ModernBERT domain-adapted encoders reach 76.3 average F1 across nine climate benchmarks, 2.8 points above vanilla ModernBERT-Base.

The authors continue pretraining ModernBERT-Base on three climate corpora - academic text, climate-filtered web data, and synthetic documents - and compare joint mixtures against parameter-space merging of specialized checkpoints. The best model achieves 76.3 average F1 across nine climate NLP benchmarks, a 2.8-point improvement over the vanilla baseline. Academic climate corpora provide the strongest adaptation signal, and parameter-space merging outperforms joint multi-source training while preserving complementary corpus information; all variants are released.

arXiv cs.AI / cs.LG / cs.CL · 9d agoAI research

NCP-ArchPreview Technical Report: Moving towards Latent Space Language Models through Next Concept Prediction

An 8.9B-parameter latent-space language model using next-concept prediction matches OLMo-3-7B pretraining loss with only 51.3% of the training tokens.

NCP-ArchPreview augments next-token prediction with Next Concept Prediction over a product-quantized concept vocabulary built from hidden states, trained jointly end-to-end. The 8.9B model was trained on 5.73T tokens from the Dolma-3 dataset, the largest latent-space language model demonstration to date. It consumes 51.3% of the tokens to reach OLMo-3-7B's final pretraining loss and outperforms it by 2.45 points on the downstream macro-average, including a 5.99-point GSM8K gain. The learned latent space also enables lightweight domain adaptation via a 17M-parameter VQ module and improves speculative drafting accepted length by 4.17%.

Hugging Face daily papers · 8d agoAI research1

Graph Machine: Towards Better Pretraining via Edges

Researchers propose Graph Machine, an O(n)-state sparse architecture that replaces 75% of Qwen3-0.6B dense layers with only slight loss change.

The paper introduces the Graph Machine (GM), an architecture that maintains an O(n)-sized state accessed through sparse, dynamic routing via pointer-like edges updated differentiably by a referral mechanism resembling pointer chasing. The authors replaced 75% of dense Transformer layers in Qwen3-0.6B with GM sparse layers and pretrained from scratch on 15.7B tokens. Retrieving 2 of 4,096 tokens per KV head in each sparse layer degrades loss only slightly, while retrieving 4 marginally improves loss over the dense baseline.

Hugging Face daily papers · 15d agoAI research

Rosetta at AlexandriaX-2026: LoRA-Adapted NileChat for Context-Aware Dialectal Arabic Dialogue Translation

Rosetta ranks 4th and 5th in AlexandriaX-2026 dialectal Arabic dialogue translation using a LoRA adapter on NileChat-3B, finding limited pretraining benefit.

The Rosetta system for the AlexandriaX-2026 shared task fine-tunes a LoRA adapter on NileChat-3B for context-aware English-to-dialectal Arabic dialogue translation. The adapter was additionally pretrained on MADAR and PADIC dialect corpora for the unconstrained track. It achieved spBLEU 26.10 (4th, constrained) and 25.09 (5th, unconstrained). External dialect pretraining improved only two of thirteen dialects while slightly degrading overall performance, indicating negative transfer.

arXiv cs.AI / cs.LG / cs.CL · 7d agoAI research1

Causal Foundation Models

A paper introduces causal foundation models (CFMs): pretrained networks that estimate treatment effects on new datasets via in-context learning without fine-tuning.

Causal foundation models (CFMs) apply the foundation-model paradigm to causal inference, replacing bespoke per-problem estimator pipelines with networks pretrained once at scale. CFMs estimate causal quantities such as the average treatment effect on entirely new datasets through in-context learning, without model updates. The work serves as a practical introduction to the emerging area, covering background in causal inference and machine learning and including example code and Jupyter notebooks.

Hugging Face daily papers · 15d agoAI research

Discrete Beckmann Transport Models for One-Step Language Modeling and Reasoning

DBTM achieves one-step text generation via a time-independent transport map trained directly from data, removing pretrained teacher distillation.

Discrete Beckmann Transport Models (DBTM) build a time-independent flow whose autonomous transport map provably carries any point in ambient space to a fixed point on simplex vertices in a single step. The fixed-point property is characterized by a conservation equation whose residual can be minimized directly from data, eliminating the need for a teacher flow, distillation, and time conditioning. A partial-context interpolant extension turns additional function evaluations into refinement steps rather than ODE integration steps. On language modeling and reasoning tasks, DBTM's one- and few-step generation improves quality and accuracy over discrete diffusion and continuous flow baselines.

arXiv cs.AI / cs.LG / cs.CL · 2d agoAI research1

CausalArena: Benchmarking Causal Discovery in the Foundation Model Era

Researchers introduce CausalArena, a unified benchmark revealing that causal discovery rankings shift substantially across structural causal model families and protocols.

The paper presents CausalArena, a unified and evolvable benchmark for causal discovery combining synthetic structural causal models, semantically grounded operational SCMs, formula-grounded scientific SCMs, and public real-world datasets. Experiments across classical, neural, and pretrained causal discovery foundation models show large ranking shifts between benchmark regimes. The authors identify pretraining-evaluation overlap and benchmark diversity as central evaluation challenges.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research

Axis Robotics Releases AXIS: A Browser-Based Data Engine With 207 Robot Manipulation Tasks and 50,129 Trajectories

Axis Robotics and academic partners released AXIS, a browser-based teleoperation system yielding 207 manipulation tasks and 50,129 trajectories that lifts pi0.5 to 88.8 on LIBERO-Plus.

A team from Axis Robotics, UC Berkeley, Georgia Tech, and NTU introduced AXIS, a browser-based data engine where contributors teleoperate a simulated Franka Research 3 in a MuJoCo WebAssembly frontend while GPU backends handle task generation, training, and evaluation. The released snapshot holds 207 tasks, 50,129 episodes, and 60K+ task or scene variants from more than 70,000 community contributors. Continual pretraining of pi0.5 on AXIS data raises LIBERO-Plus performance from 83.9 to 88.8, versus 57.5 for a volume-matched RoboCasa365 control; the 2.36 TB dataset is gated for non-commercial academic use.

MarkTechPost · 9d agoAI research

Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing

Researchers introduce KnowChange, a framework that uses pretrained vision-language models to synthesize realistic change-detection training data for remote sensing.

KnowChange is a knowledge-guided change data synthesis framework that leverages pretrained vision-language models to reason about plausible change locations and class transitions from pre-change scenes and desired change types. It addresses the limited class-transition coverage and inflexibility of handcrafted rule-based synthesis methods, enabling diverse change types in a unified pipeline. Experiments show KnowChange-generated data outperforms existing synthetic datasets in both synthetic-to-real transfer and synthetic data augmentation, despite compact generation scale.

Hugging Face daily papers · 23d agoAI research

Modality-Autoregressive World-Action Models

ModAR autoregressively denoises multiple future modalities (point tracks, DINO features, depth) before predicting actions, beating prior world-action models at all data scales.

ModAR is the first world-action model (WAM) to autoregressively denoise multiple future modalities before predicting actions, letting each prediction condition on previously generated modalities. Training from scratch shows WAMs benefit from predicting point tracks, DINO features, and depth maps, while future RGB adds no consistent benefit. ModAR's sequential generation outperforms existing WAM formulations with the highest average success rate at all evaluated data scales. It slightly beats video-model-initialized Flex-π (75% vs 72% success) using roughly 20x fewer training FLOPs and no pretraining, and wins on three real-world bimanual tasks.

Hugging Face daily papers · 2d agoAI research

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

LimiX-2, a tabular foundation model built on Contextual Mechanism Networks, outperforms existing tabular models on TabArena, TALENT, and BCCO.

Researchers introduced LimiX-2, a new model in the LimiX family that adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with Context-Conditional Masked Modeling (CCMM) on synthetic datasets generated by structural causal models. Unlike tabular PFNs centered on p(y | x, D_context), CMNs learn mechanism-oriented joint modeling of p(x, y | D_context). Evaluations on TabArena, TALENT, and BCCO show LimiX-2 outperforms current dataset-specific models and tabular foundation models. Its feature attention also encodes direct causal relationships, enabling accurate causal skeleton recovery.

Hugging Face daily papersupdated · 1d agofirst · 2d agoAI research 2 sources

Disentangling Representation Evolution in Transformers through Directional Decomposition

Researchers decompose transformer updates into parallel and perpendicular components, linking representation geometry to editing robustness, compression diagnosis, and training interventions.

The paper studies transformer representation evolution as functional geometry, decomposing learned updates into parallel and perpendicular components across attention/MLP and value-aggregation spaces. Targeted edits reveal a space-dependent asymmetry: exclude-self value-space parallel manipulation is markedly more robust than residual-space and perpendicular counterparts. Full-aggregate parallel suppression during from-scratch pretraining lowers validation-loss trajectories and improves downstream averages, with the value-space variant strongest. Code is released on GitHub.

Hugging Face daily papersupdated · 2d agofirst · 3d agoAI research 2 sources

A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang proposes Recurrent Looped Transformer, carrying full decoder state across every token for unbounded temporal depth.

Yifan Zhang's technical report defines the Recurrent Looped Transformer (RLT), pairing a causal encoder with a recurrent decoder whose final output and layerwise sliding-window attention cache carry into every subsequent token with no prompt-response boundary reset. The reference configuration ties 48 encoder and 48 decoder layers, executing 96 logical blocks per token while the state path grows to 48t blocks after t tokens at fixed per-token compute. The report details RL replay contracts that rebuild all states under current parameters and exact prefix snapshots for multi-turn serving, but explicitly reports no measured efficiency, reasoning quality, or scaling results.

MarkTechPost · 3d agoAI research1

SpectralShift: Effective Context Window Extension of Gated DeltaNet via Spectral Reparameterization

SpectralShift extends Gated DeltaNet context windows via spectral reparameterization of alpha projections, improving long-context capabilities without architectural changes.

The paper studies long-context extension of Gated DeltaNet (GDN) linear attention from the spectral perspective of its transition matrix, identifying two key factors: a broad slow spectral band aligned with target dependency lengths, and preserved fast-decaying modes for state clearing and context switching. SpectralShift reparameterizes the alpha projections initialization to reshape the decay spectrum and adds a learning-rate scaling for alpha projections during long-context continual pretraining. Experiments show consistent long-context improvements over direct continued pretraining, and the code is open-sourced.

Hugging Face daily papers · 4d agoAI research

Dynin-Robotics: Omnimodal Unified Diffusion Vision-Language-Action Model

Dynin-Robotics unifies action, goal, and dynamics prediction in one omnimodal masked-diffusion VLA model, reaching 78.4% success on Franka Research 3 manipulation tasks.

Built on the Dynin-Omni masked-diffusion backbone, the model represents language, observations, goals, and actions as discrete tokens and is continually pretrained on roughly 1.33 million trajectories from 48 Open X-Embodiment datasets. The shared trajectory interface enables test-time scaling via goal prediction, action-candidate evaluation, and joint action/future-state refinement. It achieves competitive results on LIBERO and zero-shot LIBERO-Plus, 78.4% average success across four Franka Research 3 conditions, and up to 29.2x faster model-side action decoding from a block-parallel implementation.

Hugging Face daily papersupdated · 5d agofirst · 6d agoAI research 2 sources1

Breaking the Vision-Action Shortcut: Latent Interface Training for Generalizable Robotics Foundation Models

Latent Interface Training improves robot foundation model generalization by constraining visual conditioning, boosting LIBERO-Plus success up to 10.7 points.

The paper identifies vision-action shortcuts where robot policies exploit task-irrelevant visual cues that fail under distribution shift. Latent Interface Training (LIT) first trains an action expert conditioned on language, robot state, and terminal SE(3) end-effector poses without images, then constrains visual input through a pose-supervised latent interface. Across four VLA and world-action architectures (Pi0.5, MolmoAct2, FAST-WAM, ImageWAM), LIT improves LIBERO-Plus success by 3.87-10.70 percentage points. Real-world tests show 13.30-16.70 percentage-point gains under unseen cameras, lighting, and distractors.

Hugging Face daily papers · 6d agoAI research

StepAudio 3 Gen Technical Report

StepAudio 3 Gen unifies TTS, voice design, music, and sound effects via discrete autoregressive modeling over RVQ tokens.

StepAudio 3 Gen is a general-purpose audio generation model covering zero-shot TTS, voice design, vocal generation, sound effects, music, vibe speech, and mixed audio in one framework. It uses discrete autoregressive modeling over residual vector quantization (RVQ) tokens rather than the diffusion Transformer paradigm, with a StepAudio Tokenizer representing audio at 12.5 Hz in a shared 16x2048 residual code space. Key design principles include interference-aware progressive pretraining, an RVQ Adaptor for multi-codebook acoustic representations, and shared discrete autoregressive modeling. The model reports state-of-the-art performance on TTS and voice design while retaining strong generation across speech, vocals, sound effects, and music.

Hugging Face daily papers · 6d agoAI research

The Frontier AEO Tracker: What Astra Chooses (and every other frontier model, and what you can do about it)

Latent Space launches an AEO tracker scoring 7 frontier models' product recommendations across 161 categories, revealing generational bias flips.

Latent Space built a tracker measuring Answer Engine Optimization by running 6 prompt variations across 7 frontier models with search enabled over 161 product categories, scoring first choices, alternatives, mentions, and anti-recommendations. It found 28 categories with a universally dominant primary choice and observed soft biases, such as models favoring their own lab's coding agents. Analysis of Anthropic's Sol→Astra and Opus→Fable generations showed newer models consulting fewer sources and being less likely to change answers when questions are paraphrased.

Latent Space · 9d agoAI research

H Company Releases NeoMME: A Family of 260M and 800M Single-Tower Multimodal Encoders That Drop the Vision Tower and Causal Decoder

H Company released NeoMME, 260M/800M single-tower multimodal encoders matching 3.75B ColQwen2.5 on ViDoRe v3 while being 14.4x smaller, under Apache 2.0.

H Company released NeoMME, a family of 262,937,906- and 793,715,032-parameter bidirectional encoders that process text and raw 32x32 image patches in a single tower, pretrained via masked diffusion and released under Apache 2.0 with day-zero Hugging Face Transformers support. NeoMME-Retriever-260M reaches 0.523 nDCG@10 on ViDoRe v3, matching 3.75B-parameter ColQwen2.5 while being 14.4x smaller; the 800M model scores 0.556. Hierarchical token pooling with int8 and binary quantization shrinks late-interaction indexes from roughly 1.5 MB to 6 kB per page while retaining 95.19% of nDCG@10; text-only BEIR retrieval remains a weak spot.

MarkTechPost · 10d agoAI research

Reason Through the Latent! Making Latent Visual Reasoning Necessary

Researchers introduce CVRR, forcing multimodal models to rely on recurrent latent computation rather than accessible image tokens, validated via causal interventions and benchmarks.

The paper presents Causal Visual Recurrent Reasoning (CVRR), which makes recurrent hidden-state computation the required image-conditioned path for prediction in vision-language models. Before decoding, visual states and the original multimodal KV cache are removed so only the final recurrent state carries image information to the answer. CVRR retains strong performance on V*, MMVP, BLINK, and MME-RealWorld-Lite while comparable latent reasoners fail under the same constraint. Causal interventions show predictions remain sensitive to recurrent content and that persistent visual evidence causally revises the recurrent trajectory.

Hugging Face daily papers · 11d agoAI research

Agora: Git as Shared Memory for Collective AutoResearch

Agora records multi-agent research as an append-only Git DAG; 13 LLM workers ran nearly 12 days on a weight-transfer problem.

Agora stores every result, hypothesis, and verification as an immutable commit in a Git-stored DAG, with a derived index exposing the frontier and verification status of claims. In a nearly 12-day run, 13 language-model workers with no assigned tasks or central planner published 1,703 contributions on initializing a frozen 119.6M-parameter attention-SSM hybrid from 141 donor models. They improved the evaluator from 3.39 to 1.899 bits per byte, closing 62% of the gap to a trained GPT-2 124M, with 165 independent reproductions posted and none failing.

Hugging Face daily papers · 1d agoAI research

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection

Researchers introduce PANORAMA, a vision-language model grounding caption phrases in pixel masks, alongside the PanoCaps benchmark and gPQ metric.

The paper studies panoptic grounded captioning, requiring VLMs to describe foreground and background regions while grounding each phrase with pixel-level masks. The authors release PanoCaps, a human-annotated benchmark built from panoptic segmentation datasets, plus a phrase-mask matching protocol and a generalized Panoptic Quality (gPQ) metric. PANORAMA formulates grounding as selection from phrase-conditioned mask proposals generated by a pretrained segmenter, achieving the best overall grounding on PanoCaps and matching or exceeding specialized models. Code, data, and models are publicly available.

Hugging Face daily papersupdated · 19h agofirst · 1d agoAI research 2 sources

FreqSpaNet: Frequency and Spatial Learning of SFPF for Physical Layer Hardware Integrity Detection

FreqSpaNet learns spatio-frequency polarization fingerprints to detect unauthorized wireless hardware replacement, reaching 96.31% mean AUROC across seven replacement scenarios.

FreqSpaNet is a representation learning network for open-set hardware anomaly detection using spatio-frequency polarization fingerprints (SFPFs), which capture device-dependent responses across frequencies and directions. A frequency branch models local variations among neighboring frequencies while a geometry-aware spatial branch models directional relationships via angular information, combined through adaptive fusion and complementary pretraining. It achieves a mean AUROC of 96.31%, 9.05 points above the baseline, and is verified under seven hardware replacement scenarios.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research1

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

French BabyLM entry METRON-FR (125M GPT-2, 92.47M words) shows tokenizer artifacts dominate child-scale zero-shot evaluation; proposes standard diagnostics.

METRON-FR is a 125M-parameter GPT-2 pretrained on 92.47M French words, submitted to the BabyLM 2026 Strict track, scoring 85.97% on the native Quebec-French QFrBLiMP benchmark and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE protocol combining French task-data translation with rank-16 LoRA shows relational tasks gain while world-knowledge tasks regress. Bilingual Lexicon Induction reaches p@1 of 68.84%, 18x above chance, and ablations show single-token zero-shot scoring is dominated by tokenizer and template artifacts at child scale.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

OPEN-1B: A Fully Auditable Training Run

Open-1B releases a 1B-parameter model with bitwise-reproducible training, letting independent auditors verify every step of the run on commodity hardware.

The paper introduces a 'fully auditable' tier of model transparency: every training operation is reproducible with bitwise certainty on heterogeneous commodity hardware by imposing definite ordering on GPU kernel reductions, data batch ordering, and collective communication. Because replaying a full run on one machine is infeasible, a collective verification scheme lets many independent auditors certify individual steps covering the whole run. The authors release Open-1B with its full pretraining dataset, every intermediate checkpoint, the training codebase, and an audit harness. This rules out undisclosed data, injected biases, or backdoors that proof-of-learning or proof-of-training-data techniques cannot exclude.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

A Chosen Future Can Still Be Rewritten: Causal Writability in Video Models

Study shows video models often learn correct physics but fail to use it; low-dimensional 'causal writability' edits can restore correct motion.

The paper demonstrates 'causal writability' in video generation models: physically correct motion remains available inside the model even when the model outputs incorrect motion. In a red/blue mass oscillation setup, a low-dimensional edit predicted from simple physical variables restores correct fast motion, with a sharp depth boundary marking commitment. Early causal writability predicts which training errors later get corrected, and both writability and closure reproduce in a pretrained 1.3B video model.

arXiv cs.AI / cs.LG / cs.CL · 2d agoAI research

Why don't machine learning research agents overfit?

Amazon researchers explain why ML research agents avoid benchmark overfitting, attributing generalization to compressibility of successful strategies.

Amazon Science summarizes the paper "What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents," which investigates why benchmark hill-climbing loops, whether run by human communities or LLM research agents, do not produce rampant overfitting. The explanation formalizes Occam's razor via a counting argument: successful ML strategies are highly compressible, so short descriptions lack room to memorize benchmark data and must capture real structure. LLM-based agents, being resettable and controllable, allow this hypothesis to be tested empirically.

Hacker News · security · 2d agoAI researchHN 21↑ · 3 comments

RSIAgent: Autonomous Exploration for Recursive Self-improvement in New Environments

RSIAgent, a training-free multi-agent framework, builds reusable environment memory enabling Kimi-K3 and GLM-5.3 to beat GPT-6.

RSIAgent is a training-free framework for recursive self-improvement through autonomous memory construction, coordinating curriculum, actor, and verifier agents. It uses broad-then-deep exploration to capture environment structures, hidden constraints, and causal dependencies, and freezes the resulting memory for direct reuse without parameter updates. On OSWorld-v2 and Agent's Last Exam it substantially improves strong open-source models, enabling Kimi-K3 and GLM-5.3 to outperform frontier closed-source models including GPT-6.

Hugging Face daily papers · 3d agoAI research2

AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video

AlayaVista is a camera-controllable streaming video world model that decouples panoramic scene evolution from perspective synthesis, trained on a 1,318-hour 4K dataset.

AlayaVista builds a 360-degree scene prior from a single perspective image, evolves it as a camera-conditioned panoramic latent state, and maps it to perspective video via a latent viewport renderer plus a perspective refiner. Chunk-autoregressive generation and few-step distillation enable efficient streaming. The authors introduce MUGEN, a real-world panoramic video dataset with 1,318 hours of at-least-4K video and rich semantic and geometric annotations.

Hugging Face daily papers · 4d agoAI research

Fly Language Model (FLM) Wires the Full Fruit Fly Connectome Into a Frozen 1.2B LLM, and Its Own Controls Show the Wiring Does Not Help

Researchers wire the full fruit fly connectome (166,700 nodes) into a frozen LiquidAI LFM2.5-1.2B LLM, but controls show no fly-specific benefit.

The Fly Language Model (FLM) couples the complete MaleCNS v1.0 fruit fly connectome (166,700 nodes, 25,582,938 edges) to a frozen LiquidAI LFM2.5-1.2B-Instruct backbone, training only a 278,528-parameter readout (~0.0238% of backbone parameters). The fly readout improved NLL by 0.0222 nats/token (perplexity 3.98 to 3.90) on 32 SmolTalk dialogues, but a direct-input control without the graph beat it in all three seeds. Relabeling node identities removes the gain and the recurrence contracts state differences by 0.6 per token, so the connectome adds no long-range memory. The MIT-licensed code runs locally on Python 3.12, but study artifacts remain private, limiting independent reproducibility.

MarkTechPost · 4d agoAI research1

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

SAS trains a gated sparse attention selector end-to-end with the language modeling loss, beating distillation-based sparsifiers on reasoning, long-context, and agentic tasks.

The paper proposes Simple Attention Sparsification (SAS), which injects a selector's continuous scores into attention softmax logits so the language modeling loss directly optimizes context ranking instead of distilling dense attention distributions. Key design choices include log-form gates inside the softmax, normalized gates calibrated against the current block, and preserved continuous selector scores. A memory-efficient Triton kernel integrates SAS into FlashAttention-style computation for long-sequence training. SAS outperforms trainable sparse attention baselines across budgets, with the largest gains under tight attention budgets.

arXiv cs.AI / cs.LG / cs.CL · 5d agoAI research 2 sources2

Evaluating Time-Series Foundation Models and Multimodal Dietary Context for CGM Forecasting

Study finds zero-shot time-series foundation models underperform on CGM forecasting; fine-tuned Chronos-Bolt cuts RMSE up to 18.4% and dietary context adds signal.

The paper evaluates time-series foundation models for continuous glucose monitoring forecasting across eight public datasets covering Type 1 diabetes, Type 2 diabetes, and non-diabetes populations. Under a unified protocol, zero-shot foundation models did not consistently outperform baselines like Elastic Net and PatchTST, but lightweight fine-tuning did, with fine-tuned Chronos-Bolt reducing RMSE by 6.5%-18.4% in the T1D cohort and 8.6%-18.2% in the non-diabetes/T2D cohort. A residual-based fusion framework adding dietary context from CGMacros reduced overall RMSE by about 3% and postprandial RMSE by about 15% versus CGM-only baselines.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research

Model-Aware Schedules Improve Generation via Fiberwise Optimal Transport

Researchers introduce model-aware diffusion schedules via fiberwise optimal transport, cutting flow-matching FID on CIFAR-10 by 38.6% at 16 function evaluations.

The paper proposes constructing diffusion and flow-matching sampling schedules from a fiberwise prediction risk defined via optimal transport, combined with coefficient-path kinetic action, yielding a closed-form time allocation. Across DDPM and flow-matching experiments spanning targets, datasets, and architectures, the schedules beat model-agnostic baselines, including a 38.6% relative FID reduction for flow matching on CIFAR-10 at 16 function evaluations. Normalized fiberwise-risk profiles from independently trained models align closely, suggesting empirical universality, and a frozen analytic allocation template retains most of the gains.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness lets VLM agents control robots via discrete semantic action units, outperforming VLA baselines zero-shot and after light fine-tuning.

Show-Harness is an embodied agent harness that exposes discrete semantic action units a VLM reasons over, with embodiment-specific interpreters grounding them into local robot actions. It enables zero-shot robot control with closed-source frontier VLMs and low-cost adaptation of small open-source VLMs using only a few GPU-hours of fine-tuning. The companion GUMI (GUI Manipulation Interface) extends the same semantic action space to GUI-based demonstration collection without specialized teleoperation hardware. Experiments show robust generalization across tasks, embodiments, and environments, beating representative agentic and VLA paradigms.

arXiv cs.AI / cs.LG / cs.CL · 7d agoAI research

[AINews] OpenAI reports Navier-Stokes singularity find in 88 hours using Astra-next, roughly 10,000 agents and 130B tokens (>$40M), a contender for second ever Millennium Prize awarded

OpenAI-linked accounts claim roughly 10,000 AI agents produced a Navier-Stokes singularity result in 88 hours, pending mathematical verification.

OpenAI-affiliated accounts claim a system of roughly 10,000 agents, trained over about a year with multi-agent reinforcement learning, produced a finite-time singularity result related to the Navier-Stokes Millennium Problem. The claimed 88-hour runtime and 130B-token cost circulate only via social posts, and no preprint, theorem statement, or proof artifact is available. Acceptance by the mathematics community is unresolved, so the claim's epistemic status remains unknown. The roundup also notes Cognition's $48B and Mistral's $24B fundraises, GPT Image 2.5, and Meta's Muse agent relaunch.

Latent Space · 8d agoAI research1

Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs

A survey catalogs inference-efficiency techniques for video and audiovisual LLMs, mapping bottlenecks in sampling, encoding, token reduction, and LLM decoding.

This survey covers inference-efficiency mechanisms for visual and audiovisual video LLMs, reporting reductions in parameters, FLOPs, latency, memory, and token counts. It organizes methods by pipeline stage, covering frame sampling, modality encoding, connector-level token reduction, and LLM prefilling and decoding for systems built since late 2022. The authors compile accuracy-cost comparisons under shared host models and input protocols, identify gaps in audiovisual efficiency and standardized evaluation, and maintain a public repository.

Hugging Face daily papers · 8d agoAI research

Show-Harness: Just a VLM Agent Can Play Robots

Show-Harness enables VLM agents to control robots via a semantic action interface, achieving zero-shot frontier control and few-GPU-hour adaptation of small VLMs.

Show-Harness exposes discrete semantic action units that VLMs reason over, with embodiment-specific interpreters deterministically grounding them into local robot actions. It enables zero-shot closed-source frontier VLM control and adapts small open-source VLMs for low-cost deployment with a few GPU-hours of fine-tuning. The companion GUMI interface extends the same semantic action space to GUI-based demonstration collection without teleoperation hardware, and Show-Harness-equipped agents outperform representative agentic and VLA paradigms.

Hugging Face daily papers · 8d agoAI research