ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “long-context”

13 stories

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllAI research 14Model release 11AI industry 2AI safety & security 2Threat actor 1Vulnerability 1
SeverityAllcritical 0high 0medium 3low 0info 28

ConvMem: Convolutional Memory for Long-Context Reasoning

Researchers propose ConvMem, a training-free framework treating LLMs as convolutional kernels for parallelizable long-context reasoning beyond fixed context windows.

ConvMem reformulates long-context reasoning as a hierarchical convolution in which the LLM summarizes text segments hierarchically, shortening the reasoning path from a linear chain to a logarithmic tree. It uses configurable strides, skip connections, and multi-kernel convolution to capture evidence, decompose queries, and enable massive parallelization across segments and reasoning threads. On RULER-HotpotQA and RULER-2WikiMultiHopQA it outperforms training-free baselines and avoids the out-of-distribution overfitting seen in RL-trained approaches like MemAgent.

arXiv cs.AI / cs.LG / cs.CL · 7d agoAI research

Online Draft Co-Training for Speculative Decoding in Large-Scale, Long-Context RL Post-Training

NVIDIA researchers detail an end-to-end system for online draft co-training that speeds speculative decoding in large-scale long-context RL post-training.

The paper tackles scaling online draft co-training for speculative decoding in RL post-training, where rollout generation dominates cost. It extends packed, load-balanced zigzag ring attention to merge rank-local branch attention with causal main-sequence attention for context parallelism, and introduces TapChannel to transport target features across pipeline-parallel stages without changing the schedule. Experiments show co-trained drafts tracking the policy baseline with substantial rollout and end-to-end speedups up to 122B parameters and strong scaling at 256K tokens.

Hugging Face daily papers · 10d agoAI research1

PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents

PARSER uses parallel reader subagents and an RL-trained lead agent for long-context QA, beating baselines and cutting latency up to 11x.

The PARSER paper decouples reading from reasoning: frozen subagents each read one document chunk in parallel while an RL-optimized lead agent iteratively broadcasts queries and aggregates evidence in scatter-gather rounds. On multi-hop QA with 7K to 896K token contexts, a 4B-backbone PARSER beats the strongest sequential memory baseline by 5.7 points on average and 12.0 points at 896K tokens, and a 9B version surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments show robustness to evidence position, order, and distance perturbations, with inference latency reduced by up to 11x.

Hugging Face daily papers · 11d agoAI research1

SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking

SAS trains a gated sparse attention selector end-to-end with the language modeling loss, beating distillation-based sparsifiers on reasoning, long-context, and agentic tasks.

The paper proposes Simple Attention Sparsification (SAS), which injects a selector's continuous scores into attention softmax logits so the language modeling loss directly optimizes context ranking instead of distilling dense attention distributions. Key design choices include log-form gates inside the softmax, normalized gates calibrated against the current block, and preserved continuous selector scores. A memory-efficient Triton kernel integrates SAS into FlashAttention-style computation for long-sequence training. SAS outperforms trainable sparse attention baselines across budgets, with the largest gains under tight attention budgets.

arXiv cs.AI / cs.LG / cs.CL · 5d agoAI research 2 sources2

[AINews] OpenAI reports Navier-Stokes singularity find in 88 hours using Astra-next, roughly 10,000 agents and 130B tokens (>$40M), a contender for second ever Millennium Prize awarded

OpenAI-linked accounts claim roughly 10,000 AI agents produced a Navier-Stokes singularity result in 88 hours, pending mathematical verification.

OpenAI-affiliated accounts claim a system of roughly 10,000 agents, trained over about a year with multi-agent reinforcement learning, produced a finite-time singularity result related to the Navier-Stokes Millennium Problem. The claimed 88-hour runtime and 130B-token cost circulate only via social posts, and no preprint, theorem statement, or proof artifact is available. Acceptance by the mathematics community is unresolved, so the claim's epistemic status remains unknown. The roundup also notes Cognition's $48B and Mistral's $24B fundraises, GPT Image 2.5, and Meta's Muse agent relaunch.

Latent Space · 8d agoAI research1

Learning Length-Extrapolatable Recurrent Models

Researchers propose Credit Stabilization through Time, a training method letting recurrent models extrapolate up to 128x their training length.

The paper argues that length extrapolation failure in BPTT-trained recurrent models is better explained through state credit, the signal through which future losses reach earlier recurrent states. It introduces Credit Stabilization through Time (CST), which locally rescales the state-credit signal during backpropagation without rotating the corrected component or changing forward computation. Controlled experiments show improved performance beyond the training horizon, with gains at up to 128x the training length.

arXiv cs.AI / cs.LG / cs.CL · 8d agoAI research1

Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models

A seven-person independent team trained open-weight agentic cyber models via a data-centric post-training framework, ranking 10th on CyberGym and first at comparable scale.

The paper presents Feyospace-v1, a data-centric post-training framework combining five systems: Choulea (hidden reasoning analysis), SkyReal (teacher-sampling cost reduction), Hongzwang (bypassing teacher API restrictions), PSBreakup (restoring capabilities weakened by model merging), and Kreator (converting expert interventions into trainable reasoning). The data engine builds resettable coding, vulnerability, CTF, kernel-history, full-exploit, firmware, and device-backed environments, retaining only execution-verified and evidence-audited trajectories, yielding 164,269 trajectories for long-context supervised fine-tuning. Three checkpoints improve over their starting models by an average of 23.76% on the full CyberGym suite and 10.49% across pooled CTF suites. As of September 1, 2026, Feyospace-s1 achieves a 63.24% verified success rate, ranks 10th on the official CyberGym leaderboard, and all three checkpoints rank 1st among models at comparable parameter scales.

Hugging Face daily papers · 9d agoAI research

Kalman Delta Networks: Uncertainty-aware Associative Memory

Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.

Kalman Delta Networks reformulate recurrent associative memory in linear-attention models as a linear-Gaussian state-space model, allowing the Kalman gain to weight each residual write by accumulated evidence and observation reliability; Delta-rule updates emerge as a special case lacking covariance tracking. Two scan-compatible approximations, Diagonal KDN (online mean-field variational inference) and Isotropic KDN (one uncertainty scalar per head), produce Mobius-map uncertainty recurrences enabling associative scans with logarithmic parallel depth. Controlled pretraining at 750M and 1.3B parameters consistently improves perplexity and mean downstream accuracy over state-of-the-art linear-attention models.

arXiv cs.AI / cs.LG / cs.CL · 9d agoAI research1

Kalman Delta Networks: Uncertainty-aware Associative Memory

Researchers propose Kalman Delta Networks, adding Kalman-filter uncertainty tracking to delta-rule linear attention, improving perplexity and accuracy at 750M and 1.3B parameters.

The paper introduces Kalman Delta Networks (KDNs), which reformulate recurrent associative memory in linear-attention models as a linear-Gaussian state-space model where the Kalman gain weights each write by accumulated evidence and observation reliability. Two scan-compatible approximations, Diagonal KDN via online mean-field variational inference and Isotropic KDN with a single uncertainty scalar per head, enable associative scans with logarithmic parallel depth. Delta-rule updates are shown to be a special case of this formulation. KDN variants consistently improve perplexity and mean downstream accuracy over state-of-the-art linear-attention baselines in controlled pretraining at 750M and 1.3B parameters.

Hugging Face daily papers · 10d agoAI research

Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability

A controlled study finds agent memory portability varies sharply: fixed-schema knowledge graphs survive model swaps while compressed notes degrade.

The study compares preserving an agent's history as raw long context, RAG chunks, compressed natural-language notes, or fixed-schema knowledge graphs across model upgrades, using 48 synthetic histories and two open-weight sub-10B-parameter models. Fixed-schema KG accuracy changed by only +0.0004 ± 0.0020 after a writer swap, while compressed NOTES shifted asymmetrically by +9.91 or -13.28 percentage points depending on migration direction. Mixed 50/50 embedding migrations captured only 4.96 of an 11.90-point RAG re-embedding gain; 80% of the NOTES deficit came from information lost at construction, and 81% of the RAG deficit from retrieval failures. Store-only repair of NOTES failed to reach 90% recovery in all 48 cases, while retaining raw histories enabled recovery in 34 of 48 for one direction.

arXiv cs.AI / cs.LG / cs.CL · 12d agoAI research1

Unlocking Lossless Speedups in LLMs via Discrete Diffusion

Uno pairs autoregressive LLMs with lightweight diffusion weights to draw multiple tokens in parallel, delivering up to 3x lossless speedup without a draft model.

The paper introduces diffusion-augmented LLMs: autoregressive weights trained with the standard next-token objective plus lightweight diffusion weights trained via a Diffusion Distillation phase to emit multiple tokens in parallel. Psi-Spec samplers enable lossless acceleration without the separate draft model required by speculative decoding. The 8B Uno model outperforms the 26B open DiffusionGemma and proprietary Mercury 2 on agentic tool use, coding, and long-context reasoning benchmarks, with up to 3x throughput gains over the base model at all evaluated batch sizes. Code and checkpoints are released publicly.

Hugging Face daily papers · 14d agoAI research