arXiv introduces LongBanana benchmark for long-context image synthesis
LongBanana contains 90 tasks with 4,670 expert-specified evaluation items for agent-driven multi-reference image generation.
Stories per day · 30 days
9 RSS
Relatedinference5kv-cache5linear-attention4open-weights4attention3benchmarks3compression3qwen3reinforcement-learning3agents2benchmark2grpo2image-synthesis2kimi2language-models2mixture-of-experts2moe2open-source2
107 stories, newest first · one card per story
Stories tagged #long-context
LongBanana benchmark tests long-context multi-reference image synthesis
LongBanana includes 90 tasks with 4,670 expert evaluation items for agent-driven image generation.
Frontier LLMs maintain accuracy with short contexts only
Compaction shows minimal accuracy loss on long-horizon tasks, suggesting semantic state space models.
Fast Relative-Position Attention Algorithm Speeds Up Transformers
Exact tiled computation incorporates additive position terms into FlashAttention without redundant indexing or memory overhead.
DAMI improves long-context retrieval in gated linear attention models
DAMI jointly configures memory parameters for better retrieval accuracy across multiple architectures.
Paper treats long-context retrieval as value-indexed local traversal
On MiniGrid Memory, a five-token traversal head reached 0.95 mean return after 0.59M steps, ahead of absolute addressing.
Gradient-aware allocation and sliding windows for low-bit GDN states
On Qwen3.5-4B and 9B, GGA plus PW-S8 beats STEPQuant@4 by 38.39 and 41.21 points on seven-task averages.
Hybrid wave-attention beats tiny transformers on perplexity, not memory
A 10.94M hybrid scores 5.95 perplexity, but its peak VRAM matches the pure transformer.
SenseNova-Loop-9B-Preview multimodal model released
New 9B-parameter looped model handles text, images and up to 262K tokens for agentic workflows.
1
MERLIN uses key moments to route multi-state linear attention
Reported long-context QA gains are +8.3 and +7.0 points on Gated DeltaNet-1.3B and Mamba-2-780M, with larger LongBench lifts.
Mirendil launches Fern 0.1, its first model, with 10M-token context
The captured page is only a cookie notice, so architecture, weights, and access are not described.
DCKV learns end-to-end compression of hybrid-model prompt KV caches
On Qwen3.5-2B, tokens use over 55 times fewer bytes than bf16 with at most 1.8 points lost on three long-context benchmarks.
Stanford and UC Berkeley introduce TokenGate for faster long-context prefill
The title claims token-gated attention speeds long-context prefill by up to 11.3 times; the article body was not available.
1
Kimi K3.1 Pro is available on Vani Chat and API
Moonshot says unsupported claims fell from 23.35% to 6.85%, with open weights planned for early November.
Kimi introduces K3.1 Pro, a 3T model with 1M context
The model and a faster Highspeed variant are live in Kimi products now; full weights are promised by November 4.
7
BeNice! trains looped models to share one KV-cache across recurrences
In simulated low-memory long-context inference, cache hits rose up to 31.4 percent and throughput up to 23.6 percent.
DraMaze cuts repetition in long microdrama scripts
On scripts of up to 60 episodes and 60k words, repetition fell 32.1% and Logic scored 82.5, 8.5 above the best baseline.
LongZero improves LLM long-context reasoning without external data
The method self-writes multi-hop and retrieval tasks, trains with GRPO, and beat external-corpus RL on five benchmarks.
LongZero improves LLM long-context reasoning without external training data
Models write compact evidence, expand it into multi-document context, and train with GRPO on self-made tasks.
Stratified Sparse Attention narrows the sparse-to-dense gap
On Ministral-3-8B at 50x sparsity, SSA lifts RULER-HARD from 63.8% to 69.3% at 128K.
KronLSTM applies a Kronecker Delta Rule to long-context linear attention
Authors report stable bounded memory without extra normalization, and open-source code and models are released.
KEEP-Bench finds context compressors often drop required evidence
The best open-weight 31B model keeps all required evidence in 66.8% of 4,102 cases, and the best proprietary model in 61.6%.
MemT-1.0 releases Nano and Flash memory-native models
At 128K-token history, Nano is about 1.86 times faster and uses 66 times less persistent storage than a full-context KV cache.
KronLSTM adds a Kronecker Delta Rule for long-context attention
The open-source architecture revises associative memory along one matrix direction and often beats DeltaNet-family models on long context.