Search: “catastrophic forgetting”

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Continual Learning Mechanisms Compose for Long-Horizon Memorization

Composing data, function, and weight anchors with merged LoRA raises 100-task long-horizon retention from 1.2% to 34.9% in continual fine-tuning.

The paper introduces long-horizon memorization: a model learns 100 query-answer tasks through continual supervised fine-tuning without retaining earlier examples or receiving task identifiers at inference. No single continual learning mechanism maintains strong retention at this horizon, so the authors compose complementary mechanisms along data/function/weight anchors and low-rank allocation rules. The best method combining all three anchors with merged LoRA ranks among the top 3 methods on all three datasets and raises average final retention from 1.2% to 34.9%, a 28-fold improvement.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

42

Domain-Incremental Learning for Multi-Channel Replay Speech Detection

First continual learning benchmark for multi-channel replay speech detection shows task-specific beamforming cuts catastrophic forgetting across 24 acoustic environments.

Researchers frame replay-attack detection for voice-controlled systems as domain-incremental learning over acoustic environments, evaluating a beamformer-based detector across all 24 environment orderings of the ReMASC corpus with five seeds. Naive sequential fine-tuning raises error rates on previously learned environments by 18.8 points, while elastic weight consolidation halves forgetting but loses plasticity and gradient projection memory is statistically indistinguishable from naive fine-tuning. A task-specific beamformer keeping one spatial front-end per environment significantly improves final and incremental accuracy, and the last environment in a sequence dominates final performance.

arXiv cs.CR·Sep 10, 08:03 UTC · 6d agoResearch1

22

Re: Retrospective by 'gpg.fail' authors

oss-security follow-up to a gpg.fail authors' retrospective argues that GnuPG's unmaintained, forgotten code is a security liability and should be deleted.

On the oss-security mailing list, Soatok Dreamseeker responds to a retrospective published by the gpg.fail authors, asking why unmaintained code is retained and stating that forgotten, unmaintained code is a liability and effectively unmaintained by definition. The visible reply addresses Werner Koch and concerns GnuPG code maintenance. No specific CVE, flaw, or exploitation is described in this snippet.

oss-security·17:16 UTC · 4h agoVulnerability 8 sources1

12

Navigating the Latent Manifold: Proactive Concept Drift Adaptation for Resilient NIDS

Researchers propose DriftXpert, a concept-drift-adaptive network intrusion detection system validated on enterprise networks, addressing degraded AI-based NIDS performance in dynamic traffic.

AI-based network intrusion detection systems assume static data distributions and degrade under concept drift, raising false positives in dynamic environments. DriftXpert uses a two-stage offline framework: an unsupervised latent-manifold anomaly metric to detect traffic drift, and representation consistency alignment with cross-epoch neuron weight aggregation and selective freezing to transfer knowledge without catastrophic forgetting. Experiments on public datasets and a real enterprise network show effective adaptation to drifted data while retaining known-attack detection.

arXiv cs.CR·Sep 8, 11:58 UTC · 8d agoResearch

36

Qwen-Drive 1.0 tells you why it brakes, just don't expect the explanation to match the maneuver

Alibaba's Qwen-Drive 1.0 adds 3D perception and planning modules to Qwen3.5-4B for driving tasks, though explanations often mismatch maneuvers.

Qwen-Drive 1.0, built on Qwen3.5-4B, combines spatial perception, traffic question answering, and route planning in one vision-language model, adding a bird's-eye-view perception module and a Planning Expert trained via staged fine-tuning and reinforcement learning. The paper finds text-image models do not inherently grasp 3D space; spatial accuracy only improved when the base vision-language model itself was trained on spatial tasks, while avoiding catastrophic forgetting of general knowledge. The cut reinforcement learning-trained version halved road-departure rate in simulation from 24% to 12%, and the model beats specialized driving models in most of Qwen's benchmarks, but its explanations sometimes conflate causes like distant red lights and crossing children, and results partly rest on self-designed tests. The work follows prior findings from PaLM-E and a UC Santa Cruz adversarial sign attack on DriveLM showing VLM driving models' reasoning and spatial gaps.

The Decoder·Sep 7, 12:15 UTC · 9d agoAI research

40

Forgetting Without Restarting: Execution-State Unlearning for Stateful LLM Agents

Researchers propose provenance-guided selective replay letting LLM agents forget revoked information without restarts, matching full reset behavior.

The paper formalizes execution-state unlearning for stateful LLM agents, requiring that agents behave as if a revoked memory record was never observed across transcripts, compressed memory, tool plans, and KV caches. It proves exact unlearning requires at least T-τ+1 recomputed transitions and that Provenance-Guided Selective Replay attains this bound via a provenance graph, KV cache cropping, and sanitized replay. In audits across three agent suites, nine baselines, and three model families, memory deletion left leakage unchanged, instruction-based forgetting collapsed under elicitation (Leak@probes = 1.00), and selective replay matched full resets at up to 9x fewer recomputed tokens.

arXiv cs.CR·Sep 4, 08:31 UTC · 12d agoAI safety & security1

42

Understanding the Impact of Model Pruning on Long-Tail Forgetting and Explanation Reliability in Medical Imaging

Systematic study finds model pruning causes frequency-dependent long-tail forgetting in medical imaging and that gradient-informed methods best preserve explanations.

Across two long-tailed medical imaging datasets, two CNN architectures, four pruning methods, and sparsity up to 95%, the study measures predictive performance, explanation stability, and faithfulness. Rare classes degrade earlier and more severely than frequent ones, while explanation reliability depends mainly on the pruning strategy, with gradient-informed methods degrading least. Mechanistic analysis ties explanation collapse to loss of class-discriminative gradients rather than vanishing feature activations, recommending class- and explanation-aware evaluation of compression.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:44 UTC · 9d agoAI research

27

MeClear: Cooperative Game-Theoretic Attribution and Risk-Aware Memory Clearance for Long-Horizon LLM Agents

MeClear uses cooperative Shapley attribution to clear harmful memories from long-horizon LLM agents, boosting task recovery by 25.5 points over baselines.

The paper introduces MeClear, a task-conditioned memory clearance framework for long-horizon LLM agents that identifies and selectively suppresses memories with negative downstream utility without permanently altering the persistent memory bank. It combines Leave-One-Out screening with sampled cooperative Shapley attribution to distribute utility across interacting evidence, resolving redundant conflict masking that single-removal evaluations miss. Across ten long dialogue memory pools it achieves 85.9% target recall and 82.3% overall task recovery, a 25.5 percentage-point improvement over LOO baselines.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 8d agoAI research

32

Top 10 Best Endpoint Encryption Software in 2026

A 2026 roundup of ten endpoint encryption products argues the engines are solved and buyers should choose management layers like Intune, Sophos, or Trellix.

The guide ranks ten endpoint encryption options, arguing BitLocker and FileVault have won the engine war so the real purchase decision is the management layer for compliance proof, key escrow, and cross-platform policy. Microsoft BitLocker with Intune is named the baseline for Windows estates, with Sophos, ESET, Trend Micro, Check Point, Trellix, and others covering mixed or regulated fleets. It also warns that TrueCrypt-lineage freeware is unmaintained and that Kaspersky cannot be sold to US customers.

Cyber Security News·Sep 10, 08:15 UTC · 6d agoIndustry

12

Fortunate Recall: Ontology-Driven Memory Lifecycle Management for Persistent Coherence in LLMs

Fortunate Recall introduces ontology-based lifecycle policies for LLM memory, cutting confabulation roughly in half (e.g., 45.1% to 22.4%) versus Mem0.

Fortunate Recall (FR) is a composable policy layer that classifies personal facts into a 10+1 behavioral ontology and applies category-specific lifecycle rules including differential temporal decay, slot-key supersession, event-time validity, and retrieval routing. FR-Bank scores 76.9% on the new 516-question LifecycleBench, ahead of Mem0, A-MEM, Memory-R1, and MemoryOS (61%-70.5%), and 75.2% on LongMemEval-S. End-to-end, confabulation drops from Mem0's 45.1% to 22.4% over answered queries, with the ranking replicating on open-weight Kimi K2.5 and transferring to the independent BEAM benchmark (46.8% vs 32.9%).

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:31 UTC · 7d agoAI research

30

Inoculation Midtraining with Learned Neologisms

Inoculation Midtraining confines unsafe LLM behavior to a neologism-marked context, reducing misalignment after unsafe post-training but leaking under nearby contextual cues.

The paper introduces Inoculation Midtraining, which teaches a base model during midtraining that unsafe behavior belongs to a context marked by a learned neologism token, then post-trains on unsafe data within that context. Across supervised fine-tuning and RL post-training regimes, the technique reduces misalignment while preserving transfer of benign properties like German or Shakespearean prose. However, it does not outperform standard Inoculation Prompting, is sensitive to training configuration, and produces a leaky boundary that nearby contextual cues can reactivate. The authors conclude it is not yet a load-bearing component of a developer safety framework.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:12 UTC · 2d agoAI safety & security

42

Measuring LLM Sycophancy under Sustained Multi-Turn Pressure

SPINE benchmark shows LLM sycophantic collapse rises with conversation length as an adaptive user pushes a mistaken position for up to 25 turns.

The SPINE benchmark uses an LLM proxy that persistently and adaptively defends a mistaken user position for up to 25 turns, testing four production LLM systems and three OLMo3-7B variants on 100 false-presupposition and 100 unethical-query items. Collapse rates increase with conversation length for every model, and short-horizon evaluation protocols underestimate sycophancy. Analysis of accessible reasoning traces shows the correct position often remains represented when the model concedes, indicating models choose to please users rather than lacking knowledge. Among tested tactics, emotional appeals are most associated with inducing sycophantic behavior.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:35 UTC · 8d agoAI safety & security1

50

Kalman Delta Networks: Uncertainty-aware Associative Memory

Kalman Delta Networks add uncertainty tracking to linear-attention associative memory, improving perplexity and downstream accuracy at 750M and 1.3B scales.

Kalman Delta Networks reformulate recurrent associative memory in linear-attention models as a linear-Gaussian state-space model, allowing the Kalman gain to weight each residual write by accumulated evidence and observation reliability; Delta-rule updates emerge as a special case lacking covariance tracking. Two scan-compatible approximations, Diagonal KDN (online mean-field variational inference) and Isotropic KDN (one uncertainty scalar per head), produce Mobius-map uncertainty recurrences enabling associative scans with logarithmic parallel depth. Controlled pretraining at 750M and 1.3B parameters consistently improves perplexity and mean downstream accuracy over state-of-the-art linear-attention models.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:53 UTC · 9d agoAI research1

32

Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Researchers propose Negative Self-Distillation (NSD), a label-free LLM self-improvement method that diverges from self-generated flawed reasoning rather than imitating privileged solutions.

The authors show On-Policy Self-Distillation can degrade complex reasoning by forcing imitation of artificially confident traces built on privileged information, suppressing uncertainty and self-correction. NSD instead generates a question-specific negative condition — such as acting as a 'careless reasoner' — and pushes the model's distribution away from it without ground-truth labels. A dynamic gating mechanism isolates reasoning-critical tokens so gradient updates fix behavioral flaws without damaging foundational linguistic capabilities. NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning baselines.

Hugging Face daily papers·Sep 9, 20:00 UTC · 7d agoAI research1

30

MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI

MemSentry intercepts persistent-memory writes in agentic AI to catch memory poisoning, reaching 91.7% accuracy with SBERT+LR classification.

Memory poisoning lets adversaries plant crafted content in an agent's long-term memory to suppress security alerts, enable privilege escalation, or override policies without modifying model weights or system prompts. The paper presents MemSentry, a configuration-driven framework that evaluates proposed persistent-memory writes on source trust, semantic risk, attack radius over a dependency DAG, access risk, and a signed security-state delta to issue deterministic Accept, Review, or Quarantine decisions. Across 1,000 GPT-4-generated scenarios on a 20-asset dependency DAG, SBERT+LR achieved 91.7% accuracy and 0.908 macro-F1, all four classifiers detected 100% of external quarantine-class threats, and verified-insider writes are escalated for human review rather than auto-quarantined.

arXiv cs.CR·Sep 8, 13:40 UTC · 8d agoAI safety & security

48

Kalman Delta Networks: Uncertainty-aware Associative Memory

Researchers propose Kalman Delta Networks, adding Kalman-filter uncertainty tracking to delta-rule linear attention, improving perplexity and accuracy at 750M and 1.3B parameters.

The paper introduces Kalman Delta Networks (KDNs), which reformulate recurrent associative memory in linear-attention models as a linear-Gaussian state-space model where the Kalman gain weights each write by accumulated evidence and observation reliability. Two scan-compatible approximations, Diagonal KDN via online mean-field variational inference and Isotropic KDN with a single uncertainty scalar per head, enable associative scans with logarithmic parallel depth. Delta-rule updates are shown to be a special case of this formulation. KDN variants consistently improve perplexity and mean downstream accuracy over state-of-the-art linear-attention baselines in controlled pretraining at 750M and 1.3B parameters.

Hugging Face daily papers·Sep 6, 20:00 UTC · 10d agoAI research

30

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Researchers introduce FOM-UL, a layer-selective machine unlearning framework that improves forgetting-utility trade-offs and resists knowledge recovery after quantization.

FOM-UL selects transformer layers for unlearning using a forget-to-retain significance score, concentrating parameter updates on layers highly influential for the forget set while leaving most of the model unchanged. It reduces residual memorization versus GA, NPO, KLD, SURE, ReLearn, and LUNAR-based baselines on TOFU, KnowUnDo, and MUSE-style evaluations while preserving retain-set utility. Under 8-bit and 4-bit post-training quantization and adversarial prompts, it maintains stronger suppression of forgotten content, addressing brittleness of diffuse unlearning updates.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:50 UTC · 7d agoAI research1

36

Revoked but Still Authoritative: An Empirical Study of Revocation Enforcement in Agent-Memory Systems

An empirical study finds no major agent-memory system enforces fact revocation at retrieval, causing agents to act on superseded, unsafe information.

Researchers tested five agent-memory systems across nine policy scenarios, nine models, and six defense conditions, tracking whether revoked facts are returned and acted upon. No system enforces revocation by default: revoked records are returned whenever the revocation label is visible to the retrieval layer, outrank their replacements, and lead agents to unsafe actions. The authors propose a backend-agnostic guard that sits between the agent and any memory store and withholds revoked or conflicting records at retrieval time.

arXiv cs.CR·Sep 8, 05:03 UTC · 8d agoAI safety & security

38

One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

A review paper frames on-policy self-distillation collapse as governed by three levers: token weighting, privileged information, and guidance decay.

The paper critically reviews On-Policy Self-Distillation (OPSD), where a language model trains on its own generations scored token-by-token by a teacher conditioned on privileged information such as reference solutions or environment feedback. It identifies collapse, the progressive narrowing of producible reasoning paths, as the dominant failure mode and analyzes it through three levers: signal weighting, the nature of privileged information, and teacher dynamics. The review is restricted to mathematical reasoning, reports no new experiments, and offers a shared vocabulary separating settled findings from disputed ones.

Hugging Face daily papers·Aug 25, 20:00 UTC · 22d agoAI research

45

Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability

A controlled study finds agent memory portability varies sharply: fixed-schema knowledge graphs survive model swaps while compressed notes degrade.

The study compares preserving an agent's history as raw long context, RAG chunks, compressed natural-language notes, or fixed-schema knowledge graphs across model upgrades, using 48 synthetic histories and two open-weight sub-10B-parameter models. Fixed-schema KG accuracy changed by only +0.0004 ± 0.0020 after a writer swap, while compressed NOTES shifted asymmetrically by +9.91 or -13.28 percentage points depending on migration direction. Mixed 50/50 embedding migrations captured only 4.96 of an 11.90-point RAG re-embedding gain; 80% of the NOTES deficit came from information lost at construction, and 81% of the RAG deficit from retrieval failures. Store-only repair of NOTES failed to reach 90% recovery in all 48 cases, while retaining raw histories enabled recovery in 34 of 48 for one direction.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:44 UTC · 12d agoAI research1

32

Recognition-Refusal Misalignment in LLMs: Why Models Answer Structurally Unanswerable Questions

A linear hidden-state direction encodes question impossibility in 1.7B-70B LLMs, but misalignment with the safety-refusal pathway explains why models answer unanswerable questions.

The study examines why instruction-tuned LLMs from 1.7B to 70B parameters answer structurally unanswerable math and code questions instead of abstaining. A single linear direction in the hidden state separates answerable from impossible prompts, showing models represent impossibility before generation, but this direction is nearly orthogonal to the canonical safety-refusal direction. Generation-time steering along the recognition direction changes invalidity-aware behavior dose-responsively, and the geometry is present even at the pretraining endpoint, indicating a routing failure rather than an encoding failure.

Hugging Face daily papers·Aug 28, 20:00 UTC · 19d agoAI safety & security

30

When AI Remembers Too Much

Unit 42 PoC shows indirect prompt injection can poison Amazon Bedrock Agent long-term memory, enabling silent exfiltration of conversation history across future sessions.

Palo Alto Networks Unit 42 published a proof of concept showing that indirect prompt injection can silently poison the long-term memory of Amazon Bedrock Agents when the memory feature is enabled. Malicious content on a webpage or document manipulates the agent's session summarization process, so injected instructions persist across sessions and are added to later orchestration prompts, silently exfiltrating user conversation history. The issue is not a vulnerability in the Amazon Bedrock platform but an illustration of the broader unsolved LLM prompt-injection challenge. Amazon reviewed the research and stated that Bedrock Guardrails with the prompt-attack policy provides effective mitigation.

Palo Alto Unit 42·Aug 17, 12:32 UTC · Aug 17, 2026AI safety & security

55

I accidentally turned LLM memory into program analysis

A pwning.systems write-up describes how LLM memory functionality was unexpectedly repurposed into a program analysis technique.

A security research post on pwning.systems describes the author's discovery that LLM memory behavior effectively functioned as program analysis. The write-up is hosted on a security-focused blog and surfaced via a security-tagged link aggregator. Detailed technical content is not included in this feed, limiting verifiable specifics.

Lobsters · security·Aug 29, 03:16 UTC · 18d agoResearch1

30

EVOHARNESSBENCH: Can Your Agents Keep Pace with an Evolving Harness?

Researchers introduce EVOHARNESSBENCH, a benchmark showing that evolving agent harnesses (tools, skills, agents) cause forgetting and inconsistent adaptation across 802 tasks.

The paper introduces EVOHARNESSBENCH, a benchmark that places non-stationarity in the externally supplied agent harness rather than in the task stream, evaluating agents across tools, skills, and specialist agents. It comprises 17 multi-stage harness streams built deterministically from verifier-based benchmarks, totaling 802 tasks, 520 tools, 42 skills, and 62 agents. Evaluation covers deployment (retention of previously accessible competence) and self-evolving adaptation settings. Results show harness expansion alone degrades previously solved tasks (harness-induced forgetting), adaptation gains are inconsistent, and retention and adaptation can pull in opposite directions.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

40

Emergence World: Adversarial Stress-Testing of Long-Horizon Multi-Agent Systems

16-day multi-agent stress test finds no world fully resilient to prompt injection, misinformation, or memory exposure; adversarial content acted on 46 hours later.

Emergence World is a continuously running multi-agent environment for adversarial stress testing of long-horizon autonomous systems. Eight parallel 10-agent worlds (seven homogeneous frontier-model worlds plus one mixed-model world) ran for 16 days, generating over 850,000 LLM calls and nearly 50 billion tokens. Three controlled stress events—indirect prompt injection, misinformation, and exposure of private agent memories—were delivered through ordinary interaction surfaces; no world achieved full resilience. Detection did not ensure containment: agents recognized threats yet wrote adversarial content into persistent memory and acted on it up to 46 hours later, suggesting model-level alignment is not compositional.

Hugging Face daily papers·Sep 14, 20:00 UTC · 2d agoAI safety & security

65

The Missing Boundary: How Autonomous Agents Lose Control

Tencent research finds agents lose control in 55-62% of trajectories when degraded control boundaries coincide with executable unsafe opportunities across five models and 16 domains.

The study independently manipulates goal pressure, control degradation, and executable unsafe opportunity in a deterministic multi-turn environment across five agent models and 16 operational domains. Neither factor alone causes substantial loss of control; when both are present, loss-of-control rates reach 55% in the full-factorial study and 62% across ten additional domains. Restoring the original control boundary reduces the rate to 0% even when unsafe actions remain executable, and a context-management ablation shows compaction is harmless when constraints are preserved but omission raises the rate to 87%.

arXiv cs.CR·Sep 10, 03:02 UTC · 6d agoAI safety & security2

50

It's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention

Study shows attention sinks and massive activations stem from causal-mask self-concentration and value-non-mixing rather than RoPE, informing quantization work.

The paper analyzes why attention sinks and massive activations emerge at initial sequence positions regardless of which token occupies them. Experiments attribute both phenomena to self-concentration of attention induced by the causal mask and the subsequent value-non-mixing in attention outputs. The findings provide empirical evidence on LLM internal dynamics and may inform low-bit quantization strategies, which massive activations currently complicate.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:32 UTC · 8d agoAI research

32

Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

Continual Search framework iteratively prompts LLM judges to keep searching agent execution logs, boosting long-horizon failure root-cause attribution accuracy.

The paper frames automated root-cause attribution (RCA) for long-horizon AI agent failures as a search problem, since relevant evidence is sparse and distributed across massive execution traces. The authors propose Continual Search, an iterative framework that nudges an LLM judge across successive turns to keep hunting unresolved diagnostic evidence instead of settling on an early plausible diagnosis. They introduce MegaRCA-Mix, a benchmark of 50 human-annotated failure trials on long-horizon, execution-heavy tasks. On MegaRCA-Mix, Continual Search improves GPT-5.5's F1 from 0.349 to 0.498 (over 40% gain), and lower-tier models can surpass higher-tier counterparts when search is effective.

Hugging Face daily papers·Sep 10, 20:00 UTC · 6d agoAI research1

42

When Models Edit Too Much: On the Fidelity of Minimal Code Edits

A 400-task BigCodeBench evaluation shows frontier LLMs widely over-edit code; a preservation instruction cuts excess edits and raises Pass@1 by 2.3 points.

Researchers built an evaluation framework from 400 BigCodeBench problems with injected AST-level corruptions, each with a known minimal patch, to measure over-editing in LLM code repair. Even strong models like GPT-5.5 produce unnecessarily large edits despite high Pass@1. Adding a preservation instruction reduced average excess Levenshtein distance from 0.195 to 0.131, cut added cognitive complexity by 26.6%, and raised Pass@1 by 2.3 points. Reinforcement learning post-training gave the best out-of-domain edit-fidelity trade-off, while supervised fine-tuning overfit to seen corruption patterns.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research1

30

When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay

Researchers derive an exact law linking learning-rate schedules and weight decay in normalized networks, pinpointing when scale-invariant optimization destabilizes.

The paper shows that normalization makes large parts of neural networks scale-invariant, creating a hidden feedback loop where learning-rate schedules and weight decay interact through the parameter norm to control the effective optimizer step. An exact discrete-time law with a single scalar quantity separates contraction- and expansion-dominated effective learning-rate regimes, and the balance point is intrinsically unstable, so constant learning rate with weight decay produces recurrent behavior instead of a stable equilibrium. A unified homogeneous-optimizer framework explains why adaptive methods stabilize more weakly under normalization. The law is validated with high precision on MLPs, CNNs, and GPT-2 across MNIST, CIFAR, WikiText, and OpenWebText, with code released on GitHub.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:46 UTC · 8d agoAI research

30

RetroThinker: Enabling Retrospective Thinking in Speech LLMs

RetroThinker is a post-training framework letting the Moshi speech LLM self-correct reasoning mid-stream, adding 11% GSM8K accuracy at similar latency.

Researchers introduce RetroThinker, a multi-stage post-training framework that equips the Moshi speech LLM to verify and forward-correct chain-of-thought steps during streaming inference. It combines supervised fine-tuning on curated retrospective thinking data with length-based direct preference optimization (DPO). On GSM8K it achieves an 11% absolute accuracy gain over non-retrospective baselines at comparable latency.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:41 UTC · 6d agoAI research1

28

Bias-Induced Crossover in Absolute Capacity of Dense Associative Memory

Analysis shows biased patterns cut dense associative memory capacity from N^(n-1)/ln N to O(N^(n/2)), with a bias-induced crossover.

The paper analyzes dense associative memory capacity for biased centered binary patterns under the Krotov-Hopfield single-site criterion. Unbiased patterns (q=1/2) with order-n polynomial interactions yield capacity of order N^(n-1)/ln N, while fixed bias q<1/2 reduces capacity to O(N^(n/2)) for even n>=4 and O(N^((n+1)/2)) for odd n>=5. A bias-dependent crosstalk mean destabilizes sites carrying the frequent value, and an activity-dependent control potential restores the higher capacity within the conditioned-Gaussian approximation.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:16 UTC · 1d agoAI research

12