ZeroHour

Search: “unlearning”

21 items

Forgetting Only What Matters: Layer-Selective Unlearning toward Robust LLMs

Researchers introduce FOM-UL, a layer-selective machine unlearning framework that improves forgetting-utility trade-offs and resists knowledge recovery after quantization.

FOM-UL selects transformer layers for unlearning using a forget-to-retain significance score, concentrating parameter updates on layers highly influential for the forget set while leaving most of the model unchanged. It reduces residual memorization versus GA, NPO, KLD, SURE, ReLearn, and LUNAR-based baselines on TOFU, KnowUnDo, and MUSE-style evaluations while preserving retain-set utility. Under 8-bit and 4-bit post-training quantization and adversarial prompts, it maintains stronger suppression of forgotten content, addressing brittleness of diffuse unlearning updates.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research1

Machine Unlearning as Private Retroactive Algorithms

A cs.CR paper defines private retroactive algorithms, showing machine unlearning is a data-maintenance problem and giving DP constructions for linear statistics, clustering, histograms.

The paper argues that machine unlearning's requirement to emulate retraining from scratch carries no meaningful privacy semantics against adversaries observing sequences of releases, recasting it as a data-maintenance question addressed by retroactive algorithms. It defines private retroactive algorithms, combining retroactivity with differential privacy under continual observation. Constructions achieve privacy and retroactivity at no asymptotic cost over privacy alone for linear statistics, clustering, and histograms, alongside impossibility results.

arXiv cs.CR · 11d agoResearch

Forgetting Without Restarting: Execution-State Unlearning for Stateful LLM Agents

Researchers propose provenance-guided selective replay letting LLM agents forget revoked information without restarts, matching full reset behavior.

The paper formalizes execution-state unlearning for stateful LLM agents, requiring that agents behave as if a revoked memory record was never observed across transcripts, compressed memory, tool plans, and KV caches. It proves exact unlearning requires at least T-τ+1 recomputed transitions and that Provenance-Guided Selective Replay attains this bound via a provenance graph, KV cache cropping, and sanitized replay. In audits across three agent suites, nine baselines, and three model families, memory deletion left leakage unchanged, instruction-based forgetting collapsed under elicitation (Leak@probes = 1.00), and selective replay matched full resets at up to 9x fewer recomputed tokens.

arXiv cs.CR · 12d agoAI safety & security1

Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning

Researchers show malicious federated learning clients can probe broadcast classifiers to recover deleted samples, exposing exact label leakage on MNIST and CIFAR-10.

The paper shows that federated unlearning systems broadcasting updated linear classifiers leak compact additive training summaries to clients. A malicious client can submit known changes, identify server states from returned classifiers, and compare states around an isolated deletion to expose the deleted sample, class, or client summary, potentially enabling reinsertion. On MNIST and CIFAR-10, high-precision broadcasts allowed exact label recovery for every tested deletion, while lower precision sharply reduced fine-grained recovery.

arXiv cs.CR · 12d agoResearch

Negative Self-Distillation: Learning to Reason by Avoiding Flaws

Researchers propose Negative Self-Distillation (NSD), a label-free LLM self-improvement method that diverges from self-generated flawed reasoning rather than imitating privileged solutions.

The authors show On-Policy Self-Distillation can degrade complex reasoning by forcing imitation of artificially confident traces built on privileged information, suppressing uncertainty and self-correction. NSD instead generates a question-specific negative condition — such as acting as a 'careless reasoner' — and pushes the model's distribution away from it without ground-truth labels. A dynamic gating mechanism isolates reasoning-critical tokens so gradient updates fix behavioral flaws without damaging foundational linguistic capabilities. NSD consistently outperforms OPSD and other label-free, self-bootstrapping reinforcement learning baselines.

Hugging Face daily papers · 6d agoAI research1