ZeroHour
DashboardNewsCVEsExploitsIncidentsAIBriefAnalyticsEntitiesWatchlistIOCsAsk the feedSourcesFeeds & APIKiosk mode

Search: “knowledge distillation”

7 stories in the last 30d

Searchkeywordsmart Sortlatesthotimportancerelevance
DomainAllCyberAIWindow24h3d7d30dAllIn the wildImportant onlyShow duplicates
CategoryAllAI safety & security 8AI research 7AI industry 5Threat actor 4Model release 4Advisory 2AI policy 1Exploit / PoC 1Policy & legal 1Research 1
SeverityAllcritical 1high 5medium 8low 0info 20

Label-Guided Knowledge Distillation for 3D-CNNs in Action Recognition

LGKD uses ground-truth labels to guide feature distillation for 3D-CNNs, combining sample-wise and class-wise distillation for action recognition.

The paper proposes Label-Guided Knowledge Distillation (LGKD) for 3D-CNNs, noting that most video feature distillation methods are simple adaptations of image techniques that neglect temporal-dimension differences. LGKD combines sample-wise distillation, which uses label information and the teacher's probability distribution to guide features impacting temporal accuracy, with class-wise distillation employing a prototype network to capture relational knowledge among same-category samples. Experiments on the UCF101 and HMDB51 action recognition benchmarks achieve competitive results.

arXiv cs.AI / cs.LG / cs.CL · 5d agoAI research

Lightweight Vision Transformer Compression for On-Device Plant Disease Detection in Resource-Constrained Agricultural Field Conditions

A unified ViT compression pipeline (H-BAC pruning, quantization, distillation) cuts plant-disease models 54.5x to 6.01 MB while keeping 95.13% accuracy.

Researchers combined Hessian-Balanced Adaptive Block Pruning (H-BAC), guided by second-order sensitivity estimation, with quantization and attention-based knowledge distillation to compress Vision Transformers for on-device chilli plant disease detection in India. On a 3-class cross-village, cross-device out-of-distribution dataset, the integrated pipeline reduced model size from 327.42 MB to 6.01 MB (54.5x) at 95.13 +/- 2.32% accuracy, matching the 95.13% FP32 baseline. Ablations also show a directly trained 6.01 MB INT8 student reaches 94.87% accuracy, indicating where pruning and distillation add limited value.

arXiv cs.AI / cs.LG / cs.CL · 12d agoAI research

OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining

OpenWAM releases an open modular stack for world-action model pretraining, plus OpenWAM-alpha trained on about 6,400 hours of egocentric and robot data.

OpenWAM is an open research stack that factorizes World-Action Model pretraining into composable infrastructure, study, and model components with unified training, inference, and evaluation. Controlled experiments distill three principles on knowledge inheritance, world-action synergy, and out-of-domain generalization gains from embodied co-training. The resulting OpenWAM-alpha, pretrained on roughly 6,400 hours of egocentric human and robot data, achieves top-tier results across eight simulation benchmarks and real-robot tests spanning single-arm, bimanual, and dexterous embodiments. The full stack, including infrastructure, evaluation protocols, pretrained models, and data recipes, is released openly.

Hugging Face daily papers · 10d agoAI research

Learning 3D Editing without Paired Supervision via Generative Prior Distillation

New framework distills 2D editing and VLM priors into a feed-forward 3D editing model without paired 3D training data.

The method, PriorEdit3D, learns feed-forward instruction-guided 3D editing by distilling knowledge from foundation models instead of using ground-truth 3D pairs. Through a differentiable rendering pipeline it supervises a 2D visual prior from an image editing model at the main view and a Vision-Language Model semantic prior at novel views for instruction fidelity and identity preservation. A 3D-aware Distribution Matching regularization constrains outputs to the manifold of realistic 3D assets defined by a pretrained image-to-3D teacher. Experiments report superior instruction fidelity and cross-view consistency over state-of-the-art baselines, with code released on GitHub.

Hugging Face daily papers · 13d agoAI research

OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning

OmniHarness learns symbolic policies for visual generation agents, reaching a 95.0% resolve rate on ComfyBench Creative tasks, 27.5 points above the strongest baseline.

OmniHarness abstracts verified executions into symbolic policies for visual generation task families, which are instantiated, adapted, and composed for new tasks while model parameters remain fixed. Intermediate verification guides refinement and failure recovery during execution, and self-directed inquiry generates practice tasks near capability limits before downstream objectives are specified. Experiments across six benchmarks, three MLLM backbones, and three visual agent frameworks show strong performance; on ComfyBench Creative tasks it achieves a 95.0% resolve rate, exceeding the strongest baseline by 27.5 percentage points. Frozen policy snapshots improve existing visual agent systems through plug-and-play reuse.

Hugging Face daily papers · 4d agoAI research

Revisiting Complete Reasoning Traces for Post-Training

Researchers show full reasoning traces provide limited benefit in LLM post-training, with heavily truncated or endpoint-only trajectories performing comparably.

A pilot study plus attention-based analyses and controlled token-removal studies show intermediate tokens in reasoning trajectories contribute minimally to final reasoning quality. Partial trajectories remain effective even under heavy truncation, and training on endpoints alone leads to consistent changes in reasoning behavior. The finding also benefits reinforcement-learning and on-policy distillation post-training; code is released at github.com/naver-ai/revisiting-trace.

Hugging Face daily papers · 10d agoAI research

Why don't machine learning research agents overfit?

Amazon researchers explain why ML research agents avoid benchmark overfitting, attributing generalization to compressibility of successful strategies.

Amazon Science summarizes the paper "What fits (into few tokens) doesn't overfit: Compression and generalization in ML research agents," which investigates why benchmark hill-climbing loops, whether run by human communities or LLM research agents, do not produce rampant overfitting. The explanation formalizes Occam's razor via a counting argument: successful ML strategies are highly compressible, so short descriptions lack room to memorize benchmark data and must capture real structure. LLM-based agents, being resettable and controllable, allow this hypothesis to be tested empirically.

Hacker News · security · 2d agoAI researchHN 21↑ · 3 comments