VisionHOPE: Visual Backbones as Self-Modifying Learning Systems
VisionHOPE introduces a self-modifying visual backbone where memory content and learning rules co-evolve per image, with competitive ImageNet, COCO, and ADE20K results.
VisionHOPE is presented as the first generic visual backbone formulated as a self-modifying learning system, building on Nested Learning with five coupled memories that store content, generate key/value representations, and govern learning rate and retention. The authors derive a stability-matched step-size control (soft cap on self-referential injection plus spectral clamp) and prove non-expansive memory dynamics along each scan. The model achieves competitive results on ImageNet-1K, COCO, and ADE20K using chunked directional scans for 2D feature maps. Code is released on GitHub.
- First visual backbone formulated as a self-modifying learning system with five coupled memories.
- Stability scheme (soft cap plus spectral clamp) provably yields non-expansive memory dynamics.
- Competitive results on ImageNet-1K, COCO, and ADE20K; code released publicly.
- Organizations
- VisionHOPE
From the original article
1 min read at huggingface.co
Visual backbones have evolved from Convolutional Neural Networks (CNNs) with local aggregation to Vision Transformers (ViTs) with global interactions, State-Space Models (SSMs) with input-dependent state transitions, and Test-Time Training (TTT) layers that adapt an inner learner while processing an image.
By Siran Peng, Tianshuo Zhang, Tianyu Fu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.