Diffusion Models and Concept Formation
Paper argues diffusion models implicitly form Cobweb-like concept hierarchies, with a basic level emerging at intermediate noise levels.
The authors draw a formal correspondence between diffusion models and Cobweb, a classic incremental concept-hierarchy learner, noting both are hierarchical Bayesian density models with Gaussian prototypes. Modes of the diffusion model's noisy marginals form a hierarchy whose basic level sits at intermediate noise, where class identity commits. The correspondence is tested on MNIST and Fashion-MNIST via mode-finding. Diffusion is reframed as a cognitive model of concept formation.
- Noisy marginals of diffusion models mirror Cobweb probabilistic prototype trees
- Basic level of categorization emerges at intermediate noise levels
- Correspondence validated on MNIST and Fashion-MNIST via mode-finding
From the original article
1 min read at arxiv.orgHumans organize knowledge into a taxonomy of concepts with nested levels of abstraction and a \emph{basic level} at which people recognize and name objects with the least cognitive effort. Cobweb is a classic cognitive account of this ability, an incremental learner that builds a probabilistic concept hierarchy by maximizing category utility.
By Zekun Wang · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.