Study rechecks where Platonic representation alignment appears in newer models
The capability-alignment trend holds when captions are clean, but same-modality alignment still far exceeds cross-modal scores.
Stories per day · 30 days
2 RSS
Relatedjepa4embeddings2imagenet2interpretability2language-models2mteb2multimodal2scaling-laws2agi-theory1alignment1astrophysics1backward-compatibility1brillm1byol1compositionality1compression1contrastive-learning1cross-architecture1
58 stories, newest first · one card per story
Stories tagged #representation-learning
Persona prompting reorganizes LLM latent representations
LLM persona adoption alters semantic axis projections in ideological directions.
Geometric Morphology Learning integrates geometry into foundation models
New framework enables AI models to learn geometric and topological structures for reasoning and action.
Assur modules improve limited-data learning of planar mechanisms
Assur-derived features give a geometric-mean 1.8× sample-efficiency gain over flattened adjacency and position inputs.
Paper turns the Platonic Representation Hypothesis into a JEPA theorem
A 109M-parameter text JEPA beat language models up to 72B on alignment with DINOv2 and CLIP image encoders.
TALOR rectifies open-set test-time features without backprop
On ImageNet-C with six csOOD datasets it beats UniEnt by 2.2% H-score while using 12% of the GPU memory.
Paper estimates ReLU-net mutual information from routing over linear regions
The routing term uses a plug-in estimator and, when well sampled, approximates a lower bound on joint information.
Factorized vector quantization matches continuous visual tokens on understanding
The discrete model beats UniAR and Emu3-Chat by 22.8% and 59.2% on a nine-metric average.
Theory: noise can shape what linear RNNs learn
In the rich regime, high noise can make a non-degenerate solution suboptimal when parameter scales are limited.
BriLLM-MetaPred implements MetaPred language modeling from AGI theory
One dynamical state is both context and prediction readout, with reported parity to a same-size GPT baseline.
Study measures how neural representations form during training
Path tortuosity, not endpoint displacement, predicted later drift and sensitivity to parameter perturbations.
I-JEPA predictor width decides which shared image factors stay readable
On 0.7M-parameter encoders, a half-width I-JEPA predictor scored about 40% of SimCLR, and matching encoder width reversed that at every seed.
OmniSpectra encodes spectra by wavelength instead of token rank
The 12.2M-parameter model, pretrained on eight surveys, beats seven baselines and larger models.
Sensorimotor Alignment proposed as offline proxy for robot policy success
The training-free S^2 metric correlates with closed-loop robot task success and can rank VLM backbones before VLA fine-tuning.
SCARCE learns label-free graph representations robust to distribution shift
A two-stage autoencoder confuses environment signals while preserving reconstruction and leads label-free results on DrugOOD.
Paper proposes simplex features for backward-compatible retrieval
The method needs no extra compatibility loss, learned map, or architecture change, including when classes are added.
JEPA pretraining underperforms masked models on MIMIC-IV EHRs
Three joint-embedding formulations trained without collapse but still lost to standard masked language modelling downstream.
Tessera learns direction-sensitive spatial transcriptomics representations
The method claims better representation quality than existing spatial foundation models and zero-shot use on tissues up to 16 times larger.
SpecMRL beats full-size MRL with half the embedding size
On MTEB, SpecMRL outperforms standard MRL at every tested reduced dimension, with larger gains under more aggressive compression.
Paper characterizes JEPA feature selection as predictive closure
Unpredictable features can still be encoded when they help predict other features.
SpecMRL improves compressed embeddings over Matryoshka prefixes
On MTEB, SpecMRL beats full-dimensional MRL while using only half the embedding dimensions.
Perspective maps a four-stage agenda for JEPA latent-world interpretability
Authors say existing work only covers grounding, while structure, causal checks, and intervention remain largely unexplored.
SimOPD: Relational Knowledge Transfer for On-Policy Distillation
SimOPD distills teacher knowledge through response similarities, gaining up to 3.1 points cross-architecture.
Generative Representation Learning for Scalable Latent Tokenization
A generative tokenizer avoids representation collapse and reaches FID-50K 2.18 with JiT-L.