Generative Latent Priors interpret and steer protein models
Original title: Generative Latent Priors for Interpreting and Steering Protein Language Models
Diffusion models on ESM-2 activations serve as assumption-free priors for editing protein representations.
GLP learns activation distributions directly from frozen ESM-2 models at 8M and 650M scales. Single meta-neurons probe Swiss-Prot concepts better than InterPLM. On-manifold editing restores plausibility but multi-round decoding compromises structural validity.
- No structural assumptions imposed on activations
- Meta-neurons strongest probes of discrete concepts
- On-manifold correction makes biochemical steering effective
- Naturalness fails after several decoding rounds
- GLPs released on Hugging Face
From the original article
1 min read at arxiv.orgDominant methods for interpreting large models impose strong structural assumptions. Generative Latent Priors (GLP) are a complementary approach: a diffusion model that learns the distribution of a frozen model's activations directly, imposing no assumptions.
By Daniel Hinjos · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.