SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
SAEScientist-Bench evaluates whether AI agents can autonomously conduct SAE interpretability research in Gemma-2-9B-IT, finding frontier agents trail expert baselines.
SAEScientist-Bench tests if AI agents can act as scientists using SAE tools for autonomous mechanistic discovery, requiring them to design contrastive probes and navigate a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT. Across 10 agent configurations and 20 tasks, frontier agents showed genuine discovery capability but remained well behind expert reference features, lagging most in causal steering. Agents frequently misinterpreted experimental measurements even when designing effective contrasts.
- New benchmark targets autonomous mechanistic interpretability research
- Agents navigate 131K+ SAE features in Gemma-2-9B-IT
- Frontier agents approach expert level on contrastive separation but lag on causal steering
- Code publicly released on GitHub
- Vendors
- GoogleHugging Face
- Products
- Gemma ScopeNeuronpedia
- AI models
- Gemma-2-9B-ITSAEScientist-Bench
From the original article
1 min read at huggingface.co
While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment.
By Yuqiao Tan, Shizhu He, Jun Zhao · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.