SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?
SAEScientist-Bench tests whether AI agents can autonomously run SAE interpretability research on Gemma-2-9B-IT; frontier agents trail expert baselines.
The benchmark requires agents to design contrastive probes and navigate the Gemma Scope dictionary of over 131K features in Gemma-2-9B-IT to discover optimal interpretable features, scored against expert-curated references on Neuronpedia via activation rank, concept selectivity, and causal steering. Across 10 agent configurations and 20 tasks, frontier agents demonstrate genuine discovery capability and approach expert levels at separating target concepts from controls, but lag substantially in causal steering and frequently misinterpret experimental measurements. The authors frame this as establishing experimental model understanding as a measurable capability for closed-loop autonomous AI R&D and post-hoc monitoring for recursive self-improvement.
- Evaluates 10 agent configurations across 20 tasks on SAE-based mechanistic discovery.
- Agents design good contrasts but frequently misinterpret experimental measurements such as steering.
- Search space is the Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT.
- Motivated by post-hoc monitoring and auditing for safe recursive self-improvement.
- AI models
- Gemma-2-9B-IT
From the original article
1 min read at arxiv.orgWhile research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment.
By Yuqiao Tan · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.