Local Sparsity Enables Unsupervised LLM Safety Detection
Proposes locally masked sparse-autoencoder anomaly detection that flags unsafe LLM outputs unsupervised, using only 1-2% of SAE neurons.
The paper proposes an unsupervised deployment-time LLM safety method based on locally masked sparse-autoencoder (SAE) anomaly detection, avoiding the need for unsafe training data. Under the linear representation hypothesis, nearby activation points share a small common active support, making high-dimensional anomaly detection statistically feasible with theoretical justification. Validated across architectures and both capability-testing and safety-specific datasets, the method uses only 1-2% of SAE neurons and reaches near-optimal performance when calibrated with 1% out-of-distribution data.
- Unsupervised anomaly detection avoids assuming access to unsafe training data
- Local sparsity in SAE concept space justifies high-dimensional detection
- Uses only 1-2% of SAE neurons for computation
- Near-optimal performance with 1% out-of-distribution calibration data
From the original article
1 min read at arxiv.orgDeployment-time safety methods for large language models (LLMs) are predominantly supervised and assume access to unsafe training data. Nevertheless, new attacks and harm categories regularly arise, not captured by models trained in such a supervised fashion.
By Xin Chen · arxiv.org
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.