Steering Geometry: Validating Human Value Geometry in LLM Steering Space
Researchers show LLM activation steering vectors encode human value geometry predicted by Schwartz's theory, with fidelity improving at scale but degrading after instruction tuning.
The study tests whether latent geometry of activation steering vectors reflects theory-specified structure in human values, using Schwartz's Theory of Basic Human Values and a new 26K-sample benchmark spanning 20 values. Distribution-driven methods (CAA, SphericalSteer, ODESteer) recover predicted value topologies with Spearman correlation up to 0.51 (p < 10^-13), while behavior-centric methods (COLD-Steer, BiPO) steer comparably but show little geometric fidelity. Geometric alignment improves with model scale but drops after instruction tuning. Better geometric alignment also yields more human-consistent cross-value transfer, steering one value lifting compatible values and suppressing opposing ones.
- Introduces 26K-sample benchmark covering 20 human values from Schwartz's framework
- Distribution-driven steering methods recover theoretical value geometry (Spearman rho up to 0.51)
- Behavior-centric methods (COLD-Steer, BiPO) steer well but lack geometric fidelity
- Geometric fidelity improves with model scale, drops after instruction tuning
- Aligned geometry yields human-consistent transfer across compatible and opposing values
- Vendors
- Hugging Face
- Organizations
- DeepRCL
From the original article
1 min read at huggingface.co
As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inference-time alternative to fine-tuning methods (e. g. , RLHF, DPO) for behavioral control.
By Mohammad Mahdi Abootorabi, Armin Saghafian, Ali Bazshoushtari · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.