SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
SAS trains a gated sparse attention selector end-to-end with the language modeling loss, beating distillation-based sparsifiers on reasoning, long-context, and agentic tasks.