arXiv cs.AI / cs.LG / cs.CL·18d agoIt's Not RoPE that Creates Sinks: The Role of Self-Concentration and Value-Non-Mixing in Attention#attention-sinks#interpretability#llmAI research