Huginn-style looped models beat Ouro at matched scale
A Huginn-style MoE uses 75% fewer resident parameters than a 112-layer feedforward baseline under matched FLOPs.
42
A Huginn-style MoE uses 75% fewer resident parameters than a 112-layer feedforward baseline under matched FLOPs.
Today 1.4K readers · 1.6K views · 308 reads · all time 110K views