FragToken: Amplifying LLM Inference Costs through Noncanonical Token Generation
FragToken trains LLMs to emit longer non-canonical tokens, roughly doubling inference cost while keeping answers usable.
1 stories, newest first · one card per story
FragToken trains LLMs to emit longer non-canonical tokens, roughly doubling inference cost while keeping answers usable.
Today 1.5K readers · 2.4K views · 756 reads · all time 189K views