MixTok: Adaptive Visual Tokenization with Global 1D Tokens and Sparse 2D Refinement
MixTok mixes compact 1D tokens with sparse 2D refinement for budgeted visual tokenization.
MixTok is a hybrid visual tokenizer that pairs a compact 1D global representation with sparse 2D tokens for local refinement. A content-adaptive router ranks locations and chooses how many 2D tokens each image gets under a target average budget. On ImageNet 256x256 it reports rFID 1.195 and gFID 1.95 at roughly 128 and 131 content tokens per image.
- MixTok combines compact 1D global tokens with sparse 2D local refinement.
- A router sets the 2D token count from image complexity under a budget.
- ImageNet 256x256: rFID 1.195 near 128 tokens and gFID 1.95 near 131 tokens.
From the original article
1 min read at huggingface.co
Efficient visual tokenization requires balancing representation compactness with the preservation of fine-grained image content. Conventional grid-based tokenizers allocate tokens uniformly across spatial locations despite non-uniform visual complexity, whereas compact 1D tokenizers aggregate global information but can struggle to preserve local details.
By Chenge Li, Yefei He, Xinyu Liu · huggingface.co
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.