BPE tokenizers can leak whether a dataset was in training
RTAM-Fusion uses rare-token mass and fewer shadow tokenizers to beat Vocabulary Overlap on a membership benchmark.
Stories per day · 30 days
4 RSS
2 stories, newest first · one card per story
Stories tagged #bpe
Objective vs. Search: Decomposing What Makes a Good Tokeniser
New tokeniser study shows search procedure, not optimisation objective, drives bits-per-byte performance across model sizes, vocabulary sizes, and multilingual settings.