ttok 1.0
Simon Willison releases ttok 1.0, switching the default tokenizer from GPT-4 to the GPT-5 family.
Simon Willison released ttok 1.0, a command-line token counter, after version 0.4 still defaulted to the GPT-4 tokenizer. The 1.0 default uses the GPT-5 family tokenizer, which he also applies to GPT-6. OpenAI has not officially confirmed that GPT-6 shares that tokenizer, but an experiment by William Liu found seven GPT 5.x and 6 models each reported 44,794 tokens and matched on all 31 fixtures.
30