ttok 0.4
Simon Willison released ttok 0.4, a CLI token counter built on OpenAI's tiktoken library.
Simon Willison released ttok 0.4, a command-line tool that counts tokens using OpenAI's open-source tiktoken library. The update, the first in about two years, fixes a Click warning, refreshes continuous integration, and adds a --list-models command. ttok can be run with uvx, for example by piping a file into uvx ttok.
- ttok 0.4 fixes a Click warning and updates CI.
- A new --list-models flag lists tiktoken-supported models.
- Users can count tokens via uvx without installing the package.
Release: ttok 0.4 ttok is my CLI tool for counting tokens, using OpenAI's open source tiktoken library. It hasn't been in updated in a couple of years, but I finally fixed a Click warning, updated CI, and added a --list-models command to list available models. It works with uvx , so you can count tokens in anything like this: cat file.txt | uvx ttok Tags: projects , ai , openai , generative-ai , llms , tokenization
This source does not provide full text. Read it at simonwillison.net.