RTK reports token savings, but our cost benchmarks disagree
Quesma's $1,500 benchmark found RTK cuts reported token output but changes Claude Code and DeepSeek coding costs by only about 5% on Terminal-Bench 2.1.
Quesma benchmarked RTK (Rust Token Killer), a popular tool with 79k GitHub stars that filters terminal output for AI coding agents, whose README claims up to 90% output reduction. Across 1,740 Terminal-Bench 2.1 attempts running Claude Code with Fable 5.0 and OpenCode with DeepSeek V4 Pro 0813, total costs moved only -5% for Fable and +5% for DeepSeek, with pass rates dropping 1-2%. RTK's own rtk gain metric reported 349.2 million tokens saved (an 89% reduction) across 445 DeepSeek attempts, but this did not correlate with actual cost savings, and cached terminal-output reads cost as little as 1/10 to 1/30 of regular input tokens. A bug in rtk find 0.45.0 caused one agent to loop with 339 consecutive errors, costing roughly 9x the baseline attempt, though the task still passed.
- Over $1,500 in tokens spent across 1,740 benchmarked attempts
- RTK changed total costs by only about 5% in either direction
- rtk gain counts removed bytes, not billed tokens or money saved
- Read, Grep, and Glob tool calls bypass RTK's shell rewriting
- An unsupported find flag triggered 339 errors and a 9x-cost attempt
- AI models
- Fable 5.0DeepSeek V4 Pro 0813
From the original article
6 min read at quesma.com
RTK (Rust Token Killer) filters and compresses terminal output before the AI agent reads it. With over 79k GitHub stars today, RTK is one of the most popular tools to make AI coding cheaper. One X post saying RTK could cut Claude Code tokens by up to 60% reached 313K views.
By michalwarda · quesma.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.