Saving another 100TB of RAM with math (and Rust)
Cloudflare optimized consistent hashing in pingora-ketama, reclaiming over 100TB of RAM globally across its Pingora-based load-balancing service.
Cloudflare's Performance team investigated excessive memory use in the Pingora Backend Router caused by its open-source consistent hashing library, pingora-ketama. Algorithmic and Rust-level changes to the hashing structures significantly cut the service's memory footprint, reclaiming more than 100TB of RAM across the global fleet. This follows a similar 100TB saving by the DNS team the previous month. The post explains the statistics behind hash ring balance, including expected value, standard deviation, and coefficient of variation.
- Excessive memory use traced to pingora-ketama structures in the Pingora Backend Router
- Algorithmic and Rust optimizations reclaimed over 100TB of RAM globally
- Follows a prior 100TB memory saving by Cloudflare's DNS team
- Post details probability of uneven hash ranges and how balancing fixes load skew
- Vendors
- Cloudflare
- Products
- Pingorapingora-ketama
- Organizations
- Cloudflare
From the original article
14 min read at blog.cloudflare.com
Cloudflare's global network is immense but not limitless. As we look for small ways to trim our resource usage, we sometimes get lucky and we can cut significantly more. Here’s how we reduced one of our Pingora-based service's RAM usage with statistics.
By Kevin Guthrie · blog.cloudflare.com
A short excerpt; the article and its images belong to the publisher. Summary, key facts and assessment above are AI-written from the article and can be wrong.