Cloudflare trims 100TB of edge RAM by redesigning consistent hashing in Pingora
Discover how Cloudflare’s innovative redesign of consistent hashing in Pingora has freed up over 100TB of edge RAM, enhancing efficiency and performance.
Cloudflare has reportedly recovered more than 100TB of memory across its global edge footprint by changing the way its Pingora Backend Router (PBR) performs consistent hashing—specifically in the pingora-ketama component that drives request-to-origin selection for cacheable traffic. The engineering effort combined a more memory-efficient in-memory data representation, a math-backed reduction in the number of hash
Frequently Asked Questions
What does Cloudflare mean by “consistent hashing” in Pingora (pingora-ketama), and why does it matter for edge traffic?
Consistent hashing is a technique for mapping requests to backends so that when the backend set changes, only a small portion of mappings need to move. In pingora-ketama, this mapping decides which origin serves cacheable traffic, helping keep cache distribution stable. Stable mapping reduces churn, which can otherwise increase misses and load during scaling or failures.
How can changing consistent hashing “recover” up to 100TB of edge RAM—what exactly is being reduced?
The report attributes the savings to a more memory-efficient in-memory representation of the hash ring (how ketama stores routing points) and a math-backed reduction in how many hash points are used. Together, these changes shrink the data structures the edge needs to compute request-to-origin selection, cutting per-node memory footprints across Cloudflare’s global network.
Does reducing the number of hash points affect routing accuracy or cache behavior?
The key risk with fewer hash points is uneven distribution or higher remapping when the backend set changes. The engineering effort described indicates the reduction was guided by math, aiming to keep the distribution quality close to the prior design. The intent is to preserve routing stability so cacheable traffic remains well-balanced and cache hit rates don’t degrade meaningfully.
Will this redesign increase latency due to more computation during request handling?
Memory optimizations don’t automatically imply extra CPU cost, but any change in hashing logic could. The article’s framing suggests the work primarily targeted data representation and the size of the ring representation, not a slower algorithm. If ring lookup remains efficient, latency impact should be minimal, while memory savings reduce pressure on caches and runtime overhead.
How does Cloudflare ensure consistent behavior during deployments if the hash ring representation changes?
When hash ring logic changes, different versions could momentarily map requests differently, potentially shifting load and cache residency. Large-scale systems typically handle this by rolling deployments carefully and keeping mapping changes deterministic per configuration/version. While the exact rollout strategy isn’t detailed, the claim of massive recovered memory implies the change was made in a controlled way that avoids chaotic remapping.