Saving another 100TB of RAM(blog.cloudflare.com)
475 points by f311a 25 days ago | 118 comments
tl;dr: Cloudflare reduced memory usage in its Pingora Backend Router by optimizing the consistent hashing implementation in pingora-ketama, which was consuming excessive RAM due to dozens of hash rings needed for different feature combinations. Two changes drove the savings: packing the Point struct from 8 to 6 bytes by shrinking the server index to 16 bits (25% reduction), and cutting hashes-per-server by 90% after math showed diminishing returns and 32-bit hash collisions actually worsened distribution at high counts. The migration was rolled out gradually per-datacenter to avoid cache invalidation, ultimately reclaiming 100TB of RAM globally.
HN Discussion:
  • ↑Appreciation for optimization work as a return to resource-conscious engineering
  • ↓Questions the approach and suggests alternative algorithms would be better than optimizing consistent hashing
  • ~Suggests further memory savings are possible with additional techniques like pointer compression
  • ↑Enjoyed the mathematical derivation and technical depth of the article
  • •Reflects on scale, noting 100TB is becoming less significant in modern infrastructure