Cloudflare frees 100 TB by redesigning 1.1.1.1’s DNS cache

Cloudflare says it recovered roughly 100 terabytes of memory across its fleet by optimizing how 1.1.1.1 stores DNS cache entries. The company detailed the work in a CloudFlare engineering post, explaining how small data-structure changes add up at extreme scale.

The Big Pineapple platform, which powers 1.1.1.1 and several other DNS services, holds more than 250 billion cache entries at any given time. At that volume, even one unnecessary byte per entry costs more than 250 GB of memory.

Cloudflare replaced growable Rust Vec and String fields with fixed-size Box<[T]> and Box<str> representations. Because cached responses are not modified after insertion, they do not need spare capacity. That change saved 64 bytes per entry and more than 15 TB overall.

The team also consolidated separate DNS record lists, replaced larger pointers and lengths with compact offsets, and packed boolean values into bitflags to reduce alignment padding. Together, five successive changes cut the per-entry memory footprint by more than 50%.

The optimization was not merely a capacity exercise. Cloudflare reports a 43% increase in insert throughput and a 19% reduction in lookup latency, helped by fewer allocations and improved memory locality. Its benchmarks modeled production traffic with 56% A records, 25% AAAA records, and 19% TXT records, then validated memory use on live production instances during rollout.

For teams building AI infrastructure, the lesson is broadly applicable: immutable data should not pay the overhead of mutable containers. At billions of objects, careful layout, allocation behavior, and padding can matter as much as algorithm choice—especially for caches, retrieval systems, and other high-volume services.

Source: CloudFlare