How Zalando Built an In-Process Client-Side Load Balancer for One Million Requests per Second
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Detailed architecture of a client-side load balancer at scale is highly technical and actionable for backend engineers.
Zalando's engineering team built an in-process client-side load balancer for their Product Read API, handling 1 million requests per second with single-digit-millisecond latency across 25 markets. By reimplementing Skipper's consistent hashing algorithm (xxHash64, 100 virtual nodes per endpoint) inside the calling process, they eliminated dependency on the shared edge load balancer for high fan-out internal traffic, reducing Skipper's fleet from 50+ pods to 8 and cutting daily deployment costs from $450 to $110. The migration used N-ring fade-in over 30 seconds on a ^2.5 curve to avoid scale-up latency spikes, while a watch-based Kubernetes informer replaced polling for control-plane stability.