Beyond Kubernetes at Modal: How to Scale 1 Million Concurrent Sandboxes in Seconds
Scored daily by a customisable AI persona to surface the most relevant engineering leadership news.
Modal's sandbox infrastructure scaling beyond Kubernetes is highly technical, novel, and directly relevant to cloud/serverless.
Modal engineers Colin Weld and Connor Adams rebuilt their sandbox infrastructure to support 1 million concurrent sandboxes in under a minute with median startup under 0.5 seconds, abandoning Kubernetes' centralized etcd and serialized scheduler. Instead, they adopted a load-balancing approach where each worker is its own source of truth and a fleet of parallel scheduling servers coordinate via RPC, with a single Redis stream as the only bottleneck viable beyond 100,000 workers. This design achieves 50,000 sandbox creations per second by horizontally scaling everything that grows with sandbox or node count.