Lesson 59 — System Design III: Load Balancing

~25 minutes · Day 77 · system design

An interviewer can turn a tiny service into a scaling question with one sentence: “What happens when one server is not enough?” Load balancing is the first answer. It is not magic capacity; it is a routing layer that decides which healthy, interchangeable server should receive each request.

The invariant

A load balancer spreads requests across interchangeable servers; consistent hashing keeps most keys in place when a server joins or leaves the pool.

Keep those two halves separate. Plain load balancing is for interchangeable servers. Consistent hashing is for keyed work where moving every key would cause cache misses, session loss, or noisy data movement.

The script

Use the same seven-step system-design script every time: requirements → estimate scale → API → data model → high-level boxes → deep-dive the bottleneck → failure modes.

  1. Requirements. Route web requests to healthy app servers. Keep latency low. Survive one server failing. Allow adding servers without a risky deploy.
  2. Estimate scale. Pick a round number: 20,000 requests per second at peak, 10 app servers, mostly stateless reads. That gives a first-pass average of 2,000 requests per second per server before headroom.
  3. API. The client still calls normal HTTP endpoints like GET /feed or POST /checkout. The load balancer is invisible to the API contract.
  4. Data model. Keep app servers stateless when possible. If a request must stick to a cache shard or session owner, choose a stable key such as user_id, session_id, or cache_key.
  5. High-level boxes. Client → DNS → load balancer → healthy server pool. A health checker continuously removes bad targets from rotation.
  6. Deep-dive the bottleneck. The interesting question is selection: round robin, least connections, or consistent hash?
  7. Failure modes. Servers fail, health checks lie, one key becomes hot, one load balancer becomes a single point of failure, and deploys can drain too slowly.

The architecture diagram

Read the diagram left to right first, then study the ring. The left side handles ordinary stateless requests. The right side shows the special case: keyed routing where most keys should stay put as the pool changes.

Load balancer architecture with a consistent hash ring Clients reach DNS, then a load balancer, then a healthy server pool. A health checker watches the pool. Keyed traffic can use consistent hashing so only one arc of keys moves when a server joins. Clients phones, browsers DNS service name Load balancer choose a healthy target Server A Server B Server C interchangeable app pool Health checker remove bad targets keyed request: hash(user_id) A B C D consistent hash ring When D joins, only this arc moves.

The bottleneck deep-dive

The bottleneck is the load balancer's selection policy. NGINX documents several real policies, including round robin, least connections, IP hash, and generic hash with a consistent option. In an interview, name the policy and the trade-off.

The clean interview sentence: stateless traffic starts with round robin or least connections; stateful keyed traffic needs consistent hashing or another sticky-routing strategy.

Numbers that matter

FigureWhy it mattersSource
20,000 RPS / 10 servers = 2,000 RPS eachFirst-pass capacity math before headroom, uneven load, or failover.Interview arithmetic
One server lost from 10 means survivors take about 11% moreFailover needs spare capacity, not just healthy routing.Interview arithmetic
ALB health checks default to 30s for instance or IP targetsBad targets are not removed instantly; detection time affects errors.AWS docs
ALB idle timeout defaults to 60sLong quiet connections can close at the balancer before the app expects.AWS docs
Adding server 11 moves about 1/11 of keys with consistent hashingMost cache keys stay warm instead of causing a full reshuffle.NGINX docs

Practice — answer before you scroll past

1. A service is stateless and every server is identical. Which first policy is easiest to explain?

2. When a cache server joins a keyed pool, what does consistent hashing protect?

3. Long file uploads make connections pile up, but CPU looks fine. Which policy reacts better than plain round robin?

4. A load balancer marks slow servers healthy because /health only checks the process. What should change?

Your move (tonight)

Close this page and redraw the architecture diagram from memory on paper: clients, DNS, load balancer, healthy server pool, health checker, and consistent-hash ring. Then explain the whole design out loud in 5 minutes as if to an interviewer, using the seven-step script.

One primary source

Read NGINX — HTTP Load Balancing after your redraw, as the compare step. Focus on how each policy changes the answer to “which server gets this request?”


Stuck or curious? Ask your teacher (the agent) — that's what it's for.