Load shedding — turning the excess away
Past its capacity, a server that tries to serve everyone slows for everyone until requests time out. Turning the excess away fast and cheaply keeps the accepted requests in time, so more orders succeed, not fewer — and starting a little before full keeps each one quick and the server off the edge.
12 minMediumStation: Load sheddingMembers lesson
Membership isn’t open yet. The free lessons are open — 16 of them.
- Flaky payments
- Bulkhead
- Load shedding
- Backpressure
- Rate limiting
What you'll be able to do
- 1Explain why quickly rejecting the excess keeps more requests succeeding than trying to serve them all.
- 2Predict goodput and latency for a server pushed past its capacity, with and without load shedding.
- 3Choose where to shed (gateway, load balancer or server), when to start, and which requests to turn away first.
Inside the lesson
- steps
- 10
- questions
- 8
- simulations
- 3
You guess first, then run the system and see if you were right.
Step 1 · The setup
At the last stop, search's flooded server went down behind its wall. A bulkhead keeps a flood from spreading; it doesn't keep the flooded server up. This stop is about how a server protects itself.
Sale day again, for all of the shop server's traffic. It can take about 92 requests a second before it's past full; on a busy afternoon it gets 60, each answered in about 5 ms. At t = 10 a flash sale sends 150 a second. Autoscaling takes minutes to add servers — the rush arrives in seconds.
Steps 2–10 are for members
More on the Resilience Line
- Failure modes and availabilityOpen9 min
- Timeouts — capping the waitMembers9 min
- Retries — with backoff and jitterMembers12 min
- Circuit breakerMembers10 min
- Bulkhead — walls between workloadsMembers12 min
- Backpressure — a queue that says noMembers11 min
- Rate limiting — a budget for each callerMembers12 min