Resilience LineLesson
Retries — with backoff and jitter
A retry can turn a brief glitch into a success, but every retry is more load on a struggling service — so retry only safe, transient failures, a few times, waiting longer each time with some randomness, at one layer.
12 minMediumStation: Retries with backoff and jitterMembers lesson
Membership isn’t open yet. The free lessons are open — 16 of them.
- Failure modes & availability
- Timeouts
- Retries with backoff and jitter
- Circuit breaker
- Flaky payments
What you'll be able to do
- 1Choose which failures to retry — transient ones, on operations that are safe to repeat — and which to fail fast.
- 2Explain why waiting longer after each attempt, with random jitter, keeps retries from arriving in synchronized waves.
- 3Predict how retries at several layers multiply the load on a failing dependency.
Inside the lesson
- steps
- 11
- questions
- 11
- simulations
- 6
You guess first, then run the system and see if you were right.
Step 1 · The setup
At 7 pm, Checkout's payments service restarts and is down for 8 seconds. Checkout reacts the way a lot of code does: it sends each failed call again — right away, up to three times.
It sounds harmless. Each call gets more chances, and payments is only gone for a moment.
Steps 2–11 are for members
More on the Resilience Line
- Failure modes and availabilityOpen9 min
- Timeouts — capping the waitMembers9 min
- Circuit breakerMembers10 min
- Bulkhead — walls between workloadsMembers12 min
- Load shedding — turning the excess awayMembers12 min
- Backpressure — a queue that says noMembers11 min
- Rate limiting — a budget for each callerMembers12 min