Load leveling
Notify's push provider takes 100 sends a second, and evening waves go well above that. What does a queue in front buy, and what does it cost?
14 minMediumStation: Load levelingMembers lesson
Membership isn’t open yet. The free lessons are open — 16 of them.
- Pixel's postsUsers
- Notify APIWeb API
- Push provider's cap (100/s)Rate limiter
- Push providerService
What you'll be able to do
- 1Explain how a queue lets a service with a fixed capacity, such as a provider with a send cap, work at its own steady rate — sized for the average, not the peak — and why a consumer that is down then only delays the work.
- 2Predict, from a run's rates, the backlog a peak leaves and when it clears — new work keeps arriving, so only the capacity left over from normal traffic works it off.
- 3Choose between leveling (accept all the work, finish it later) and refusing the excess (rate limiting, backpressure), from how long the work can wait and whether the average fits the capacity.
Inside the lesson
- steps
- 10
- questions
- 7
- simulations
- 4
You guess first, then run the system and see if you were right.
Step 1 · The setup
Pixel's uploads already answer in about 5 ms: its workers take each "notify followers" job from a queue. They hand the notifications to the notification service — now Notify, a service of its own — one call (a post) per notification; on a normal evening about 60 a second.
Notify passes each one to an outside push provider, which delivers it to phones. Like many such services, it caps how fast one customer may send: at most 100 sends a second; the rest are refused. Each evening a wave lifts posts to 160 a second for about 10 seconds. Today Notify sends each notification straight away and answers the post once the provider does.
Watch the evening wave.
Steps 2–10 are for members