Sale day

The brief

Sale day at Checkout. One medium shop server runs three web APIs: shoppers’ orders (10 a second, all day), shoppers’ searches (40 a second) and the product API a partner app calls (20 a second). At second 30 a sale banner goes up and searches jump to 200 a second. At second 90 a bug makes the partner app loop its calls: 240 a second, to the end of the run. Orders and the product API stay on the shop server, and it has to stay up the whole sale, for under $115 a month. A medium server handles about 92 requests a second, a large about 185; none that fits the budget can serve either flood. In this sim, a server kept above 95% busy for 5 seconds goes down and stays down, and above 80% it slows (up to 3× at 95%). Refused requests fail fast and don’t count as served. The API gateway from the load-shedding stop is billed per request — at these rates it would cost more than the whole budget — so here a rate limiter does the shedding: one shared cap for all searches — a fixed rate a second, the excess turned away at once.

What it checks

1. 1Explain why quickly rejecting the excess keeps more requests succeeding than trying to serve them all.
2. 2Choose where to shed (gateway, load balancer or server), when to start, and which requests to turn away first.
3. 3Choose a limit's key, rate and burst from tested capacity, and how a client should react to a 429.

Brush up first

• Bulkhead
• Load shedding
• Rate limiting

More practice

1. Keep it aliveOpen8 min
2. Launch dayFree with an account20 min