Black Friday

The brief

Black Friday at Checkout: 400 checkout pages a second. A load balancer spreads them over two zones, two large servers in each, and takes a server out after three failed checks. Each page asks the orders database and the recommendations service at once. At second 40 zone A goes dark with everything in it: its servers, the recommendations service (it only runs there) and the database’s primary copy. Keep checkout pages coming, every one with its order taken. A page without recommendations still counts — that’s the degraded mode, and it’s allowed; a page without its order doesn’t. In this sim a large server serves about 185 pages a second before it passes 95% busy, an xlarge about 375, and a server kept above 95% for 5 seconds goes down and stays down. You can resize servers, copy a zone’s server group (the copy stays in that zone), change how a page treats each service it calls, and change the database’s settings. Before you build: once zone A is gone, how busy is each server left in zone B?

What it checks

1. 1Predict whether the zones left after a failure can carry the load, from how many zones there are and how busy each one runs before the loss.
2. 2Explain how sorting dependencies into critical and optional lets a page keep its core job when an optional dependency fails, and why waiting on an optional call can still take the page down.
3. 3Choose how checks react — shallow checks for the load balancer, deeper checks to a central monitor, fail open when every server looks sick — and how a database primary fails over to a standby.

Brush up first

• Multi-AZ redundancy
• Graceful degradation
• Health checks and failover

More practice

1. Keep it aliveOpen8 min
2. Launch dayFree with an account20 min