Server capacity & scaling up
Server capacity & scaling up
Gold
A server can only do so much work per second. As utilization nears 100%, requests queue and latency shoots up. A bigger machine buys headroom until the biggest one runs out.
- Use when
- Early systems, where one well-sized server is simple and cheap · Stateful parts that are hard to spread across machines
- Avoid when
- When losing that one server would be an outage you cannot accept · When expected load is beyond the largest machine available
- Trade-offs
- Every size up costs more per hour, and there is a largest size · Resizing usually means a restart, so the site is down meanwhile · One server is one point of failure · Running near full leaves no buffer for spikes
- Requires
- Latency & percentiles, Client–server & request/response
3 objectives · 9 misconceptions
- predict Predict how latency changes as utilization rises from 50% to above 90%.
- diagnose Find which resource — CPU, memory or connections — limits a server under load.
- choose Decide when a bigger machine is the right fix and when it hits its ceiling.
- Not true: Latency grows in a straight line with load.
- Not true: Running a server at 100% CPU is efficient.
- Not true: You can always buy a bigger server.
- Not true: When a server is slow, its CPU must be the limit.
- Not true: Moving a server to a bigger size happens with no downtime.
- Not true: A resource only slows requests down once it reaches 100%.
- Not true: A bigger server also makes the system more reliable.
- Not true: Only hardware — CPU, memory, disk — can be a server's bottleneck.
- Not true: An overloaded server keeps serving at full capacity and simply turns the rest away.
Learn
Server capacity & scaling upFree with an account~10 minPractice
Launch dayFree with an accountmediumCheck
Station test-out80% to passStories here
Neighborhood
→Server capacity & scaling up→
contrast withHorizontal scaling with a stateless tier