Latency & percentiles
Latency & percentiles
Gold
Latency is how long one request takes. Percentiles (p50, p95, p99) describe the spread across many requests, and the slow tail is what users and calling services actually feel.
- Use when
- Setting latency targets and alerts · Comparing two designs under the same load · Spotting queueing before errors appear
- Avoid when
- Debugging one slow request — read its trace instead · Tiny samples, where p99 is mostly noise
- Trade-offs
- A tighter p99 target costs headroom: more capacity at lower utilization · Percentiles cannot be averaged across servers — merge histograms instead
3 objectives · 6 misconceptions
- explain Read p50, p95 and p99 from a latency distribution and say what each means for users.
- predict Predict how the tail grows when one request fans out to several backends.
- choose Choose percentiles over averages for targets and alerts, and explain why.
- Not true: The average latency describes what users experience.
- Not true: p99 only hurts 1% of users, so it can be ignored.
- Not true: Lower latency and higher throughput are the same thing.
- Not true: The maximum latency is the safest number to target.
- Not true: Percentiles from several servers combine by averaging them or taking the worst.
- Not true: A page that waits on many backends is about as slow as its typical backend.
Neighborhood
builds onnothing — a starting point
→Latency & percentiles→