Z1 · Compute & Execution

Queueing & Little's law

Bronze

Requests arrive, wait their turn and get served. Little's law ties three averages together — the number of requests inside a system equals the rate they arrive times how long each one stays — for a queue, a server, a pool or a whole service. A queue only settles while arrivals stay below what its server can serve.

Use when: Estimating how many requests, threads or connections are busy at once from traffic and latency · Reading a queue's wait from its backlog and arrival rate · Sanity-checking a load test — the in-flight count, rate and latency it reports must agree
Avoid when: Predicting percentiles such as p99 — the law relates averages only · Short windows in which a backlog is building up or draining
Trade-offs: Holds for long-run averages whatever the traffic pattern, but says nothing about spread or spikes · Needs a stable system — arrivals below what the server can serve
Requires: Latency & percentiles

3 objectives · 8 misconceptions

explain Explain why the average number of requests inside a system equals their arrival rate times the average time each spends inside, wherever you draw the boundary — a queue, a server, a pool.
predict Work out the third quantity from the other two — requests in flight from rps and latency, or a queue's wait from its backlog — and predict how a slowdown at unchanged traffic changes it.
explain Explain utilization as arrival rate × service time, and why a queue only settles while utilization stays below 100%.

Not true: The number of requests in flight only goes up when traffic goes up.
Not true: Requests per second and requests in flight are the same number.
Not true: Little's law is about waiting lines; it doesn't apply to a whole server, a pool or a service.
Not true: Little's law only holds for particular traffic, such as random arrivals served first come, first served.
Not true: From Little's law you can work out the p99 latency.
Not true: If requests arrive a little faster than a server can serve them, its queue grows a bit and then levels off.
Not true: A server that serves requests exactly as fast as they arrive keeps a short, steady queue.
Not true: A request's time in a system is the time spent working on it.

Learn

Little's law — how many are inside~12 min

Practice

No katas practice this yet.

Check

Station test-out80% to pass

Neighborhood

builds onLatency & percentiles

→Queueing & Little's law→

unlocksConnection pools

On linesMachine Line