Z1 · Caching & Delivery

Memory vs disk vs network

Bronze

Reading from memory takes nanoseconds, from an SSD tens to hundreds of microseconds, from a spinning disk milliseconds — and a network round trip adds anything from a millisecond or two inside one region to well over a hundred between continents. The steps are not small, which is why a copy of hot data kept in memory, close to the code that reads it, pays off.

Use when: Estimating a request's latency on the back of an envelope · Deciding where hot data should live — the app's memory, a shared cache or the database
Avoid when: As exact figures — hardware and networks vary, so measure your own system
Trade-offs: The faster a tier, the smaller and costlier per gigabyte it is · Memory inside the app is quickest, but every server then keeps its own copy · A shared cache gives every server the same data, at the price of a network hop
Requires: Latency & percentiles

3 objectives · 6 misconceptions

recall Put memory, an SSD, a spinning disk, a round trip inside one region and one between continents in order of how long a read takes, each with its unit — nanoseconds, microseconds or milliseconds.
predict Estimate a request's latency by adding up its steps — memory reads, disk reads, network round trips — and name the step that dominates it.
choose Choose where a copy of hot data should live — in the app's own memory, in a shared cache one network hop away, or on local disk — weighing speed against every server holding its own copy.

Not true: Memory, disk and the network differ by a factor of two or three — the same league.
Not true: An SSD is fast enough that reading it costs about the same as reading memory.
Not true: Inside one data centre, a call to another machine costs about as much as reading local memory.
Not true: A shared cache server answers as fast as a cache inside the app's own process.
Not true: A database read is slow only because of its disk; the trip to the database costs nothing.
Not true: A faster server on another continent answers as quickly as a nearby one.

Learn

Memory vs disk vs network — the latency ladder~11 min

Practice

No katas practice this yet.

Check

Station test-out80% to pass

Neighborhood

builds onLatency & percentiles

→Memory vs disk vs network→

unlocksCache basics: hit ratio, TTL, eviction

On linesCache Line