Z3 · Messaging & Events

Delivery guarantees: at most, at least, exactly once

Gold

A queue promises one of three things about each message. At most once — the consumer confirms before it works, so a crash in the middle loses the message. At least once — it confirms after the work, so nothing is lost but a crash, a lost confirm or a timeout runs the message again. Exactly once is what everyone wants, but between machines it is built, not given — at-least-once delivery plus a consumer that recognizes and drops repeats. So duplicates are normal, and the design question is what a loss or a repeat costs.

Use when: Deciding what a worker does when it crashes halfway — lose the message or do it again · Reading a broker's delivery promise before building on it · Work where a repeat is harmless or can be made harmless — most background jobs
Avoid when: Expecting the broker alone to make a payment or a push happen exactly once — that needs the consumer (core/idempotent-consumer) · Paying for at-least-once on data that is worthless a moment later, such as frequent sensor readings — at most once may do
Trade-offs: At most once is simple and never repeats, but a crash or a lost transfer drops the message · At least once loses nothing, but every consumer must cope with repeats · Broker exactly-once features cut duplicates but add latency, cover only part of the path and sometimes only one region
Requires: Competing consumers

3 objectives · 6 misconceptions

explain Explain at-most-once, at-least-once and exactly-once by when the consumer confirms a message relative to doing the work, and what each one loses or repeats when a worker fails.
predict Predict where duplicate deliveries come from and how many reach downstream — a lost confirm, a worker crash after the work, a hiding timeout shorter than the work, a producer resending after an unclear answer.
choose Choose a guarantee for a job from what losing it or doing it twice would cost, and read a broker's "exactly-once" feature for what it covers — the queue, not the consumer's side effects.

Not true: A message queue delivers every message exactly once, so a consumer never sees the same message twice.
Not true: Confirming a message as soon as it arrives, before doing the work, is the safe default.
Not true: A duplicate delivery means the broker is broken or misconfigured.
Not true: A producer that resends after a timeout can't create a duplicate, because the first send must have failed.
Not true: A broker feature labelled exactly-once also makes the consumer's side effects — a push sent, a card charged — happen exactly once.
Not true: Losing a message is never acceptable, so at-most-once is never the right choice.

Learn

Delivery guarantees: at most, at least, exactly onceMembers~13 min

Practice

Deliver onceMembersmedium

Check

Station test-out80% to pass

Stories here

Segment: delivering billions of messages exactly onceMembersstory

Neighborhood

builds onCompeting consumers

→Delivery guarantees: at most, at least, exactly once→

unlocksIdempotent consumer

On the pathChapter 12 · Deliver once