Idempotent consumer: making a repeat harmless
At least once means some messages arrive twice. How does a consumer spot a repeat, what key does it trust, where does the record live — and which repeats can it still miss?
13 minMediumStation: Idempotent consumerMembers lesson
Membership isn’t open yet. The free lessons are open — 17 of them.
- Pixel's postsUsers
- Notify APIWeb API
- Send queueQueue
- Sender 1Worker
- Sender 2Worker
- Sender 3Worker
- Push provider's cap (300/s)Rate limiter
- Push providerService
What you'll be able to do
- 1Explain how a stable idempotency key and a record of handled keys, written together with the work in one transaction, turn at-least-once delivery into effectively-once processing — and why checking first and recording afterwards leaves a gap.
- 2Choose between making the work naturally idempotent and keeping a dedup record, and spot the side effects — a push, a charge — that also need the key passed to the outside service.
- 3Predict which repeats a deduplication window misses — a late redelivery, a dead-letter replay, a producer resend after the keys expired — and size the window from how long the broker can redeliver.
Inside the lesson
- steps
- 12
- questions
- 9
- simulations
- 3
You guess first, then run the system and see if you were right.
Step 1 · The setup
Notify kept at least once: its senders confirm a notification only after the push provider accepts it, so nothing is lost. The price came at the last stop — a lost answer, a queue's spare copy or a resend, and a user gets "Your order shipped" twice.
Losing pushes is worse, so the team keeps the repeats coming and makes them harmless instead. First, how bad is it? In this sim the queue hands 1 in 20 notifications out a second time — a high share, so you can see it.
Steps 2–12 are for members