Idempotent consumer

Since a message can arrive twice, the consumer makes the second time harmless. The easiest way is work that is naturally safe to repeat — setting a status rather than adding to a count. Otherwise every message carries a stable key for the operation, and the consumer keeps a record of the keys it has handled, written in the same transaction as the work, so a repeat is spotted and skipped. Side effects outside that transaction, like sending a push or charging a card, need the key passed on to the provider. Keys are kept for longer than the broker could still redeliver.

Use when

Consuming from any at-least-once queue or topic — the default for most brokers · A repeat would do harm — a second charge, a second order, a second push to a user · Several workers share one queue, so two copies can be handled at the same moment

Avoid when

The work is already naturally idempotent, such as an upsert by id — a dedup table only adds cost · An occasional repeat is cheaper than running the dedup store

Trade-offs

Natural idempotency costs nothing to run, but not every action can be phrased as a set or an upsert · A dedup record catches repeats, but needs a store, a transaction around the work, and a cleanup policy · A longer key window catches later repeats, but the store grows and every new message is checked against it · Broker deduplication removes producer resends cheaply, but only within its window and never consumer redeliveries

Requires

Delivery guarantees: at most, at least, exactly once