Exactly-once semantics

What exactly-once really is

• Delivery: The message will be delivered to the consumer exactly once.
• Processing: the final side effect (writing to the database, changing the balance, issuing another event) will happen exactly once, even if there were more deliveries or attempts.

In distributed systems, it is more reliable to talk about processing semantics. Delivery exactly once is difficult (duplicates and replicates are inevitable), but the resulting state can be made to be equivalent to a single treatment.

When EOS is needed and not

• Cash transactions and balances: Double write-offs are not allowed.
• License/quota accounting, billing counters.
• Irreversible external calls (for example, one-time key activation).

• The effects are reversible or compensable (sagas, returns).
• Temporary duplicates are allowed in storefronts/logs.
• It is cheaper to provide an idempotent sink than to drag transactions through the entire path.

Model: end-to-end vs. hop-by-hop

Hop-by-hop EOS: each section (source → processor → receiver) guarantees that it will apply its action exactly once.
End-to-end EOS: The whole chain ensures that from "fact" to "side effect," the result is equivalent to a single treatment.

In practice, end-to-end is achieved by a combination of transactions and/or idempotency on each hop.

Basic building blocks

1. Idempotent operations

Repeating the same query on the operation key produces the same result.

Ключи: `idempotency_key`/`event_id`/`operation_id`.
Implementation: table of "seen" operations with TTL ≥ presentation of the input log.

2. Read-process-write transactions

In one atomic unit of work, both the side effect and the reading progress (offsets/position) are recorded. This eliminates "ghosts" when falling between steps.

3. Versioning/SEQUENCE

A version/counter is stored for the aggregate; changes apply only if 'expected _ version' matches. Repetitions of the same event do not increase the version → effect once.

4. Deduplication

Index on '(consumer_id, event_id)' or on the natural 'business _ id' of the transaction.

Implementation patterns

1) Transactional log + transactional sink with offset fixation

Ideal for stream processing.

We read from the log (only confirmed entries).
We carry out processing.

• a) write the effect in sink (database/table),
• b) fix "read to offset N" (in the same database).
• Commit. When restarting, either everything is discounted (and the offset is shifted), or nothing.

Properties: repeated execution is not harmful; "exactly once" in effect, even if the message was read twice.

2) Outbox + idempotent consumer

For transactional producer services.

In one database transaction: change the domain record and write the event to the outbox.
The repeater delivers the event to the bus with the same 'event _ id'.
Consumers apply events idempotently (dedup by 'event _ id').

Properties: Producer ensures no fact is lost; consumers guarantee exactly one effect.

3) EOS in Kafka/Flink-like systems (conceptually)

Idempotent producer: protects against takes on sending retreats.
Producer transactions: a group of entries in topics + a consummer shift are committed atomically; readers use'read _ committed'isolation.
The processing side stores the state store and commits it along with the transaction.

Properties: Re-starting the store/drag does not lead to a double effect; duplicates "not visible" downstream.

4) Idempotent "sinks" via upsert/merge

Sink takes' operation _ id '/' event _ id'and executes' UPSERT... WHERE NOT EXISTS`.

The side effect (for example, accrual) is performed atomically with the check "has not already been applied."

Properties: cheap EOS method at the edge with storage, without distributed transactions.

Key Implementation Details

Operation IDs

Must be deterministic for repeats (do not generate a new UUID when retracting).
Have a stable scope (on the consumer/unit/system).

Deduplication Table

Колонки: `consumer_id`, `operation_id`, `applied_at`, `ttl_expires_at`.
Indexes on '(consumer_id, operation_id)'.
TTL ≥ the maximum repeat window (log retention + potential delays).

Optimistic competition

In the write model, store the version of the unit.
When applying an event/command, use'WHERE version =: expected '; duplicate will not increase version.

Order/Order

EOS is not "exactly the same order." Ensure consistency through the batch key (all aggregate events → one batch) and/or the'sequence 'comparison.

Idempotent external calls

For insecure methods (for example, HTTP webhooks to a third-party service), add 'Idempotency-Key' and require the partner to support it.

Frequent traps

EOS in only one place: if sink is idempotent, but you emit secondary events without idempotency, you will get "exactly many times" downstream.
Two commits: first in the database, then the offset commit in the broker - the fall between them creates duplicate effects.
Raw CDCs out: Changing the DB scheme breaks consumer idempotency.
Unstable keys: 'operation _ id' depends on time/random and changes during retray.

Cost and trade-offs

Latency: transactions/isolated reads → p95/p99 growth.
Overhead storage: deduplication tables, state stores, transaction logs.
Operational complexity: transaction timeouts, thread rebalance, stuck sessions.
Diagnostics: more states ("in kamit," "visible as read_committed", "rolled back").

Choose EOS pointwise: for critical aggregates and effects; cover the rest with idempotency and compensation.

Testing exactly-once

1. Fault-injection: the drop in the process between the steps "recorded the effect" and "recorded the offset."

2. Duplicates: download the same message 2-5 times, make sure of one effect.
3. Restarts and rebalances: stop/restart of workers, check for the absence of double processing.
4. Network flappies: mid-transaction timeouts, commit retry.

5. Load tests: queue growth → whether there is no degradation to "forever in transaction."

Mini Templates (Pseudo)

Idempotent sink with offset fixing

pseudo begin tx if not exists(select 1 from dedup where consumer_id=:c and op_id=:id)
then apply_effect(...) -- upsert / merge / add_one_time_action insert into dedup(c, id, applied_at) values(:c,:id, now)
end if update offsets set pos=:pos where consumer_id=:c commit

Command with unit version

pseudo begin tx update account set balance = balance +:delta,
version = version + 1 where id=:account_id and version=:expected_version;
if row_count=0 then error CONCURRENT_MODIFICATION commit

Safety and compliance

PII/PCI in deduplication tables: store a minimum, use tokens instead of raw data.
Audit: log 'operation _ id', 'trace _ id', outcome (APPLIED/ALREADY_APPLIED).
Storage policy: TTL on dedup tables, archiving of offsets/logs.

Anti-patterns

"Real exactly-once delivery": an attempt to exclude duplicates at the transport protocol level without effect idempotency.
Global distributed transactions for everything: XA/2PC through all services is fragile and slow.
Mixing of non-idempotent side effects (for example, e-mail sent before the offset commit).
Lack of operation keys: relying on the "uniqueness" of the payload.

Production checklist

• Each critical effect has an idempotent key.
• The offset/read position is fixed in one transaction with effect.
• Deduplication tables indexed; TTL ≥ log retention.
• Optimistic competition (version/sequence) is enabled for aggregates.
• Threads/Topics are read in "Comp Only" mode (if available).
• Duplicate and drop tests are present in the CI/CD.
• Dashboards: share of repetitions, failed transactions, blocking time, lags.
• Integrator documentation for'Idempotency-Key '/retries/timeouts.

FAQ

Can EOS be provided without transactions?
Often yes - through the idempotency of sink's (upsert/merge) and versioning of aggregates. Transactions simplify warranties but increase cost.

Does everyone need exactly-once?
No, it isn't. It's expensive. Apply pointwise where compensation is not possible/expensive.

How to associate letters/webhooks with EOS?
Buffer the notification before the commit, send after fixing the effect; store 'notification _ id' and make sending idempotent.

What is more important - delivery or processing?
Processing. Deliveries may be repeated; the final state must be correct and the only one.

Total

Exactly-once is about the correctness of the effect, and not about the absence of duplicates in the wiring. It is achieved by a combination of idempotency, atomic fixation of effect and reading progress, reasonable partitioning and versioning discipline. Apply EOS where the cost of error is unacceptable, and test its reality with falls and takes tests - no belief in transport.