Transactional Messaging

Transactional messaging is a set of architectural techniques that ensure consistency between local state changes (database/cache) and messages on the broker/bus. Purpose: "state is fixed ↔ message is not lost or duplicated" in case of failures, retrays, scaling and multi-tenancy.

1) Delivery semantics

At-most-once: Quick and cheap, losses possible, no takes.
At-least-once: does not lose messages, duplicates are possible → idempotency is required.
(Effective) Exactly-once: no loss and no visible takes for business effects, achieved by a combination of techniques (outbox/inbox, producer/consumer transactions, dedup).

2) Why "two-letter" is dangerous

Naive logic "first write to the database, then send to the bus" (or vice versa) breaks when falling between steps: the data is fixed, and the event is lost; either the event is gone, but there is no data. Transactional messaging bridges this gap.

3) Basic patterns

3. 1 Outbox (manufacturer)

In one local transaction, we write the business change and the row to the 'outbox' table; a separate publisher reads outbox and publishes in a broker with retras and backoff. Losses excluded; doubles are extinguished by idempotency among consumers.

3. 2 Inbox/Idempotent Consumer

Before executing the effect, the consumer makes'INSERT 'in' inbox (consumer, event_id) 'as the primary key. Key conflict = event already processed → skip. This is how "effectively exactly-once" is achieved.

3. 3 Read-Process-Write with Offset Transaction

Template for log-oriented buses: the consumer reads the batch, in the same transaction records the business change and the "passed offset." After the commit, the broker considers the messages consumed. This eliminates "read → fall → repeat" without duplicates in the effect.

3. 4 TSS/Sagas for interservice effects

When you need a consistent multi-step process, use TCC or sagas; messages - transport of commands/events, and transactionality - at the level of steps and compensations.

4) Idempotent producers and consumers

Producer: stable 'message _ id '/' idempotency _ key', resending with the same key does not create new effects for subscribers; maintain sequence by key.
Consumer: 'inbox '+ business idempotency (upsert/merge, check latest version/revision).

5) Order and causality

Participate by business key (for example, 'aggregate _ id', 'tenant _ id') so that the events of one object arrive in order.

Keep consecutive numbers/timestamps inside the lot; when redrawing from DLQ, observe "by key and sequentially."

If global order is not critical, ensure local order by key and fix domain invariants.

6) Offsets and fixing effects

Option A: "Offset in DB"

Write "last processed offset (partition, offset)" to the same transaction where you change domain data. When restarting, continue from the next offset, avoiding a repeated effect.

Option B: Broker Transaction

Some brokers support atomic recording of messages and offsets in one producer/consumer transaction. Use if available, but always supplement with idempotency on the consumer.

7) Retrai, backoff, DLQ

Repeat only retrable errors (timeouts, 5xx), with exponential backoff and jitter.
Non-retrable (schema/validation) - immediately in DLQ with metadata (tenant, key, offset, reason).
Dose the redrave from DLQ (batch, rate limit), check the circuit before repeating, observe the order by key.

8) Multi-tenancy and regions

Include 'tenant _ id', 'plan', 'region' in message metadata and partition keys.
Per-tenant fairness: Limit publishing/processing so that the "noisy" client does not deduct the budget from the rest.
Residency: store messages and outbox in the same region as domain data; interregional replications - asynchronous aggregates.

9) Observability and audit

Tracing: correlation 'event _ id '/' aggregate _ id '/' saga _ id', spans "read → process → write/commit."

Metrics: publishing/processing lag (p95/p99), success rate, DLQ-rate, redrive success, "duplicates suppressed."

Logs: short for success; details on errors (reason, attempt, key, offset).
Audit: who redrawn/rolled back, what batch and with what result.

10) Safety and compliance

Minimize PII in payload; Mask when transferring to DLQ/logs.
Sign/encrypt messages for external buses; use mTLS between services.
Manage shelf life and "right to forget" per tenant/region.

11) Typical integration schemes

1. Service source (write-side)

Local transaction: domain record + outbox.
Publisher: batches, 'SKIP LOCKED', backoff, limits per tenant.
Monitoring lag'now − occurred_at'.

2. Service-consumer (read-side)

Reading the batch → trying to 'INSERT inbox (consumer, event_id)' → if successful, we execute the effect.
In the same transaction, we fix the "passed offset" (option A) or rely on the broker's transaction (option B).
On error: retray or DLQ by policy.

3. Projection/Materialized View

Only idempotent updates (upsert), compact deduplication keys, periodic checksum verification.

12) Configuration templates (example)

yaml producer:
idempotency_key: event_id partition_key: "{tenant_id}:{aggregate_id}"
retry:
max_attempts: 8 initial_ms: 200 max_ms: 8000 strategy: exponential_full_jitter

consumer:
batch: 500 offset_commit: "with_domain_tx"  # или "broker_tx"
inbox_enabled: true concurrency_per_partition: 4 dlq:
enabled: true batch_redrive: 200 rate_limit_per_sec: 50 order_by_key: true

observability:
metrics:
- processing_lag_ms
- publish_success_ratio
- dlq_rate
- redrive_success_ratio tracing_tags: [event_id, tenant_id, aggregate_id, partition, offset]

13) Pre-sale checklist

• Eliminated "two-letter": outbox on the producer or fixing the offset and effect in one transaction at the consumer.
• Idempotent consumer: 'inbox '/dedup journal, business idempotency of operations.
• Partitioning by business key, local order is followed.
• Backoff + jitter retraces, error classification, metadata rich DLQ.
• Redrave dosed, safe; there are playbooks.
• Multi-tenant limits and priorities; 'tenant _ id/plan/region'tags.
• Telemetry: lags, success rate, "duplicates suppressed," alerts by p95/p99.
• PII/retention/encryption policies are enforced.
• Tests: drop between steps, duplicates, key order, mass redraw.

14) Typical errors

Sending to the bus and writing to the database in separate steps without an outbox/offset transaction.
A consumer without idempotency → duplicates side effects.
The global order "come what may" is expensive and rarely justified; enough order by key.
A massive redraw without limits → a secondary incident.

Lack of tracing/lag metrics → "hidden degradation."

PII mixing in DLQ/logs.

15) Quick recipes

SaaS events: Outbox + idempotent consumer (inbox), partitioning by 'tenant _ id: aggregate _ id'.
ETL/projections: Read-process-write with fixing offsets in one transaction, batches 500-1000, upsert.
High load: Publishing shardings, 'SKIP LOCKED', WFQ per tenant, lag control.
Strict compliance zone: regional outbox, payload encryption, retention and audit of redrives.

Conclusion

Transactional messaging is the discipline of connecting data and messages. By combining outbox/inbox, idempotency, offsets fixation along with effects and managed retrays with DLQ, you get practical exactly-once behavior without global locks and retain SLO even with crashes, peaks and complex multi-tenant exploitation.