The Outbox Pattern: Reliable Messaging Without Distributed Transactions

Every distributed system eventually runs into the same problem: a service updates its database and needs to publish a message to the outside world. A user signs up — the users row is written and an email-welcome event should be published. An order is placed — the orders row is written and a “send this to fulfillment” event should flow. Two writes, to two systems, ideally both happening or neither. If one succeeds and the other fails, you’ve created an invisible bug.

The obvious solution — “just use a distributed transaction” — doesn’t work at scale. XA transactions across a database and a message broker exist, but they’re slow, fragile, lock across systems, and most modern brokers don’t support them. Saying “we just do both and hope for the best” produces bugs you’ll chase for years.

The outbox pattern is the elegant answer the industry converged on. It turns the two-write problem into one write plus a reliable relay. This post is the full tour: why the naive approaches fail, what the outbox pattern actually is, the two main implementation flavors, the operational realities, and the patterns that make it robust.

The dual-write problem

Start concrete. An order service:

1
2
3

def place_order(order):
    db.execute("INSERT INTO orders (...) VALUES (...)", order)
    broker.publish("orders.placed", order_event(order))

Four failure modes:

1. Both succeed. Good.
2. DB fails. Bad, but clean — broker publish never happens, the user sees an error, no harm done.
3. Broker fails after DB success. The order exists; nobody was told. The fulfillment service never processes it. The user’s email never sends. The order is stuck.
4. DB commit after broker publish. Swap the order — publish first, then insert. If DB insert fails, you’ve told the world about an order that doesn’t exist. Downstream systems look for it, don’t find it, get confused.

Case 3 and case 4 are the quiet killers. They happen rarely — maybe one in a million — but in a system doing millions of operations a day, they happen every day. The downstream inconsistency is hard to detect and harder to recover from.

“Just retry” isn’t a fix. A retry can still fail. A retry of the broker publish can succeed, but you’ve committed to “publish after commit,” which means the retry logic needs to survive service crashes between commit and publish. Now you’re building a durable queue on the side — which is exactly what the outbox pattern is.

The outbox pattern

The core idea: when a business operation produces a message, write the message to the same database in the same transaction. The database now atomically records both the state change and the intent to publish. A separate relay process reads unpublished messages from the database and publishes them to the broker.

1
2
3
4
5
6
7

def place_order(order):
    with db.transaction():
        db.execute("INSERT INTO orders (...) VALUES (...)", order)
        db.execute(
            "INSERT INTO outbox (id, topic, payload) VALUES (?, ?, ?)",
            uuid(), "orders.placed", json.dumps(order_event(order))
        )

One transaction. Either both rows exist or neither does. The outbox row is the durable promise that a message will eventually be published.

Separately, a relay process:

1. Reads unpublished outbox rows.
2. Publishes them to the broker.
3. Marks them published (or deletes them).

The relay can crash. The broker can be temporarily unreachable. The relay can be slow. None of that produces the “lost message” bug — the outbox row is durable until it’s successfully published. At-least-once delivery is the floor.

The outbox table

A reasonable shape:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

CREATE TABLE outbox (
    id UUID PRIMARY KEY,
    aggregate_type TEXT NOT NULL,
    aggregate_id TEXT NOT NULL,
    topic TEXT NOT NULL,
    payload JSONB NOT NULL,
    headers JSONB,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
    published_at TIMESTAMPTZ
);

CREATE INDEX outbox_unpublished_idx ON outbox (created_at)
    WHERE published_at IS NULL;

Key design points:

• id is the message ID. Use it as the idempotency key on the consumer side.
• aggregate_id allows message ordering guarantees per aggregate (critical if order matters).
• payload as JSONB is convenient but binary/protobuf is fine. The outbox doesn’t care about format.
• headers carries context — trace IDs, tenant IDs, event type version. Don’t couple this to the payload; downstream middleware cares about it.
• Partial index on unpublished rows — the relay’s hot query is “give me unpublished messages.” The index keeps that query fast even when the table has millions of published rows.

Some teams prefer a separate published_events archive table, keeping outbox lean. Others delete rows after a retention window. Either works; don’t keep published rows forever unless you genuinely need them for audit.

Two flavors: polling relay vs CDC

There are two main ways to implement the relay.

Polling relay

Simplest approach. A dedicated worker polls the outbox table:

1
2
3
4
5
6

SELECT id, topic, payload, headers
  FROM outbox
 WHERE published_at IS NULL
 ORDER BY created_at
 LIMIT 100
 FOR UPDATE SKIP LOCKED;

FOR UPDATE SKIP LOCKED is the key incantation for multi-worker polling — each worker grabs a different slice, nobody blocks. Workers publish the claimed messages to the broker, then:

1

UPDATE outbox SET published_at = NOW() WHERE id = ANY($1);

Delete instead of update if you don’t need the published archive.

Pros:

• Works on any database.
• No infrastructure beyond the existing DB and broker.
• Easy to reason about.
• Simple to operate — one stateless worker type.

Cons:

• Polling latency (even at 100ms poll interval, that’s 100ms added to end-to-end delivery).
• Database load from the poll query. On busy systems, this gets noisy.
• Each poll finds rows, claims them, publishes, updates — four DB operations per batch.

For low-to-medium throughput systems, polling is often the right answer. At 10–100 messages per second it’s barely noticeable. At 10,000 per second, you need to batch aggressively and tune the polling interval, but it still works.

CDC-based relay

The alternative: skip polling entirely. Use the database’s change-data-capture stream to observe inserts to the outbox table as they happen.

For PostgreSQL: logical replication via a slot. Debezium (Kafka Connect-based) or native pglogical subscribers read the replication stream, filter for outbox inserts, and publish to Kafka.

Postgres WAL
    -> Debezium connector (logical decoding)
    -> Filter: table = outbox
    -> Kafka topic: (routed by outbox.topic field)

Pros:

• Near-zero latency (milliseconds from commit to Kafka).
• No polling load on the database.
• Leverages the battle-tested replication infrastructure.
• Debezium’s outbox event router transforms outbox rows into first-class Kafka messages.

Cons:

• Operational complexity. Debezium + Kafka Connect + replication slots is a lot of moving parts.
• Replication slot gotchas. An offline connector causes WAL retention to grow unboundedly — can fill disk if not monitored.
• Strong coupling to Kafka (or whatever the CDC target is).

For systems already running Kafka at scale, Debezium + outbox is the industry-standard setup. Confluent and Red Hat have written extensively about it; Debezium’s outbox event router (io.debezium.transforms.outbox.EventRouter) is purpose-built for this pattern.

For non-Kafka shops, CDC gets more custom. PostgreSQL → NATS, PostgreSQL → RabbitMQ, PostgreSQL → SQS — all possible, mostly with home-rolled or smaller-community tooling.

Ordering guarantees

The outbox pattern gives you per-aggregate ordering, not global ordering — if you’re careful.

Per aggregate (same aggregate_id):

• Polling relay: ORDER BY created_at within the aggregate, one worker per aggregate partition, sequential publishing. Works.
• CDC relay: WAL preserves insert order. Debezium publishes in order. Works.

Globally:

• Polling relay with multiple workers: no, messages can be published out of order across aggregates (some workers are slower than others).
• CDC relay: strict commit order is preserved, which is a stronger guarantee than most systems need.

For most domains, per-aggregate ordering is what matters. OrderPlaced before OrderShipped for the same order is essential. OrderPlaced for order A before OrderPlaced for order B is irrelevant.

If you need strict global ordering, your options narrow:

• Single-worker polling relay (throughput ceiling).
• CDC relay (Debezium gives you this naturally).
• Revisit whether you actually need it (usually you don’t).

At-least-once delivery and idempotency

The outbox pattern provides at-least-once delivery. The same message may be published multiple times — if the relay crashes after publishing but before marking the row as published, the next relay run re-publishes. This is fundamental; exactly-once is not achievable in practice without consumer-side deduplication.

Consumer obligation: be idempotent. Process id=abc123 twice and produce the same end state both times.

Patterns:

• Deduplication table. Consumer records (message_id, processed_at); skip if already present.
• Idempotency in the business logic. INSERT ... ON CONFLICT DO NOTHING on a natural key. UPDATE ... WHERE status = 'pending' — conditional updates that only apply once.
• Check-and-set. Read current state; apply only if state matches expectation; no-op otherwise.

The producer’s outbox pattern is trivial compared to the consumer’s idempotency discipline. Teams adopting the outbox pattern without investing in idempotent consumers will have better delivery guarantees but still ship bugs when duplicates arrive.

Message formatting: events vs commands

What should go into the outbox?

Events — facts about what happened (OrderPlaced, UserSignedUp). Past tense. Multiple consumers. The producer doesn’t know or care who consumes.

Commands — instructions to a specific consumer (SendWelcomeEmail, ReserveInventory). Imperative. Usually one consumer.

Both work. Events scale better for loosely-coupled architectures — adding a new consumer doesn’t require the producer to change. Commands are sometimes clearer for orchestration scenarios where you want intent to be explicit.

Prefer events unless you have a specific reason to prefer commands. If you need to invoke a specific action, publish an event that carries enough context, and let the consumer decide what command it represents internally.

Schema and versioning

The outbox table doesn’t care about payload schema. The consumer does.

Treat outbox messages as a public API:

• Use a schema registry (Confluent Schema Registry, Apicurio, etc.) for Avro/Protobuf schemas, or JSON Schema for JSON payloads.
• Version messages explicitly (event_type, schema_version headers).
• Evolve additively — add fields, never remove or change meaning. If you need a breaking change, create OrderPlaced.v2 alongside.
• Consumer contract tests validate schema compatibility in CI.

A team that ships good outbox events treats them with the same rigor as a REST API contract. A team that treats the outbox as a private log ships breaking changes that cascade across every downstream consumer.

Handling poison messages

Some messages can’t be published. Maybe the broker is rejecting the schema, maybe the message exceeds size limits, maybe a bug in serialization. Without care, one bad message blocks all subsequent ones (the relay keeps retrying the same message forever).

Strategies:

• Error column. Track publish_attempts and last_error on the outbox row. After N attempts, move to a dead-letter state or a separate table for human review.
• Exponential backoff. Retry at increasing intervals rather than tight loop.
• Alerting. Messages in error state should page someone. This is not a state to let accumulate silently.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

ALTER TABLE outbox
    ADD COLUMN publish_attempts INT NOT NULL DEFAULT 0,
    ADD COLUMN last_error TEXT,
    ADD COLUMN next_retry_at TIMESTAMPTZ;

-- Relay claim query
SELECT * FROM outbox
 WHERE published_at IS NULL
   AND (next_retry_at IS NULL OR next_retry_at <= NOW())
   AND publish_attempts < 10
 ORDER BY created_at
 LIMIT 100
 FOR UPDATE SKIP LOCKED;

Outbox and the aggregate lifecycle

Subtle but important: the outbox write must be in the same transaction as the business write, or the guarantee breaks.

This sometimes surprises teams using ORMs that batch writes. If your ORM commits the business write before the outbox write (via autocommit or separate transactions), you have the same dual-write problem the outbox was meant to solve.

Code reviews should enforce this. Ideal pattern:

1
2
3
4
5

def place_order(order):
    with db.transaction() as tx:
        repository.save(tx, order)
        outbox.emit(tx, "orders.placed", event=order_event(order))
    # commit is the atomic boundary

Some frameworks make this seamless (Spring’s @TransactionalEventListener with publishAfterCommit, .NET’s MediatR with transactional behaviors, Rails’ after_commit callbacks paired with an outbox gem). Use them when available.

Tombstones and cleanup

The outbox will grow. Without management:

• Published rows accumulate. Even if you keep only unpublished rows by deleting on publish, busy systems have millions of rows in flight.
• Long-retained rows slow the partial index.
• Backup time grows.

Cleanup policy:

• Hard delete on publish — simplest, loses audit.
• Retain published rows for N days, then archive or delete — balance audit with storage.
• Partition the table by date — modern Postgres supports declarative partitioning. Drop old partitions rather than deleting rows.

Whichever policy, run cleanup as a scheduled job with bounded batch sizes. Don’t let it turn into a surprise long-running transaction that locks the outbox.

Transactional outbox vs other patterns

Worth placing in context:

Transactional outbox (this post): DB write + message in one transaction; relay publishes.

Listen-to-yourself: service publishes to broker; subscribes to its own events; writes to DB in handler. Reverses the problem — broker is source of truth. Works only if message delivery is truly reliable (Kafka with strong delivery semantics).

Change Data Capture without outbox: CDC the business tables directly. Downstream consumers interpret DB row changes. Fragile — downstream couples to schema — and DB changes ≠ business events (a single business event often spans multiple rows).

Saga orchestration: for multi-step workflows, combine outbox with a saga/workflow engine (Temporal, Cadence) that tracks step completion and issues next commands.

Event sourcing: the event log IS the source of truth, and projections drive reads. Outbox is one step short of this — you have events but still have a “current state” table as source of truth.

Implementing on Postgres

Practical Postgres-specific tips:

Use partial indexes. WHERE published_at IS NULL keeps the index small regardless of table size.

Set fillfactor on the outbox table. WITH (fillfactor = 70) leaves space for HOT updates when marking rows published. Saves index bloat on write-heavy tables.

Autovacuum tuning. High churn table; default autovacuum lags. Configure aggressive per-table settings:

1
2
3
4

ALTER TABLE outbox SET (
    autovacuum_vacuum_scale_factor = 0.01,
    autovacuum_vacuum_cost_delay = 10
);

Avoid long-running transactions. A transaction that inserts into the outbox and holds open blocks vacuum and grows bloat. Keep outbox-inserting transactions short.

Consider NOTIFY for low-latency polling. Instead of fixed-interval polls, have the insert trigger notify listeners:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

CREATE TRIGGER outbox_notify_trigger
    AFTER INSERT ON outbox
    FOR EACH STATEMENT EXECUTE FUNCTION notify_outbox();

CREATE FUNCTION notify_outbox() RETURNS TRIGGER AS $$
BEGIN
    PERFORM pg_notify('outbox_new', '');
    RETURN NULL;
END;
$$ LANGUAGE plpgsql;

Poller does LISTEN outbox_new and wakes immediately on insert. Near-zero latency without CDC. Still need to fall back to periodic polling for safety.

Observability

Critical metrics:

• Outbox lag — age of the oldest unpublished row. Primary health metric. Alert above a few seconds in steady state.
• Publish rate — outbox inserts per second vs publishes per second. If insert rate exceeds publish rate sustained, backlog is growing.
• Publish error rate — % of attempts that failed. Low persistent error rate is normal; spikes mean the broker or downstream is unhealthy.
• Poison message count — messages with publish_attempts >= threshold. Alert on any > 0.
• End-to-end latency — from business write to consumer ack. This is the user-visible metric.

Expose these as Prometheus counters/gauges. Grafana dashboard with a single “outbox health” row per service.

When not to use the outbox pattern

The outbox pattern solves a specific problem. It’s not free; use it where the cost is justified.

Don’t use for purely internal logging. If you emit structured logs to a log aggregation system, you don’t need exactly-once — logs are tolerant to loss and duplicates.

Don’t use when the broker is your primary datastore. If you’re fully event-sourced into Kafka with Kafka as source of truth, there’s nothing to dual-write against.

Don’t use if you’re okay with best-effort. Analytics events often are — losing 0.01% of impressions doesn’t break anything. Skip the outbox, publish directly.

Do use for: user-visible actions (emails, notifications), business-critical state transitions (orders, payments), cross-service coordination where missed messages cause data divergence.

A pragmatic rollout

For a team introducing the outbox pattern:

1. Pick one bounded context. Don’t roll it out everywhere at once.
2. Start with polling relay. Simpler; you can move to CDC later if throughput demands.
3. Use Postgres outbox table with partial index. Keep the design simple.
4. Treat messages as a public contract. Schema registry, versioning, docs.
5. Invest in consumer idempotency first. An outbox producer with non-idempotent consumers is worse than no outbox (you now have reliable duplicates).
6. Build observability before you need it. Lag and error metrics from day one.
7. Plan for poison messages. Have a process for handling them before one happens.
8. Partition or clean up regularly. Don’t let the table grow unbounded.
9. Move to CDC (Debezium) if you outgrow polling. It’s a real complexity increase; make sure you need it.
10. Celebrate: you’ve eliminated a whole class of silent bugs. This pattern routinely fixes “weird edge cases” that plague dual-write architectures.

The outbox pattern isn’t glamorous. It’s plumbing — durable, reliable, boring plumbing that makes the rest of the system possible. The teams I’ve seen that invested in it early spend almost no time on dual-write bugs; the ones that skipped it eventually built something that looks like an outbox but worse. Learn it once, apply it where it fits, and one whole category of distributed-systems failure mode goes away.