The Outbox Pattern with Go and PostgreSQL

The problem

A service that creates an order usually needs to do two things: write the order to its own database, and tell the rest of the system it happened — publish an order.created event so the shipping service, the billing service, and the recommendation pipeline can react. The database and the message broker are two separate systems with no shared transaction between them, so there is no way to make both writes atomic by wrapping them in one BEGIN/COMMIT.

The naive approach writes to the database, then publishes:

The transaction commits — the order exists, the customer sees a confirmation — and then the publish call fails: the broker is briefly unreachable, the network blips, or the process is killed right after COMMIT returns but before the publish call goes out. The order is real. No downstream service ever finds out. The shipping service never ships it. Nobody gets paged, because nothing errored from the customer's point of view — the order API returned 201 Created. The event just silently never happened.

The failure is symmetric, not one-sided. Publish first and write second, and a different failure appears instead: the publish succeeds, then the transaction fails to commit. Now a downstream service reacts to an order that doesn't exist in the database at all. Neither ordering is safe on its own.

Why it matters

This isn't a rare edge case — it's a property of the architecture. Any service that writes to a database and publishes an event as two separate operations has this failure mode, regardless of how reliable the broker is, because "reliable" still isn't "transactional with your database." As event-driven systems grow — more consumers depending on more event types — a silently dropped event stops being a minor inconsistency and starts being a customer-visible outage that's hard to diagnose, because the symptom shows up in a completely different service than the one that has the bug.

The Outbox Pattern

The fix is to stop publishing directly from the request path at all. Instead, write the event to an outbox_events table in the same database transaction as the business write. Both rows succeed or fail together, atomically, because they're now just two inserts in one PostgreSQL transaction — no cross-system coordination required. A separate outbox worker then polls that table, publishes each pending event to the broker, and marks it published.

The atomicity guarantee moves from "database write and network publish happen together" (impossible) to "database write and outbox-row write happen together" (a normal transaction) plus "the outbox worker eventually publishes every row that exists" (a retry loop against a durable table, which is a much easier problem than a network call that must not fail).

Practical implementation

The outbox table

CREATE TABLE outbox_events (
    id             UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    aggregate_type TEXT NOT NULL,        -- e.g. "order"
    aggregate_id   UUID NOT NULL,        -- the order's ID
    event_type     TEXT NOT NULL,        -- e.g. "order.created"
    payload        JSONB NOT NULL,
    status         TEXT NOT NULL DEFAULT 'pending', -- pending | published | failed
    created_at     TIMESTAMPTZ NOT NULL DEFAULT now(),
    published_at   TIMESTAMPTZ
);
 
CREATE INDEX idx_outbox_pending ON outbox_events (created_at)
    WHERE status = 'pending';

The partial index on status = 'pending' keeps the worker's polling query fast even after millions of published rows have accumulated — it only ever scans the small pending slice.

Writing the order and the event together

func (r *OrderRepository) CreateOrder(ctx context.Context, order Order) error {
	tx, err := r.db.Begin(ctx)
	if err != nil {
		return fmt.Errorf("begin transaction: %w", err)
	}
	defer tx.Rollback(ctx) // no-op if Commit succeeds
 
	const insertOrder = `
		INSERT INTO orders (id, customer_id, total_cents, status)
		VALUES ($1, $2, $3, $4)
	`
	if _, err := tx.Exec(ctx, insertOrder, order.ID, order.CustomerID, order.TotalCents, order.Status); err != nil {
		return fmt.Errorf("insert order: %w", err)
	}
 
	payload, err := json.Marshal(orderCreatedEvent{
		OrderID:    order.ID,
		CustomerID: order.CustomerID,
		TotalCents: order.TotalCents,
	})
	if err != nil {
		return fmt.Errorf("marshal event payload: %w", err)
	}
 
	const insertOutboxEvent = `
		INSERT INTO outbox_events (aggregate_type, aggregate_id, event_type, payload)
		VALUES ($1, $2, $3, $4)
	`
	if _, err := tx.Exec(ctx, insertOutboxEvent, "order", order.ID, "order.created", payload); err != nil {
		return fmt.Errorf("insert outbox event: %w", err)
	}
 
	return tx.Commit(ctx)
}

If either insert fails, defer tx.Rollback(ctx) undoes both — there is no state where the order exists without its corresponding event, or vice versa.

The outbox worker

Multiple worker instances typically run for availability, which means they need to poll the same table without two workers publishing the same event twice. PostgreSQL's SELECT ... FOR UPDATE SKIP LOCKED handles this directly: each worker locks a batch of rows it's about to process, and any other worker's concurrent SELECT simply skips rows that are already locked instead of blocking on them.

func (w *OutboxWorker) processBatch(ctx context.Context) error {
	tx, err := w.db.Begin(ctx)
	if err != nil {
		return fmt.Errorf("begin transaction: %w", err)
	}
	defer tx.Rollback(ctx)
 
	const selectPending = `
		SELECT id, event_type, payload
		FROM outbox_events
		WHERE status = 'pending'
		ORDER BY created_at
		LIMIT 50
		FOR UPDATE SKIP LOCKED
	`
	rows, err := tx.Query(ctx, selectPending)
	if err != nil {
		return fmt.Errorf("select pending events: %w", err)
	}
	defer rows.Close()
 
	var events []outboxEvent
	for rows.Next() {
		var e outboxEvent
		if err := rows.Scan(&e.ID, &e.EventType, &e.Payload); err != nil {
			return fmt.Errorf("scan event: %w", err)
		}
		events = append(events, e)
	}
 
	for _, e := range events {
		if err := w.publisher.Publish(ctx, e.EventType, e.Payload); err != nil {
			w.logger.Error("publish failed, leaving event pending for retry",
				"event_id", e.ID, "error", err)
			continue // leave status = 'pending'; picked up again next poll
		}
 
		const markPublished = `
			UPDATE outbox_events SET status = 'published', published_at = now()
			WHERE id = $1
		`
		if _, err := tx.Exec(ctx, markPublished, e.ID); err != nil {
			return fmt.Errorf("mark event published: %w", err)
		}
	}
 
	return tx.Commit(ctx)
}

Locking the batch inside the same transaction that later marks rows published means a crash mid-batch simply rolls the lock back — the events stay pending and get picked up by the next poll, from this worker or another one.

Duplicate events and idempotent consumers

A pending event that fails to publish is just left pending — no separate "failed" state is required for a transient failure, since the next poll retries it automatically. A backoff between polls (rather than retrying a specific event immediately in a tight loop) is enough for most outbox workers; if a particular event needs its own backoff schedule because it fails repeatedly, the retry/backoff/jitter approach from Retries, Timeouts and Backoff in Go applies directly to the publish call inside the worker loop. A genuinely poisoned event (the broker rejects it as malformed, for example) should move to an explicit failed status after a bounded number of attempts, so it stops being retried forever and instead surfaces for manual inspection.

That retry is also why this pattern only ever promises at-least-once delivery, never exactly-once. A worker can publish an event to the broker successfully and then crash — or lose its database connection — before it commits the UPDATE ... SET status = 'published'. On restart, that row is still pending, and the worker publishes it again. This isn't a bug to chase down; it's the trade-off the pattern makes in exchange for never losing an event. The alternative — marking the row published before confirming the publish succeeded — trades a rare duplicate for a much more common silent loss, which is a worse trade.

Consumers need to treat a duplicate delivery as a normal case, using the same idempotency-key thinking covered in Idempotency in APIs — typically the outbox_events.id itself makes a natural idempotency key for consumers to deduplicate against, since it's stable and unique per event no matter how many times the row gets published.

Cleanup

Published events accumulate. Once a downstream consumer has had time to process an event, it doesn't need to stay in outbox_events — a scheduled job that deletes or archives rows with status = 'published' AND published_at < now() - interval '7 days' keeps the table (and its partial index) small indefinitely. Skipping this step is how outbox tables grow into a multi-million-row liability that nobody planned for.

DELETE FROM outbox_events
WHERE status = 'published'
  AND published_at < now() - interval '7 days';

Run it on a schedule, in batches if the table is large — a single unbounded DELETE against millions of rows will hold locks and bloat pg_stat_activity for longer than most operators are comfortable with. A table partitioned by created_at (monthly, say) turns this into a DROP TABLE on the oldest partition instead of a row-by-row delete, which is worth doing once volume justifies it.

Ordering considerations

Processing events for the same aggregate_id (the same order) in the order they were created is straightforward with a single worker: the ORDER BY created_at in the polling query combined with one worker processing rows sequentially preserves that order. The moment a second worker instance runs concurrently for scale, that guarantee weakens — worker A can be mid-publish on an older row while worker B picks up and publishes a newer row for the same aggregate first, because FOR UPDATE SKIP LOCKED deliberately lets workers grab different rows out of order rather than queue behind each other.

That's fine for most events, but not all of them: if a consumer needs order.created to be processed strictly before order.updated for the same order, per-aggregate ordering matters and a free-for-all pool of workers breaks it. The usual fix is to shard work by aggregate_id — hash the aggregate ID to a fixed number of partitions and let each worker own one partition exclusively — rather than have every worker pull from the same unpartitioned queue. Global ordering across every aggregate in the table is a much harder property to provide, and most consumers don't actually need it: they care about the sequence of events for one entity, not a total order across the entire system.

Production considerations

Outbox lag is the metric that matters. The single most useful signal is the age of the oldest unpublished row:

SELECT extract(epoch from now() - min(created_at)) AS oldest_pending_seconds
FROM outbox_events
WHERE status = 'pending';

Export that as a gauge and alert on it. A growing value means the worker is falling behind, stuck, or down — a rising row count alone can be misleading (a single stuck poisoned row skews the count without meaning the pipeline is unhealthy), but a rising oldest-pending-age always means something downstream is late to find out about real events.

Scale workers horizontally, not by polling faster. Running one worker with a short poll interval increases broker load and database contention without much upside; running several worker instances, each doing SELECT ... FOR UPDATE SKIP LOCKED LIMIT 50, lets them share the pending backlog safely and catch up in parallel when lag spikes. Keep the batch size (LIMIT 50 above) tuned to the publish latency of the broker — too large a batch held inside one transaction increases how long its locks are held; too small a batch means more round trips per event published.

When Kafka is appropriate vs. PostgreSQL alone

Kafka (or a similar broker) earns its operational cost when there are multiple independent consumers that each need their own read position, when events need to be replayed or retained for longer than "until the next service processes them," or when throughput is high enough that a polling worker against PostgreSQL becomes the bottleneck. For a small number of internal consumers reacting to moderate event volume, publishing straight from the outbox worker via PostgreSQL LISTEN/NOTIFY, or even having consumers poll the outbox table directly, avoids running and operating a broker before there's a real need for one — the outbox pattern's guarantees don't require Kafka specifically.

Common mistakes

• Publishing before committing the transaction. If the event is published from inside the request handler before the database transaction commits, a rollback afterward leaves a published event for something that never happened — this is the exact problem the pattern exists to avoid, reintroduced.
• Assuming exactly-once delivery. Consumers that aren't idempotent will eventually double-process an event; this is a "when," not an "if."
• Unbounded outbox table growth. No cleanup job means the table (and every index on it) grows forever, degrading both the worker's poll query and routine maintenance like VACUUM.
• A poisoned event retried forever with no failed state, silently consuming worker capacity on every single poll cycle indefinitely.

Testing

The transaction boundary is the entire mechanism this pattern depends on, so it deserves a test that checks it directly rather than trusting the code review: when the business write fails, its outbox row must fail with it, since both are inserts in the same transaction.

//go:build integration
 
func TestCreateOrder_FailedInsert_RollsBackOutboxRow(t *testing.T) {
	db := setupTestDB(t)
	repo := NewOrderRepository(db)
 
	orderID := uuid.New()
	valid := Order{ID: orderID, CustomerID: "cust_1", TotalCents: 4999, Status: "pending"}
	if err := repo.CreateOrder(context.Background(), valid); err != nil {
		t.Fatalf("seed order: %v", err)
	}
 
	// Reusing the same primary key forces the second INSERT INTO orders to
	// violate a unique constraint — after the outbox row for this attempt
	// would otherwise have been written inside the same transaction.
	duplicate := valid
	err := repo.CreateOrder(context.Background(), duplicate)
	if err == nil {
		t.Fatal("expected duplicate order ID to fail")
	}
 
	var count int
	err = db.QueryRow(context.Background(),
		`SELECT count(*) FROM outbox_events WHERE aggregate_id = $1`, orderID,
	).Scan(&count)
	if err != nil {
		t.Fatalf("count outbox events: %v", err)
	}
	if count != 1 {
		t.Fatalf("expected exactly 1 outbox row (from the first, successful "+
			"insert), got %d — the second attempt's outbox row should have "+
			"rolled back along with its failed order insert", count)
	}
}

If the order insert and the outbox insert ever end up in separate transactions — the single most damaging way to implement this pattern wrong — this test catches it immediately: the second attempt's outbox row would survive the failed order insert, and an event would exist in the table for an order that was never actually created.

Summary

The dual-write problem — a database write and a message publish that can't be made atomic together — is solved by making them atomic with each other instead: write the event to an outbox table in the same transaction as the business change, and let a separate worker handle the unreliable part (the actual network publish) with retries against a durable table. SELECT ... FOR UPDATE SKIP LOCKED lets multiple workers share that job safely. The trade-off you accept is at-least-once delivery, which means every consumer must be idempotent — that's not a workaround, it's the actual contract this pattern provides.