Idempotency in APIs: Preventing Duplicate Payments and Requests

The problem

A client sends POST /payments. The server does exactly what it's supposed to: it charges the card, writes the payment record, and starts sending back a 201 Created. Somewhere between the server finishing and the client receiving that response, the connection drops — a load balancer times out, a mobile client loses signal, a proxy resets the socket. The client never sees a response.

From the client's point of view, this looks identical to a request that never reached the server at all. The correct, expected behavior for any well-behaved client is to retry. So it does — same endpoint, same body, same intent: charge this customer for this order.

The server has no way to tell "this is the same payment, retried" apart from "this is a second, separate payment the customer authorized twice." Without something extra in the request, it processes it again. The customer is charged twice.

This isn't a hypothetical. It's the default outcome of combining "networks are unreliable" with "clients retry on timeout" — both of which are true of essentially every production system — unless the server is explicitly built to recognize a retried request as the same request.

Why it matters

Anywhere a request has a side effect that costs money, sends something, or can't be trivially undone, an unprotected retry turns a transient network blip into a real, customer-visible incident: double charges, duplicate orders shipped, duplicate emails sent. These are also some of the hardest bugs to reproduce, because they only show up under real network conditions — they won't appear in a local dev environment where requests always succeed on the first try.

The fix has to live on the server. You cannot fix this by telling clients "don't retry" — retrying on timeout is correct client behavior, and disabling it just trades duplicate-charge bugs for dropped-payment bugs.

The idempotency-key approach

The client generates a unique token — typically a UUID — once per logical operation, and sends it on every attempt (including retries) of that operation as an Idempotency-Key header:

POST /payments
Idempotency-Key: 7c9e6679-7425-40de-944b-e07fc1f90ae7
 
{"orderId": "order_123", "amountCents": 4999}

The server's contract: the first time it sees a given key, it processes the request normally and stores the result against that key. Every subsequent request with the same key returns the stored result without processing anything again — regardless of whether the first attempt succeeded, is still in flight, or came from a genuine retry after a lost response.

That's the concept. The part that actually makes it safe is where the uniqueness is enforced.

Implementation: schema and request fingerprint

The idempotency record lives in PostgreSQL, not in application memory:

CREATE TABLE idempotency_keys (
    key             TEXT PRIMARY KEY,
    request_fingerprint TEXT NOT NULL,
    status          TEXT NOT NULL DEFAULT 'pending', -- pending | completed | failed
    response_status INT,
    response_body   JSONB,
    created_at      TIMESTAMPTZ NOT NULL DEFAULT now(),
    completed_at    TIMESTAMPTZ
);

key is the client-supplied Idempotency-Key, and it's the primary key — that's the constraint doing the real work, covered below.

request_fingerprint is a hash of the request body (e.g. SHA-256). It exists to catch a specific mistake: a client reusing the same idempotency key for a different request. Without the fingerprint, that would silently return the cached response from the first request — the wrong payment confirmation for the second one. With it, a mismatched fingerprint is a clear client error, not silent data corruption:

type IdempotencyStore struct {
	db *pgxpool.Pool
}
 
func fingerprint(body []byte) string {
	sum := sha256.Sum256(body)
	return hex.EncodeToString(sum[:])
}
 
var (
	ErrKeyInFlight       = errors.New("request with this idempotency key is already being processed")
	ErrFingerprintMismatch = errors.New("idempotency key reused with a different request body")
)

Handling the request

func (s *PaymentHandler) CreatePayment(w http.ResponseWriter, r *http.Request) {
	key := r.Header.Get("Idempotency-Key")
	if key == "" {
		http.Error(w, "Idempotency-Key header is required", http.StatusBadRequest)
		return
	}
 
	body, err := io.ReadAll(r.Body)
	if err != nil {
		http.Error(w, "invalid request body", http.StatusBadRequest)
		return
	}
	fp := fingerprint(body)
 
	record, err := s.store.Begin(r.Context(), key, fp)
	switch {
	case errors.Is(err, ErrFingerprintMismatch):
		http.Error(w, "idempotency key reused with a different request", http.StatusUnprocessableEntity)
		return
	case errors.Is(err, ErrKeyInFlight):
		http.Error(w, "request already in progress", http.StatusConflict)
		return
	case err == nil && record.Status == "completed":
		// A genuine retry of a completed request: replay the stored result.
		w.Header().Set("Content-Type", "application/json")
		w.WriteHeader(record.ResponseStatus)
		w.Write(record.ResponseBody)
		return
	case err != nil:
		http.Error(w, "internal error", http.StatusInternalServerError)
		return
	}
 
	// First time seeing this key: process the payment for real.
	var req createPaymentRequest
	if err := json.Unmarshal(body, &req); err != nil {
		s.store.Fail(r.Context(), key)
		http.Error(w, "invalid request body", http.StatusBadRequest)
		return
	}
 
	payment, err := s.payments.Charge(r.Context(), req.OrderID, req.AmountCents)
	if err != nil {
		s.store.Fail(r.Context(), key)
		http.Error(w, "payment failed", http.StatusUnprocessableEntity)
		return
	}
 
	respBody, _ := json.Marshal(payment)
	s.store.Complete(r.Context(), key, http.StatusCreated, respBody)
 
	w.Header().Set("Content-Type", "application/json")
	w.WriteHeader(http.StatusCreated)
	w.Write(respBody)
}

Begin is where the concurrency-safety actually happens — next.

Concurrent requests with the same key

Two requests carrying the same idempotency key can arrive at the same time: a client's retry firing just as the original request is still mid-flight, or a client (incorrectly) firing the same request twice in parallel. A check-then-insert in application code — SELECT ... WHERE key = $1, and if nothing comes back, INSERT — has a race window between the read and the write. Two goroutines (on the same instance or, worse, on different replicas) can both read "nothing exists yet" and both proceed to charge the card.

The key TEXT PRIMARY KEY constraint is what closes that window, because the insert itself is atomic at the database level:

func (s *IdempotencyStore) Begin(ctx context.Context, key, fp string) (*Record, error) {
	const insert = `
		INSERT INTO idempotency_keys (key, request_fingerprint, status)
		VALUES ($1, $2, 'pending')
	`
	_, err := s.db.Exec(ctx, insert, key, fp)
	if err == nil {
		// We won the race — we're the one processing this request.
		return nil, nil
	}
 
	var pgErr *pgconn.PgError
	if !errors.As(err, &pgErr) || pgErr.Code != "23505" { // unique_violation
		return nil, fmt.Errorf("insert idempotency key: %w", err)
	}
 
	// Someone already holds this key. Load what they recorded.
	existing, err := s.get(ctx, key)
	if err != nil {
		return nil, err
	}
	if existing.RequestFingerprint != fp {
		return nil, ErrFingerprintMismatch
	}
	if existing.Status == "pending" {
		return nil, ErrKeyInFlight
	}
	return existing, nil
}

Exactly one of two concurrent INSERTs can succeed — PostgreSQL guarantees that at the storage layer, regardless of how many application instances are racing to write it. The loser doesn't get an ambiguous result: it gets a 23505 unique_violation, reads back what the winner recorded, and responds accordingly. This is the entire reason the constraint lives in the database and not as an in-memory map or a "check first" query — an in-memory map doesn't help at all once there's more than one replica, and even on a single instance, a mutex around a check-then-act sequence is strictly more code than a constraint the database already enforces for free.

Expiration and cleanup

Idempotency keys shouldn't live forever. A reasonable default is 24 hours — long enough to cover realistic client retry windows (including a client that waits and retries manually), short enough that the table doesn't grow unbounded and a legitimately new request from the same client, made days later, isn't mistakenly treated as a duplicate if a key were ever reused.

DELETE FROM idempotency_keys
WHERE created_at < now() - interval '24 hours';

Run this as a periodic job (a cron-triggered task or a lightweight background worker), not inline on the request path.

Failure recovery

If the server crashes after inserting the pending row but before calling Complete, the key is left stuck in pending indefinitely — a retry against that key would receive ErrKeyInFlight forever, even though nothing is actually in flight anymore.

Handle this by treating a pending record older than a short timeout (a few minutes — longer than any legitimate request should take, short enough to recover quickly) as abandoned, and allowing it to be retried as if it were new:

if existing.Status == "pending" && time.Since(existing.CreatedAt) > 5*time.Minute {
	return s.reclaim(ctx, key, fp) // reset to pending, caller proceeds
}

This is a genuine trade-off: too short a timeout risks two attempts processing the same request if the first is simply slow rather than dead; too long leaves a real crash unrecoverable for an uncomfortable window. Size it against your actual p99 request latency, not a guess.

Production considerations

Not every endpoint needs this. Idempotency keys matter for non-idempotent, side-effecting writes — creating a payment, placing an order, sending a notification. A GET is already idempotent by definition. A PUT that sets a resource to an exact target state is usually naturally idempotent too (setting a user's email to x twice has the same end state as setting it once). Reserve idempotency keys for the operations where "did this already happen?" is genuinely ambiguous and genuinely expensive to get wrong.

This pairs directly with Retries, Timeouts and Backoff in Go: a retry policy on the client only becomes safe to apply to a payment-creation call once the server side is idempotent. Without this, "just add retries" to a payments client is how you get duplicate charges in production.

Common mistakes

• Application-level-only checks. A check-then-insert without a database constraint behind it looks like it works in testing and fails exactly when it matters — under concurrent load.
• No request fingerprint. Without one, a key reused for a different request body silently returns the previous response instead of surfacing a clear error — the client thinks it got a real answer to the new request when it didn't.
• No expiration. The table grows forever, and old keys eventually become a real storage and index-bloat problem.
• Applying this to every endpoint. Idempotency keys add a database round-trip and real implementation complexity. Reserve them for operations where a duplicate is actually costly.

Testing

The concurrency guarantee is the part worth testing directly — fire two requests with the same key at the same time and assert exactly one payment is created:

func TestCreatePayment_ConcurrentSameKey_OnlyOnePaymentCreated(t *testing.T) {
	store := newTestIdempotencyStore(t) // real Postgres, migrated schema
	payments := &countingPaymentService{}
	handler := NewPaymentHandler(store, payments)
 
	key := uuid.NewString()
	body := []byte(`{"orderId":"order_1","amountCents":4999}`)
 
	var wg sync.WaitGroup
	results := make([]int, 2)
	for i := range 2 {
		wg.Add(1)
		go func(i int) {
			defer wg.Done()
			req := httptest.NewRequest(http.MethodPost, "/payments", bytes.NewReader(body))
			req.Header.Set("Idempotency-Key", key)
			rec := httptest.NewRecorder()
			handler.CreatePayment(rec, req)
			results[i] = rec.Code
		}(i)
	}
	wg.Wait()
 
	if payments.chargeCount() != 1 {
		t.Fatalf("expected exactly 1 charge, got %d", payments.chargeCount())
	}
	if results[0] != http.StatusCreated && results[1] != http.StatusCreated {
		t.Fatal("expected at least one request to succeed with 201")
	}
}

Run this against a real PostgreSQL instance — the guarantee being tested lives in the database's constraint enforcement, which a fake or mocked store can't exercise honestly.

Summary

An idempotency key turns "did this request already happen?" from an unanswerable question into a database lookup. The key itself is just a client-supplied token; the safety comes entirely from enforcing uniqueness with a real database constraint, not an application-level check, because only the database can atomically decide which of two concurrent identical requests gets to proceed. Add a request fingerprint to catch key reuse, expire old keys so the table doesn't grow forever, and reserve this pattern for the writes where a duplicate is genuinely expensive — not every endpoint needs it.