Advanced Go Concurrency: Worker Pools, Backpressure and Bounded Concurrency

The problem

for _, item := range items {
	go process(item)
}

This compiles, runs, and works fine in development against ten items. Against ten thousand, it launches ten thousand goroutines simultaneously — each one opening a database connection, calling a downstream API, or allocating a buffer, all at once, with nothing in the loop that could possibly slow it down. Building Background Workers in Go already covered the bounded worker pool pattern for job queues; this article goes further into why unbounded concurrency degrades performance rather than improving it, and how to reason about the right amount of concurrency for a given workload.

Why it matters

More goroutines does not mean more throughput. Goroutines are cheap compared to OS threads, but they aren't free, and "cheap per-goroutine" doesn't stay cheap when the number is unbounded:

• CPU-bound work run with more goroutines than CPU cores doesn't run faster — the Go scheduler still has to time-slice them across the same fixed number of cores, and the extra goroutines just add scheduling and context-switch overhead without adding capacity.
• I/O-bound work (network calls, database queries) benefits from some concurrency, since goroutines spend most of their time blocked waiting, not competing for CPU — but past a certain point, the bottleneck moves downstream (the database's connection limit, a rate-limited API), and more goroutines just means more of them waiting in line rather than more useful work happening.
• Memory grows linearly with the unbounded count — each goroutine's stack starts small but grows with what it's doing, and ten thousand goroutines each holding a buffer or a partially-built struct adds up fast, exactly the failure mode the naive worker pattern is prone to.

Bounded concurrency with a semaphore

The simplest correct fix for "run at most N of these at a time" is a buffered channel used as a counting semaphore — acquire a slot before doing work, release it after:

func processAll(ctx context.Context, items []Item, maxConcurrent int) error {
	sem := make(chan struct{}, maxConcurrent)
	var wg sync.WaitGroup
	errCh := make(chan error, len(items))
 
	for _, item := range items {
		select {
		case sem <- struct{}{}:
		case <-ctx.Done():
			wg.Wait()
			return ctx.Err()
		}
 
		wg.Add(1)
		go func(item Item) {
			defer wg.Done()
			defer func() { <-sem }()
 
			if err := process(ctx, item); err != nil {
				errCh <- fmt.Errorf("process item %v: %w", item.ID, err)
			}
		}(item)
	}
 
	wg.Wait()
	close(errCh)
 
	for err := range errCh {
		return err // return the first error; see errgroup below for richer handling
	}
	return nil
}

The channel's buffer size is the concurrency limit: the maxConcurrent+1th attempt to send into sem blocks until a running goroutine releases its slot. This is the mechanism — the same shape as the bounded worker pool from Building Background Workers in Go, just expressed as a semaphore rather than a fixed set of long-lived worker goroutines pulling from a queue. Both are valid; the semaphore form is often simpler when the total item count is known up front rather than an ongoing stream.

errgroup: bounded concurrency with real error propagation

The pattern above returns only the first error and swallows the rest into a buffered channel nobody drains further. golang.org/x/sync/errgroup (a real, widely-used extension of the standard library's concurrency primitives) does this more cleanly, including canceling the shared context the moment any goroutine returns an error — so the remaining in-flight work gets a cancellation signal instead of continuing pointlessly:

func processAll(ctx context.Context, items []Item, maxConcurrent int) error {
	g, ctx := errgroup.WithContext(ctx)
	sem := make(chan struct{}, maxConcurrent)
 
	for _, item := range items {
		item := item
		select {
		case sem <- struct{}{}:
		case <-ctx.Done():
			return g.Wait()
		}
 
		g.Go(func() error {
			defer func() { <-sem }()
			return process(ctx, item)
		})
	}
 
	return g.Wait() // returns the first non-nil error, if any
}

g.Wait() blocks until every launched goroutine returns, then returns the first non-nil error — and because ctx here is the group's own derived context, every process(ctx, item) call sees cancellation as soon as one of its siblings fails, instead of each goroutine only finding out at its own natural completion.

Backpressure and queue depth

Backpressure is what a bounded semaphore or worker pool gives you almost for free: once every slot is occupied, the producer — the loop launching work — blocks too, rather than continuing to pile up unstarted work in memory. This is the property the naive unbounded version has none of: nothing in go process(item) ever slows the loop down, so the loop finishes submitting all ten thousand goroutines before any meaningful number of them have finished, and every one of them exists in memory simultaneously in the worst case.

Queue depth — how many items are waiting versus how many are actively being processed — is the signal to watch in production. A queue depth that stays near zero means the pool is keeping up; a queue depth that climbs steadily means the producer is outpacing the consumers, and the fix is either more concurrency (if the bottleneck is genuinely idle capacity) or addressing whatever's slow downstream (if it isn't) — adding workers against a saturated database doesn't help, it just moves the queueing from your process's memory to the database's own connection queue.

Context cancellation and goroutine leaks

Every goroutine launched inside a bounded-concurrency helper must actually check ctx.Done() somewhere in its own work, not just at launch — a goroutine blocked on a channel receive, an unbounded time.Sleep, or a network call with no timeout attached to ctx will keep running (and holding its semaphore slot, or worse, never releasing it) even after the caller has given up and moved on. This is a goroutine leak: a goroutine that outlives any code that could still observe its result, silently consuming memory and, if it's holding a semaphore slot, silently reducing your effective concurrency limit for good. The fix is mechanical but easy to skip: every blocking operation inside a launched goroutine needs to either derive its own bound from ctx, or select on ctx.Done() alongside whatever else it's waiting on.

CPU-bound vs I/O-bound: choosing a concurrency limit

There's no universal "right" number, but the reasoning differs by workload:

• CPU-bound — set maxConcurrent at or near runtime.NumCPU(). More than that just adds scheduling overhead for work that's genuinely limited by available cores.
• I/O-bound — the useful limit is usually set by what's downstream, not by your own CPU: the database's connection pool size (see PostgreSQL Connection Pooling in Go), a rate limit on an external API, or a fixed budget you've decided not to exceed regardless of what the dependency could technically handle.

A benchmark: bounded vs unbounded

A simple illustrative comparison — 5,000 jobs, each simulating 5ms of I/O wait, run unbounded versus bounded to 50 concurrent:

func BenchmarkUnbounded(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var wg sync.WaitGroup
		for j := 0; j < 5000; j++ {
			wg.Add(1)
			go func() {
				defer wg.Done()
				time.Sleep(5 * time.Millisecond)
			}()
		}
		wg.Wait()
	}
}
 
func BenchmarkBounded50(b *testing.B) {
	for i := 0; i < b.N; i++ {
		sem := make(chan struct{}, 50)
		var wg sync.WaitGroup
		for j := 0; j < 5000; j++ {
			sem <- struct{}{}
			wg.Add(1)
			go func() {
				defer wg.Done()
				defer func() { <-sem }()
				time.Sleep(5 * time.Millisecond)
			}()
		}
		wg.Wait()
	}
}

On a modest machine, the unbounded version finishes in roughly the same wall-clock time as the bounded one for this simulated-I/O workload (since sleeping goroutines barely touch the scheduler) — but its peak goroutine count and memory footprint are two orders of magnitude higher throughout the run, and against real I/O (an actual database or HTTP call instead of time.Sleep), the unbounded version instead saturates the real downstream dependency immediately and its effective latency gets far worse, not just its memory profile. The lesson isn't "bounded is always faster" — for pure sleep it isn't — it's that unbounded concurrency's cost shows up as resource pressure and downstream saturation, which a synthetic sleep-based benchmark under-represents.

Production recommendations

• Default to a bounded semaphore or worker pool for any concurrency over a handful of items — treat unbounded go in a loop as a code smell that needs a specific justification, not a default.
• Size the limit based on the actual downstream constraint (database pool, external rate limit, CPU count) — not an arbitrary round number.
• Always thread context.Context through the launched work and check it inside any blocking call, not just at the top of the goroutine.
• Watch queue depth / semaphore saturation as a production metric, the same way you'd watch database connection pool utilization.

Common mistakes

• go in a loop with no bound, deferring the concurrency-limit decision to whatever happens to run out first (memory, connections, downstream patience).
• A goroutine that doesn't check ctx.Done() anywhere in its own blocking work, leaking past the point its result is still wanted.
• Sizing concurrency off CPU count for I/O-bound work, or off nothing at all for CPU-bound work — the two workloads need opposite reasoning.

Testing

Verify the bound is actually enforced, not just present in the code:

func TestProcessAll_RespectsMaxConcurrent(t *testing.T) {
	var current, peak int32
	process := func(ctx context.Context, item Item) error {
		n := atomic.AddInt32(&current, 1)
		defer atomic.AddInt32(&current, -1)
		for {
			old := atomic.LoadInt32(&peak)
			if n <= old || atomic.CompareAndSwapInt32(&peak, old, n) {
				break
			}
		}
		time.Sleep(2 * time.Millisecond)
		return nil
	}
 
	items := make([]Item, 200)
	if err := processAllWith(context.Background(), items, 10, process); err != nil {
		t.Fatalf("unexpected error: %v", err)
	}
	if peak > 10 {
		t.Fatalf("expected peak concurrency <= 10, got %d", peak)
	}
}

Summary

Unbounded goroutines aren't dangerous because goroutines are expensive — they're dangerous because nothing limits how many exist simultaneously, so memory, database connections, and downstream capacity all get consumed at whatever rate the producer happens to generate work, with no feedback loop to slow it down. A semaphore or bounded worker pool, ideally paired with errgroup for clean error propagation and cancellation, restores that feedback loop: concurrency capped at a number chosen from the real constraint — CPU count for CPU-bound work, downstream capacity for I/O-bound work — and backpressure that makes the producer wait instead of piling up unbounded work in memory.