Graceful Shutdown in Go HTTP Servers

The problem

When a Go process receives SIGKILL, or is killed without any signal handling at all, every in-flight HTTP request is dropped mid-response. A client mid-checkout gets a connection reset. A background write that had already been committed to the database but not yet acknowledged to the caller now looks like a failure to retry. None of this is a Go-specific problem — it's what happens whenever a process managing live connections disappears without warning.

net/http.Server gives you a way to shut down deliberately: stop accepting new connections, let in-flight requests finish within a bounded window, then exit. Getting this wired up correctly is a small amount of code with an outsized effect on reliability during deploys.

Why it matters

Deploys, autoscaling, and node maintenance all end the same way for a running process: something sends it a termination signal and expects it to exit soon after. In a containerized environment this happens constantly — every rolling deploy terminates the previous generation of pods.

• Without graceful shutdown, every deploy produces a burst of failed requests for whatever was in flight at the moment the old process died.
• Health checks and load balancers need time to notice a pod is going away and stop routing to it — if the process exits before that propagates, requests get routed to a socket that no longer accepts connections.
• Background work (queue consumers, in-memory batching, outbox flushing) needs an explicit signal to stop pulling new work and flush what it has.

How it works

http.Server.Shutdown(ctx) stops the listener from accepting new connections, then waits for active requests to complete before returning. If the context passed to Shutdown is canceled or its deadline passes first, Shutdown returns immediately and leaves any still-active connections to be closed however the caller decides.

The standard pattern: listen for SIGTERM and SIGINT with signal.NotifyContext, run the server in a goroutine, and call Shutdown once a signal arrives, bounded by a timeout.

func main() {
	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
	defer stop()
 
	srv := &http.Server{
		Addr:         ":8080",
		Handler:      newRouter(),
		ReadTimeout:  5 * time.Second,
		WriteTimeout: 10 * time.Second,
		IdleTimeout:  120 * time.Second,
	}
 
	go func() {
		if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
			log.Fatalf("listen: %v", err)
		}
	}()
 
	<-ctx.Done()
	stop() // stop listening for further signals; a second SIGTERM should force-kill
	log.Println("shutdown signal received, draining connections")
 
	shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
	defer cancel()
 
	if err := srv.Shutdown(shutdownCtx); err != nil {
		log.Printf("graceful shutdown failed: %v", err)
		_ = srv.Close() // force-close remaining connections
	}
}

ListenAndServe always returns a non-nil error; after Shutdown is called it returns http.ErrServerClosed, which is the expected, successful exit path — not something to treat as a startup failure.

Draining more than just HTTP connections

A real service usually has more than one thing that needs to stop cleanly: a database connection pool, a background queue consumer, an in-memory metrics flusher. Wire their shutdown into the same signal so nothing keeps running after the HTTP server has already stopped accepting traffic.

func run(ctx context.Context, db *sql.DB, consumer *queue.Consumer, srv *http.Server) error {
	consumerDone := make(chan struct{})
	go func() {
		consumer.Run(ctx) // returns when ctx is canceled
		close(consumerDone)
	}()
 
	go func() {
		if err := srv.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
			log.Printf("listen: %v", err)
		}
	}()
 
	<-ctx.Done()
 
	shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
	defer cancel()
 
	if err := srv.Shutdown(shutdownCtx); err != nil {
		return fmt.Errorf("shutdown http server: %w", err)
	}
 
	select {
	case <-consumerDone:
	case <-shutdownCtx.Done():
		log.Println("queue consumer did not stop before shutdown deadline")
	}
 
	return db.Close()
}

The consumer's own ctx — the same one canceled by the signal — is what tells it to stop pulling new work; Shutdown only owns the HTTP listener.

Production considerations

In Kubernetes, SIGTERM and "stop receiving traffic" are not the same event. A pod is removed from Service endpoints asynchronously with the SIGTERM being sent to the container — there's no guarantee the endpoint update has propagated to every kube-proxy and load balancer before your process starts draining. A preStop hook that sleeps for a few seconds before the container's main process receives SIGTERM is the usual fix, giving the network layer time to stop sending new traffic first:

lifecycle:
  preStop:
    exec:
      command: ["sleep", "5"]
terminationGracePeriodSeconds: 30

terminationGracePeriodSeconds must exceed your shutdown timeout, including the preStop delay. If the grace period expires first, Kubernetes sends SIGKILL regardless of how much of your Shutdown call had completed.

Readiness, not liveness, controls traffic. If your process also exposes a /ready endpoint, flip it to unready as the first step of shutdown — before calling Shutdown — so any readiness-probe-driven load balancer stops routing new requests as early as possible, independent of the Kubernetes endpoint propagation delay.

Common mistakes

• No shutdown handling at all — relying on the orchestrator to just kill the process, which is exactly the failure mode this pattern avoids.
• Shutdown timeout longer than the grace period. The process gets SIGKILLed mid-Shutdown, which is no better than not calling it.
• Treating http.ErrServerClosed as a fatal error in the goroutine running ListenAndServe, which turns every clean shutdown into a logged crash.
• Closing shared resources (DB pool, cache client) before Shutdown returns, which fails in-flight requests that were otherwise going to complete successfully within the drain window.
• Not handling a second signal. If Shutdown hangs longer than expected, an operator sending a second SIGTERM/SIGINT should be able to force an immediate exit rather than waiting out the full grace period.

Testing

Shutdown's contract — it waits for active requests, then returns — is straightforward to test directly with httptest, using a handler that blocks until told to proceed:

func TestGracefulShutdown_WaitsForActiveRequest(t *testing.T) {
	release := make(chan struct{})
	started := make(chan struct{})
 
	srv := &http.Server{
		Handler: http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
			close(started)
			<-release
			w.WriteHeader(http.StatusOK)
		}),
	}
 
	ln, err := net.Listen("tcp", "127.0.0.1:0")
	if err != nil {
		t.Fatal(err)
	}
	go srv.Serve(ln)
 
	go func() {
		resp, err := http.Get("http://" + ln.Addr().String())
		if err != nil {
			t.Errorf("request failed: %v", err)
			return
		}
		resp.Body.Close()
	}()
 
	<-started
 
	shutdownDone := make(chan error, 1)
	go func() {
		shutdownDone <- srv.Shutdown(context.Background())
	}()
 
	select {
	case <-shutdownDone:
		t.Fatal("Shutdown returned before the in-flight request completed")
	case <-time.After(50 * time.Millisecond):
	}
 
	close(release)
 
	if err := <-shutdownDone; err != nil {
		t.Fatalf("Shutdown returned error: %v", err)
	}
}

The test asserts the negative case explicitly — Shutdown must still be blocked while a handler is mid-flight — which is the behavior that actually matters in production.

Summary

Graceful shutdown means listening for SIGTERM/SIGINT, calling http.Server.Shutdown with a bounded timeout so new connections stop and in-flight ones get to finish, and draining any other long-running work (queue consumers, background jobs) on the same signal before closing shared resources like a database pool. In Kubernetes, the shutdown timeout, preStop delay, and terminationGracePeriodSeconds all have to agree with each other, or the orchestrator will SIGKILL the process before your own shutdown logic finishes.