Skip to content
Cloud

Kubernetes for Go Developers: Deploying a Production-Ready Go API

What actually happens when Kubernetes sends your Go process SIGTERM, and the Deployment, Service, probe, and resource config a production Go API needs.

GoBackend.dev13 min read
GoKubernetesDockerCloud NativeDevOps

The problem

Your Go process received SIGTERM. What happens next?

If the answer is "the process exits," that's the problem. Kubernetes sends SIGTERM as the first step of a normal, healthy shutdown — a rolling deployment, a node scaling down, a pod being rescheduled — not as an emergency. By the time your process sees that signal, Kubernetes has already removed the pod from the Service's list of endpoints, which means no new traffic should be routed to it (though in-flight connections and DNS/kube-proxy caching mean this isn't instantaneous — a request or two can still land in the brief window right after removal). What your process does in the seconds after SIGTERM determines whether that's an invisible, routine event or a burst of dropped connections on every single deploy.

Why it matters

A Go service that doesn't handle this correctly fails visibly and repeatedly — not once, but on every rollout, every autoscale-down event, every node maintenance cycle. That's often mistaken for "flaky infrastructure" when it's actually a missing five lines of shutdown code.

The pod termination lifecycle

The actual sequence, in order:

Rendering diagram…

SIGTERM isn't a request to stop immediately — it's a request to start shutting down. This is exactly what Graceful Shutdown in Go HTTP Servers implements: catching the signal, calling http.Server.Shutdown(ctx) so the server stops accepting new connections but lets in-flight requests finish, then closing the database pool and any other resources before exiting. That article's code isn't optional infrastructure trivia — it's the specific mechanism that makes this Kubernetes lifecycle work without dropping requests on every deployment.

terminationGracePeriodSeconds must be longer than your application's own shutdown timeout. If Kubernetes' grace period is shorter than the time your graceful shutdown needs to finish in-flight requests, Kubernetes sends SIGKILL before your cleanup completes — and the graceful shutdown code might as well not exist. If your app's shutdown timeout is 15s, set terminationGracePeriodSeconds to at least 30s to leave margin.

Practical implementation

A Deployment for a Go API, with the graceful-shutdown-relevant field set explicitly:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: gobackend-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: gobackend-api
  template:
    metadata:
      labels:
        app: gobackend-api
    spec:
      terminationGracePeriodSeconds: 30
      containers:
        - name: api
          image: registry.example.com/gobackend-api:1.4.0
          ports:
            - containerPort: 8080
          envFrom:
            - configMapRef:
                name: gobackend-config
            - secretRef:
                name: gobackend-secrets
          resources:
            requests:
              cpu: "250m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "256Mi"
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            periodSeconds: 5
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            periodSeconds: 10

A Service routing to those pods:

apiVersion: v1
kind: Service
metadata:
  name: gobackend-api
spec:
  selector:
    app: gobackend-api
  ports:
    - port: 80
      targetPort: 8080

Non-secret configuration in a ConfigMap, secrets in a Secret — both mounted as environment variables, which is exactly what the GoBackend Starter's config.Load() already expects:

apiVersion: v1
kind: ConfigMap
metadata:
  name: gobackend-config
data:
  ENV: "production"
  PORT: "8080"
---
apiVersion: v1
kind: Secret
metadata:
  name: gobackend-secrets
type: Opaque
stringData:
  DATABASE_URL: "postgres://user:pass@postgres:5432/gobackend?sslmode=require"
  JWT_SECRET: "replace-with-a-real-32+-character-secret"

The architecture this produces:

Rendering diagram…

Readiness vs liveness probes

These answer different questions, and conflating them causes real outages:

  • Readiness — "can this pod serve traffic right now?" Failing it removes the pod from the Service's endpoints temporarily; the pod keeps running and can rejoin once it passes again. Point this at /ready — the GoBackend Starter's readiness endpoint, which checks the database connection.
  • Liveness — "is this process alive, or hung?" Failing it gets the pod killed and restarted. Point this at /health — a check that only confirms the process itself is responsive, nothing more.

A liveness probe that checks downstream dependencies (the database, an external API) is a common and dangerous mistake: if the database gets slow, every pod's liveness probe starts failing, and Kubernetes restarts every pod in the deployment — replacing a slow-database problem with a full outage, caused entirely by the liveness check itself. Dependency health belongs in the readiness probe, which only takes the pod out of rotation, not the liveness probe, which kills it.

Resource requests and limits

Requests affect scheduling — the scheduler only places a pod on a node that has the requested CPU/memory actually available. Limits affect runtime behavior once running — exceeding the CPU limit gets you throttled (slower, not killed); exceeding the memory limit gets the pod OOM-killed immediately, no grace period.

The common mistake is setting the memory limit too close to steady-state usage — a normal traffic spike (a burst of larger request bodies, a GC cycle running behind on collection) pushes past the limit and the pod is killed mid-request, which looks like a random crash rather than what it actually is: a limit set with no headroom.

Rolling deployments

Kubernetes replaces pods gradually during a rollout: new pods are created, and only once a new pod passes its readiness probe does Kubernetes proceed to terminate an old one. This makes the readiness probe load-bearing on every single deployment, not just at steady state — a new pod that starts accepting traffic before it's actually ready (before its database connections are established, for example) produces a visible spike of errors on every deploy, and a readiness probe that's too lenient (or missing) is usually the reason.

The old pods being terminated during that same rollout go through the exact SIGTERM lifecycle described above — so a rolling deployment exercises both halves of the correctness requirement simultaneously: new pods must be honestly not ready until they actually are, and old pods must shut down gracefully rather than dropping in-flight work.

Horizontal scaling and shared dependencies

Adding replicas multiplies how many instances hit the same downstream dependencies. The connection pool math from PostgreSQL Connection Pooling in Go applies directly here: MaxConns per replica × replica count has to stay under what PostgreSQL's max_connections can actually support — scaling a Deployment from 3 replicas to 10 without revisiting per-replica pool size can push a previously fine configuration past the database's ceiling, with no change to the application code at all.

Common mistakes

  • terminationGracePeriodSeconds shorter than the app's own shutdown timeout, so Kubernetes SIGKILLs the process before graceful shutdown finishes.
  • Liveness probes checking downstream dependencies, turning a slow database into a mass pod-restart event.
  • No resource limits, letting one pod consume a node's resources and degrade its neighbors.
  • A readiness probe that always returns 200 regardless of actual state, which defeats its entire purpose during rollouts and outages alike.

Summary

The Kubernetes concepts that matter for a Go API aren't abstract — they're the concrete answer to "what happens to my process, and my in-flight requests, during a deploy or a scale-down." Graceful shutdown from earlier in this series is what makes SIGTERM safe; a correctly separated readiness/liveness pair is what makes rolling deployments and dependency hiccups survivable instead of cascading into restarts; and resource limits plus connection-pool math sized for your real replica count are what keep scaling out from becoming its own outage.