File Descriptor Limits Under Connection Spikes

Too many open sockets crashed the gateway with 'too many open files' — ulimits, kernel tuning, and Kubernetes limits.

At ~8,000 concurrent connections, the process died. Default ulimit -n was 1024.

Layers to Check

LayerDefaultOur target
Process (ulimit -n)102465535
/proc/sys/fs/file-maxvariesaudit host
K8s pod specnonedocument in runbook

Kubernetes Fix

securityContext:
  # Requires privileged init or sysctl on node for some clusters
containers:
  - name: gateway
    resources:
      limits:
        memory: 512Mi
    # Use an init container or custom image with raised limits

We baked raised limits into the container entrypoint and documented the sysctl requirements for bare-metal fallback environments.

Connection ceiling moved from ~1K to ~50K per pod before memory became the bottleneck instead.

linux , networking , limits · Linux , Kubernetes

More in Linux