Sizing a Payment Gateway for Peak Traffic

Before writing a line of code for the gateway rewrite, I documented expected load and failure modes. This note captures the model we used — and where it was wrong.

Assumptions

Parameter	Estimate	Source
Peak RPS	12,000	Last Black Friday + 30% buffer
P99 latency target	150ms	Product SLA
Avg payload	2.4 KB	Access logs sample
DB writes per request	0.3	Mostly reads, audit on mutations

The Model

Required app instances = Peak RPS / (RPS per pod at P99 target)
RPS per pod ≈ 500 (measured under load test with 150ms P99)
Instances = 12,000 / 500 = 24 pods (+ 20% headroom) → 30 pods

Connection math mattered more than CPU:

• 30 pods × 20 DB connections = 600 client connections
• PgBouncer pool capped at 100 → bottleneck identified before prod

What We Got Wrong

Load tests used uniform traffic. Real Black Friday traffic spiked in 90-second bursts — 3× average for short windows. HPA lagged by 45 seconds. Fix: custom metrics on request rate, not CPU.