Load Testing

Load testing measures application behavior under a specified workload, such as concurrent checkout sessions, API requests per second, or messages arriving at a queue. Its purpose is not simply to generate large amounts of traffic, but to determine whether the system meets measurable business requirements. For example, an ordering service might need to sustain 2,000 requests per second with a 99th-percentile response time below 500 milliseconds and an error rate below 0.1 percent.

Start by defining a realistic workload model. Include the production mix of operations, request sizes, authentication flows, data distributions, cache behavior, and user think time. Concurrent users and requests per second are not interchangeable: slower responses can reduce the request rate generated by a fixed population of simulated users. When evaluating an arrival-rate requirement, use a tool configuration that maintains the intended arrival rate and reports when generators cannot keep up. Otherwise, an overloaded application can appear healthier because the test unintentionally reduces demand.

Run tests against an isolated, production-like environment with comparable topology, scaling policies, quotas, and dependency behavior. Distributed load generators can simulate larger workloads, but their own CPU, network bandwidth, and connection limits must be monitored. Use controlled test data, protect credentials, and avoid sending unintended transactions to payment, email, or other external systems. Replacing dependencies with stubs can reduce cost and risk, but it also limits conclusions about end-to-end performance.

Collect client-side latency percentiles, throughput, timeouts, and errors alongside server-side metrics, logs, and traces. Average latency alone can hide severe delays affecting a small but important fraction of requests. Correlate application measurements with CPU utilization, memory pressure, database connections, lock waits, storage latency, queue age, throttling, and service quotas. High CPU may justify additional compute capacity, whereas database contention or an exhausted connection pool may require query optimization, connection management, caching, or architectural changes.

To validate scaling, increase demand gradually and observe the complete sequence from metric detection through capacity provisioning, application startup, health checks, and useful request handling. A checkout tier might scale out correctly while requests still fail because its database connection limit is reached. Adding more application instances can worsen that bottleneck. Other remediations include selecting a better scaling signal, adjusting instance warmup, increasing minimum capacity, using scheduled scaling for predictable peaks, or requesting quota increases. Repeat the same test after remediation to verify the improvement and its cost.

Distinguish related performance tests. Load testing validates behavior under defined expected demand; stress testing pushes beyond expected capacity to discover failure limits and recovery behavior. Spike testing examines abrupt changes, while soak testing exposes problems that emerge over extended operation. Benchmarking compares measurements across controlled configurations or software versions. These techniques complement one another rather than serving as interchangeable labels.

Integrate versioned test scripts into CI/CD after deploying the candidate application. Run short performance checks frequently, with larger distributed or endurance tests on a schedule or before important releases. Define automated pass/fail gates using explicit thresholds and retain results with the application version, infrastructure configuration, dataset, and test settings. Compare equivalent runs and account for environmental variability before declaring a regression. Stop unsafe tests automatically, clean up temporary resources, and budget for compute, requests, data transfer, and observability charges. Passing a load test demonstrates performance under the tested conditions, not unlimited scalability or correctness under every failure scenario.