TCP Congestion Control

TCP congestion control prevents senders from persistently overwhelming a network path. A sender maintains a congestion window, commonly called cwnd, that limits unacknowledged data in flight. The receiver separately advertises a receive window, or rwnd, to protect its available buffering. Transmission is constrained by both windows, as well as application behavior and local resources. Congestion control protects the network; receive-window flow control protects the receiver. Confusing these mechanisms can lead engineers to increase network capacity when the actual problem is a slow-consuming application.

Many TCP implementations use slow start to increase the congestion window rapidly at the beginning of a connection, followed by a more measured congestion-avoidance phase. Loss-based algorithms interpret packet loss as a congestion signal and reduce their sending allowance. Explicit Congestion Notification, when negotiated and supported along the path, can signal congestion through packet marking rather than requiring a drop. Algorithms differ: CUBIC primarily uses loss-based feedback, while BBR estimates delivery bandwidth and propagation delay to guide transmission. Their behavior and suitability depend on the operating system, implementation, and network conditions.

Cloud applications encounter congestion control across virtual networks, internet paths, VPNs, and private connectivity. A high-bandwidth connection does not guarantee that one TCP stream will fill it. The bandwidth-delay product estimates the data that must remain in flight to sustain a target rate: a 1 Gbit/s path with an 80-millisecond round-trip time requires approximately 10 MB in flight, before considering overhead. Small socket buffers, a restrictive receive window, packet loss, or short connection lifetimes can prevent that rate. Reusing connections can reduce repeated startup costs, but increasing parallel connections may worsen contention rather than fix the underlying bottleneck.

For packet-level troubleshooting, capture both directions of the connection, preferably at both endpoints. In AWS, VPC Traffic Mirroring can copy traffic from supported network interfaces to an analysis appliance for inspection with tools such as Wireshark. Examine sequence and acknowledgment numbers, selective acknowledgment information, retransmissions, duplicate acknowledgments, advertised windows, round-trip-time trends, and ECN signals. A retransmission does not prove congestion: reordering, capture loss, and other packet-loss causes can produce similar evidence. TCP does not transmit its congestion window in a header, so correlate packet observations with sender-side TCP statistics, such as those exposed by Linux ss.

Traffic shaping deliberately queues or paces traffic to enforce a rate, whereas policing commonly drops or marks traffic that exceeds a configured allowance. A download that plateaus at a repeatable rate while round-trip time rises may be encountering a shaping queue. Repeated loss during bursts near a rate limit can suggest policing, but neither pattern is conclusive without configuration and metric checks. Compare packet timing with host queue statistics, interface counters, and cloud network allowance metrics. Sustained throughput alone cannot identify which device or policy imposed the limit.

Capture quality is part of the diagnosis. Mirrored traffic can be dropped or truncated, and host offloads can make local captures appear to contain unusually large segments or invalid checksums. TLS encryption generally leaves TCP headers available for analysis, but encrypted application content remains unreadable without appropriate decryption material. VPC Flow Logs provide connection-level metadata, not packet sequences or timing detail sufficient to reconstruct congestion behavior. Validate the observation point before interpreting apparent loss.

Optimization should follow the identified constraint. Reduce unnecessary latency, correct loss or MTU problems, size buffers appropriately, and select suitable instance networking capacity before changing congestion-control algorithms. Large queues may improve utilization while increasing latency through bufferbloat. More aggressive sending or additional parallel streams can improve one workload at the expense of competing traffic, so test throughput, tail latency, and fairness together.