CLOUD CONCEPT

IP Fragmentation

IP fragmentation divides an IP datagram into smaller fragments so it can traverse a link whose maximum transmission unit is smaller than the original datagram.

Cloud Concepts Hub →

What is IP Fragmentation

Fragmentation allows oversized IP datagrams to cross smaller-MTU links, but increases packet-processing overhead and makes delivery more sensitive to loss. Cloud network design generally aims to avoid fragmentation by aligning packet sizes with the complete path, accounting for tunnel overhead, and enabling appropriate path MTU discovery.

IP fragmentation addresses a mismatch between an IP datagram's size and a link's maximum transmission unit, or MTU. The IP MTU is the largest IP packet that a link can carry without fragmentation; it excludes Ethernet headers and trailers. An Ethernet link with a 1,500-byte IP MTU therefore carries frames larger than 1,500 bytes. Distinguishing frame size from IP MTU is important when configuring switches, virtual interfaces, and cloud connections.

For IPv4, a sender or router can fragment an oversized datagram when the Don't Fragment, or DF, flag is clear. Each fragment has its own IP header and carries information that lets the destination reconstruct the original datagram. If DF is set, a router cannot fragment the packet and normally drops it while returning an ICMP Destination Unreachable message indicating that fragmentation is needed. IPv6 routers never fragment packets: they return ICMPv6 Packet Too Big messages, and only the source can perform IPv6 fragmentation.

The effective path MTU is the smallest IP MTU along the route. A virtual machine configured with a 9,001-byte MTU does not gain a 9,001-byte path to every destination. Traffic may traverse an internet connection, firewall, transit router, or encrypted tunnel with a smaller limit. Encapsulation adds outer headers, reducing the space available for the inner packet unless the underlying network supports a sufficiently larger MTU.

Jumbo frames can improve bulk-transfer efficiency by carrying more application data per packet and reducing header and packet-processing overhead. They are useful for compatible private paths, but they are not an automatic performance improvement. In AWS, supported EC2 interfaces can use a 9,001-byte MTU, while AWS Transit Gateway supports an 8,500-byte MTU on supported non-VPN attachments. AWS Direct Connect supports different MTUs by virtual-interface type: private virtual interfaces can support 9,001 bytes, transit virtual interfaces can support 8,500 bytes, and public virtual interfaces use 1,500 bytes. All relevant components must support the chosen size.

For example, two instances exchanging storage data over a validated jumbo-capable private path may benefit from larger packets. If that traffic is rerouted through an IPsec VPN, the original packet size may no longer fit. Rather than relying on fragmentation, configure an appropriate tunnel or route MTU and, for TCP, consider maximum segment size clamping. With a 1,500-byte MTU and ordinary headers, TCP MSS is 1,460 bytes for IPv4 or 1,440 bytes for IPv6. Additional headers reduce those values. MSS controls TCP payload size; it does not protect UDP applications from sending oversized datagrams.

Classic path MTU discovery uses ICMP feedback to help senders reduce packet sizes. Blocking the necessary ICMP messages can produce an MTU black hole: connection establishment and small requests succeed, but larger transfers stall. Packetization-layer path MTU discovery can reduce dependence on ICMP where the transport and implementation support it. During troubleshooting, use controlled-size probes with fragmentation disabled where supported, packet captures, and tests in both directions; asymmetric paths can have different limits.

Fragmentation is not equivalent to TCP segmentation or NIC offload. Segmentation creates independently transported TCP segments, whereas IP fragmentation splits one datagram that must be reassembled before delivery to the upper-layer protocol. Losing one fragment prevents reconstruction of that datagram, and reassembly consumes memory and processing resources. Some middleboxes also filter fragments. Optimize for the largest reliably supported packet size across each actual path, then measure throughput, CPU use, loss, and retransmissions rather than assuming that larger frames always reduce costs or improve performance.

Key aspects of IP Fragmentation

Avoid unnecessary fragmentation by matching packet sizes to the complete path, accounting for encapsulation, and validating discovery and transport behavior.

Path MTU governs usable packet size

The smallest MTU along the route determines the largest packet that can traverse it without fragmentation. Validate the full path, including alternate routes, rather than relying only on the endpoint interface setting.

IPv4 and IPv6 handle fragmentation differently

IPv4 routers can fragment packets when DF is clear; IPv6 routers cannot fragment packets. Permit the appropriate ICMP error messages so senders can learn when their packets exceed a downstream limit.

Encapsulation reduces effective MTU

IPsec and other overlays add headers around the original packet. Budget for their actual overhead and adjust inner packet sizes instead of assuming a 1,500-byte underlay supports a 1,500-byte inner packet.

Jumbo frames require compatible paths

Larger packets can reduce per-packet processing and header overhead for bulk transfers. Their benefit depends on support across interfaces, appliances, and connection types, and should be confirmed through measurement.

Fragment loss increases delivery risk

The destination needs every fragment to reconstruct a datagram. A missing fragment invalidates the complete datagram, while reassembly adds resource consumption and can expose compatibility issues with middleboxes.

TCP MSS is a targeted mitigation

MSS negotiation or clamping can limit TCP segment payloads to fit a smaller path MTU. It does not resize UDP datagrams or replace MTU planning for non-TCP traffic.

Cloud services for IP Fragmentation

EC2 supplies the workload interfaces, while Direct Connect, Site-to-Site VPN, and Transit Gateway introduce connection-specific MTU boundaries. A private transfer may support jumbo packets across Direct Connect, yet require smaller packets when crossing Transit Gateway or failing over to an encrypted VPN. Design packet sizes for each actual route, including backup routes, rather than applying one jumbo-frame setting everywhere. Coordinate endpoint MTUs, customer router settings, tunnel overhead, and TCP MSS policies, and allow necessary ICMP feedback. Validate large transfers as well as small connectivity probes before treating a path as fragmentation-free.

Amazon EC2 image Amazon EC2

Supported EC2 network interfaces can use a 9,001-byte MTU for jumbo-capable paths. Configure the guest operating system appropriately and verify downstream limits before using large packets across private, internet, or hybrid routes.

Open Amazon EC2 →
AWS Direct Connect image AWS Direct Connect

Direct Connect MTU support depends on the virtual-interface type: private virtual interfaces support up to 9,001 bytes, transit virtual interfaces up to 8,500 bytes, and public virtual interfaces use 1,500 bytes. Customer routers and the remaining cloud path must also support the selected MTU.

Open AWS Direct Connect →
AWS Site to Site VPN image AWS Site-to-Site VPN

IPsec encapsulation reduces the packet size available to workloads, and Site-to-Site VPN does not support jumbo frames. Its maximum supported MTU is 1,446 bytes, with the usable value potentially lower depending on encryption and encapsulation overhead; configure customer gateway MTU and TCP MSS accordingly.

Open AWS Site-to-Site VPN →
AWS Transit Gateway image AWS Transit Gateway

Transit Gateway supports an 8,500-byte MTU on supported non-VPN attachment paths, while VPN paths have smaller limits. A 9,001-byte endpoint configuration therefore does not imply that equally large packets can cross a transit-gateway-based network.

Open AWS Transit Gateway →