VXLAN

VXLAN, short for Virtual Extensible LAN, builds an overlay network on top of an IP underlay. The overlay defines logical Ethernet segments, while the underlay provides IP connectivity between tunnel endpoints. A VXLAN tunnel endpoint, or VTEP, encapsulates an Ethernet frame in VXLAN, UDP, and outer IP headers; the receiving VTEP removes those headers and delivers the original frame to the appropriate logical segment. The standard UDP destination port is 4789, although some implementations use a different port.

Each VXLAN segment is identified by a 24-bit VXLAN Network Identifier, or VNI. This provides approximately 16 million possible segment identifiers, compared with roughly 4,000 usable VLAN identifiers. Devices in different VNIs remain in separate logical Layer 2 domains unless a routing function connects them. A VNI is a segmentation identifier, not an encryption key or an authorization mechanism.

For hybrid-cloud design, distinguish the routes that deliver encapsulated traffic from the connectivity inside the overlay. An on-premises VTEP needs an IP route to its cloud-side peer, and the cloud-side peer needs a return route. AWS Direct Connect or AWS Site-to-Site VPN can provide that routed underlay connectivity. Neither service automatically extends an on-premises VLAN into a VPC or acts as a general-purpose VXLAN endpoint.

For example, an organization might run a virtual network appliance on Amazon EC2 and establish a VXLAN segment with an on-premises appliance. That segment could support a legacy workload that requires Ethernet adjacency. The design must explicitly define which interfaces participate in the overlay, where default gateways reside, how MAC addresses are learned, and how broadcast traffic reaches remote participants. An ordinary VPC subnet does not become a customer-controlled Ethernet broadcast domain simply because an EC2 instance runs a VTEP. Workload attachment and bridging or routing must be supported by the chosen appliance and guest-network design.

VXLAN specifies encapsulation, not a complete control plane. Implementations can learn remote MAC locations from traffic, use configured peers, or use a control plane such as BGP EVPN. Broadcast, unknown-unicast, and multicast traffic requires an explicit distribution mechanism, such as ingress replication to remote VTEPs. Do not assume a cloud underlay supports the multicast behavior expected by an on-premises VXLAN fabric.

Encapsulation also reduces the usable packet size. With typical Ethernet framing and an IPv4 underlay, VXLAN requires about 50 bytes of additional capacity relative to the inner IP packet. An underlay IP MTU of 1500 therefore commonly requires an inner IP MTU of 1450, with further reductions when additional encapsulation is present. Validate the complete path, including VPN limits, and test large packets. TCP MSS adjustment can help TCP traffic, but it does not solve oversized UDP packets or all path MTU discovery failures.

VXLAN does not provide confidentiality or integrity protection by itself. Restrict tunnel traffic to intended peers, enforce segmentation at the endpoints, and use an encrypted transport when required. Monitor both layers: a healthy underlay route does not prove that VNI mappings, MAC learning, or overlay gateways are correct. Troubleshooting should examine outer IP reachability, UDP filtering, MTU, endpoint configuration, and inner traffic separately.

For architecture and certification scenarios, prefer ordinary routed connectivity when workloads do not require Layer 2 adjacency. Stretching an Ethernet segment can preserve addressing or satisfy a legacy dependency, but it also extends broadcast scope and couples failure domains across locations. VXLAN offers overlay flexibility; it does not remove the need for resilient routing, loop prevention, security policy, or application-aware migration planning.