Layer 2 extension using VXLAN with Brocade VDX

VXLAN. I’m sure you’ve heard a lot about it, and you might have even configured it. For those not in the know: it’s a fairly easy way to overlay L2 networks over a L3 infrastructure, allowing you to stretch a L2 domain between L3 PODs or even between sites. One of the applications could be vMotion for example, allowing you to transfer VMs on the fly between PODs.

In short VXLAN overlays are tunnels between VXLAN Tunnel End Points (VTEPs), with the underlying L3 infrastructure not having to know anything about VXLAN. One thing to note is that due to the VXLAN encapsulation, the frame size increases (+8 bytes) so you might have to adjust the MTU size on your routers as VXLAN packets should not be fragmented.

For those interested, here’s the RFC (RFC7348).

Brocade VDX6740 and VDX6940 platforms support layer 2 extension using VXLAN in two ways:

  1. Layer 2 extension mode
  2. VCS Gateway for NSX

The first option works without any controllers, and using the VDX platform as the VTEPs and to build the tunnels. The second option has the VDXs connect to the NSX Controller to automatically create tunnels in conjunction with NSX. This blog post focuses on option 1, but don’t worry: I’ll dedicate one of my next articles to NSX and the integration with VDX as this is becoming an increasingly hot topic.

There are a few ways you can set this up, check with your local SE what the best way is to do this.

I’ve got it running in our lab like this, with VDX 6940s as the spine, and 6740s as leaf switches: VXLAN Console

What you see is two PODs, connected via L3. There’s a VXLAN gateway setup on each of the VDX 6940s in the different sites (RBridge 101 & RBridge 201), which are VTEP 1 and VTEP 2 respectively. I only have 1 device per site available, you could have multiple of these running in VRRP-E mode for example.

There are two servers which are connected to VDX 6740s in different sites, with a different VLAN per site but both in the same subnet. I could have used the same VLAN on both sides, but for illustration I used different VLANs to show that the one thing that really matters is the VXLAN Network Identifier (VNI).

To get the layer 2 extension to work, the following is required:

  • Layer 3 connectivity between the VTEPs.
  • Define what switch(es) you want to use as VXLAN gateway(s).
  • Specify site names, one for each site you want to connect to.
  • Define the VLAN to VXLAN Network Identifier (VNI) mapping,. This is a 1:1 mapping, the VLAN per site can be different but the VNI needs to match on both sides.
  • On each site, define the VLANs you want to extend.

To make it work, you need to configure this as follows:

RBridge 101

Configuration Explanation
overlay-gateway vxlan-gw1 Define the name of the overlay-gateway.
type layer2-extension Type of the Overlay Gateway, either NSX or Layer2 Extension.
ip interface Ve 150 vrrp-extended-group 150 Specify the source L3 interface for the tunnel, either a Ve interface or Loopback interface.
attach rbridge-id add 101 Specify the RBridge IDs of the switches that need to be used for the VXLAN gateway instance.
map vlan 99 vni 5000 Maps the VLAN to a VXLAN Network Identifier (VNI). This VNI needs to match on both sides!
site POD2 Name of the site you want to connect to.
ip address The IP address of the other site.
extend vlan add 99 The VLAN(s) you want to extend between these sites.
activate Activate the configuration.

RBridge 201

overlay-gateway vxlan-gw1
type layer2-extension
ip interface Ve 250 vrrp-extended-group 250
attach rbridge-id add 201
map vlan 199 vni 5000
site POD1
ip address
extend vlan add 199

We now have a VXLAN tunnel running over our L3 network, extending the L2 domain between the two PODs over VNI 5000 and allowing the servers to connect to each other.

Last but not least some verification commands, to check the status of our VLANs and the associated tunnel (one per VNI): RBridge 101 RBridge 102 Tunnel Brief

That’s it! Hope you learned a thing or two about getting started with VXLAN, feel free to reach out if you have any questions and stay tuned for some updates on this topic!

Related Articles

comments powered by Disqus