◎ Comparing Arista and Brocade VXLAN VTEP Hardware Termination

Today, Arista has announced the 7150S device. It’s low latency, 10 Gigabit and VXLAN terminating.

Ivan Pepelnjak, as usual, has done a great job of getting details that I missed in the briefing. Check his post Arista Launches The First Hard Termination Device for more information about it’s Ethernet functions. I’m more interested in VXLAN It’s my understanding that both Arista and Brocade were demonstrating VXLAN termination at VMworld 2012 ( and others too I think), so claims of first are somewhat, shall we say, tenuous.

What’s interesting to me is that Brocade and Arista are solving the same problem in different ways. Ivan has determined that Arista have decided to use the Intel chipset (I’m guessing the SM6000?) and then enable the tunnel termination features in the software.

Brocade has elected to deliver the first phase of their VTEP service using the ADX platform. The ADX has unique hardware architecture that is a Brocade silicon fabric internally with 3 FPGAs providing services. Today, the ADX configures the FGPA software to provide SSL termination, load balancing and the fabric acts as a low latency high performance network connection. Frames cross the network fabric and are passed to the FPGAs for handling (not unlike the Arista 7124FX Application Switch). Thus it’s a high performance network load balancer because the FPGA software is configured to load balance packets. The Brocade ADX can be adapted to parse and mangle VXLAN protocols by upgrading the FPGA software. This approach might be superior when you consider that the VXLAN control plane is not yet complete and likely to go through a number of changes. In principle, it should offer more flexibility.

On the other hand, Brocade won’t have VTEP in hardware for their VDX until the next silicon revision arrives. Arista can deliver the VTEP right down to the edge of network.

My current design challenge with VTEP is achieving L3 separation in a secure manner. There are two requirements here

  1. I have existing production and non-production zone in the physical network that must interoperate.
  2. Some servers must connect to a differnt security zones on physical firewalls

For both of these, the traffic flows must not ‘mix’. Today, VLANs are not accepted as a security tool, and VTEP terminates into simple VLANs at the termination points. I need to be able to have MPLS VRFs or some other L3 control to make VTEP useful.

Next year, the Broadcom Trident2 will be shipping and I’m told that it will have support for hardware termination for tunnels. This opens the way for Avaya, AlcaLu and minor players to deliver the feature. I’m not certain what Cisco or Juniper is planning in this space.

The EtherealMind View

Being able to terminate VXLAN in hardware is the first step in the solution. The box is ticked.

Now, to us this feature I need to terminate VXLAN correctly to match the existing security processes. This means inside an MPLS VRF, or PVLAN, or some other L3 protected area in the OS.

At this time, neither Brocade or Arista does this. My broad conclusion is that VXLAN and VTEP have a long road ahead before they will be widely useful. Don’t hold your breath, it will be while before the software is completely finished in the network layer, and VMware still hasn’t finished VXLAN in vSphere either.

I’m planning for VXLAN but not rushing into it.

Brocade Press Release is here

Arista Press Release is here

Disclosure

I attend Brocade Analyst and Tech Day in Sep 2012 as a guest of Brocade who paid for my travel and accomodation where I learned about the Brocade ADX. I received an advance briefing from Arista under Embargo over the Internet.

  • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

    > Today, VLANs are not accepted as a security tool

    Would you care to elaborate, or to point out a URL where you may have elaborated on it previously?

    Cheers!

  • http://twitter.com/dgourlay Douglas Gourlay

    Greg, a few corrections:

    1) The Intel Chipset is the FM6000, not the SM6000.

    and

    2) The Arista switch terminates the VXLAN tunnels in hardware, on the Intel FM6000, not in software. This enables about 640Gb/s of VXLAN tunnel termination and does carry an incremental performance delay of 40ns per frame.

    I believe the other system you mention uses the FPGA to terminate the VXLAN tunnels and I don’t remember seeing it actually work at VMworld – plus its 1/8th the performance at >2x the price :)

  • http://twitter.com/dgourlay Douglas Gourlay

    Also, am not sure I concur with your design challenge. I can map a VXLAN-VNI to a port, port+vlan, vlan, or loopback adapter in the system. This way if as you indicate VLANs are not a trusted solution I can map the VNI to a physical port and support VNI per physical interface on the firewall or other security policy enforcement point.

    The control plane traffic per VNI is as separate as it is in most/any VRF/MPLS setup. i.e. the tables are shared but each route/segment is tagged appropriately and the overall routing process is still largely shared – i.e. same PID but managing table structure based on VRF Label first, then calculating a graph per label.

    This is not markedly different than using a separate mRoute/IGMP Group per VNI or stacking associated VNIs within a specific IGMP group provided you feel broadcast/unknown/multicast frame distribution to a VTEP who doesn’t really care and will drop and not forward is acceptable. Fortunately you have both options and can implement that which makes your team the most comfortable.