You are here: Home / Blog / VMware vCloud, Isolated Networks and L2 network overlays.

VMware vCloud, Isolated Networks and L2 Network Overlays.

12th October 2010 by 5 Comments

Yeah, so I’m at VMworld in Copenhagen. It’s nice here.

VMware vCloud and networking

VMware vCloud is an interesting spin on the virtualisation story that focusses on automated provisioning and deployment of virtual servers. And part of provisioning servers is the networking connectivity. Most people will know that VMware has the vSwitch and Distributed vSwitch as part of the VMware that already removes a lot of networking configuration from the physical devices. Lets assume for the moment that this is acceptable from a security and performance perspective. Note that Cisco’s Nexus 1000V rounds out the software switching options with further feature enhancements on the Distributed Switch.

In the vCloudDirector platform, you must create pools of networks resources that are then used by the servers to connect to the physical platform. Firstly, for this to work, the network resources need to be deployed between many servers and implicitly, the physical servers must by hyperconnected to many VLANs.

Network pools can be one of these types:

  • VLAN-backed – a range of VLAN IDs and a vNetwork distributed switch are available in vSphere. The VLAN IDs must be valid IDs that are configured in the physical switch to which the ESX/ESXi servers are connected.
  • vCloud isolated networks – An isolation-backed network pool does not require pre-existing port groups in vSphere but needs a vSphere vNetwork distributed switch. It uses portgroups which are dynamically created. A Cloud isolated network spans hosts, provides traffic isolation from other networks and is the best source for vApp networks.
  • vSphere port groups – Unlike other types of network pools, a network pool that is backed by port groups does not require a vNetwork distributed switch. This is the only type of network pool that works with Cisco Nexus 1000V virtual switches.

Lets pretend, for a minute, that that it’s NOT a really bad idea to enable all VLANs on all ports because of STP/MSTP/RSTP/PVST, or security, or backbone performance, or broadcast storms created by ARP floods etc etc. Right, got that, lets pretend that. As Ivan Pepelnjak at IOS Hints notes (and we have often discussed) the size of L2 domains is a major concern in scaling out VMware clouds, and watching senior VMware engineers simply ignore these issues is somewhat chilling.

So, you would would want to be able to deploy a server to any VLAN, anywhere. OK, I can kind of go with this (KIND OF).

vCloud Isolated Networks

Of the three types of vCloud Networks, the most interesting the ISOLATED network. This means that vCloud servers can have their own private network. This private network is defined independent of the network admins, to quote “it’s too painful to ring up the Network Ops and get VLANs configured so we set this up so you don’t need to”.

The slides then go on to describe this network as using MAC – in – MAC encapsulation. This means that a private Layer 2 Network is being overlaid on the physical network between VMware physical hosts and the vSwitches

Yeah, that’s a problem waiting to happen. I’ve got some research to do on that one to understand the processing impact of de-encapsulating Ethernet frames and what VMware is using for loop prevention. Until now, I think VMware has used software rules to ensure that vSwitch configuration are never able to build looped configurations, but by creating a L2 overlay on top of an existing L2 network is going to have some interesting challenges for Spanning Tree and switch loads and frame MTUs.

The road to the cloud isn’t getting any easier.

Disclosure

HP have provided travel, accomodation and entry to VMworld Copenhagen 2010 however, my opinions and thoughts remain stubbornly my own.

Filed Under: Blog Tagged With: , ,
About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

  • http://packetpushers.net Ethan Banks

    MAC in MAC encapsulation…a solution in search of a problem, at least with the justification as explained thus far. Everything in me screams that it’s a bad idea, a bad road to go down for VMware.

    Just because they can doesn’t mean they should. To me, it sounds like VMware is trying to…in a way…engineer the network out of the picture, make it irrelevant. That won’t scale.

    • http://etherealmind.com Greg Ferro

      Yeah, I spoke with some senior people about it and got this gushing view that L2 scales and works.

      So we will be having a podcast with some senior VMware people……..

    • Paul Fowler

      Mac-In-Mac is not a solution looking for a problem. Case in point. Our network is managed by a different State organization. It takes 6 weeks to get a new VLAN if approved. On average we take 6 months to set up a virtual machine in vSphere and the @#$*@#! network group is the problem. No automation, no self-service, no mult-tenancy. Changes can only occur during change windows that are at obsurd hours after multiple weeks of notification… This is due to the physical hardware dependencies.

      Network folk need to realize that physical networking is like physical servers. It only exists to provide capacity and redundancy. You add physical networking when you need more resources. Carving up the physical resources must be enabled at a higher logical level and the network groups need to realize this. Continually reconfigure physical switches for a cloud infrastructure? Really????? Sorry, this post hit a nerve. I won’t even begin to describe how the network folk prevent viable Disaster Recovery for our organization by preventing location transparency with their dumb coupling of physical resources (like local physical firewalls rules) to virtual machines.

      The world needs more network admins that are tuned to the needs of public/private cloud infrastructure.

  • http://www.tali.com tali

    VCDNI is NOT MAC-in-MAC , look for lab-manager encapsulation (akimbi) to find out more , and look for port-group-isolation technology to see how it works, bottom line – NOT secure like VLAN, isolation done through clear text packet built , not port-based mapping !

    • http://etherealmind.com Greg Ferro

      VCDNI is MAC-in-MAC and is clearly described as such in the documentation.