You are here: Home / Blog / Usage of Overlay and Underlay Networking

Usage of Overlay and Underlay Networking

16th November 2012 by 11 Comments

The cloud platforms are talking about being able to get better software control of networking with “overlay networks”. These overly networks use protocols like VXLAN, STT, NVGRE or NVO3 (and more to come, I’m sure) and run between the virtual switches.

But when you are designing and planning for overlay networks, you need to constantly refer to the features in the physical networks. The problem is that there are dozens of virtualisation features already used the physical network – things like MPLS, VLANs, Device Contexts, Switch Virtual Interfaces. This leads to some confusing names for thing. It quickly gets confusing as to what features are where – I’ve had discussions where “virtual physical” and “virtual virtual” were tried. That didn’t work. 

So I’ve start using the term underlay network.

For example, the VXLAN overlay network needs to extensive IP and Ethernet multicast features enabled in the underlay network. While its possible for VMs to also use Multicast protocols in the overlay networks this may create performance problems in the underlay networks.

Here is another example:

The VXLAN ID in the overlay network is hidden from the VLAN ID in the underlay network. The physical limits of the underlay network is a maximum of 4096 VLANs while VXLAN has up to 16 million identifiers (users a 24 bit tag). 

I’d be interested to hear of any solutions to the problems of referring to virtual features in the physical network when having discussions around the virtual features of the virtual network.

Got it ?

Filed Under: Blog, Dictionary Tagged With:
About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

  • http://twitter.com/rovinguser rovinguser

    Maybe stick to the WLAN way of dealing with architecture : control, data and management plane. Should transfer nicely into SDN designed networks. Underlay and overlay in SDN networks sounds not appliccable to me. Nice post on very important topic: concepts

    • http://etherealmind.com Etherealmind

      The thing about OpenFlow is that tunnelling to build overlay networks is optional. If you use SDN/OpenFlow to configure flow for VXLAN or STT, then the overlay/underlay metaphor stills applies.

      For a direct flow based network, you don’t need overlays so the language becomes redundant.

  • sh0x

    I tend to use overlay and underlay as well, but sometimes I use ‘substrate’ or ‘physical’ network to refer to the underlay. The overlay can be virtualized with L2/L3 too. I’ve been using the term ‘virtual datacenter’ or ‘container’ to reference a tenants virtual network topology in the overlay.

    • http://thenetworksherpa.com/ john harrington

      +1 for “substrate”, instead of underlay. May just be familiarity with substrate or the fact that underlay makes me think of carpet.

  • http://twitter.com/DmitriKalintsev Dmitri Kalintsev

    I use “Transport” and “Service”.

    • http://twitter.com/icemarkom Marko Milivojevic

      While I like this better than Overlay and Underlay, it may lead to confusion when talking to someone familiar with QinQ, where Service VLAN refers to the tag use by the transport network.

  • http://twitter.com/chris_marino Chris Marino

    I tend to think of VXLAN and their ilk as part of the underlay network. They’re using physical addressing, etc. To me, it’s the tenant network that the overlay. They don’t see anything related to the physical network’s VLAN or VXLAN extensions. However, I often hear VXLAN used to describe the tenant network, so it gets confusing real fast.

  • Umair Hoodbhoy

    I’m okay with any new terms that don’t begin with ‘v’ or ‘virtual’. For many years I used to interpret ‘NIC’ and ‘network adapter’ as the same, and I’m sure I was not alone. This all changed when virtualization became dominant. On the one hand, you have ‘virtual physical NICs’ that are created by SR-IOV and are the logical representation of the physical card that plugs into the physical server. Ivan Pepeljnak uses the term ‘vNIC’ and ‘virtual physical NIC’ interchangeably in his webinars. On the other hand, there is the Virtual Ethernet Adapter, which is the term given to each VM’s NIC.

  • Alexandra Stanovska

    Makes sense, it’s “underlying infrastructure” anyway.

    By the way if we have hardware, software and possibly firmware, will actual devices that make underlay called “underware”?

  • Pingback: Network Dictionary – oVLAN and uVLAN — EtherealMind

  • Pingback: Network Dictionary – cVLAN and sVLAN — EtherealMind