If you don’t like Overlay Networking, it’s probably worth remembering that all of the alternatives solutions have failed to gain acceptance. Early alternative standards like Edge Virtual Bridging/VEB/802.1Qbg and VEPA/802.1BR have achieved only minor adoption for specific use cases such as blade server networking and fancy NICs.
Replacing Spanning Tree with TRILL for scalable & reliable L2 ECMP has not seen significant adoption. In part due to cost of software licensing but also because of the long delay in new hardware to support the TRILL protocols. The fact that major vendors have delivered proprietary versions is yet another drag on adoption. The IEEE wasn’t able to get its standards finished in a reasonable amount of time ( as usual ) and Shortest Path Bridging is struggling to find mind share.
The standards bodies that control MPLS are slow and may never be able to agree on developing an API that would allow programmatic API configuration of Label Paths. MPLS otherwise remains complex and expensive to deploy in the LAN for many reasons.
The scale/capacity of silicon is preventing large numbers of endpoints. Top of Rack switches with 16K CAM table entries are common but so are data centres with 1000 physical servers and running 20 virtual servers on each for a conservative total of 20000 MAC addresses. Extending VLANs in this type of data centre isn’t reliable.
It’s impractical to increase the 802.1q VLAN tag size to accommodate more than 4096 VLANs yet many data centres will need to solve this problem. Equally, MST and RSTP tree BPDU processing is beginning to exceed the CPU capacity on many switches, most switches are limited to just 64 MSTP domains due to silicon processing limits.
And finally, there is not any consistent and predictable way to configure network devices from all vendors using an API. SNMP was declared a failure for configuration in 2003, vendors and customer never seriously adopted NETCONF or other standards and instead focussed on ever more complicated command line interfaces or JAVA clients that load in a web page. Once this was OK because network engineers could work around the problem but customers are demanding NetDevsOps today, now, immediately. Instead of offering a solution, Cisco developed a proprietary API with onePK while other vendors are working with the ONF before starting again with OpenDaylight. Maybe one of these will develop an API.
Overlay networking has been repeatedly used for the last 20 years or more. IPinIP, GRE, NetBIOS over IP, DLSW and STUN are just a few I can call to mind. Overlays are not new. Controller based management of networks is not new either although it’s been lot longer since we saw that on mainframes as the 3745 Front End Processor.
Before you declare that Overlay Networking with Controller based management as the wrong solution that won’t work, consider that all of the other choices have failed to deliver. The “technology leadership” from networking vendors whom we trust to provide innovation and technological progress did none of these things.
Many vendors defend their positions by claiming that they gave customers what they asked for. Namely, more of the same. More CLI, more complex knobs, increasingly larger devices, faster speeds. No one wanted change, they say. Failure of leadership on both sides would be my judgement today.
So now we have Overlay Networking. Instead of an integrated virtual network offering that is tightly bound to the physical network, we have a completely abstracted network built and managed in software. It works, it is usable and products are shipping today.
Overlay Networking is the way forward because all of the other options failed in some way or another proving that once again better technologies didn’t reach the market but the one that works well enough and costs the least to implement does. Fifteen years ago the market “chose” Ethernet instead FDDI or Token Ring for exactly the same reason.
As they say, if you can’t love the one you want, love the one you are with.
Update: incorrectly named the 3745 Front End Processor as a 3270 FEP. Oh, the horror.