@FStevenChalmers left three questions on the Do Networking Vendors Make Their Own Products ? – EtherealMind post.
Question One: This disaggregation tends to work best when there’s a single ASIC family for the software vendors to focus on. In servers, one price of disaggregation has been the decline of other CPU architectures in favor of Intel x86. In data center switch disaggregation, the market already seems to favor a certain line of Broadcom switch ASICs. Is it time for the market to make a single choice yet, to the exclusion of all competitors?
From my current (and admittedly insufficient) knowledge of the Broadcom Trident/StrataDNX chipsets, current silicon is designed for 1RU or 2 RU devices that are best suited for top of rack switching. In my current view, Broadcom has decided to target the high volume market and instead of producing a full spectrum of silicon solutions. The feature set of the existing chipsets is incomplete with limited TCAMs and poor pipeline flexibility. I’ve written before about Cavium Xpliant
Compare this approach to Intel Pentium x86 CPUs that have options for a wide range of use cases – server, desktop, laptop and mobile. Intel covers the entire market.
Why is Broadcom only producing low end silicon ?
- My best guess is that there isn’t enough money in switch chassis to make it worthwhile.
- networking silicon isn’t a large market and not worth the investment for Broadcom. This could be good for Arista / Brocade / Cisco / Juniper who can continue to use their own silicon and not write down the sunk cost of those products.
For this reason, there are other vendors who are looking to enter the market. Cavium Xpliant, Intel FM, Centec Networks are just a few of about 15 vendors. Its possible that Broadcom can dominate but I judge it unlikely in the current market unless the wave of buyouts continues to extremes.
Hardware Independence Is Under Way
We have a way forward, the Switch Abstraction Interface (SAI) is developing a standard AP to abstract the switch silicon from the software for the most common switching features. Vendors of network operating systems (NOS) will have substantially reduced development and silicon choices if their low cost, low performance and low feature devices use the SAI instead of developing custom kernel modules for each vendor chip.
Note: Only the kernel module needs to development and startups such as Big Switch, Cumulus and Pluribus have demonstrated that it takes very little funding to make this happen. This isn’t to say that its easy, it certainly is not. However, a small team of experts can develop such code and sustain profitability. Large teams and big money are not required.
Reference: Networking/SpecsAndDesigns – OpenCompute
Observation: Microsoft did attempt CPU independence in the early 2000’s that ultimately failed because Intel was willing to invest in partnerships and provide significant rebates to Microsoft and server makers. There are no signs that Broadcom is doing this for Trident/StrataDNX but it is providing resources to software developers eg. Facebook for FBOSS.
Question 2: Do the available disaggregated switch software sets have the features you need yet, and at current pricing will they be able to invest to achieve those feature sets?
Certainly, the current software feature sets are incomplete. But the speed of development shown by Arista to equal (or improve) the best efforts of Cisco or Juniper in five years suggests that the velocity of development is changed. And Big Switch and Cumulus Networks show that software-only companies can repeat the formula.
Today, every company making a NOS is duplicating 90% of the features of a competitor. There is small amounts of differentiation which is basically a game of trivial pursuit. If we continue to build network software in the same way, then
Note that reliability is not a feature but minimum competency for a product. In 2015, discussion of a vendor as reliable is reprehensible.
Consider that all vendors must produce a BGP application to run on their operating system. Each vendor duplicates the effort to develop, test and validate it. Why ? Does each vendor truly produce the best BGP application ? 1 Why do vendors waste so much money on competing by copying BGP which should be indetical on every device ?
As an example, consider the ONIE project that has developed a common process for booting operating system on devices. Think how much pointless time is wasted on learning the various ways to load an operating system on, say, Cisco devices. Each hardware platforms uses a totally different process – and what benefits do customers get ?
So I would say that vendors currently waste vast sums of money on pointless “competitive” positions in device software. How many vendors making Microsoft Word do we need ? Disaggregation gives me hope that vendors will stop duplicating the same effort and reuse open source components for the core operating and differentiate products in hardware or other software capabilities.
Features: I find myself questioning whether networking is, for practical purposes, is “feature complete”. Do we really need more BGP features ? How much customer demand is there for MPLS-TE or RSVP-TE ? We know that QoS doesn’t work reliably and is “best effort”, how much time do we waste on pointless tweaks ?
How many customers really need the density & performance of an Ethernet chassis ?
Question Three: Will you have staff to support the combined software and hardware you want to deploy as switches? If not, who do you intend to call, how will they be funded, and (given that replacing broken hardware is the easy part) what will they do when your call is about a software bug it takes CCIE level talent to even understand the description of?
In this case, I would look to how servers are operated. Who supports Linux on a Dell/HP Server ? Who troubleshoot the hardware or software in a server platform when running MS Windows Server ? If disaggregation was a fundamental technology problem, then x86 servers could never be disaggregated.
It is also true that Intel invests billions of dollars every year to make ensure software/hardware compatibility. Thousands of head count are allocated to work with developers from Microsoft and Linux to ensure that their kernels are reliable and compatible while also working on new hardware/software integrations.
Second, we already staff to “support the combined software and hardware you want to deploy as switches”, thats the job of operations today. The vendor help desk is a single telephone number but the hardware support is totally separate from the software team on the helpdesk. This won’t change if desegregation occurs.
- In fact, most of them are very poor judging by the number of bugs found and regularly shipped to customers. ↩
Other Posts in This Series
- Three Questions on Network Switch & Operating System Disaggregation (9th June 2015)
- Commodity Manufacture is the Majority of Switch Products (24th March 2015)
- Four Different Types of Ethernet & Whitebox Products (24th March 2015)