EtherealMind

Software Defined & Intent Based Networking

You are here: Home / Archives for Blog / Briefing

Briefing: Project Calico – BGP-driven SDN Without Overlays

By Filed Under: Briefing

The following is a “paper review” of Project Calico following a recent briefing. I’ve conducted a short review of the technology and business issues around the product and conclude that its unlikely to be competitive with Docker libnetwork that was announced a few weeks back.

What is it ?

Its an open-source project sponsored/owned by Metaswitch that promotes a model of programming Open vSwitch on Linux (see below for response from Calico team). using BGP as an API to configure Linux iptables for FIB manipulation and forwarding control. This provides an the low quality baseline for an SDN solution for networking in Linux / KVM / OpenStack for cloud services.

On each compute host, Calico uses the proxy ARP technique to intercept all ARP requests from each workload, returning the MAC address of the compute host as the next hop. As Calico is responding to all ARP requests from a workload, there is no distribution of MAC addresses between compute nodes and, hence, none of the usual proxy ARP scalability issues arise. Frequently Asked Questions — Project Calico 0.27 documentation

This is a substantially different approach to most other SDN solutions. ARP hijacking does present operational risk to certain types traffic loads in the enterprise based on my experience.

Why Use It ?

  1. Because BGP is the solution to every networking problem ever.
  2. Because high quality BGP code exists in open source libraries and reduces development time for Yet Another Network Controller.
  3. Because you don’t believe in encapsulation or overlays.
  4. Because you want a network stack that works in OpenStack and Docker.
  5. Because you only have a data centre LAN.
  6. Because Metaswitch has a product of interest or you believe that Metaswitch can build a community around Project Calico that is better than Docker, Weave, NSX, Contrail, ACI, Nuage Networks or other SDN solution for Linux hosts.

Reference: Why Calico? | Project Calico

Concerns

Licensing Project is open source with an Apache license but Metaswitch is the project owner and controls contributions and some rather odd patent assertions that are onerous and need further investigation. I’m guessing that Metaswitch wants to mainline community contributions into its product for service providers and didn’t keep the lawyers under control or carefully watched. Contribution Guidelines

Security I have concerns around the integrity of Calico as cloud infrastructure. The use of next hop addressing means that spoof attacks could be a practical attack vector. Calico does configure iptables on hosts but this doesn’t protect against spoofing. While Calico implement a endpoint security as a form of stateless firewall using profiles, it doesn’t seem to address in network attack surfaces. Would require more research to confirm interpretation.

Server Resource This might be solved but the resource costs of holding a large BGP table and metadata in the server OS needs research. Servers are high performance and networking in Linux is really fast, but I would like some more data about performance impacts.

Data Centre Only The Calico solution relies on Linux host running all VMs. The documentation discusses gateways and access to the main network but I could not easily establish how connectivity to external networks is managed. A data centre require seamless integration to WAN, Wireless or Internet/DMZ networks and must be seamless, therefore incurring substantial technical debt.

Testing These Regressions

BGP BGP is a fine protocol for distributing state especially when there is limited CPU or memory (which is not true in 2015). I take the general view that it is not a good protocol for configuration, monitoring or API manipulation. Much better alternatives such as NETCONF, OpenFlow/OBSDB, exist and are well proven.

Because you cannot query BGP instances and validate or test that data is consistent or coherent across all instances. Using BGP to share state requires dozens of other protocols to monitor, analyse and operate ? Complexity like this must have a payback elsewhere to be worth building technical debt like this.

Encapsulations Networking has been using overlay networks through protocol encapsulation for more than 30 years. VLAN tagging, MPLS Tagging, GRE, IPSec, SSL VPN are the most common and there are dozens of other lost to history.[1] Overlay networking isn’t new either and overlay networking in SDN is widely implemented for good technical reasons. I reject totally the assertion that overlay networking isn’t the best solution. Issues around tunnelling are directly related to the limited capabilities of path management and decades old routing protocols which have limited functions. Recursive paths are routing problems not limitations of encapsulation “per se”.

Docker and VM networking Docker has introduced native networking support. It works with lots of existing products. Its even an enabler for Project Calico but mostly people will be using Cisco / VMware/Nuage for running their data centers. There are dozens of SDN solutions that offer this feature, Calico is just another one with relatively limited features at low cost.

EtherealMind View

I did some searching on Reddit and Hacker News, and decisions on using Project Calico seems to focus on what developers have “found” rather than rational and knowledgeable decision making. Signs of adoption are positive for the project but I’m not convinced that it can achieve momentum since Docker has announced a native solution.

The technology assertions don’t make sense to me. The lack of overlay networking and the use of BGP as a programming interface is a substantial technical debt that isn’t offset by gains in other areas.

It is my perception that Metaswitch is focussed on building solutions for Service Provider (SP) Market and targeting Calico as an SDN solution to support its NFV products. In this context, the use of BGP and lack of overlays may be appealing to those customers but for cloud developers or enterprise customers, the product doesn’t seem to be relevant.

References

Blessay: Linux on Network Switches Makes Sense. For Your Career – EtherealMind

Networking Receives an Upgrade | Docker Blog

  1. CLAIM  ↩

 

Response from Project Calico

The team at Project Calico have posted a useful response which addresses some of the points raised in this piece. In particular, I made an error about Open vSwitch

Firstly, Calico does not program, or use Open vSwitch in any way.  In fact you can remove Open vSwitch in a Calico network, as its functions are completely bypassed. Instead, Calico programs the native routing function of  the Linux kernel. Secondly, we do not use BGP as an API, we use it as it was intended, a routing protocol that tells other Calico nodes (and the rest of the infrastructure) where workloads are at any given point in time.. Our API is in the form of an etcd data model. Our use of BGP should be totally opaque to anything trying to program Calico.

Briefing: Corsa Technology

By Filed Under: Briefing

Takeaway – Corsa Technology is focussed on provided a hardware data plane for SDN WAN use cases.

briefing-logo-opt

WAN Networks are different A WAN network has more comprehensive requirements for fabric, shapers and buffers to manage data in a more heavily loaded environment. For example, Broadcom Trident 2 has shallow buffers so that queueing is less effective. For the Data Centre, dropping a few packets at high speed and low latency is perfectly acceptable but in the WAN is not.

SDN Data Plane Corsa Technology has built “forwarding engines” to be a WAN data plane for SDN networking. In SDN you are always buying a data plane to move packets across the network. But can you do this with same silicon everywhere ? For WAN applications you need a flexible silicon fabric that can adapt to the different requirements.

Technology Corsa Technology has built a forwarding engine that uses FPGA technology instead of more common ASIC. FPGA use software programs to implant functions and capability in the silicon plane. The advantage is ultimate flexibility for functions and features at the price of increase software development and moderate cost (ASICs are generally cheaper in volume).

Corsa Technology would propose that FPGA are suitable for SDN WAN application since they can deliver rich functionality and even change the performance of forwarding plane over time.

Today, Corsa is OpenFlow 1.3 compliant and would work with most other OpenFlow controllers.

Importantly, Corsa Technology is shipping 100Gbe networking today.

This hardware is significantly different a typical 1RU whitebox Ethernet switch. The data forwarding capabilities are significantly different and more capable. In the Data Centre traffic is about volume and speed and fibre costs very little.

In the WAN, there is an alternate use case where fibre costs outweigh equipment costs and silicon investments enable better returns.

corsa-networks-data-plane

The EtherealMind View

SDN WAN is the a major trend in the SDN/NFV space and Corsa Technology is well positioned to bring FPGA technology to address some use cases. FPGA have had mixed success in the last 10 years due to software complexity but I believe that new software toolchains and programming practices may see better technology.

I’ll be looking closely at this technology in preparation for a 3hour techtorial at Interop New York on the “10 year strategy”  future of the WAN.

Perspective on Arista IPO and Market Positioning

By Filed Under: Blog, Briefing

Arista had their IPO and is now valued at $3.4Billion or so. Most coverage has highlighted that Arista is competing with Cisco but really their products and strategy looks more like HP or Brocade.

Merchant Silicon

For the last 20 years, a “silicon moat” that has protected Cisco and other networking vendors from market competition. The cost, time and risk in designing and building the ASICs for networking equipment prevented market entrants and the result has been what is colloquially known “Cisco and the eight dwarves”.

Arista has taken merchant silicon,open source software and modern software development methodologies and built a hardware product that competes equally with the top networking vendors. The value is not in the hardware but in the software that drives it.

Here is how I see Merchant Silicon fitting into the Arista value chain.

 

arista-product-values-opt

Focussed and Rapid Software Development

At this point, Arista has had good traction in the data centre and especially with financial organisation in the trading floors. Arista was able to use its rapid software pipeline to add features to the EOS operating system that addressed the transactional market in HFT. When Microsoft came calling with a BGP SDN requirement, Arista leveraged EOS and rapid development to add features to win major revenue (22% of company sales according to IPO documents or $80 MM) and a signature customer.

Arista has resisted customer pressure to add features. EOS implements features that solve core networking problems and, so far, Arista has been measured in adding features. Code bloat will lead to the same problems that Cisco IOS faces in terms cost, service overheads and most importantly, reliability.

Hardware Portfolio

Today Arista is continuing with chassis-centric product strategy which is a differentiator to whitebox Ethernet. Chassis products are complex to design and customers are less likely to regard them as consumable items.

As hardware, the Arista product range is comparable to any competitors. While there are differences, there is no specific weaknesses.

Growth

Arista has many growth options outside of the data centre. The most likely destination is to carrier/service provider via NFV technology because the EOS architecture is well suited to integration with OpenStack and other cloud orchestration tools. The rapid development cycle of EOS can provide the momentum to bypass incumbents at the edge of the carrier network.

Threats

Whitebox Ethernet

  • merchant silicon enables many companies to innovate on hardware on a similar basis to Arista. Broadcom has it’s own operating system and companies like Facebook are building their own hardware and software. This shrinks the available market.
  • Ethernet Chassis products are becoming less relevant outside of scale network as new ECMP technologies in 1RU switches mature.
  • Virtualization means that total ports are 1/10th of what 1GbE port sales were previously.

Converged Infrastructure

  • Companies like Nutanix and VCE have proved that there is demand for converged infrastructure. The network is no longer separate from servers and storage.
  • shrinks the addressable market and prevent entry to some markets.
  • to date, Arista involvements in OpenStack and Opendaylight is limited with few partnerships in this space.

Briefing: HP Helion, Virtual Cloud Networking, FlexFabric 7900, SDN and OpenStack at HP Discover

By Filed Under: Blog, Briefing

HP Discover is happening this week and have three announcements
that I’m summarising here. They are Virtual Cloud Network (VCN), FlexFabric 7900 switch and a supporting package of consulting to implement cloud in your organisation.

Virtual Cloud Networking

Virtual Cloud Networking (VCN) is the HP networking element stack for OpenStack. It will be a key element of HP Helion, the commercial OpenStack distro from HP. HP continues to expand it’s commitment to OpenStack with Helion transforming into public and private option. VCN uses VAN to handle the network control functions so you can consider VCN as an SDN app. OpenStack has reasonably poor networking and for HP Helion to be successful it needs to use VAN SDN components and the VCN is that element. Since VAN SDN platform is well established it seems reasonable to be positive about this move.

Like other Cloud & SDN networking strategies, VCN goes beyond simple overlay networking to provide full stack integration for cloud orchestration. I haven’t seen a demo but I would consider this to be somewhat competitive with Cisco ACI strategy. Like Cisco, this strategy will for customers who already own and use HP switches which have been SDN ready for some time.

FlexFabric 7900 Seriesbriefing-logo-opt

A hardware announcement is a rare thing these days but HP has announced Flexfabric 7900 . I would broadly compare it to the Cisco Nexus 6000 family or Dell Z9500 in terms of 40G / 100G

There are some differentiators.

Midplane Design switching is on the line cards and all the other benefits of using mid-plane architecture accrue.

Broadcom Arad Chipsets – The Flexfabric 7900 uses the next generation Broadcom Arad chipset. I can’t find anything to tell me how the Arad is different to the Trident2 and Broadcom doesn’t let vendors talk about their product so who knows whether these are upgrades or downgraded silicon. Either way, it uses merchant silicon and this means that the product focus has been on features not basic functions. The switch runs ComWare including IRF and others.

Virtual Output Queues The Broadcom chipsets often have limited buffer capacity which can have performance impacts. Most larger vendors have workarounds and HP has implemented VOQs to improve the performance to line rate.

Professional Services

One of the more interesting aspects is that HP has developed an business unit delivering professional services around cloud implementation. This isn’t new since Cisco has a flexible services packet in InterCloud and Flexpod, VCE has their one size fits all package and startups like Nutanix are in the mid-market.

The potential difference here is HP is offering an framework and methodology that is more like a management consultancy. There are many companies who are struggling to switch to the cloud and this type of service is attractive.

This service is important because Cisco is reaching into this market with consulting methodologies through InterCloud or marketing packages like FlexPod. HP is one of the few vendors with an end-to-end portfolio for the data centre and this could be strongly leveraged with the right approach.

The success of converged infrastructure is the services package that goes with the deployment. I often consider this type of sale as “comfort package” where end-to-end ownership appears to devolve responsibility to a third party. Puts HP Cloud infrastructure on a better footing to compete with IBM and other integrators.

The EtherealMind View

HP continues to show signs of convergence between the business units. The stronger ties between servers / network / storage bode well for customers who want to certainty of a single supplier.

Network Break Podcast

Network Break is round table podcast on news, views and industry events. Join Ethan, Drew and myself as we talk about what happened this week in networking. In the time it takes to have a coffee.

Packet Pushers Weekly

A podcast on Data Networking where we talk nerdy about technology, recent events, conduct interviews and more. We look at technology, the industry and our daily work lives every week.

Our motto: Too Much Networking Would Never Be Enough!

Find Me on Social Media