You are here: Home / Blog / Juniper QFabric – My Speculations

Juniper QFabric – My Speculations

1st June 2011 by 10 Comments

Ivan has blogged about how he thinks Juniper QFabric works Speculation: This is How I Would Build a QFabric, while I agree in terms of the software elements ( at least, they look like good guesses to me), I would like to discuss more about the hardware:

There are three components in a QFabric kit list :

  • Interconnect
  • Edge Switch
  • Manager

My understanding is that the Interconnect chassis is a simple Fat Clos tree silicon and the QF/Edge acts like a blade in chassis that performs all the forwarding lookups, traffic classification and backbone encapsulation. The QF/Director acts as a pure management plane only.

Qfabric guess01

In this understanding, the uplinks between the QF/Edge switches and QF/Interconnect are effectively “backplane” connection and traffic on the uplinks is tagged / munged / processed into a form for forwarding across the Silicon Fabric that is in the InterConnect chassis, and most likely the software uses the software mechanisms that Ivan describes in his post. This is the same idea as the connections between line cards and the supervisor in a traditional switch chassis which use a frame/packet encapsulation devised to meet the platform requirements.

The QF/Interconnect has line cards that have low cost silicon for network interfaces and not much else (no Distributed Feature functions or local switching) , and a bunch of silicon switchesblades that provide a Fat Tree Clos switching fabric for forwarding plane for the entire QFabric

Aside from the fact that Juniper performs the forwarding decision is performed at the QF/Edge, it’s the same concept Cisco FabricExtenders (FEX) use by extending the backplane to external modules with a proprietary tagging scheme between the edge device and the core switch. The big difference is that the Cisco approach means that a “fat switch” with high capex (good for Cisco’s business) is needed to scale this, where the Juniper solution means that InterConnect is cheaper and the intelligent edge scales more effectively. Each purchase of an QF/Edge increases the forwarding performance of the entire system, and the funding process is better aligned to a private / public cloud revenue model. This is really vital to successful cloud deployments and causing major change in the storage industry as well.

Postscript

This is the unit that was on display at Interop – its look huge, and it IS big but don’t be scared – but it’s the full size unit to show off to the press – there are quarter and half sized switching planes to be released in the near future.

Juniper QFabric - as seen as InterOp (click for larger)

So a couple of points about this unit that I’ve been told :

  • just because it’s big, doesn’t mean it is expensive. I’m assured it’s simplicity means it’s low cost.
  • there are smaller and cheaper versions than this (half and quarter size).
  • because it is a simple silicon fabric, it’s essentially a dumb element when compared to existing approaches such as Brocade or Cisco
  • it’s big to have enough ports for a large number of edge switches
  • It needs a lot of air to keep that silicon cool – that’s part of the reason it is so large.

The EtherealMind View

I’ve approached Juniper to be on the Packet Pushers Podcast and talk about QFabric in detail – I think it would make a good show. However, their PR/marketing department appear to be scared of independent bloggers and social media and keep blowing me off. I’m still hopeful, but I’m not going to bother much longer.

I do think that the QFabric approach is different, up to a point. In the sense that they reinvented the wheel to make it better , the concept of using distributed edge switching is a old technical idea returning for another day in the sun 1. With developments in software and software development today, the problems of reliable coherence between all elements in the edge ( to allow distributed forwarding) is a practical problem, and the use of ISIS in the LAN seems have to gained acceptance in the marketplace.

So far, I think QFabric is well suited to the idea of Cloud networks because it provides greater overall flexibility, and a purchasing model that is better aligned to purchasing that the conventional Cisco/Brocade approach of buying all the performance in one big hit up front (which just so happens to be good for their bottom line and not yours).

I’ll be interested is seeing more information at it arrives.

Footnotes

  1. Networking Truth 11 – Every old idea will be proposed again with a different name and a different presentation, regardless of whether it works – http://etherealmind.com/twelve-networking-truths-rfc-9125/ ↩

Other posts in the series

  1. ◎ What's Happening Inside an Ethernet Switch ? ( Or Network Switches for Virtualization People )
  2. Tech Notes: Juniper QFabric - A Perspective on Scaling Up
  3. Switch Fabrics: Input and Output Queues and Buffers for a Switch Fabric
  4. Switch Fabrics: Fabric Arbitration and Buffers
  5. What is an Ethernet Fabric ?
  6. What is the Definition of a Switch Fabric ?
  7. Juniper QFabric - My Speculations (This post)
Filed Under: Blog, Featured Tagged With: ,
About Greg Ferro

Greg Ferro is a Network Engineer/Architect, mostly focussed on Data Centre, Security Infrastructure, and recently Virtualization. He has over 20 years in IT, in wide range of employers working as a freelance consultant including Finance, Service Providers and Online Companies. He is CCIE#6920 and has a few ideas about the world, but not enough to really count.

He is a host on the Packet Pushers Podcast, blogger at EtherealMind.com and on Twitter @etherealmind and Google Plus

  • http://bradhedlund.com Brad Hedlund

    Greg,
    I think something is missing here. In a traditional switch chassis the line cards are typically connected to the control engine via a separate management traces from the main traces used for packet data. This provides an always on and reliable control link to the linecards. Where is that in this architecture?

    I’m willing to bet that a real QFabric implementation would involve a separate out of band network from the QF-Edge to the QF-Director.

    Cheers,
    Brad

    • Ferro Greg

      I’m now told that is correct, but hopeful of some confirmation from Juniper at some point.

      • Ducati_man78

        Brad, that’s correct.  The director piece has separate connections to the QFX edge devices.  No management traffic rides the interconnects.

  • http://abnerg.tumblr.com Abner Germanow

    Hi Greg,

    Nice write up. A couple of notes:

    As you correctly point out, the interconnect is a dumb, fast, huge capacity chassis. While a switch must both process and transport traffic, the interconnect is transport only. This enables high capacity connectivity inside a server pod or across a datacenter if you choose to grow the fabric that large. For those looking carefully at the pictures, the cards in the back connect the cards in the front – which in turn connect to the QF Nodes – where the QFabric connects to anything else via Ethernet or Fiberchannel.

    Why? In a world full of distributed application components, we can’t expect to know what application bits the business will want to combine next. In a 1 gig world, Juniper can get all those app components fairly close to each other in a two tier network and traditional chassis switches can mostly handle the load up to a few thousand servers. However, as you transition to a 10G world, the ability for today’s chassis switches to both process and transport traffic maxes out around 400 to 800 10G servers (depending on which vendor’s chassis switch you use) In essence you want to make sure the core of the network has way more capacity than the edge can push into it.

    On the control plane front, Ivan has made some solid assumptions and yes there is an out-of-band network connecting the QFDirector servers. We generally leave it out of high level discussions because the application traffic doesn’t care and we have a hard enough time with people over complicating QFabric by treating the single entity as a network instead of as a switch. [1]

    Does a small out of band network complicate deployment? If you consider the number of independent switches in a large datacenter and the risk associated with getting all interactions between those switches right everyday vs the simplicity of a single switch for all production traffic and then a couple switches that don’t require much care and feeding to give the QFabric yet another layer of redundancy… Well I have yet to have a customer bat an eyelash on that front.

    Since QFabric is such a significant departure from how we have classically connected compute, data, and services in the datacenter, we spend most of our time with customers talking about how it solves business agility, migration, scaling up/down, and overall redundancy. For those customers who want all the gory details on how it works, we are glad to supply it, but lately we simply bring them into the lab to show them what it does.

    On the control plane front, we have kept a number of the details under NDA because, that’s the hardest bit of the problem. There have been a number of public reports of competitors attempting to copy the QFabric architecture, so apologies for our paranoia on that front. It’s a nice testament to the networking community that so many have landed on that topic. What I can tell you today is that every layer of a QFabric is redundant, resilient, and distributed. If, for example, that out-of-band network failed for some reason? – Say you didn’t RTFM and you plugged all your QFD servers into the same power supply – which then failed? The fabric would continue to operate.

    I think we’ll have a few QFabric peeps for Packetpushers to talk to shortly. Keep the questions coming.

    Cheers,
    Abner

    Director, Enterprise Marketing – Juniper Networks

    [1] I’ve been informed we spoke about the out of band network during a webcast titled, “Unleashing the Power of the Virtualized Data Center” that can be found here: http://www.juniper.net/us/en/dm/exponential-dc/ It’s behind a reg wall and I know how you feel about that, but there it is.

    • Ferro Greg

      Thanks for the feedback Abner.

      Sigh, registration walls for public content. Do marketing people really think that annoying suspect and prospect customers is a good idea ? (Don’t answer that).

      PS. There are often security reasons (data leakage) why these forms cannot be used and prevent some people from ever having access.

  • joel spencer

    I think you mentioned a couple of weeks ago (on the podcast) that QF/Director would not be released until sometime in 2012?  Is that still your understanding?

    • http://twitter.com/abnerg Abner Germanow

      The QF Director and Interconnect will become generally available in Q3 2011.

  • Pingback: Show 51 – Juniper QFabric

  • Ironcc77

    I remember talk about the idea that it
    is better to take L2 then L3 service from ISP because good network
    admin should have more control over his network. Is using closed,
    proprietary protocol that you have never seen before to control
    communication in your datacenter good idea?

    • http://etherealmind.com Etherealmind

      There is a big difference between WAN and LAN Ethernet. For QFabric, it might be easier to think of the connections between the QFedge devices and QFInterconnect as backplane connections. I wrote some more about this here http://etherealmind.com/juniper-qfabric-my-speculation-too/