In the article, “What Is the Definition of a Switch Fabric ?” on Switch Fabrics I looked at how a Crossbar switching fabric allow for concurrent circuit forwarding and how this is used to build a fabric.
In most cases, frames would be received and forwarded from an input to an output, as show in this diagram since the value of a crossbar fabric is that multiple simultaneous inputs are connected to multiple outputs.
But it’s inevitable that eventually, two frames will need to be forwarded to the same output either at the same time, or during the forwarding of a frame. In these examples of collisions, data is already being forward from input port 5:output port 2 and a new frame for input port 1:output port 2 is not possible.
In reality, this is a common event. For example, output port 2 might be a server port that has hundreds of clients sending data to it. Or an uplink to the network core from an aggregation switch with thousands of downstream desktops on access switches that connect. Or several other possible conditions in a distributed path network.
This collision can not be permitted in a fabric and must be prevented. The silicon circuits within the chip are simply not designed for it to keep them simple. The silicon fabric is controlled by a CPU-like controller. It controls a range of system functions for the chip such as the clocking, buffers and others.
We can represent this like so:
To control the input process to the chip, the current theory is that using buffers that hold a small numbers of frames will optimise the overall performance of the fabric. The buffer is an amount of memory that is synchronised and managed by the Fabric CPU which is known as the Fabric Arbitration.
The Fabric Arbitration is able to detect that a collision would occur, pause the frame in the buffer and wait for the output to clear.
The frame would then be forwarded once the Fabric Arbitration knows that the output is available.
Managing Fabric bandwidth
The importance of this management cannot be overstated. The performance of a Crossbar Switch is determined by it’s ability to forward as many frames, without collision, over the available circuits as possible. The problem with networks is that contention are inherently part of a hierarchical network design. Thus Fabric Arbritration is vitally important silicon design process to.
The path from the buffer to the Arbiter ASIC is separate from the Fabric I/O to ensure that the Arbiter ASIC is always able to control the Crossbar.
Buffer Management
The real challenge now becomes the buffer management process. After all, there is no contention within the fabric as frames do not enter unless the output is available.
Consider what happens if output buffer becomes full, or what happens when the input buffer overloads. What if the input buffer becomes full of frames for a single output ? It would drop frames even though other outputs are free and ready to receive to data. This is the topic of the next post.
Other posts in the series
- ◎ What's Happening Inside an Ethernet Switch ? ( Or Network Switches for Virtualization People )
- Tech Notes: Juniper QFabric - A Perspective on Scaling Up
- Switch Fabrics: Input and Output Queues and Buffers for a Switch Fabric
- Switch Fabrics: Fabric Arbitration and Buffers (This post)
- What is an Ethernet Fabric ?
- What is the Definition of a Switch Fabric ?
- Juniper QFabric - My Speculations
