Was reading through some documentation as noticed that the Nexus 5500 series has some unusual behaviours for Store and Forward. I made myself notes about the functional modes.
Cut-Through vs Store and Forward
In cut-through mode of operation a switch will start transmitting a frame before the frame has been completely received and this is the low latency mode of operation. Store-and-forward switches, on the other hand, cannot achieve low latency since the packet must be completely stored in switch memory or ASIC before being propagated out the port.
Store and Forward was popular in late 1990’s because it was common for Ethernet frames to be corrupted on the wire before reaching the switch due to collisions and broadcasts which would cause runts. Cable quality would also cause bit damage in frame payloads (this is quite rare today). Therefore, only forwarding valid frames was a real advance. Today, it mostly adds latency for little gain.
But not all cut-through switches are created the same when it comes to latency as the underlying switch architecture makes a big difference. For example, in the Nexus 5500 switches it will automatically use the following modes for frame handling
|Source Interface||Destination Interface||Switching Mode|
|10 GigabitEthernet||10 GigabitEthernet||Cut-Through|
|10 GigabitEthernet||1 GigabitEthernet||Cut-Through|
|1 GigabitEthernet||1 GigabitEthernet||Store-and-Forward|
|1 GigabitEthernet||10 GigabitEthernet||Store-and-Forward|
|Fibre Channel||Fibre Channel||Store-and-Forward|
I’ve highlighted the one line which is surprising. Store and Forward is almost mandatory for FibreChannel standards compliance to maintain low error rates. Equally, you can’t transmit a frame from a low speed interface to a high speed interface until you have received the entire frame.
The reverse is not true however since Cut Through for 10G to 1G is practical as the frame would be received on 10G port before transmission can start on the 1G.
Rules of Thumb.
Faster to Slower is Cut Through – No buffering needed to transmit the frame.
Slower to Faster is Store and Forward because the frame must transmit in a single elements, you can’t start transmitting the frame at 10Gbps and then pause while you wait for the rest to be received at 1Gbps.
FibreChannel to anything is Store And Forward. probably for error checking as part of ANSI requirements
The surprising feature here is the use of store and forward for 1 gigabit connections. I’m not sure why this is true.