Recently I noticed that Cisco is selling “Fabric Ethernet Transceivers” for the Nexus switch family. Some research shows that these are replacements for 10GBaseSX SFP modules. they come in bundles of product, cannot be purchased separately and I can’t find any detail on how FET works at a technical level. I’m guessing that it uses a proprietary PHY layer that strips out unused or pointless Ethernet requirements 1.
Rules for use
Some sketchy details here say the following:
You can’t use FET’s for anything but connections to Cisco NX7K, NX5K, NX2K
- Supported for fabric links only (Cisco Nexus 2000 Series to Cisco parent switch)
- Cisco Fabric Extender Transceiver must be connected to another Cisco Fabric Extender Transceiver
- Supported on Cisco Nexus 2200 platform uplinks
- Supported on Cisco Nexus 5000 and Nexus 7000 Series Switch fabric links
You can find a description on the Cisco FET-10G. Here is the relevant bit.
The Cisco FET-10G Fabric Extender Transceiver support link lengths up to 100m on laser-optimized OM3 or OM4 multimode fiber. It is supported on fabric links only from a Nexus 2000 to a Cisco parent switch. __Note this product is not orderable individually.__
From the Nexus 2000 Hardware Guide
| Transceiver | Cable Type | Connector Type | Wavelength (nm) | Core Size (microns) | Modal Bandwidth (MHz-km) | Maximum Cable Distance |
|---|---|---|---|---|---|---|
| N2200 FET | MMF | Dual LC/PC | 850 | 50 | 500 | 82 feet (25 meters) |
| 50 | 2000 | 328 feet (100 meters) | ||||
| SFP-10G-SR | MMF | Dual LC/PC | 850 | 62.5 | 160 | 85 feet (26 meters) |
| 62.5 | 200 | 108 feet (33 meters) | ||||
| 50.0 | 400 | 216 feet(66 meters) | ||||
| 50.0 | 500 | 269 feet (82 meters) | ||||
| 50.0 | 2000 | 984 feet (984 meters) | ||||
| SFP-10G-LR | SMF | Dual LC/PC | 1310 | G.652 fiber | — | 6.2 miles (10 km) |
As you can see, the FET doesn’t support the older fibre optic cabling types such as 62.5 micron fibre, or the low modal bandwidth cabling types. In effect, this means you need OM2 or OM3 type multimode fibre cabling and you might to upgrade your fibre to use them. In my view, it’s cheaper to install new cable than to buy the 10GBaseSR SFPs (more on the pricing comparison below).
Pricing – They are cheap
So, here is the good news. They are cheap. The bad news ? You can’t buy them as a standalone product.
How cheap ? Consider the following US List Price (for September 2011)
| Part Number | Description | US$ List Price |
|---|---|---|
| N5548PM-6N2248TF | Nexus 5548P/Expansion Module/6xN2248TP/48xFET | $82,000.00 |
| N5548PM-6N2248TP | Nexus 5548P/Expansion Module/6xN2248TP | $74,500.00 |
| N5548PM-6N2248TR | Nexus 5548P/Expansion Module/6xN2248TP/30xSFP-10G-SR | $108,000.00 |
| N5020P-N2248TP-BE | Nexus 5020/6x2248TP/30xSR Bundle | $105,000.00 |
| N5010P-4N2248TF-B | Nexus 5010P/4x2248TP/32xFET Bundle | $53,000.00 |
| N5020P-4N2232PF-B | Nexus 5020P/4x2232PP/64xFET Bundle | $85,500.00 |
| N5020P-6N2248TF-B | Nexus 5020P/6x2248TP/48xFET Bundle | $79,500.00 |
| N5K-C5548UP-FA | Nexus 5548 UP Chassis, 32 10GbE Ports, 2 PS, 2 Fans | $25,600.00 |
So if you were using standards-based 10GbaseSR with the ‘N5020P-N2248TP-BE Nexus 5020/6x2248TP/30xSR Bundle’ with 30 SFPs included, it would cost $108000. Compare that with the ‘N5548PM-6N2248TF Nexus 5548P/Expansion Module/6xN2248TP/48xFET’ which has 48 FETs and costs USD$82000. That’s a USD$26000 saving.
If you think having proprietary FETs is concern, then consider that you are getting 48 FETS, using as two lots of 24 for each side of the north/south connection ie. 24 FETs on the NX5K and 24 FETs for NX2K which means 4 FETs per NX2K. Thats the maximum number of FEX uplinks ready to go on day one.
Nexus 2000 FET Bundles
Lets say that you needs some more Nexus 2000 units with FET, there an answer for that.
| Part Number | Description | US$ List Price |
|---|---|---|
| N2K-C2248TF-1GE | Nexus 2248TP with 8 FET | $11,000.00 |
| N2K-C2232PF-10GE | Nexus 2232PP with 16 FET | $14,000.00 |
| N2K-C2224TF-1GE | Nexus 2224TP with 4 FET | $8,000.00 |
Compare that with the pricing for bare NX2Ks
| Part Number | Description | US$ List Price |
|---|---|---|
| N2K-C2224TP-1GE | N2K GE, 2PS, 1 Fan Module, 24×100/1000-T+2x10GE (req SFP+) | $7,000.00 |
| N2K-C2248TP-1GE | N2K GE, 2PS, 1 Fan Module, 48×100/1000-T+4x10GE (req SFP+) | $9,000.00 |
| N2K-C2232PP-10GE | N2K 10GE, 2PS, 1 Fan Module, 32×1/10GE+8x10GE (req SFP+) | $10,000.00 |
| SFP-10G-SR= | 10GBASE-SR SFP Module | $1,495.00 |
Cheaper to buy new cables
By extension, the FETs cost about USD$250 each (at list price, apply your discount). All of a sudden, 10 gigabit Ethernet is actually possible instead of bloody expensive. With a USD$1000 difference _PER SFP, that’s USD$2000 per cable run just to use 10GBaseSR SFPs.
Lets say you get a 40% discount on your Cisco kit, that’s still USD$1200 for every cable run between switches to use 10GbaseSR SFP modules.
WIN¡
Power Consumption
Power : Approximately 1W per transceiver. Comparable or less that 10GbaseSR.
The EtherealMind View
I’m not comfortable with the lack of technical detail to tell or advise me what I’m not getting but if the transceivers are simply optimised to work with 50 micron fibre thus reducing the manufacturing cost of the launch laser then that concept works for me. Some feedback from Cisco would be appreciated.
The price of 10GbaseSR SFPs is hideously expensive, and for any good sized 10Gb network makes up large dollar value of the network kit and thus makes this a good option for consideration. The IEEE 802.3 committee does try too hard to be backwards compatible and this is probably an example of that. The cost of the those 10GBaseSR SFPs is way overpriced. I’ve proven that for Cisco products, it’s currently cheaper to replace your fibre and use the FETs than to buy 10GBaseSR SFPs. On this basis, it’s probably time for the IEEE to reconsider their ‘Microsoft method of support all twenty year old standards’ way of working and look to cut the backwards compatibility if it’s going to cost this much to implement.
On balance, I’m cautiously positive about FETs. 10Gb networking is important for the networking industry to move forward, and the current SFP pricing is holding up customer adoption. Cisco has big enough volumes to make custom hardware possible, and should act as a kick in the pants to IEEE to do the smart thing more often. When the price of recabling is cheaper than the SFPs, the backward compatibility should be discarded.
PS. You should not be using copper for 10GBaseT in your data centre. That’s just silly. Always use 10GbaseSR. Always. Copper has no future in your data centre.
Update 20111122: John points out in the comments that these are actually Ultra Short Range SFP+ optics. So I did some research and summarised it
What Are 10 Gigabit Ultra Short Reach (USR) Optics ?
- The Ethernet standard has built up a lot of cruft over the years from what I can understand. I’m assuming that some part of the interoperability requirements ↩
Pingback: What are 10 Gigabit Ultra Short Reach (USR) Optics ? — My Etherealmind