iSCSI Network Designs: Part 1 — Introduction and Basics

I am work­ing on a Data Centre that has more than 500 serv­ers and we are con­sid­er­ing what our stor­age strategy should be. This is why I have been look­ing at FCoE and iSCSI and estab­lish­ing the bene­fits (or not).

My next few posts will be a “stream of con­scious­ness” on pos­sible designs for an iSCSI infrastructure.

Administrative Concerns

In my mind, I have con­cerns that many net­work­ing people do not take enough care when con­fig­ur­ing their switches. A short inter­rup­tion in net­work access is often not noticed, but for an iSCSI con­nec­tion an out­age of more than a few seconds will cause prob­lems. For cer­tain types of soft­ware (data­bases) any inter­rup­tion will cause cor­rup­tion and server outages.

Servers Intensity

In simple terms, I clas­sify serv­ers as low intens­ity and high intensity.

Low intens­ity would refer to serv­ers that :

1) use lim­ited resources (CPU, Memory, Disk) and thus require a low level of performance

2) are not critical.

Typical serv­ers in this cat­egory are email, fax gate­ways, WSUS, net­work man­age­ment systems.

High intens­ity would refer to serv­ers that are high resource and /​ or busi­ness critical.

1) require CPU /​ Memory /​ Disk in large volumes, or to be able to spike to large volumes to meet per­form­ance (either real or perceived).

2) are mis­sion crit­ical and would severely impact the busi­ness if an out­age occurred.

3) if the OS or Server plat­form suf­fers a fail­ure, then the res­tor­a­tion pro­cess would require rebuild­ing e.g. data­base applic­a­tion where a cor­rup­tion of the data­base would require hours to rebuild.

iSCSI ini­ti­ator performance

When imple­ment­ing iSCSI in your server oper­at­ing sys­tem you have two choices, soft­ware or hard­ware (using Host Block Adapter).

A soft­ware ini­ti­ator is identical to what hap­pens in your server today. The TCP/​IP packet is gen­er­ated by the oper­at­ing sys­tem using your core CPU. This includes some quite intens­ive oper­a­tions such gen­er­at­ing the CRC for the TCP Packet, sequence cal­cu­la­tions and track­ing, ses­sion ini­ti­ation and tear­down, slid­ing win­dow cal­cu­la­tions and so on. The oper­at­ing sys­tem ker­nel then pushes the pre­pared packet to the NIC for transmission.

A Host Bus Adapter is a PCI card that per­forms the TCP pro­cessing. This is known as TCP Offload and the cards are gen­er­ally known as TCP Offload Engines or TOE.

TOE basic­ally means that there a ded­ic­ated chip on the net­work card that per­forms the TCP func­tions. TOE cards are con­cep­tu­ally sim­ilar to GPU’s for graph­ics pro­cessing. And just like GPU cards, the qual­ity of the TOE cards var­ies accord­ing to the man­u­fac­turer and price. TOE Network Drivers are also import­ant (in the same way the Nvidia drivers for graph­ics are important.

I would sug­gest that you can use iSCSI soft­ware ini­ti­at­ors for low intens­ity serv­ers and the per­form­ance will be accept­able. You should cer­tainly con­sider iSCSI HBA for high intens­ity server.

While prices for HBA are rel­at­ively expens­ive, they offer sig­ni­fic­ant per­form­ance gains. Take time to ensure that you pur­chase a qual­ity product as the soft­ware drivers will be vital to your success.

iSCSI Security

FCoE has made a lot of noise about their encryp­tion fea­ture. If thats import­ant to you then you should remem­ber that iSCSI has had a IPSec in the RFC since April 2004.

It seems to be agreed that prac­tical imple­ment­a­tions of IPSec have not been pos­sible because of per­form­ance con­cerns. However, I sus­pect that HBA will soon (if they don’t already) include sil­icon to per­form the IPSec pro­cessing. Conceptually I can envis­age stand­ard IPsec encryp­tion sil­icon from the major chip fabs to appear on iSCSI HBAs without too much problem.(3)

I like the idea of IPSec, I already know about it and can use my ‘off the shelf’ know­ledge to set it up.

Network Requirements

In my read­ing, many art­icles point out that Storage Area Networks have a require­ment for low latency and low jitter.

So my first method of address­ing this is to use the fol­low­ing design of a single layer Ethernet switch to con­nect serv­ers and iSCSI storage.

There a a num­ber of fea­tures about this design that I like.

Conclusion

In some ways, this looks like a simple Fibrechannel design, without the over­head of a Fibrechannel switch sys­tem. The max­imum dens­ity of the sys­tem is determ­ined by the num­ber of ports on the switch, and they must all be phys­ic­ally loc­ated near the switch.

Another lim­it­a­tion is that band­width is lim­ited to 1Gb/​s (unless you are pur­chas­ing a 10GB Ethernet switch and NIC’s, but there are not many 10GB HBA around).

So the next step will be to grow this design to encom­pass a lar­ger num­ber of serv­ers, and to increase the bandwidth.

Feedback

As I am learn­ing from text­books and vari­ous Internet sources, please feel free to con­tact me or com­ment. I would value some input. myether­e­al­mind (at} gmail​.com.

Footnotes

(1) Stream of con­scious­ness: phrase used by William James in 1890 to describe the unbroken flow of thought and aware­ness of the wak­ing mind

(2) Note that Fibrechannel has exactly the same issues as FC uses the same con­cepts to improve performance.

(3) Let me know if these already exist and I will update.

Please rate this post:

(1 votes, average: 5.00 out of 10)

 Loading ...