I’ve been having thought exercises around the impact of hard disk drives that have ethernet interfaces. The basic concept of connecting a a multi-terabye HDD directly to an Ethernet switch is powerful since it provides options to those very expensive arrays in the data centre. There is obvious value in this idea.
- Switch silicon is fast, lossless and reliable.
- Ethernet fabric chips make for high speed bus backplanes inside some vendor arrays already.
- Disk drives are using I/O standards that hopelessly out of date like FibreChannel or SATA.
- magnetic disk drives are moving to low cost, low value solutions while SSDs will take over critical loads.
Possible use cases.
- disk spindle could be used for near line backup or long term storage.
- house Ethernet drives in low cost cases
- consider a Backblaze approach.
Replace the Array with A Ethernet Switch
At a basic level, a storage array is simply a network bus between the disk drives and controller motherboard with a CPU. The controller runs software that presents the disks to other software – like servers as LUNs, or data stream for Dedpulication, or blocks for tape drives.
So, connect HDD to a switch, allocate some x86 servers as a controller. Since we are using off the shelf hard are, the hardest work is mostly done. The relatively “easy” part, of course, is the software that turns this into something useful.
The controller needs to deliver a few key functions.
- stripe data across disks for redundancy,
- monitor status/integrity
- determine the fastest or best access for read/write
- value add functions like deduplications.
Comparing with Distributed Storage
Distributed storage is the current direction for low end storage using software CEPH, Lustre or GFS. A X86 server acts as an “Ethernet – SATA” bridge for the disk drives. Enhancements like SSD caching are done locally since it simplifies the software. Storage startups are only capable of limited amount in the few years they have to prove viability and producing a complex SSD caching for this type of system is possibly beyond them.
Switch Inside a x 86 Server
As Ethernet silicon becomes more available, it’s possible that we could see an Ethernet switch inside an x86 server. The Intel FM7000 has excellent networking features and Intel could conceivably provide motherboard that has a switch onboard. The server metalwork could hold a bunch of drives connected over an Ethernet bus.
Could be interesting.
From a networking point of view, this changes the network design. Most storage arrays are barnacles on the network architecture with a couple of “fat Ethernet” connections and large amounts of traffic that creates congestion points. An improved network design would have the load more equally distributed over many interfaces. Current storage technology creates “elephant” flows that cause all kinds of nasty hacks to be implemented.
Having a directly connected Ethernet switch allows for many new ways to smooth the network architecture.
The EtherealMind View
The idea of Ethernet connected hard drives seems to be a recurrent theme but the storage folks are notoriously resistant to new ideas so I don’t expect this to happen. That said, its interesting to consider this ideas and wonder how networking could be different. Thanks for following along.