Re: [gentoo-user] Re: Strategy for using SAN/NAS for storage with Gentoo...

[Stroller <stroller@stellar.eclipse.co.uk>]

On 15 Mar 2010, at 16:26, Steve wrote:
> ...
> From ages ago, I remember iSCSI being bandied about.  Did that ever go
> anywhere (i.e. is this easy to do from Gentoo?)

I believe it is quite widely used - it is mentioned often on the linux- 
poweredge list. I would imagine the Linux kernel allows mounting and  
sharing by iSCSI - check `make menuconfig` and type "/iscsi".

It's hard to be more specific without knowing your usage.

For storage of a "mere terabyte" you can buy a networked storage  
enclosure which will accommodate two drives. These are cheap, do  
mirroring, will accommodate standard 1TB, 1.5TB, 2TB drives, but are  
probably not too fast.

One reads a lot posted by people who have large movie collections  
stored on the network, whether they be MythTV users or the mutineer  
sailors of 17th century galleons. A PC-based solution gives you more  
room for this - you can fit perhaps 4 drives in a standard PC case you  
find at the tip, or you can get 12 or 16 drives in a dedicated  
rackmount server case. This allows capacity of upto 32TB with current  
drives, if you can afford that, or to use cheaper drives (1TB or 1.5TB  
are best gigabytes-per-dollar at present, I think; 500gb drives seem  
recently to have become disproportionately expensive) and have better  
RAID levels.

The Norco one is popular amongst enthusiasts, because it's really  
cheap [1]; it uses 2 x standard ATX power supplies, one for the  
mainboard, one for the drives. You can get similar cases with the  
option of hot-swap PSUs - Chenbro used to be the main brand for this,  
I think, but in the last couple of years TST <http://TSTcom.com> have  
started producing nicer cases; I use a TST ESR-316, which is utterly  
lush, but which was expensive. I have one slight reservation about the  
TST, which I will not spend time detailing unless you ask.

I use only half the TST's capacity at present, but it is a pleasure  
and a relief to have so much room available; expansion of network  
drive capacity is never a problem - just slap a drive in and you're  
ready to go. Even with as many as 6 or 8 drive bays there are corner  
cases which can make expansion a bit of a headache (at least if uptime  
is important).

Since these cases accommodate standard ATX motherboards, you get to  
use an old Pentium 4 motherboard salvaged from an old PC or an Atom- 
based motherboard for £100 or so. The latter price is a bit shocking,  
IMO, compared to (say) the Asus EE-PC, but it reflects the demand for  
them; they're prolly only $100 in the US. These atom motherboards have  
minimal expansion slots, but if you only want to use it for storage  
then you're probably fine with just one.

If you build your own server you can use software or hardware RAID.  
Fast hardware RAID, based on an PCIe controller card, is expensive.  
You can get PCI or PCI-X hardware RAID very cheaply on eBay these  
days, but it's slow. That is to say that PCI or PCI-X hardware RAID is  
fast enough to stream a couple of movies at the same time, fast enough  
to copy 5gb files only a couple of minutes, but production server  
systems (if you were buying a database server for work) would be  
expected to use a PCIe-based hard-drive controller. Hardware RAID is  
nice in its ability to hot-swap out a failed hard-drive without  
interruption. I have not found non-RAID SATA controllers that satisfy  
me with their ability to do hot-swap (although I would love to).

Managing RAID on a PC-based server - rather than a dedicated NAS  
enclosure - very easily allows expansion. With RAID5 or 6 you can just  
add in another drive and expand on to it. I use an old PCI-X (fits in  
a PCI slot) 3ware 9500 card, and it *seems* like if you have a RAID1  
(haven't tried RAID5) on two drives of capacity X, then remove each of  
those drives in turn, rebuilding onto drives of X+Y capacity, then  
upon completion the array appears to the o/s as the larger X+Y size. I  
think some LSI cards do this, also. I would not bet on the ability of  
low-end NAS boxes to do this.

A company called Drobo makes some high-end NAS hardware with space for  
plenty of drives (on some models) and some fancy features. I find UK  
prices a bit shocking, but depending upon your application they might  
be justified; the US prices seem quite reasonable to me.

I wouldn't get too het up about Samba / CIFS vs NFS. Samba / CIFS can  
be faster than NFS, even in an all-Linux environment. Other times it's  
not. This seems pretty much random, depending upon whom is doing the  
benchmarking. On an intellectual level, at least, I find neither  
wholly satisfying - it would be really nice to have a Linux-native  
network filesystem that does authentication / permissions properly.  
But both do work.

I looked at ZFS, but decided that Solaris, from a look at the HCL, was  
too picky over hardware. I think ZFS is great, I no longer think it's  
the future. My selection of cheap hardware is far wider under Linux, I  
can install Gentoo and just `emerge mediatomb` and stream movies to my  
PS3.

So there ya go. Lots of options, budget from dead cheap to mega money.  
Depends how much you can justify.

Stroller.



[1] http://www.newegg.com/product/product.aspx?item=n82e16811219021