[gentoo-user] RAID/LVM machine - install questions

[gentoo-user] RAID/LVM machine - install questions

[Mark Knecht]

Hi,
   I'm starting to put together a server-type machine with multiple purposes:

1) MythTV server
2) General backups for another fast machine I'm going to build
3) Wife's new desktop

   Since all of these requirements are pretty modest but I want high
reliability I'd like to do software RAID with (I think) LVM on top of
that. I've not decided yet exactly what RAID type to use, and I know
nothing about LVM, so I'm reading here to learn:

http://en.gentoo-wiki.com/wiki/LVM
http://www.gentoo.org/doc/en/lvm2.xml

   The LVM Install doc is pretty clear about not putting these in LVM:

/etc, /lib, /mnt, /proc, /sbin, /dev, and /root

which seems sensible. From an install point of view I'm wondering
about RAID and how I should treat /, /boot and swap? As I'm planning
on software RAID it seems that maybe those part of the file system
should not even be part of RAID. Is that sensible? I always want /
available to mount the directories above. /boot on RAID means (I
guess) that I'd need RAID in the kernel instead of modular, and why do
I need swap on RAID?

   The most important task of this machine is to keep data safe. If
the machine has a hardware problem and is down for a day or two that's
not going to be a big problem as long as the Myth recordings, my
wife's home directory and the backups I'm doing and placing on the
machine are relatively safe. the processor is going to be WAY overkill
with 8 cores but it hardly makes financial sense to go smaller as
maybe I can even run some computer cycles on it, like building binary
packages for my other machines or something like that.

   Interested in whatever thoughts folks have about the best way to
set this up. If I haven't outlined the requirements well enough let me
know. I'm at least a week from starting to build the hardware so
there's no rush on this but maybe comments will effect motherboard
selection, etc.

Thanks,
Mark

Re: [gentoo-user] RAID/LVM machine - install questions

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 1928 bytes --]

Am 19.03.2010 23:40, schrieb Mark Knecht:
[...]
> 
>    The LVM Install doc is pretty clear about not putting these in LVM:
> 
> /etc, /lib, /mnt, /proc, /sbin, /dev, and /root
> 

/boot shouldn't be there, either. Not sure about /bin

> which seems sensible. From an install point of view I'm wondering
> about RAID and how I should treat /, /boot and swap? As I'm planning
> on software RAID it seems that maybe those part of the file system
> should not even be part of RAID. Is that sensible? I always want /
> available to mount the directories above. /boot on RAID means (I
> guess) that I'd need RAID in the kernel instead of modular, and why do
> I need swap on RAID?
>[...] 

If you use kernel based software RAID (mdadm, not dmraid), you can put
everything except of /boot on RAID. Even for /boot, there are
workarounds. I think there was a thread about it very recently right on
this list.

If you don't want to use an initrd (and believe me, you don't), you
cannot build the RAID components as modules, of course. But why would
you want? You need it anyway all the time between bootup and shutdown.

You don't need to put swap on a RAID. Swap has its own system for
implementing RAID-1 or RAID-0-like functionality. Using a RAID-1 for it
prevents the machine from crashing if the disk on which swap resides
dies. RAID-0 would be faster, of course.

I personally find it easier to put swap on LVM in order to make
management easier. However, if you want to use suspend-to-disk (a.k.a.
hibernate), you would need an initrd, again.

Alternatively, you can also use LVM for mirroring (RAID-1) or striping
(RAID-0) single volumes. I think this only makes sense if you just want
to protect some single volumes. After using it for some time, I found it
not worth the effort. With current disk prices, just mirror everything
and live easy ;-)

Hope this helps,
Florian Philipp


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] RAID/LVM machine - install questions

[KH]

Mark Knecht schrieb:
> Hi,
[...]
> 3) Wife's new desktop
[...]
> I want high reliability
[...]
> The most important task of this machine is to keep data safe.
[...]
> 
> Thanks,
> Mark
> 

Hi Mark,

For me it sounds like those points just don't fit together ;-)

Regards
kh

Re: [gentoo-user] RAID/LVM machine - install questions

[Mark Knecht]

On Sat, Mar 20, 2010 at 9:38 AM, KH <gentoo-user@konstantinhansen.de> wrote:
> Mark Knecht schrieb:
>>
>> Hi,
>
> [...]
>>
>> 3) Wife's new desktop
>
> [...]
>>
>> I want high reliability
>
> [...]
>>
>> The most important task of this machine is to keep data safe.
>
> [...]
>>
>> Thanks,
>> Mark
>>
>
> Hi Mark,
>
> For me it sounds like those points just don't fit together ;-)
>
> Regards
> kh
>
>
<Smiling broadly...> :-) Yeah.. Well, keeping my wife's data safe
keeps me happy. :-)

So the chassis and drives for this 1st machine are on order. 6 1TB
green drives. Now I just need to decide what sort of RAID to use. I
don't need much speed writing so I'm thinking maybe a 3 drive RAID1
setup with a hot spare managed using mdadm and then LVM on top of it.

The main backup data will be coming from another machine I'm building
that runs a new i7 980x 12 core processor with 24GB of DRAM. That
machine will run 5 copies of VirtualBox/Windows 7 using 2 cores + 4GB
DRAM for each, leaving 2 cores and 4GB for Gentoo as the host OS. Each
Windows instance crunches numbers 24/7 and needs to be backed up once
a day with each backup being about 20GB. I'll move approximately 100GB
across the network each night to the 1st machine at least once a day,
possibly more. This data needs to be very safe so once every week it
goes offsite also.

Other than that the 1st machine will also be a MythTV backend server.
That's a pretty light load hardware wise but uses a little computing
muscle to do commercial detection and the like.

And then yes, my wife can use it as her main desktop as most of the
above work takes place overnight with Myth being in the evening and
backups of machine #1 occurring at 4AM, etc.

- Mark

Re: [gentoo-user] RAID/LVM machine - install questions

[Mark Knecht]

On Sat, Mar 20, 2010 at 6:22 AM, Florian Philipp
<lists@f_philipp.fastmail.net> wrote:
> Am 19.03.2010 23:40, schrieb Mark Knecht:
> [...]
>>
>>    The LVM Install doc is pretty clear about not putting these in LVM:
>>
>> /etc, /lib, /mnt, /proc, /sbin, /dev, and /root
>>
>
> /boot shouldn't be there, either. Not sure about /bin
>
>> which seems sensible. From an install point of view I'm wondering
>> about RAID and how I should treat /, /boot and swap? As I'm planning
>> on software RAID it seems that maybe those part of the file system
>> should not even be part of RAID. Is that sensible? I always want /
>> available to mount the directories above. /boot on RAID means (I
>> guess) that I'd need RAID in the kernel instead of modular, and why do
>> I need swap on RAID?
>>[...]
>
> If you use kernel based software RAID (mdadm, not dmraid), you can put
> everything except of /boot on RAID. Even for /boot, there are
> workarounds. I think there was a thread about it very recently right on
> this list.

I'm thinking I'll keep it as simple as possibly and just spread out
the Gentoo install over the multiple hard drives without using RAID,
but maybe not. It would be nice to have everything on RAID but I don't
know if I should byte that off for my first taste of building RAID.


>
> If you don't want to use an initrd (and believe me, you don't), you
> cannot build the RAID components as modules, of course. But why would
> you want? You need it anyway all the time between bootup and shutdown.

No initrd. I've never used it in 10 years of running Linux and I
wouldn't know how to start or even why I would use it. I suppose if I
had hardware RAID then maybe I'd need to but that's not my plan.

>
> You don't need to put swap on a RAID. Swap has its own system for
> implementing RAID-1 or RAID-0-like functionality. Using a RAID-1 for it
> prevents the machine from crashing if the disk on which swap resides
> dies. RAID-0 would be faster, of course.
>
> I personally find it easier to put swap on LVM in order to make
> management easier. However, if you want to use suspend-to-disk (a.k.a.
> hibernate), you would need an initrd, again.
>
> Alternatively, you can also use LVM for mirroring (RAID-1) or striping
> (RAID-0) single volumes. I think this only makes sense if you just want
> to protect some single volumes. After using it for some time, I found it
> not worth the effort. With current disk prices, just mirror everything
> and live easy ;-)
>
> Hope this helps,
> Florian Philipp
>
>

Very helpful. Thanks!

Cheers,
Mark

Re: [gentoo-user] RAID/LVM machine - install questions

[KH]

Am 20.03.2010 19:26, schrieb Mark Knecht:
[...]
> So the chassis and drives for this 1st machine are on order. 6 1TB
> green drives. []
> - Mark
>

Hi Mark,

What do you mean by "green drives"? I had been told - but never searched 
for confirmation - that those energy saving drives change spinning and 
also do spin down. The problem would be that the drives than might drop 
out of the raid since they are not reachable fast.

Don't know if that is true. I bought me some black label drives for the 
longer warranty.

kh

Re: [gentoo-user] RAID/LVM machine - install questions

[KH]

Am 20.03.2010 19:29, schrieb Mark Knecht:
[...]
>
> I'm thinking I'll keep it as simple as possibly and just spread out
> the Gentoo install over the multiple hard drives without using RAID,
> but maybe not. It would be nice to have everything on RAID but I don't
> know if I should byte that off for my first taste of building RAID.
>
> [...]
>
> Very helpful. Thanks!
>
> Cheers,
> Mark
>

Hi,

I have boot on raid1 and everything else on raid5. Also swap is raid5. 
It wasn't hard to do that.
If I did it again, I would also create a small (5GB) raid5 for testing 
stuff. Like when I try to reassemble or change something. Copy some 
movies and music to that drive. Whenever you need to change something 
with your real raid, do it with the test one first and see if you can 
still listen to your music.

Regards
kh

Re: [gentoo-user] RAID/LVM machine - install questions

[Florian Philipp]

[-- Attachment #1: Type: text/plain, Size: 661 bytes --]

Am 20.03.2010 19:26, schrieb Mark Knecht:
> On Sat, Mar 20, 2010 at 9:38 AM, KH <gentoo-user@konstantinhansen.de> wrote:
>> Mark Knecht schrieb:
>>>
>>
> <Smiling broadly...> :-) Yeah.. Well, keeping my wife's data safe
> keeps me happy. :-)
> 
> So the chassis and drives for this 1st machine are on order. 6 1TB
> green drives. Now I just need to decide what sort of RAID to use. I
> don't need much speed writing so I'm thinking maybe a 3 drive RAID1
> setup with a hot spare managed using mdadm and then LVM on top of it.
> 

With 4 drives, you could build a RAID-6, too. It's like a RAID-5 but
protects against failure of any two drives.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 262 bytes --]

Re: [gentoo-user] RAID/LVM machine - install questions

[Paul Hartman]

On Sun, Mar 21, 2010 at 7:12 AM, KH <gentoo-user@konstantinhansen.de> wrote:
> Am 20.03.2010 19:26, schrieb Mark Knecht:
> [...]
>>
>> So the chassis and drives for this 1st machine are on order. 6 1TB
>> green drives. []
>> - Mark
>>
>
> Hi Mark,
>
> What do you mean by "green drives"? I had been told - but never searched for
> confirmation - that those energy saving drives change spinning and also do
> spin down. The problem would be that the drives than might drop out of the
> raid since they are not reachable fast.
>
> Don't know if that is true. I bought me some black label drives for the
> longer warranty.

If it is a WD drive, google "TLER" for info about possible problems in RAID use.

Re: [gentoo-user] RAID/LVM machine - install questions

[Mark Knecht]

On Mon, Mar 22, 2010 at 8:51 AM, Paul Hartman
<paul.hartman+gentoo@gmail.com> wrote:
> On Sun, Mar 21, 2010 at 7:12 AM, KH <gentoo-user@konstantinhansen.de> wrote:
>> Am 20.03.2010 19:26, schrieb Mark Knecht:
>> [...]
>>>
>>> So the chassis and drives for this 1st machine are on order. 6 1TB
>>> green drives. []
>>> - Mark
>>>
>>
>> Hi Mark,
>>
>> What do you mean by "green drives"? I had been told - but never searched for
>> confirmation - that those energy saving drives change spinning and also do
>> spin down. The problem would be that the drives than might drop out of the
>> raid since they are not reachable fast.
>>
>> Don't know if that is true. I bought me some black label drives for the
>> longer warranty.
>
> If it is a WD drive, google "TLER" for info about possible problems in RAID use.
>
>
Yeah, those issues do get discussed at times on the Linux RAID list.
I've asked questions about it and been told that Linux software RAID
depends totally on what the driver tells it and nothing seems to be
don (as best I can tell) based on any fixed time. That's more of a
hardware controller issue. I was told that if the drive by itself
doesn't fail at the system level when it's spinning up, then it won't
fail at the RAID level either. However what it does if it has a
hardware error is a bit beyond me at this point. My intention is to
try and get better with smartd so that the drive is continually
monitored and see if I can get ahead of a failure with that.