[gentoo-amd64] Upgrading to Raid

[gentoo-amd64] Upgrading to Raid

[scotthathcock]

I currently have an ASUS K8V Deluxe system with an 80GB SATA drive. It
used to have a old 10GB PATA disk for M$ dual boot. That disk died and
was interfering with boot, so it's in the trash. 

The dead disk reminded me of the pain I will suffer if I loose my linux
system. Even with backups of my home dir, which I'm not as good about as
I should be, a full fresh install would be a pain. Thus, I am
considering Raid.

Does anyone have experience with RAID on this Board? 
Is the Via or Promise controller better for this? 
Is there a good Howto on migrating from a non Raid disk to Raid? I
recall seeing one but can't find it now.
Am I wasting time and money? Should I just do a better job of backup? Is
there a resource on what to backup to allow a fast regeneration (or
duplication) of an existing gentoo system?

Finally, thanks for the great distribution. I use a commercial distro at
work and I am much happier with Gentoo. :)

Scott
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Nuitari]

> Does anyone have experience with RAID on this Board?
> Is the Via or Promise controller better for this?
> Is there a good Howto on migrating from a non Raid disk to Raid? I
> recall seeing one but can't find it now.
> Am I wasting time and money? Should I just do a better job of backup? Is
> there a resource on what to backup to allow a fast regeneration (or
> duplication) of an existing gentoo system?

You could also look into Linux Software RAID which I found to be so much 
better then motherboard dependant hardware raid as both are CPU bound, but 
there is no hardware costs for the Linux one...

Also if your motherboard changes the Linux one is much easier to move 
around.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Chris S]

> You could also look into Linux Software RAID which I found to be so 
> much better then motherboard dependant hardware raid as both are CPU 
> bound, but there is no hardware costs for the Linux one...
>
> Also if your motherboard changes the Linux one is much easier to move 
> around.

I agree, Linux software raid is the only way to go unless you wanna fork 
out for a real raid controller like Areca or 3Ware.
and you'd prob go raid 1 (mirror) with 2 harddrives though I would get 3 
drives and go raid 5 ;)
-c
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Chris S]

Perhaps I should endeavour to answer a few other questions too..

>Does anyone have experience with RAID on this Board? 
>  
>
No, however it's the same as other on-board raid controllers, they are 
sudo raid, in that they are Windows software based driver driven.
There was some support for them under 2.4, but not very well and 
certainly not for SATA - plus you want to use libata with built in sata 
support.

>Is the Via or Promise controller better for this? 
>  
>
well you won't use the built in raid function so it doesn't matter. If 
you're talking which has better SATA support, I'd go for Promise.

>Is there a good Howto on migrating from a non Raid disk to Raid? I recall seeing one but can't find it now.
>  
>
well if you're using linux software raid (lsr) then you can just tarball 
your whole system, build your raid and extract it, re-do your bootloader 
and you're done.
See forums.gentoo.org there are lots of posts re raid.

>Am I wasting time and money? Should I just do a better job of backup? 
>
depends - can you afford downtime. if the answer is no, then go raid 1 
or higher.
if the answer is yes, then perhaps save your money, invest in a hdd and 
usb 2 / firewire drive and backup.
use rsync for incremental backups. u can also backup other pcs over the 
network.
this also gives you the ability to recover deleted files and versions, 
which raid does no (unless you setup your raid with CVS or subversion)

>Is
>there a resource on what to backup to allow a fast regeneration (or
>duplication) of an existing gentoo system?
>  
>
it's linux! so just boot to a livecd, tarball your entire drives and 
mounts, then extract to your new hdd setup.

-c

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Billy Holmes]

Chris S wrote:
> if the answer is yes, then perhaps save your money, invest in a hdd and 
> usb 2 / firewire drive and backup.

in my experience usb and firewire drives are not good backup devices. A 
good backup device is one that does not go tits up when you need to 
retreive the data.

usb and firewire enclosed drives get banged around a lot - even if 
you're careful.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Kyle Liddell]

On Wed, 2005-08-31 at 21:46 -0600, scotthathcock@comcast.net wrote:
> Is there a good Howto on migrating from a non Raid disk to Raid? I
> recall seeing one but can't find it now.

If you're really lazy (like me), you can convert your current running
system over to RAID without messing with reinstalling from backups
(although you probably should make the backups, hopefully you won't have
to use them).  Assuming you're doing RAID1, get your partitions and such
set up on your new drive.  Then set up the raid array (the howto on
tldp.org is helpful, although be sure to use the newer mdadm setup even
though the docs are a little thinner in the howto) on your new drive,
marking the other drive in the array as bad/down.  Copy say, /dev/hdxy
to /dev/mdx, and then edit the proper config files so that when you
reboot you'll be using the raid array.  Once you're in it, change the
other drive in the array from bad to the old drive, and it'll rebuild
the array on the fly.  It's pretty neat.

(If you do that method though, read the docs very carefully to make sure
you're doing it right.  It's been a while since I've done it so I may
have gotten something backwards or some such.)

I wouldn't worry about any performance hits:  A few months ago I
converted an old P2 450mhz system to RAID 1+0 and noticed no performance
hits, and even went to raid1 + lvm and again no problems.  I've since
added an additional CPU to that box, but hard drives are just so much
slower than your CPU that it'd take a lot of work to create a slowdown.

Also, if there's any chance you'll want to maybe resize your partitions,
or especially, add more drives to the system, I would very strongly
suggest looking into LVM (Linux Volume Manager).  It plays well with
raid (instead of raid1+0 I'm doing the equivalent with raid1+lvm, but no
worries about adding more disks) and lets you alter "partitions" on the
fly.  If you want to add more storage, just buy a pair of drives, raid1
them, then just append it to the end of your volume, and run your
favorite on the fly partition resizer and enjoy.

By the way, a $15 promise ATA100 two channel IDE PCI card + a pair of
$80 120gb seagate HDDs is quite a nice deal.  I've got two of the
Promise IDE cards in my server, and they seem to be pretty good PATA
cards if you're into that sort of thing.

Kyle


-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Francisco Perez]

Scott,
I have a 3Ware Escalade 4 chaneel hardware raid controller running raid 
10.  Here are my thoughts:  The card itself was $175.00 and the 4 250GB 
brives were $150 bucks a piece, so the cost is considerable.  If you are 
only going to have two disks mirrored, you can get the two channel card 
and two drives and mirror them which is obviously less money.  My system 
is a Tyan Transport GX-28 with 2 Opteron 246's and originally I had the 
drives on the onboard controller.  First, there is a very noticable 
increase in performance going from the integrated (software) raid 
controller to the hardware raid controller.  For my percieved needs it 
was definitely worth the money.  Second, its a nice reassurance to know 
that if a drive (or even two drives) fails the hardware controller can 
restore the array by popping in a new disk without me having to do 
much...I'm not sure if this is the case with the integrated or linux 
raid.  With the hardware controller, if I wanted a bit more storage 
space, I could have chosen to make a raid five array, something that as 
far as I know is unavailable in both integrated and Linux array (at 
least now without a significant performance hit.)  Hope that helps.

Frank

scotthathcock@comcast.net wrote:
> I currently have an ASUS K8V Deluxe system with an 80GB SATA drive. It
> used to have a old 10GB PATA disk for M$ dual boot. That disk died and
> was interfering with boot, so it's in the trash. 
> 
> The dead disk reminded me of the pain I will suffer if I loose my linux
> system. Even with backups of my home dir, which I'm not as good about as
> I should be, a full fresh install would be a pain. Thus, I am
> considering Raid.
> 
> Does anyone have experience with RAID on this Board? 
> Is the Via or Promise controller better for this? 
> Is there a good Howto on migrating from a non Raid disk to Raid? I
> recall seeing one but can't find it now.
> Am I wasting time and money? Should I just do a better job of backup? Is
> there a resource on what to backup to allow a fast regeneration (or
> duplication) of an existing gentoo system?
> 
> Finally, thanks for the great distribution. I use a commercial distro at
> work and I am much happier with Gentoo. :)
> 
> Scott
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Joshua Hoblitt]

[-- Attachment #1: Type: text/plain, Size: 3298 bytes --]

I have a large number (more then a dozen) 3Ware 8500 and 9500 cards.
The majority of these are 12-port SATA cards with RAID 5 volumes on
them.

Three quick observations:

* Software RAID 5 on Linux WILL NOT remap bad blocks/sectors like a
hardware RAID controller.  If you care about your data, software RAID
simply isn't an option.

* The RAID 5 performance of 3Ware controllers is terrible.  The 9500
series cards can push 50-55MB/s with xfs and in the neighborhood of
45MB/s with ext3 (with an enlarged journal, etc.) for sequential writes,
random I/O is even worse.  The 8500 cards are about 10% slower compared
to the 9500 once you fill up the on-card cache.

* Neither xfs or ext3 are reliable on volumes greater then 2TB.  Nor can
fdisk even partition them (but lvm2 can handle them).

Cheers,

-J

--
On Thu, Sep 01, 2005 at 12:36:08AM -0500, Francisco Perez wrote:
> Scott,
> I have a 3Ware Escalade 4 chaneel hardware raid controller running raid 
> 10.  Here are my thoughts:  The card itself was $175.00 and the 4 250GB 
> brives were $150 bucks a piece, so the cost is considerable.  If you are 
> only going to have two disks mirrored, you can get the two channel card 
> and two drives and mirror them which is obviously less money.  My system 
> is a Tyan Transport GX-28 with 2 Opteron 246's and originally I had the 
> drives on the onboard controller.  First, there is a very noticable 
> increase in performance going from the integrated (software) raid 
> controller to the hardware raid controller.  For my percieved needs it 
> was definitely worth the money.  Second, its a nice reassurance to know 
> that if a drive (or even two drives) fails the hardware controller can 
> restore the array by popping in a new disk without me having to do 
> much...I'm not sure if this is the case with the integrated or linux 
> raid.  With the hardware controller, if I wanted a bit more storage 
> space, I could have chosen to make a raid five array, something that as 
> far as I know is unavailable in both integrated and Linux array (at 
> least now without a significant performance hit.)  Hope that helps.
> 
> Frank
> 
> scotthathcock@comcast.net wrote:
> >I currently have an ASUS K8V Deluxe system with an 80GB SATA drive. It
> >used to have a old 10GB PATA disk for M$ dual boot. That disk died and
> >was interfering with boot, so it's in the trash. 
> >
> >The dead disk reminded me of the pain I will suffer if I loose my linux
> >system. Even with backups of my home dir, which I'm not as good about as
> >I should be, a full fresh install would be a pain. Thus, I am
> >considering Raid.
> >
> >Does anyone have experience with RAID on this Board? 
> >Is the Via or Promise controller better for this? 
> >Is there a good Howto on migrating from a non Raid disk to Raid? I
> >recall seeing one but can't find it now.
> >Am I wasting time and money? Should I just do a better job of backup? Is
> >there a resource on what to backup to allow a fast regeneration (or
> >duplication) of an existing gentoo system?
> >
> >Finally, thanks for the great distribution. I use a commercial distro at
> >work and I am much happier with Gentoo. :)
> >
> >Scott
> -- 
> gentoo-amd64@gentoo.org mailing list
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Upgrading to Raid

[Matt Randolph]

That's valuable information indeed.  Since migrating from Windows to 
Gentoo I have had to abandon an old PATA RAID 5 card because its Linux 
drivers haven't been maintained in years.  I had thought about going the 
3Ware route, so I'm glad to hear about your experiences with their 
cards.  I guess I have some more homework to do.  Thanks for the heads-up!

Joshua Hoblitt wrote:

>I have a large number (more then a dozen) 3Ware 8500 and 9500 cards.
>The majority of these are 12-port SATA cards with RAID 5 volumes on
>them.
>
>Three quick observations:
>
>* Software RAID 5 on Linux WILL NOT remap bad blocks/sectors like a
>hardware RAID controller.  If you care about your data, software RAID
>simply isn't an option.
>
>* The RAID 5 performance of 3Ware controllers is terrible.  The 9500
>series cards can push 50-55MB/s with xfs and in the neighborhood of
>45MB/s with ext3 (with an enlarged journal, etc.) for sequential writes,
>random I/O is even worse.  The 8500 cards are about 10% slower compared
>to the 9500 once you fill up the on-card cache.
>
>* Neither xfs or ext3 are reliable on volumes greater then 2TB.  Nor can
>fdisk even partition them (but lvm2 can handle them).
>
>Cheers,
>
>-J
>  
>
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Joshua Hoblitt]

[-- Attachment #1: Type: text/plain, Size: 1720 bytes --]

Supposedly the Areca cards offer much better RAID 5 performance.  Since
there is a driver in the -mm tree now I'm about to start testing a
24-port Areca SATA card.  I have *no* experience with them as of yet so
I can't recommend them.

    http://www.areca.com.tw/index/html/

-J

--
On Fri, Sep 02, 2005 at 09:22:34AM -0400, Matt Randolph wrote:
> That's valuable information indeed.  Since migrating from Windows to 
> Gentoo I have had to abandon an old PATA RAID 5 card because its Linux 
> drivers haven't been maintained in years.  I had thought about going the 
> 3Ware route, so I'm glad to hear about your experiences with their 
> cards.  I guess I have some more homework to do.  Thanks for the heads-up!
> 
> Joshua Hoblitt wrote:
> 
> >I have a large number (more then a dozen) 3Ware 8500 and 9500 cards.
> >The majority of these are 12-port SATA cards with RAID 5 volumes on
> >them.
> >
> >Three quick observations:
> >
> >* Software RAID 5 on Linux WILL NOT remap bad blocks/sectors like a
> >hardware RAID controller.  If you care about your data, software RAID
> >simply isn't an option.
> >
> >* The RAID 5 performance of 3Ware controllers is terrible.  The 9500
> >series cards can push 50-55MB/s with xfs and in the neighborhood of
> >45MB/s with ext3 (with an enlarged journal, etc.) for sequential writes,
> >random I/O is even worse.  The 8500 cards are about 10% slower compared
> >to the 9500 once you fill up the on-card cache.
> >
> >* Neither xfs or ext3 are reliable on volumes greater then 2TB.  Nor can
> >fdisk even partition them (but lvm2 can handle them).
> >
> >Cheers,
> >
> >-J
> > 
> >
> -- 
> gentoo-amd64@gentoo.org mailing list
> 

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Upgrading to Raid

[Florian D.]

Francisco Perez wrote:

> With the hardware controller, if I wanted a bit more storage 
> space, I could have chosen to make a raid five array, something that as 
> far as I know is unavailable in both integrated and Linux array
no, this is possible with software-raid, too. It is even possible to 
dynamically increase your array by adding new disks to it or by 
increasing the size of each partition.
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Homer Parker]

On Sat, 2005-09-03 at 11:54 +0200, Florian D. wrote:
> no, this is possible with software-raid, too. It is even possible to 
> dynamically increase your array by adding new disks to it or by 
> increasing the size of each partition.

	I'd be interested in documentation supporting the resizing of Linux
RAID partitions, as all of the documentation I found said it wasn't
possible.

-- 
Homer Parker
Gentoo/AMD64 Arch Tester Strategic Lead
hparker@gentoo.org

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Florian D.]

Homer Parker wrote:
> On Sat, 2005-09-03 at 11:54 +0200, Florian D. wrote:
> 
>>no, this is possible with software-raid, too. It is even possible to 
>>dynamically increase your array by adding new disks to it or by 
>>increasing the size of each partition.
> 
> 
> 	I'd be interested in documentation supporting the resizing of Linux
> RAID partitions, as all of the documentation I found said it wasn't
> possible.
> 
In the raidtools-package, there is a utility called raidreconf, which 
allows you to add new disks to a raid5 array. With mdadm (which is 
otherwise to be favoured, because it doesn´t rely on a configuration 
file), it is 'only' possible to add devices to a raid1 array - or to 
resize the partitions of a raid1/4/5/6 array accordingly (mdadm --grow) 
- alas, you´re right, documentation is very scanty and I cannot offer 
other sources of information as man-pages or sth. like:
http://www.issociate.de/board/post/213676/

Everything of the above should also be possible with EVMS, but this 
requires a lot of reading, because they are using a different terminology



-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Florian D.]

because there are so many people praising software-raid, you´ll probably 
find this story interesting, which happened to me yesterday (my system 
is -for the biggest part- on a raid5 partition):

while I emerged koffice, played some mp3´s and let matlab calculate sth. 
in the background, I tried to start a windows program with wine. I 
really shouldn´t do that, because Linux crashed and after a hard reboot 
it could not boot any more. It stopped during the execution of the 
init-scripts and i was not able to do anything, thus: 2nd hard reboot. I 
have another(old) gentoo installation on my computer and this time I 
booted into that one. By accident I realized that one of the discs of my 
raid system has been marked as faulty and it was syncing in the 
background. /proc/mdstat said that it would need 20min or so, but after 
15min it said 200min! After another 10min it was sth >300min (and a lot 
of IO errors). So I rebooted again, this time it synced successfully in 
12min. Then it was possible to boot and work with my normal system 
without any further difficulties or IO errors.

Summing up, I can say:

+ I successfully wrecked my file system, but software raid was able to 
repair it
- without a second Linux-installation on another partition, you´re lost
- restoring of a faulty raid disk is not reliable
- it seems that it is not possible to boot or work with a system, whose 
system disk is being restored at the same time (Linux-2.6.12.6)
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Francisco Perez]

Correct me if I am wrong since I have yet to have to deal with restoring 
failed disk in an array, but in a hardware raid, with my card I would 
just enter the card's bios after rebooting and then restore the new 
replacement drive from the secondary image (remember I am using raid 10 
with 4 disks.)  Is there something I'm forgetting?

Frank

Florian D. wrote:
> because there are so many people praising software-raid, you´ll probably 
> find this story interesting, which happened to me yesterday (my system 
> is -for the biggest part- on a raid5 partition):
> 
> while I emerged koffice, played some mp3´s and let matlab calculate sth. 
> in the background, I tried to start a windows program with wine. I 
> really shouldn´t do that, because Linux crashed and after a hard reboot 
> it could not boot any more. It stopped during the execution of the 
> init-scripts and i was not able to do anything, thus: 2nd hard reboot. I 
> have another(old) gentoo installation on my computer and this time I 
> booted into that one. By accident I realized that one of the discs of my 
> raid system has been marked as faulty and it was syncing in the 
> background. /proc/mdstat said that it would need 20min or so, but after 
> 15min it said 200min! After another 10min it was sth >300min (and a lot 
> of IO errors). So I rebooted again, this time it synced successfully in 
> 12min. Then it was possible to boot and work with my normal system 
> without any further difficulties or IO errors.
> 
> Summing up, I can say:
> 
> + I successfully wrecked my file system, but software raid was able to 
> repair it
> - without a second Linux-installation on another partition, you´re lost
> - restoring of a faulty raid disk is not reliable
> - it seems that it is not possible to boot or work with a system, whose 
> system disk is being restored at the same time (Linux-2.6.12.6)
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] Upgrading to Raid

[Nuitari]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2432 bytes --]

There is a huge difference between RAID5 and RAID10.

Software-RAID5 should be avoided as it is CPU intensive and there are many 
good true hardware solutions for it that are supported in Linux.


On Fri, 2 Sep 2005, Francisco Perez wrote:

> Date: Fri, 02 Sep 2005 20:24:44 -0500
> From: Francisco Perez <fperez@albrookdata.com>
> Reply-To: gentoo-amd64@lists.gentoo.org
> To: gentoo-amd64@lists.gentoo.org
> Subject: Re: [gentoo-amd64] Upgrading to Raid
> 
> Correct me if I am wrong since I have yet to have to deal with restoring 
> failed disk in an array, but in a hardware raid, with my card I would just 
> enter the card's bios after rebooting and then restore the new replacement 
> drive from the secondary image (remember I am using raid 10 with 4 disks.) 
> Is there something I'm forgetting?
>
> Frank
>
> Florian D. wrote:
>>  because there are so many people praising software-raid, you´ll probably
>>  find this story interesting, which happened to me yesterday (my system is
>>  -for the biggest part- on a raid5 partition):
>>
>>  while I emerged koffice, played some mp3´s and let matlab calculate sth.
>>  in the background, I tried to start a windows program with wine. I really
>>  shouldn´t do that, because Linux crashed and after a hard reboot it could
>>  not boot any more. It stopped during the execution of the init-scripts and
>>  i was not able to do anything, thus: 2nd hard reboot. I have another(old)
>>  gentoo installation on my computer and this time I booted into that one.
>>  By accident I realized that one of the discs of my raid system has been
>>  marked as faulty and it was syncing in the background. /proc/mdstat said
>>  that it would need 20min or so, but after 15min it said 200min! After
>>  another 10min it was sth >300min (and a lot of IO errors). So I rebooted
>>  again, this time it synced successfully in 12min. Then it was possible to
>>  boot and work with my normal system without any further difficulties or IO
>>  errors.
>>
>>  Summing up, I can say:
>>
>>  + I successfully wrecked my file system, but software raid was able to
>>  repair it
>>  - without a second Linux-installation on another partition, you´re lost
>>  - restoring of a faulty raid disk is not reliable
>>  - it seems that it is not possible to boot or work with a system, whose
>>  system disk is being restored at the same time (Linux-2.6.12.6)
>