[gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[J. Roeleveld]

Hi All,

I am currently installing a new server and am using Linux software raid to 
merge 6 * 1.5TB drives in a RAID5 configuration.

Creating the RAID5 takes over 20 hours (according to " cat /proc/mdstat ")

Is there a way that will speed this up? The drives are new, but contain random 
data left over from some speed and reliability tests I did. I don't care about 
keeping the current 'data', as long as when the array is reliable later.

Can I use the " --assume-clean " option with mdadm and then expect it to keep 
working, even through reboots?
Or is this a really bad idea?

Many thanks,

Joost Roeleveld

Re: [gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[Kyle Bader]

Most of the wait I would assume is due to the size of the volume and
creating parity.  If it was my array I'd probably just sit tight and
wait it out.

On 2/1/10, J. Roeleveld <joost@antarean.org> wrote:
> Hi All,
>
> I am currently installing a new server and am using Linux software raid to
> merge 6 * 1.5TB drives in a RAID5 configuration.
>
> Creating the RAID5 takes over 20 hours (according to " cat /proc/mdstat ")
>
> Is there a way that will speed this up? The drives are new, but contain
> random
> data left over from some speed and reliability tests I did. I don't care
> about
> keeping the current 'data', as long as when the array is reliable later.
>
> Can I use the " --assume-clean " option with mdadm and then expect it to
> keep
> working, even through reboots?
> Or is this a really bad idea?
>
> Many thanks,
>
> Joost Roeleveld
>
>

-- 
Sent from my mobile device


Kyle

Re: [gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[Stroller]

On 1 Feb 2010, at 11:58, J. Roeleveld wrote:
> ...
> I am currently installing a new server and am using Linux software  
> raid to
> merge 6 * 1.5TB drives in a RAID5 configuration.
>
> Creating the RAID5 takes over 20 hours (according to " cat /proc/ 
> mdstat ")
>
> Is there a way that will speed this up? The drives are new, but  
> contain random
> data left over from some speed and reliability tests I did. I don't  
> care about
> keeping the current 'data', as long as when the array is reliable  
> later.
>
> Can I use the " --assume-clean " option with mdadm and then expect  
> it to keep
> working, even through reboots?
> Or is this a really bad idea?


It wasn't my intention to chide you - I don't use software RAID  
myself, and your question piqued my curiosity - but the first three  
Google hits for "assume-clean" indicate that this isn't safe to use  
with RAID5.

The 4th Google hit contains an extract from the manpage:

               ... It can
               also be used when creating a RAID1 or RAID10 if you want
               to avoid the initial resync, however this practice --
               while normally safe -- is not recommended. Use this
               only if you really know what you are doing.


I have to say that I don't fully understand this. I would have thought  
that one could pretend the entire array was empty, and the RAID driver  
would just overwrite the disk as you write to the filesystem. The  
parts used by the filesystem are the only parts you care about, and I  
wouldn't have thought it would matter if the unused parts weren't in  
sync. I would be delighted if someone could explain me.

I kinda expected this 20 hours to be spent verifying that the disks  
contain no bad sectors, which would really hose you if it were the case.

But OTOH, 20 hours does not seem an outrageous amount of time for  
building a 7.5TB array. You're not going to do this often, and you  
want it done right.

It would be interesting to know whether hardware RAID would behave any  
differently or allow the sync to perform in the background. I have  
only 1.5TB in RAID5 across 4 x 500gb drives at present; IIRC the  
expansion from 3 x drives took some hours, but I can't recall the  
initial setup.

Stroller.

Re: [gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[J. Roeleveld]

On Monday 01 February 2010 14:20:28 Stroller wrote:
> On 1 Feb 2010, at 11:58, J. Roeleveld wrote:
> > ...
> > I am currently installing a new server and am using Linux software
> > raid to
> > merge 6 * 1.5TB drives in a RAID5 configuration.
> >
> > Creating the RAID5 takes over 20 hours (according to " cat /proc/
> > mdstat ")
> >
> > Is there a way that will speed this up? The drives are new, but
> > contain random
> > data left over from some speed and reliability tests I did. I don't
> > care about
> > keeping the current 'data', as long as when the array is reliable
> > later.
> >
> > Can I use the " --assume-clean " option with mdadm and then expect
> > it to keep
> > working, even through reboots?
> > Or is this a really bad idea?
> 
> It wasn't my intention to chide you - I don't use software RAID
> myself, and your question piqued my curiosity - but the first three
> Google hits for "assume-clean" indicate that this isn't safe to use
> with RAID5.
> 
> The 4th Google hit contains an extract from the manpage:
> 
>                ... It can
>                also be used when creating a RAID1 or RAID10 if you want
>                to avoid the initial resync, however this practice --
>                while normally safe -- is not recommended. Use this
>                only if you really know what you are doing.

I did find the same results on Google, but not really a proper explanation as 
to why it's a "bad idea". Unfortunately, my budget doesn't extend to a 
hardware raid solution. (The cheap cards offload it to the CPU anyway and are 
generally considered slower in various benchmarks)

> I kinda expected this 20 hours to be spent verifying that the disks
> contain no bad sectors, which would really hose you if it were the case.

True, but I already ran "badblocks" twice on each disk to verify that the 
disks are fine. (No badblocks found).

> But OTOH, 20 hours does not seem an outrageous amount of time for
> building a 7.5TB array. You're not going to do this often, and you
> want it done right.

Good point, and I agree, which is why I will let it finish it's course, but I 
also expected it could be done quicker.

> It would be interesting to know whether hardware RAID would behave any
> differently or allow the sync to perform in the background. I have
> only 1.5TB in RAID5 across 4 x 500gb drives at present; IIRC the
> expansion from 3 x drives took some hours, but I can't recall the
> initial setup.

I'm hoping someone with more knowledge about RAID-systems can throw in his/her 
2cents.

Thanks,

Joost

Re: [gentoo-user] Raid 5 creation is slow - Can this be done quicker?

[Kyle Bader]

>> It would be interesting to know whether hardware RAID would behave any
>> differently or allow the sync to perform in the background. I have
>> only 1.5TB in RAID5 across 4 x 500gb drives at present; IIRC the
>> expansion from 3 x drives took some hours, but I can't recall the
>> initial setup.

LSI, 3ware and Areca hardware raid controllers are capable of doing a
background init  but their performance is impacted, I can't speak on
other controllers as I haven't used them before.  I've built many
RAID6 arrays with all three controllers - 8x 1TB and 8x 1.5TB and I'll
usually start a foreground init and let them run overnight because it
does take a long time.  Also, RAID10 is much faster to get up and
running because it doesn't have to calculate parity.

-- 

Kyle

Re: [gentoo-user] Raid 5 creation is slow - Can this be done quicker? [SOLVED]

[J. Roeleveld]

On Monday 01 February 2010 12:58:49 J. Roeleveld wrote:
> Hi All,
> 
> I am currently installing a new server and am using Linux software raid to
> merge 6 * 1.5TB drives in a RAID5 configuration.
> 
> Creating the RAID5 takes over 20 hours (according to " cat /proc/mdstat ")
> 
> Is there a way that will speed this up? The drives are new, but contain
>  random data left over from some speed and reliability tests I did. I don't
>  care about keeping the current 'data', as long as when the array is
>  reliable later.
> 
> Can I use the " --assume-clean " option with mdadm and then expect it to
>  keep working, even through reboots?
> Or is this a really bad idea?
> 
> Many thanks,
> 
> Joost Roeleveld
> 

Hi all,

Many thanks for all the input, I did wait the 20 hours, but when it was 
finished, the performance was still slow. And trying out different options for 
the array didn't actually help.

Thanks to the thread "1-Terabyte drives - 4K sector sizes? -> bar performance 
so far" I figured out the problem (4KB sectors).
After changing the partitions to use sector 64 as start (as opposed to 63) a 
build of the array should only take 6 hours.
Hopefully, the raid-array will also show a better performance when this is 
finished.

--
Joost Roeleveld