Re: [gentoo-amd64] Re: Is my RAID performance bad possibly due to starting sector value?

[Rich Freeman <rich0@gentoo.org>]

On Fri, Jun 21, 2013 at 10:27 AM, Duncan <1i5t5.duncan@cox.net> wrote:
> Rich Freeman posted on Fri, 21 Jun 2013 06:28:35 -0400 as excerpted:
>
>> That is what is keeping me away.  I won't touch it until I can use it
>> with raid5, and the first common containing that hit the kernel weeks
>> ago I think (and it has known gaps).  Until it is stable I'm sticking
>> with my current setup.
>
> Question:  Would you use it for raid1 yet, as I'm doing?  What about as a
> single-device filesystem?  Do you believe my estimates of reliability in
> those cases (almost but not quite stable for single-device, kind of in
> the middle for raid1/raid0/raid10, say a year behind single-device and
> raid5/6/50/60 about a year behind that) reasonably accurate?

If I wanted to use raid1 I might consider using btrfs now.  I think it
is still a bit risky, but the established use cases have gotten a fair
bit of testing now.  I'd be more confident in using it with a single
device.

>
> Because if you're waiting until btrfs raid5 is fully stable, that's
> likely to be some wait yet -- I'd say a year, likely more given that
> everything btrfs has seemed to take longer than people expected.

That's my thought as well.  Right now I'm not running out of space, so
I'm hoping that I can wait until the next time I need to migrate my
data (from 1TB to 5+TB drives, for example).  With such a scenario I
don't need to have 10 drives mounted at once to migrate the data - I
can migrate existing data to 1-2 drives, remove the old ones, and
expand the new array. To migrate today would require finding someplace
to dump all the data offline and migrate the drives, as there is no
in-place way to migrate multiple ext3/4 logical volumes on top of
mdadm to a single btrfs on bare metal.

Without replying to anything in particular both you and Bob have
mentioned the importance of multiple redundancy.

Obviously risk goes down as redundancy goes up.  If you protect 25
drives of data with 1 drive of parity then you need 2/26 drives to
fail to hose 25 drives of data.  If you protect 1 drive of data with
25 drives of parity (call them mirrors or parity or whatever - they're
functionally equivalent) then you need 25/26 drives to fail to lose 1
drive of data.  RAID 1 is actually less effective - if you protect 13
drives of data with 13 mirrors you need 2/26 drives to fail to lose 1
drive of data (they just have to be the wrong 2).  However, you do
need to consider that RAID is not the only way to protect data, and
I'm not sure that multiple-redundancy raid-1 is the most
cost-effective strategy.

If I had 2 drives of data to protect and had 4 spare drives to do it
with, I doubt I'd set up a 3x raid-1/5/10 setup (or whatever you want
to call it - imho raid "levels" are poorly named as there really is
just striping and mirroring and adding RS parity and everything else
is just combinations).  Instead I'd probably set up a
RAID1/5/10/whatever with single redundancy for faster storage and
recovery, and an offline backup (compressed and with
incrementals/etc).  The backup gets you more security and you only
need it in a very unlikely double-failure.  I'd only invest in
multiple redundancy in the event that the risk-weighted cost of having
the node go down exceeds the cost of the extra drives.  Frankly in
that case RAID still isn't the right solution - you need a backup node
someplace else entirely as hard drives aren't the only thing that can
break in your server.

This sort of rationale is why I don't like arguments like "RAM is
cheap" or "HDs are cheap" or whatever.  The fact is that wasting money
on any component means investing less in some other component that
could give you more space/performance/whatever-makes-you-happy.  If
you have $1000 that you can afford to blow on extra drives then you
have $1000 you could blow on RAM, CPU, an extra server, or a trip to
Disney.  Why not blow it on something useful?

Rich