Re: [gentoo-amd64] Planning a new box - AMD Opteron or Intel Xeon?

[Bob Sanders <rsanders@sgi.com>]

Conway S. Smith, mused, then expounded:
> 
> About registered vs. unbuffered memory, my understanding is that for
> systems w/ lots of memory (more than 4 GiB), the registers are a Good
> Thing(tm). I don't really understand the electrical engineering behind
> how memory & memory controllers work, so I don't claim to really grok
> why registered memory is so important in systems w/ lots of RAM, but
> since I'm planning on loading it with 8 or 16 GiB, I was planning on
> going with registered anyway.
>

Registered memory is needed when the system is running 24x7 and when
a large number of DIMMS are used due to trace lengths to the DIMMS.

Adding registers causes power comsuption to go up as each DIMM now
has around 5 W of standing power, even when the memory is not in use.
It also results in a performance hit as the registers cause more latency.

Whether registered memory (fo FB-DIMMs for the Intel side) are required,
is really up to you and your choices.  However, without a really good
memory diag that runs on the os, along with logging the sel/event errors
and knowing that the running bios is actually logging memory errors
properly, registered memory is close to worthless for the majority
of home builders.  Because without the proper tools, you don't know
that your memory is functioning properly.  And the more DIMMs you
use the more errors you'll see (assuming the bios is reporting them)
and the more DIMMs you'll go through during the burn-in period, until
you finally get a stable system.
 
> 
> Interesting, most of what I've heard about Tyan has been really good.
> Is there something specific about Tyan that's been a problem for you?
> I've also heard good things about Gigabyte, but Tyan really seemed to
> stand out as excellent.  The two dual-socket F Gigabyte boards on
> NewEgg are cheaper than the Tyans I've been considering, but
> unfortunately they don't support the new Barcelona Opterons, and I'd
> have to wait for new versions to be released.  But then the Tyan
> motherboard I liked best is in the same situation, w/ a new
> Barcelona-compatible version expected later this month.  I'll keep the
> Gigabytes in mind.
>

Server motherboards tend to be better tested and have less bleeding edge
hardware - I had to compile the Agere GigE driver for the ECS motherboard
I had, before I grew tired of the other flakeyness and swapped in a Tyan
server motherboard.
 
> 
> RAM I mentioned above.  Hard disk space was actually the main thing
> that's prompting this new box, as I'm filling up all my current disks.
> I eventually plan on filling the case w/ as many hard disks as it can
> fit, probably at least 10+.  This box is going to be my home fileserver
> for a long time to come.  But for starters I'm thinking I'll get 3x
> 1TiB in RAID5, and then grow the RAID as it fills up & drive prices
> drop.  This will be my first time setting up RAID, I'm planning on
> following the HOWTO_Setup_fully_crypted_Gentoo_on_EVMS in the
> gentoo-wiki.
>

After spending significant time with both software RAID5 and hardware 
RAID1 and RADI5, and trashing both in ways all the docs say is not 
possible, I find RAID is very oversold for it's supposed benefits.  
I'd like to offer some suggestions (which are worth exactly what 
you're paying to see them) -

   - Make the file server a sperate box, do not run your desktop on
     the same box.  You'll trash it one way or another at some point.
     Also, that allows you to power down the the desktop and leave
     the file server running, should you feel wasting electricity
     is justified.  And the fileserver can use less powerful cpus.

   - Minimize the number of drives.  The more drives, the sooner the
     box will fail.  Stay away from RAID if it's possible.  Use something
     like LVM and individual drives - Google around for Linux video
     recorder for some experiences in this area.

   - Don't buy drives from the same lot.  If one fails then there is a
     significantly high chance another will fail at the same time.  If
     you really insist on a RAID above 0 or 1, buy double the number
     of drives and expect only half of them will actually work in
     the target RAID after you burn them in for 2 weeks, including
     power cycling.

   - Statistically, RAID 0/1 will provide greater reliability, due
     to having only 2 drives, thus providing higher reliability.  More
     than 2 drives starts to lower reliability, requireing the need
     to have an ECC drive (RAID5) or two (RAID6).  For better reliability,
     one needs to go to SAS drives, but then one has to use a better
     controller than is found on most motherboards, thus increasing the
     cost and parts count, lowering the overall reliabilty, requiring
     more drives - hot spares.

   - If you intend on ever moving the drives from one system to another,
     stay far away from hardware RAID.  It will bite you big time on this
     as the raid must be re-integrated each time the drives move.  Even
     pulling the drives out and re-installing.  But it is dependant upon
     the specific firmware used in the specific RAID card.  And that
     firmware varies even though the exact same RAID chip is implemented.

   - Get to know you're recovery software and procedures.  RAIDs will fail.
     Indeed, cause the failures (or, like me use a crappy motherboard that
     will cause failures).  I've re-built my running RAID more times
     than I care to name, and in my case, am damn glad I was was running
     XFS so I could recover from the failures.  But for most people XFS
     is a poor choice - see the previously mentioned LVM + XFS Linux
     video recorder.

Given that there are now 1 TB drives, the need for a RAID at home is becoming
less and less.

Bob
-- 
-  
-- 
gentoo-amd64@gentoo.org mailing list