From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from lists.gentoo.org ([140.105.134.102] helo=robin.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1IrdMQ-0003TI-4B for garchives@archives.gentoo.org; Mon, 12 Nov 2007 17:46:10 +0000 Received: from robin.gentoo.org (localhost [127.0.0.1]) by robin.gentoo.org (8.14.2/8.14.0) with SMTP id lACHgmB8024810; Mon, 12 Nov 2007 17:42:48 GMT Received: from relay.sgi.com (relay1.corp.sgi.com [192.48.171.29] (may be forged)) by robin.gentoo.org (8.14.2/8.14.0) with ESMTP id lACHglYI024772 for ; Mon, 12 Nov 2007 17:42:47 GMT Received: from cthulhu.engr.sgi.com (cthulhu.engr.sgi.com [192.26.80.2]) by relay1.corp.sgi.com (Postfix) with ESMTP id 40DB48F815D for ; Mon, 12 Nov 2007 09:42:46 -0800 (PST) Received: from conejo.engr.sgi.com (conejo.engr.sgi.com [150.166.8.69]) by cthulhu.engr.sgi.com (8.12.10/8.12.10/SuSE Linux 0.7) with ESMTP id lACHgjke007616 for ; Mon, 12 Nov 2007 09:42:45 -0800 Received: from conejo.engr.sgi.com (localhost [127.0.0.1]) by conejo.engr.sgi.com (SGI-8.12.11.20060308/8.12.11) with ESMTP id lACHgjOr212284 for ; Mon, 12 Nov 2007 09:42:45 -0800 (PST) Received: (from rsanders@localhost) by conejo.engr.sgi.com (SGI-8.12.11.20060308/8.12.11/Submit) id lACHgjXu211857 for gentoo-amd64@lists.gentoo.org; Mon, 12 Nov 2007 09:42:45 -0800 (PST) Date: Mon, 12 Nov 2007 09:42:45 -0800 From: Bob Sanders To: gentoo-amd64@lists.gentoo.org Subject: Re: [gentoo-amd64] Planning a new box - AMD Opteron or Intel Xeon? Message-ID: <20071112174245.GE207902@sgi.com> Mail-Followup-To: gentoo-amd64@lists.gentoo.org References: <20071110081152.4065e4db@mandalor.homelinux.net> <20071110095206.7a73544e@mandalor.homelinux.net> <20071111053814.06ce20f3@mandalor.homelinux.net> Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-amd64@gentoo.org Reply-to: gentoo-amd64@lists.gentoo.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071111053814.06ce20f3@mandalor.homelinux.net> Organization: SGI, Mountain View, California, U.S.A. User-Agent: Mutt/1.5.16 (2007-06-09) X-Archives-Salt: cd3a638a-a5df-469c-874a-6fa351fd972d X-Archives-Hash: 72f3fe93b3c42877c6eb4f7d1a1e2eb3 Conway S. Smith, mused, then expounded: > > About registered vs. unbuffered memory, my understanding is that for > systems w/ lots of memory (more than 4 GiB), the registers are a Good > Thing(tm). I don't really understand the electrical engineering behind > how memory & memory controllers work, so I don't claim to really grok > why registered memory is so important in systems w/ lots of RAM, but > since I'm planning on loading it with 8 or 16 GiB, I was planning on > going with registered anyway. > Registered memory is needed when the system is running 24x7 and when a large number of DIMMS are used due to trace lengths to the DIMMS. Adding registers causes power comsuption to go up as each DIMM now has around 5 W of standing power, even when the memory is not in use. It also results in a performance hit as the registers cause more latency. Whether registered memory (fo FB-DIMMs for the Intel side) are required, is really up to you and your choices. However, without a really good memory diag that runs on the os, along with logging the sel/event errors and knowing that the running bios is actually logging memory errors properly, registered memory is close to worthless for the majority of home builders. Because without the proper tools, you don't know that your memory is functioning properly. And the more DIMMs you use the more errors you'll see (assuming the bios is reporting them) and the more DIMMs you'll go through during the burn-in period, until you finally get a stable system. > > Interesting, most of what I've heard about Tyan has been really good. > Is there something specific about Tyan that's been a problem for you? > I've also heard good things about Gigabyte, but Tyan really seemed to > stand out as excellent. The two dual-socket F Gigabyte boards on > NewEgg are cheaper than the Tyans I've been considering, but > unfortunately they don't support the new Barcelona Opterons, and I'd > have to wait for new versions to be released. But then the Tyan > motherboard I liked best is in the same situation, w/ a new > Barcelona-compatible version expected later this month. I'll keep the > Gigabytes in mind. > Server motherboards tend to be better tested and have less bleeding edge hardware - I had to compile the Agere GigE driver for the ECS motherboard I had, before I grew tired of the other flakeyness and swapped in a Tyan server motherboard. > > RAM I mentioned above. Hard disk space was actually the main thing > that's prompting this new box, as I'm filling up all my current disks. > I eventually plan on filling the case w/ as many hard disks as it can > fit, probably at least 10+. This box is going to be my home fileserver > for a long time to come. But for starters I'm thinking I'll get 3x > 1TiB in RAID5, and then grow the RAID as it fills up & drive prices > drop. This will be my first time setting up RAID, I'm planning on > following the HOWTO_Setup_fully_crypted_Gentoo_on_EVMS in the > gentoo-wiki. > After spending significant time with both software RAID5 and hardware RAID1 and RADI5, and trashing both in ways all the docs say is not possible, I find RAID is very oversold for it's supposed benefits. I'd like to offer some suggestions (which are worth exactly what you're paying to see them) - - Make the file server a sperate box, do not run your desktop on the same box. You'll trash it one way or another at some point. Also, that allows you to power down the the desktop and leave the file server running, should you feel wasting electricity is justified. And the fileserver can use less powerful cpus. - Minimize the number of drives. The more drives, the sooner the box will fail. Stay away from RAID if it's possible. Use something like LVM and individual drives - Google around for Linux video recorder for some experiences in this area. - Don't buy drives from the same lot. If one fails then there is a significantly high chance another will fail at the same time. If you really insist on a RAID above 0 or 1, buy double the number of drives and expect only half of them will actually work in the target RAID after you burn them in for 2 weeks, including power cycling. - Statistically, RAID 0/1 will provide greater reliability, due to having only 2 drives, thus providing higher reliability. More than 2 drives starts to lower reliability, requireing the need to have an ECC drive (RAID5) or two (RAID6). For better reliability, one needs to go to SAS drives, but then one has to use a better controller than is found on most motherboards, thus increasing the cost and parts count, lowering the overall reliabilty, requiring more drives - hot spares. - If you intend on ever moving the drives from one system to another, stay far away from hardware RAID. It will bite you big time on this as the raid must be re-integrated each time the drives move. Even pulling the drives out and re-installing. But it is dependant upon the specific firmware used in the specific RAID card. And that firmware varies even though the exact same RAID chip is implemented. - Get to know you're recovery software and procedures. RAIDs will fail. Indeed, cause the failures (or, like me use a crappy motherboard that will cause failures). I've re-built my running RAID more times than I care to name, and in my case, am damn glad I was was running XFS so I could recover from the failures. But for most people XFS is a poor choice - see the previously mentioned LVM + XFS Linux video recorder. Given that there are now 1 TB drives, the need for a RAID at home is becoming less and less. Bob -- - -- gentoo-amd64@gentoo.org mailing list