[gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[Stroller]

Just a quick question to see if any of the list members are using  
Gentoo - or any other Linux distro for that matter - on Dell  
PowerEdge 2600 or 2800 servers?

A site I manage has had from new a 2800 running Windows, which we're  
quite happy with (the 2800, that is, not Windows ;). We really need  
new hardware for our Linux-based mailserver & similar systems seem to  
be quite affordable on the secondhand market, and it would make quite  
a bit of sense for us to use one of these.

I haven't done much digging yet, but thought a quick show of hands  
here might save some time. It looks like the SCSI hot-swap / RAID  
controller uses an AMI / LSI MegaRAID driver which is (?) part of the  
main kernel - anyone know if that does status updates (dead-hard  
drives &c) to the syslog? Does it depend on any userland utilities  
that are only available as RPM or whatever?

I know RedHat &/or Suse are supported on this machine, but I've been  
using Gentoo so long now I find it hard to use them thar binary  
distros. It'd also be nice if power-supply failures were logged in  
the same way - anyone know? I've had some experience in the past with  
a Compaq Proliant 6500 and certain utilities for that would only  
report problems via SNMP, which was a bit of a pain.

Cheers,

Stroller.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[kashani]

Stroller wrote:
> Just a quick question to see if any of the list members are using Gentoo 
> - or any other Linux distro for that matter - on Dell PowerEdge 2600 or 
> 2800 servers?
> 
> A site I manage has had from new a 2800 running Windows, which we're 
> quite happy with (the 2800, that is, not Windows ;). We really need new 
> hardware for our Linux-based mailserver & similar systems seem to be 
> quite affordable on the secondhand market, and it would make quite a bit 
> of sense for us to use one of these.
> 
> I haven't done much digging yet, but thought a quick show of hands here 
> might save some time. It looks like the SCSI hot-swap / RAID controller 
> uses an AMI / LSI MegaRAID driver which is (?) part of the main kernel - 
> anyone know if that does status updates (dead-hard drives &c) to the 
> syslog? Does it depend on any userland utilities that are only available 
> as RPM or whatever?
> 
> I know RedHat &/or Suse are supported on this machine, but I've been 
> using Gentoo so long now I find it hard to use them thar binary distros. 
> It'd also be nice if power-supply failures were logged in the same way - 
> anyone know? I've had some experience in the past with a Compaq Proliant 
> 6500 and certain utilities for that would only report problems via SNMP, 
> which was a bit of a pain.

	I used Redhat, Fedora, and Gentoo on 2550, 1650, 2650, 1750, 1850, and 
2850 PowerEdge servers. Never had an issue and never had driver issues 
other than early tg3 ether driver problems with Redhat 8. I'd assume the 
2800 and 2600s are roughly the same.
	Other than the CPU/RAM the main different between 2650, 2850, and 2950 
was the SCSI card. I'd choose the 2850 over the 2650 given a choice for 
anything with heavy I/O and the 2950 are noticeably faster than the 2850 
for our db stuff.

The SCSI on 2850's should be megaraid and you want the megaraid-new 
driver and Linux kernels would have issues if you tried to build both 
new and old so just pick new. (this might have changed in the past year 
since I've built a custom kernel for a 2850). I never had driver issues 
with any distro provided kernel or my own kernels.

IIRC you can pull the megarc RPMs from Dell's website and install them. 
I never got around to making them work with Gentoo, but it shouldn't be 
terribly hard. I don't know of anything in the normal driver that will 
tell you any ifo about status or failed drives, but I never looked that 
hard.

I bought most of my 2850's about two years ago. Dual Xeon's, 8GB, 6 x 
10k 146GB drives, and remote management card for about $4000. Discount 
as appropriate.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[Steve Dommett]

[-- Attachment #1: Type: text/plain, Size: 1492 bytes --]

On Thursday 20 December 2007, Stroller wrote:
> I haven't done much digging yet, but thought a quick show of hands
> here might save some time. It looks like the SCSI hot-swap / RAID
> controller uses an AMI / LSI MegaRAID driver which is (?) part of the
> main kernel - anyone know if that does status updates (dead-hard
> drives &c) to the syslog? Does it depend on any userland utilities
> that are only available as RPM or whatever?
I maintain a few Poweredges, I think mostly 2950.  Just yesterday we swapped a 
drive on the Fusion MPT SAS controller.  We were prompted to take the drive 
out of service by an email from 'smartd'.  I couldn't find any evidence of 
bad sectors or I/O timeouts in /var/log/messages, so this must be the SMART 
prefailure it purported to be in the email.  In /etc/smartd.conf I use:
DEVICESCAN -H -l error -l selftest -t -I 194 -W 5,45,48 -R 5 -R 194 -R 231 -m 
alerts@mydomain.com

After failing and removing the drive from the array using 'mdadm', we tried 
hotswapping the drive, and whilst nothing untoward happened when we pulled 
the drive there were no kernel messages either.  I was expecting something 
similar to when I've hotplugged SATA drives on my desktop machine.  We had to 
reboot the server to get it to see the replacement drive.  Perhaps there's 
some /proc/ or /sys/ setting to trigger a rescan of the SCSI bus, but I 
couldn't find it.

Other than those oddities the drive swap went well.

Cheers,
  Steve.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[Stroller]

On 20 Dec 2007, at 07:26, kashani wrote:
> Stroller wrote:
>> Just a quick question to see if any of the list members are using  
>> Gentoo - or any other Linux distro for that matter - on Dell  
>> PowerEdge 2600 or 2800 servers?
>> ...
> 	I used Redhat, Fedora, and Gentoo on 2550, 1650, 2650, 1750, 1850,  
> and 2850 PowerEdge servers ...

Blimey! You obviously know your stuff. So how do you find Gentoo  
measures up to Redhat / Fedora on these machines?

> 	Other than the CPU/RAM the main different between 2650, 2850, and  
> 2950 was the SCSI card. I'd choose the 2850 over the 2650 given a  
> choice for anything with heavy I/O and the 2950 are noticeably  
> faster than the 2850 for our db stuff.

Ours is a 2800, and it's the 2600 that I find most readily / cheaply  
available. Looks like the xx50 models are the rack-mount & lower- 
profile models of the same generation. Looks like they're more  
expensive secondhand and it's not obvious if hot-swap PSUs are  
available?

The machines at this site aren't under high-load, so that's not  
really a problem. We like this class of servers for the redundancy of  
the moving-and-failure-prone kind of parts (PSU & disks).

If I might ask some follow-up questions:
Are the SCSI cards in these models the same brand / chipset / Linux  
driver, please?
Or are they completely different?

> The SCSI on 2850's should be megaraid and you want the megaraid-new  
> driver and Linux kernels would have issues if you tried to build  
> both new and old so just pick new. (this might have changed in the  
> past year since I've built a custom kernel for a 2850). I never had  
> driver issues with any distro provided kernel or my own kernels.

Thanks for that pointer.

> IIRC you can pull the megarc RPMs from Dell's website and install  
> them. I never got around to making them work with Gentoo, but it  
> shouldn't be terribly hard. I don't know of anything in the normal  
> driver that will tell you any ifo about status or failed drives,  
> but I never looked that hard.

Hmmmn... googling a bit further I find that `megarc` are the  
userspace utilities for these cards, and that they're only available  
as binaries. I feel my enthusiasm for these units flagging - the cost  
savings of buying secondhand aren't so much that I wouldn't rather  
find a fully-OSS alternative.

> I bought most of my 2850's about two years ago. Dual Xeon's, 8GB, 6  
> x 10k 146GB drives, and remote management card for about $4000.

Yeah, we paid £1300, I think, at about the same time. Dual Xeons &  
the DRAC, but much less RAM & disk-space.

Stroller.--
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[Stroller]

On 20 Dec 2007, at 07:31, Steve Dommett wrote:
> On Thursday 20 December 2007, Stroller wrote:
>
> I maintain a few Poweredges, I think mostly 2950.  Just yesterday  
> we swapped a
> drive on the Fusion MPT SAS controller.  We were prompted to take  
> the drive
> out of service by an email from 'smartd'...
>
> After failing and removing the drive from the array using 'mdadm',  
> we tried
> hotswapping the drive, and whilst nothing untoward happened when we  
> pulled
> the drive there were no kernel messages either. ... We had to
> reboot the server to get it to see the replacement drive.

Funnily enough, I've experienced something similar on this 2800 of  
ours the last couple of days, also a prefailure. This machine is  
running Windows, and the new drive was recognised in OpenManage  
Server Administrator <https://localhost:1311> but despite it showing  
exactly the same size (68.24gig) as the other two already installed  
(as RAID1) when I tried to assign it as global hot-spare I got  
directed to a message saying that it was insufficient to accommodate  
all virtual disks.

The Dell tech support guy - who has been BRILLIANTLY helpful over a  
simple failed drive, by the way - advised installing the latest  
firmware updates. After rebooting the drive has been fine, and I was  
able to allocate as hot-spare with a single click, although I guess I  
can't say whether this is because of the updates or of just the  
reboot. But the engineer also mentioned these updates, so multiple  
sources concur, at least. The DRAC remote-administration unit also  
seems much more responsive with the newer firmware.


> ... I was expecting something
> similar to when I've hotplugged SATA drives on my desktop machine.

What controller is in that, please?

Does it do hardware RAID, or is it just a regular SATA controller?

I've been reading a little about hotplugging SATA recently, and as I  
understand it hotplugging is a part of the SATA specification in a  
way that it's not in EIDE (or even SCSI?). But what I read also  
stated that SATA controllers are not _required_ to support hot- 
plugging, either. This makes choosing an SATA more complicated, of  
course - I can't hep thinking it's easiest to plump for a SATA  
controller advertised to do hot-swap hardware RAID - I imagine this  
might be better marketed than a regular SATA controller that happens  
to support hot-swapping (but no RAID).

Due simply to the price of disks we'd tend to choose hot-plug SATA  
RAID over hot-plug SCSI, if were to buy new.

Stroller.
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[kashani]

Stroller wrote:
> On 20 Dec 2007, at 07:26, kashani wrote:
>>     I used Redhat, Fedora, and Gentoo on 2550, 1650, 2650, 1750, 1850, 
>> and 2850 PowerEdge servers ...
> 
> Blimey! You obviously know your stuff. So how do you find Gentoo 
> measures up to Redhat / Fedora on these machines?

	Never had an issue with Gentoo on any of them. The SCSI and ether 
drivers were well supported.

>>     Other than the CPU/RAM the main different between 2650, 2850, and 
>> 2950 was the SCSI card. I'd choose the 2850 over the 2650 given a 
>> choice for anything with heavy I/O and the 2950 are noticeably faster 
>> than the 2850 for our db stuff.
> 
> Ours is a 2800, and it's the 2600 that I find most readily / cheaply 
> available. Looks like the xx50 models are the rack-mount & lower-profile 
> models of the same generation. Looks like they're more expensive 
> secondhand and it's not obvious if hot-swap PSUs are available?

I am not sure about the xx00 series, but you could hot swap PSUs in the 
xx50 machines.

> The machines at this site aren't under high-load, so that's not really a 
> problem. We like this class of servers for the redundancy of the 
> moving-and-failure-prone kind of parts (PSU & disks).
> 
> If I might ask some follow-up questions:
> Are the SCSI cards in these models the same brand / chipset / Linux 
> driver, please?
> Or are they completely different?

Hmmm the SCSI card was onboard and you could get RAID by adding the 
memory dimm/unlocker doohicky if your system didn't come with it. We hit 
Ebay and picked up a bunch for cheap. Within a series the SCSI card was 
always the same other than maybe minor revision. Perc3i ver 3, ver 2, 
and etc in the 2600 and then Perc4i ver 1, ver 2 in the 2800.
	You'd never have an issue with an early rev or later rev having issues 
in any 2.6 kernel I ran.

kashani
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] Gentoo on Dell PowerEdge 2600 / 2800? AMI / LSI MegaRAID driver?

[Steve Dommett]

On Thursday 20 December 2007, Stroller wrote:
> > ... I was expecting something
> > similar to when I've hotplugged SATA drives on my desktop machine.
>
> What controller is in that, please?
>
> Does it do hardware RAID, or is it just a regular SATA controller?
I've done it using both the onboard controllers: nVidia nForce4 CK804 SATA, 
and Silicon Image SiI 3114.  They both claim RAID but I'm sure it's done by 
the driver in both cases.

> stated that SATA controllers are not _required_ to support hot-
> plugging, either. This makes choosing an SATA more complicated, of
Eek!  Maybe I'm just lucky, but I've swapped the SATA drive from my laptop to 
my desktop and back quite a few times without an issue.  Friends have 
hotplugged their drives into this machine too many times with no ill effects.  
We plug the power into the drive first, then once it's spun up insert the 
SATA data cable.
-- 
gentoo-user@gentoo.org mailing list