[gentoo-amd64] SATA mdraid woes

[gentoo-amd64] SATA mdraid woes

[Peter Humphrey]

A few weeks ago I had a hardware problem, and the upshot is that I now have
a new motherboard, a SuperMicro H8DCE. I now can't boot my Gentoo system.

This box has two IDE drives on the primary IDE channel (and two optical
drives on the secondary), and I have two small ext2 partitions on /dev/hda1
to boot Linux. Windows lives in hda3, and I'm using it now to write Webmail.
Grub lives in /dev/hda1, pointed to by BootMagic in the MBR. Hdb is mostly
to keep backups of other things, being 200 GB.

Gentoo lives on two SATA drives, which the BIOS shows me as ide3 master and
ide4 master. I have a small boot partition on each of them, rarely used,
then the rest is given over to several RAID-1 partitions. E.g. the root
partition is on /dev/md2, which is assembled from /dev/sd[ab]5.

When I first got the box back I tried booting with no changes. Blank screen
and no keyboard as soon as I hit the default choice in the grub screen. That
was ok as several hardware changes have occurred. So I compiled a new kernel
to use the new graphics card (PCI Express instead of AGP) and motherboard
chipset (nForce Pro 2200/2050 instead of VIA) and network card (forcedeth
instead of tg3). Still the same, so I backed up all the data, deleted the md
and sd partitions and recreated them all afresh, then restored all the
backed-up data.

Now I get a can't-find-root error. I've experimented with lots of kernel
parameters, both when compiling and on the command line, but I can't get the
system to boot. Here's a selection of diagnostics, which I hope I've
transcribed aright:

--
ACPI: PCI Interrupt Link [LTID] enabled at IRQ22
ata1: SATA max UDMA/133 cmd 0xC800 ctl 0xC402 bmdma 0xB800 irq22
ata1: SATA max UDMA/133 cmd 0xC800 ctl 0xBC02 bmdma 0xB800 irq22
[that's the ordinary IDE subsystem]
...
scsi2: sata_nv
[so the nForce SATA functionality is compiled in ok]
ata1: SATA link up 1.5Gbps (SStatus 113 SControl 300)
ata1.00: ATA-7, max UDMA/133, 398297088 sectors: LBA48 NCQ (depth 0/32)
ata1.00: ata1: dev 0 multi count 16
ata1.00: configured for UDMA/133
scsi3: sata_nv
ata2: SATA link up 1.5Gbps (SStatus 113 SControl 300)
ata2.00: ATA-7, max UDMA/133, 398297088 sectors: LBA48 NCQ (depth 0/32)
ata2.00: ata2: dev 0 multi count 16
ata2.00: configured for UDMA/133
scsi 2:0:0:0 Direct-Access ATA Maxtor 6L200M0 BANC PQ: ANSI: 5
scsi 3:0:0:0 Direct-Access ATA Maxtor 6L200M0 BANC PQ: ANSI: 5
[that's the two SATA drives]
ACPI: PCI Interrupt Link [LTIE] enabled at IRQ 21
ACPI: PCI Interrupt 0000:00:08.0[A] -> Link [LTIE] -> GSI 21 (level, low) ->
IRQ21
...
(ata3 to ata8 links down)
...
md: linear personality registered for level -1
md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
[so I've got mdraid compiled in]
...
Activating mdev
Detected real_root as a md device. Setting up the device node
Determining root device...
Mounting root...
...
The root block device is unspecified or not detected.
--
[end of transcript]
Then I'm invited to specify another device, or enter a shell. I use the
shell to say "ls -l /dev/md2", which shows the block device I expect to see,
but "cat /dev/md2" returns an empty result. If I do that from the
installation CD I get a dump of the contents of the md disk, so it seems
that the node exists but it isn't connected to the array /dev/md2.

All I can think of is that I've made an error in creating the RAID-1 arrays,
but can anyone point me to what that might be?

--
Rgds
Peter.


________________________________________________
Message sent using UebiMiau 2.7.2

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] SATA mdraid woes

[Antoine Martin]

On Thu, 2006-12-28 at 18:06 +0000, Peter Humphrey wrote:
> A few weeks ago I had a hardware problem, and the upshot is that I now have
> a new motherboard, a SuperMicro H8DCE. I now can't boot my Gentoo system.
I had a similar problem resulting from a similar upgrade.
It was because of the order of the drives, which was different in the
BIOS (as seen by grub) and the Linux kernel.
mdadm should not care and it should be able to re-assemble the array no
matter what the partitions are named (sdc|d instead of sda|b in my case)
Some of the arrays were out of sync (I had mounted one of the raid-1
partitions separately for making a backup, etc) - booting using knoppix
(or using a simple recovery ramdisk) allowed me to re-assemble them.
Reboot, done.
It can be useful to keep a ~200MB partition to install something small
like Slackware/Busybox for emergencies, this would allow you to boot on
a single drive and see what the kernel and mdadm tools make of your
array.
Hope this helps!

Antoine

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: SATA mdraid woes

[Duncan]

"Peter Humphrey" <prh@gotadsl.co.uk> posted
20061228180653.40B4A2B8ADF@smtp.nildram.co.uk, excerpted below, on  Thu,
28 Dec 2006 18:06:48 +0000:

[snipped]

> md: linear personality registered for level -1
> md: raid0 personality registered for level 0
> md: raid1 personality registered for level 1
> [so I've got mdraid compiled in]
> ...
> Activating mdev
> Detected real_root as a md device. Setting up the device node
> Determining root device...
> Mounting root...
> ...
> The root block device is unspecified or not detected.
> --
> [end of transcript]
> Then I'm invited to specify another device, or enter a shell. I use the
> shell to say "ls -l /dev/md2", which shows the block device I expect to see,
> but "cat /dev/md2" returns an empty result. If I do that from the
> installation CD I get a dump of the contents of the md disk, so it seems
> that the node exists but it isn't connected to the array /dev/md2.
> 
> All I can think of is that I've made an error in creating the RAID-1 arrays,
> but can anyone point me to what that might be?

>From what I've seen, there aren't a lot of folks on this list doing RAID,
and some of the ones that are, are using the DM-RAID firmware-RAID stuff,
rather than md-RAID.

I'm doing RAID, but RAID-only, no non-RAID boot and no initramfs, so that
aspect of it I'm unfamiliar with and that seems to be the problem, so I'll
be of limited help.  I'm /guessing/ the most likely list to have real RAID
experts on it is going to be the gentoo-server list, which I've never
subscribed to so I can't say for sure /what/ they call topical there.

FWIW tho I don't see that it's going to help you presently, unless you
decide to rework to do something similar, I'm setup using md RAID-0, -1,
and -6 on four identically partitioned SATA drives. RAID-0 for /boot since
GRUB can work with it.  Partitioned RAID-6 for my main system, with root
(including everything portage writes to, so much of /var and /usr, on
root, keeping portage in sync with what's on the partition) and a backup
root image on two of the RAID-6 partitions (I'd go with a second backup
image if I redid it) and an LVM2 managed RAID-6 partition as well for
data.  The RAID-0 covers all the temp and redownloadable stuff such as the
portage tree.

Critically, my root and backup root partitions are directly on the
partitioned RAID-6, not on LVM2, so I don't need an initramfs.  md-RAID is
built-in and can be configured on the kernel command line from GRUB, while
LVM2 requires userspace configuration, thus an initramfs, which I can
avoid by placing my root and backup directly on partitioned RAID-6
partitions.

Thus, in the event of motherboard/SATA-chipset hardware failure, all
that's needed to get going again is a new mobo and the ability to compile
a kernel with the appropriate standard SATA drivers for the new chipset. 
The kernel is pointed at the correct root from its command line directly,
no initramfs or the like needed, and lvm2 loads from the main root and
only manages non-system data, so I have a fully working root complete with
all the usual binaries to work with if I have lvm2 issues.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] SATA mdraid woes

[Boyd Stephen Smith Jr.]

[-- Attachment #1: Type: text/plain, Size: 1722 bytes --]

On Thursday 28 December 2006 12:06, "Peter Humphrey" <prh@gotadsl.co.uk> 
wrote about '[gentoo-amd64] SATA mdraid woes':
> Then I'm invited to specify another device, or enter a shell. I use the
> shell to say "ls -l /dev/md2", which shows the block device I expect to
> see, but "cat /dev/md2" returns an empty result. If I do that from the
> installation CD I get a dump of the contents of the md disk, so it seems
> that the node exists but it isn't connected to the array /dev/md2.

I have the same symptoms on my RAID 0, although I have a slightly different 
setup.  I am able to boot because although the RAID 0 is a pv in my single 
vg LVM setup, all the data for my root lv is on other disks, but I haven't 
been able to boot unattended for many months now.

Anyway, I have two suggestions for you.  First, use your favorite fdisk 
tool to confirm all your RAID partitions are set to be RAID autodetect; 
some initrds and particularly the kernel code for RAID detection is very 
picky about this.  [Unfortunately, this is not possible for me, since I 
put whole disks into the RAID array; not partitions.]

If that fails to fix the problem, you might create your own 
initrd/initramfs that includes the full mdadm tool and your mdadm.conf, 
because genkernel (and indeed all "mkinitrd"s I've run under Gentoo) just 
doesn't seem to be able to handle all RAID correctly -- in particular my 
setup has never worked, independent of the options I provide the script.

-- 
"If there's one thing we've established over the years,
it's that the vast majority of our users don't have the slightest
clue what's best for them in terms of package stability."
-- Gentoo Developer Ciaran McCreesh

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [gentoo-amd64] Re: SATA mdraid woes

[Mike Doty]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Duncan wrote:
> "Peter Humphrey" <prh@gotadsl.co.uk> posted
> 20061228180653.40B4A2B8ADF@smtp.nildram.co.uk, excerpted below, on  Thu,
> 28 Dec 2006 18:06:48 +0000:
> 
> [snipped]
> 
>> md: linear personality registered for level -1
>> md: raid0 personality registered for level 0
>> md: raid1 personality registered for level 1
>> [so I've got mdraid compiled in]
>> ...
>> Activating mdev
>> Detected real_root as a md device. Setting up the device node
>> Determining root device...
>> Mounting root...
>> ...
>> The root block device is unspecified or not detected.
>> --
>> [end of transcript]
>> Then I'm invited to specify another device, or enter a shell. I use the
>> shell to say "ls -l /dev/md2", which shows the block device I expect to see,
>> but "cat /dev/md2" returns an empty result. If I do that from the
>> installation CD I get a dump of the contents of the md disk, so it seems
>> that the node exists but it isn't connected to the array /dev/md2.
>>
>> All I can think of is that I've made an error in creating the RAID-1 arrays,
>> but can anyone point me to what that might be?
> 
> From what I've seen, there aren't a lot of folks on this list doing RAID,
> and some of the ones that are, are using the DM-RAID firmware-RAID stuff,
> rather than md-RAID.
damn ricers!

here are the 2 most commonly overlooked items:  the sdXY partitions must
be type FD (linux raid) and you must use a persistent superblock.

- --
=======================================================
Mike Doty                      kingtaco -at- gentoo.org
Gentoo/AMD64 Strategic Lead
Gentoo Council
Gentoo Developer Relations
Gentoo Recruitment Lead
Gentoo Infrastructure
GPG: E1A5 1C9C 93FE F430 C1D6  F2AF 806B A2E4 19F4 AE05
=======================================================
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.5 (GNU/Linux)

iQCVAwUBRZSTt4BrouQZ9K4FAQJtJAQAzI6r4+AetIHB4pmFtmQFL7mCfm090u2V
uS+96fSwweS46YMEoWJSNS/OB9PDJIgGpSCLUwcxl55+7T96yNQl0HVwba0C7IGV
P7s7UAH43pRXTpb1S4XClUNnrK6whle5ohCH72imU2a5SaGEiDBedFwng1C3JRPD
MyNwgw8aHuA=
=fUn9
-----END PGP SIGNATURE-----
-- 
gentoo-amd64@gentoo.org mailing list

Re: [gentoo-amd64] SATA mdraid woes

[Thomas Rösner]

Peter Humphrey wrote:
> A few weeks ago I had a hardware problem, and the upshot is that I now have
> a new motherboard, a SuperMicro H8DCE. I now can't boot my Gentoo system.
>
> [...]
> All I can think of is that I've made an error in creating the RAID-1 arrays,
> but can anyone point me to what that might be?
>   

Did you create the partitions with type "Linux RAID Autodetect"?

Regards,
    T.
-- 
gentoo-amd64@gentoo.org mailing list