[gentoo-user] problem with raid1: error while booting

[gentoo-user] problem with raid1: error while booting

[Jarry]

Hi,

I'm trying to install gentoo on a small server in full sw-raid1 setup,
using info in http://www.gentoo.org/doc/en/gentoo-x86-tipsntricks.xml

I have a similar setup with /boot / and swap on md. After creating md's
I continued with installation using info in
http://www.gentoo.org/doc/en/handbook/2005.0/index.xml

I recompilled kernel with md-support (no modules), installed boot-loader
(lilo, on /dev/md0), created /etc/fstab, emerged mdadm, etc, etc...
Everything was OK until 10.d: Rebooting the System

When booting into new system (this time from disks, not from
installation cd), I got following messages:
---------------------------------------------------------------------
... <up to here everything normal, all md's correctly autodetected>...

* mounting sysfs at sys ...
can't create lock file /etc/mtab1008: Read only file-system
(use -n flag to override)                           [!!]

...<again everything normal up to>...

* checking root filesystem
ext2fs_check_if_mount: No such file or directory while determining
whether /dev/md1 is mounted
fsck.ext3: No such file or directory while trying to open /dev/md1
/dev/md1:
The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contain an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock: e2fsck -b 8193 <device>

* Filesystem could not be fixed :-(                 [!!]

Give root-password for maintenance
(or type Control-D for normal startup):
---------------------------------------------------------------------

Then I logged as root and tried that "e2fsck -b 8193 /dev/md1" with
the same error. Rebooting did not change anything...

BTW, I repeated the whole installation 3 times, even with different
pair of disks (I thought there is some hw-error), but no difference...
What now? Could someone tell me where the problem is?

Thanks,
	Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Kurt Guenther]

I did a similar setup, but I used Linux raid-1 rather then the software
raid.

If I remember right, I tried the software raid, but I think you made it
further then I did.  I couldn't get the software raid working under the
live CD.

Sorry, no help other then to recommend Linux raid instead of the
software raid.  Usually the performance is better (from what I hear)
anyways.

--Kurt



Jarry wrote:

> Hi,
>
> I'm trying to install gentoo on a small server in full sw-raid1 setup,
> using info in http://www.gentoo.org/doc/en/gentoo-x86-tipsntricks.xml
>
> I have a similar setup with /boot / and swap on md. After creating md's
> I continued with installation using info in
> http://www.gentoo.org/doc/en/handbook/2005.0/index.xml
>
> I recompilled kernel with md-support (no modules), installed boot-loader
> (lilo, on /dev/md0), created /etc/fstab, emerged mdadm, etc, etc...
> Everything was OK until 10.d: Rebooting the System
>
> When booting into new system (this time from disks, not from
> installation cd), I got following messages:
> ---------------------------------------------------------------------
> ... <up to here everything normal, all md's correctly autodetected>...
>
> * mounting sysfs at sys ...
> can't create lock file /etc/mtab1008: Read only file-system
> (use -n flag to override)                           [!!]
>
> ...<again everything normal up to>...
>
> * checking root filesystem
> ext2fs_check_if_mount: No such file or directory while determining
> whether /dev/md1 is mounted
> fsck.ext3: No such file or directory while trying to open /dev/md1
> /dev/md1:
> The superblock could not be read or does not describe a correct ext2
> filesystem. If the device is valid and it really contain an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate
> superblock: e2fsck -b 8193 <device>
>
> * Filesystem could not be fixed :-(                 [!!]
>
> Give root-password for maintenance
> (or type Control-D for normal startup):
> ---------------------------------------------------------------------
>
> Then I logged as root and tried that "e2fsck -b 8193 /dev/md1" with
> the same error. Rebooting did not change anything...
>
> BTW, I repeated the whole installation 3 times, even with different
> pair of disks (I thought there is some hw-error), but no difference...
> What now? Could someone tell me where the problem is?
>
> Thanks,
>     Jarry


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Jarry]

Kurt Guenther wrote:
> I did a similar setup, but I used Linux raid-1 rather then the software
> raid.

Pardon for my ignorance, but what do you mean with "Linux raid-1 rather
than the software raid"??? Is it not the same? Or do you mean some
hw-raid?

BTW, I'm already on the track of my problem:
After booting (actually not booting) I entered into that "maintenance
mode" and found out, that there are no /dev/md? files (although I
created them during installation)! Why??? Should not udev create them
automatically? Or did I misconfigured something concerning udev/devfs
in kernel?

At least it is clear now that error message of "ext2fs_check_if_mount"
and "fsck.ext3": there is no /dev/md1, so it is not possible check it...

Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[A. Khattri]

On Tue, 12 Jul 2005, Jarry wrote:

> Pardon for my ignorance, but what do you mean with "Linux raid-1 rather
> than the software raid"??? Is it not the same? Or do you mean some
> hw-raid?

Some motherboards claim to have RAID built-in but really it is software
RAID in the BIOS. Linux's software RAID is usually better though. I think
this is what was implied.

> BTW, I'm already on the track of my problem:
> After booting (actually not booting) I entered into that "maintenance
> mode" and found out, that there are no /dev/md? files (although I
> created them during installation)! Why??? Should not udev create them
> automatically? Or did I misconfigured something concerning udev/devfs
> in kernel?

Firstly, is RAID support built-in to your kernel? Also did you set the
partition types to "Linux raid autodetect" when running fdisk?


-- 

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Richard Fish]

A. Khattri wrote:

>On Tue, 12 Jul 2005, Jarry wrote:
>
>  
>
>>Pardon for my ignorance, but what do you mean with "Linux raid-1 rather
>>than the software raid"??? Is it not the same? Or do you mean some
>>hw-raid?
>>    
>>
>
>Some motherboards claim to have RAID built-in but really it is software
>RAID in the BIOS. Linux's software RAID is usually better though. I think
>this is what was implied.
>
>  
>
>>BTW, I'm already on the track of my problem:
>>After booting (actually not booting) I entered into that "maintenance
>>mode" and found out, that there are no /dev/md? files (although I
>>created them during installation)! Why??? Should not udev create them
>>automatically? Or did I misconfigured something concerning udev/devfs
>>in kernel?
>>    
>>
>
>Firstly, is RAID support built-in to your kernel? Also did you set the
>partition types to "Linux raid autodetect" when running fdisk?
>
>
>  
>

Just an FYI, the md driver does not create sysfs entries and thus udev
does not create device nodes if autodetection is not performed by the
driver.  Autodetection is not performed if either:

1. The md driver is a module (not a problem according to your original
email)
2. You are using an initramfs (vs an initrd) to bring up the system
3. or if you do not have the right partition types defined.

-Richard

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Jarry]

A. Khattri wrote:

> Some motherboards claim to have RAID built-in but really it is software
> RAID in the BIOS.

OK, now I understand. I call it "pseudo-hw-raid". It still depends on
software driver and cpu.

> Linux's software RAID is usually better though.

I agree. And that is what I'm trying to use, linux software-raid built
in kernel...

> Firstly, is RAID support built-in to your kernel? Also did you set the
> partition types to "Linux raid autodetect" when running fdisk?

Yes, of course. All partitions are marked as "auto-detect" (fd), and
raid support is in my kernel (not as module). As I wrote previously,
all md's are autodetected without any problem:

--------------------------------------------------------
md: Autodetecting RAID arrays.
md: autorun...
...
md:  adding hdg1
md:  adding hde1
md: created md0
md: bind<hde1>
md: bind<hdg1>
md: running: <hde1> <hdg1>
raid1: raid set md0 active running 2 out of 2 mirrors
md: ... autorun DONE
<similar for all md's>
---------------------------------------------------------


Richard Fish wrote:

 > Just an FYI, the md driver does not create sysfs entries and thus udev
 > does not create device nodes if autodetection is not performed by the
 > driver.  Autodetection is not performed if either:
 >
 > 1. The md driver is a module (not a problem according to your original
 > email)
 > 2. You are using an initramfs (vs an initrd) to bring up the system
 > 3. or if you do not have the right partition types defined.

I think I can unselect options 1. and 3. (md-support is in kernel, and
partitions are marked as type "fd - Linux raid autodetect").

Concerning 2, I'm not sure if I understand correctly:
Do you mean turning off initramfs support in kernel-configuration?

Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Richard Fish]

Jarry wrote:

> Richard Fish wrote:
>
> > Just an FYI, the md driver does not create sysfs entries and thus udev
> > does not create device nodes if autodetection is not performed by the
> > driver.  Autodetection is not performed if either:
> >
> > 1. The md driver is a module (not a problem according to your original
> > email)
> > 2. You are using an initramfs (vs an initrd) to bring up the system
> > 3. or if you do not have the right partition types defined.
>
> I think I can unselect options 1. and 3. (md-support is in kernel, and
> partitions are marked as type "fd - Linux raid autodetect").
>
> Concerning 2, I'm not sure if I understand correctly:
> Do you mean turning off initramfs support in kernel-configuration?
>
> Jarry


Actually, it is pretty much impossible to turn off initramfs support in
the kernel.  A couple of quick definitions:

1. initrd - an initial ramdisk of a fixed size loaded into memory by the
boot loader used to get the system ready to mount the root filesystem. 
The /linuxrc script in the ramdisk is responsible for this, and requires
"RAM disk support" and "Initial RAM disk (initrd) support" in the kernel
configuration.

2. initramfs - A compressed (optional) cpio archive linked directly into
the kernel, or possibly loaded into memory by the boot loader, used to
provide early-userspace services.  If the initramfs contains /init, it
will be executed and is expected to mount the root filesystem, move the
system root to it, and execute the real init on the root.  If the user
doesn't specify an initramfs, the kernel makes a very small one of its
own, without the /init.  This is the "rootfs" you see in /proc/mounts.

#1 you are probably already familiar with...it is the traditional way
for loading modules needed for the root filesystem to become available.

#2 is fairly recent...but growing in usage.  fbsplash uses
early-userspace to provide a splash image as soon as the graphics system
is initialized in the kernel, for example.  Unless you have specified
something for "Device Drivers->Block Devices->Initramfs source file" in
the kernel configuration, or your "initrd" option to grub/lilo specifies
an initramfs, this is not a problem for you.

In short, if you don't really know what initramfs is, you are probably
not using it!  So I am not sure why you are having this problem.  Could
you double check that /etc/udev/rules.d/50-udev.rules contains:

# md block devices
KERNEL=="md[0-9]*",     NAME="md/%n", SYMLINK+="%k", GROUP="disk"

Also, do you have any custom rules files in /etc/udev/rules.d?

In the maintenance mode, does /sys/block/md0/* exist?  What does "cat
/proc/mdstat" report?

-Richard


-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Jarry]

Richard Fish wrote:

> In short, if you don't really know what initramfs is, you are probably
> not using it! So I am not sure why you are having this problem.

Neither am I. I already installed a couple of servers with full raid1
using various distros (RedHat, Caldera, Debian), now I'm trying
Gentoo and I can not get past the first booting...   :-(

> Could you double check that /etc/udev/rules.d/50-udev.rules contains:
> # md block devices
> KERNEL=="md[0-9]*",     NAME="md/%n", SYMLINK+="%k", GROUP="disk"

Yes, I do have it there...

> Also, do you have any custom rules files in /etc/udev/rules.d?

No custom rules. Did not have time to make them, you now, my system
is not booting at all...

> In the maintenance mode, does /sys/block/md0/* exist?  What does "cat
> /proc/mdstat" report?

mdstat does not report anything. No /dev/md* exist, so no /dev/md*
is running...

BTW, on gentoo-forum I got answer with link pointing to debian list:

http://lists.debian.org/debian-kernel/2005/02/msg00253.html

It seems (or at least it is discussed there) that this (udev does not
create /dev/md* at startup) is some kernel-bug! If some of our kernel
developers is watching this list, could he confirm or refuse it?
Can I somehow get rid of udev, when it is causing problems to me?

In the meantime I'm trying to update my system to 2.6.12-r4
(up to now I used 2.6.11-r3 from 2005.0 universal installation cd)...

Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Mike Williams]

On Wednesday 13 July 2005 15:09, Jarry wrote:
> It seems (or at least it is discussed there) that this (udev does not
> create /dev/md* at startup) is some kernel-bug! If some of our kernel
> developers is watching this list, could he confirm or refuse it?
> Can I somehow get rid of udev, when it is causing problems to me?

http://article.gmane.org/gmane.linux.gentoo.devel/29788

-- 
Mike Williams
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Richard Fish]

Mike Williams wrote:

>On Wednesday 13 July 2005 15:09, Jarry wrote:
>  
>
>>It seems (or at least it is discussed there) that this (udev does not
>>create /dev/md* at startup) is some kernel-bug! If some of our kernel
>>developers is watching this list, could he confirm or refuse it?
>>Can I somehow get rid of udev, when it is causing problems to me?
>>    
>>
>
>http://article.gmane.org/gmane.linux.gentoo.devel/29788
>
>  
>

Just to expand a bit on the linked thread:

Gentoo now has support for creating the device nodes at boot time, in
the stable baselayout, which should be 1.11.12-r4 at this point.

To enable it, you need:

1. A proper /etc/raidtab or /etc/mdadm.conf (mdadm.conf preferred)
2. The mdadm tools emerged.
3. Add "md" to RC_VOLUME_ORDER in /etc/conf.d/rc.

The above should trigger then execution of
/lib/rcscripts/addons/raid-start.sh during startup which will create the
device nodes for you.

I did not know any of this before I posted regarding the RAID device
nodes in the linked thread...

-Richard

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting - SOLVED !

[Jarry]

> In the meantime I'm trying to update my system to 2.6.12-r4
> (up to now I used 2.6.11-r3 from 2005.0 universal installation cd)...

Actually, I did not updated the system, but installed once more.
This time as a "network installation", with latest portage and
stage1 downloaded from some mirror-server (previously I installed
without network, portage and stage3 from gentoo-2005.0 universal CD).

And (to my surprise!) everything works without a problem. Halleluyah!
And I even used the same kernel-config from 2.6.11-r3 for 2.6.12-r4
(was too lazy to go through the whole "make menuconfig" again)...

Jarry

-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting - SOLVED !

[Richard Fish]

And this will teach me to scan all messages in a thread before I start
replying!!!

Glad you got it working.  If the kernel config is truly the same, then I
suspect you got an updated version of baselayout or udev that fixed it.

-Richard

Jarry wrote:

>
>> In the meantime I'm trying to update my system to 2.6.12-r4
>> (up to now I used 2.6.11-r3 from 2005.0 universal installation cd)...
>
>
> Actually, I did not updated the system, but installed once more.
> This time as a "network installation", with latest portage and
> stage1 downloaded from some mirror-server (previously I installed
> without network, portage and stage3 from gentoo-2005.0 universal CD).
>
> And (to my surprise!) everything works without a problem. Halleluyah!
> And I even used the same kernel-config from 2.6.11-r3 for 2.6.12-r4
> (was too lazy to go through the whole "make menuconfig" again)...
>
> Jarry
>
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting - SOLVED !

[Jarry]

Richard Fish wrote:

 > Glad you got it working.  If the kernel config is truly the same,
 > then I suspect you got an updated version of baselayout or udev
 > that fixed it.

Yes, you are right. Baselayout which is on 2005.0 install-cd must
have some bug, which I eliminated during network-install by updating
baselayout scripts (as advised in manual) before finishing installation
and rebooting. And one more user on forum reported the same - with
baselayout from 2005.0 universal-cd udev doesn't create /dev/md* ...

Jarry
-- 
gentoo-user@gentoo.org mailing list

Re: [gentoo-user] problem with raid1: error while booting

[Richard Fish]

Jarry wrote:

>> In the maintenance mode, does /sys/block/md0/* exist?  What does "cat
>> /proc/mdstat" report?
>
>

Actually, if the kernel autodetection runs at boot time, /proc/mdstat
should show the devices regardless of whether the device nodes exist or
not....so something is a bit strange with your kernel configuration.

Either you are missing a RAID driver in the kernel, or a necessary disk
driver.  Take a careful look at what "grep =m /usr/src/linux/.config"
reports...something in there is almost certainly the problem!

You can also follow my notes about creating an mdadm.conf file in
another message (sorry for the multiple replies!).

> It seems (or at least it is discussed there) that this (udev does not
> create /dev/md* at startup) is some kernel-bug! If some of our kernel
> developers is watching this list, could he confirm or refuse it?


I don't know if I would call it a kernel bug.  It is mostly just
functionality that isn't clear where it belongs, and there is no obvious
right thing to do.  The autodetection feature of the MD driver came from
a different era of kernel development, and would almost certainly not be
accepted today.


> Can I somehow get rid of udev, when it is causing problems to me?


Well, you can always revert back to static device nodes.  See the
RC_DEVICES entry in /etc/conf.d/rc.  If you set this, you will need to
run a bunch of /sbin/MAKEDEV commands to get the static nodes created.

If you do decide to go this route, I encourage you to consider it a
temporary solution until you get the system up and running. 

-Richard

-- 
gentoo-user@gentoo.org mailing list