[gentoo-amd64] Re: Root on Raid and LVM - Solved

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Edward Middleton]

>Only one issue now, and that is that upon shutdown, Gentoo cannot stop
>/dev/md5 because it is in use.
>Is there a fix for this? Perhaps in the order of the shutdown?

hi Chris,

Did you work out how to fix this.  I am having exactly the same
problem.  I understand why it is happening (the root partition is still
in use), but presumably there is some way of getting around this.

Edward

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Duncan]

Edward Middleton posted <4365CBC8.2080502@nikon-sys.co.jp>, excerpted
below,  on Mon, 31 Oct 2005 16:46:16 +0900:

>>Only one issue now, and that is that upon shutdown, Gentoo cannot stop
>>/dev/md5 because it is in use.
>>Is there a fix for this? Perhaps in the order of the shutdown?
> 
> hi Chris,
> 
> Did you work out how to fix this.  I am having exactly the same
> problem.  I understand why it is happening (the root partition is still
> in use), but presumably there is some way of getting around this.

Well, normally the root partition /will/ still be in use, so can't be
unmounted.  However, it's then mounted read-only, instead, forcing out all
cached writes to bring the fs into a consistent state and then allowing no
more writes to it after it's mounted read-only.  Thus, the system can
still shut down even with it mounted.

The thing is, that's expected for your root partition, so it's normally
included in the init scripts to mount it read-only instead of unmounting
it, and for that one, it normally doesn't cause a warning, whereas for
others, it would (and should, in the general case, anyway).

The problem here is that for some reason the system apparently isn't
recognizing that partition as root, so it's still warning when it can't be
unmounted.  As mentioned, it could be an order thing, or it could be that
particular raid or lvm config isn't setup correctly in the initscripts, so
it's not recognizing them as devices containing filesystems, at all.

In a few days, I'll have mine setup and be playing around with it, so may
or may not be able to confirm the issue then, but I expect to be able to
trace it if I do have it.  (Note that I'm running ~amd64 and the
1.12.0-rc's of baselayout.  I don't expect I'd be reverting to stable, if
the issue's happening there.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Edward Middleton]

Duncan wrote

>>Well, normally the root partition /will/ still be in use, so can't be
>>unmounted.  However, it's then mounted read-only, instead, forcing out all
>>cached writes to bring the fs into a consistent state and then allowing no
>>more writes to it after it's mounted read-only.  Thus, the system can
>>still shut down even with it mounted.
>  
>

I gather you mean in this situation the raid doesn't need to be deactivated.


>>The problem here is that for some reason the system apparently isn't
>>recognizing that partition as root, so it's still warning when it can't be
>>unmounted.  As mentioned, it could be an order thing, or it could be that
>>particular raid or lvm config isn't setup correctly in the initscripts, so
>>it's not recognizing them as devices containing filesystems, at all.
>  
>

Going on what you said before the /etc/conf.d/rc variable should be set to RC_VOLUME_ORDER="lvm" if you are using raidlvm.  There is a bug in the /lib/rcscripts/addons/lvm-stop.sh which assumes that mount returns the symbolic link /dev/[volume group]/[logical partition] but it actually returns the actual device /dev/mapper/[volume group]-[logical partition].  The following path patch fixes this.

--- /lib/rcscripts/addons/lvm-stop.sh.orig    2005-11-01 19:07:25.000000000 +0900
+++ /lib/rcscripts/addons/lvm-stop.sh    2005-11-01 19:48:38.000000000 +0900
@@ -46,6 +46,8 @@
         then
            
             ROOT_DEVICE=`mount|grep " / "|awk '{print $1}'`
+            [ -L ${ROOT_DEVICE} ] && ROOT_DEVICE="`readlink ${ROOT_DEVICE}`"
+            [ -L ${x} ] && x="`readlink ${x}`"
             if [ ! ${ROOT_DEVICE} = ${x} ]
             then
                 ewarn "  Unable to shutdown: ${x} "


Edward

-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Duncan]

Edward Middleton posted <4366CA93.3020400@nikon-sys.co.jp>, excerpted
below,  on Tue, 01 Nov 2005 10:53:23 +0900:

> Going on what you said before the /etc/conf.d/rc variable should be set
> to RC_VOLUME_ORDER="lvm" if you are using raidlvm.  There is a bug in
> the /lib/rcscripts/addons/lvm-stop.sh which assumes that mount returns
> the symbolic link /dev/[volume group]/[logical partition] but it
> actually returns the actual device /dev/mapper/[volume group]-[logical
> partition].  The following path patch fixes this.

[patch snipped]

Interesting, especially since I have lvm/dm merged in preparation for
going RAID later this week. Thanks!

You didn't mention what package that belonged to, or what version you have
merged.  However, I checked here and it looks as the patch says the
original does, and here's what I have:

$ equery b /lib/rcscripts/addons/lvm-stop.sh
[ Searching for file(s) /lib/rcscripts/addons/lvm-stop.sh in *... ]
sys-fs/lvm2-2.01.14 (/lib/rcscripts/addons/lvm-stop.sh)

So... that script is part of sys-fs/lvm2, version 2.01.14, here.

$cat /p/sys-fs/lvm2/lvm2-2.01.14.ebuild|grep amd64
KEYWORDS="~alpha ~amd64 ~hppa ~mips ~ppc ~ppc64 ~sparc ~x86"

So... that ebuild is keyworded ~amd64.  What version are you running,
stable (equery l -p says 2.01.09 is latest stable) or ~?  Have you filed a
bug on it with your patch included?

Investigating a bit more...  All versions in the tree install the same
scripts, as found in sys-fs/lvm2/files.  So, the bug would apply to all
versions in the tree.

Checking bugzilla...  I didn't see anything there that looked like this
bug, or filed by you (name or address), altho there were 118 bugs for lvm2
listed.

It appears most aren't running into this bug, as the closest bugs I see
relate to a missing xargs (xargs in /usr/bin, /usr on an lvm, so once it's
unmounted at system shutdown, xargs can't be found, so the command
using xargs in the shutdown script failed -- that set of bugs and dups
fixed), yet the number of bugs suggests there's a fair number of users.

However, this appears to depend on which /dev entry is listed in fstab to
be mounted.  Perhaps the handbook or some other Gentoo doc says to use the
one, so everybody is, and nobody has run into this assumption busting
situation yet.

Anyway, I'd certainly recommend filing the bug.  As for me personally,
I'll keep this in mind, since I'll be working with it in the next few
days.  You may have saved me a quite a bit of going nuts over this,
myself! =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Edward Middleton]

>You didn't mention what package that belonged to, or what version you have
>merged.  However, I checked here and it looks as the patch says the
>original does, and here's what I have:
The latest stable lvm2-2.01.09 but as you said latter they all use the
same script.

>It appears most aren't running into this bug,
>yet the number of bugs suggests there's a fair number of users.
This bug will only effect users of root on lvm2 users

>However, this appears to depend on which /dev entry is listed in fstab to
>be mounted.
Thats what I thought but it doesn't.  lvdisplay which is used to get the
list of drives allways seems to get the symbolic link(using
sys-fs/lvm2-2.01.09).  mount allways seems to get the actual device
(using sys-apps/util-linux-2.12r).

>Anyway, I'd certainly recommend filing the bug.
I added it to the end of this bug which seemed relivant.
http://bugs.gentoo.org/show_bug.cgi?id=57229
I also updated the patch so it should work with /usr folders on lvm2 and
error message will show link no actual device name

--- /lib/rcscripts/addons/lvm-stop.sh.orig      2005-11-01 19:07:25.000000000 +0900
+++ /lib/rcscripts/addons/lvm-stop.sh   2005-11-02 20:35:05.000000000 +0900
@@ -46,7 +46,10 @@
                then

                        ROOT_DEVICE=`mount|grep " / "|awk '{print $1}'`
-                       if [ ! ${ROOT_DEVICE} = ${x} ]
+                       MOUNTED_DEVICE=${x}
+                       [ -L ${ROOT_DEVICE} ] && ROOT_DEVICE="`/bin/readlink ${ROOT_DEVICE}`"
+                       [ -L ${x} ] && MOUNTED_DEVICE="`/bin/readlink ${x}`"
+                       if [ ! ${ROOT_DEVICE} = ${MOUNTED_DEVICE} ]
                        then
                                ewarn "  Unable to shutdown: ${x} "
                        fi


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Duncan]

Edward Middleton posted <4368286D.90909@nikon-sys.co.jp>, excerpted below,
 on Wed, 02 Nov 2005 11:46:05 +0900:

>>It appears most aren't running into this bug, yet the number of bugs
>>suggests there's a fair number of users.
> This bug will only effect users of root on lvm2 users

BTW, I'm not fully up and working yet, but I've decided to put root
directly on (a partitioned) raid6, so it can be directly auto-assembled by
the kernel and I won't have to use an initrd/initramfs, as (I believe??) I
would if root was on LVM2.  (The software raid howto is outdated when it
says Linux raid can't be partitioned.  The 2.6 kernel handles that just
fine, but with different device numbers than the traditional
non-partitionable raid.)

I got 4-300 gig Seagates -- 1.2 terabyte of raw disk space (plus my
existing PATA drive space).  One of the reasons I'm not up and running yet
is that I double-checked the raw drives by running badblocks -w on each
entire drive.  That's four patterns, written out one at a time to four
drives, 300 gig each, then read back in and verified.  4x4x300 gig = 4.8
terabytes written, 4.8 terabytes read.  That took me a couple days!  =8^)
Fortunately, I didn't feel it much at all, save for the wait, because that
was traffic on drives I'm not yet using in production, and while it did
run 1 CPU to 100% for the entire period, that was mostly I/O wait, and my
other CPU kept up with the normal desktop stuff just fine.  Anyway... it
was waiting for that, that I realized just how much drive space I now
have! =8^)

I have the raid6 partitioned three ways, my traditional root and
backup-root-snapshot as two partitions with the third being the big one --
to hold the rest of my raid6 volumes in LVM, again, for the additional
flexibility.  There's enough space on the LVM-on-raid6 partition to keep a
working and a snapshot copy of everything there, too, guarding against
fat-fingering as well as hard drive failure.  (A second copy of everything
means very little risk playing with reiser4 or the like, as well! =8^)

Right now, I have three raids set up (altho I haven't installed
filesystems on them yet), a small raid0 (mirrored) for booting, the
majority of the space as the three-way-partitioned raid6 mentioned above,
and a raid0 (striped for speed, non-redundant) for semi-volatile storage,
the portage tree as it can easily be resynced for recovery, /tmp, etc (no,
not /etc! <g>).  However, after reading up on LVM and realizing it had
built-in striping capabilities, I may kill the raid0 and simply use
striped LVM directly on the physical volume partitions underneath.

Anyway... it was quite cool to realize the 2.6 kernel handles partitioned
raid, now, and with direct boot-time kernel raid assembly, I could run
root on raid6 without an initrd/initramfs!  As I was dreading the
complexity of an initrd solution, I grabbed the opportunity to avoid it
and still have a raid-backed root, when I saw it!  So, no root on LVM
here!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Edward Middleton]

>Anyway... it was quite cool to realize the 2.6 kernel handles partitioned
>raid, now, and with direct boot-time kernel raid assembly, I could run
>root on raid6 without an initrd/initramfs!
That is pretty cool that you recognize a partitioned raid at boot.

>As I was dreading the
>complexity of an initrd solution, I grabbed the opportunity to avoid it
>and still have a raid-backed root, when I saw it!

The initrd part of setup is easy if you use genkernel.  The only issue I
had was using an out of kernel driver but that can be handled by the
--callback="emerge driver name" and adding the driver to
/usr/share/genkernel/x86_64/modules_load
You will also need to set lvmraid=[raid device] real_root=[lvm device]
on your kernel line in /boot/grub/menu.lst

> So, no root on LVM here!
The reason for lvm root is that you can resize the root partition.  It
is also trivial to setup a secondary rescue root partition in case you
mess up your system.

Edward
-- 
gentoo-amd64@gentoo.org mailing list

[gentoo-amd64] Re: Root on Raid and LVM - Solved

[Duncan]

Edward Middleton posted <4372AD03.8080400@nikon-sys.co.jp>, excerpted
below,  on Thu, 10 Nov 2005 11:14:27 +0900:

>>Anyway... it was quite cool to realize the 2.6 kernel handles
>>partitioned raid, now, and with direct boot-time kernel raid assembly, I
>>could run root on raid6 without an initrd/initramfs!
> That is pretty cool that you recognize a partitioned raid at boot.
> 
>> So, no root on LVM here!
> The reason for lvm root is that you can resize the root partition.  It
> is also trivial to setup a secondary rescue root partition in case you
> mess up your system.

OK, not like anyone's probably interested in this, but I worked hard the
last few weeks and need somewhere to brag a bit about the result! =8^)

FWIW, the raid system is 100% up and running, including switching from
LILO to GRUB and learning a /ton/ about raid/lvm/grub all three!  I'm only
using lvm on my non-root raid6 partition, and as explained, the kernel
handles the raid on its own, so no need for an initrd/initramfs!  Yeah!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html


-- 
gentoo-amd64@gentoo.org mailing list