[gentoo-user] fsck check of /usr on a separate partition fails during boot

[gentoo-user] fsck check of /usr on a separate partition fails during boot

[Andrew Barchuk]

Hi folks,

I've posted about this problem to the forums[1] without luck despite
getting more than a thousand views so I thought I'll try here.

My system boots successfully but filesystem check fails for /usr which
is on a separate partition:

 * Checking local filesystems  ...
/dev/mapper/MacVg-gentoo--root: clean, 2390/65536 files, 30938/262144 blocks
/dev/mapper/MacVg-gentoo--usr is mounted.
e2fsck: Cannot continue, aborting.


/dev/mapper/MacVg-gentoo--var: clean, 22647/65536 files, 59083/262144 blocks
/dev/mapper/MacVg-gentoo--home: clean, 8080/917504 files, 243397/3670016 blocks
/dev/mapper/MacVg-data: clean, 5293/3145728 files, 8945157/12582912 blocks
 * Operational error
 [ !! ] 

I use LVM on LUKS container for my partitions and an initramfs built
with genkernel.

My fstab:

/dev/MacVg/gentoo-root  /       ext4    defaults                0 1
/dev/MacVg/gentoo-usr   /usr    ext4    defaults                0 2
/dev/MacVg/gentoo-var   /var    ext4    nodev,nosuid            0 2
/dev/MacVg/gentoo-home  /home   ext4    nodev,nosuid            0 2
/dev/MacVg/data         /data   ext4    nodev,nosuid,noexec     0 2
LABEL=EFI               /boot   vfat    noauto,umask=0022       0 2
/dev/MacVg/swap         none    swap    defaults                0 0
tmpfs                   /tmp    tmpfs   nosuid,nodev,size=1G,mode=1777 0 0
tmpfs                   /var/tmp tmpfs  nosuid,nodev,size=8G,mode=1777 0 0 

Any ideas what is going on and how do I make the fsck check succeed?
Maybe I should file it as an OpenRC bug but I'm not completely sure if
it's not me doing something wrong.
Thanks in advance for any help.

1. https://forums.gentoo.org/viewtopic-t-1075174-highlight-.html

---
Andrew

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[John Covici]

On Sat, 13 Jan 2018 05:58:43 -0500,
Andrew Barchuk wrote:
> 
> Hi folks,
> 
> I've posted about this problem to the forums[1] without luck despite
> getting more than a thousand views so I thought I'll try here.
> 
> My system boots successfully but filesystem check fails for /usr which
> is on a separate partition:
> 
>  * Checking local filesystems  ...
> /dev/mapper/MacVg-gentoo--root: clean, 2390/65536 files, 30938/262144 blocks
> /dev/mapper/MacVg-gentoo--usr is mounted.
> e2fsck: Cannot continue, aborting.
> 
> 
> /dev/mapper/MacVg-gentoo--var: clean, 22647/65536 files, 59083/262144 blocks
> /dev/mapper/MacVg-gentoo--home: clean, 8080/917504 files, 243397/3670016 blocks
> /dev/mapper/MacVg-data: clean, 5293/3145728 files, 8945157/12582912 blocks
>  * Operational error
>  [ !! ] 
> 
> I use LVM on LUKS container for my partitions and an initramfs built
> with genkernel.
> 
> My fstab:
> 
> /dev/MacVg/gentoo-root  /       ext4    defaults                0 1
> /dev/MacVg/gentoo-usr   /usr    ext4    defaults                0 2
> /dev/MacVg/gentoo-var   /var    ext4    nodev,nosuid            0 2
> /dev/MacVg/gentoo-home  /home   ext4    nodev,nosuid            0 2
> /dev/MacVg/data         /data   ext4    nodev,nosuid,noexec     0 2
> LABEL=EFI               /boot   vfat    noauto,umask=0022       0 2
> /dev/MacVg/swap         none    swap    defaults                0 0
> tmpfs                   /tmp    tmpfs   nosuid,nodev,size=1G,mode=1777 0 0
> tmpfs                   /var/tmp tmpfs  nosuid,nodev,size=8G,mode=1777 0 0 
> 
> Any ideas what is going on and how do I make the fsck check succeed?
> Maybe I should file it as an OpenRC bug but I'm not completely sure if
> it's not me doing something wrong.
> Thanks in advance for any help.
> 
> 1. https://forums.gentoo.org/viewtopic-t-1075174-highlight-.html
> 

I bet you are using genkernel or gentoo-next to generate your initrd.
If so, its because the script they have mounts the /usr at the wrong
time, and you cannot do e2fsck to a mounted disk.  You might have
better luck using dracut -- be sure to have the use flag of -systemd
if you are using openrc.

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici wb2una
         covici@ccs.covici.com

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Alan McKinnon]

On 13/01/2018 12:58, Andrew Barchuk wrote:
> Hi folks,
> 
> I've posted about this problem to the forums[1] without luck despite
> getting more than a thousand views so I thought I'll try here.
> 
> My system boots successfully but filesystem check fails for /usr which
> is on a separate partition:
> 
>  * Checking local filesystems  ...
> /dev/mapper/MacVg-gentoo--root: clean, 2390/65536 files, 30938/262144 blocks
> /dev/mapper/MacVg-gentoo--usr is mounted.
> e2fsck: Cannot continue, aborting.
> 
> 
> /dev/mapper/MacVg-gentoo--var: clean, 22647/65536 files, 59083/262144 blocks
> /dev/mapper/MacVg-gentoo--home: clean, 8080/917504 files, 243397/3670016 blocks
> /dev/mapper/MacVg-data: clean, 5293/3145728 files, 8945157/12582912 blocks
>  * Operational error
>  [ !! ] 
> 
> I use LVM on LUKS container for my partitions and an initramfs built
> with genkernel.
> 
> My fstab:
> 
> /dev/MacVg/gentoo-root  /       ext4    defaults                0 1
> /dev/MacVg/gentoo-usr   /usr    ext4    defaults                0 2
> /dev/MacVg/gentoo-var   /var    ext4    nodev,nosuid            0 2
> /dev/MacVg/gentoo-home  /home   ext4    nodev,nosuid            0 2
> /dev/MacVg/data         /data   ext4    nodev,nosuid,noexec     0 2
> LABEL=EFI               /boot   vfat    noauto,umask=0022       0 2
> /dev/MacVg/swap         none    swap    defaults                0 0
> tmpfs                   /tmp    tmpfs   nosuid,nodev,size=1G,mode=1777 0 0
> tmpfs                   /var/tmp tmpfs  nosuid,nodev,size=8G,mode=1777 0 0 
> 
> Any ideas what is going on and how do I make the fsck check succeed?
> Maybe I should file it as an OpenRC bug but I'm not completely sure if
> it's not me doing something wrong.
> Thanks in advance for any help.
> 
> 1. https://forums.gentoo.org/viewtopic-t-1075174-highlight-.html
> 
> ---
> Andrew
> 


By far the easiest way to deal with this without having to predict if
maybe /usr is mounted or not, or if maybe your intiramfs has the correct
files in place and all sorts of other maybes, is the following:

- find any old LiveCD/installer/whatever on CD or thumb drive (the
gentoo minimal install CD works just fine, so does ubuntu-server
installer (it boots quite quickly)
- set your BIOS to boot from that device
- reboot
- use the fsck tool on that system (which is independent of your main
system) to fix the broken fs for /usr
- reboot as normal

Yes, you *could* fiddle with your initramfs to provide a shell and fs
tools. How often are you going to use it or test it? As you are not
RedHat with paying customers, I'd say "almost never". so rescue disk ftw
-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Floyd Anderson]

On Sat, 13 Jan 2018 11:58:43 +0100
Andrew Barchuk <andrew@raindev.io> wrote:
>Hi folks,
>
[…]
>
>Any ideas what is going on and how do I make the fsck check succeed?

It seems the init script(s) within your initramfs implements no 
logic/hooks for fsck but just mount your /usr partition. After switching 
to real root your OpenRC fsck init script try to check /usr partition 
due to the specified 2 for passno (sixth field in /usr fstab entry) and 
fails because it is already mounted.

You have to implement your own logic in initramfs init script(s) and set 
passno to 0 for /usr partition to prevent the OpenRC fsck. Have a look 
at [1] as a starting point. Also check if /etc/initramfs.mounts exists 
(comes with genkernel-next and probably with other packages, I am not 
sure at this) and what is defined in there.

The steps to realise a fsck check hook within an initramfs init script 
may not be trivial and time consuming on building/testing, so, as Alan 
suggested earlier, checking your partition with a booted rescue live CD 
or similar, may prevent some headaches.


References:
  - [1] <https://wiki.gentoo.org/wiki/Early_Userspace_Mounting>



-- 
Regards,
floyd

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Andrew Barchuk]

John,

> I bet you are using genkernel or gentoo-next to generate your initrd.

Exactly. Probably got lost in between the file contents:

> I use LVM on LUKS container for my partitions and an initramfs built
with genkernel.

> You might have better luck using Dracut

Thank you for the suggestion, I've tried Dracut and it got me almost
there. First I've tried to add 'ro' command line option to mount both
root and /usr as read-only and allow fsck OpenRC service to check the
filesystems but it wasn't sufficient: I got the same error from e2fsck
complaining that /usr is mounted.

As Dracut implements fsck in the initramfs I've decided to leverage it
and disabled fsck for both root and /usr in fstab. Now OpenRC fsck
succeeds. But if I'm reading the logs correctly Dracut is not checking
/usr, only root (199bb83d-c783-4254-a6eb-fdbb83c33144):

Jan 13 19:55:24 machine kernel: <30>[    2.611986] dracut: luksOpen /dev/sdb3 luks-2acb7668-fff1-492d-b46e-f05ead26d153
Jan 13 19:55:24 machine kernel: [    7.318082] random: crng init done
Jan 13 19:55:24 machine kernel: <30>[   30.842143] dracut: Scanning devices dm-0  for LVM logical volumes MacVg/swap MacVg/gentoo-root MacVg/gentoo-usr
Jan 13 19:55:24 machine kernel: <30>[   30.856028] dracut: inactive '/dev/MacVg/arch-root' [10.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.858497] dracut: inactive '/dev/MacVg/arch-var' [6.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.860752] dracut: inactive '/dev/MacVg/swap' [8.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.862977] dracut: inactive '/dev/MacVg/arch-home' [6.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.865294] dracut: inactive '/dev/MacVg/data' [48.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.867488] dracut: inactive '/dev/MacVg/gentoo-root' [1.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.869669] dracut: inactive '/dev/MacVg/gentoo-var' [1.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.871936] dracut: inactive '/dev/MacVg/gentoo-home' [2.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.874062] dracut: inactive '/dev/MacVg/gentoo-usr' [12.00 GiB] inherit
Jan 13 19:55:24 machine kernel: <30>[   30.961553] dracut: Scanning devices dm-0  for LVM volume groups MacVg
Jan 13 19:55:24 machine kernel: <30>[   30.967503] dracut: Reading all physical volumes. This may take a while...
Jan 13 19:55:24 machine kernel: <30>[   30.975165] dracut: Found volume group "MacVg" using metadata type lvm2
Jan 13 19:55:24 machine kernel: <30>[   31.022883] dracut: 9 logical volume(s) in volume group "MacVg" now active
Jan 13 19:55:24 machine kernel: [   31.031599] PM: Starting manual resume from disk
Jan 13 19:55:24 machine kernel: [   31.034193] PM: Image not found (code -22)
Jan 13 19:55:24 machine kernel: [   31.045037] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: data=ordered
Jan 13 19:55:24 machine kernel: <30>[   31.079313] dracut: Checking ext4: /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144
Jan 13 19:55:24 machine kernel: <30>[   31.083597] dracut: issuing e2fsck -a  /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144
Jan 13 19:55:24 machine kernel: <30>[   31.099671] dracut: /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144: clean, 2477/65536 files, 33002/262144 blocks
Jan 13 19:55:24 machine kernel: <30>[   31.106003] dracut: Mounting /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144 with -o rw,relatime,data=ordered
Jan 13 19:55:24 machine kernel: [   31.116662] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: data=ordered
Jan 13 19:55:24 machine kernel: <30>[   31.124793] dracut: Mounted root filesystem /dev/mapper/MacVg-gentoo--root
Jan 13 19:55:24 machine kernel: <30>[   31.136286] dracut: Mounting /usr with -o defaults
Jan 13 19:55:24 machine kernel: [   31.518944] EXT4-fs (dm-3): recovery complete
Jan 13 19:55:24 machine kernel: [   31.524685] EXT4-fs (dm-3): mounted filesystem with ordered data mode. Opts: (null)
Jan 13 19:55:24 machine kernel: <30>[   31.568023] dracut: Switching root

My Dracut kernel command line options:

rd.vconsole.keymap=workman rd.vconsole.font=ter-132n root=UUID=199bb83d-c783-4254-a6eb-fdbb83c33144 rd.luks.uuid=2acb7668-fff1-492d-b46e-f05ead26d153 rd.lvm.vg=MacVg

And /etc/dracut.conf:

hostonly="yes"

Any suggestions why Dracut is not performing /usr filesystem check as it
is supposed to accordingly to the man pages?

---
Andrew

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Alan McKinnon]

On 13/01/2018 21:30, Andrew Barchuk wrote:
> Alan, Floyd,
> 
> Thanks for your responses.
> Indeed I prefer to not maintain my own initramfs scripts. Right now I
> use genkernel initramfs but it seems to not be doing the right thing
> regarding /usr partition mounting (as I understand now it's not a
> problem with OpenRC fsck service). On the other hand I prefer to not
> have to remember to run fsck manually, I'm pretty bad at such things.
> 
> ---
> Andrew
> 

fwiw, fsck here runs automagically at startup whenever the fs is dirty,
and I do not use an initramfs at all. Not sure exactly what code does
this, I assume it's something in OpenRC.

-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Andrew Barchuk]

Alan, Floyd,

Thanks for your responses.
Indeed I prefer to not maintain my own initramfs scripts. Right now I
use genkernel initramfs but it seems to not be doing the right thing
regarding /usr partition mounting (as I understand now it's not a
problem with OpenRC fsck service). On the other hand I prefer to not
have to remember to run fsck manually, I'm pretty bad at such things.

---
Andrew

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Andrew Barchuk]

Alan,

> Not sure exactly what code does this, I assume it's something in OpenRC.

It's OpenRC service fsck that performs filesystem checks on boot runlevel
(/etc/init.d/fsck):

$ rc-status boot | grep fsck
 fsck                                                              [  started  ]

---
Andrew

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 511 bytes --]

On Sat, 13 Jan 2018 21:29:12 +0200, Alan McKinnon wrote:

> fwiw, fsck here runs automagically at startup whenever the fs is dirty,
> and I do not use an initramfs at all. Not sure exactly what code does
> this, I assume it's something in OpenRC.

It is, and the reason it works is that you do not use an initramfs that
mounts /usr before openrc gets a look in. If you use an initramfs, that
should take care of running fsck when needed.


-- 
Neil Bothwick

Puns are bad, but poetry is verse...

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[John Johnson]

[-- Attachment #1: Type: text/plain, Size: 794 bytes --]

Shouldn't that be taken care of by the "/etc/fstab" entries?
Obviously, if "/usr" is on a separate partition, it needs to be mounted at
the time when "/usr/sbin/fsck" is expected to be present.


On Sat, Jan 13, 2018 at 2:54 PM, Neil Bothwick <neil@digimed.co.uk> wrote:

> On Sat, 13 Jan 2018 21:29:12 +0200, Alan McKinnon wrote:
>
> > fwiw, fsck here runs automagically at startup whenever the fs is dirty,
> > and I do not use an initramfs at all. Not sure exactly what code does
> > this, I assume it's something in OpenRC.
>
> It is, and the reason it works is that you do not use an initramfs that
> mounts /usr before openrc gets a look in. If you use an initramfs, that
> should take care of running fsck when needed.
>
>
> --
> Neil Bothwick
>
> Puns are bad, but poetry is verse...
>

[-- Attachment #2: Type: text/html, Size: 1393 bytes --]

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1362 bytes --]

On Sat, 13 Jan 2018 14:57:59 -0600, John Johnson wrote:

> Shouldn't that be taken care of by the "/etc/fstab" entries?

Those say whether the filesystem should be checked, not when.

> Obviously, if "/usr" is on a separate partition, it needs to be mounted
> at the time when "/usr/sbin/fsck" is expected to be present.

fsck is in /sbin, but that's not the point. If you have an initramfs,
fsck should be in it and run before /usr is mounted rw, which means it
has to be done by the initramfs. It's too late to do it when control has
been handed over because then /usr is already mounted rw.
 
> 
> On Sat, Jan 13, 2018 at 2:54 PM, Neil Bothwick <neil@digimed.co.uk>
> wrote:
> 
> > On Sat, 13 Jan 2018 21:29:12 +0200, Alan McKinnon wrote:
> >  
> > > fwiw, fsck here runs automagically at startup whenever the fs is
> > > dirty, and I do not use an initramfs at all. Not sure exactly what
> > > code does this, I assume it's something in OpenRC.  
> >
> > It is, and the reason it works is that you do not use an initramfs
> > that mounts /usr before openrc gets a look in. If you use an
> > initramfs, that should take care of running fsck when needed.
> >
> >
> > --
> > Neil Bothwick
> >
> > Puns are bad, but poetry is verse...
> >  




-- 
Neil Bothwick

If a book about failures doesn't sell, is it a success?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Alan McKinnon]

On 13/01/2018 23:16, Neil Bothwick wrote:
> On Sat, 13 Jan 2018 14:57:59 -0600, John Johnson wrote:
> 
>> Shouldn't that be taken care of by the "/etc/fstab" entries?
> 
> Those say whether the filesystem should be checked, not when.
> 
>> Obviously, if "/usr" is on a separate partition, it needs to be mounted
>> at the time when "/usr/sbin/fsck" is expected to be present.
> 
> fsck is in /sbin, but that's not the point. If you have an initramfs,
> fsck should be in it and run before /usr is mounted rw, which means it
> has to be done by the initramfs. It's too late to do it when control has
> been handed over because then /usr is already mounted rw.


So what does the dirty check and fsck of / ?

I don't have an initramfs, I don't have a separate /usr, I run OpenRC
and the kernel command line says where / is for mounting


-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Dale]

Alan McKinnon wrote:
> On 13/01/2018 23:16, Neil Bothwick wrote:
>> On Sat, 13 Jan 2018 14:57:59 -0600, John Johnson wrote:
>>
>>> Shouldn't that be taken care of by the "/etc/fstab" entries?
>> Those say whether the filesystem should be checked, not when.
>>
>>> Obviously, if "/usr" is on a separate partition, it needs to be mounted
>>> at the time when "/usr/sbin/fsck" is expected to be present.
>> fsck is in /sbin, but that's not the point. If you have an initramfs,
>> fsck should be in it and run before /usr is mounted rw, which means it
>> has to be done by the initramfs. It's too late to do it when control has
>> been handed over because then /usr is already mounted rw.
>
> So what does the dirty check and fsck of / ?
>
> I don't have an initramfs, I don't have a separate /usr, I run OpenRC
> and the kernel command line says where / is for mounting
>
>


I think without a init thingy, it mounts / ro at first, runs the checks
and then remounts rw.  I think it does the same with /usr.  I'm not sure
what decides that tho. 

The last I rebooted, the checks are done within the init thingy for /
and /usr, while mounted ro of course.  Once / and /usr are in the clear,
it swaps from the init thingy and the normal kernel/OS boot starts.  I
think it checks /home after the init thingy is gone.  I think.  It's
been a while since I rebooted.  167 days so far.  The power company is
doing a good job of keeping our power going. 

If I ever redo my setup, /usr will be on / and hopefully no init
thingy.  With drives and file systems like they are now, it's just not
worth the trouble. 

Dale

:-)  :-) 

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1566 bytes --]

On Sat, 13 Jan 2018 23:16:19 +0200, Alan McKinnon wrote:

> On 13/01/2018 23:16, Neil Bothwick wrote:
> > On Sat, 13 Jan 2018 14:57:59 -0600, John Johnson wrote:
> >   
> >> Shouldn't that be taken care of by the "/etc/fstab" entries?  
> > 
> > Those say whether the filesystem should be checked, not when.
> >   
> >> Obviously, if "/usr" is on a separate partition, it needs to be
> >> mounted at the time when "/usr/sbin/fsck" is expected to be
> >> present.  
> > 
> > fsck is in /sbin, but that's not the point. If you have an initramfs,
> > fsck should be in it and run before /usr is mounted rw, which means it
> > has to be done by the initramfs. It's too late to do it when control
> > has been handed over because then /usr is already mounted rw.  
> 
> 
> So what does the dirty check and fsck of / ?

OpenRC AFAIK.
 
> I don't have an initramfs, I don't have a separate /usr,

You need an initramfs and a separate /usr to experience this problem. You
have neither so you have avoided it twice, well done :-)

On systems where I have both, I also have a filesystem that does not use
fsck, which is a third way of avoiding the issue.

> I run OpenRC and the kernel command line says where / is for mounting

And the kernel mounts it ro, openrc remounts/ rw later on. It seems the
problem here is the initramfs mounting /usr rw before the attemt to run
fsck. If I felt like finger-pointing, I'd be tempted to point at the
initramfs.


-- 
Neil Bothwick

Windows to 486/50 mhz cpu: Don't rush me, don't rush me...

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Dale]

Neil Bothwick wrote:
> On Sat, 13 Jan 2018 23:16:19 +0200, Alan McKinnon wrote:
>
>> I run OpenRC and the kernel command line says where / is for mounting
> And the kernel mounts it ro, openrc remounts/ rw later on. It seems the
> problem here is the initramfs mounting /usr rw before the attemt to run
> fsck. If I felt like finger-pointing, I'd be tempted to point at the
> initramfs.
>
>

That's my thinking as well.  If one is using a init thingy, the init
thingy should be checking those, / and /usr at the least.  It seems it
has two fingers pointing at it now.  ;-) 

Dale

:-)  :-) 

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[David Haller]

Hello,

On Sat, 13 Jan 2018, Andrew Barchuk wrote:
[..]
>My fstab:
>
>/dev/MacVg/gentoo-root  /       ext4    defaults                0 1
>/dev/MacVg/gentoo-usr   /usr    ext4    defaults                0 2
[..]
>Any ideas what is going on and how do I make the fsck check succeed?

Try changing the last field for /usr/ to "1", i.e.:

/dev/MacVg/gentoo-usr   /usr    ext4    defaults                0 1

HTH,
-dnh

-- 
"Of course it does not work, but look how fast it is!"
         -- Unknown

[gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Ian Zimmerman]

On 2018-01-13 15:49, Dale wrote:

> I think without a init thingy, it mounts / ro at first, runs the checks
> and then remounts rw.

Right.

> I think it does the same with /usr.

No, other filesystems are not mounted at all until they're checked, in
this situation (which is the traditional one, fsck is older than any
init thingy concept and a separate /usr was once highly recommended).

:-P :-P

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet, fetch the TXT record for the domain.

Re: [gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Dale]

Ian Zimmerman wrote:
> On 2018-01-13 15:49, Dale wrote:
>
>> I think without a init thingy, it mounts / ro at first, runs the checks
>> and then remounts rw.
> Right.
>
>> I think it does the same with /usr.
> No, other filesystems are not mounted at all until they're checked, in
> this situation (which is the traditional one, fsck is older than any
> init thingy concept and a separate /usr was once highly recommended).
>
> :-P :-P
>

You may be right.  I recall at least / being done during the init thingy
part.  I thought /usr was to, since it is mounted along with / within
the init part before the regular OS boots.  That's my understanding of
the purpose of the init thingy is to mount / and /usr and then pivot
over to the regular boot process.  Maybe it mounts /usr ro or something. 

Yea, it used to be recommended and in a way it can still be a good
idea.  I use LVM for example and I can increase /usr, /var, /home or
whatever without having to redo my drive setup.  The only thing I can't
change is / which is a regular file system.  Just have to cross that
bridge when I get there.  Oh, I had a log file file up /var once. 
System was still running and I was able to figure out the problem before
it got worse.  I can't recall what the problem was now but messages was
huge, I mean HUGE. 

Dale

:-)  :-) 

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Alan McKinnon]

On 14/01/2018 01:36, Neil Bothwick wrote:
> On Sat, 13 Jan 2018 23:16:19 +0200, Alan McKinnon wrote:
> 
>> On 13/01/2018 23:16, Neil Bothwick wrote:
>>> On Sat, 13 Jan 2018 14:57:59 -0600, John Johnson wrote:
>>>   
>>>> Shouldn't that be taken care of by the "/etc/fstab" entries?  
>>>
>>> Those say whether the filesystem should be checked, not when.
>>>   
>>>> Obviously, if "/usr" is on a separate partition, it needs to be
>>>> mounted at the time when "/usr/sbin/fsck" is expected to be
>>>> present.  
>>>
>>> fsck is in /sbin, but that's not the point. If you have an initramfs,
>>> fsck should be in it and run before /usr is mounted rw, which means it
>>> has to be done by the initramfs. It's too late to do it when control
>>> has been handed over because then /usr is already mounted rw.  
>>
>>
>> So what does the dirty check and fsck of / ?
> 
> OpenRC AFAIK.
>  
>> I don't have an initramfs, I don't have a separate /usr,
> 
> You need an initramfs and a separate /usr to experience this problem. You
> have neither so you have avoided it twice, well done :-)

I'm an engineer, so I don't add unneeded things that serve no purpose
and no benefit :-)



-- 
Alan McKinnon
alan.mckinnon@gmail.com

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Andrew Barchuk]

Thanks everyone, I've nailed it \o/ A more detailed story follows. 

After taking the approach of offloading / and /usr checks to Dracut I've
disabled fsck for those partitions in /etc/fstab so OpenRC fsck wouldn't
attempt to check /usr (and fail) by setting passno (the last column to
0). It turned out that Dracut honours fstab for fsck too (root get
special treatment and is checked anyways). After enabling fsck in fstab
for / and /usr Dracut started to check /usr - a small win already. No
the only thing left to do was to _not_ check / and /usr by the init fsck
which was natural to do by setting fsck_passno=">1" in /etc/conf.d/fsck.

Now all the filesystems are checked during the boot: the ones having
passno 1 (root and /usr) by the initramfs and passno 2 (/var, /home,
/data) by the init. The relevant excerpts from the final configuration
are below.

/etc/fstab:
/dev/MacVg/gentoo-root	/       ext4    defaults                0 1
/dev/MacVg/gentoo-usr	/usr    ext4    defaults                0 1
/dev/MacVg/gentoo-var	/var    ext4    nodev,nosuid            0 2
/dev/MacVg/gentoo-home	/home   ext4    nodev,nosuid            0 2
/dev/MacVg/data		/data   ext4    nodev,nosuid,noexec     0 2

/etc/dracut.conf:
hostonly="yes"
use_fstab="yes"
no_fsck="no"

Note that the two latter Dracut options are not required and applied by
default in my installation but showing explicitly what configuration is
used.

/etc/conf.d/fsck:
fsck_passno=">1"

As can be seen from logs / and /usr are checked by Dracut and /var,
/home and /data by OpenRC.

/var/log/dmesg:

[    2.607796] dracut: luksOpen /dev/sdb3 luks-2acb7668-fff1-492d-b46e-f05ead26d153
[    5.930575] random: crng init done
[   11.017574] dracut: Scanning devices dm-0  for LVM logical volumes MacVg/swap MacVg/gentoo-root MacVg/gentoo-usr
[   11.028712] dracut: inactive '/dev/MacVg/arch-root' [10.00 GiB] inherit
[   11.031019] dracut: inactive '/dev/MacVg/arch-var' [6.00 GiB] inherit
[   11.033121] dracut: inactive '/dev/MacVg/swap' [8.00 GiB] inherit
[   11.035193] dracut: inactive '/dev/MacVg/arch-home' [6.00 GiB] inherit
[   11.037305] dracut: inactive '/dev/MacVg/data' [48.00 GiB] inherit
[   11.039686] dracut: inactive '/dev/MacVg/gentoo-root' [1.00 GiB] inherit
[   11.042072] dracut: inactive '/dev/MacVg/gentoo-var' [1.00 GiB] inherit
[   11.044452] dracut: inactive '/dev/MacVg/gentoo-home' [2.00 GiB] inherit
[   11.046791] dracut: inactive '/dev/MacVg/gentoo-usr' [12.00 GiB] inherit
[   11.131447] dracut: Scanning devices dm-0  for LVM volume groups MacVg
[   11.137382] dracut: Reading all physical volumes. This may take a while...
[   11.146511] dracut: Found volume group "MacVg" using metadata type lvm2
[   11.187955] dracut: 9 logical volume(s) in volume group "MacVg" now active
[   11.198562] PM: Starting manual resume from disk
[   11.200830] PM: Image not found (code -22)
[   11.211669] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: data=ordered
[   11.244303] dracut: Checking ext4: /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144
[   11.248218] dracut: issuing e2fsck -a  /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144
[   11.264264] dracut: /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144: clean, 2477/65536 files, 33012/262144 blocks
[   11.267463] dracut: Mounting /dev/disk/by-uuid/199bb83d-c783-4254-a6eb-fdbb83c33144 with -o rw,relatime,data=ordered
[   11.278896] EXT4-fs (dm-2): mounted filesystem with ordered data mode. Opts: data=ordered
[   11.286288] dracut: Mounted root filesystem /dev/mapper/MacVg-gentoo--root
[   11.302264] dracut: Checking ext4: /dev/MacVg/gentoo-usr
[   11.305391] dracut: issuing e2fsck -a  /dev/MacVg/gentoo-usr
[   13.144541] dracut: /dev/MacVg/gentoo-usr: recovering journal
[   13.146699] dracut: /dev/MacVg/gentoo-usr: clean, 439895/1572864 files, 2264868/3145728 blocks
[   13.148865] dracut: Mounting /usr with -o defaults
[   13.167299] EXT4-fs (dm-3): mounted filesystem with ordered data mode. Opts: (null)
[   13.226192] dracut: Switching root

/var/log/rc.log:

* Checking local filesystems [passno >1]  ...
/dev/mapper/MacVg-gentoo--home: clean, 6057/131072 files, 211958/524288 blocks
/dev/mapper/MacVg-data: clean, 5831/3145728 files, 10243129/12582912 blocks
/dev/mapper/MacVg-gentoo--var: clean, 22475/65536 files, 61158/262144 blocks
 [ ok ]

P. S. A bonus point is I no longer have to patch genkernel to support
custom keymaps and fonts.

---
Andrew

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Floyd Anderson]

Hi Andrew,

On Sun, 14 Jan 2018 12:06:23 +0100
Andrew Barchuk <andrew@raindev.io> wrote:
>Thanks everyone, I've nailed it \o/ A more detailed story follows.

Thank you for your conclusion and that you share your solution which 
will certainly have some benefit for future readers.

Well done, keep it up.

-- 
Regards,
floyd

Re: [gentoo-user] fsck check of /usr on a separate partition fails during boot

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 530 bytes --]

On Sun, 14 Jan 2018 11:15:51 +0200, Alan McKinnon wrote:

> > You need an initramfs and a separate /usr to experience this problem.
> > You have neither so you have avoided it twice, well done :-)  
> 
> I'm an engineer, so I don't add unneeded things that serve no purpose
> and no benefit :-)
> 
I'm a geek so I have to ;-)

Actually, I do need an initramfs because I have an encrypted root
partition. I don't use one if it provides no benefit.


-- 
Neil Bothwick

Does someone know the cheats for WindowsXP?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 559 bytes --]

On Sat, 13 Jan 2018 20:59:56 -0600, Dale wrote:

> Yea, it used to be recommended and in a way it can still be a good
> idea.  I use LVM for example and I can increase /usr, /var, /home or
> whatever without having to redo my drive setup.  The only thing I can't
> change is / which is a regular file system.  Just have to cross that
> bridge when I get there.

You can put / on LVM too, but then you'll need an init-thingy and I don't
think any of us want to go there ;-)


-- 
Neil Bothwick

Unsupported service (adj): Broken (see Demon)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Dale]

Neil Bothwick wrote:
> On Sat, 13 Jan 2018 20:59:56 -0600, Dale wrote:
>
>> Yea, it used to be recommended and in a way it can still be a good
>> idea.  I use LVM for example and I can increase /usr, /var, /home or
>> whatever without having to redo my drive setup.  The only thing I can't
>> change is / which is a regular file system.  Just have to cross that
>> bridge when I get there.
> You can put / on LVM too, but then you'll need an init-thingy and I don't
> think any of us want to go there ;-)
>
>


Yea, keep in mind, I didn't want a init thingy at all.  Thing is, I had
already set up my partitions when that requirement came along. 

In a way, I'd either like to have everything on LVM or BTRFS or nothing
but /home on one of those.  In other words, if I have to have a init
thingy, everything is where I can shrink/expand as needed or if no init
thingy then all those are on a regular file system.  Likely going with
the later next time around. 

Maybe by then BTRFS will be 100% ready to go then.  ;-) 

Dale

:-)  :-) 

Re: [gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Rich Freeman]

On Sun, Jan 14, 2018 at 12:53 PM, Dale <rdalek1967@gmail.com> wrote:
>
> Yea, keep in mind, I didn't want a init thingy at all.

Could have fooled us...

-- 
Rich

Re: [gentoo-user] Re: fsck check of /usr on a separate partition fails during boot

[Dale]

Rich Freeman wrote:
> On Sun, Jan 14, 2018 at 12:53 PM, Dale <rdalek1967@gmail.com> wrote:
>> Yea, keep in mind, I didn't want a init thingy at all.
> Could have fooled us...
>

That's either a touch of sarcasm or you missed my messages.  ;-)

Dale

:-)  :-)