[gentoo-user] HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[gentoo-user] HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Grant Taylor]

On 8/4/19 7:26 PM, Grant Taylor wrote:
> I am also using a bit of a hack that I think could be (re)used to allow 
> /usr being a separate file system without /requiring/ an initramfs / 
> initrd.  (I'll reply in another email with details to avoid polluting 
> this thread.)

I think that a variation of a technique I'm using for LUKS encrypted 
/home on a VPS could be used to allow booting without an initramfs / 
initrd while maintaining a separate /usr file system.

The problem is that /bin & /sbin would be symbolic links to /usr/bin & 
/usr/sbin.  So, any commands that would be needed to mount the /usr file 
system would need to be directly accessible in /bin & /sbin paths, or 
indirectly accessible in /usr/bin & /usr/sbin.

IMHO this is a whopper of a hack.

Create the bin and sbin directories inside of the /usr directory that is 
the mount point so that they are on the underlying file system that /usr 
is mounted over top of.  Then copy the needed binaries to the /usr/bin & 
/usr/sbin directories on the underlying file system.  That way, 
/sbin/fsck -> /usr/sbin/fsck still exists even before the real /usr is 
mounted.

I did say this is a whopper of a hack.

It's trivial to access these directories even when the normal / full 
/usr is mounted.

1)  mkdir /mnt/root-underlay
2)  mount -o bind / /mnt/root-underlay
3)  ls /mnt/root-underlay/bin /mnt/root-underlay/sbin

This "technique" / "trick" / "hack" works because the /bin & /sbin 
""directories are sym-links to the /usr/bin & /usr/sbin directories. 
There is nothing that means that the contents of (/usr)/(s)bin can't 
change from one command invocation to another.  The /(s)bin sym-links 
just need to point to a valid directory.  They can easily be on the 
root-underlay file system that /usr gets mounted on top of.



-- 
Grant. . . .
unix || die

Re: [gentoo-user] HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Mick]

[-- Attachment #1: Type: text/plain, Size: 2600 bytes --]

On Monday, 5 August 2019 02:36:31 BST Grant Taylor wrote:
> On 8/4/19 7:26 PM, Grant Taylor wrote:
> > I am also using a bit of a hack that I think could be (re)used to allow
> > /usr being a separate file system without /requiring/ an initramfs /
> > initrd.  (I'll reply in another email with details to avoid polluting
> > this thread.)
> 
> I think that a variation of a technique I'm using for LUKS encrypted
> /home on a VPS could be used to allow booting without an initramfs /
> initrd while maintaining a separate /usr file system.
> 
> The problem is that /bin & /sbin would be symbolic links to /usr/bin &
> /usr/sbin.  So, any commands that would be needed to mount the /usr file
> system would need to be directly accessible in /bin & /sbin paths, or
> indirectly accessible in /usr/bin & /usr/sbin.
> 
> IMHO this is a whopper of a hack.
> 
> Create the bin and sbin directories inside of the /usr directory that is
> the mount point so that they are on the underlying file system that /usr
> is mounted over top of.  Then copy the needed binaries to the /usr/bin &
> /usr/sbin directories on the underlying file system.  That way,
> /sbin/fsck -> /usr/sbin/fsck still exists even before the real /usr is
> mounted.
> 
> I did say this is a whopper of a hack.
> 
> It's trivial to access these directories even when the normal / full
> /usr is mounted.
> 
> 1)  mkdir /mnt/root-underlay
> 2)  mount -o bind / /mnt/root-underlay
> 3)  ls /mnt/root-underlay/bin /mnt/root-underlay/sbin
> 
> This "technique" / "trick" / "hack" works because the /bin & /sbin
> ""directories are sym-links to the /usr/bin & /usr/sbin directories.
> There is nothing that means that the contents of (/usr)/(s)bin can't
> change from one command invocation to another.  The /(s)bin sym-links
> just need to point to a valid directory.  They can easily be on the
> root-underlay file system that /usr gets mounted on top of.

Interesting concept, thanks for sharing.

Unless I misunderstand how this will work, it will create duplication of the 
fs for /bin and /sbin, which will both use extra space and require managing.

Will you mount -bind the underlying fs in fstab?

How will you make sure installations of the same binaries are installed/copied 
in both underlying and mounted /usr/* fs and kept in sync?  By changing all 
affected ebuilds?

It is a hack alright, to restore the previous default /usr functionality, so a 
useful option to consider.  If I were to be asked my preference would be to 
revert the systemd inspired changes which caused this loss of functionality.  
;-)

-- 
Regards,

Mick

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Grant Taylor]

On 8/5/19 5:45 AM, Mick wrote:
> Interesting concept, thanks for sharing.

You're welcome.

> Unless I misunderstand how this will work, it will create duplication 
> of the fs for /bin and /sbin, which will both use extra space and 
> require managing.

Yes, it will create some duplication.  Though I don't think that /all/ 
of the contents of /bin and /sbin would need to be duplicated.  Think 
about the minimum viable binaries that are needed.

Perhaps something like busybox would even suffice.

> Will you mount -bind the underlying fs in fstab?

You could.  I have done so in some VMs that I've tested various things.

My use case on my VPS is for an encrypted data partition.  So I have 
things like the following:

/home     -> /var/LUKS/home
/etc/mail -> /var/LUKS/etc/mail
/etc/bind -> /var/LUKS/etc/bind

/var/LUKS/home/gtaylor does have an absolute minimum directory structure 
so that I can ssh in with my key and run a script to unlock / open and 
mount the LUKS volume and start some services (mostly email and DNS 
related).

> How will you make sure installations of the same binaries are 
> installed/copied in both underlying and mounted /usr/* fs and kept 
> in sync?  By changing all affected ebuilds?

I don't have an answer to this qustion.  I've not needed an answer to 
this question.

I think I would likely create a script that would copy specific files 
from the /usr path to the underlying /(usr) path as needed.

I doubt there would be many files.

I don't see any need to alter an untold number of ebuilds for a system 
architecture / file system decision.

> It is a hack alright, to restore the previous default /usr 
> functionality, so a useful option to consider.

That's why I shared it.

It's also an example of an idea that works for my use case that you are 
free to take and modify for your use case.  I don't need to know about 
your use case, much less have an answer for it, when I'm sharing my use 
case.  (Harking back to the different types of communities in the 
previous email.)

> If I were to be asked my preference would be to revert the systemd 
> inspired changes which caused this loss of functionality.  ;-)

Fair enough.

Though I would question just how much and what is broken by having a 
separate /usr file system without systemd.  }:-)  Specifically, is it 
truly broken?  Or does it need some minor tweaks?



-- 
Grant. . . .
unix || die

Re: [gentoo-user] HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 767 bytes --]

On Mon, 5 Aug 2019 10:30:58 -0600, Grant Taylor wrote:

> > How will you make sure installations of the same binaries are 
> > installed/copied in both underlying and mounted /usr/* fs and kept 
> > in sync?  By changing all affected ebuilds?  
> 
> I don't have an answer to this qustion.  I've not needed an answer to 
> this question.
> 
> I think I would likely create a script that would copy specific files 
> from the /usr path to the underlying /(usr) path as needed.
> 
> I doubt there would be many files.

You could use rsync with the --existing flag to update all files in your
bind mounted directories, and run it as a post-emerge script.

It's a horrible hack, I like it!


-- 
Neil Bothwick

C Error #011: First C Program, huh?

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Ian Zimmerman]

On 2019-08-04 19:36, Grant Taylor wrote:

> Create the bin and sbin directories inside of the /usr directory that
> is the mount point so that they are on the underlying file system that
> /usr is mounted over top of.  Then copy the needed binaries to the
> /usr/bin & /usr/sbin directories on the underlying file system.  That
> way, /sbin/fsck -> /usr/sbin/fsck still exists even before the real
> /usr is mounted.

Don't you have to go through some extra hoops (a flag to the mount
command or something) to mount over a non-empty directory?

-- 
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Manuel McLure]

[-- Attachment #1: Type: text/plain, Size: 487 bytes --]

On Mon, Aug 5, 2019 at 4:53 PM Ian Zimmerman <itz@very.loosely.org> wrote:

>
> Don't you have to go through some extra hoops (a flag to the mount
> command or something) to mount over a non-empty directory?
>
>
Not in my experience, I've done it many times (sometimes even on purpose :)
)
-- 
Manuel A. McLure WW1FA <manuel@mclure.org> <http://www.mclure.org>
...for in Ulthar, according to an ancient and significant law,
no man may kill a cat.                       -- H.P. Lovecraft

[-- Attachment #2: Type: text/html, Size: 1021 bytes --]

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Grant Taylor]

On 8/5/19 5:52 PM, Ian Zimmerman wrote:
> Don't you have to go through some extra hoops (a flag to the mount 
> command or something) to mount over a non-empty directory?

Nope.

I don't recall ever needing to do anything like that in Linux.

I do know that other traditional Unixes are more picky about it.  AIX 
will refuse to use a populated directory as a mount point.

As I type this, perhaps ZFS on Linux complains, but I don't recall.



-- 
Grant. . . .
unix || die

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 411 bytes --]

On Mon, 5 Aug 2019 18:04:00 -0600, Grant Taylor wrote:

> > Don't you have to go through some extra hoops (a flag to the mount 
> > command or something) to mount over a non-empty directory?  
> 
> Nope.
> 
> I don't recall ever needing to do anything like that in Linux.

Fuse complain about this, that's the only time I've seen it.


-- 
Neil Bothwick

Top Oxymorons Number 39: Almost exactly

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Jack]

On 2019.08.05 19:52, Ian Zimmerman wrote:
> On 2019-08-04 19:36, Grant Taylor wrote:
> 
>> Create the bin and sbin directories inside of the /usr directory  
>> that is the mount point so that they are on the underlying file  
>> system that /usr is mounted over top of.  Then copy the needed  
>> binaries to the /usr/bin & /usr/sbin directories on the underlying  
>> file system.  That way, /sbin/fsck -> /usr/sbin/fsck still exists  
>> even before the real /usr is mounted.
> 
> Don't you have to go through some extra hoops (a flag to the mount  
> command or something) to mount over a non-empty directory?

As others have said, no.  However, I keep wondering if an overlay file  
system might not be of some use here.  Start with /bin, containing only  
what's necessary to boot before /usr is available.  Once /usr is  
mounted, overlay mount /usr/bin on /bin (or would it be the other way  
around?)

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Grant Taylor]

On 8/5/19 6:28 PM, Jack wrote:
> However, I keep wondering if an overlay file system might not be of 
> some use here.  Start with /bin, containing only what's necessary to 
> boot before /usr is available.

I wonder how much of what would need to be in the pre-/usr /bin 
directory can be provided by busybox.  (Assuming that busybox is 
compiled with everything living in / (root).

> Once /usr is mounted, overlay mount /usr/bin on /bin (or would it be 
> the other way around?)

An overlay mount (mount -o bind /usr/bin /bin) would be an additional 
mount.  Which in and of itself is not a bad thing.  But the sym-link 
from /bin -> /usr/bin would avoid the additional mount.  Admittedly, you 
might need one additional (bind) mount somewhere to be able to access 
the underlay while /usr is mounted.

Unless....

...

Even bigger hack.

What if the underlay (/ (root)) file system had the following structure:

/bin     -> /usr/bin
/usr/bin -> /.bin

That would mean that the pre-/usr /bin contents would still be 
accessible via /.bin even after /usr is mounted.  And /bin would still 
point to /usr/bin as currently being discussed with /usr merge.



-- 
Grant. . . .
unix || die

Re: [gentoo-user] Re: HACK: Boot without an initramfs / initrd while maintaining a separate /usr file system.

[Grant Taylor]

On 8/5/19 8:45 PM, Grant Taylor wrote:
> Even bigger hack.

I wouldn't be me if I didn't lob these two words out there:

mount namespaces

/me will see himself out now.



-- 
Grant. . . .
unix || die