[gentoo-dev] separate /usr without initramfs

[gentoo-dev] separate /usr without initramfs

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1712 bytes --]

Hey all,

I have been advised to bring this topic back to the list before taking
any action, so here it is.

First, I need to clarify what I'm *NOT* talking about.

This discussion has nothing to do with whether or not you have the
split-usr use flag turned on; all of us officially have that on because
/bin, /lib* and /sbin are directories in the official Gentoo setup. In
other words, I am *not* talking about forcing the /usr merge.

Unfortunately, the concept of separate usr has gotten wrapped up in the
split-usr use flag and doesn't have to be.  For the record, I mean something
very specific when I say "separate usr". I am talking about the situation
where /usr is a mount point separate from /, so in this thread, let's stick
to "separate usr" for that situation. I am *not* even saying that using
separate usr is wrong or unsupported. You can even run separate usr with
split-usr turned off if you would like to do so.

Now for the use case I want to talk about, and that is using separate
/usr without using an initramfs to boot your system and pre-mount /usr.

If you do this, many things are broken, and this is why the binary
distros all use an initramfs if you do this. This configuration is also
unsupported officially in Gentoo [1] [2], and it is not shown as the
example setup in our handbook.

I want to hear from people who have / and /usr on separate partitions
and who are not using an initramfs.

If you are in this group, I have a very specific question. Why aren't
you using an initramfs?

Thanks,

William

[1] https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
[2] https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=a79dd69b0cca439bc0c483c9193c79e0554819d0

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[Dale]

William Hubbs wrote:
> Hey all,
>
> I have been advised to bring this topic back to the list before taking
> any action, so here it is.
>
> First, I need to clarify what I'm *NOT* talking about.
>
> This discussion has nothing to do with whether or not you have the
> split-usr use flag turned on; all of us officially have that on because
> /bin, /lib* and /sbin are directories in the official Gentoo setup. In
> other words, I am *not* talking about forcing the /usr merge.
>
> Unfortunately, the concept of separate usr has gotten wrapped up in the
> split-usr use flag and doesn't have to be.  For the record, I mean something
> very specific when I say "separate usr". I am talking about the situation
> where /usr is a mount point separate from /, so in this thread, let's stick
> to "separate usr" for that situation. I am *not* even saying that using
> separate usr is wrong or unsupported. You can even run separate usr with
> split-usr turned off if you would like to do so.
>
> Now for the use case I want to talk about, and that is using separate
> /usr without using an initramfs to boot your system and pre-mount /usr.
>
> If you do this, many things are broken, and this is why the binary
> distros all use an initramfs if you do this. This configuration is also
> unsupported officially in Gentoo [1] [2], and it is not shown as the
> example setup in our handbook.
>
> I want to hear from people who have / and /usr on separate partitions
> and who are not using an initramfs.
>
> If you are in this group, I have a very specific question. Why aren't
> you using an initramfs?
>
> Thanks,
>
> William
>
> [1] https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> [2] https://gitweb.gentoo.org/data/gentoo-news.git/commit/?id=a79dd69b0cca439bc0c483c9193c79e0554819d0


I have a separate /usr among others and always have.  The reason I do
that, /boot and / are normal partitions but everything else is LVM.  I
can adjust the size of everything BUT /boot and /.  At the time I did
that, the init thingy was not needed if I recall correctly.  I might
add, I've had to grow /usr and /var a couple times.  Before LVM, it
meant copying over to another drive, repartitioning and then restoring
to the old drive.  Time consuming and one wrong command could ruin a
install.

While I have a init thingy, I do not like it.  I've had a couple
failures already with those things.  Luckily I keep older kernels and
such for that.  If I had my wish, I would not need a init thingy, ever. 
It's just one more thing that can cause problems.  There's already more
than enough things that can break.  While I understand the problem comes
from upstream, I still think it sucks.  It's easy enough to have a
unbootable kernel as it is.  Adding another layer for booting to fail
should be avoided.  BTW, I use dracut.  I tried to build it other ways
but couldn't get it to work.  Bad thing is, when one fails even built
with dracut, I have no clue how it works really so no idea how to fix
other than using a older kernel or just rerunning dracut and hoping for
the best.

I'm also not looking forward to the other situation you mentioned
either.  At some point, having separate partitions won't be easy with or
without a init thingy.  I can't easily resize / without reworking the
whole thing.

Just my point of view on why I don't like the thing and wish I didn't
have to have one. 

Dale

:-)  :-) 

Re: [gentoo-dev] separate /usr without initramfs

[Jaco Kroon]

Hi,

On 2019/10/25 20:14, William Hubbs wrote:
> Hey all,
>
> I have been advised to bring this topic back to the list before taking
> any action, so here it is.
>
> First, I need to clarify what I'm *NOT* talking about.
>
> This discussion has nothing to do with whether or not you have the
> split-usr use flag turned on; all of us officially have that on because
> /bin, /lib* and /sbin are directories in the official Gentoo setup. In
> other words, I am *not* talking about forcing the /usr merge.
>
> Unfortunately, the concept of separate usr has gotten wrapped up in the
> split-usr use flag and doesn't have to be.  For the record, I mean something
> very specific when I say "separate usr". I am talking about the situation
> where /usr is a mount point separate from /, so in this thread, let's stick
> to "separate usr" for that situation. I am *not* even saying that using
> separate usr is wrong or unsupported. You can even run separate usr with
> split-usr turned off if you would like to do so.
>
> Now for the use case I want to talk about, and that is using separate
> /usr without using an initramfs to boot your system and pre-mount /usr.
>
> If you do this, many things are broken, and this is why the binary
> distros all use an initramfs if you do this. This configuration is also
> unsupported officially in Gentoo [1] [2], and it is not shown as the
> example setup in our handbook.
>
> I want to hear from people who have / and /usr on separate partitions
> and who are not using an initramfs.
>
> If you are in this group, I have a very specific question. Why aren't
> you using an initramfs?

Because until recently it wasn't an issue.  So for me the final kicker 
was still not a separate /usr.  For my use case everything except a big 
warning about md5sum not being available during boot works.  This is NOT 
desktop setup for MOST of my systems.  On the few that are I don't 
generally have things like bluetooth keyboards etc that I need available 
during early(ish) boot or anything crazy.  So the argument that "many 
things break" may be accurate, but I've yet to find a concrete example 
that bugs me enough to care.

There is basically one thing that I found that broke "out of the box" 
with a separate /usr - and that's if I have one of those stupid LTE 
modem things that pretend to be a disk drive/cdrom or something similar 
until you tell it to switch to network mode. The same thing that, for 
me, breaks with suspend resume (after resume I have to kill 
modem_manager, and start it *before* replugging the modem or it simply 
will not work - another discussion).

So frankly, I just don't see the benefit.  The reason I've eventually 
bothered with setting up and creating an initramfs now was because of 
/lib/firmware which I need available during module load (pre 
/etc/init.d/localmount) so that firmware is available as soon as amdgpu 
and i915 loads or else I end up with a borked screen.  Only uefi fb 
compiled into the kernel (required to get hand-over from grub2 ... at 
least, the only way I could get it to work smoothly).

The initramfs we ONLY pull in on systems where it's required (one 
currently, the machine I'm typing this on).

Why do I have /lib/firmware on a separate partition now?  Because 512MB 
for / used to be plenty, and I have MANY history systems out there which 
is non-trivial to make partitioning changes to (I keep / in a raw 
partition).  Now just /lib/firmware is larger than that.

Like William, / and /boot are definite partitions, and I want to keep 
these small.  Everything else is LVM.  Most systems have >50% of VG 
space unallocated because people always overestimate what they really need.

Separate partitions means I can set up separate mount options 
(nosuid,nodev etc ...)

Like William, I feel more cogs means more opportunities for breakage. I 
roll my own vanilla kernel, with one or two of my own patches.  I roll 
my own init script for initramfs.  Why - because system bootability is 
of utmost importance.  So I absolutely have to KNOW that it'll work, and 
when it doesn't that I can walk someone through fixing it over the 
phone. Never used the default gentoo initramfs.  genkernel always pulls 
in stuff I don't want nor need.  Sometimes it's just simpler I guess.  
But that's the thing - Gentoo gives me CHOICE.  Which I don't get from 
other distributions.  These are not the kind of things I like to leave 
to chance, or in the hands of a continuously changing tool (eg, dracut 
as mentioned by William).

Recently we ran into a bug causing filesystem corruption on /usr/portage 
(for some reason it was always under /usr/portage). With /usr on a 
separate partition we could recover that by setting appropriate boot 
options *remotely*.  No need to drive out.  Some of these systems were 
 >100km away.

So another motivation for separate / and /usr for me is that / is much 
more read-heavy than /usr, and as such, with the lower write ratio lower 
risk of corruption.  Better ability to recover.

As to why not use the initramfs - simple:  It's not needed.

The only potential advantage in my opinion is if you can build a 
recovery system in there that's small enough, and contains all 
conceivable tools required to recover from just about any boot failure.  
Work-in-progress I guess.

I'm not sure that answers your question, and this is one of those 
debates that can run in circles for hours, days and even weeks. So I 
hope I at least managed to give you some insight into my thinking and 
reasoning.

Kind Regards,
Jaco

Re: [gentoo-dev] separate /usr without initramfs

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 2517 bytes --]

Hi Dale,

I would like to call your attention to a couple of things in my message.

On Fri, Oct 25, 2019 at 01:40:10PM -0500, Dale wrote:
> William Hubbs wrote:

*snip*

> > I want to hear from people who have / and /usr on separate partitions
> > and who are not using an initramfs.
> >
> > If you are in this group, I have a very specific question. Why aren't
> > you using an initramfs?

*snip*

> I have a separate /usr among others and always have.  The reason I do
> that, /boot and / are normal partitions but everything else is LVM.  I
> can adjust the size of everything BUT /boot and /.  At the time I did
> that, the init thingy was not needed if I recall correctly.  I might
> add, I've had to grow /usr and /var a couple times.  Before LVM, it
> meant copying over to another drive, repartitioning and then restoring
> to the old drive.  Time consuming and one wrong command could ruin a
> install.
> 
> While I have a init thingy, I do not like it.  I've had a couple
> failures already with those things.  Luckily I keep older kernels and
> such for that.  If I had my wish, I would not need a init thingy, ever. 
> It's just one more thing that can cause problems.  There's already more
> than enough things that can break.  While I understand the problem comes
> from upstream, I still think it sucks.  It's easy enough to have a
> unbootable kernel as it is.  Adding another layer for booting to fail
> should be avoided.  BTW, I use dracut.  I tried to build it other ways
> but couldn't get it to work.  Bad thing is, when one fails even built
> with dracut, I have no clue how it works really so no idea how to fix
> other than using a older kernel or just rerunning dracut and hoping for
> the best.
 
 You just stated that you have an initramfs, so you did not thoroughly
 read my message. I specifically asked to hear from folks who aren't
 using one. All of this is irrelivent since you are.

> I'm also not looking forward to the other situation you mentioned
> either.  At some point, having separate partitions won't be easy with or
> without a init thingy.  I can't easily resize / without reworking the
> whole thing.
 
 Like I said above, separate partitions without an initramfs has been
 broken for many years. We have been doing some downstream hacking that
 made it work for some people. You are obviously not one of those people
 since you use an initramfs.

Can you please not add irrelivent noise to this thread?

 William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[Dale]

William Hubbs wrote:
> Hi Dale,
>
> I would like to call your attention to a couple of things in my message.
>
> On Fri, Oct 25, 2019 at 01:40:10PM -0500, Dale wrote:
>> William Hubbs wrote:
> *snip*
>
>>> I want to hear from people who have / and /usr on separate partitions
>>> and who are not using an initramfs.
>>>
>>> If you are in this group, I have a very specific question. Why aren't
>>> you using an initramfs?
> *snip*
>
>> I have a separate /usr among others and always have.  The reason I do
>> that, /boot and / are normal partitions but everything else is LVM.  I
>> can adjust the size of everything BUT /boot and /.  At the time I did
>> that, the init thingy was not needed if I recall correctly.  I might
>> add, I've had to grow /usr and /var a couple times.  Before LVM, it
>> meant copying over to another drive, repartitioning and then restoring
>> to the old drive.  Time consuming and one wrong command could ruin a
>> install.
>>
>> While I have a init thingy, I do not like it.  I've had a couple
>> failures already with those things.  Luckily I keep older kernels and
>> such for that.  If I had my wish, I would not need a init thingy, ever. 
>> It's just one more thing that can cause problems.  There's already more
>> than enough things that can break.  While I understand the problem comes
>> from upstream, I still think it sucks.  It's easy enough to have a
>> unbootable kernel as it is.  Adding another layer for booting to fail
>> should be avoided.  BTW, I use dracut.  I tried to build it other ways
>> but couldn't get it to work.  Bad thing is, when one fails even built
>> with dracut, I have no clue how it works really so no idea how to fix
>> other than using a older kernel or just rerunning dracut and hoping for
>> the best.
>  
>  You just stated that you have an initramfs, so you did not thoroughly
>  read my message. I specifically asked to hear from folks who aren't
>  using one. All of this is irrelivent since you are.
>
>> I'm also not looking forward to the other situation you mentioned
>> either.  At some point, having separate partitions won't be easy with or
>> without a init thingy.  I can't easily resize / without reworking the
>> whole thing.
>  
>  Like I said above, separate partitions without an initramfs has been
>  broken for many years. We have been doing some downstream hacking that
>  made it work for some people. You are obviously not one of those people
>  since you use an initramfs.
>
> Can you please not add irrelivent noise to this thread?
>
>  William


I did read your message.  The reason I posted, I wish I did NOT have to
have one and I was in the situation you describe.  From what people
post, here and elsewhere, my system may not boot without one.  So, if it
is possible to NOT have a init thingy, I'd love to see that supported. 

I might also add, I originally thought this was on -user not -dev.  I
did see that wrong. 

Dale

:-)  :-) 

Re: [gentoo-dev] separate /usr without initramfs

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 3541 bytes --]

On Fri, Oct 25, 2019 at 03:16:19PM -0500, Dale wrote:
> William Hubbs wrote:
> > Hi Dale,
> >
> > I would like to call your attention to a couple of things in my message.
> >
> > On Fri, Oct 25, 2019 at 01:40:10PM -0500, Dale wrote:
> >> William Hubbs wrote:
> > *snip*
> >
> >>> I want to hear from people who have / and /usr on separate partitions
> >>> and who are not using an initramfs.
> >>>
> >>> If you are in this group, I have a very specific question. Why aren't
> >>> you using an initramfs?
> > *snip*
> >
> >> I have a separate /usr among others and always have.  The reason I do
> >> that, /boot and / are normal partitions but everything else is LVM.  I
> >> can adjust the size of everything BUT /boot and /.  At the time I did
> >> that, the init thingy was not needed if I recall correctly.  I might
> >> add, I've had to grow /usr and /var a couple times.  Before LVM, it
> >> meant copying over to another drive, repartitioning and then restoring
> >> to the old drive.  Time consuming and one wrong command could ruin a
> >> install.
> >>
> >> While I have a init thingy, I do not like it.  I've had a couple
> >> failures already with those things.  Luckily I keep older kernels and
> >> such for that.  If I had my wish, I would not need a init thingy, ever. 
> >> It's just one more thing that can cause problems.  There's already more
> >> than enough things that can break.  While I understand the problem comes
> >> from upstream, I still think it sucks.  It's easy enough to have a
> >> unbootable kernel as it is.  Adding another layer for booting to fail
> >> should be avoided.  BTW, I use dracut.  I tried to build it other ways
> >> but couldn't get it to work.  Bad thing is, when one fails even built
> >> with dracut, I have no clue how it works really so no idea how to fix
> >> other than using a older kernel or just rerunning dracut and hoping for
> >> the best.
> >  
> >  You just stated that you have an initramfs, so you did not thoroughly
> >  read my message. I specifically asked to hear from folks who aren't
> >  using one. All of this is irrelivent since you are.
> >
> >> I'm also not looking forward to the other situation you mentioned
> >> either.  At some point, having separate partitions won't be easy with or
> >> without a init thingy.  I can't easily resize / without reworking the
> >> whole thing.
> >  
> >  Like I said above, separate partitions without an initramfs has been
> >  broken for many years. We have been doing some downstream hacking that
> >  made it work for some people. You are obviously not one of those people
> >  since you use an initramfs.
> >
> > Can you please not add irrelivent noise to this thread?
> >
> >  William
> 
> 
> I did read your message.  The reason I posted, I wish I did NOT have to
> have one and I was in the situation you describe.  From what people
> post, here and elsewhere, my system may not boot without one.  So, if it
> is possible to NOT have a init thingy, I'd love to see that supported. 

We are doing a bit of custom hackery at this point to make it work as
well as it does, so the reason for this thread is to determine why
people in that situation still are.
Expanding support would mean more custom hackery and forcing more things
to install in / instead of /usr, so I don't see us going there.


> I might also add, I originally thought this was on -user not -dev.  I
> did see that wrong. 

It doesn't matter that this is on -dev.

William

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[Joshua Kinard]

On 10/25/2019 14:14, William Hubbs wrote:
> Hey all,
> 
> I have been advised to bring this topic back to the list before taking
> any action, so here it is.
> 
> First, I need to clarify what I'm *NOT* talking about.
> 
> This discussion has nothing to do with whether or not you have the
> split-usr use flag turned on; all of us officially have that on because
> /bin, /lib* and /sbin are directories in the official Gentoo setup. In
> other words, I am *not* talking about forcing the /usr merge.
> 
> Unfortunately, the concept of separate usr has gotten wrapped up in the
> split-usr use flag and doesn't have to be.  For the record, I mean something
> very specific when I say "separate usr". I am talking about the situation
> where /usr is a mount point separate from /, so in this thread, let's stick
> to "separate usr" for that situation. I am *not* even saying that using
> separate usr is wrong or unsupported. You can even run separate usr with
> split-usr turned off if you would like to do so.
> 
> Now for the use case I want to talk about, and that is using separate
> /usr without using an initramfs to boot your system and pre-mount /usr.
> 
> If you do this, many things are broken, and this is why the binary
> distros all use an initramfs if you do this. This configuration is also
> unsupported officially in Gentoo [1] [2], and it is not shown as the
> example setup in our handbook.
> 
> I want to hear from people who have / and /usr on separate partitions
> and who are not using an initramfs.
> 
> If you are in this group, I have a very specific question. Why aren't
> you using an initramfs?
> 
> Thanks,
> 
> William


I use this approach.  /usr is its own partition (usually /dev/sda6 in my
buildouts), and it mounts a few seconds after init is started.  I've been
doing it this way on multiple architectures (x86, amd64, sparc64, mips)
since pretty much 2003, when I first started experimenting w/ Gentoo and
later went on to become a developer.  None of my existing Gentoo installs
have *ever* needed an initramfs to boot with /usr on a separate partition.
Yes, there's a small bit in OpenRC that whines apparently, but I have not
seen any ill effects.

My hardware layout is fairly static.  amd64 box uses hardware RAID5
(LSI/3Ware 9750), and LILO as bootloader, and the SGI machines use MD-RAID
in either RAID1 or RAID5 mode, and they all netboot (manually).  Maybe it's
because I stick to very simplistic builds, that I've avoided problems.  ::
shrug ::

Why do I not like an initramfs, though?  Well, for one, it complicates the
kernel compiles (and it makes them bigger, something which is an issue on
the old SGI systems at times).  Two, it's another layer that I have to
maintain.  Three, it violates, in my mind, the simplicity of keeping the
kernel and userland separated (e.g., kernel does kernel-y things, userland
does userland-y things).

Put simply, the kernel's single purpose, as nothing more than a
hyper-complex while loop, is to get the hardware up into a usable state and
then hand off to userland, then sit and service userland's needs as called
upon.  The kernel should have all of the subsystems loaded into it necessary
to accomplish this task.  The fact that the userland, in the current
ill-conceived case, cannot get itself up and running simply because /usr is
on a yet-to-be-mounted partition is not a concern of the kernel's.  Thus,
the loading of an initramfs into the kernel to solve this issue is, in
principle, a violation (and a Cthulhu-awful hack).

Maybe I'm just a old codger who refuses to accept change.  I'm fine with
that description.  I like things to remain somewhat simple, and my view of
Linux, both kernel and userland, over the last few years is one of growing
dismay due to the constant introduction of subsystem layer atop subsystem
layer for very little gain.  How much longer until we need a kernel to boot
the kernel to mount the userland that mounts the userland (yo dawg)?

And there's that tiny, tiny issue of some kiddos from another, commercial,
distribution that just up and declared one day that the idea of
/usr-on-another-partition was "broken", and they then used their market
position to effectively force all other distributions to bow to their
belief.  I took, and still to this day, take, great umbrage at that.  I'll
probably never get over it.  Surely, if it works for me on multiple systems
*today*, it can't be broken, right?  It certainly was *not* broken back
then.  So why change it?  Why completely reinstall all of my systems because
non-Gentoo developers said something I did was wrong?  No, I don't play that
game.

Oh, and for a technical reason, I like setting mount flags on partitions
like /usr, /var, and /tmp for security.  It's really as simple as that.  All
of the fluff above is just the philosophical pudding that I use to justify
my obstinance.

---

That all said, I've fallen madly in love with ZFS.  I've got two network
appliances and an old laptop running FreeBSD 12.0-RELEASE with ZFS, and the
things I can do with ZFS eliminate the need to have multiple physical
partitions on the disks in those systems (outside of FreeBSD's needed boot
and swap partitions).  I get separate datasets mounted at /usr, /var, /tmp,
and quite a few others, and each can have its own mount properties set that
also achieve the security aims I cherish.  I can even replicate VM-like
snapshot capabilities where I can break part of the install with some dumb
idea (like testing 12.1-RC2 out last weekend), then switch boot environments
back to the working one and the system comes back up as if nothing happened.

So lately, I've been messing around w/ ZFS On Linux using a crappy old ~2012
2.5" 5400rpm drive in my dev box, attached to the LSI 9750.  And with ZFS,
that little drive *screams*.  It's virtually faster than the RAID5 array on
the other three 9750 ports running XFS.  If that drive was a heavy metal
singer, it'd take over Bruce Dickinson's day job.

As such, I am contemplating switching to ZFS Root soon on that machine, once
I work out some minor details (like can I still use LILO, because GRUB is
meh).  If that happens, well, that's one machine that won't ever have to
worry about the /usr-on-a-sep-partition mess again (even though the juvenile
logic behind the no-separate-/usr movement is completely wrong, but I
digress in a troll-like fashion).

Of course, this leaves my poor SGI systems in a questionable state.  ZFS is
completely reliant on checksums, and ARC needs gobs upon gobs of RAM.  So
that right there means ZFS might not be viable on those old systems (at
least w/ SHA checksumming -- fletcher4 might be fine).  And that's assuming
ZFS even compiles for a mips64 target and runs without turning a disk into a
large pool of entropic bit slop.

Well, we'll just cross that bridge when we get there...

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] separate /usr without initramfs

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1523 bytes --]

On Sun, 27 Oct 2019 05:38:48 -0400
Joshua Kinard <kumba@gentoo.org> wrote:

> Why do I not like an initramfs, though?  Well, for one, it complicates the
> kernel compiles (and it makes them bigger, something which is an issue on
> the old SGI systems at times).  Two, it's another layer that I have to
> maintain.  Three, it violates, in my mind, the simplicity of keeping the
> kernel and userland separated (e.g., kernel does kernel-y things, userland
> does userland-y things).

You make it sound like the initramfs has to be built into the kernel
image. It can be but it usually isn't. I suspect you know that though?
Admittedly that does depend on support from your bootloader. While GRUB
and U-Boot have supported this for years, I forget what oddball
bootloaders your hardware may be using.

> Maybe I'm just a old codger who refuses to accept change.  I'm fine with
> that description.  I like things to remain somewhat simple, and my view of
> Linux, both kernel and userland, over the last few years is one of growing
> dismay due to the constant introduction of subsystem layer atop subsystem
> layer for very little gain.  How much longer until we need a kernel to boot
> the kernel to mount the userland that mounts the userland (yo dawg)?

Isn't that what the BIOS and bootloader do? On the plus side, you can
now boot straight from UEFI to kernel without a bootloader but on the
other hand, UEFI is horrifically over-complex.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[Benda Xu]

Joshua Kinard <kumba@gentoo.org> writes:

> Put simply, the kernel's single purpose, as nothing more than a
> hyper-complex while loop, is to get the hardware up into a usable state and
> then hand off to userland, then sit and service userland's needs as called
> upon.  The kernel should have all of the subsystems loaded into it necessary
> to accomplish this task.  The fact that the userland, in the current
> ill-conceived case, cannot get itself up and running simply because /usr is
> on a yet-to-be-mounted partition is not a concern of the kernel's.  Thus,
> the loading of an initramfs into the kernel to solve this issue is, in
> principle, a violation (and a Cthulhu-awful hack).
>
> Maybe I'm just a old codger who refuses to accept change.  I'm fine with
> that description.  I like things to remain somewhat simple, and my view of
> Linux, both kernel and userland, over the last few years is one of growing
> dismay due to the constant introduction of subsystem layer atop subsystem
> layer for very little gain.  How much longer until we need a kernel to boot
> the kernel to mount the userland that mounts the userland (yo dawg)?

This a very well put argument that I find enjoyable reading.  Thank you
Joshua.

Yours,
Benda

Re: [gentoo-dev] separate /usr without initramfs

[Matt Turner]

On Sun, Oct 27, 2019 at 3:06 AM James Le Cuirot <chewi@gentoo.org> wrote:
>
> On Sun, 27 Oct 2019 05:38:48 -0400
> Joshua Kinard <kumba@gentoo.org> wrote:
>
> > Why do I not like an initramfs, though?  Well, for one, it complicates the
> > kernel compiles (and it makes them bigger, something which is an issue on
> > the old SGI systems at times).  Two, it's another layer that I have to
> > maintain.  Three, it violates, in my mind, the simplicity of keeping the
> > kernel and userland separated (e.g., kernel does kernel-y things, userland
> > does userland-y things).
>
> You make it sound like the initramfs has to be built into the kernel
> image. It can be but it usually isn't. I suspect you know that though?
> Admittedly that does depend on support from your bootloader. While GRUB
> and U-Boot have supported this for years, I forget what oddball
> bootloaders your hardware may be using.

Though he's likely not using it, GRUB2 supports all the platforms he
mentioned (x86, amd64, sparc64, [sgi] mips).

Re: [gentoo-dev] separate /usr without initramfs

[Michael Everitt]

[-- Attachment #1.1: Type: text/plain, Size: 1131 bytes --]

On 27/10/19 16:12, Matt Turner wrote:
> On Sun, Oct 27, 2019 at 3:06 AM James Le Cuirot <chewi@gentoo.org> wrote:
>> On Sun, 27 Oct 2019 05:38:48 -0400
>> Joshua Kinard <kumba@gentoo.org> wrote:
>>
>>> Why do I not like an initramfs, though?  Well, for one, it complicates the
>>> kernel compiles (and it makes them bigger, something which is an issue on
>>> the old SGI systems at times).  Two, it's another layer that I have to
>>> maintain.  Three, it violates, in my mind, the simplicity of keeping the
>>> kernel and userland separated (e.g., kernel does kernel-y things, userland
>>> does userland-y things).
>> You make it sound like the initramfs has to be built into the kernel
>> image. It can be but it usually isn't. I suspect you know that though?
>> Admittedly that does depend on support from your bootloader. While GRUB
>> and U-Boot have supported this for years, I forget what oddball
>> bootloaders your hardware may be using.
> Though he's likely not using it, GRUB2 supports all the platforms he
> mentioned (x86, amd64, sparc64, [sgi] mips).
>
FWIW, I do believe I saw LILO mentioned ..


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1557 bytes --]

On Sun, 27 Oct 2019 16:17:04 +0000
Michael Everitt <gentoo@veremit.xyz> wrote:

> On 27/10/19 16:12, Matt Turner wrote:
> > On Sun, Oct 27, 2019 at 3:06 AM James Le Cuirot <chewi@gentoo.org> wrote:  
> >> On Sun, 27 Oct 2019 05:38:48 -0400
> >> Joshua Kinard <kumba@gentoo.org> wrote:
> >>  
> >>> Why do I not like an initramfs, though?  Well, for one, it complicates the
> >>> kernel compiles (and it makes them bigger, something which is an issue on
> >>> the old SGI systems at times).  Two, it's another layer that I have to
> >>> maintain.  Three, it violates, in my mind, the simplicity of keeping the
> >>> kernel and userland separated (e.g., kernel does kernel-y things, userland
> >>> does userland-y things).  
> >> You make it sound like the initramfs has to be built into the kernel
> >> image. It can be but it usually isn't. I suspect you know that though?
> >> Admittedly that does depend on support from your bootloader. While GRUB
> >> and U-Boot have supported this for years, I forget what oddball
> >> bootloaders your hardware may be using.  
> > Though he's likely not using it, GRUB2 supports all the platforms he
> > mentioned (x86, amd64, sparc64, [sgi] mips).
> >  
> FWIW, I do believe I saw LILO mentioned ..

https://wiki.gentoo.org/wiki/Early_Userspace_Mounting#Configuring_LILO

Phew. ;-)

Actually I was getting confused between initramfs support and device
tree support. I think every bootloader has supported initramfs since
forever.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] separate /usr without initramfs

[Joshua Kinard]

On 10/27/2019 06:06, James Le Cuirot wrote:
> On Sun, 27 Oct 2019 05:38:48 -0400
> Joshua Kinard <kumba@gentoo.org> wrote:
> 
>> Why do I not like an initramfs, though?  Well, for one, it complicates the
>> kernel compiles (and it makes them bigger, something which is an issue on
>> the old SGI systems at times).  Two, it's another layer that I have to
>> maintain.  Three, it violates, in my mind, the simplicity of keeping the
>> kernel and userland separated (e.g., kernel does kernel-y things, userland
>> does userland-y things).
> 
> You make it sound like the initramfs has to be built into the kernel
> image. It can be but it usually isn't. I suspect you know that though?
> Admittedly that does depend on support from your bootloader. While GRUB
> and U-Boot have supported this for years, I forget what oddball
> bootloaders your hardware may be using.

As mentioned (and later pointed out in the thread), I use LILO.  Considering
upstream is dead now, how much longer that will persist is an open question.
 I am a creature very wedded to habit, and LILO, for whatever its ills and
warts are, has always just worked for me, so I have stuck with it.  Having
to edit a config file before each kernel update is the trivialest of tasks,
especially since I tend to only update the kernel once every few months.

The SGI systems all boot using netboot because I compile their kernels on my
dev box and then host them over TFTP.  It's just a lot easier that way,
especially since ARCS (the SGI version of a BIOS) has had network booting
built in since pretty much 1990 (and earlier, I think).  It's really the SGI
Indy (IP22) that refuses to netboot if the kernel's size is >11MB.  All of
those kernels are monolithic, since the hardware in such systems is pretty
much fixed for all time.  Granted, my Indy gave up the ghost some time ago
(probably the infamous RTC battery issue), so I am really just left with an
O2 (which refuses to netboot anything >41MB), Octane, and two IP27-class
systems, which don't have size limits (that I've found yet) for what they'll
boot.

For actual disk booting on those systems, well, there's really just the
super-old ArcLoad project.  Written by the mastermind behind Octane's Linux
port, he aimed to make it very GRUB-like, so it has some of GRUB's
filesystem code in it for direct booting, or it can boot kernels stored in a
special area of the disk called the volume header (on disks w/ SGI's
partition layout).  Except that imported GRUB code is beyond ancient (11+
years old?), so it probably won't load a kernel off of ext4 or even XFSv5
partitions now.  Thus you really just have the volume header approach, and
that has its own limits.

That all said, it doesn't really matter if the initramfs is built into the
kernel or loaded somewhere into memory by the bootloader.  It's still a
wholly-separate userland layer that needs its own libc and whatever small
amount of tools to help mount required partitions and/or flash CPU microcode
before the kernel tries invoking /sbin/init.  That's just a terrible design.
 I mean, you can put lipstick on a pig, but it's still a bloody pig.


>> Maybe I'm just a old codger who refuses to accept change.  I'm fine with
>> that description.  I like things to remain somewhat simple, and my view of
>> Linux, both kernel and userland, over the last few years is one of growing
>> dismay due to the constant introduction of subsystem layer atop subsystem
>> layer for very little gain.  How much longer until we need a kernel to boot
>> the kernel to mount the userland that mounts the userland (yo dawg)?
> 
> Isn't that what the BIOS and bootloader do? On the plus side, you can
> now boot straight from UEFI to kernel without a bootloader but on the
> other hand, UEFI is horrifically over-complex.

Yeah, UEFI is one of those Eldritch horrors spoken of in the Necronomicon.
It's really just there so gamerz bois can have their uber-fancy gaming
system fully specced out.  Ironically, SGI's ARCS was capable of doing most
of what UEFI does 20 years ago, including pretty GFX and even playing music
when the system boots up (e.g., Indy's infamous "ass trumpet").

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] separate /usr without initramfs

[Joshua Kinard]

On 10/27/2019 12:12, Matt Turner wrote:
> On Sun, Oct 27, 2019 at 3:06 AM James Le Cuirot <chewi@gentoo.org> wrote:
>>
>> On Sun, 27 Oct 2019 05:38:48 -0400
>> Joshua Kinard <kumba@gentoo.org> wrote:
>>
>>> Why do I not like an initramfs, though?  Well, for one, it complicates the
>>> kernel compiles (and it makes them bigger, something which is an issue on
>>> the old SGI systems at times).  Two, it's another layer that I have to
>>> maintain.  Three, it violates, in my mind, the simplicity of keeping the
>>> kernel and userland separated (e.g., kernel does kernel-y things, userland
>>> does userland-y things).
>>
>> You make it sound like the initramfs has to be built into the kernel
>> image. It can be but it usually isn't. I suspect you know that though?
>> Admittedly that does depend on support from your bootloader. While GRUB
>> and U-Boot have supported this for years, I forget what oddball
>> bootloaders your hardware may be using.
> 
> Though he's likely not using it, GRUB2 supports all the platforms he
> mentioned (x86, amd64, sparc64, [sgi] mips).

Nope, never took to GRUB.  Never bought into the selling point of "you don't
have to edit a config file for every kernel update".  Also don't have a
working sparc64 box anymore, as the Blade 100 died quite a long time ago.  I
still have a SunFire v240, but it's got a PROM release that can't netboot,
and the updated version that can was released 3 days after Oracle bought Sun
and locked SunSolve up (thanks, Oracle).  Plus the thing is as loud as a jet
engine.

As far as GRUB working on SGI systems, that's news to me.  ARCS is pretty
unique on each SGI system, so it's not like one can write a tiny bit of code
to cover them all.  I'll have to investigate GRUB's documentation to see if
can deal w/ machines running ARCS, and whether it can handle IP27, IP30, and
even IP35.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic