[gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Andreas K. Huettel]

Hi all, 

why do we *copy* the timezone file to /etc/localtime, instead of symlinking it like everyone else?

1) Our handbook recommends:

echo "Europe/Brussels" > /etc/timezone
emerge --config sys-libs/timezone-data

which is effectively doing 

echo "Europe/Brussels" > /etc/timezone
cp -f /usr/share/zoneinfo/Europe/Brussels /etc/localtime

2) Most other distros seem to just do

ln -sf /usr/share/zoneinfo/Europe/Brussels /etc/localtime

and use the link content as timezone name (this is also what is required by systemd).

Does anyone remember the reason for 1) ? Or is that lost in history?

Cheers,
Andreas

PS See also bug https://bugs.gentoo.org/737914 , where support for 1) was removed from Qt.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Nils Freydank]

[-- Attachment #1: Type: text/plain, Size: 706 bytes --]

Hi Andreas,

Am Samstag, den 20.03.2021 um 16:37:23 Uhr +0100 schrieb "Andreas K. Huettel" <dilfridge@gentoo.org>:
> [...]
> Does anyone remember the reason for 1) ? Or is that lost in history?

I just quote comment 3 from the linked bug https://bugs.gentoo.org/737914#c3:
"copying the zonefile to /etc/localtime is a good idea, as /usr could be on a separate partition. How about creating the

/etc/TZ -> /etc/timezone softlink by default?"

In my opinion so many tools tend to expect an always-mounted /usr that the
symlink would not do any harm. FWIW I use symlinks on a handfull Gentoo machines
and had no issues in years, but I'm pretty close to mainstream (amd64, glibc,
OpenRC).

Best regards,
Nils

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 1293 bytes --]

On Sat, Mar 20, 2021 at 05:04:06PM +0100, Nils Freydank wrote:
> Hi Andreas,
> 
> Am Samstag, den 20.03.2021 um 16:37:23 Uhr +0100 schrieb "Andreas K. Huettel" <dilfridge@gentoo.org>:
> > [...]
> > Does anyone remember the reason for 1) ? Or is that lost in history?
> 
> I just quote comment 3 from the linked bug https://bugs.gentoo.org/737914#c3:
> "copying the zonefile to /etc/localtime is a good idea, as /usr could be on a separate partition. How about creating the
> 
> /etc/TZ -> /etc/timezone softlink by default?"
> 
> In my opinion so many tools tend to expect an always-mounted /usr that the
> symlink would not do any harm. FWIW I use symlinks on a handfull Gentoo machines
> and had no issues in years, but I'm pretty close to mainstream (amd64, glibc,
> OpenRC).

I'm not sure which symlink you are talking about, so I am adding my
comments here.

/etc/localtime should definitely be a symlink to the proper file in
/usr/share/zoneinfo.

This works fine if /usr is on a separate partition *and* you are using
an initramfs. The only time it doesn't work is if /usr is separate
without using an initramfs.

Council decided years ago that we don't support separate /usr without an
initramfs, but we haven't completed that transition yet.

William


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 514 bytes --]

>>>>> On Sat, 20 Mar 2021, William Hubbs wrote:

> /etc/localtime should definitely be a symlink to the proper file in
> /usr/share/zoneinfo.

> This works fine if /usr is on a separate partition *and* you are using
> an initramfs. The only time it doesn't work is if /usr is separate
> without using an initramfs.

> Council decided years ago that we don't support separate /usr without
> an initramfs, but we haven't completed that transition yet.

Which doesn't imply that we deliberately break things.

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Alec Warner]

On Sat, Mar 20, 2021 at 9:19 PM Ulrich Mueller <ulm@gentoo.org> wrote:
>
> >>>>> On Sat, 20 Mar 2021, William Hubbs wrote:
>
> > /etc/localtime should definitely be a symlink to the proper file in
> > /usr/share/zoneinfo.
>
> > This works fine if /usr is on a separate partition *and* you are using
> > an initramfs. The only time it doesn't work is if /usr is separate
> > without using an initramfs.
>
> > Council decided years ago that we don't support separate /usr without
> > an initramfs, but we haven't completed that transition yet.
>
> Which doesn't imply that we deliberately break things.

Not sure I follow.. how is updating the handbook breaking anything?

-A

>
> Ulrich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 388 bytes --]

>>>>> On Sun, 21 Mar 2021, Alec Warner wrote:

>> Which doesn't imply that we deliberately break things.

> Not sure I follow.. how is updating the handbook breaking anything?

Both configurations (regular file and symlink) work just fine, and
sys-libs/timezone-data supports them. I don't see a good reason why we
would suddenly declare the regular file to be an invalid option.

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Alec Warner]

On Sat, Mar 20, 2021 at 10:27 PM Ulrich Mueller <ulm@gentoo.org> wrote:
>
> >>>>> On Sun, 21 Mar 2021, Alec Warner wrote:
>
> >> Which doesn't imply that we deliberately break things.
>
> > Not sure I follow.. how is updating the handbook breaking anything?
>
> Both configurations (regular file and symlink) work just fine, and
> sys-libs/timezone-data supports them. I don't see a good reason why we
> would suddenly declare the regular file to be an invalid option.

https://bugs.gentoo.org/737914 seems to imply for some upstreams, it
being a file is not a valid option anymore?

(I'm ignoring the logic of that decision of course, but this was the
original reason this was raised.)

-A

>
> Ulrich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Ulrich Mueller]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

>>>>> On Sun, 21 Mar 2021, Alec Warner wrote:

> https://bugs.gentoo.org/737914 seems to imply for some upstreams, it
> being a file is not a valid option anymore?

> (I'm ignoring the logic of that decision of course, but this was the
> original reason this was raised.)

Indeed, that's a strange design decision. It also violates the principle
of least surprise, because nobody will expect a symlink to behave any
different from a literal copy of the file (or even from a hardlink) when
reading it.

Ulrich

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 507 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

On Sun, 21 Mar 2021 05:18:52 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> >>>>> On Sat, 20 Mar 2021, William Hubbs wrote:  
> 
> > /etc/localtime should definitely be a symlink to the proper file in
> > /usr/share/zoneinfo.  
> 
> > This works fine if /usr is on a separate partition *and* you are using
> > an initramfs. The only time it doesn't work is if /usr is separate
> > without using an initramfs.  
> 
> > Council decided years ago that we don't support separate /usr without
> > an initramfs, but we haven't completed that transition yet.  
> 
> Which doesn't imply that we deliberately break things.

How about making emerge --config dynamic, copying if it's on a
different partition and symlinking if it's not? You can't accurately
determine the use of an initramfs but at least this is closer to what
we want.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 450 bytes --]

On 2021-03-20 16:37, Andreas K. Huettel wrote:
> 2) Most other distros seem to just do

No, not most distros are doing that. systemd is forcing that downstream 
(the result is the same)!

It was added via 
https://github.com/systemd/systemd/commit/92c4ef2d357baeef78b6f82f119b92f7ed12ac77 
without mentioning a reason.


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Mike Gilbert]

On Sun, Mar 21, 2021 at 7:58 PM Thomas Deutschmann <whissi@gentoo.org> wrote:
>
> On 2021-03-20 16:37, Andreas K. Huettel wrote:
> > 2) Most other distros seem to just do
>
> No, not most distros are doing that. systemd is forcing that downstream
> (the result is the same)!
>
> It was added via
> https://github.com/systemd/systemd/commit/92c4ef2d357baeef78b6f82f119b92f7ed12ac77
> without mentioning a reason.

Based on that commit message, it looks systemd switched to looking at
the symlink target instead of /etc/timezone well *after* some major
distro started using a symlink for /etc/localtime. I suspect Kay
Sievers noticed that the content of /etc/timezone and /etc/localtime
were redundant on his development machine, and added a TODO entry to
eliminate the redundant /etc/timezone file.

In other words, this isn't a case of systemd forcing distros to
symlink /etc/localtime; they were already doing that anyway.

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Mike Gilbert]

On Sat, Mar 20, 2021 at 11:37 AM Andreas K. Huettel
<dilfridge@gentoo.org> wrote:
>
> Hi all,
>
> why do we *copy* the timezone file to /etc/localtime, instead of symlinking it like everyone else?
>
> 1) Our handbook recommends:
>
> echo "Europe/Brussels" > /etc/timezone
> emerge --config sys-libs/timezone-data
>
> which is effectively doing
>
> echo "Europe/Brussels" > /etc/timezone
> cp -f /usr/share/zoneinfo/Europe/Brussels /etc/localtime
>
> 2) Most other distros seem to just do
>
> ln -sf /usr/share/zoneinfo/Europe/Brussels /etc/localtime
>
> and use the link content as timezone name (this is also what is required by systemd).
>
> Does anyone remember the reason for 1) ? Or is that lost in history?

I think the most obvious reason would be to support late-mounting of
/usr, after the system clock has been reset with a UTC offset on boot.
To make this work, the /etc/localtime file would need to be available
when the hwclock init script runs.

These days, I suspect there are not many users running systems with a
hardware clock in local time, and a separate /usr filesystem, and no
initramfs to mount /usr early.

I created a PR to adjust the logic in timezone-data to promote
symlinks for new installs, while keeping regular files in place for
existing users.

https://github.com/gentoo/gentoo/pull/20050

If we added some logic to detect if /usr is a separate filesystem, we
could possibly be even more aggressive about this. But I'm not sure I
want to rock the boat that much in one commit.

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 1554 bytes --]

On 2021-03-22 03:06, Mike Gilbert wrote:
> Based on that commit message, it looks systemd switched to looking at
> the symlink target instead of /etc/timezone well *after* some major
> distro started using a symlink for /etc/localtime. I suspect Kay
> Sievers noticed that the content of /etc/timezone and /etc/localtime
> were redundant on his development machine, and added a TODO entry to
> eliminate the redundant /etc/timezone file.
> 
> In other words, this isn't a case of systemd forcing distros to
> symlink /etc/localtime; they were already doing that anyway.

I just downloaded and tested some old distributions:

Debian 9 was the first Debian release with systemd. Because of systemd, 
/etc/localtime became a symlink. In Debian 8 or when you install Debian 
9 without systemd, it is a regular file.

Ubuntu 12.04.5 is the same: No systemd, /etc/localtime is a regular 
file. Once they moved to systemd it became a symlink.

In Fedora 17, which is already using systemd but a version before linked 
commit, /etc/localtime is also a regular file. But once Fedora upgraded 
to >=systemd-190 it became a symlink.

That's why from my P.O.V. this is clearly caused by systemd. But does 
this matter? I doubt that systemd will even think about removing what I 
believe to be a false warning when systemd detects that /etc/localtime 
is a regular file. So let's focus on dealing with the fallout...


-- 
Regards,
Thomas Deutschmann / Gentoo Linux Developer
fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Thomas Deutschmann]

[-- Attachment #1.1: Type: text/plain, Size: 139 bytes --]

PS: Even Debian is mentioning "to follow systemd" when they updated 
their tzdata package end of 2015, https://bugs.debian.org/803144.


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Mike Gilbert]

On Sun, Mar 21, 2021 at 11:06 PM Thomas Deutschmann <whissi@gentoo.org> wrote:
>
> On 2021-03-22 03:06, Mike Gilbert wrote:
> > Based on that commit message, it looks systemd switched to looking at
> > the symlink target instead of /etc/timezone well *after* some major
> > distro started using a symlink for /etc/localtime. I suspect Kay
> > Sievers noticed that the content of /etc/timezone and /etc/localtime
> > were redundant on his development machine, and added a TODO entry to
> > eliminate the redundant /etc/timezone file.
> >
> > In other words, this isn't a case of systemd forcing distros to
> > symlink /etc/localtime; they were already doing that anyway.
>
> I just downloaded and tested some old distributions:
>
> Debian 9 was the first Debian release with systemd. Because of systemd,
> /etc/localtime became a symlink. In Debian 8 or when you install Debian
> 9 without systemd, it is a regular file.
>
> Ubuntu 12.04.5 is the same: No systemd, /etc/localtime is a regular
> file. Once they moved to systemd it became a symlink.
>
> In Fedora 17, which is already using systemd but a version before linked
> commit, /etc/localtime is also a regular file. But once Fedora upgraded
> to >=systemd-190 it became a symlink.

Thanks for looking into it. I wonder how Kay's system ended up that
way then. Just a curiosity.

> That's why from my P.O.V. this is clearly caused by systemd. But does
> this matter?

You're the one who brought it up; I'm not sure what the point of that
was, other than to complain about systemd.

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Andreas Sturmlechner]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

On Sonntag, 21. März 2021 08:58:34 CET James Le Cuirot wrote:
> How about making emerge --config dynamic, copying if it's on a
> different partition and symlinking if it's not? You can't accurately
> determine the use of an initramfs but at least this is closer to what
> we want.

We will not be able to carry the revert for Qt5Core forever. I suspect it will 
be gone with Qt6, and then all we can do is ewarn users that their clock will 
be broken if their localtime is not a symlink.

Keeping it as a file would mean convincing upstream(s) that the symlink is a 
systemd-ism, not a standard.

Regards

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 977 bytes --]

On Mon, 22 Mar 2021 09:59:11 +0100 Andreas Sturmlechner wrote:

>On Sonntag, 21. März 2021 08:58:34 CET James Le Cuirot wrote:
>> How about making emerge --config dynamic, copying if it's on a
>> different partition and symlinking if it's not? You can't accurately
>> determine the use of an initramfs but at least this is closer to what
>> we want.
>
>We will not be able to carry the revert for Qt5Core forever. I suspect
>it will be gone with Qt6, and then all we can do is ewarn users that
>their clock will be broken if their localtime is not a symlink.
>
>Keeping it as a file would mean convincing upstream(s) that the
>symlink is a systemd-ism, not a standard.
>
>Regards

With enough motivation we can carry that revert for a very long time. I
know that because I still carry reverts in my udev packages from when
it was devoured by systemd.

-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1059 bytes --]

> > Council decided years ago that we don't support separate /usr without
> > an initramfs, but we haven't completed that transition yet.
> 
> Which doesn't imply that we deliberately break things.

That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.

Why? Because the number of required hacks and complexity will only increase, as will the number of uncooperative upstreams. It's called a strategic retreat. :D

My suggestion would be that the next profile version (21? 22?) declares separate /usr a broken configuration, and explicitly encourages devs to introduce all ebuild simplifications that are made possible by this. (Like this symlink - no more conditional code.) No more discussions about "not breaking things" at that point.

(Or to put it another way, I think we should stop wasting time and effort here just to be able to live in the past.)

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Andreas Sturmlechner]

[-- Attachment #1: Type: text/plain, Size: 320 bytes --]

On Monday, 22 March 2021 21:18:32 CET Lars Wendler wrote:
> With enough motivation we can carry that revert for a very long time. I
> know that because I still carry reverts in my udev packages from when
> it was devoured by systemd.

It is 11.2 KiB worth of patch that at least I know I won't take responsibility 
over.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Jaco Kroon]

Hi Andreas,

On 2021/03/23 00:54, Andreas K. Huettel wrote:
>>> Council decided years ago that we don't support separate /usr without
>>> an initramfs, but we haven't completed that transition yet.
>> Which doesn't imply that we deliberately break things.
> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
>
> Why? Because the number of required hacks and complexity will only increase, as will the number of uncooperative upstreams. It's called a strategic retreat. :D
>
> My suggestion would be that the next profile version (21? 22?) declares separate /usr a broken configuration, and explicitly encourages devs to introduce all ebuild simplifications that are made possible by this. (Like this symlink - no more conditional code.) No more discussions about "not breaking things" at that point.

Why was it ever copied in the first place?  For as long as I can recall
(been using Gentoo since 2003) I've always been using a symlink here,
not to mention a separate /usr (which has only been in the last year or
so mounted from initrd along with /lib/firmware - which was the trigger
point when it became too big to be contained on our normal / partition,
the other fix would have been to be more selective in which firmware to
install but that would be a higher administrative overhead).

To this day I still believe that / should contain a minimal viable
bootable system (give me a shell and just enough to perform basic tasks
like activating LVM, repairing and mounting filesystems, and ideally
some or another editor such as busybox vi is good enough, mostly
everything else can go to /usr even with it being "split").

I still don't see why a split /usr is a bad thing.  In fact, there are a
significant number cases where I've had FS corruption which I was able
to recover without the need for additional bootable media (which when
working remotely via IP KVMs can be difficult at best) due to "living in
the past" as you say.

There is no reason a symbolic link can't cross a filesystem boundary ...
it's hard links that can't.

Kind Regards,
Jaco

> (Or to put it another way, I think we should stop wasting time and effort here just to be able to live in the past.)
>

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Rich Freeman]

On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
>
> > > Council decided years ago that we don't support separate /usr without
> > > an initramfs, but we haven't completed that transition yet.
> >
> > Which doesn't imply that we deliberately break things.
>
> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
>

Just to clarify - it is already unsupported at a distro level.  It is
just that some individual packages still work with it.

The current Council decisions on the issue are (just providing for
general reference):

- "Since that particular setup may already be subtly broken today
  depending on the installed software, Council recommends using an
  early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
  is on a separate partition."
  Accepted unanimously. [1]

- "The intention is to eventually not require maintainers to support
  a separate /usr without an early boot mechanism once the Council
  agrees that the necessary docs/migration path is in place."
  Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]

- "The Council agrees that all preparations for dropping support for
  separate /usr without an initramfs or similar boot mechanism are
  complete. A news item will be prepared, and users will be given one
  month to switch after the news item has been sent."
  Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]

Current policy documentation:
Developers are not required to support using separate /usr filesystem
without an initramfs. [3]

1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202

-- 
Rich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/23/2021 07:31, Rich Freeman wrote:
> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
>>
>>>> Council decided years ago that we don't support separate /usr without
>>>> an initramfs, but we haven't completed that transition yet.
>>>
>>> Which doesn't imply that we deliberately break things.
>>
>> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
>>
> 
> Just to clarify - it is already unsupported at a distro level.  It is
> just that some individual packages still work with it.
> 
> The current Council decisions on the issue are (just providing for
> general reference):
> 
> - "Since that particular setup may already be subtly broken today
>   depending on the installed software, Council recommends using an
>   early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
>   is on a separate partition."
>   Accepted unanimously. [1]
> 
> - "The intention is to eventually not require maintainers to support
>   a separate /usr without an early boot mechanism once the Council
>   agrees that the necessary docs/migration path is in place."
>   Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
> 
> - "The Council agrees that all preparations for dropping support for
>   separate /usr without an initramfs or similar boot mechanism are
>   complete. A news item will be prepared, and users will be given one
>   month to switch after the news item has been sent."
>   Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
> 
> Current policy documentation:
> Developers are not required to support using separate /usr filesystem
> without an initramfs. [3]
> 
> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202

Is there a list of software/ebuilds that currently do this "subtle" handling
of separate /usr w/o initramfs?

I've got just my MIPS systems left that use a separate /usr and do not boot
with initramfs because I build fully monolithic kernels and that makes the
resulting vmlinux images run up against hard size limits in the SGI PROM
(firmware, BIOS, etc) on these machines if I try to pack too large of an
initramfs in.  I can check for any software that may be switched over soon
to a hard initramfs requirement and look at my options.

I kinda wish the Linux kernel had an ability to partially boot, init the
networking subsystem, then fetch an initramfs image over TFTP like it can do
with NFS Root.  That would solve the problem on my MIPS system(s) (and make
install netboots better).  I've dug around, but this does not seem to be a
capability currently in the kernel, unless I have over looked something.

Otherwise in the future, I may just have to setup an initramfs into an NFS
Root and teach the SGI's to somehow deal with it.  Which all still seems
unnecessarily complicated because some other distro thinks it knows what's
best for everyone else (but I digress...).

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

On Sat, 27 Mar 2021 17:43:34 -0400
Joshua Kinard <kumba@gentoo.org> wrote:

> I kinda wish the Linux kernel had an ability to partially boot, init the
> networking subsystem, then fetch an initramfs image over TFTP like it can do
> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
> install netboots better).  I've dug around, but this does not seem to be a
> capability currently in the kernel, unless I have over looked something.
> 
> Otherwise in the future, I may just have to setup an initramfs into an NFS
> Root and teach the SGI's to somehow deal with it.  Which all still seems
> unnecessarily complicated because some other distro thinks it knows what's
> best for everyone else (but I digress...).

NBD may be a slightly simpler alternative and a bit more like an
initramfs. nbdkit can do all sorts of weird things. I thought it might
have a plugin for cpio archives, allowing you to use a regular
initramfs generated by Dracut or similar directly. It doesn't appear to
but plugins are quite easy to write. Alternatively you could just
extract the initramfs and use nbdkit-linuxdisk-plugin.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/27/2021 18:16, James Le Cuirot wrote:
> On Sat, 27 Mar 2021 17:43:34 -0400
> Joshua Kinard <kumba@gentoo.org> wrote:
> 
>> I kinda wish the Linux kernel had an ability to partially boot, init the
>> networking subsystem, then fetch an initramfs image over TFTP like it can do
>> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
>> install netboots better).  I've dug around, but this does not seem to be a
>> capability currently in the kernel, unless I have over looked something.
>>
>> Otherwise in the future, I may just have to setup an initramfs into an NFS
>> Root and teach the SGI's to somehow deal with it.  Which all still seems
>> unnecessarily complicated because some other distro thinks it knows what's
>> best for everyone else (but I digress...).
> 
> NBD may be a slightly simpler alternative and a bit more like an
> initramfs. nbdkit can do all sorts of weird things. I thought it might
> have a plugin for cpio archives, allowing you to use a regular
> initramfs generated by Dracut or similar directly. It doesn't appear to
> but plugins are quite easy to write. Alternatively you could just
> extract the initramfs and use nbdkit-linuxdisk-plugin.

Can NBD be used like I described?  Never played with it before.  The MIPS
machine has functioning local disk drives, and currently, it boots fine by
just pulling a kernel off my TFTP server and booting from the local drive,
no initramfs needed because I compiled everything into it.  If we ever force
sep-usr to end, then I'll need a way to teach it to mount /usr before
dropping into /bin/init (and nothing else).

I wonder if there's a small C program out there that can call whatever the
kernel functions are to mount a disk partition that could be embedded into a
tiny initramfs, then pivot_root to $REAL_ROOT and run /bin/init?

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 5573 bytes --]

On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
> On 3/23/2021 07:31, Rich Freeman wrote:
> > On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
> >>
> >>>> Council decided years ago that we don't support separate /usr without
> >>>> an initramfs, but we haven't completed that transition yet.
> >>>
> >>> Which doesn't imply that we deliberately break things.
> >>
> >> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
> >>
> > 
> > Just to clarify - it is already unsupported at a distro level.  It is
> > just that some individual packages still work with it.
> > 
> > The current Council decisions on the issue are (just providing for
> > general reference):
> > 
> > - "Since that particular setup may already be subtly broken today
> >   depending on the installed software, Council recommends using an
> >   early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
> >   is on a separate partition."
> >   Accepted unanimously. [1]
> > 
> > - "The intention is to eventually not require maintainers to support
> >   a separate /usr without an early boot mechanism once the Council
> >   agrees that the necessary docs/migration path is in place."
> >   Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
> > 
> > - "The Council agrees that all preparations for dropping support for
> >   separate /usr without an initramfs or similar boot mechanism are
> >   complete. A news item will be prepared, and users will be given one
> >   month to switch after the news item has been sent."
> >   Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
> > 
> > Current policy documentation:
> > Developers are not required to support using separate /usr filesystem
> > without an initramfs. [3]
> > 
> > 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> > 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> > 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
> 
> Is there a list of software/ebuilds that currently do this "subtle" handling
> of separate /usr w/o initramfs?
> 
> I've got just my MIPS systems left that use a separate /usr and do not boot
> with initramfs because I build fully monolithic kernels and that makes the
> resulting vmlinux images run up against hard size limits in the SGI PROM
> (firmware, BIOS, etc) on these machines if I try to pack too large of an
> initramfs in.  I can check for any software that may be switched over soon
> to a hard initramfs requirement and look at my options.

One group of packages involved in this is any package that calls
gen_usr_ldscript. We have this function for a couple of reasons.

Gentoo has a policy that bans *.a static libraries from being
installed in /lib*. I think this policy is unique to Gentoo,
because Most build systems I've seen do not
have separate paths for static vs dynamic libraries, so all libraries
from a package are installed in /usr/lib* or /lib*. This policy was
originally hard coded into portage some time back (I can find the commit
if you want it) before it was made a formal policy by the qa team.
It has since become a formal policy.

Because of this policy, we have to tell upstream build systems to
install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
have separate usr with no initramfs. In that case, you have binaries on
/ that link to shared libraries on /usr. When we saw this happening, we
started manually moving shared libraries from /usr/lib* to /lib* if
someone needed the package at boot time. Then we hit bug 4411 [1]
where the linker was prioritizing static libraries in /usr/lib* over shared
libraries in /lib*. 

I don't know if we tried to address that upstream or not, but ultimately
gen_usr_ldscript came out of it.
 
Several folks have wanted to get rid of this function for years [2].

I have looked into it before, and there are several things that can be done.

We can have packages that currently build static libraries and
use gen_usr_ldscript stop building static libraries and use the
appropriate setting in their build systems to install libraries in
/$(get_libdir). This is the path OpenRC is taking per request of the qa
lead. The next release will not support the static-libs use flag. This
will actively break anyone who is expecting this use flag to exist for OpenRC.

If a package does not build static libraries in the first place, there is
really no reason  to call gen_usr_ldscript; the ebuild should be using
the upstream build system to put the libraries where they need to be.

If static libraries are needed for a package that is using
gen_usr_ldscript to move shared libraries to /lib*, the only option you
have is to drop the gen_usr_ldscript call. If you do that, all of the
libraries will stay in /usr/lib*, but as soon as one package takes this
path, there will be active breakage for someone who is using a separate
/usr without an initramfs.

The most controversial thing to do would be the /usr merge. It would
affect far more users than the other paths for getting rid of
gen_usr_ldscript, but it would make all "separate /usr" concerns go
away along with giving us a number of other benefits.

The short version is, anything we do to remove the gen_usr_ldscript
function will actively break some group of users.

William

[1] https://bugs.gentoo.org/4411
[2] https://bugs.gentoo.org/417451

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Rich Freeman]

On Sat, Mar 27, 2021 at 5:43 PM Joshua Kinard <kumba@gentoo.org> wrote:
>
> I kinda wish the Linux kernel had an ability to partially boot, init the
> networking subsystem, then fetch an initramfs image over TFTP like it can do
> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
> install netboots better).  I've dug around, but this does not seem to be a
> capability currently in the kernel, unless I have over looked something.

Support exists, in the form of an initramfs.  Bear with me for a second.

An initramfs does not need to be large.  It is just software.  Sure,
they're often implemented in bash with all sorts of userspace tools in
the image, and that uses lots of space.  However, an initramfs could
be nothing but a small compiled binary and nothing else - a few
kilobytes even.

I'm not aware if anything like this has already been created, but you
could do something reminiscent of coreboot, and it would be
distro-agnostic.  Just build a minimal kernel with only support for
whatever you need to read a kernel and initramfs (from disk, network,
whatever).  Then build an initramfs that minimally fetches that kernel
- it could just be a tiny binary, or it could use tools like curl/etc.
Then you'd kexec the new kernel/initramfs which would do the full
boot, and this wouldn't have any size limitation.

An initramfs is just a way to automate the kernel boot process using
userspace code, vs needing to do a lot of fancy stuff in kernel space.
They can get a bit bloated when you make them full-featured/generic,
but if you just want a secondary bootloader you can shrink both the
kernel and the initramfs considerably, and use that to boot another
kernel.

If I were to look anywhere for something I could use out-of-the-box it
would be with coreboot.  That is a linux-based kernel/initramfs
designed to be a bootloader really.

-- 
Rich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/27/2021 21:00, Rich Freeman wrote:
> On Sat, Mar 27, 2021 at 5:43 PM Joshua Kinard <kumba@gentoo.org> wrote:
>>
>> I kinda wish the Linux kernel had an ability to partially boot, init the
>> networking subsystem, then fetch an initramfs image over TFTP like it can do
>> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
>> install netboots better).  I've dug around, but this does not seem to be a
>> capability currently in the kernel, unless I have over looked something.
> 
> Support exists, in the form of an initramfs.  Bear with me for a second.
> 
> An initramfs does not need to be large.  It is just software.  Sure,
> they're often implemented in bash with all sorts of userspace tools in
> the image, and that uses lots of space.  However, an initramfs could
> be nothing but a small compiled binary and nothing else - a few
> kilobytes even.
> 
> I'm not aware if anything like this has already been created, but you
> could do something reminiscent of coreboot, and it would be
> distro-agnostic.  Just build a minimal kernel with only support for
> whatever you need to read a kernel and initramfs (from disk, network,
> whatever).  Then build an initramfs that minimally fetches that kernel
> - it could just be a tiny binary, or it could use tools like curl/etc.
> Then you'd kexec the new kernel/initramfs which would do the full
> boot, and this wouldn't have any size limitation.
> 
> An initramfs is just a way to automate the kernel boot process using
> userspace code, vs needing to do a lot of fancy stuff in kernel space.
> They can get a bit bloated when you make them full-featured/generic,
> but if you just want a secondary bootloader you can shrink both the
> kernel and the initramfs considerably, and use that to boot another
> kernel.
> 
> If I were to look anywhere for something I could use out-of-the-box it
> would be with coreboot.  That is a linux-based kernel/initramfs
> designed to be a bootloader really.

I believe kexec support is fairly new on MIPS, and I am not 100% certain it
works with the older SGI hardware.  Each SGI machine is not entirely
dissimilar from a unique sub-architecture, with their own memory map layouts
that are enforced at the PROM level.  I believe that kexec is picky about
where it stores or jumps into the newer kernel, and on the SGI machines,
that's going to be tricky, since there's only one or two FreeMemory() areas
available to load things into.

And it may not even work at all on the IP27 platform that is ccNUMA-aware
and can have memory scattered across multiple physical machine nodes (and I
lack the hardware and 240V plugs to even test this).  As such, this might be
an endeavor that has too much overhead to attempt.

As it stands, I currently only have 5.4 LTS kernels working on Octane (IP30)
and Origin/Onyx (IP27).  There were massive changes from 5.5 through 5.7
that deal with the mainlining of the SGI Octane code (and cleanups in the
IP27 PCI core) that I haven't had time to deal with yet, so I am quite a
ways behind on things.  Hope to take a crack at porting to 5.10 in the next
few months (which is why I am sticking to LTS kernels now -- mainline
kernels move too fast these days).

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/27/2021 20:32, William Hubbs wrote:
> On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
>> On 3/23/2021 07:31, Rich Freeman wrote:
>>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
>>>>
>>>>>> Council decided years ago that we don't support separate /usr without
>>>>>> an initramfs, but we haven't completed that transition yet.
>>>>>
>>>>> Which doesn't imply that we deliberately break things.
>>>>
>>>> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
>>>>
>>>
>>> Just to clarify - it is already unsupported at a distro level.  It is
>>> just that some individual packages still work with it.
>>>
>>> The current Council decisions on the issue are (just providing for
>>> general reference):
>>>
>>> - "Since that particular setup may already be subtly broken today
>>>   depending on the installed software, Council recommends using an
>>>   early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
>>>   is on a separate partition."
>>>   Accepted unanimously. [1]
>>>
>>> - "The intention is to eventually not require maintainers to support
>>>   a separate /usr without an early boot mechanism once the Council
>>>   agrees that the necessary docs/migration path is in place."
>>>   Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
>>>
>>> - "The Council agrees that all preparations for dropping support for
>>>   separate /usr without an initramfs or similar boot mechanism are
>>>   complete. A news item will be prepared, and users will be given one
>>>   month to switch after the news item has been sent."
>>>   Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
>>>
>>> Current policy documentation:
>>> Developers are not required to support using separate /usr filesystem
>>> without an initramfs. [3]
>>>
>>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
>>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
>>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
>>
>> Is there a list of software/ebuilds that currently do this "subtle" handling
>> of separate /usr w/o initramfs?
>>
>> I've got just my MIPS systems left that use a separate /usr and do not boot
>> with initramfs because I build fully monolithic kernels and that makes the
>> resulting vmlinux images run up against hard size limits in the SGI PROM
>> (firmware, BIOS, etc) on these machines if I try to pack too large of an
>> initramfs in.  I can check for any software that may be switched over soon
>> to a hard initramfs requirement and look at my options.
> 
> One group of packages involved in this is any package that calls
> gen_usr_ldscript. We have this function for a couple of reasons.
> 
> Gentoo has a policy that bans *.a static libraries from being
> installed in /lib*. I think this policy is unique to Gentoo,
> because Most build systems I've seen do not
> have separate paths for static vs dynamic libraries, so all libraries
> from a package are installed in /usr/lib* or /lib*. This policy was
> originally hard coded into portage some time back (I can find the commit
> if you want it) before it was made a formal policy by the qa team.
> It has since become a formal policy.
> 
> Because of this policy, we have to tell upstream build systems to
> install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
> have separate usr with no initramfs. In that case, you have binaries on
> / that link to shared libraries on /usr. When we saw this happening, we
> started manually moving shared libraries from /usr/lib* to /lib* if
> someone needed the package at boot time. Then we hit bug 4411 [1]
> where the linker was prioritizing static libraries in /usr/lib* over shared
> libraries in /lib*. 
> 
> I don't know if we tried to address that upstream or not, but ultimately
> gen_usr_ldscript came out of it.
>  
> Several folks have wanted to get rid of this function for years [2].
> 
> I have looked into it before, and there are several things that can be done.
> 
> We can have packages that currently build static libraries and
> use gen_usr_ldscript stop building static libraries and use the
> appropriate setting in their build systems to install libraries in
> /$(get_libdir). This is the path OpenRC is taking per request of the qa
> lead. The next release will not support the static-libs use flag. This
> will actively break anyone who is expecting this use flag to exist for OpenRC.
> 
> If a package does not build static libraries in the first place, there is
> really no reason  to call gen_usr_ldscript; the ebuild should be using
> the upstream build system to put the libraries where they need to be.

The number of packages calling gen_usr_ldscript looks to be fairly small.
Creating a TRACKER bug and sub-bugs for checking or removing the need for
gen_usr_ldscript might be a way to go to pair the list down and reduce the
footprint:

app-accessibility/brltty        sys-apps/util-linux
app-arch/xz-utils               sys-apps/systemd
app-arch/bzip2                  sys-apps/dmapi
dev-libs/libiconv               sys-auth/skey
dev-libs/libaio                 sys-fs/reiserfsprogs
dev-libs/libpcre2               sys-fs/zfs
dev-libs/libedit                sys-fs/sysfsutils
dev-libs/lzo                    sys-fs/udev
dev-libs/libusb-compat          sys-fs/e2fsprogs
dev-libs/libpwquality           sys-fs/xfsprogs
dev-libs/libintl                sys-fs/reiser4progs
dev-libs/libusb                 sys-libs/gpm
dev-libs/expat                  sys-libs/libcap
dev-libs/libpcre                sys-libs/pam
eclass/usr-ldscript.eclass      sys-libs/ncurses
eclass/toolchain-funcs.eclass   sys-libs/glibc
net-firewall/iptables           sys-libs/pwdb
net-libs/libnftnl               sys-libs/cracklib
net-libs/libtirpc               sys-libs/libaal
net-libs/libmnl                 sys-libs/zlib
sys-apps/keyutils               sys-libs/readline
sys-apps/acl                    sys-libs/e2fsprogs-libs
sys-apps/openrc                 sys-process/audit
sys-apps/attr                   sys-process/procps
sys-apps/tcp-wrappers


> If static libraries are needed for a package that is using
> gen_usr_ldscript to move shared libraries to /lib*, the only option you
> have is to drop the gen_usr_ldscript call. If you do that, all of the
> libraries will stay in /usr/lib*, but as soon as one package takes this
> path, there will be active breakage for someone who is using a separate
> /usr without an initramfs.

Correct me if wrong, but static libraries are only needed during
compilation, right?  The *.a files are merged into the binary at link time
and thus that binary can stand on its own regardless of whether the *.a
files are available or not.  They're not like shared libs which are needed
by the loader to resolve symbols at run time.

A scenario where the *.a static libs aren't available because such a /usr is
unmounted, but are needed for some function, would be an anomalous system
state in the first place.  That would imply one was trying to do something
like executing part of the system toolchain, which should be impossible in
that case because the actual binaries for gcc, ld, as, and others live on /usr.

We're mostly talking about the small window during boot where, on a system
with /usr on a separate disk partition, /usr might not be available until
some userspace tool in /bin or /sbin runs to make it available.  Running the
system compiler during boot would be a rare and very unique scenario (not to
mention a sign of questionable development processes).


> The most controversial thing to do would be the /usr merge. It would
> affect far more users than the other paths for getting rid of
> gen_usr_ldscript, but it would make all "separate /usr" concerns go
> away along with giving us a number of other benefits.
> 
> The short version is, anything we do to remove the gen_usr_ldscript
> function will actively break some group of users.

If I read the temperature right, it seems like there is desire to eliminate
gen_usr_ldscript regardless of sep-usr or not.  If so, then that seems the
way to go.  Make the eventual /usr merge a separate issue to tackle some
time further down the road.


> William
> 
> [1] https://bugs.gentoo.org/4411
> [2] https://bugs.gentoo.org/417451
> 


-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 2699 bytes --]

On Sat, 27 Mar 2021 18:40:52 -0400
Joshua Kinard <kumba@gentoo.org> wrote:

> On 3/27/2021 18:16, James Le Cuirot wrote:
> > On Sat, 27 Mar 2021 17:43:34 -0400
> > Joshua Kinard <kumba@gentoo.org> wrote:
> >   
> >> I kinda wish the Linux kernel had an ability to partially boot, init the
> >> networking subsystem, then fetch an initramfs image over TFTP like it can do
> >> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
> >> install netboots better).  I've dug around, but this does not seem to be a
> >> capability currently in the kernel, unless I have over looked something.
> >>
> >> Otherwise in the future, I may just have to setup an initramfs into an NFS
> >> Root and teach the SGI's to somehow deal with it.  Which all still seems
> >> unnecessarily complicated because some other distro thinks it knows what's
> >> best for everyone else (but I digress...).  
> > 
> > NBD may be a slightly simpler alternative and a bit more like an
> > initramfs. nbdkit can do all sorts of weird things. I thought it might
> > have a plugin for cpio archives, allowing you to use a regular
> > initramfs generated by Dracut or similar directly. It doesn't appear to
> > but plugins are quite easy to write. Alternatively you could just
> > extract the initramfs and use nbdkit-linuxdisk-plugin.  
> 
> Can NBD be used like I described?  Never played with it before.  The MIPS
> machine has functioning local disk drives, and currently, it boots fine by
> just pulling a kernel off my TFTP server and booting from the local drive,
> no initramfs needed because I compiled everything into it.  If we ever force
> sep-usr to end, then I'll need a way to teach it to mount /usr before
> dropping into /bin/init (and nothing else).

Apologies, I went to experiment with this idea but I'd forgotten that
booting over NBD is not a built-in kernel feature and requires... an
initramfs. NFS looks like the only option with this general approach.

Rich is right though, the initramfs can be tiny. Dracut images are
heavy because they are very flexible and pack a lot of features but
none of that is required.

The kexec idea would be a nice bonus for you, if it works. Limiting
what kernel features you can enable must be frustrating.

It's also worth bearing in mind that you could just generate an image
with Dracut and extract it to a disk partition. That may seem a little
pointless if you're not using something like LVM or encryption but I
don't see the point in separate /usr either. ;) I'm guessing you don't
want to change your partition layout at this point though.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 8614 bytes --]

On Sat, Mar 27, 2021 at 10:51:11PM -0400, Joshua Kinard wrote:
> On 3/27/2021 20:32, William Hubbs wrote:
> > On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
> >> On 3/23/2021 07:31, Rich Freeman wrote:
> >>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
> >>>>
> >>>>>> Council decided years ago that we don't support separate /usr without
> >>>>>> an initramfs, but we haven't completed that transition yet.
> >>>>>
> >>>>> Which doesn't imply that we deliberately break things.
> >>>>
> >>>> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
> >>>>
> >>>
> >>> Just to clarify - it is already unsupported at a distro level.  It is
> >>> just that some individual packages still work with it.
> >>>
> >>> The current Council decisions on the issue are (just providing for
> >>> general reference):
> >>>
> >>> - "Since that particular setup may already be subtly broken today
> >>>   depending on the installed software, Council recommends using an
> >>>   early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
> >>>   is on a separate partition."
> >>>   Accepted unanimously. [1]
> >>>
> >>> - "The intention is to eventually not require maintainers to support
> >>>   a separate /usr without an early boot mechanism once the Council
> >>>   agrees that the necessary docs/migration path is in place."
> >>>   Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
> >>>
> >>> - "The Council agrees that all preparations for dropping support for
> >>>   separate /usr without an initramfs or similar boot mechanism are
> >>>   complete. A news item will be prepared, and users will be given one
> >>>   month to switch after the news item has been sent."
> >>>   Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
> >>>
> >>> Current policy documentation:
> >>> Developers are not required to support using separate /usr filesystem
> >>> without an initramfs. [3]
> >>>
> >>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
> >>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
> >>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
> >>
> >> Is there a list of software/ebuilds that currently do this "subtle" handling
> >> of separate /usr w/o initramfs?
> >>
> >> I've got just my MIPS systems left that use a separate /usr and do not boot
> >> with initramfs because I build fully monolithic kernels and that makes the
> >> resulting vmlinux images run up against hard size limits in the SGI PROM
> >> (firmware, BIOS, etc) on these machines if I try to pack too large of an
> >> initramfs in.  I can check for any software that may be switched over soon
> >> to a hard initramfs requirement and look at my options.
> > 
> > One group of packages involved in this is any package that calls
> > gen_usr_ldscript. We have this function for a couple of reasons.
> > 
> > Gentoo has a policy that bans *.a static libraries from being
> > installed in /lib*. I think this policy is unique to Gentoo,
> > because Most build systems I've seen do not
> > have separate paths for static vs dynamic libraries, so all libraries
> > from a package are installed in /usr/lib* or /lib*. This policy was
> > originally hard coded into portage some time back (I can find the commit
> > if you want it) before it was made a formal policy by the qa team.
> > It has since become a formal policy.
> > 
> > Because of this policy, we have to tell upstream build systems to
> > install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
> > have separate usr with no initramfs. In that case, you have binaries on
> > / that link to shared libraries on /usr. When we saw this happening, we
> > started manually moving shared libraries from /usr/lib* to /lib* if
> > someone needed the package at boot time. Then we hit bug 4411 [1]
> > where the linker was prioritizing static libraries in /usr/lib* over shared
> > libraries in /lib*. 
> > 
> > I don't know if we tried to address that upstream or not, but ultimately
> > gen_usr_ldscript came out of it.
> >  
> > Several folks have wanted to get rid of this function for years [2].
> > 
> > I have looked into it before, and there are several things that can be done.
> > 
> > We can have packages that currently build static libraries and
> > use gen_usr_ldscript stop building static libraries and use the
> > appropriate setting in their build systems to install libraries in
> > /$(get_libdir). This is the path OpenRC is taking per request of the qa
> > lead. The next release will not support the static-libs use flag. This
> > will actively break anyone who is expecting this use flag to exist for OpenRC.
> > 
> > If a package does not build static libraries in the first place, there is
> > really no reason  to call gen_usr_ldscript; the ebuild should be using
> > the upstream build system to put the libraries where they need to be.
> 
> The number of packages calling gen_usr_ldscript looks to be fairly small.
> Creating a TRACKER bug and sub-bugs for checking or removing the need for
> gen_usr_ldscript might be a way to go to pair the list down and reduce the
> footprint:
> 

Yes, I can start this work.

> 
> > If static libraries are needed for a package that is using
> > gen_usr_ldscript to move shared libraries to /lib*, the only option you
> > have is to drop the gen_usr_ldscript call. If you do that, all of the
> > libraries will stay in /usr/lib*, but as soon as one package takes this
> > path, there will be active breakage for someone who is using a separate
> > /usr without an initramfs.
> 
> Correct me if wrong, but static libraries are only needed during
> compilation, right?  The *.a files are merged into the binary at link time
> and thus that binary can stand on its own regardless of whether the *.a
> files are available or not.  They're not like shared libs which are needed
> by the loader to resolve symbols at run time.
 
You are correct, and actually in Gentoo we prefer not to link to static
libraries, so we build them but we do not use them.

*snip*

> We're mostly talking about the small window during boot where, on a system
> with /usr on a separate disk partition, /usr might not be available until
> some userspace tool in /bin or /sbin runs to make it available.  Running the
> system compiler during boot would be a rare and very unique scenario (not to
> mention a sign of questionable development processes).

The problem is, there's a chicken-and-egg problem in the scenario where
/ and /usr are on separate partitions, and this is why a number of linux
distros have moved to requiring an initramfs in this situation.
I'm linking systemd's description here, only because it is the best
writeup of the issue I've found [1].
Anything that is needed in the early boot process requires all of its libraries,
dependent libraries, binaries, data files, etc to be on /, and this has
become a moving target.

> > The most controversial thing to do would be the /usr merge. It would
> > affect far more users than the other paths for getting rid of
> > gen_usr_ldscript, but it would make all "separate /usr" concerns go
> > away along with giving us a number of other benefits.
> > 
> > The short version is, anything we do to remove the gen_usr_ldscript
> > function will actively break some group of users.
> 
> If I read the temperature right, it seems like there is desire to eliminate
> gen_usr_ldscript regardless of sep-usr or not.  If so, then that seems the
> way to go.  Make the eventual /usr merge a separate issue to tackle some
> time further down the road.

The way I see it, when we start to remove the gen_usr_ldscript calls,
people using a sep-usr mount without an initramfs will run into one or
both of these issues:

- they might have to increase the size of their root partition depending
  on what gets added to /lib*
- if one package in that list drops gen_usr_ldscript without installing
  libraries in /lib*, it will mean they need an initramfs.

From my pov, it is better to start using initramfs in that situation,
and if you do that you fix all of the other issues mentioned on
the page linked above. It feels like the usr merge is less of a big deal
once we know everyone who has /usr separate is using an initramfs.

William

[1] https://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/28/2021 16:05, William Hubbs wrote:
> On Sat, Mar 27, 2021 at 10:51:11PM -0400, Joshua Kinard wrote:
>> On 3/27/2021 20:32, William Hubbs wrote:
>>> On Sat, Mar 27, 2021 at 05:43:34PM -0400, Joshua Kinard wrote:
>>>> On 3/23/2021 07:31, Rich Freeman wrote:
>>>>> On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel <dilfridge@gentoo.org> wrote:
>>>>>>
>>>>>>>> Council decided years ago that we don't support separate /usr without
>>>>>>>> an initramfs, but we haven't completed that transition yet.
>>>>>>>
>>>>>>> Which doesn't imply that we deliberately break things.
>>>>>>
>>>>>> That's right. Though we should at some point start thinking about an end of support for separate usr without initramfs.
>>>>>>
>>>>>
>>>>> Just to clarify - it is already unsupported at a distro level.  It is
>>>>> just that some individual packages still work with it.
>>>>>
>>>>> The current Council decisions on the issue are (just providing for
>>>>> general reference):
>>>>>
>>>>> - "Since that particular setup may already be subtly broken today
>>>>>   depending on the installed software, Council recommends using an
>>>>>   early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
>>>>>   is on a separate partition."
>>>>>   Accepted unanimously. [1]
>>>>>
>>>>> - "The intention is to eventually not require maintainers to support
>>>>>   a separate /usr without an early boot mechanism once the Council
>>>>>   agrees that the necessary docs/migration path is in place."
>>>>>   Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]
>>>>>
>>>>> - "The Council agrees that all preparations for dropping support for
>>>>>   separate /usr without an initramfs or similar boot mechanism are
>>>>>   complete. A news item will be prepared, and users will be given one
>>>>>   month to switch after the news item has been sent."
>>>>>   Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]
>>>>>
>>>>> Current policy documentation:
>>>>> Developers are not required to support using separate /usr filesystem
>>>>> without an initramfs. [3]
>>>>>
>>>>> 1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
>>>>> 2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
>>>>> 3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202
>>>>
>>>> Is there a list of software/ebuilds that currently do this "subtle" handling
>>>> of separate /usr w/o initramfs?
>>>>
>>>> I've got just my MIPS systems left that use a separate /usr and do not boot
>>>> with initramfs because I build fully monolithic kernels and that makes the
>>>> resulting vmlinux images run up against hard size limits in the SGI PROM
>>>> (firmware, BIOS, etc) on these machines if I try to pack too large of an
>>>> initramfs in.  I can check for any software that may be switched over soon
>>>> to a hard initramfs requirement and look at my options.
>>>
>>> One group of packages involved in this is any package that calls
>>> gen_usr_ldscript. We have this function for a couple of reasons.
>>>
>>> Gentoo has a policy that bans *.a static libraries from being
>>> installed in /lib*. I think this policy is unique to Gentoo,
>>> because Most build systems I've seen do not
>>> have separate paths for static vs dynamic libraries, so all libraries
>>> from a package are installed in /usr/lib* or /lib*. This policy was
>>> originally hard coded into portage some time back (I can find the commit
>>> if you want it) before it was made a formal policy by the qa team.
>>> It has since become a formal policy.
>>>
>>> Because of this policy, we have to tell upstream build systems to
>>> install libraries in ${ED}/usr/$(get_libdir). This is fine unless you
>>> have separate usr with no initramfs. In that case, you have binaries on
>>> / that link to shared libraries on /usr. When we saw this happening, we
>>> started manually moving shared libraries from /usr/lib* to /lib* if
>>> someone needed the package at boot time. Then we hit bug 4411 [1]
>>> where the linker was prioritizing static libraries in /usr/lib* over shared
>>> libraries in /lib*. 
>>>
>>> I don't know if we tried to address that upstream or not, but ultimately
>>> gen_usr_ldscript came out of it.
>>>  
>>> Several folks have wanted to get rid of this function for years [2].
>>>
>>> I have looked into it before, and there are several things that can be done.
>>>
>>> We can have packages that currently build static libraries and
>>> use gen_usr_ldscript stop building static libraries and use the
>>> appropriate setting in their build systems to install libraries in
>>> /$(get_libdir). This is the path OpenRC is taking per request of the qa
>>> lead. The next release will not support the static-libs use flag. This
>>> will actively break anyone who is expecting this use flag to exist for OpenRC.
>>>
>>> If a package does not build static libraries in the first place, there is
>>> really no reason  to call gen_usr_ldscript; the ebuild should be using
>>> the upstream build system to put the libraries where they need to be.
>>
>> The number of packages calling gen_usr_ldscript looks to be fairly small.
>> Creating a TRACKER bug and sub-bugs for checking or removing the need for
>> gen_usr_ldscript might be a way to go to pair the list down and reduce the
>> footprint:
>>
> 
> Yes, I can start this work.
> 
>>
>>> If static libraries are needed for a package that is using
>>> gen_usr_ldscript to move shared libraries to /lib*, the only option you
>>> have is to drop the gen_usr_ldscript call. If you do that, all of the
>>> libraries will stay in /usr/lib*, but as soon as one package takes this
>>> path, there will be active breakage for someone who is using a separate
>>> /usr without an initramfs.
>>
>> Correct me if wrong, but static libraries are only needed during
>> compilation, right?  The *.a files are merged into the binary at link time
>> and thus that binary can stand on its own regardless of whether the *.a
>> files are available or not.  They're not like shared libs which are needed
>> by the loader to resolve symbols at run time.
>  
> You are correct, and actually in Gentoo we prefer not to link to static
> libraries, so we build them but we do not use them.
> 
> *snip*
> 
>> We're mostly talking about the small window during boot where, on a system
>> with /usr on a separate disk partition, /usr might not be available until
>> some userspace tool in /bin or /sbin runs to make it available.  Running the
>> system compiler during boot would be a rare and very unique scenario (not to
>> mention a sign of questionable development processes).
> 
> The problem is, there's a chicken-and-egg problem in the scenario where
> / and /usr are on separate partitions, and this is why a number of linux
> distros have moved to requiring an initramfs in this situation.
> I'm linking systemd's description here, only because it is the best
> writeup of the issue I've found [1].
> Anything that is needed in the early boot process requires all of its libraries,
> dependent libraries, binaries, data files, etc to be on /, and this has
> become a moving target.

Yeah, I've read systemd's explanation, and generally disagree with it.  They
created the problem in the first place, then invented their own solution for
it, and now everyone acts like they're the wise men on the mountain for it.

I still don't see the connection to the static *.a libs and whether they're
on /lib or /usr/lib, though.  Unless we're implying that where the *.a's go,
so too do the *.so's go, then THAT makes sense because *.so's ARE needed at
program runtime, whereas *.a's are not.

I wonder if we couldn't shovel all static libs off to a dedicated folder
somewhere, like '/usr/lib/static/<pkg>/*.a', similar to the way debug files
are now consolidated under '/usr/lib/debug'.  Since they're only needed
during a specific kind of compilation that we don't support out-of-the-box
that happens long after the system is fully booted, stuffing them off
somewhere unimportant would make some sense.  Most modern software should be
using shared libs by default, and if it ain't, that's either a bug or that
software is for a very specific function (like a bootloader).


>>> The most controversial thing to do would be the /usr merge. It would
>>> affect far more users than the other paths for getting rid of
>>> gen_usr_ldscript, but it would make all "separate /usr" concerns go
>>> away along with giving us a number of other benefits.
>>>
>>> The short version is, anything we do to remove the gen_usr_ldscript
>>> function will actively break some group of users.
>>
>> If I read the temperature right, it seems like there is desire to eliminate
>> gen_usr_ldscript regardless of sep-usr or not.  If so, then that seems the
>> way to go.  Make the eventual /usr merge a separate issue to tackle some
>> time further down the road.
> 
> The way I see it, when we start to remove the gen_usr_ldscript calls,
> people using a sep-usr mount without an initramfs will run into one or
> both of these issues:
> 
> - they might have to increase the size of their root partition depending
>   on what gets added to /lib*
> - if one package in that list drops gen_usr_ldscript without installing
>   libraries in /lib*, it will mean they need an initramfs.

I tend to make my root partitions ~4GB, which has often been plenty of room
for well over 15 years.  But again, location of the *.a static libs is
irrelevant during system boot.  They are not needed nor referenced when a
program executes.  A statically-compiled program has all of its dependencies
lumped inside of it, so you could put it pretty much anywhere on the
filesystem and run it (ignoring for a moment 'noexec' potentially being
set).  Or even more to the point, you could e-mail a static binary to
someone else on a different distro and it'll probably execute just fine.

The important bit here is certain software packages defaulting their install
into /usr that are needed to bring the system up (like encryption, lvm,
etc).  It's when using those setups that you pretty much have to use an
initramfs, sometimes regardless if you have /usr on another partition or not.


> From my pov, it is better to start using initramfs in that situation,
> and if you do that you fix all of the other issues mentioned on
> the page linked above. It feels like the usr merge is less of a big deal
> once we know everyone who has /usr separate is using an initramfs.

Yeah, and the problem there has been the generation of those initramfs
archives.  I'm stuck between a rock and a hard place because the classic
tools like dracut target popular archs like amd64 or arm64, but not mips,
and especially not 20+ year-old hardware, and go on to make assumptions
under that pretense.  Luckily, my MIPS machine setups are simple and really
only rely on mdraid auto-assembling via 0.90 metadata (otherwise, I'd need
an initramfs to get the things to boot, cause someone decided at some point
that it ain't the kernel's job assemble the array, and newer metadata
signatures enforce this...).

Add on to that, my specific use-case, I really just need the tiniest binary
that tells the kernel to go and mount /usr as an XFS partition on /dev/md2
(after array assembly).  No one seems to have cobbled together an initramfs
generator that simple.  Instead, the Linux world went straight for the Rube
Goldberg versions of initramfs.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Joshua Kinard]

On 3/28/2021 05:25, James Le Cuirot wrote:> On Sat, 27 Mar 2021 18:40:52
-0400 Joshua Kinard <kumba@gentoo.org>
> wrote:
>
>> On 3/27/2021 18:16, James Le Cuirot wrote:
>>> On Sat, 27 Mar 2021 17:43:34 -0400 Joshua Kinard <kumba@gentoo.org>
>>> wrote:
>>>
>>>> I kinda wish the Linux kernel had an ability to partially boot,
>>>> init the networking subsystem, then fetch an initramfs image over
>>>> TFTP like it can do with NFS Root.  That would solve the problem
>>>> on my MIPS system(s) (and make install netboots better).  I've dug
>>>> around, but this does not seem to be a capability currently in the
>>>> kernel, unless I have over looked something.
>>>>
>>>> Otherwise in the future, I may just have to setup an initramfs
>>>> into an NFS Root and teach the SGI's to somehow deal with it.
>>>> Which all still seems unnecessarily complicated because some other
>>>> distro thinks it knows what's best for everyone else (but I
>>>> digress...).
>>>>
>>>
>>> NBD may be a slightly simpler alternative and a bit more like an
>>> initramfs. nbdkit can do all sorts of weird things. I thought it
>>> might have a plugin for cpio archives, allowing you to use a regular
>>>  initramfs generated by Dracut or similar directly. It doesn't
>>> appear to but plugins are quite easy to write. Alternatively you
>>> could just extract the initramfs and use nbdkit-linuxdisk-plugin.
>>
>> Can NBD be used like I described?  Never played with it before.  The
>> MIPS machine has functioning local disk drives, and currently, it
>> boots fine by just pulling a kernel off my TFTP server and booting from
>> the local drive, no initramfs needed because I compiled everything into
>> it. If we ever force sep-usr to end, then I'll need a way to teach it
>> to mount /usr before dropping into /bin/init (and nothing else).
>
> Apologies, I went to experiment with this idea but I'd forgotten that
> booting over NBD is not a built-in kernel feature and requires... an
> initramfs. NFS looks like the only option with this general approach.

I've kinda come to this conclusion myself.  I could probably host an
extracted, micro-initramfs on my NFS server that would be loaded by the
kernel to jump to $REAL_ROOT.  Not *too* much of an issue on the Octane,
because I can store the kernel command line args in a PROM variable so that
all I have to do is type "$foo" at the PROM prompt to load something.  But,
SGI did their best to be inconsistent, and IP27 systems cannot permanently
save variables to NVRAM.  Once you power cycle, all user-defined PROM vars
are lost.  And Linux's NFS Root command args are somewhat complicated from
what I remember.  Upshot is I boot the IP27 over serial console, so I can
just save bootp() args in a text file on my desktop and paste it into the
console for those instances.

In any event, I kinda wish there was some way to just boot the kernel to a
point, have it copy an initramfs image via TFTP, load it, run it, jump to
$REAL_ROOT.  If I had the time, maybe I could put something together like
that myself, but time is not something I have much of right now.


> Rich is right though, the initramfs can be tiny. Dracut images are heavy
> because they are very flexible and pack a lot of features but none of
> that is required.
>
> The kexec idea would be a nice bonus for you, if it works. Limiting what
> kernel features you can enable must be frustrating.

My primary kernel image for daily use on the Octane isn't terribly limited.
 The final vmlinux output is ~39MB, due to loading debugging symbols in.
But the cutoff in size for Octane seems to be somewhere around 43-45MB,
before the PROM errors out due to no available contiguous FreeMemory() areas
(the mem region defined by the ARCS PROM where stuff can be loaded and
executed).  Netboot kernels for installs can creep up there if you go too
crazy on kernel features and pack the netboot with supporting packages
(mainly filesystem tools).  This already impacts older systems, like SGI
Indy, which cannot boot current MIPS netboot images because they have
smaller FreeMemory() areas.

I thought switching to musl would make the netboots smaller, but that oddly
didn't work and musl netboots are about ~2MB larger than the older uclibc-ng
versions.  And all of my uclibc-ng stages have become unusable for some
unknown reason.  As soon as you rebuild ncurses in one, all binaries linked
to it throws SIGSEGVs (which kills portage).  glibc-based netboots are too
big and will not load at all, even on the Octane.


> It's also worth bearing in mind that you could just generate an image
> with Dracut and extract it to a disk partition. That may seem a little
> pointless if you're not using something like LVM or encryption but I
> don't see the point in separate /usr either. ;) I'm guessing you don't
> want to change your partition layout at this point though.

No, no LUKS, encryption, btrfs, etc.  Just mdraid w/ 0.90 metadata for
auto-assemble.  Which is why I hate having to use an initramfs for something
so simple.  Unnecessary complexity annoys me (unless you're working on a
non-functional art piece).

In any event, I've already rebuilt the system from a RAID5 down to a RAID1,
so re-doing that to merge / and /usr would not be difficult when the time comes.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
rsa6144/5C63F4E3F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[William Hubbs]

[-- Attachment #1: Type: text/plain, Size: 6565 bytes --]

This is getting really long, so I'm going to remove more things I'm not
answering directly.

On Sun, Mar 28, 2021 at 07:31:02PM -0400, Joshua Kinard wrote:
> > The problem is, there's a chicken-and-egg problem in the scenario where
> > / and /usr are on separate partitions, and this is why a number of linux
> > distros have moved to requiring an initramfs in this situation.
> > I'm linking systemd's description here, only because it is the best
> > writeup of the issue I've found [1].
> > Anything that is needed in the early boot process requires all of its libraries,
> > dependent libraries, binaries, data files, etc to be on /, and this has
> > become a moving target.
> 
> Yeah, I've read systemd's explanation, and generally disagree with it.  They
> created the problem in the first place, then invented their own solution for
> it, and now everyone acts like they're the wise men on the mountain for it.

http://lists.busybox.net/pipermail/busybox/2010-December/074114.html

Rob Landley had a lot to say about how linux should be using initramfs
to handle early boot long before systemd came  along, so this has been
an issue a lot longer than systemd; the systemd guys just amplified it.

> I still don't see the connection to the static *.a libs and whether they're
> on /lib or /usr/lib, though.  Unless we're implying that where the *.a's go,
> so too do the *.so's go, then THAT makes sense because *.so's ARE needed at
> program runtime, whereas *.a's are not.

It is a linker behavior issue we discovered and  worked around.

In the past, Gentoo has gone an extra mile to make some things useable
by people who have separate /usr without an initramfs by moving only
shared libraries to /lib*. We leave everything else that would be in
libdir in the default location which is /usr/lib*. This includes things
like static libs, pkgconfig files etc.

When we started doing this we found that the linker favors libraries in
/usr/lib* over libraries in /lib*, so if you are linking to libfoo and
there's a static version in /usr/lib* and a shared version in /lib* you
will link to the static version. That's what bug 4411 is about. 

gen_usr_ldscript gets around this by creating a linker script at the
original location of the shared library in /usr/lib* when it moves the
shared library to /lib*.
See /usr/lib64/libbz2.so for an example of the generated linker script.

Getting rid of gen_usr_ldscript would move the shared libraries back to
their upstream location (/usr/lib*) and remove the linker scripts. This
would also allow the removal of the usr-ldscript eclass and the
gen_usr_ldscript function from eutils.eclass.

The up side of this is that it allows us to get rid of some of our
custom code in ebuilds, and it is completely transparent to most of our
user base.

The down side is that if you are using separate /usr with no
initramfs, libraries that you were accessing in /lib* during early boot will
moved to /usr/lib* and not be available before /usr is mounted.
This would cause breakage until you reconfigured your system to boot
with an initramfs and mount / and /usr before jumping into the real
system.

> I wonder if we couldn't shovel all static libs off to a dedicated folder
> somewhere, like '/usr/lib/static/<pkg>/*.a', similar to the way debug files
> are now consolidated under '/usr/lib/debug'.  Since they're only needed
> during a specific kind of compilation that we don't support out-of-the-box
> that happens long after the system is fully booted, stuffing them off
> somewhere unimportant would make some sense.  Most modern software should be
> using shared libs by default, and if it ain't, that's either a bug or that
> software is for a very specific function (like a bootloader).

If you started creating separate static libs folders you would need one per
abi, so it would end up being pretty ugly. We don't build that many
static libraries right now, so I'm not sure it is worth moving them to
some other folder.

If we put the shared libs back in the upstream expected location
(/usr/lib*) we would eliminate the linker issue.

*snip*

> > The way I see it, when we start to remove the gen_usr_ldscript calls,
> > people using a sep-usr mount without an initramfs will run into one or
> > both of these issues:
> > 
> > - they might have to increase the size of their root partition depending
> >   on what gets added to /lib*
> > - if one package in that list drops gen_usr_ldscript without installing
> >   libraries in /lib*, it will mean they need an initramfs.
> 
> I tend to make my root partitions ~4GB, which has often been plenty of room
> for well over 15 years.  But again, location of the *.a static libs is
> irrelevant during system boot.  They are not needed nor referenced when a
> program executes.  A statically-compiled program has all of its dependencies
> lumped inside of it, so you could put it pretty much anywhere on the
> filesystem and run it (ignoring for a moment 'noexec' potentially being
> set).  Or even more to the point, you could e-mail a static binary to
> someone else on a different distro and it'll probably execute just fine.

I think I wasn't very clear. In the second bullet point above I meant
shared libraries, which leads to what you say below.

> The important bit here is certain software packages defaulting their install
> into /usr that are needed to bring the system up (like encryption, lvm,
> etc).  It's when using those setups that you pretty much have to use an
> initramfs, sometimes regardless if you have /usr on another partition or not.
 
 Correct, and this is the issue. Gentoo has done extra work in the past
 to make more things useable  without an initramfs, and a good part of that
 work is in the gen_usr_ldscript function.

If I start removing calls to that function, more shared libraries will
move back to /usr/lib*, breaking systems that use separate /usr without
initramfs.

If I override the build systems of these packages and install all of the
library info in /$(get_libdir), I would have to develop custom code to
move unnecessary things back to /usr/$(get_libdir). This would also be a
bandaid because separate /usr without initramfs would be impossibly
broken if we do the /usr merge eventually.

If I do go forward with removing gen_usr_ldscript, I want to be reasonable
about how I do it, so let me know what your thoughts are about how it should
happen.

Let me know what you think.

William


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 1035 bytes --]

On Sun, 28 Mar 2021 19:46:32 -0400
Joshua Kinard <kumba@gentoo.org> wrote:

> I've kinda come to this conclusion myself.  I could probably host an
> extracted, micro-initramfs on my NFS server that would be loaded by the
> kernel to jump to $REAL_ROOT.  Not *too* much of an issue on the Octane,
> because I can store the kernel command line args in a PROM variable so that
> all I have to do is type "$foo" at the PROM prompt to load something.  But,
> SGI did their best to be inconsistent, and IP27 systems cannot permanently
> save variables to NVRAM.  Once you power cycle, all user-defined PROM vars
> are lost.  And Linux's NFS Root command args are somewhat complicated from
> what I remember.  Upshot is I boot the IP27 over serial console, so I can
> just save bootp() args in a text file on my desktop and paste it into the
> console for those instances.

Have you seen CONFIG_CMDLINE? It lets you bake command line args into
the kernel image itself.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 275 bytes --]

On 2021-03-29 10:06, James Le Cuirot wrote:

> Have you seen CONFIG_CMDLINE? It lets you bake command line args into
> the kernel image itself.

And since 5.6, there is also bootconfig:

https://www.kernel.org/doc/html/latest/admin-guide/bootconfig.html

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Marek Szuba]

[-- Attachment #1.1: Type: text/plain, Size: 1114 bytes --]

On 2021-03-27 23:40, Joshua Kinard wrote:

> The MIPS machine has functioning local disk drives, and currently, it
> boots fine by just pulling a kernel off my TFTP server and booting
> from the local drive, no initramfs needed because I compiled
> everything into it.

Out of curiosity, if your kernel images already come from the TFTP 
servers why not simply put separate initramfs files there too?

> I wonder if there's a small C program out there that can call
> whatever the kernel functions are to mount a disk partition that
> could be embedded into a tiny initramfs, then pivot_root to
> $REAL_ROOT and run /bin/init?

You might be interested in this FOSDEM 2020 talk:

https://archive.fosdem.org/2020/schedule/event/ema_boot_linux_only/

Not exactly what you have asked for but the problem they are trying to 
solve is the same as yours - boot Linux on a system whose first-stage 
bootloader impose considerable size constraints. And since it uses kexec 
at its core, it's essentially what Rich has suggested - except it's 
already been (at least partially) done :-)

-- 
MS


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]