[gentoo-user] Gentoo chroot with old glibc

[gentoo-user] Gentoo chroot with old glibc

[Hervé Guillemet]

Hello,

I need to distribute some linux binaries and the one built with my 
up-to-date gentoo sytem won't run on distributions using older glibc.

My idea is too maintain a gentoo chroot dedicated for compiling my 
binaries which would (package.)mask recent versions of glibc and gcc 
ebuilds.

What's the better way to go ? If I start with some of the stage3 
available for download, I won't be able to downgrade the glibc.

Or do you have any suggestion for alternatives to this gentoo chroot ? 
(I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).

-- 
Hervé

Re: [gentoo-user] Gentoo chroot with old glibc

[Michael]

[-- Attachment #1: Type: text/plain, Size: 862 bytes --]

On Friday, 19 June 2020 22:19:39 BST Hervé Guillemet wrote:
> Hello,
> 
> I need to distribute some linux binaries and the one built with my
> up-to-date gentoo sytem won't run on distributions using older glibc.
> 
> My idea is too maintain a gentoo chroot dedicated for compiling my
> binaries which would (package.)mask recent versions of glibc and gcc
> ebuilds.
> 
> What's the better way to go ? If I start with some of the stage3
> available for download, I won't be able to downgrade the glibc.
> 
> Or do you have any suggestion for alternatives to this gentoo chroot ?
> (I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).

Once you chroot, you're in the chrooted env.  As long as you have a stage 3 
old enough to contain the requisite glibc, you should be good to go:

http://gentoo.osuosl.org/releases/amd64/

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Gentoo chroot with old glibc

[Hervé Guillemet]

Le 21/06/2020 à 19:06, Michael a écrit :
>> I need to distribute some linux binaries and the one built with my
>> up-to-date gentoo sytem won't run on distributions using older glibc.
>>
>> My idea is too maintain a gentoo chroot dedicated for compiling my
>> binaries which would (package.)mask recent versions of glibc and gcc
>> ebuilds.
>>
>> What's the better way to go ? If I start with some of the stage3
>> available for download, I won't be able to downgrade the glibc.
>>
>> Or do you have any suggestion for alternatives to this gentoo chroot ?
>> (I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).
> 
> Once you chroot, you're in the chrooted env.  As long as you have a stage 3
> old enough to contain the requisite glibc, you should be good to go:
> 
> http://gentoo.osuosl.org/releases/amd64/
> 

That's what I did: I found a 2017 stage3 with a still older glibc and 
managed to upgrade to a 2020 gentoo while masking the last glibc 
versions. That was tricky because I had to git-checkout intermediate 
versions of the portage tree in order to deal with the EAPI changes but 
I have a working chroot now. Thanks.

-- 
Hervé

Re: [gentoo-user] Gentoo chroot with old glibc

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1767 bytes --]

On Sunday, 21 June 2020 20:43:59 BST Hervé Guillemet wrote:
> Le 21/06/2020 à 19:06, Michael a écrit :
> >> I need to distribute some linux binaries and the one built with my
> >> up-to-date gentoo sytem won't run on distributions using older glibc.
> >> 
> >> My idea is too maintain a gentoo chroot dedicated for compiling my
> >> binaries which would (package.)mask recent versions of glibc and gcc
> >> ebuilds.
> >> 
> >> What's the better way to go ? If I start with some of the stage3
> >> available for download, I won't be able to downgrade the glibc.
> >> 
> >> Or do you have any suggestion for alternatives to this gentoo chroot ?
> >> (I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).
> > 
> > Once you chroot, you're in the chrooted env.  As long as you have a stage
> > 3
> > old enough to contain the requisite glibc, you should be good to go:
> > 
> > http://gentoo.osuosl.org/releases/amd64/
> 
> That's what I did: I found a 2017 stage3 with a still older glibc and
> managed to upgrade to a 2020 gentoo while masking the last glibc
> versions. That was tricky because I had to git-checkout intermediate
> versions of the portage tree in order to deal with the EAPI changes but
> I have a working chroot now. Thanks.


I hadn't understood you wanted a current state of an OS, plus current system 
and other packages, BUT with a deprecated version of glibc.  I thought you 
would be OK to use a stage 3 from back then as it was in its totality, frozen 
in time, with no updates or backported packages.

However, since you managed to hold back glibc while updating everything else, 
you have proven what you wanted could be done!  I would think this on its own 
is a feat worth a HowTo.  :-)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Gentoo chroot with old glibc

[Hervé Guillemet]

Le 21/06/2020 à 22:03, Michael a écrit :

> 
> I hadn't understood you wanted a current state of an OS, plus current system
> and other packages, BUT with a deprecated version of glibc.  I thought you
> would be OK to use a stage 3 from back then as it was in its totality, frozen
> in time, with no updates or backported packages.

Yes, I need up-to-date or near-up-to-date build tools like cmake and gcc.

> 
> However, since you managed to hold back glibc while updating everything else,
> you have proven what you wanted could be done!  I would think this on its own
> is a feat worth a HowTo.  :-)

So far gentoo doesn't complain, besides keeping remind me my installed 
glibc is masked. But it's a quite minimalist system.
I found the existing HowTo 'Update Old Gentoo'[1] useful for setting 
this up.
I

[1] https://wiki.gentoo.org/wiki/User:NeddySeagoon
/HOWTO_Update_Old_Gentoo


-- 
Hervé

Re: [gentoo-user] Gentoo chroot with old glibc

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 638 bytes --]

> That's what I did: I found a 2017 stage3 with a still older glibc and
> managed to upgrade to a 2020 gentoo while masking the last glibc
> versions. That was tricky because I had to git-checkout intermediate
> versions of the portage tree in order to deal with the EAPI changes but
> I have a working chroot now. Thanks.

That's the easy way to do it, yes.

The hard way is to treat this as a cross-compilation problem and bootstrap 
your own stages from scratch. Instructions would be a bit longer...

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-user] Gentoo chroot with old glibc

[Laurence Perkins]

On June 30, 2020 1:26:48 AM PDT, "Andreas K. Huettel" <dilfridge@gentoo.org> wrote:
>> That's what I did: I found a 2017 stage3 with a still older glibc and
>> managed to upgrade to a 2020 gentoo while masking the last glibc
>> versions. That was tricky because I had to git-checkout intermediate
>> versions of the portage tree in order to deal with the EAPI changes
>but
>> I have a working chroot now. Thanks.
>
>That's the easy way to do it, yes.
>
>The hard way is to treat this as a cross-compilation problem and
>bootstrap 
>your own stages from scratch. Instructions would be a bit longer...


The medium way to do it is to use portage's ability to install packages to a different root directory. Which still has its caveats and challenges but generally avoids EAPI mismatches.

LMP

[gentoo-user] Grub: for the love of almighty Zardoz the magniicent....

[Alan Grimes]

[-- Attachment #1: Type: text/plain, Size: 1018 bytes --]

https://omsi.edu/calendar/zardoz


I RMA'd my normal mobo as previously discussed. 
I went to grab my previous motherboard and it turns out that it had actually released a goodly chunk of it's Magic Smoke (tm) while I hadn't been looking.  So I went down to the Quickie Mart and grabbed a motherboard that was still compatible with the rusty old 1800x... 
The error was 
Unable to load "/grub/i386-pc/normal.mod" 
I had to buy some more USB sticks (my optical drive seems functional but will not boot the machine...) and used gparted to get into the partition, 
.........
=(((
All my .mod files were in x86_64-efi   
>>> AND IT WAS WORKING ON THE OTHER MOTHERBOARD!!! <<< 
=~((((((((
Dear beloved Zardoz, what have I done to deserve your wrath??? 

I need to have a run of brass plaques fabricated with the inscription "If it can fail then the design is wrong" and mail them to each of the penguins that cause me this much heartburn. 
I have no idea where even to begin fixing this mess... 

[-- Attachment #2: Type: text/html, Size: 2833 bytes --]

Re: [gentoo-user] Grub: for the love of almighty Zardoz the magniicent....

[Neil Bothwick]

[-- Attachment #1: Type: text/plain, Size: 1098 bytes --]

On Wed, 1 Jul 2020 01:40:54 +0000 (UTC), Alan Grimes wrote:

> I RMA'd my normal mobo as previously discussed. 
> I went to grab my previous motherboard and it turns out that it had
> actually released a goodly chunk of it's Magic Smoke (tm) while I
> hadn't been looking.  So I went down to the Quickie Mart and grabbed a
> motherboard that was still compatible with the rusty old 1800x... The
> error was Unable to load "/grub/i386-pc/normal.mod" I had to buy some
> more USB sticks (my optical drive seems functional but will not boot
> the machine...) and used gparted to get into the partition, .........
> =((( All my .mod files were in x86_64-efi   
> >>> AND IT WAS WORKING ON THE OTHER MOTHERBOARD!!! <<<   
> =~((((((((
> Dear beloved Zardoz, what have I done to deserve your wrath??? 

Is the replacement motherboard UEFI and is t set that way in the firmware
menus?

I think Zardoz would ask why you are subjecting yourself to GRUB when you
have much simpler options with UEFI hardware.


-- 
Neil Bothwick

I backed up my hard drive and ran into a bus.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Grub: for the love of almighty Zardoz the magniicent....

[Michael]

[-- Attachment #1: Type: text/plain, Size: 1891 bytes --]

On Wednesday, 1 July 2020 09:15:10 BST Neil Bothwick wrote:
> On Wed, 1 Jul 2020 01:40:54 +0000 (UTC), Alan Grimes wrote:
> > I RMA'd my normal mobo as previously discussed. 
> > I went to grab my previous motherboard and it turns out that it had
> > actually released a goodly chunk of it's Magic Smoke (tm) while I
> > hadn't been looking.  So I went down to the Quickie Mart and grabbed a
> > motherboard that was still compatible with the rusty old 1800x... The
> > error was Unable to load "/grub/i386-pc/normal.mod" 

Some short explanation of what Live image you're trying to boot with and at 
least your partitioning scheme might help.

The GRUB error is indicative of GRUB not finding the partition in which the 
normal.mod module is stored.


> > I had to buy some
> > more USB sticks (my optical drive seems functional but will not boot
> > the machine...) 

Why not?  There should be no difference between a LiveCD and a LiveUSB.  If 
anything booting off a USB on some MoBos could be more troublesome.  Have you 
looked into this problem to find out if there was some MoBo setting you had to 
change (physical jumper/switch/cable, or firmware)?


> > and used gparted to get into the partition, .........
> > =((( All my .mod files were in x86_64-efi   
> > 
> > >>> AND IT WAS WORKING ON THE OTHER MOTHERBOARD!!! <<< 
> > 
> > =~((((((((
> > Dear beloved Zardoz, what have I done to deserve your wrath??? 
> 
> Is the replacement motherboard UEFI and is t set that way in the firmware
> menus?
> 
> I think Zardoz would ask why you are subjecting yourself to GRUB when you
> have much simpler options with UEFI hardware.

Assuming this is a UEFI MoBo you can boot any kernel images on disk directly, 
as long as you can point the UEFI firmware to them.

Have a look here:

https://wiki.gentoo.org/wiki/EFI_stub_kernel

and perhaps here:

https://wiki.gentoo.org/wiki/Efibootmgr

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [gentoo-user] Grub: for the love of almighty Zardoz the magniicent....

[Alan Grimes]

Got the motherboard running, the motherboard was IGNORING the "UEFI
only" BIOS setting, it was attempting to boot my system in BIOS mode, I
found that I had to efibootmgr and tell the stupid BIOS what is what...
why the BIOS doesn't scan the first two directories under /EFI for *.efi
and list them, then allow the user to select the default is byond me...
I mean it would be much much too easy to do it that way. =\


The magic smoke had left my previous motherboard, literally a component
was burned off the board, so couldn't put it back in service even if it
could have been booted. The temporary build actually looks pretty nice.
It has an absolutely absurd amount of RGB on it...

hmm, I wonder how many of you good ppl have a similarly burned component
that you haven't detected for some reason... If your build is more than
~2 years old, I suggest a close visual inspection of all components...

The RGB on the ram does cause a serious heat issue when I have 4 sticks
stacked up on the threadripper, but on this temporary build I'm only
using two sticks and it looks very nice.


-- 
The vaccine is a LIE. 
#TheHonklerIsReal

Powers are not rights.

Re: [gentoo-user] Gentoo chroot with old glibc

[Thomas Mueller]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

> > That's what I did: I found a 2017 stage3 with a still older glibc and
> > managed to upgrade to a 2020 gentoo while masking the last glibc
> > versions. That was tricky because I had to git-checkout intermediate
> > versions of the portage tree in order to deal with the EAPI changes but
> > I have a working chroot now. Thanks.

> That's the easy way to do it, yes.

> The hard way is to treat this as a cross-compilation problem and bootstrap
> your own stages from scratch. Instructions would be a bit longer...

> Andreas K. Hüttel

I have looked through crossdev.  Is that what it would take to cross-compile and bootstrap stages from scratch?

Could that be done from (instead of an old glibc) musl, uClibc, or FreeBSD or NetBSD?

Tom

Re: [gentoo-user] Gentoo chroot with old glibc

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

Am Mittwoch, 1. Juli 2020, 03:30:59 CEST schrieb Thomas Mueller:
> > > That's what I did: I found a 2017 stage3 with a still older glibc and
> > > managed to upgrade to a 2020 gentoo while masking the last glibc
> > > versions. That was tricky because I had to git-checkout intermediate
> > > versions of the portage tree in order to deal with the EAPI changes but
> > > I have a working chroot now. Thanks.
> > 
> > That's the easy way to do it, yes.
> > 
> > The hard way is to treat this as a cross-compilation problem and bootstrap
> > your own stages from scratch. Instructions would be a bit longer...
> > 
> > Andreas K. Hüttel
> 
> I have looked through crossdev.  Is that what it would take to cross-compile
> and bootstrap stages from scratch?
> 
> Could that be done from (instead of an old glibc) musl, uClibc, or FreeBSD
> or NetBSD?

It could be done from anywhere to anywhere in principle.

(Like, building an old-glibc x86 stage on an arm64 machine...)

This is how I bootstrapped the first riscv stages. (Yes I know we need newer 
ones...) But I don't claim it was a straightforward process. Took some time.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-user] Gentoo chroot with old glibc

[Thomas Mueller]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 749 bytes --]

> > That's what I did: I found a 2017 stage3 with a still older glibc and
> > managed to upgrade to a 2020 gentoo while masking the last glibc
> > versions. That was tricky because I had to git-checkout intermediate
> > versions of the portage tree in order to deal with the EAPI changes but
> > I have a working chroot now. Thanks.

> That's the easy way to do it, yes.

> The hard way is to treat this as a cross-compilation problem and bootstrap
> your own stages from scratch. Instructions would be a bit longer...

> Andreas K. Hüttel

I have looked through crossdev.  Is that what it would take to cross-compile and bootstrap stages from scratch?

Could that be done from (instead of an old glibc) musl, uClibc, or FreeBSD or NetBSD?

Tom

Re: [gentoo-user] Gentoo chroot with old glibc

[Andreas K. Hüttel]

[-- Attachment #1: Type: text/plain, Size: 2075 bytes --]

Am Montag, 4. Januar 2021, 13:57:37 EET schrieb Thomas Mueller:
> > > That's what I did: I found a 2017 stage3 with a still older glibc and
> > > managed to upgrade to a 2020 gentoo while masking the last glibc
> > > versions. That was tricky because I had to git-checkout intermediate
> > > versions of the portage tree in order to deal with the EAPI changes but
> > > I have a working chroot now. Thanks.
> > 
> > That's the easy way to do it, yes.
> > 
> > The hard way is to treat this as a cross-compilation problem and bootstrap
> > your own stages from scratch. Instructions would be a bit longer...
> > 
> > Andreas K. Hüttel
> 
> I have looked through crossdev.  Is that what it would take to cross-compile
> and bootstrap stages from scratch?
> 
> Could that be done from (instead of an old glibc) musl, uClibc, or FreeBSD
> or NetBSD?

In principle yes, but every experimental piece can add more problems. 
It's probably best to start with a base that is as boring as possible.

What crossdev does is (simplified)
* "create" (as symlinks) ebuilds for cross-compiler, cross-binutils, and 
target glibc
* build cross-compiler, cross-binutils, and target glibc
* installs a wrapper for emerge that uses these

For example, you end up on an amd64 system with an additional directory /usr/
riscv64-unknown-linux-gnu that contains 
* a gcc that runs on amd64 (CBUILD) but generates code for riscv64 (CTARGET)
* a binutils that runs on amd64 but works with files for riscv64
* a glibc for riscv64

Then, using commands like
riscv64-unknown-linux-gnu-emerge -av @system
you can build in /usr/riscv64-unknown-linux-gnu pieces of the target system. 

This works only in a very limited way, since many upstream build systems do 
not support cross-compiling. However, with some patience you can get to the 
point where the directory can be tarred up and used as a chroot on the target 
architecture.



-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer 
(council, qa, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 963 bytes --]

Re: [gentoo-user] Gentoo chroot with old glibc

[Rich Freeman]

On Fri, Jun 19, 2020 at 5:19 PM Hervé Guillemet <hg@apteryx.fr> wrote:
>
> Or do you have any suggestion for alternatives to this gentoo chroot ?
> (I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).

You're of course free to do it any way you wish, but if I wanted to
create packages for various distros, I'd probably just follow their
instructions for doing so.

If you're making Ubuntu 16.04 packages I suspect it would be just a
lot less fuss all around to do it from an Ubuntu 16.04 container, and
so on.

And if you're just building binaries and creating tarballs with them,
well, why?  If people want to manually deal with stuff the source is
already fine.  If they want the benefits of a package manager they're
going to want packages.  And if you're hoping to encourage distros to
do the packaging for you, they're probably only going to do that from
source anyway.

I imagine most distros have a fairly straightforward packaging system,
and I suspect a lot of CI systems have plugins to churn out packages
for them automatically.  So why maintain some Gentoo chroot and
carefully curate every single library on them to match some entirely
different distro?  You're going to run into stuff where Gentoo doesn't
have the version you need in the repo, and you'll be fighting auto
updates, and so on.

But, sure, you can get Gentoo to install whatever you want as long as
you don't mind manually picking packages, or maintaining your own repo
where you carefully curate this stuff.

-- 
Rich

Re: [gentoo-user] Gentoo chroot with old glibc

[Hervé Guillemet]

Le 21/06/2020 à 23:08, Rich Freeman a écrit :
> On Fri, Jun 19, 2020 at 5:19 PM Hervé Guillemet <hg@apteryx.fr> wrote:
>>
>> Or do you have any suggestion for alternatives to this gentoo chroot ?
>> (I'd prefer avoid installing some CentOS or Ubuntu as virtual guests).
> 
> You're of course free to do it any way you wish, but if I wanted to
> create packages for various distros, I'd probably just follow their
> instructions for doing so.
> 
> If you're making Ubuntu 16.04 packages I suspect it would be just a
> lot less fuss all around to do it from an Ubuntu 16.04 container, and
> so on.
> 
> And if you're just building binaries and creating tarballs with them,
> well, why?  If people want to manually deal with stuff the source is
> already fine.  If they want the benefits of a package manager they're
> going to want packages.  And if you're hoping to encourage distros to
> do the packaging for you, they're probably only going to do that from
> source anyway.

In my case the application is a Java software embedding some native 
components. I don't want to limit the target system to some specific 
distros and specific versions, I just want the software to run on a 
large variety of linux boxes. All I have to do is to avoid linking with 
too recent libc and libstdc++.
Version 2.29+ of libc is particularly annoying because they introduce 
new version of pow(), log() and exp()... so any simple mathematical 
library compiled with 2.29+ won't run on linux distro using 2.28-.
As for the packaging, I'm using a jlink image in a tarball, and will 
probably use jpackage soon.

> 
> I imagine most distros have a fairly straightforward packaging system,
> and I suspect a lot of CI systems have plugins to churn out packages
> for them automatically.  So why maintain some Gentoo chroot and
> carefully curate every single library on them to match some entirely
> different distro?  You're going to run into stuff where Gentoo doesn't
> have the version you need in the repo, and you'll be fighting auto
> updates, and so on.
> 
> But, sure, you can get Gentoo to install whatever you want as long as
> you don't mind manually picking packages, or maintaining your own repo
> where you carefully curate this stuff.
> 

Thanks for the suggestion. I believe my local build system with the 
chroot is enough for my current needs but I'll probably have to use a CI 
system if it becomes untractable or decide to use target distro 
packaging system.

-- 
Hervé