[gentoo-dev] Arch Status and Future Plans

[gentoo-dev] Arch Status and Future Plans

[Arthur Zamarin]

[-- Attachment #1.1: Type: text/plain, Size: 6794 bytes --]

Hi all, this will be a long mail, and might be confusing, I'll try to
organize it, but this is a mess, so bear with me.

As you all know, Gentoo supports many various arches, in various degrees
(stable, dev, exp). Let me explain those 3 statuses fast:

* stable arch - meaning we have stable profile for this arch, and stable
keywords across base-system + varying degree of seriousness. We stable
stuff after ~30 days in tree, and are mostly happy. For example the well
known and common amd64 arch.

* dev arch - meaning we have complete dep-tree (no broken dep-trees),
but no stable profile. If you break here a package (for example
introduce new dep, previously unkeyworded) you are expected to dekeyword
and ask for rekeywording. For example the nearly unknown arch s390.

* exp arch - meaning we support what we support, with possible broken
dep-tree. This is the "scary" state of arch, since it can break at any
moment. For example the noisy (because of the physical fans) arch alpha.

So now that you know the arch statuses each arch can be, we have the
next mess - sub-arch profile. Did you know the ppc64 gentoo arch
consists of ppc64ul (big-endian) and ppc64le (little-endian), with the
latter having a much better support from upstreams. On sparc we have
both sparc32 (32-bit) and sparc64 (64-bit), with the former nearly not
working.

And the next important knowledge to know for this discussion is the
devbox situation. Arches that have a devbox (for example for sparc we
have catbus, a machine running sparc arch) are easier to maintain, since
we have the tattoo cluster (automated handler of arch stable/keyword
bugs), so it is simpler to maintain them. An unofficial requirement for
an arch to be stable arch, it should have a devbox.

Finally, many keywords on arches are continues because of inertia.
Someone added an arch 10-15 years ago, it collected dependencies, we
keyworded it, increase the dep tree, and the cycle continues. For a long
time the default keyword for new package was "~amd64 ~x86" - you see the
point...

So, are you ready for seeing the mess? Here we go.

======== amd64 & arm64 ========

Stable Arches in the best state possible. We recommend people to stable
stuff for those arches, users have great experience with those, handling
on arch testing team is great and simple. No need to think on it.

======== 32-bit arches ========

This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
maybe more. Those are in much worse situation, with a mess on various
fronts, some of them super hard to continue support. For example
qtwebengine is less and less likely to manage to compile on a
real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
minimize our work on those arches, meaning mass-destable and even
mass-dekeyword, with potentially full drop of stable status.

======== x86 ========

Stable 32-bit arch. I'll be honest, I don't believe at all this should
be stable arch anymore. I propose making it dev arch, and mass-dekeyword
stuff we got because of inertia. This arch is close to HW die. (let's
not talk about i486 vs i686).

======== arm ========

Stable 32-bit arch, split into many sub-arches, based on the arm
generation (4, 5, 6, 7, ...). We use a 32-bit chroot on arm64 host to
handle this arch. While not urgent since the host is strong, it is
already collecting tech debt.

I propose we mass-destable and mass-dekeyword stuff. Should still remain
stable arch status.

======== ppc ========

Stable 32-bit arch. Becoming harder and harder with time, with more
broken stuff (which I just destable/dekeyword).

I propose we convert it into dev arch status, not stable. If folks
disagree, once again mass-dekeyword.

======== ppc64 ========

Stable 64-bit arch. So, this is a mess of an arch. Consists of both
ppc64ul (big-endian) and ppc64le (little-endian). The latter is much
better supported by upstream. The profiles inheritance inside is a mess
(we even added running 32 userspace on 64 bit kernel, called ppc64/32ul
- just why?). We have devboxes for both BE and LE, so mostly fine. The
profile inheritance is the messiest I've even seen.

I would hope to split this arch into the two endianness, but I suspect
nobody has the energy to do it. Oh well.

Next proposal is to cleanup profiles: remove the ppc64/32ul, cleanup
profile inheritance, cleanup the masks and unmasks, and continue with
both ppc64ul & ppc64le supported.

======== sparc ========

Stable 32-bit and 64-bit arch. Has the best devbox (just seeing all the
CPUs in htop is warming a Gentoo heart).

32-bit sparc is something which is being removed from the kernel, and
didn't really exist (?), so we can just clean it up and drop support for
that.

64-bit sparc works quite well, with even rust support (what a
surprise!), and we should just cleanup keywords we got ebcause of
inertia. Don't be afraid to dekeyword and destable stuff (consult with
Arch Team), but nothing as mass work.

======== hppa ========

Sigh. Stable 64-bit arch. Out main Gentoo devbox died, and the second
one is always stuck compiling gcc for stage3 (a compilation takes 7
days). Here we have a fight in Arch Team. I prefer to destable it, Sam
prefers to stable it. This one is tough.

======== ia64 ========

Dev 64-bit arch. Kernel dropped support, glibc dropped support, devbox
died - days are short before full exp status or full removal of arch.

======== s390 ========

Dev arch. Has 2 sub-arches: s390 (32-bit) and s390x (64-bit). The latter
works well for its job, a good devbox.

For s390 32-bit we want to drop the profile, to simplify the profile
mess it has (mask&unmasks) for packages that work for 64-bit (like all
the rust stuff) and 32-bit stuff (inherits wd40 profile).

======== loong ========

Dev arch. I have no info on it, but it works. Let's not touch :)

======== riscv ========

Dev arch. I don't have much info on it, but I heard some mess with
riscv32 and riscv64, so maybe time to split it? I leave it to riscv arch
team, which works quite well, but I'll be happy to open discussion for it.

======== alpha ========

Exp arch, with nearly (or maybe already) full correct dep-tree. matoro
did a lot of great work here, so I think we should promote it to dev
arch, so dep-tree remains unbroken. We dekeyworded a lot of stuff,
cleaned it up, so a nice "completion bonus".

======== m68 ========

Exp arch, works ? maybe? I've no idea. Let's not touch :)

======== mips ========

Exp arch, with mostly good dep-tree. Does mips team want to make it dev
arch?

-- 
Arthur Zamarin
arthurzam@gentoo.org
Gentoo Linux developer (Python, pkgcore stack, QA, Arch Teams, GURU)

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [gentoo-dev] Arch Status and Future Plans

[matoro]

On 2024-06-25 13:33, Arthur Zamarin wrote:
> So, are you ready for seeing the mess? Here we go.

Thanks Arthur, I can add some input as another AT.

> ======== 32-bit arches ========
> 
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.

I think all of these need to be moved to dev.  Particular attention on ppc 
because somebody seems to have gone on a rampage throwing keywords at it and 
so I frequently see packages with things like KEYWORDS="amd64 ppc" which 
makes no sense.

Also, hppa is 32-bit as well.  The hardware is 64-bit, and so is the kernel, 
but the userspace is 32-bit.

sparc32 and s390 I think need to be dropped or exp at least.  For the former 
we need to get sys-boot/silo out of tree.
Also, in https://wiki.gentoo.org/wiki/Handbook:Main_Page#SPARC_Handbook we 
specifically call out:
> In Gentoo, only SPARC64-compatible CPUs are supported.

> ======== ppc64 ========
> 
> Stable 64-bit arch. So, this is a mess of an arch. Consists of both
> ppc64ul (big-endian) and ppc64le (little-endian). The latter is much
> better supported by upstream. The profiles inheritance inside is a mess
> (we even added running 32 userspace on 64 bit kernel, called ppc64/32ul
> - just why?). We have devboxes for both BE and LE, so mostly fine. The
> profile inheritance is the messiest I've even seen.
> 
> I would hope to split this arch into the two endianness, but I suspect
> nobody has the energy to do it. Oh well.
> 
> Next proposal is to cleanup profiles: remove the ppc64/32ul, cleanup
> profile inheritance, cleanup the masks and unmasks, and continue with
> both ppc64ul & ppc64le supported.

Agree that this needs to be split.  I could do the profile reorganization, 
but would need a developer who would be able to in one shot replace all 
KEYWORDS with both entries.

> ======== hppa ========
> 
> Sigh. Stable 64-bit arch. Out main Gentoo devbox died, and the second
> one is always stuck compiling gcc for stage3 (a compilation takes 7
> days). Here we have a fight in Arch Team. I prefer to destable it, Sam
> prefers to stable it. This one is tough.

FWIW as the one handling these for now I also would prefer destable.  I would 
say that any wd40 profiles should be ineligible for stable.

> ======== ia64 ========
> 
> Dev 64-bit arch. Kernel dropped support, glibc dropped support, devbox
> died - days are short before full exp status or full removal of arch.

I would prefer to see this go straight from dev to removed.  Also, a timeline 
on removal with respect to kernel/glibc EOL would be nice so that it doesn't 
just stay up in the air indefinitely.  Our python deprecation schedule is a 
model of clarity.

> ======== alpha ========
> 
> Exp arch, with nearly (or maybe already) full correct dep-tree. matoro
> did a lot of great work here, so I think we should promote it to dev
> arch, so dep-tree remains unbroken. We dekeyworded a lot of stuff,
> cleaned it up, so a nice "completion bonus".

Yes, this has been basically fully-correct for months now and it's very 
annoying to play catch-up constantly.  Please make this dev at the soonest 
opportunity.

Also, current tree correctness is stuck on this:  
https://github.com/gentoo/gentoo/pull/36711

> ======== mips ========
> 
> Exp arch, with mostly good dep-tree. Does mips team want to make it dev
> arch?

Unfortunately this is in much worse state than alpha.  For a long while it 
was stuck on rust but luckily we now have rust and llvm, so I should be able 
to make more progress on it, but it's slow going.  Dekeywording efforts would 
be hugely appreciated here.

Re: [gentoo-dev] Arch Status and Future Plans

[James Le Cuirot]

[-- Attachment #1: Type: text/plain, Size: 2322 bytes --]

On Tue, 2024-06-25 at 20:33 +0300, Arthur Zamarin wrote:
> ======== x86 ========
> 
> Stable 32-bit arch. I'll be honest, I don't believe at all this should
> be stable arch anymore. I propose making it dev arch, and mass-dekeyword
> stuff we got because of inertia. This arch is close to HW die. (let's
> not talk about i486 vs i686).

I don't use stable, so I'm biased, but I think this is long overdue.

> ======== ppc ========
> 
> Stable 32-bit arch. Becoming harder and harder with time, with more
> broken stuff (which I just destable/dekeyword).
> 
> I propose we convert it into dev arch status, not stable. If folks
> disagree, once again mass-dekeyword.

I'm inclined to kill it. I actively used hardware once upon a time, but that
was nearly 20 years ago. When I dropped Java support for it around 2016ish, I
literally one found user. I've never heard of one since.

> ======== ppc64 ========
> 
> Stable 64-bit arch. So, this is a mess of an arch. Consists of both
> ppc64ul (big-endian) and ppc64le (little-endian). The latter is much
> better supported by upstream. The profiles inheritance inside is a mess
> (we even added running 32 userspace on 64 bit kernel, called ppc64/32ul
> - just why?). We have devboxes for both BE and LE, so mostly fine. The
> profile inheritance is the messiest I've even seen.
> 
> I would hope to split this arch into the two endianness, but I suspect
> nobody has the energy to do it. Oh well.
> 
> Next proposal is to cleanup profiles: remove the ppc64/32ul, cleanup
> profile inheritance, cleanup the masks and unmasks, and continue with
> both ppc64ul & ppc64le supported.

I wasn't really in favour of having LE and BE under the same keyword, but I
was overruled. My fears have largely proven true. We have seen a lot of issues
around graphical stuff. Sure, we've handled it in profiles, but it is a pain.


> ======== m68k ========
> 
> Exp arch, works ? maybe? I've no idea. Let's not touch :)

It's not too bad and generally works. I'm a little slow on keywording requests
sometimes, but I mostly keep up. I have thought about promoting it from exp to
dev now that the tree is in a better state than it was, but there is still
some work to do. We have an important ABI breakage ahead, but that is being
discussed.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 858 bytes --]

Notion of stable depgraph vs stable keywords (Re: [gentoo-dev] Arch Status and Future Plans)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 993 bytes --]

Arthur Zamarin <arthurzam@gentoo.org> writes:

> Hi all, this will be a long mail, and might be confusing, I'll try to
> organize it, but this is a mess, so bear with me.

Absolutely - thanks for doing this. I'm going to split my replies with
alt subject to help keep it organised.

>
> As you all know, Gentoo supports many various arches, in various degrees
> (stable, dev, exp). Let me explain those 3 statuses fast:
>
> * stable arch - meaning we have stable profile for this arch, and stable
> keywords across base-system + varying degree of seriousness. We stable
> stuff after ~30 days in tree, and are mostly happy. For example the well
> known and common amd64 arch.

This mixes the notion of keywords vs profiles.

You can have a stable profile in profiles.desc without any stable
keywords at all.

'stable' in profiles.desc means we require CI to pass for its depgraph
consistency. 'dev' means we warn on it. 'exp' means it doesn't even show
up unless you opt-in with pkgcheck etc.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

[gentoo-dev] Re: Notion of stable depgraph vs stable keywords (Re: Arch Status and Future Plans)

[Duncan]

Sam James posted on Wed, 26 Jun 2024 01:06:12 +0100 as excerpted:

> Arthur Zamarin <arthurzam@gentoo.org> writes:
> 
>> As you all know, Gentoo supports many various arches, in various
>> degrees (stable, dev, exp). Let me explain those 3 statuses fast:
>>
>> * stable arch - meaning we have stable profile for this arch, and
>> stable keywords across base-system + varying degree of seriousness. We
>> stable stuff after ~30 days in tree, and are mostly happy. For example
>> the well known and common amd64 arch.
> 
> This mixes the notion of keywords vs profiles.
> 
> You can have a stable profile in profiles.desc without any stable
> keywords at all.
> 
> 'stable' in profiles.desc means we require CI to pass for its depgraph
> consistency. 'dev' means we warn on it. 'exp' means it doesn't even show
> up unless you opt-in with pkgcheck etc.

While that may clear things up from a developer perspective, it's still 
confusing from a user perspective (even a long-time user like me who 
religiously follows this list, tho being on amd64 personally with no 
question on it staying stable it doesn't really affect me personally at 
this time... tho not /too/ long ago I was still running a 32-bit-only atom 
netbook (tho only upgraded perhaps every year or two... which always made 
it difficult but possible with some time and patience) so it /could/ still 
affect me and I'm concerned about others still affected).

Taking the one most likely to affect the greatest number of users as an 
example, what practical effects would dropping x86 to dev (I'm assuming no 
one's suggesting dropping it straight to experimental) have on remaining 
x86 users?

How would it differ if they're already running ~x86 vs stable x86 
(keywording), assuming the same currently stable x86 profile?

And (again from a user perspective) how does dropping x86 to dev differ 
from the mentioned apparently worse alternative, mass dekeywording?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Re: Notion of stable depgraph vs stable keywords (Re: Arch Status and Future Plans)

[Sam James]

Duncan <1i5t5.duncan@cox.net> writes:

> Sam James posted on Wed, 26 Jun 2024 01:06:12 +0100 as excerpted:
>
>> Arthur Zamarin <arthurzam@gentoo.org> writes:
>> 
>>> As you all know, Gentoo supports many various arches, in various
>>> degrees (stable, dev, exp). Let me explain those 3 statuses fast:
>>>
>>> * stable arch - meaning we have stable profile for this arch, and
>>> stable keywords across base-system + varying degree of seriousness. We
>>> stable stuff after ~30 days in tree, and are mostly happy. For example
>>> the well known and common amd64 arch.
>> 
>> This mixes the notion of keywords vs profiles.
>> 
>> You can have a stable profile in profiles.desc without any stable
>> keywords at all.
>> 
>> 'stable' in profiles.desc means we require CI to pass for its depgraph
>> consistency. 'dev' means we warn on it. 'exp' means it doesn't even show
>> up unless you opt-in with pkgcheck etc.
>
> While that may clear things up from a developer perspective,

The discussion we're currently having *is* from a developer perspective
where I'm trying to clarify something Arthur said for the purposes of
further discussion.

>
> How would it differ if they're already running ~x86 vs stable x86 
> (keywording), assuming the same currently stable x86 profile?
>
> And (again from a user perspective) how does dropping x86 to dev differ 
> from the mentioned apparently worse alternative, mass dekeywording?

An inconsistent depgraph is a very poor experience for users because
there's no guarantee emerge can resolve things.

Misc arch plans (Re: [gentoo-dev] Arch Status and Future Plans)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 5891 bytes --]

Arthur Zamarin <arthurzam@gentoo.org> writes:

> Hi all, this will be a long mail, and might be confusing, I'll try to
> organize it, but this is a mess, so bear with me.
>
> [...]
>
> ======== 32-bit arches ========
>
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.
>
> ======== x86 ========
>
> Stable 32-bit arch. I'll be honest, I don't believe at all this should
> be stable arch anymore. I propose making it dev arch, and mass-dekeyword
> stuff we got because of inertia. This arch is close to HW die. (let's
> not talk about i486 vs i686).

We need to be clear on the rationale for making something a dev arch.

If we're going to mass-dekeyword, the need for it being a dev arch
lessens, no?

There's also the frustrating issue of keywords in general with regard to
"did this ever work" vs "we support it and want to keep it going", which
is why pruning keywords is a bit of a shame - though I think we should
do it. (It's annoying for stuff which actually imposes no new depgraph
constraints and works fine, and creates work when something gains a dep
on it.)

>
> ======== arm ========
>
> Stable 32-bit arch, split into many sub-arches, based on the arm
> generation (4, 5, 6, 7, ...). We use a 32-bit chroot on arm64 host to
> handle this arch. While not urgent since the host is strong, it is
> already collecting tech debt.
>
> I propose we mass-destable and mass-dekeyword stuff. Should still remain
> stable arch status.
>

Sounds OK, I think. I'd want "lightweight desktops" to have stable KWs
there (lxqt/xfce).

> ======== ppc ========
>
> Stable 32-bit arch. Becoming harder and harder with time, with more
> broken stuff (which I just destable/dekeyword).
>
> I propose we convert it into dev arch status, not stable. If folks
> disagree, once again mass-dekeyword.
>

I can't know if I disagree or not until we're clear on what the
rationale is for depgraph consistency.

dev arch status won't help here if mass-dekeywording is
*separate* because then we just get very noisy CI.


> ======== ppc64 ========
>
> Stable 64-bit arch. So, this is a mess of an arch. Consists of both
> ppc64ul (big-endian) and ppc64le (little-endian). The latter is much
> better supported by upstream. The profiles inheritance inside is a mess
> (we even added running 32 userspace on 64 bit kernel, called ppc64/32ul
> - just why?). We have devboxes for both BE and LE, so mostly fine. The
> profile inheritance is the messiest I've even seen.
>
> I would hope to split this arch into the two endianness, but I suspect
> nobody has the energy to do it. Oh well.
>
> Next proposal is to cleanup profiles: remove the ppc64/32ul, cleanup
> profile inheritance, cleanup the masks and unmasks, and continue with
> both ppc64ul & ppc64le supported.
>

I'd like to see it split too but I don't want to do the work myself.

> ======== sparc ========
>
> Stable 32-bit and 64-bit arch. Has the best devbox (just seeing all the
> CPUs in htop is warming a Gentoo heart).
>
> 32-bit sparc is something which is being removed from the kernel, and
> didn't really exist (?), so we can just clean it up and drop support for
> that.

Agreed.

>
> 64-bit sparc works quite well, with even rust support (what a
> surprise!), and we should just cleanup keywords we got ebcause of
> inertia. Don't be afraid to dekeyword and destable stuff (consult with
> Arch Team), but nothing as mass work.

Agreed.

>
> ======== hppa ========
>
> Sigh. Stable 64-bit arch. Out main Gentoo devbox died, and the second
> one is always stuck compiling gcc for stage3 (a compilation takes 7
> days). Here we have a fight in Arch Team. I prefer to destable it, Sam
> prefers to stable it. This one is tough.
>

hppa is 32-bit right now. The userland work is stalled on glibc support.

I think I can live with it being ~arch after reflection, but I'll talk
about my feelings for ~arch vs stable in general for these arches in a minute.

> ======== ia64 ========
>
> Dev 64-bit arch. Kernel dropped support, glibc dropped support, devbox
> died - days are short before full exp status or full removal of arch.

Yeah, no interest in ia64, sorry. I'd like it to just go.

>
> ======== s390 ========
>
> Dev arch. Has 2 sub-arches: s390 (32-bit) and s390x (64-bit). The latter
> works well for its job, a good devbox.
>
> For s390 32-bit we want to drop the profile, to simplify the profile
> mess it has (mask&unmasks) for packages that work for 64-bit (like all
> the rust stuff) and 32-bit stuff (inherits wd40 profile).
>

Let's kill s390 (31-bit). s390x should stay.

> ======== loong ========
>
> Dev arch. I have no info on it, but it works. Let's not touch :)
>
> ======== riscv ========
>
> Dev arch. I don't have much info on it, but I heard some mess with
> riscv32 and riscv64, so maybe time to split it? I leave it to riscv arch
> team, which works quite well, but I'll be happy to open discussion for it.
>

I still think we should split the keywords.

> ======== alpha ========
>
> Exp arch, with nearly (or maybe already) full correct dep-tree. matoro
> did a lot of great work here, so I think we should promote it to dev
> arch, so dep-tree remains unbroken. We dekeyworded a lot of stuff,
> cleaned it up, so a nice "completion bonus".
>

OK.

> ======== m68 ========
>
> Exp arch, works ? maybe? I've no idea. Let's not touch :)
>

Me neither.

> ======== mips ========
>
> Exp arch, with mostly good dep-tree. Does mips team want to make it dev
> arch?

No idea from my end.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

ia64, was: Re: Misc arch plans (Re: [gentoo-dev] Arch Status and Future Plans)

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 657 bytes --]

> > ======== ia64 ========
> >
> > Dev 64-bit arch. Kernel dropped support, glibc dropped support, devbox
> > died - days are short before full exp status or full removal of arch.
> 
> Yeah, no interest in ia64, sorry. I'd like it to just go.

This is probably unavoidable given that our devbox and stage builder is unstable
now regularly dies (and that there is also no qemu support).

I'm all for keeping what we can, but we need
* people to do the work
* hardware to test

If *both* is missing it's kinda hopeless.

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: ia64, was: Re: Misc arch plans (Re: [gentoo-dev] Arch Status and Future Plans)

[matoro]

On 2024-06-26 16:29, Andreas K. Huettel wrote:
>> > ======== ia64 ========
>> >
>> > Dev 64-bit arch. Kernel dropped support, glibc dropped support, devbox
>> > died - days are short before full exp status or full removal of arch.
>> 
>> Yeah, no interest in ia64, sorry. I'd like it to just go.
> 
> This is probably unavoidable given that our devbox and stage builder is 
> unstable
> now regularly dies (and that there is also no qemu support).
> 
> I'm all for keeping what we can, but we need
> * people to do the work
> * hardware to test
> 
> If *both* is missing it's kinda hopeless.

I will continue to keep ia64 in shape until its final removal from Gentoo.

Also, your earlier point about exp arches is fantastic.  It should be a 
transition state only, either to dev or to removal.

On the value (or not?) of stable keywords (Re: [gentoo-dev] Arch Status and Future Plans)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1370 bytes --]

Arthur Zamarin <arthurzam@gentoo.org> writes:

> Hi all, this will be a long mail, and might be confusing, I'll try to
> organize it, but this is a mess, so bear with me.
>
> [...]
> ======== 32-bit arches ========
>
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.

On stable keywords, there's a few thoughts I have:
* Stable keywords help keep a platform sustainable because you can keep
 it up to date and not be buried under heavy updates (e.g. I sometimes
 get asked to not keyword new GCC versions for the benefit of ~arch-only
 arches)

* For arches without stable keywords, we don't have any reason to
  regularly run the testsuite, so issues which might even be trivially
  solvable go unnoticed. It goes from "running tests whenever there's a
  regular stablereq" -> "once in a blue moon if rekeywording is
  required".

  This part is a shame, but it's also precisely why there's pressure to
  destable things. It's tricky :(

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

time64 & LFS for 32-bit arches (Re: [gentoo-dev] Arch Status and Future Plans)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1191 bytes --]

Arthur Zamarin <arthurzam@gentoo.org> writes:

> Hi all, this will be a long mail, and might be confusing, I'll try to
> organize it, but this is a mess, so bear with me.
>
> ======== 32-bit arches ========
>
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.
>

We haven't yet migrated to 64-bit time_t (and off_t - Large File
Support/LFS) which will fix a lot of general test failures.

Doing that before making any decision has value in two ways:
1) It might make the situation a lot better, who knows?

2) Even if we do then do a mass purge after, it leaves users on such
platforms in a good state to keep stabilisation until after we're done,
to help find any possible issues.

dilfridge is currently beginning the time64 prep work (it requires LFS
too).

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

x86 FP issues (Re: [gentoo-dev] Arch Status and Future Plans)

[Sam James]

[-- Attachment #1: Type: text/plain, Size: 1183 bytes --]

Arthur Zamarin <arthurzam@gentoo.org> writes:

> Hi all, this will be a long mail, and might be confusing, I'll try to
> organize it, but this is a mess, so bear with me.
> [...]
> ======== 32-bit arches ========
>
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.
>
> ======== x86 ========
>
> Stable 32-bit arch. I'll be honest, I don't believe at all this should
> be stable arch anymore. I propose making it dev arch, and mass-dekeyword
> stuff we got because of inertia. This arch is close to HW die. (let's
> not talk about i486 vs i686).

I think the mfpmath=sse thing [0] makes this a bit better but I still
sympathise with your point.

[0] https://public-inbox.gentoo.org/gentoo-dev/ce894afe6c2b324fef012da9bb9387cfde7aed03.camel@gentoo.org/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 377 bytes --]

Re: x86 FP issues (Re: [gentoo-dev] Arch Status and Future Plans)

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

On Wed, 2024-06-26 at 01:20 +0100, Sam James wrote:
> Arthur Zamarin <arthurzam@gentoo.org> writes:
> 
> > Hi all, this will be a long mail, and might be confusing, I'll try to
> > organize it, but this is a mess, so bear with me.
> > [...]
> > ======== 32-bit arches ========
> > 
> > This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> > maybe more. Those are in much worse situation, with a mess on various
> > fronts, some of them super hard to continue support. For example
> > qtwebengine is less and less likely to manage to compile on a
> > real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> > minimize our work on those arches, meaning mass-destable and even
> > mass-dekeyword, with potentially full drop of stable status.
> > 
> > ======== x86 ========
> > 
> > Stable 32-bit arch. I'll be honest, I don't believe at all this should
> > be stable arch anymore. I propose making it dev arch, and mass-dekeyword
> > stuff we got because of inertia. This arch is close to HW die. (let's
> > not talk about i486 vs i686).
> 
> I think the mfpmath=sse thing [0] makes this a bit better but I still
> sympathise with your point.

I'd just like to point out that `-mfpmath=sse` is a two-edged sword.  It
generally makes results more consistent with other architectures which
is good and resolves some test failures when people *aren't testing with
x86*.  However, it breaks stuff when people are specifically testing
with x86 and accounting for i387 math (sigh).

> 
> [0] https://public-inbox.gentoo.org/gentoo-dev/ce894afe6c2b324fef012da9bb9387cfde7aed03.camel@gentoo.org/

-- 
Best regards,
Michał Górny


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 512 bytes --]

[gentoo-dev] Re: Arch Status and Future Plans

[Florian Schmaus]

[-- Attachment #1.1.1: Type: text/plain, Size: 1875 bytes --]

Hi Arthur,

thanks for taking the time to write this mail.

On 25/06/2024 19.33, Arthur Zamarin wrote:
> ======== x86 ========
> 
> Stable 32-bit arch. I'll be honest, I don't believe at all this should
> be stable arch anymore.

I have the impression as well. The time to drop stable keywords for x86 
probably has come. But I always wonder if there is a x86 use-case we are 
not aware of. Therefore, if there is a group of x86 Gentoo users out 
there, then they should speak now and ideally elaborate a bit on their 
use case.


> ======== hppa ========
> 
> Sigh. Stable 64-bit arch. Out main Gentoo devbox died, and the second
> one is always stuck compiling gcc for stage3 (a compilation takes 7
> days). Here we have a fight in Arch Team. I prefer to destable it, Sam
> prefers to stable it. This one is tough.

It's not a fight, just a disagreement, or is it? ;)

Probably time would be better invested in other architectures. However, 
I remember sam saying that he has a special interested in hppa and there 
is nothing better than working on stuff that you care about. So maybe 
just give stable hppa a try and re-evaluate the situation in a year or so?


> ======== riscv ========
> 
> Dev arch. I don't have much info on it, but I heard some mess with
> riscv32 and riscv64, so maybe time to split it? I leave it to riscv arch
> team, which works quite well, but I'll be happy to open discussion for it.

IMHO should be split into 32- and 64-bit keywords. Ideally we would also 
consider renaming the 'riscv' keyword to 'riscv64', but maybe this ship 
has sailed?

What about (64-bit) RISC-V becoming stable? Not sure if they time for it 
has already come, but it certainly will come. I might gain access to a 
64-core RISC-V system (Milk-V Pioneer), so I could probably help with 
riscv stabilization.

- Flow



[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 21567 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 618 bytes --]

[gentoo-dev] Re: Arch Status and Future Plans

[Christian Bricart]

Am 26.06.24 um 09:38 schrieb Florian Schmaus:
> Hi Arthur,
> 
> thanks for taking the time to write this mail.
> 
> On 25/06/2024 19.33, Arthur Zamarin wrote:
>> ======== x86 ========
>>
>> Stable 32-bit arch. I'll be honest, I don't believe at all this should
>> be stable arch anymore.
> 
> I have the impression as well. The time to drop stable keywords for x86 
> probably has come. But I always wonder if there is a x86 use-case we are 
> not aware of. Therefore, if there is a group of x86 Gentoo users out 
> there, then they should speak now and ideally elaborate a bit on their 
> use case.

well - at least *I* am running x86 on a bunch of Intel Atom boards 
(-march=bonnell, "Diamondville") which are mostly purposed as routers, 
but also keeping some of those early "Netbooks" based on that processor 
alive.
All equipped with (the platform's max of) 2 Gi RAM, they still suit 
their purpose well…

Cheers
   Christian

Re: [gentoo-dev] Re: Arch Status and Future Plans

[Immolo]

Hi all,

As a 32bit user on many arches I'll try to answer Flow's question below.

On Wed, 26 Jun 2024 at 07:38, Florian Schmaus <flow@gentoo.org> wrote:
>
> Hi Arthur,
>
> thanks for taking the time to write this mail.
>
> On 25/06/2024 19.33, Arthur Zamarin wrote:
> > ======== x86 ========
> >
> > Stable 32-bit arch. I'll be honest, I don't believe at all this should
> > be stable arch anymore.
>
> I have the impression as well. The time to drop stable keywords for x86
> probably has come. But I always wonder if there is a x86 use-case we are
> not aware of. Therefore, if there is a group of x86 Gentoo users out
> there, then they should speak now and ideally elaborate a bit on their
> use case.

I personally support the move to ~arch for all 32bit arches that I use
(x86, ppc and arm) as I don't feel many people are testing these
platforms, so I've found generally a better experience since switching
as fixes seem to get sorted a lot faster.

My only concern is around running the testing toolchain on these
systems as we have had 2 x86 breakages with glibc causing a user to
need to reinstall in at least one occasion and there was also the
issue with ppc not working with GCC13 and 14 for a very long time.
Now for me I enjoy the challenge of issues like this so I wouldn't
have an issue however not everyone is like me so I would advocate that
we would have a system in place for a phased keywording around glibc,
gcc, llvm, clang and binutils. This way users can have a stable
toolchain with minimal issues and users such as myself can help you
all by giving you early warning.
This wouldn't have to be a permanent plan in place however, I believe
it would make the transition a lot smoother for both devs and users.

Bugs:
https://bugs.gentoo.org/933764
https://sourceware.org/bugzilla/show_bug.cgi?id=31316

Kind regards,

Immolo

nomenclature, was: Re: [gentoo-dev] Arch Status and Future Plans

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 2547 bytes --]

> As you all know, Gentoo supports many various arches, in various degrees
> (stable, dev, exp). Let me explain those 3 statuses fast:
> 
> * stable arch - meaning we have stable profile for this arch, and stable
> keywords across base-system + varying degree of seriousness. We stable
> stuff after ~30 days in tree, and are mostly happy. For example the well
> known and common amd64 arch.
> 
> * dev arch - meaning we have complete dep-tree (no broken dep-trees),
> but no stable profile. If you break here a package (for example
> introduce new dep, previously unkeyworded) you are expected to dekeyword
> and ask for rekeywording. For example the nearly unknown arch s390.
> 
> * exp arch - meaning we support what we support, with possible broken
> dep-tree. This is the "scary" state of arch, since it can break at any
> moment. For example the noisy (because of the physical fans) arch alpha.

This classification is good enough for practical purposes (and for the
discussion of the status of architectures) but technically not correct.

I'll try to clarify below, but mostly to clean up confusion about wording.

You have to keep apart two things, A) architecture status, and B) profile
status, which are formally independent of each other.

A) architecture status, defined in profiles/arches.desc

A.1) stable
amd64, arm, arm64, hppa, ppc, ppc64, sparc, x86
Architectures that have a stable keyword and where packages undergo stabilization.

A.2) transitional
(currently none)
Architectures that have a stable keyword and where packages undergo stabilization, 
but the dependency tree is only consistent for ~arch.
This is useful for upgrading architectures to stable.

A.3) testing
all other
Architectures that only have testing, ~arch keywords

B) profile status, defined in profiles/profiles.desc

B.1) stable
The dependency tree is checked and enforced by the CI and by pkgcheck by default.
Unsatisfied dependencies etc generate errors and "break the tree".

B.2) dev
The dependency tree is checked by the CI and by pkgcheck (by default).
Unsatisfied dependencies etc generate warnings.

B.3) exp
No checking (by default)

Many combinations of these two properties exist. For example, 

amd64:
is A.1 and has many B.1 profiles (but also some B.2 and B.3)

loong:
is A.3 and has B.1, B.2, B.3 profiles

m68k:
is A.3 and has only B.3 profiles



-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

Re: [gentoo-dev] Arch Status and Future Plans

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1623 bytes --]

> ======== alpha ========
> 
> Exp arch, with nearly (or maybe already) full correct dep-tree. matoro
> did a lot of great work here, so I think we should promote it to dev
> arch, so dep-tree remains unbroken. We dekeyworded a lot of stuff,
> cleaned it up, so a nice "completion bonus".
> 
> ======== m68 ========
> 
> Exp arch, works ? maybe? I've no idea. Let's not touch :)
> 
> ======== mips ========
> 
> Exp arch, with mostly good dep-tree. Does mips team want to make it dev
> arch?

What you call "exp arches" (i.e. architectures with no "stable profiles" at
all) are a pain in the behind. Nobody checks if dependencies are correct, 
and the dependency tree becomes randomly broken.

This should ideally only be a temporary status, 
* either on the way to a consistent deptree (~arch)
* or on the way out the door.

My 2ct:

alpha: well, if it is in a decent state, and if Matoro promises to keep up
with keywording requests (important!), let's upgrade it.

m68k: dito as soon as Chewi thinks it's ready

mips: well... personally I'd love to have it back to a consistent deptree, 
but right now it has still waaay too many packages keyworded for that.
also, this is a horrible beast, with 32bit, mixed, and 64bit ABI and both
big and little endian. we have no hardware at all, and even if we had it
would be slow... 
that said, I dont see us dropping support for mips either. in particular
since we're one of the last distros to deal with it...




-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

riscv, was: Re: [gentoo-dev] Arch Status and Future Plans

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

> ======== riscv ========
> 
> Dev arch. I don't have much info on it, but I heard some mess with
> riscv32 and riscv64, so maybe time to split it? I leave it to riscv arch
> team, which works quite well, but I'll be happy to open discussion for it.

riscv is something new and growing, but for now hardware is still quite slow.
We do not have any hardware devbox so far.

In its very beginning I proposed to use different keywords for 32bit and 64bit
(if not for that, for what then???), but enthusiasm was limited.
** Reminder, people actually have to do the keywording and maybe stabilization,
and more keywords means more work. **

If we split it, then please let's keep riscv for 64bit and e.g. use riscv32 
and one day in the futre riscv128 for the other variants...

Also, this is a potential candidate for a future stable arch (i.e. having stable
keywords and stabilization).

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

32bit vs 64bit, was: Re: [gentoo-dev] Arch Status and Future Plans

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]

> ======== 32-bit arches ========
> 
> This includes stable arches x86, arm, ppc, sparc32, dev arches s390, and
> maybe more. Those are in much worse situation, with a mess on various
> fronts, some of them super hard to continue support. For example
> qtwebengine is less and less likely to manage to compile on a
> real-hardware, and not 32-bit chroot on 64-bit host. Arch Team want to
> minimize our work on those arches, meaning mass-destable and even
> mass-dekeyword, with potentially full drop of stable status.

We've got several different types of problems here.

1) Different data type sizes
This is the least problematic point, since testing in a chroot works fine.

2) Limitations, e.g. limited memory address space
Also not that problematic, since testing in a chroot on a 64bit kernel
circumvents the limitation somewhat.
Whether it makes sense to be able to build something in a chroot but never 
on real hardware is another question.

3) time64
This is work in progress. I guess we'll get it done until 2038.

4) isa- and hardware-related regressions.
This is the real problem that cannot be caught via chroots and testing
on fast 64bit machines...
We recently had an upstream regression in glibc that (from memory) broke
all machines without SSE. It took a surprisingly long time for anyone to
notice that, since of course any x86-64 machine understands these 
instructions. The only way to really reproduce it on modern hardware is
in qemu-system with disabled kvm.
Can we call an architecture "stable" if we never test on real (and not
"downward-emulated") hardware?

-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]

splitting keywords, was: Re: [gentoo-dev] Arch Status and Future Plans

[Andreas K. Huettel]

[-- Attachment #1: Type: text/plain, Size: 1012 bytes --]

> I would hope to split this arch into the two endianness, but I suspect
> nobody has the energy to do it. Oh well.
[...]
> Dev arch. I don't have much info on it, but I heard some mess with
> riscv32 and riscv64, so maybe time to split it? I leave it to riscv arch
> team, which works quite well, but I'll be happy to open discussion for it.

So, technically, splitting a keyword into two is easy to do.

You start with duplicating the KEYWORDS entry in all ebuilds, and duplicating the
profiles. Then both profiles and keywords are pruned where it makes sense,
and they develop independently.

Happy to help with this part as it's mostly mechanical.

The really important part however is what comes afterwards: Separate keywords mean
* separate keywording requests
* separate stable requests
and there should be at least some sort of arch team taking care of that.


-- 
Andreas K. Hüttel
dilfridge@gentoo.org
Gentoo Linux developer
(council, toolchain, base-system, perl, libreoffice)

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 981 bytes --]