Re: [gentoo-dev] new profile layout with flavors and mix-ins

[Joshua Kinard <kumba@gentoo.org>]

On 07/03/2014 03:32, Michał Górny wrote:
> Dnia 2014-07-03, o godz. 02:18:23
> Joshua Kinard <kumba@gentoo.org> napisał(a):
> 
>> [...]
>>
>> And so on.  The goal was to have profiles/ be extremely flat, with limited
>> nesting only for categorization purposes.  Each component would contain all
>> of the information specific to that component, and rely on OOP-like
>> inheritance to override parent-level variables only within that component
>> (although, <arch> and <kernel> would have to be a little bit more broad in
>> scope).
> 
> Another sub-thread, the same question: how do you handle information
> that needs to be applied to the intersection of the profiles? CHOST for
> amd64 FreeBSD, for example.

Well, as I stated, I thought the bulk of this up *years* ago (at least 6-7
years ago, probably longer).  The issues you're raising now weren't issues
back then.  I only updated it slightly based on what I know now about how we
do things and what things are supported in Gentoo.  I am only proposing it
as a base from which others to think about and either build upon or enhance,
or strike it down.


>> PROFILE also had a set order for the first few pieces, if only to make
>> parsing and building the profile stack saner for the Portage developers:
>>    PROFILE="base:<KERNEL>:<ARCH>[:<SUBARCH>]:<LIBC>:<EVERYTHING_ELSE>"
>>
>> base - Core Gentoo/Portage/whatever information/vars/foobar/kittens
> 
> I'm against plain 'base' since it quickly becomes a mess. I would
> prefer having flavor-specific bases, like for example arch/base to mask
> stuff available only in 1/2 arches and revert it in another arch/.

The only reason I put a 'base' folder in was so that where was some kind of
anchor point for Portage to build off of.  I didn't put any thought into
what it might actually contain.  For all intents and purposes, it might just
contain empty variable declarations, kinda like an abstract base class in an
OOP language.  By itself, it's unusable and must be overridden, but it's the
point from which everything derives.

Maybe that's not needed now in current Portage.  Dunno.


>> kernel - Specifies the OS kernel.  At the time, only Linux existed, but
>>          I *think* Debian was eyeballing kFreeBSD at the time.  So I *think*
>>          I accounted for it back then.
> 
> What about kernel-ABI intersection? What about Prefix?

I know very little about Prefix.  Any new profile system we create should
focus only what is in-tree *first*, then once we work the bugs and kinks
out, we can figure out how Prefixes will be incorporated.

But that said, what little I do know about Prefix is that it's Portage
ported to other OSes, like Solaris, MacOS, Interix, IRIX, etc.  I guess an
out-of-tree Prefix overlay would just define missing bits in its profiles/*
folder.  I.e., for Solaris, you'd have a kernel/solaris, libc/solaris-libc, etc.

I'm not the best at OOP-designs, but if we set the in-tree stuff up right,
it should be trivial for out-of-tree overlays to extend and override to
support things, without going down the endlessly-nested directory snafu that
the current profile system is.  In-tree should be as flat as possible,
Prefix and other external overlays can extend it out however far they want.

As for kernel-ABI intersection, can you expand on that a little more?  A
given kernel can support multiple ABIs simultaneously.  Using MIPS as the
classic example, a 32-bit MIPS kernel only does o32, but a 64-bit kernel
does n64 natively, and you can enable o32 and n32 support individually
within the configuration.  And MIPS has even more obscure ABIs that aren't
well-known, either.


>> arch - Machine arch-specific generic information -- doesn't handle
>>        lower-level things like ISAs/ABIs/Endian, etc.
>>
>> subarch - Defines items specific given to given subarch of a main arch.
>>           Items under this directory would carry their parent arch's
>>           name for clarity only.  Again, at the time, MIPS would have been
>>           the only real user of this, and, back then, it would've dealt
>>           with SGI-machine-specific packages and USE flags, such as
>>           differentiating an ip32 machine from an ip27 machine or a
>>           cobalt.  Now that amd64/x86_64 is considering its own form of
>>           n32 (x32), that would go here, and I imagine arm would
>>           make heavy use of it as well.
> 
> This is a no-go for multilib support. We need to work on a consistent
> arch profile tree, not more splitting.

Well, it'll have to be nested somehow.  A given arch can handle multiple
ABIs, so there should at least be a top-level arch/ folder that defines very
high-level, very generic things specific to a given arch.  How we fold the
multilib and multiple ABI stuff in probably needs a lot of thought.  It may
not be possible with this proposed idea, so, if not, we'll need to think of
something else.

Also keep in mind that with MIPS, we up the fun fun with the ISA
(Instruction Set Architecture) mess (mips1 to mips4, mips32r2, mips64r2,
etc), which are also mostly independent of the ABI (meaning, n32/n64 on
mips3 or greater, but o32 only for mips1 and mips2.  No idea about mips32r*
-- other MIPS devs will have to speak for those).


>> libc - Defines the main C library used.  A bit redundant for *BSD, because
>>        they really only have one, but might help if we ever add k*BSD
>>        support (such as freebsd/glibc-based or such).  For Linux...could be
>>        tricky, since there are so many libc possibilities there.  I feel it
>>        fits better getting defined after <SUBARCH>, especially in the case
>>        of MIPS.
> 
> I think you need to handle kernel & libc in the same group.

If this was BSD, maybe.  But Linux the kernel is independent from the
userland libc.  You can have Linux/uclibc, Linux/glibc, and even Linux/musl,
among others.  And as Debian has shown, kFreeBSD marries a FreeBSD kernel
with a GNU glibc-based userland (which probably chafes diehard BSD users).

Hence the purpose of me breaking them up that way was to make individual
elements as discrete as possible.  I.e., the old UNIX philosophy of having
lots of little things that each do one thing very, very well.  Not a few big
things that try to do many things poorly or halfway (like the IOC3 chip in
certain SGI systems, but I digress).

And, by organizing them one directory deep, it even adheres to our current
Portage philosophy of <category>/<package>, which also would keep
/usr/portage/profiles somewhat clean (once we got rid of the old stacking
system many years down the road).


>> eapi - EAPI didn't exist back when this topic was visited.  Basically,
>>        everything was EAPI=0 and there was only Portage (Paludis nor
>>        pkgcore had been invented yet).
> 
> And what is this supposed to do?

I dunno, I threw it in there just in case there was some nit about EAPIs
that I didn't know about that could be defined at a profile level.  If it's
not relevant, then ignore it.


>> releases - I don't think this existed back then.  How it'd operate
>>            nowadays, I have no idea.  Probably would contain
>>            version-specific maskings for specific @system packages
>>            to facilitate upgrading, and help us if we ever tried to cut
>>            actual, fixed release points again (like what FreeBSD does
>>            w/ -RELEASE-x.y)
> 
> I don't think releases would make any sense without heavy intersection
> with other profiles.

Possibly.  It's just another thing to think about, though.  Arguably, if we
ever migrate the tree to git, setting release-specific branches is better
done there, though.  But, it's just building on how we currently do
release-specific profiles in the tree.  Maybe it's not need anymore either.

Brainstorming doesn't mean everything I am proposing is right.  It might all
be wrong.  Some of it might be right.  Maybe it inspires someone to think of
something better.  But if I don't put it out there, it'll never get
discussed in the first place.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic