Re: [gentoo-mips] multilib problems on mips64 profiles

[Joshua Kinard <kumba@gentoo.org>]

On 09/21/2014 21:11, Anthony G. Basile wrote:
> On 09/21/14 18:53, Joshua Kinard wrote:
>> On 09/21/2014 17:55, Anthony G. Basile wrote:
>>> On 09/17/14 16:19, Anthony G. Basile wrote:
>> [snip]
>>>
>>> This isn't going to work.  The problem is that the above structure doesn't
>>> distinguish between mips/o32 and mips64/o32 (and equivalent el versions).  Now
>>> mips64/o32 is funny, but possible: it would be o32 (a 32-bit abi) on 64 bit
>>> arch/kernel, like ppc64-32ul.  The structure is possible at the level of
>>> profile/arch, but it would mean a deep restructuring of
>>> default/linux/mips/13.0.
>>
>> Err, I've been using this setup for the last 9+ years?  My Octane and O2 both
>> boot mips64 kernels and run an o32-only userland.  It works fine with the
>> current profile setup.  You just use a mips-unknown-linux-gnu CHOST.  I may not
>> be able to fully utilize all of the CPU registers like I would under n32, but
>> that's never been an issue for me.
>>
>> So there should be no need to differentiate at the userland level between
>> mips/o32 and mips64/o32.  As long as the CHOST is set correctly, o32 will work
>> fine under a mips64 kernel (caveat: make sure CONFIG_MIPS32_O32 is set in the
>> kernel).
> 
> I agree that there shouldn't be any userland difference, but I'm not 100%
> sure.  Anyhow, let's go with that assumption for now.

There isn't.  If there is a difference that can be detected (meaning, with a
machine that can run in 32-bit or 64-bit mode with the same userland), then
you're looking at a kernel problem and that would get handled via upstream
channels.  You're actually at a big disadvantage running o32 under a 64-bit
kernel, because of not being able to use all of the registers in the CPU, can
only pass the first four function args via registers (8 in n32), etc.  o32 is
for all intents and purposes, the MIPS-I and MIPS-II ISAs, which are core to
the overall ISA set, and so all MIPS CPUs to date should (in theory, because I
do not have access to mips64r* hardware) run o32 just fine.

You can compile o32 with higher ISAs, i.e., mips3 and mips4, but it's always
been debatable how much extra that helps you (Debian used to question our logic
on this back in the day).  Can't use 64-bit features of 64-bit CPUs via that,
but it is supposed to enable things like the square-root instructions available
at the mips4 level.

The only reason I'm still running o32 is I just haven't been motivated enough
to do a full re-install of both systems.  Even if I switch to n32, I'll
probably keep o32 chroots around just to make sure things don't break there.
Might do this soon, though, as I just bought ten new hard drives for all of my
MIPS systems.  Just need to get the right brackets for them (because they're
2.5" 10K SCA drives) so I can mount everything and do data migration.


>>
>> Correct me if wrong, but it seems the core problem here is that multilib
>> inherits from the o32 base profile.  While I think the proper longterm fix is
>> to have more discrete, modular/pluggable profile components (like OOP and
>> multiple base classes), that's not going to happen in Portage for a long time.
> 
> Not exactly.  The problem is that everything inherits from profiles/arch/mips
> and currently that forces o32 with mgorny's multilib stuff.  We need to get
> that out of the way for the other profiles that inherit it.
> 

Agreed.  This worked fine in the past when o32 was our primary focus and
n32/n64 were still experimental (and multilib didn't exist).  Nowadays, the
base profile shouldn't prescribe a default ABI at all.  That needs to be
handled under a subprofile somehow.

The real problem is Portage's stacking profiles feature, while soving a lot of
problems that the previous profile system had, wasn't thought up with all of
MIPS' happyfun included.  Because endianness, ABI, ISA, kernel, libc, compiler,
and even specific machine support can all be interchangeable in MIPS, this
causes the stacking profile system to become a thing that would frighten
Yog-Sothoth itself, if we were to fully implement all of the combinations
possible under this system.


>> So I think the solution is to split the profiles into "mips" and "mips64",
>> i.e., 32/ and 64/ under the 'mips' base profile folder.  32/ contains
>> everything needed to run pure o32-only under either a mips or mips64 kernel.
>> I.e., mips-unknown-linux-gnu CHOST.  So on my Octane, /etc/portage/make.profile
>> symlinks to /usr/portage/default/linux/mips/32/<VERSION>/
> 
> That's one approach, but I'm trying to find the least intrusive. I do have the
> problem fixed with the following:
> 
>   [1]   default/linux/mips/13.0/o32
>   [2]   default/linux/mips/13.0/n32
>   [3]   default/linux/mips/13.0/n64
>   [4]   default/linux/mips/13.0/multilib/o32
>   [5]   default/linux/mips/13.0/multilib/n32
>   [6]   default/linux/mips/13.0/multilib/n64
>   [7]   default/linux/mips/13.0/mipsel/o32
>   [8]   default/linux/mips/13.0/mipsel/n32
>   [9]   default/linux/mips/13.0/mipsel/n64
>   [10]  default/linux/mips/13.0/mipsel/multilib/o32
>   [11]  default/linux/mips/13.0/mipsel/multilib/n32
>   [12]  default/linux/mips/13.0/mipsel/multilib/n64
> 
> and would like to push this through as a fix for now.  1 and 4 have
> CHOST=mips-unknown-linux-gnu and mipsel-unknown-linux-gnu respectively.  2,3
> and 8,9 have mips64-unknown-linux-gnu and mips64el-unknown-linux-gnu
> respectively.  Only users of profiles 1 and 7 need to change their sym link one
> level down to ../o32.  That was necessary to move the o32 stuff out of the way
> of profiles/arch/mips.

This sounds reasonable.  I've got the Octane tasked again on building glibc to
hunt down the R10000 bug I've been stuck on for the past three months, but I
can sync later and test this under your hardened overlay...maybe.  I just tried
overlays on Gentoo/FreeBSD for nigoro's 9.3 profiles and gave up in utter
frustration.

I still think these need to be under a separate versioned directory structure,
though.  14.0/ or 15.0/ or testing/ or pikachu/, whatever.  Then we mark 13.0
as deprecated and allow time for users to migrate off before purging them out
of the tree in a month or two.  It makes for a messy commit (thanks, CVS), but
it avoids breaking people's systems out there who may not be on this mailing list.

Also, we should get a news item added as well, specific to MIPS systems only.


>>
>> 64/ is where the fun is at.  I think the default ABI there should be n32 at the
>> base, with a subprofile for multilib that itself contains subprofiles to handle
>> the combinations of ABIs desired.  Still have to workout what CHOST you use for
>> the base (GNU standard for n32 is mips64-unknown-linux-gnueabin32).  multilib
>> would just use mips64-unknown-linux-gnu, and -mabi would be passed to gcc to
>> pick the ABI to build for.
>>
>> Also, if we are going to do any kind of reorganizing of the profiles, let's not
>> do it under 13.0.  Personally, I'd like to get away from datestamps as profile
>> versions (13.0 refers to 2013) and either pick a fixed version number that we
>> increment or come up with some other kind of versioning scheme.  Or just do
>> away with it altogether and have something like a "stable" profile where things
>> work and an "unstable" branch that only exists when we're doing insane changes
>> to the profiles, which after testing, becomes "stable" (and current "stable"
>> becomes "oldstable" or such).
>>
>> If we have to stick with datestamped versions, then use 15.0.  Cause it'll be
>> 2015 by the time this goes active.
>>
>> rough draft of the top-level layout.  Doesn't care about chosen libc, ISA, or
>> <VERSION>, but does reflect endianness.
>>
>> default/linux/mips
>>     |
>>     |-32/
>>     |  |-be/
>>     |  |-le/
>>     |
>>     |-64/
>>     |  |-be/
>>     |  |-le/
>>     |  |
>>     |  |-multilib/
>>     |  |  |-be/
>>     |  |  |-le/
>>     |  |
>>     |
>>
> 
> I like this, but the problem is disentangling the situation under arch/mips,
> not under default/linux/mips.  Again, the inheritance problem stems from
> everything under profiles/arch/mips inheriting from profiles/arch/mips.  So
> even if you do implement the above, you still have to address the issue of how
> the arch/mips stuff is stacking.

I'll poke around arch/mips later and see if some of this can't be extruded
there.  I don't know if arch/mips is for us exclusively, or if we share it with
a few other project teams (other than -embedded -- do we have an OpenBSD MIPS
team or such??).


-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And our
lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic