[gentoo-mips] Reducing the number of the MIPS supported stages

[gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

Hi all,

Right now the number of stages for each endianness is 8:

- mips1
- mips32
- mips32r2
- mips3
- mips4
- mips4_r10
- mips64
- mips64r2

==> 16 stages in total.

This takes quite a bit of time for all stages to be built (by the time 
everything is built, we are one month passed the time the snapshot was 
taken). How about stop building stages for mips1, mips3 and mips4? We 
keep the existing stages on the mirrors but we will no longer update 
them (or maybe we do on per user or per case basis). I understand there 
is hardware for these ISAs but how often do people actually use the new 
stages?

Just to be clear, I am not suggesting for the team to stop supporting 
these ISAs but to stop building new stages and let the users of such 
ISAs, grab an old stage3 and do the update themselves if needed.

This will free up some hardware resources for building different stages 
for the newer ISAs (maybe more non-multilib n32 and n64 variants etc)

What does everyone think?

(CC'ing releng just to keep them in the loop)

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Panagiotis Christopoulos]

[-- Attachment #1: Type: text/plain, Size: 1611 bytes --]

On 15:06 Mon 05 May     , Markos Chandras wrote:
> Hi all,
Hi,

> ... 
> This takes quite a bit of time for all stages to be built (by the time 
> everything is built, we are one month passed the time the snapshot was 
> taken). How about stop building stages for mips1, mips3 and mips4? We 
> keep the existing stages on the mirrors but we will no longer update 
> them (or maybe we do on per user or per case basis). I understand there 
> is hardware for these ISAs but how often do people actually use the new 
> stages?
> 
> Just to be clear, I am not suggesting for the team to stop supporting 
> these ISAs but to stop building new stages and let the users of such 
> ISAs, grab an old stage3 and do the update themselves if needed.
> 
> This will free up some hardware resources for building different stages 
> for the newer ISAs (maybe more non-multilib n32 and n64 variants etc)
> 
> What does everyone think?

I agree. If we don't have the resources to build everything for everyone, lets
just build what is mainstream at the moment. I don't know catalyst's internals
or how painful it would be to update the whole set once per year or a bit longer, but it could
be nice, because I'm not sure how possible would be to update a stage3 after >
1-2 years.

But again, because I'm not in the MIPS industry, so can't say whether people are really
using the older ISAs stages, if you suspect nobody is using them just do as you
say and stop updating them, and we build any upon request.

Panagiotis

-- 
Panagiotis Christopoulos ( pchrist )
    ( Gentoo Lisp Project )

[-- Attachment #2: Type: application/pgp-signature, Size: 291 bytes --]

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Justin Cormack]

[-- Attachment #1: Type: text/plain, Size: 1322 bytes --]

On May 5, 2014 3:07 PM, "Markos Chandras" <hwoarang@gentoo.org> wrote:
>
> Hi all,
>
> Right now the number of stages for each endianness is 8:
>
> - mips1
> - mips32
> - mips32r2
> - mips3
> - mips4
> - mips4_r10
> - mips64
> - mips64r2
>
> ==> 16 stages in total.
>
> This takes quite a bit of time for all stages to be built (by the time
everything is built, we are one month passed the time the snapshot was
taken). How about stop building stages for mips1, mips3 and mips4? We keep
the existing stages on the mirrors but we will no longer update them (or
maybe we do on per user or per case basis). I understand there is hardware
for these ISAs but how often do people actually use the new stages?
>
> Just to be clear, I am not suggesting for the team to stop supporting
these ISAs but to stop building new stages and let the users of such ISAs,
grab an old stage3 and do the update themselves if needed.
>
> This will free up some hardware resources for building different stages
for the newer ISAs (maybe more non-multilib n32 and n64 variants etc)
>
> What does everyone think?
>

Agree in general but not quite sure about the choice. Mips3 (fuloong) is
still quite widely used I think. Mips32 might not be though, Android seems
to be mips32r2

Has anyone asked Imagination for resources to support this?

Justin

[-- Attachment #2: Type: text/html, Size: 1651 bytes --]

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 05/05/2014 06:13 PM, Justin Cormack wrote:
> On May 5, 2014 3:07 PM, "Markos Chandras" <hwoarang@gentoo.org>
> wrote:
>> 
>> Hi all,
>> 
>> Right now the number of stages for each endianness is 8:
>> 
>> - mips1 - mips32 - mips32r2 - mips3 - mips4 - mips4_r10 - mips64 
>> - mips64r2
>> 
>> ==> 16 stages in total.
>> 
>> This takes quite a bit of time for all stages to be built (by the
>> time
> everything is built, we are one month passed the time the snapshot
> was taken). How about stop building stages for mips1, mips3 and
> mips4? We keep the existing stages on the mirrors but we will no
> longer update them (or maybe we do on per user or per case basis).
> I understand there is hardware for these ISAs but how often do
> people actually use the new stages?
>> 
>> Just to be clear, I am not suggesting for the team to stop
>> supporting
> these ISAs but to stop building new stages and let the users of
> such ISAs, grab an old stage3 and do the update themselves if
> needed.
>> 
>> This will free up some hardware resources for building different
>> stages
> for the newer ISAs (maybe more non-multilib n32 and n64 variants
> etc)
>> 
>> What does everyone think?
>> 
> 
> Agree in general but not quite sure about the choice. Mips3
> (fuloong) is still quite widely used I think. Mips32 might not be
> though, Android seems to be mips32r2
> 
> Has anyone asked Imagination for resources to support this?

I work for Imagination :) and resources is not huge problem. Like I
said, I am happy to do it provided there is good reason (and userbase)
to do it. Regarding mips3/fuloong users can still get an existing
stage3 and update afterwards. Like I said before, I never proposed to
drop these stages from mirrors, just to stop updating them.

Ideas are welcomed, my initial email was just a suggestion and my goal
was to kick off a discussion

- -- 
Regards,
Markos Chandras
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQF8BAEBCgBmBQJTZ8dbXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w
ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGRDlGMzA4MUI2MzBDODQ4RDBGOEYxMjQx
RjEwRUQ0QjgxREVCRjE5AAoJEB8Q7UuB3r8ZuLAIAJjoeKWyuzGgBopc8vs5gB2E
YIeuHp4HXHBiLIeqfUz+pNc+FSlPu3dQPUe5Q3XQndIrQjNsRVuzK8N2deEGEFGI
q50wllAH2d4Mtc2sFv1EaXc0YM6Xlu7+aehT7YYUvoN/mbiWAw9r30FaMCZx6aHe
gsalycjgdzlGItfPvHUMjRPdE3O+idJr7IMfDKBQ2H7VgEUXPAeSLovgS/CejNJx
1sfxtWPBKzMEOAzzWUXB49zw3ygDiR2WyvxjlkVcAs+HfBWEzDOyHmA1yvYHwcVU
9rip/gEiiP48NnxQZkcaxRfAPHXbWr9iJjpuX1OFbp6he6Mir5C1A+ba07X0gLE=
=Y7sq
-----END PGP SIGNATURE-----

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/05/2014 06:04 PM, Panagiotis Christopoulos wrote:
> On 15:06 Mon 05 May     , Markos Chandras wrote:
>> Hi all,
> Hi,
> 
>> ... 
>> This takes quite a bit of time for all stages to be built (by the time 
>> everything is built, we are one month passed the time the snapshot was 
>> taken). How about stop building stages for mips1, mips3 and mips4? We 
>> keep the existing stages on the mirrors but we will no longer update 
>> them (or maybe we do on per user or per case basis). I understand there 
>> is hardware for these ISAs but how often do people actually use the new 
>> stages?
>>
>> Just to be clear, I am not suggesting for the team to stop supporting 
>> these ISAs but to stop building new stages and let the users of such 
>> ISAs, grab an old stage3 and do the update themselves if needed.
>>
>> This will free up some hardware resources for building different stages 
>> for the newer ISAs (maybe more non-multilib n32 and n64 variants etc)
>>
>> What does everyone think?
> 
> I agree. If we don't have the resources to build everything for everyone,

Resources is not a huge problem at the moment. Like I explained on the
other email, I am mainly interested in discussing whether doing so is
desired or not.

 lets
> just build what is mainstream at the moment. I don't know catalyst's internals
> or how painful it would be to update the whole set once per year or a bit longer, but it could
> be nice, because I'm not sure how possible would be to update a stage3 after >
> 1-2 years.

Reducing the frequency of such stages can also be an option. I didn't
quite rule that out yet

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Matt Turner]

On Mon, May 5, 2014 at 7:06 AM, Markos Chandras <hwoarang@gentoo.org> wrote:
> Hi all,
>
> Right now the number of stages for each endianness is 8:
>
> - mips1
> - mips32
> - mips32r2

Do we need r1 and r2 stages?

(Isn't r3 a thing now?)

> - mips3
> - mips4
> - mips4_r10

Big endian only.

> - mips64
> - mips64r2

Do we need r1 and r2 stages?

>
> ==> 16 stages in total.
>
> This takes quite a bit of time for all stages to be built (by the time
> everything is built, we are one month passed the time the snapshot was
> taken). How about stop building stages for mips1, mips3 and mips4?

How often are you building stages?

Resources aren't a problem but it takes a month to build everything?
Don't you have a 16 or 32-core build system? Cavium gave me ssh access
to one that I was planning to build stages on, but I never tried.

I guess I don't understand the problem you're facing.

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/05/2014 06:22 PM, Matt Turner wrote:
> On Mon, May 5, 2014 at 7:06 AM, Markos Chandras <hwoarang@gentoo.org> wrote:
>> Hi all,
>>
>> Right now the number of stages for each endianness is 8:
>>[...]
> 
> Do we need r1 and r2 stages?

Why not? r2 has been around ~10 years. But r1 devices are still present.

> 
>>
>> ==> 16 stages in total.
>>
>> This takes quite a bit of time for all stages to be built (by the time
>> everything is built, we are one month passed the time the snapshot was
>> taken). How about stop building stages for mips1, mips3 and mips4?
> 
> How often are you building stages?
Roughly every 3 months.

> 
> Resources aren't a problem but it takes a month to build everything?
Well, my free time is also limited so it's not just the resources that
may or may not cause some problem :) Preparing catalyst and solving
blockers in ~arch takes a considerable amount of my Gentoo time.

> Don't you have a 16 or 32-core build system? Cavium gave me ssh access
> to one that I was planning to build stages on, but I never tried.
> 
> I guess I don't understand the problem you're facing.
> 

My question (and not a problem) is whether building stages for old ISAs
is desired or not. If it is, then that's fine. If not, then maybe we can
stop building stages or maybe we can build them once a year? Like you
said, with the additions of newer ISAs (like -r3) the total number of
stages will grow even more. I think we are the arch with the most stages
in Gentoo, which is a great thing(!) but maybe we can reconsider our
options, and make room for newer ISAs and variants?

-- 
Regards,
Markos Chandras

[gentoo-mips] Re: Reducing the number of the MIPS supported stages

[Anthony G. Basile]

On 05/05/2014 10:06 AM, Markos Chandras wrote:
> Hi all,
>
> Right now the number of stages for each endianness is 8:
>
> - mips1
> - mips32
> - mips32r2
> - mips3
> - mips4
> - mips4_r10
> - mips64
> - mips64r2
>
> ==> 16 stages in total.
>
> This takes quite a bit of time for all stages to be built (by the time 
> everything is built, we are one month passed the time the snapshot was 
> taken). How about stop building stages for mips1, mips3 and mips4? We 
> keep the existing stages on the mirrors but we will no longer update 
> them (or maybe we do on per user or per case basis). I understand 
> there is hardware for these ISAs but how often do people actually use 
> the new stages?
>
> Just to be clear, I am not suggesting for the team to stop supporting 
> these ISAs but to stop building new stages and let the users of such 
> ISAs, grab an old stage3 and do the update themselves if needed.
>
> This will free up some hardware resources for building different 
> stages for the newer ISAs (maybe more non-multilib n32 and n64 
> variants etc)
>
> What does everyone think?
>
> (CC'ing releng just to keep them in the loop)
>

FYI, for uclibc I'm only doing big endian mips32r2 and little endian 
mips3 (aka mipsel3).  Those are in use (atheros ar71xx boards and 
lemote, respectively).

Since you can build a lower level ISA on a board capable of a higher 
level ISA, maybe you want to choose along those lines, eg you can build 
mips1 using catalyst on an ar71xx board running mips32r2.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA

Re: [gentoo-mips] Re: Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/05/2014 08:19 PM, Anthony G. Basile wrote:
> On 05/05/2014 10:06 AM, Markos Chandras wrote:
>> Hi all,
>>
>> Right now the number of stages for each endianness is 8:
>>
>> - mips1
>> - mips32
>> - mips32r2
>> - mips3
>> - mips4
>> - mips4_r10
>> - mips64
>> - mips64r2
>>
>> ==> 16 stages in total.
>>
>> This takes quite a bit of time for all stages to be built (by the time
>> everything is built, we are one month passed the time the snapshot was
>> taken). How about stop building stages for mips1, mips3 and mips4? We
>> keep the existing stages on the mirrors but we will no longer update
>> them (or maybe we do on per user or per case basis). I understand
>> there is hardware for these ISAs but how often do people actually use
>> the new stages?
>>
>> Just to be clear, I am not suggesting for the team to stop supporting
>> these ISAs but to stop building new stages and let the users of such
>> ISAs, grab an old stage3 and do the update themselves if needed.
>>
>> This will free up some hardware resources for building different
>> stages for the newer ISAs (maybe more non-multilib n32 and n64
>> variants etc)
>>
>> What does everyone think?
>>
>> (CC'ing releng just to keep them in the loop)
>>
> 
> FYI, for uclibc I'm only doing big endian mips32r2 and little endian
> mips3 (aka mipsel3).  Those are in use (atheros ar71xx boards and
> lemote, respectively).
> 
> Since you can build a lower level ISA on a board capable of a higher
> level ISA, maybe you want to choose along those lines, eg you can build
> mips1 using catalyst on an ar71xx board running mips32r2.
> 

Yes there are a lot of options. Sadly we have no metrics on how many
people use the old ISA stages or how many of them build their own thing.
I don't want to decide anything by myself, I want to know what others
think and make a decision as a team and community (if we are going to
decide anything at all) :)

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/05/2014 13:22, Matt Turner wrote:
> On Mon, May 5, 2014 at 7:06 AM, Markos Chandras <hwoarang@gentoo.org> wrote:
>> Hi all,
>>
>> Right now the number of stages for each endianness is 8:
>>
>> - mips1
>> - mips32
>> - mips32r2
> 
> Do we need r1 and r2 stages?
> 
> (Isn't r3 a thing now?)

I cannot speak for anything outside of the standard/original MIPS ISAs, but
I thought we killed off mips1 long ago.  When did that come back?  Almost
anything out there should be able to handle mips2 at a bare minimum (only
R2000 and R3000-based systems, like certain DECStations, would need mips1).
 mips2 is also the branch point for the mips32r* ISAs, so if any of the
original, 32-bit ISAs should be kept, that would be mips2.  mips1 can go.


>> - mips3
>> - mips4
>> - mips4_r10
> 
> Big endian only.
> 

Seconded.  We still support SGI systems, which max out at the mips4 ISA.  If
we drop support for mips3, that will cut off any R4x00-based Indigo2 and
Indy systems, but I believe those to be fairly rare anyways (and the R4600
still has quirks about it that can pose challenges).

mips4 little-endian is only useful for the old Cobalt RaQ2 and Qube
hardware, which I don't think is reasonable anymore.  Modern gcc takes long
enough on an SGI system with a secondary cache -- the Cobalt's low-powered
RM5231 with no L2 would make 4.8 or 4.9 even more excruciating.  Max memory
there is 256MB of RAM, and I don't know if gcc-4.9 will build under that.

And I don't see a point in doing an R10K-specific mips4 build.  The standard
mips4 build is good enough, and users of R10K systems can rebuild to gain
the R10K enhancements if needed.

So, keep mips4 big-endian as a standard stage build, let's do a mips3
big-endian build maybe once or twice a year, and drop mips4 little-endian
and R10K mips4.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Matt Turner]

On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
> And I don't see a point in doing an R10K-specific mips4 build.  The standard
> mips4 build is good enough, and users of R10K systems can rebuild to gain
> the R10K enhancements if needed.

That's not true, as far as I'm aware. glibc for instance has
compile-time work-arounds for R10k errata. Hangs result without the
fix.

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Matt Turner]

On Mon, May 5, 2014 at 10:29 AM, Markos Chandras <hwoarang@gentoo.org> wrote:
> On 05/05/2014 06:22 PM, Matt Turner wrote:
>> On Mon, May 5, 2014 at 7:06 AM, Markos Chandras <hwoarang@gentoo.org> wrote:
>>> Hi all,
>>>
>>> Right now the number of stages for each endianness is 8:
>>>[...]
>>
>> Do we need r1 and r2 stages?
>
> Why not? r2 has been around ~10 years. But r1 devices are still present.
>
>>
>>>
>>> ==> 16 stages in total.
>>>
>>> This takes quite a bit of time for all stages to be built (by the time
>>> everything is built, we are one month passed the time the snapshot was
>>> taken). How about stop building stages for mips1, mips3 and mips4?
>>
>> How often are you building stages?
> Roughly every 3 months.

I wouldn't mind reducing this to every 6 months or so. It was
certainly a "when I felt like it" schedule for me.

>> Resources aren't a problem but it takes a month to build everything?
> Well, my free time is also limited so it's not just the resources that
> may or may not cause some problem :) Preparing catalyst and solving
> blockers in ~arch takes a considerable amount of my Gentoo time.

It was my experience that fixing the ~arch issues was something that
you did once for every batch of stages. You just reused the fixed up
portage snapshot for everything. It wasn't a trivial effort by any
means though.

>> Don't you have a 16 or 32-core build system? Cavium gave me ssh access
>> to one that I was planning to build stages on, but I never tried.
>>
>> I guess I don't understand the problem you're facing.
>>
>
> My question (and not a problem) is whether building stages for old ISAs
> is desired or not. If it is, then that's fine. If not, then maybe we can
> stop building stages or maybe we can build them once a year? Like you
> said, with the additions of newer ISAs (like -r3) the total number of
> stages will grow even more. I think we are the arch with the most stages
> in Gentoo, which is a great thing(!) but maybe we can reconsider our
> options, and make room for newer ISAs and variants?

Once a year for stages with questionable usefulness seems reasonable to me.

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Matt Turner]

On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
> I cannot speak for anything outside of the standard/original MIPS ISAs, but
> I thought we killed off mips1 long ago.  When did that come back?  Almost
> anything out there should be able to handle mips2 at a bare minimum (only
> R2000 and R3000-based systems, like certain DECStations, would need mips1).
>  mips2 is also the branch point for the mips32r* ISAs, so if any of the
> original, 32-bit ISAs should be kept, that would be mips2.  mips1 can go.

We've had this discussion before. If you're going to have >mips2
stages, then there's zero reason to have mips2 stages since mips2
effectively doesn't exist.

mips1 is a significantly smaller build than other stages, since
there's only one ABI. It's really not much time, relatively speaking.

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/05/2014 19:36, Matt Turner wrote:
> On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
>> I cannot speak for anything outside of the standard/original MIPS ISAs, but
>> I thought we killed off mips1 long ago.  When did that come back?  Almost
>> anything out there should be able to handle mips2 at a bare minimum (only
>> R2000 and R3000-based systems, like certain DECStations, would need mips1).
>>  mips2 is also the branch point for the mips32r* ISAs, so if any of the
>> original, 32-bit ISAs should be kept, that would be mips2.  mips1 can go.
> 
> We've had this discussion before. If you're going to have >mips2
> stages, then there's zero reason to have mips2 stages since mips2
> effectively doesn't exist.

It's been a while, but I thought we only kept a mips2 stage1 around for
those that wanted a baseline to build their own stage2 or stage3's from.
You can do this with a mips1 as well, but mips2 is, more or less, the
baseline from which all other possible ISAs and stages can be built from, as
long as you don't care about R2k or R3k CPUs.  stage3's can be the higher
mips32r* ISAs.

I personally don't see a point in having mips1 or mips2 stage3's, only a
stage1 to use as a bootstrap for new machines or ISAs.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/05/2014 19:30, Matt Turner wrote:
> On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
>> And I don't see a point in doing an R10K-specific mips4 build.  The standard
>> mips4 build is good enough, and users of R10K systems can rebuild to gain
>> the R10K enhancements if needed.
> 
> That's not true, as far as I'm aware. glibc for instance has
> compile-time work-arounds for R10k errata. Hangs result without the
> fix.

Ah, that bug.  Been a while indeed.  My R10K IP28 is in the closet and R10K
O2, last time I tried a kernel, hated me rather fantastically.

The Octane, on the other hand, partially boots now in 3.14.1.  At least, I
can get to a working Odyssey console w/ SCSI, keyboard, and even the mouse
looks to work.  Still fighting the IOC3 over RTC interrupts, and SCSI is
abysmally slow (too many interrupts, I think -- ~4MB/sec r/w).  But it works
again.  No serial, RAD1, Impact or SMP yet.

When I fix a few more things, I'll probably attempt to bootstrap a new
userland on it.  Current one is from 2009.  So, when that happens, I'll be
able to re-verify glibc and the R10k thing again, I guess.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/06/2014 01:09 AM, Joshua Kinard wrote:
> On 05/05/2014 19:36, Matt Turner wrote:
>> On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
>>> I cannot speak for anything outside of the standard/original MIPS ISAs, but
>>> I thought we killed off mips1 long ago.  When did that come back?  Almost
>>> anything out there should be able to handle mips2 at a bare minimum (only
>>> R2000 and R3000-based systems, like certain DECStations, would need mips1).
>>>  mips2 is also the branch point for the mips32r* ISAs, so if any of the
>>> original, 32-bit ISAs should be kept, that would be mips2.  mips1 can go.
>>
>> We've had this discussion before. If you're going to have >mips2
>> stages, then there's zero reason to have mips2 stages since mips2
>> effectively doesn't exist.
> 
> It's been a while, but I thought we only kept a mips2 stage1 around for
> those that wanted a baseline to build their own stage2 or stage3's from.
> You can do this with a mips1 as well, but mips2 is, more or less, the
> baseline from which all other possible ISAs and stages can be built from, as
> long as you don't care about R2k or R3k CPUs.  stage3's can be the higher
> mips32r* ISAs.
> 
> I personally don't see a point in having mips1 or mips2 stage3's, only a
> stage1 to use as a bootstrap for new machines or ISAs.
> 

(picking up a random thread)

Ok thanks for the replies.

Ok I think it's safe to proceed with the following:
- Stop mips1 builds (we don't have mips2)
- Reduce the frequency to once-a-year for mips3 and mips4. Updating
these stages every year with catalyst will be a lot of fun ;)

@kumba: You mentioned too many times that I wanted to "drop" support for
mips3 and mips4. I never said that (I am sort-of tired keep repeating
that). All I said (again) was to reduce the frequency or stop building
them at all. Users can still get an existing mips3/mips4 stage3 and
update themselves

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/06/2014 03:07, Markos Chandras wrote:
> @kumba: You mentioned too many times that I wanted to "drop" support for
> mips3 and mips4. I never said that (I am sort-of tired keep repeating
> that). All I said (again) was to reduce the frequency or stop building
> them at all. Users can still get an existing mips3/mips4 stage3 and
> update themselves

Maybe it's just my way of interpretation, but in your opening paragraph,
even though you said we wouldn't drop support, you did suggest not creating
new stages for mips1-mips4.

Given a sufficiently long-enough time, that effectively drops support due to
bitrot.  Like I mentioned w/ the 2009-era userland on this Octane, I am not
going to even try to update that, simply due to the amount of time it would
take, even if I figure the IRQ prioritization bugs out.

So, my apologies if I read it wrong, but that's just how I see it.


> (picking up a random thread)
> 
> Ok thanks for the replies.
> 
> Ok I think it's safe to proceed with the following:
> - Stop mips1 builds (we don't have mips2)

I'll defer to Matt to chime in to my last message and correct me anymore, if
needed, but, I think we'll want to keep either a mips1 or a mips2, but not
both.  As well as decide whether it's a full stage3 or just a simple
stage1/stage2 tarball so people have a base from which to start a port to a
new MIPS machine if needed.  That can get updated once a year, especially if
it's a stage1 which shouldn't take long at all.


> - Reduce the frequency to once-a-year for mips3 and mips4. Updating
> these stages every year with catalyst will be a lot of fun ;)

NAK, At least once every 6 months, and preferably shortly after the .1
release of a new major gcc rev, given gcc's absurd compile time now.  Gentoo
moves fast, and a lot can change in a year.

Anyone looking to work with older SGI hardware needs something to start
from, and if a stage3 is too old, it can take those machines a while to
update everything.  I know most others on the team don't care for the SGI
hardware anymore, but it's still the most widely-available MIPS hardware for
hobbyists and tinkerers.

Otherwise, just e-mail me your mips3/mips4/mips4_r10 spec files, any custom
tweaks/changes to catalyst, and any specific instructions you do
before/during/after a catalyst build and I'll put the O2 to work if needed.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Anthony G. Basile]

On 05/06/2014 03:07 AM, Markos Chandras wrote:
> On 05/06/2014 01:09 AM, Joshua Kinard wrote:
>> On 05/05/2014 19:36, Matt Turner wrote:
>>> On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
>>>> I cannot speak for anything outside of the standard/original MIPS ISAs, but
>>>> I thought we killed off mips1 long ago.  When did that come back?  Almost
>>>> anything out there should be able to handle mips2 at a bare minimum (only
>>>> R2000 and R3000-based systems, like certain DECStations, would need mips1).
>>>>   mips2 is also the branch point for the mips32r* ISAs, so if any of the
>>>> original, 32-bit ISAs should be kept, that would be mips2.  mips1 can go.
>>>
>>> We've had this discussion before. If you're going to have >mips2
>>> stages, then there's zero reason to have mips2 stages since mips2
>>> effectively doesn't exist.
>>
>> It's been a while, but I thought we only kept a mips2 stage1 around for
>> those that wanted a baseline to build their own stage2 or stage3's from.
>> You can do this with a mips1 as well, but mips2 is, more or less, the
>> baseline from which all other possible ISAs and stages can be built from, as
>> long as you don't care about R2k or R3k CPUs.  stage3's can be the higher
>> mips32r* ISAs.
>>
>> I personally don't see a point in having mips1 or mips2 stage3's, only a
>> stage1 to use as a bootstrap for new machines or ISAs.
>>
>
> (picking up a random thread)
>
> Ok thanks for the replies.
>
> Ok I think it's safe to proceed with the following:
> - Stop mips1 builds (we don't have mips2)
> - Reduce the frequency to once-a-year for mips3 and mips4. Updating
> these stages every year with catalyst will be a lot of fun ;)
>
> @kumba: You mentioned too many times that I wanted to "drop" support for
> mips3 and mips4. I never said that (I am sort-of tired keep repeating
> that). All I said (again) was to reduce the frequency or stop building
> them at all. Users can still get an existing mips3/mips4 stage3 and
> update themselves
>

mipsel3 is used for the lemote.  Are you sure we should drop it?

-- 
Anthony G. Basile, Ph. D.
Chair of Information Technology
D'Youville College
Buffalo, NY 14201
(716) 829-8197

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/06/2014 12:32 PM, Anthony G. Basile wrote:
> On 05/06/2014 03:07 AM, Markos Chandras wrote:
>> On 05/06/2014 01:09 AM, Joshua Kinard wrote:
>>> On 05/05/2014 19:36, Matt Turner wrote:
>>>> On Mon, May 5, 2014 at 3:02 PM, Joshua Kinard <kumba@gentoo.org> wrote:
>>>>> I cannot speak for anything outside of the standard/original MIPS
>>>>> ISAs, but
>>>>> I thought we killed off mips1 long ago.  When did that come back? 
>>>>> Almost
>>>>> anything out there should be able to handle mips2 at a bare minimum
>>>>> (only
>>>>> R2000 and R3000-based systems, like certain DECStations, would need
>>>>> mips1).
>>>>>   mips2 is also the branch point for the mips32r* ISAs, so if any
>>>>> of the
>>>>> original, 32-bit ISAs should be kept, that would be mips2.  mips1
>>>>> can go.
>>>>
>>>> We've had this discussion before. If you're going to have >mips2
>>>> stages, then there's zero reason to have mips2 stages since mips2
>>>> effectively doesn't exist.
>>>
>>> It's been a while, but I thought we only kept a mips2 stage1 around for
>>> those that wanted a baseline to build their own stage2 or stage3's from.
>>> You can do this with a mips1 as well, but mips2 is, more or less, the
>>> baseline from which all other possible ISAs and stages can be built
>>> from, as
>>> long as you don't care about R2k or R3k CPUs.  stage3's can be the
>>> higher
>>> mips32r* ISAs.
>>>
>>> I personally don't see a point in having mips1 or mips2 stage3's, only a
>>> stage1 to use as a bootstrap for new machines or ISAs.
>>>
>>
>> (picking up a random thread)
>>
>> Ok thanks for the replies.
>>
>> Ok I think it's safe to proceed with the following:
>> - Stop mips1 builds (we don't have mips2)
>> - Reduce the frequency to once-a-year for mips3 and mips4. Updating
>> these stages every year with catalyst will be a lot of fun ;)
>>
>> @kumba: You mentioned too many times that I wanted to "drop" support for
>> mips3 and mips4. I never said that (I am sort-of tired keep repeating
>> that). All I said (again) was to reduce the frequency or stop building
>> them at all. Users can still get an existing mips3/mips4 stage3 and
>> update themselves
>>
> 
> mipsel3 is used for the lemote.  Are you sure we should drop it?
> 

No, I can still do that.

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/06/2014 09:10 AM, Joshua Kinard wrote:
> On 05/06/2014 03:07, Markos Chandras wrote:
>> @kumba: You mentioned too many times that I wanted to "drop" support for
>> mips3 and mips4. I never said that (I am sort-of tired keep repeating
>> that). All I said (again) was to reduce the frequency or stop building
>> them at all. Users can still get an existing mips3/mips4 stage3 and
>> update themselves
> 
> Maybe it's just my way of interpretation, but in your opening paragraph,
> even though you said we wouldn't drop support, you did suggest not creating
> new stages for mips1-mips4.
> 
> Given a sufficiently long-enough time, that effectively drops support due to
> bitrot.  Like I mentioned w/ the 2009-era userland on this Octane, I am not
> going to even try to update that, simply due to the amount of time it would
> take, even if I figure the IRQ prioritization bugs out.
> 
> So, my apologies if I read it wrong, but that's just how I see it.
> 
> 
>> (picking up a random thread)
>>
>> Ok thanks for the replies.
>>
>> Ok I think it's safe to proceed with the following:
>> - Stop mips1 builds (we don't have mips2)
> 
> I'll defer to Matt to chime in to my last message and correct me anymore, if
> needed, but, I think we'll want to keep either a mips1 or a mips2, but not
> both.  As well as decide whether it's a full stage3 or just a simple
> stage1/stage2 tarball so people have a base from which to start a port to a
> new MIPS machine if needed.  That can get updated once a year, especially if
> it's a stage1 which shouldn't take long at all.

I see no value for mips1 or mips2 so feel free to pick these up. Even if
someone is using them as bootstrap, then *any* mips1 stage3 would do.

> 
> 
>> - Reduce the frequency to once-a-year for mips3 and mips4. Updating
>> these stages every year with catalyst will be a lot of fun ;)
> 
> NAK, At least once every 6 months, and preferably shortly after the .1
> release of a new major gcc rev, given gcc's absurd compile time now.  Gentoo
> moves fast, and a lot can change in a year.

Forgive me but this almost sounds like an order :) This is not going to
happen, sorry :) I don't want to become a build robot and spend all my
Gentoo/MIPS time doing stages. With the introduction of new ISAs, the
total number of stages will grow even more, and like i explained
multiple times, this does not scale. There are other parts of the
architecture that needs some love too and right now I have no time for both.

And you haven't really convinced me why mips4 is desired, when mips3 can
run just fine on mips4 hardware. I think you need to be realist, and
take into consideration, not just your personal needs, but also the time
it actually takes to build and maintain all these stages. I explained
that so many times already, I am not going to do that again.
As Anthony said, mipsel3 is used by lemote, so keeping it alive is
probably a good thing (though the newer hardware is mips64 capable)

> 
> Otherwise, just e-mail me your mips3/mips4/mips4_r10 spec files, any custom
> tweaks/changes to catalyst, and any specific instructions you do
> before/during/after a catalyst build and I'll put the O2 to work if needed.
> 

There is nothing special about my spec files and I do nothing special in
catalyst so feel free to pick up the mips3 and mips4 stages. If you are
having troubles with catalyst email the gentoo-catalyst@ ML. That might
actually be a good way for you to become active again ;)

-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/06/2014 13:50, Markos Chandras wrote:
> On 05/06/2014 09:10 AM, Joshua Kinard wrote:
>> On 05/06/2014 03:07, Markos Chandras wrote:
>>> @kumba: You mentioned too many times that I wanted to "drop" support for
>>> mips3 and mips4. I never said that (I am sort-of tired keep repeating
>>> that). All I said (again) was to reduce the frequency or stop building
>>> them at all. Users can still get an existing mips3/mips4 stage3 and
>>> update themselves
>>
>> Maybe it's just my way of interpretation, but in your opening paragraph,
>> even though you said we wouldn't drop support, you did suggest not creating
>> new stages for mips1-mips4.
>>
>> Given a sufficiently long-enough time, that effectively drops support due to
>> bitrot.  Like I mentioned w/ the 2009-era userland on this Octane, I am not
>> going to even try to update that, simply due to the amount of time it would
>> take, even if I figure the IRQ prioritization bugs out.
>>
>> So, my apologies if I read it wrong, but that's just how I see it.
>>
>>
>>> (picking up a random thread)
>>>
>>> Ok thanks for the replies.
>>>
>>> Ok I think it's safe to proceed with the following:
>>> - Stop mips1 builds (we don't have mips2)
>>
>> I'll defer to Matt to chime in to my last message and correct me anymore, if
>> needed, but, I think we'll want to keep either a mips1 or a mips2, but not
>> both.  As well as decide whether it's a full stage3 or just a simple
>> stage1/stage2 tarball so people have a base from which to start a port to a
>> new MIPS machine if needed.  That can get updated once a year, especially if
>> it's a stage1 which shouldn't take long at all.
> 
> I see no value for mips1 or mips2 so feel free to pick these up. Even if
> someone is using them as bootstrap, then *any* mips1 stage3 would do.

WFM.


>>> - Reduce the frequency to once-a-year for mips3 and mips4. Updating
>>> these stages every year with catalyst will be a lot of fun ;)
>>
>> NAK, At least once every 6 months, and preferably shortly after the .1
>> release of a new major gcc rev, given gcc's absurd compile time now.  Gentoo
>> moves fast, and a lot can change in a year.
> 
> Forgive me but this almost sounds like an order :) This is not going to
> happen, sorry :) I don't want to become a build robot and spend all my
> Gentoo/MIPS time doing stages. With the introduction of new ISAs, the
> total number of stages will grow even more, and like i explained
> multiple times, this does not scale. There are other parts of the
> architecture that needs some love too and right now I have no time for both.

It's not an order, sorry.  Just a preference to avoid long compile times.
If you have a stage3 built against gcc-4.8.x, and 4.9.x becomes stable,
someone installing will have to compile 4.9, then rebuild everything to gain
the enhancements 4.9 will bring.  If a new stage3 is built after the .1
release of a new major rev (so that we avoid major bugs in the .0 release),
that should mitigate that problem.


> And you haven't really convinced me why mips4 is desired, when mips3 can
> run just fine on mips4 hardware. I think you need to be realist, and
> take into consideration, not just your personal needs, but also the time
> it actually takes to build and maintain all these stages. I explained
> that so many times already, I am not going to do that again.
> As Anthony said, mipsel3 is used by lemote, so keeping it alive is
> probably a good thing (though the newer hardware is mips64 capable)

Well, to me, mips3 != mipsel3.  Sorry about that.  When I say mips3/mips4
(lowercase), I usually refer to big-endian.  If I capitalize the ISA, i.e.,
MIPS-III or MIPS-IV, then I'm referring to the entire ISA, regardless of
endianness.

So, to re-clarify my original statement, for big-endian SGI systems, we
probably only need mips4 and I guess the mips4_r10 stages.  I don't know if
any of our users still have or run R4x00 mips3 big-endian equipment.  If so,
well, I can do that too, then.  It's just a higher electric bill :)

And it's not really my personal preference.  Based on my understanding of
what we currently support, that's what makes sense to me.  I know you work
on MIPS stuff for your day job, but it's a hobby for me, so I have to
prioritize things a bit differently.  That said, I've done catalyst runs
before, so I know how time-consuming they can be, especially if the build
breaks somewhere in the middle of a long compile.

I think what we need to do is instead of having just one person like you or
Matt do all of the stage building, separate out the ISAs/ABIs/etc to the
people that actually care most about it.  Anthony works with the mipsel3
Lemote hardware, so if he wants, he can take care of mipsel3 stages; I'll
handle the SGI stuff since I know a lot about those machines; and you can
cover whichever of the newer ISAs matter most to you.

Sound reasonable?  We can even work out a set timetable for stage building,
or just release individual stages on an as-needed basis.


>> Otherwise, just e-mail me your mips3/mips4/mips4_r10 spec files, any custom
>> tweaks/changes to catalyst, and any specific instructions you do
>> before/during/after a catalyst build and I'll put the O2 to work if needed.
>>
> 
> There is nothing special about my spec files and I do nothing special in
> catalyst so feel free to pick up the mips3 and mips4 stages. If you are
> having troubles with catalyst email the gentoo-catalyst@ ML. That might
> actually be a good way for you to become active again ;)

Back in the past, I had to tweak catalyst sometimes to get it to do stage
builds properly.  A lot of those bugs have probably been fixed by now, at
least for stage1-3.  The livecd stages and netboots, however, were much more
problematic.

If you can still send me at least one of your stage3 spec files, that'd be
appreciated.  It's been 5-6 years since I last messed with catalyst.  The
Octane was my build platform, but combined with the bitrot that prevented it
from booting, moving, a new job, etc, I never got around to setting up stage
building on the O2.  Now that I can boot Octane again, I can at least
recover my old spec files, though.  Might help if I ever attempt to tackle
the livecd or netboot builds again.

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Markos Chandras]

On 05/06/2014 10:37 PM, Joshua Kinard wrote:
> 
>> And you haven't really convinced me why mips4 is desired, when mips3 can
>> run just fine on mips4 hardware. I think you need to be realist, and
>> take into consideration, not just your personal needs, but also the time
>> it actually takes to build and maintain all these stages. I explained
>> that so many times already, I am not going to do that again.
>> As Anthony said, mipsel3 is used by lemote, so keeping it alive is
>> probably a good thing (though the newer hardware is mips64 capable)
> 
> Well, to me, mips3 != mipsel3.  Sorry about that.  When I say mips3/mips4
> (lowercase), I usually refer to big-endian.  If I capitalize the ISA, i.e.,
> MIPS-III or MIPS-IV, then I'm referring to the entire ISA, regardless of
> endianness.
> 
> So, to re-clarify my original statement, for big-endian SGI systems, we
> probably only need mips4 and I guess the mips4_r10 stages.  I don't know if
> any of our users still have or run R4x00 mips3 big-endian equipment.  If so,
> well, I can do that too, then.  It's just a higher electric bill :)

Dropping mips4 and building mips3 (yes BE) should satisfy everyone even
if they don't get the maximum optimizations right after the stage3
unpacking. But if you want to do all 3 of them, I will not stop you :)

> 
> And it's not really my personal preference.  Based on my understanding of
> what we currently support, that's what makes sense to me.  I know you work
> on MIPS stuff for your day job, but it's a hobby for me, so I have to
> prioritize things a bit differently.  That said, I've done catalyst runs
> before, so I know how time-consuming they can be, especially if the build
> breaks somewhere in the middle of a long compile.

I am not getting paid to do Gentoo/MIPS (not sure when I said the
opposite) so it's still a hobby for me that's why I want to do other
things as well. We can all agree that being an arch team member is not
just about building stages.

> 
> I think what we need to do is instead of having just one person like you or
> Matt do all of the stage building, separate out the ISAs/ABIs/etc to the
> people that actually care most about it.  Anthony works with the mipsel3
> Lemote hardware, so if he wants, he can take care of mipsel3 stages; I'll
> handle the SGI stuff since I know a lot about those machines; and you can
> cover whichever of the newer ISAs matter most to you.
> 
> Sound reasonable?  We can even work out a set timetable for stage building,
> or just release individual stages on an as-needed basis.

Works perfectly fine for me. I think each one building his stages on his
own timescale is better.

> 
> 
>>> Otherwise, just e-mail me your mips3/mips4/mips4_r10 spec files, any custom
>>> tweaks/changes to catalyst, and any specific instructions you do
>>> before/during/after a catalyst build and I'll put the O2 to work if needed.
>>>
>>
>> There is nothing special about my spec files and I do nothing special in
>> catalyst so feel free to pick up the mips3 and mips4 stages. If you are
>> having troubles with catalyst email the gentoo-catalyst@ ML. That might
>> actually be a good way for you to become active again ;)
> 
> Back in the past, I had to tweak catalyst sometimes to get it to do stage
> builds properly.  A lot of those bugs have probably been fixed by now, at
> least for stage1-3.

There are usually blockers and stuff due to MIPS being ~arch but
catalyst, as a tool, works fine.

The livecd stages and netboots, however, were much more
> problematic.

I don't think we do that anymore

> 
> If you can still send me at least one of your stage3 spec files, that'd be
> appreciated.  It's been 5-6 years since I last messed with catalyst.  The
> Octane was my build platform, but combined with the bitrot that prevented it
> from booting, moving, a new job, etc, I never got around to setting up stage
> building on the O2.  Now that I can boot Octane again, I can at least
> recover my old spec files, though.  Might help if I ever attempt to tackle
> the livecd or netboot builds again.

I will email you a complete set of stage1/2/3 for, say, mips3 and then
it's easy to figure out what do change for the rest :)


-- 
Regards,
Markos Chandras

Re: [gentoo-mips] Reducing the number of the MIPS supported stages

[Joshua Kinard]

On 05/07/2014 03:06, Markos Chandras wrote:
> 
> There are usually blockers and stuff due to MIPS being ~arch but
> catalyst, as a tool, works fine.
> 
> The livecd stages and netboots, however, were much more
>> problematic.
> 
> I don't think we do that anymore

Haven't in a *long* time.  However, it's been on my todo list for a good
while now.  The thing that killed netboots was the great uclibc break
several years ago where a chicken-and-egg scenario arose.  To compile from
uclibc-0.9.27 to .28, you needed ~gcc-4.4 (I think, maybe it was 3.4...).
However, you couldn't compile the required version of gcc, linked against
uclibc, without the .28 version.  So that broke a *lot* of people.

I have no idea how that was eventually resolved, but without uclibc,
netboots at the time were just too massive for the SGI Indy's to boot.
You'd get a "Not enough space in a FreeMemeoryArea() from ARCS.

Though...nowadays, I think that ARCS error has something to do with byte
alignment and not size.  Seems if you compile things in or out of the kernel
to increase or decrease its size, you can get a size that ARCS will be happy
with.

LiveCDs, OTOH...I do not look forward to even attempting those.  The two SGI
LiveCDs I produced used Gnome GDM, which was possible without sucking in
half of Gnome back then, which is not possible today, as far as I know.  I
don't know what's a good login manager these days, but it might be back to
classic XDM if I do attempt the things.


>> If you can still send me at least one of your stage3 spec files, that'd be
>> appreciated.  It's been 5-6 years since I last messed with catalyst.  The
>> Octane was my build platform, but combined with the bitrot that prevented it
>> from booting, moving, a new job, etc, I never got around to setting up stage
>> building on the O2.  Now that I can boot Octane again, I can at least
>> recover my old spec files, though.  Might help if I ever attempt to tackle
>> the livecd or netboot builds again.
> 
> I will email you a complete set of stage1/2/3 for, say, mips3 and then
> it's easy to figure out what do change for the rest :)

</smirk>

-- 
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
4096R/D25D95E3 2011-03-28

"The past tempts us, the present confuses us, the future frightens us.  And
our lives slip away, moment by moment, lost in that vast, terrible in-between."

--Emperor Turhan, Centauri Republic