[gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 1247 bytes --]

On 12/15/2009 05:54 PM, Denis Dupeyron wrote:
> The next meeting will be on 18 January 2010 at 1900UTC. The date was
> pushed back 2 weeks for various reasons but the main one is to let our
> livers recover. Which prompts me to make the following public
> announcement: Happy holidays! :o)
> 
> I will be following up discussions on various mailing lists to prepare
> the agenda. If you already want to suggest topics feel free to reply
> to this thread. You'll get a second chance with the meeting reminder
> approximately two weeks before the meeting. I will be sending a
> message about the two topics which did not make it last time and
> explain why. I should have sent that much earlier but well... you
> know...
> 
> Denis.
> 
> 

I will make it short, since i already requested it 3 times, did create a thread at gentoo-dev ML:

agenda topic: Discussion and approval for following item:

Adding real multilib features from current multilib-portage to currently hardmasked and testing
portage-2.2* for wider testing, more eyes looking at it and hopefully more people helping improving
it, so we can get a version, which most can accept for PMS and maybe next EAPI.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Fabian Groffen]

On 15-12-2009 09:54:36 -0700, Denis Dupeyron wrote:
> I will be following up discussions on various mailing lists to prepare
> the agenda. If you already want to suggest topics feel free to reply
> to this thread. You'll get a second chance with the meeting reminder
> approximately two weeks before the meeting. I will be sending a
> message about the two topics which did not make it last time and
> explain why. I should have sent that much earlier but well... you
> know...

I'd like to council to discuss the current *$^&!! policy of
-dev-announce and -dev.  I'd propose to at least implement the following
behaviour such that I:
- don't have to see some mails 3 (!) times and many 2 times
- don't get lost where the mail is/was
- get broken threading because the original mail was sent to another
  list

Proposed behaviour:
Aautomatically send all mail sent to -dev-announce to -dev.
Benefits:
- any reply-to hackery for -dev-announce to -dev unnecessary
- being subscrived to -dev alone is enough (alternatively -dev-announce
  can be /dev/null-ed)
- threads are complete, instead of scattered over some lists
- multiple copies can be avoided
- cross-list posting can be reduced to a minimum


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Jeremy Olexa]

Fabian Groffen wrote:
> On 15-12-2009 09:54:36 -0700, Denis Dupeyron wrote:
>> I will be following up discussions on various mailing lists to prepare
>> the agenda. If you already want to suggest topics feel free to reply
>> to this thread. You'll get a second chance with the meeting reminder
>> approximately two weeks before the meeting. I will be sending a
>> message about the two topics which did not make it last time and
>> explain why. I should have sent that much earlier but well... you
>> know...
> 
> I'd like to council to discuss the current *$^&!! policy of
> -dev-announce and -dev.  I'd propose to at least implement the following
> behaviour such that I:
> - don't have to see some mails 3 (!) times and many 2 times
> - don't get lost where the mail is/was
> - get broken threading because the original mail was sent to another
>   list
> 
> Proposed behaviour:
> Aautomatically send all mail sent to -dev-announce to -dev.
> Benefits:
> - any reply-to hackery for -dev-announce to -dev unnecessary
> - being subscrived to -dev alone is enough (alternatively -dev-announce
>   can be /dev/null-ed)
> - threads are complete, instead of scattered over some lists
> - multiple copies can be avoided
> - cross-list posting can be reduced to a minimum
> 
> 

In general there are too many mail lists to even care about the 
semantics of -dev-announce. Even this thread is being carried out on 
-dev and -council. Well, that was the attempt, but no one that has 
replied so far is on the -council list so the attempted thread on that 
list is dead too.
-Jeremy

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1072 bytes --]

On Sunday 20 December 2009 09:49:09 Fabian Groffen wrote:
> On 15-12-2009 09:54:36 -0700, Denis Dupeyron wrote:
> > I will be following up discussions on various mailing lists to prepare
> > the agenda. If you already want to suggest topics feel free to reply
> > to this thread. You'll get a second chance with the meeting reminder
> > approximately two weeks before the meeting. I will be sending a
> > message about the two topics which did not make it last time and
> > explain why. I should have sent that much earlier but well... you
> > know...
> 
> I'd like to council to discuss the current *$^&!! policy of
> -dev-announce and -dev.  I'd propose to at least implement the following
> behaviour such that I:
> - don't have to see some mails 3 (!) times and many 2 times
> - don't get lost where the mail is/was
> - get broken threading because the original mail was sent to another
>   list

get a sane mail client that automatically handles messages with duplicate ids 
and references.  cant say ive ever noticed a problem with kmail.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Fabian Groffen]

On 20-12-2009 15:01:30 -0500, Mike Frysinger wrote:
> On Sunday 20 December 2009 09:49:09 Fabian Groffen wrote:
> > On 15-12-2009 09:54:36 -0700, Denis Dupeyron wrote:
> > > I will be following up discussions on various mailing lists to prepare
> > > the agenda. If you already want to suggest topics feel free to reply
> > > to this thread. You'll get a second chance with the meeting reminder
> > > approximately two weeks before the meeting. I will be sending a
> > > message about the two topics which did not make it last time and
> > > explain why. I should have sent that much earlier but well... you
> > > know...
> > 
> > I'd like to council to discuss the current *$^&!! policy of
> > -dev-announce and -dev.  I'd propose to at least implement the following
> > behaviour such that I:
> > - don't have to see some mails 3 (!) times and many 2 times
> > - don't get lost where the mail is/was
> > - get broken threading because the original mail was sent to another
> >   list
> 
> get a sane mail client that automatically handles messages with duplicate ids 
> and references.  cant say ive ever noticed a problem with kmail.

and gmane or even archives.g.o?


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1781 bytes --]

On Sunday 20 December 2009 15:04:12 Fabian Groffen wrote:
> On 20-12-2009 15:01:30 -0500, Mike Frysinger wrote:
> > On Sunday 20 December 2009 09:49:09 Fabian Groffen wrote:
> > > On 15-12-2009 09:54:36 -0700, Denis Dupeyron wrote:
> > > > I will be following up discussions on various mailing lists to
> > > > prepare the agenda. If you already want to suggest topics feel free
> > > > to reply to this thread. You'll get a second chance with the meeting
> > > > reminder approximately two weeks before the meeting. I will be
> > > > sending a message about the two topics which did not make it last
> > > > time and explain why. I should have sent that much earlier but
> > > > well... you know...
> > >
> > > I'd like to council to discuss the current *$^&!! policy of
> > > -dev-announce and -dev.  I'd propose to at least implement the
> > > following behaviour such that I:
> > > - don't have to see some mails 3 (!) times and many 2 times
> > > - don't get lost where the mail is/was
> > > - get broken threading because the original mail was sent to another
> > >   list
> >
> > get a sane mail client that automatically handles messages with duplicate
> > ids and references.  cant say ive ever noticed a problem with kmail.
> 
> and gmane or even archives.g.o?

gmane is f-ed up already irregardless of what we do.  it eats cross-posted e-
mails for breakfast and doesnt tell anyone.

as for archives.g.o, file a bug if it isnt handling threading within a list 
properly.  i dont really see how your proposal here would break archives.g.o 
anyways.  someone sends an e-mail to both dev and dev-announce, it has the 
same id.  people respond and they all go to dev.  either way, archives.g.o 
should be seeing a sane thread on dev.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Fabian Groffen]

On 20-12-2009 22:16:30 -0500, Mike Frysinger wrote:
> gmane is f-ed up already irregardless of what we do.  it eats cross-posted e-
> mails for breakfast and doesnt tell anyone.
> 
> as for archives.g.o, file a bug if it isnt handling threading within a list 
> properly.  i dont really see how your proposal here would break archives.g.o 
> anyways.  someone sends an e-mail to both dev and dev-announce, it has the 
> same id.  people respond and they all go to dev.  either way, archives.g.o 
> should be seeing a sane thread on dev.

New devs are not announced to -dev, the mail is only sent to
-dev-announce.  The "January 2010 meeting date" mail was only sent to
-dev-announce (and -council), not to -dev, hence replies (that have to
go to -dev) are replies with the original mail missing.

If all mail that would go to -dev-announce would guaranteed be sent to
-dev as well, I didn't have to check -dev-announce, and archives.g.o
would also have the original "January 2010 meeting date" mail in the
thread on -dev.


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Richard Freeman]

On 12/21/2009 02:54 AM, Fabian Groffen wrote:
> If all mail that would go to -dev-announce would guaranteed be sent to
> -dev as well, I didn't have to check -dev-announce, and archives.g.o
> would also have the original "January 2010 meeting date" mail in the
> thread on -dev.
>
>

Or you could just subscribe to both and add this to your procmailrc:

:0 Wh: msgid.lock
| formail -D 8192 msgid.cache

:0 a:
.duplicates/new

Cross-posting in general tends to mess up threads, but there isn't much 
that can be done about that unless we just ban it.  However, most 
cross-posts are honest attempts to try to move list discussion to a 
place where it might better belong.

Honestly, list traffic is a lot better than it used to be, and I'm not 
sure that this is really a big problem these days.

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Fabian Groffen]

On 21-12-2009 06:30:23 -0500, Richard Freeman wrote:
> On 12/21/2009 02:54 AM, Fabian Groffen wrote:
> > If all mail that would go to -dev-announce would guaranteed be sent to
> > -dev as well, I didn't have to check -dev-announce, and archives.g.o
> > would also have the original "January 2010 meeting date" mail in the
> > thread on -dev.
> 
> Or you could just subscribe to both and add this to your procmailrc:
> 
> :0 Wh: msgid.lock
> | formail -D 8192 msgid.cache
> 
> :0 a:
> .duplicates/new

Does that fix archives.g.o?  No.

> Cross-posting in general tends to mess up threads, but there isn't much 
> that can be done about that unless we just ban it.  However, most 
> cross-posts are honest attempts to try to move list discussion to a 
> place where it might better belong.

For commits I can imagine, but for this, it's just pointless.


-- 
Fabian Groffen
Gentoo on a different level

[gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Denis Dupeyron]

On Sun, Dec 20, 2009 at 10:53 PM, Thomas Sachau <tommy@gentoo.org> wrote:
> I will make it short, since i already requested it 3 times, did create a thread at gentoo-dev ML:
>
> agenda topic: Discussion and approval for following item:
>
> Adding real multilib features from current multilib-portage to currently hardmasked and testing
> portage-2.2* for wider testing, more eyes looking at it and hopefully more people helping improving
> it, so we can get a version, which most can accept for PMS and maybe next EAPI.

Sorry, I forgot to send an email explaining what happened on the
council alias as promised. The consensus was that the project wasn't
mature enough to go ahead. Also more generally the council's job isn't
discussing but deciding, approving, etc... Discussing is what should
happen on mailing lists. Before you can bring that to the council we
need to see an as-much-as-possible finalized solution with any of the
following if applicable: portage branch with an implementation that
people can try, documentation, PMS patch, devmanual patches, and a
team. By team I mean: who is going to maintain this in the long run if
necessary? A one man team is a dead team, it's only a matter of time.
If the amd64 team is going to be the one doing this job, and this is
just an example buy the way, then we need them to tell us they're OK
with it.

Now don't get me wrong. I love your project and the last thing I want
is to shoot it down. Look at what happened with prefix. They wanted
the council to approve it immediately or else... We didn't cede to
pressure and worked with them to make it good enough for approval.
Right now I don't hear anybody arguing about prefix going forward. And
that's exactly what I want for your project, i.e. helping you making
it better instead of it fading and failing in the (not so) long run.

I will stop now because I'm at a bus stop near Mount Fuji and I need
to go. I hope the other council members, especially the more
technically competent ones than me, will get back to you on this and
offer help and advice. As soon as I have a better internet connection
I will contact you about this.

Denis.

[gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 5849 bytes --]

On 12/25/2009 06:10 AM, Denis Dupeyron wrote:
> On Sun, Dec 20, 2009 at 10:53 PM, Thomas Sachau <tommy@gentoo.org> wrote:
>> I will make it short, since i already requested it 3 times, did create a thread at gentoo-dev ML:
>>
>> agenda topic: Discussion and approval for following item:
>>
>> Adding real multilib features from current multilib-portage to currently hardmasked and testing
>> portage-2.2* for wider testing, more eyes looking at it and hopefully more people helping improving
>> it, so we can get a version, which most can accept for PMS and maybe next EAPI.
> 
> Sorry, I forgot to send an email explaining what happened on the
> council alias as promised. The consensus was that the project wasn't
> mature enough to go ahead. Also more generally the council's job isn't
> discussing but deciding, approving, etc... Discussing is what should
> happen on mailing lists.

Since i see noone arguing against adding the multilib features to current testing branch of portage,
the discussion part already seems to be done. so a simple approval is ok, drop the discussion request.

> Before you can bring that to the council we
> need to see an as-much-as-possible finalized solution with any of the
> following if applicable: portage branch with an implementation that
> people can try, documentation, PMS patch, devmanual patches, and a
> team.

Did you actually read my lines? I did NOT request an ACK to add it to PMS and next EAPI with a
complete spec. zmedico also has no problem with having a look and adding it, but since he was once
forced to remove an added feature, he now wants a council-ok before adding and improving it in
testing branch of main tree portage.

> By team I mean: who is going to maintain this in the long run if
> necessary? A one man team is a dead team, it's only a matter of time.

Much code is maintained by a single person, even the package maintainers have one maintainer and
some contributors. And with integrating it in main tree portage, there will additionally be the
portage team.

> If the amd64 team is going to be the one doing this job, and this is
> just an example buy the way, then we need them to tell us they're OK
> with it.

If any other team raises its voice, no problem with me, but it seems more like noone wants to do the
work.

> Now don't get me wrong. I love your project and the last thing I want
> is to shoot it down.

In this case, you will shoot it down. I wont be able to maintain the code alone, do all requested
code changes, spec writing, PMS patches etc beside my work and other projects i do within Gentoo. So
if you stop me from getting help by integrating it in *testing* portage (not the 2.1.* versions,
only the 2.2* versions, which contains more and experimental code), it will probably stay at the
current level and nothing more will happen.
I can live myself with a private fork of portage, which contains the features i like, it is easy to
only do some basic changes and some workarounds to get it personally working without much time.
But on the other hand, all people, who would like to see emul-linux-* packages dropped, would like
to actually compile recent versions of 32bit libs and would like to compile additional libs not in
those emul-linux-* packages in addition to multi-ABI support for other ARCHes and multi-SLOT support
for the different languages (support parallel install for different SLOTS of e.g. python, perl or
ruby) would have to do their own local or eclass hacks to get their thing working.

> Look at what happened with prefix. They wanted
> the council to approve it immediately or else... We didn't cede to
> pressure and worked with them to make it good enough for approval.

Something like "I/We want <x>,<y>,<z> or you wont get an approval" is no support and no "work with
them". So if you really would like to see it in, actually help with patches, SPEC writing,
discussion and code writing. Else i request an approval for getting some additional help instead of
just shooting it down.

> Right now I don't hear anybody arguing about prefix going forward. And
> that's exactly what I want for your project, i.e. helping you making
> it better instead of it fading and failing in the (not so) long run.

prefix is no one-man-team and the actual amount of people, who can and are willing to work on
portage code is limited, so which other way do you have to improve it as requested?

And yes, it is frustrating to see 3 council sessions and months going by and still no offer to
support, no discussion, no patches and no decision is made. I can see now, why such project did die
before and why people dont want to do such things, which can actually improve the experience with
Gentoo and can give our userbase more options and choice.

> 
> I will stop now because I'm at a bus stop near Mount Fuji and I need
> to go. I hope the other council members, especially the more
> technically competent ones than me, will get back to you on this and
> offer help and advice. As soon as I have a better internet connection
> I will contact you about this.

Feel free to do so.

P.S.: I dont want to affront you or anyone else personally, but the way, how it currently goes. I
know from IRC, forums and mails, that there are people around, who would like to see
multilib-features in portage. But with such frustrating none-actions like this, noone should wonder
why such things are not implemented, also there are people, who would like to see it and even
people, who would help doing it and creating code for it. That does not actually speak for Gentoo,
more against it (and it is not the only point, where Gentoo could improve, but does not, but that
could be part of another big mail).


-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Doug Goldstein]

On Fri, Dec 25, 2009 at 8:00 AM, Thomas Sachau <tommy@gentoo.org> wrote:
> On 12/25/2009 06:10 AM, Denis Dupeyron wrote:
>> On Sun, Dec 20, 2009 at 10:53 PM, Thomas Sachau <tommy@gentoo.org> wrote:
>>> I will make it short, since i already requested it 3 times, did create a thread at gentoo-dev ML:
>>>
>>> agenda topic: Discussion and approval for following item:
>>>
>>> Adding real multilib features from current multilib-portage to currently hardmasked and testing
>>> portage-2.2* for wider testing, more eyes looking at it and hopefully more people helping improving
>>> it, so we can get a version, which most can accept for PMS and maybe next EAPI.
>>
>> Sorry, I forgot to send an email explaining what happened on the
>> council alias as promised. The consensus was that the project wasn't
>> mature enough to go ahead. Also more generally the council's job isn't
>> discussing but deciding, approving, etc... Discussing is what should
>> happen on mailing lists.
>
> Since i see noone arguing against adding the multilib features to current testing branch of portage,
> the discussion part already seems to be done. so a simple approval is ok, drop the discussion request.
>
>> Before you can bring that to the council we
>> need to see an as-much-as-possible finalized solution with any of the
>> following if applicable: portage branch with an implementation that
>> people can try, documentation, PMS patch, devmanual patches, and a
>> team.
>
> Did you actually read my lines? I did NOT request an ACK to add it to PMS and next EAPI with a
> complete spec. zmedico also has no problem with having a look and adding it, but since he was once
> forced to remove an added feature, he now wants a council-ok before adding and improving it in
> testing branch of main tree portage.
>
>> By team I mean: who is going to maintain this in the long run if
>> necessary? A one man team is a dead team, it's only a matter of time.
>
> Much code is maintained by a single person, even the package maintainers have one maintainer and
> some contributors. And with integrating it in main tree portage, there will additionally be the
> portage team.
>
>> If the amd64 team is going to be the one doing this job, and this is
>> just an example buy the way, then we need them to tell us they're OK
>> with it.
>
> If any other team raises its voice, no problem with me, but it seems more like noone wants to do the
> work.
>
>> Now don't get me wrong. I love your project and the last thing I want
>> is to shoot it down.
>
> In this case, you will shoot it down. I wont be able to maintain the code alone, do all requested
> code changes, spec writing, PMS patches etc beside my work and other projects i do within Gentoo. So
> if you stop me from getting help by integrating it in *testing* portage (not the 2.1.* versions,
> only the 2.2* versions, which contains more and experimental code), it will probably stay at the
> current level and nothing more will happen.
> I can live myself with a private fork of portage, which contains the features i like, it is easy to
> only do some basic changes and some workarounds to get it personally working without much time.
> But on the other hand, all people, who would like to see emul-linux-* packages dropped, would like
> to actually compile recent versions of 32bit libs and would like to compile additional libs not in
> those emul-linux-* packages in addition to multi-ABI support for other ARCHes and multi-SLOT support
> for the different languages (support parallel install for different SLOTS of e.g. python, perl or
> ruby) would have to do their own local or eclass hacks to get their thing working.
>
>> Look at what happened with prefix. They wanted
>> the council to approve it immediately or else... We didn't cede to
>> pressure and worked with them to make it good enough for approval.
>
> Something like "I/We want <x>,<y>,<z> or you wont get an approval" is no support and no "work with
> them". So if you really would like to see it in, actually help with patches, SPEC writing,
> discussion and code writing. Else i request an approval for getting some additional help instead of
> just shooting it down.
>
>> Right now I don't hear anybody arguing about prefix going forward. And
>> that's exactly what I want for your project, i.e. helping you making
>> it better instead of it fading and failing in the (not so) long run.
>
> prefix is no one-man-team and the actual amount of people, who can and are willing to work on
> portage code is limited, so which other way do you have to improve it as requested?
>
> And yes, it is frustrating to see 3 council sessions and months going by and still no offer to
> support, no discussion, no patches and no decision is made. I can see now, why such project did die
> before and why people dont want to do such things, which can actually improve the experience with
> Gentoo and can give our userbase more options and choice.
>
>>
>> I will stop now because I'm at a bus stop near Mount Fuji and I need
>> to go. I hope the other council members, especially the more
>> technically competent ones than me, will get back to you on this and
>> offer help and advice. As soon as I have a better internet connection
>> I will contact you about this.
>
> Feel free to do so.
>
> P.S.: I dont want to affront you or anyone else personally, but the way, how it currently goes. I
> know from IRC, forums and mails, that there are people around, who would like to see
> multilib-features in portage. But with such frustrating none-actions like this, noone should wonder
> why such things are not implemented, also there are people, who would like to see it and even
> people, who would help doing it and creating code for it. That does not actually speak for Gentoo,
> more against it (and it is not the only point, where Gentoo could improve, but does not, but that
> could be part of another big mail).
>
>
> --
> Thomas Sachau
>
> Gentoo Linux Developer
>
>

People are honestly just waiting for this to hit the tree at this
point. Inaction by the council is a serious failure of the council.
The Portage team doesn't want to start integrating code that the
council will force them to remove in the future. Being a former
council member myself I can easily say, get off your ass and do
something.

-- 
Doug Goldstein

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Fabian Groffen]

Hi Thomas,

On 25-12-2009 15:00:36 +0100, Thomas Sachau wrote:
> On 12/25/2009 06:10 AM, Denis Dupeyron wrote:
> > Sorry, I forgot to send an email explaining what happened on the
> > council alias as promised. The consensus was that the project wasn't
> > mature enough to go ahead. Also more generally the council's job isn't
> > discussing but deciding, approving, etc... Discussing is what should
> > happen on mailing lists.
> 
> Since i see noone arguing against adding the multilib features to
> current testing branch of portage, the discussion part already seems
> to be done. so a simple approval is ok, drop the discussion request.

This may be a wasted effort, but I tried figuring out what you do in
your portage branch going by your earlier discussions.  The idea I got
is the following:

Triggered by some mechanism (maybe unconditional), you just run each
ebuild function executed by Portage multiple times, that is, for each
ABI that you grab from somewhere. e.g:
src_unpack() {
	for abi in ${EABIS} ; do
		mkdir ${WORKDIR}-${abi}
		cd ${WORKDIR}-${abi}
		unpack ${A}
	done
}

During configure you inject -m{32,64} in CFLAGS depending on your ABI.
This gives your multilib-compiler the right hint to do the thing you
want.

This is about what I understood.  Now here I have some questions that
may or may not be relevant.

What triggers a multilib build?  Is it unconditional, or can it be
turned on/off per package?  How does Portage resolve/verify that a
library is built for the right ABI in that case?

Unpacking sources many times feels like a terrible waste to me,
especially for things like GCC.  If we would just start building outside
of the workdir (sources) into a separate builddir, wouldn't that just
be much cleaner and a nice EAPI feature?

Since you make each compilation multiple times, you also obtain a fully
identical installation of the same package.  How do you deal with that?
Do you have /usr/bin{64,32} directories for example too?  If you only
keep libs (found by a scanelf scan or something), how do you know what's
relevant.  Alternatively, if you build the full application anyway,
isn't it a waste to throw away the result?  You could see multilib also
as two (unrelated) trees, such as e.g. a Gentoo Prefix installation.
A nasty one: how to deal with libs that actually contain hardcoded paths
to configuration from e.g. /etc or /var in your implementation?

You chose to inject -m{32,64} in CFLAGS.  Suppose you set CC to "gcc
-m{32,64}" or even "x86_64-..." or "i686-..." you could do some
cross-target stuff, I think.  I say so in the light of Darwin systems
which are capable of using Mach-O FAT objects.  Such objects can contain
multiple architectures.  This idea is available for as FATelf also, and
e.g. included in the zen-sources.  On such system you ultimately want to
handle the multilib hel^Wproblem by avoiding the different paths but
instead generate that single unified tree that contains all your ABIs in
each (FAT) file.  Is your approach flexible enough to lipo two or more
trees together in one after the two images are installed?

> > Before you can bring that to the council we
> > need to see an as-much-as-possible finalized solution with any of the
> > following if applicable: portage branch with an implementation that
> > people can try, documentation, PMS patch, devmanual patches, and a
> > team.
> 
> Did you actually read my lines? I did NOT request an ACK to add it to
> PMS and next EAPI with a complete spec. zmedico also has no problem
> with having a look and adding it, but since he was once forced to
> remove an added feature, he now wants a council-ok before adding and
> improving it in testing branch of main tree portage.

From my experience they just want to get some grip on the issue.  A
formal description helps sometimes.

> > Look at what happened with prefix. They wanted
> > the council to approve it immediately or else... We didn't cede to
> > pressure and worked with them to make it good enough for approval.
>
> Something like "I/We want <x>,<y>,<z> or you wont get an approval" is
> no support and no "work with them". So if you really would like to see
> it in, actually help with patches, SPEC writing, discussion and code
> writing. Else i request an approval for getting some additional help
> instead of just shooting it down.

Pfff, I guess like you, Thomas, we got a bit impatient.  Our experience
is that once you give the information to the council in the right
format, they seem to be much more focussed.

> > Right now I don't hear anybody arguing about prefix going forward. And
> > that's exactly what I want for your project, i.e. helping you making
> > it better instead of it fading and failing in the (not so) long run.
> 
> prefix is no one-man-team and the actual amount of people, who can and
> are willing to work on portage code is limited, so which other way do
> you have to improve it as requested?

Prefix has been more or less a one-man-team for a long long time, but
then, the project exists for about 5 years now.  I cannot really advise
you to go that route, so please try to spec it.


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 9129 bytes --]

On 12/26/2009 11:25 AM, Fabian Groffen wrote:
> Hi Thomas,
> 
> On 25-12-2009 15:00:36 +0100, Thomas Sachau wrote:
>> On 12/25/2009 06:10 AM, Denis Dupeyron wrote:
>>> Sorry, I forgot to send an email explaining what happened on the
>>> council alias as promised. The consensus was that the project wasn't
>>> mature enough to go ahead. Also more generally the council's job isn't
>>> discussing but deciding, approving, etc... Discussing is what should
>>> happen on mailing lists.
>>
>> Since i see noone arguing against adding the multilib features to
>> current testing branch of portage, the discussion part already seems
>> to be done. so a simple approval is ok, drop the discussion request.
> 
> This may be a wasted effort, but I tried figuring out what you do in
> your portage branch going by your earlier discussions.  The idea I got
> is the following:
> 
> Triggered by some mechanism (maybe unconditional), you just run each
> ebuild function executed by Portage multiple times, that is, for each
> ABI that you grab from somewhere. e.g:
> src_unpack() {
> 	for abi in ${EABIS} ; do
> 		mkdir ${WORKDIR}-${abi}
> 		cd ${WORKDIR}-${abi}
> 		unpack ${A}
> 	done
> }

Currently it is something like this for every phase:

src_unpack() {
	for abi in ${MULTILIB_ABIS}; do
		set_env $abi
		<do the usual stuff as normal>
	done
}

The ebuild phase does only see the normal WORKDIR, all work with it is done inside portage and
outside the phases, so every package doing things inside WORKDIR just works without additional
changes. The install part contains some additional helpers for headers and binaries.

But i currently try to change it as suggested by vapier:

for abi in ${MULTILIB_ABIS}; do
	set_env $abi
	pkg_setup
	src_{unpack,compile,install}
	mv ${D} ${D}.$abi
done
merge ${D}.$abi into ${D}
<continue as usual>

This way you can set ABI-dependent vars in pkg_setup (previous implementation requires to set it in
every phase, if it was no preserved var), there is nothing in ${D} during src_install, so you can do
a mv again (like some ebuilds or eclasses do or did) and, since the DEFAULT_ABI is run first, i can
detect, if there is actually the need to run everything again for other ABIs.
But i currently cannot say when i am finished with it.

> 
> During configure you inject -m{32,64} in CFLAGS depending on your ABI.
> This gives your multilib-compiler the right hint to do the thing you
> want.

Right

> 
> This is about what I understood.  Now here I have some questions that
> may or may not be relevant.
> 
> What triggers a multilib build?  Is it unconditional, or can it be
> turned on/off per package?  How does Portage resolve/verify that a
> library is built for the right ABI in that case?

Currently multilib-portage does add a USE flag called "lib32". If you enable it, you will get the
cross-compile, else just the normal install. In addition this flag is internally used like an EAPI-2
usedep, so it will require the dependencies to be built for all ABIs too.

> 
> Unpacking sources many times feels like a terrible waste to me,
> especially for things like GCC.  If we would just start building outside
> of the workdir (sources) into a separate builddir, wouldn't that just
> be much cleaner and a nice EAPI feature?

That might be an extra step, once the basic implementation works, but you will have to adjust some
things, since e.g. cmake-utils eclass does already something like that, maybe others do it too, so
you would have to change those ebuilds/eclasses or add exceptions or extra rules to portage for those.
Some packages like gcc or glibc already do this multilib-stuff internally with the multilib USE
flag, so you currently wont get any better experience for them.

> 
> Since you make each compilation multiple times, you also obtain a fully
> identical installation of the same package.  How do you deal with that?
> Do you have /usr/bin{64,32} directories for example too?  If you only
> keep libs (found by a scanelf scan or something), how do you know what's
> relevant.  Alternatively, if you build the full application anyway,
> isn't it a waste to throw away the result?  You could see multilib also
> as two (unrelated) trees, such as e.g. a Gentoo Prefix installation.
> A nasty one: how to deal with libs that actually contain hardcoded paths
> to configuration from e.g. /etc or /var in your implementation?

Currently i only work and test with amd64-ARCH, so with x86 and amd86 ABI. For those, the lib-part
is easy since the crosscompile does install the libs into /usr/lib32 while the 64bit ones go into
/usr/lib64. The headers for both ABIs are diffed and different ones are preserved, the rest is
isntalled as usual. For binaries, normally only the one for the DEFAULT_ABI, so in this case the
64bit one, will be preserved. But you can tell multilib-portage to preserve the 32bit binaries. In
that case, the binaries will be called $binary-$ABI and a symlink $binary to a wrapper created,
which calls the real binary depending on the current ABI.

> 
> You chose to inject -m{32,64} in CFLAGS.  Suppose you set CC to "gcc
> -m{32,64}" or even "x86_64-..." or "i686-..." you could do some
> cross-target stuff, I think.  I say so in the light of Darwin systems
> which are capable of using Mach-O FAT objects.  Such objects can contain
> multiple architectures.  This idea is available for as FATelf also, and
> e.g. included in the zen-sources.  On such system you ultimately want to
> handle the multilib hel^Wproblem by avoiding the different paths but
> instead generate that single unified tree that contains all your ABIs in
> each (FAT) file.  Is your approach flexible enough to lipo two or more
> trees together in one after the two images are installed?

My currently planned changes do install into different target dirs, so if those target dirs do
contain all you need, there should be no problem to add some additional lines during merge.

>>> Before you can bring that to the council we
>>> need to see an as-much-as-possible finalized solution with any of the
>>> following if applicable: portage branch with an implementation that
>>> people can try, documentation, PMS patch, devmanual patches, and a
>>> team.
>>
>> Did you actually read my lines? I did NOT request an ACK to add it to
>> PMS and next EAPI with a complete spec. zmedico also has no problem
>> with having a look and adding it, but since he was once forced to
>> remove an added feature, he now wants a council-ok before adding and
>> improving it in testing branch of main tree portage.
> 
>>From my experience they just want to get some grip on the issue.  A
> formal description helps sometimes.

As i already said: My implementation is not final nor do i request some PMS changes or EAPI bump for
it. I simply want some more help and feedback by getting it in the 2.2_rc* branch of portage and
zmedico just wants an ok from council, so that they wont force him to remove it again.

Once the implementation is final, there should be more time to create the SPEC, PMS patches and
similar for a final approval by council and inclusion in PMS/some EAPI.

>>> Look at what happened with prefix. They wanted
>>> the council to approve it immediately or else... We didn't cede to
>>> pressure and worked with them to make it good enough for approval.
>>
>> Something like "I/We want <x>,<y>,<z> or you wont get an approval" is
>> no support and no "work with them". So if you really would like to see
>> it in, actually help with patches, SPEC writing, discussion and code
>> writing. Else i request an approval for getting some additional help
>> instead of just shooting it down.
> 
> Pfff, I guess like you, Thomas, we got a bit impatient.  Our experience
> is that once you give the information to the council in the right
> format, they seem to be much more focussed.

But i currently dont have "the information", i only want a "we wont force the portage team to remove
the multilib code from portage-2.2_rc*".

>>> Right now I don't hear anybody arguing about prefix going forward. And
>>> that's exactly what I want for your project, i.e. helping you making
>>> it better instead of it fading and failing in the (not so) long run.
>>
>> prefix is no one-man-team and the actual amount of people, who can and
>> are willing to work on portage code is limited, so which other way do
>> you have to improve it as requested?
> 
> Prefix has been more or less a one-man-team for a long long time, but
> then, the project exists for about 5 years now.  I cannot really advise
> you to go that route, so please try to spec it.

Long term forking is hard, especially when you also work for other time consuming projects and have
some work, where you have no contact with Gentoo or Linux.
But i cannot give out specs, when it is not final. My main interest is more feedback, more comments,
more testing and maybe a bit more help for the code part.

-- 
Thomas Sachau

Gentoo Linux Developer


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Zac Medico]

On 12/26/2009 05:15 AM, Thomas Sachau wrote:
> On 12/26/2009 11:25 AM, Fabian Groffen wrote:
>> On 25-12-2009 15:00:36 +0100, Thomas Sachau wrote:
>>> Did you actually read my lines? I did NOT request an ACK to add it to
>>> PMS and next EAPI with a complete spec. zmedico also has no problem
>>> with having a look and adding it, but since he was once forced to
>>> remove an added feature, he now wants a council-ok before adding and
>>> improving it in testing branch of main tree portage.
>>
>> >From my experience they just want to get some grip on the issue.  A
>> formal description helps sometimes.
> 
> As i already said: My implementation is not final nor do i request some PMS changes or EAPI bump for
> it. I simply want some more help and feedback by getting it in the 2.2_rc* branch of portage and
> zmedico just wants an ok from council, so that they wont force him to remove it again.

The council needs to approve the merging of multilib support into
mainline portage since ebuild developers will be expected to
help/cooperate in fixing any issues that may arise due to
interaction with the new multilib behavior. The role of imposing new
expectations like this belongs to the council, not to me or the
portage team.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 1479 bytes --]

On Friday 25 December 2009 09:00:36 Thomas Sachau wrote:
> On 12/25/2009 06:10 AM, Denis Dupeyron wrote:
> > On Sun, Dec 20, 2009 at 10:53 PM, Thomas Sachau <tommy@gentoo.org> wrote:
> >> I will make it short, since i already requested it 3 times, did create a
> >> thread at gentoo-dev ML:
> >>
> >> agenda topic: Discussion and approval for following item:
> >>
> >> Adding real multilib features from current multilib-portage to currently
> >> hardmasked and testing portage-2.2* for wider testing, more eyes looking
> >> at it and hopefully more people helping improving it, so we can get a
> >> version, which most can accept for PMS and maybe next EAPI.
> >
> > Sorry, I forgot to send an email explaining what happened on the
> > council alias as promised. The consensus was that the project wasn't
> > mature enough to go ahead. Also more generally the council's job isn't
> > discussing but deciding, approving, etc... Discussing is what should
> > happen on mailing lists.
> 
> Since i see noone arguing against adding the multilib features to current
>  testing branch of portage, the discussion part already seems to be done.
>  so a simple approval is ok, drop the discussion request.

that's incorrect.  you still havent addressed my outstanding issues.  until 
you do, i dont see how you can push for this being added to portage.  or i 
missed some update along the way, but the last e-mail i see is from me dated 
26.10.2009 ...
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] Re: [gentoo-dev-announce] January 2010 meeting date

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 3309 bytes --]

On Saturday 26 December 2009 08:15:22 Thomas Sachau wrote:
> On 12/26/2009 11:25 AM, Fabian Groffen wrote:
> > This is about what I understood.  Now here I have some questions that
> > may or may not be relevant.
> >
> > What triggers a multilib build?  Is it unconditional, or can it be
> > turned on/off per package?  How does Portage resolve/verify that a
> > library is built for the right ABI in that case?
> 
> Currently multilib-portage does add a USE flag called "lib32". If you
>  enable it, you will get the cross-compile, else just the normal install.
>  In addition this flag is internally used like an EAPI-2 usedep, so it will
>  require the dependencies to be built for all ABIs too.

this is something that needs to be fixed per the earlier discussion

> > Unpacking sources many times feels like a terrible waste to me,
> > especially for things like GCC.  If we would just start building outside
> > of the workdir (sources) into a separate builddir, wouldn't that just
> > be much cleaner and a nice EAPI feature?
> 
> That might be an extra step, once the basic implementation works, but you
>  will have to adjust some things, since e.g. cmake-utils eclass does
>  already something like that, maybe others do it too, so you would have to
>  change those ebuilds/eclasses or add exceptions or extra rules to portage
>  for those. Some packages like gcc or glibc already do this multilib-stuff
>  internally with the multilib USE flag, so you currently wont get any
>  better experience for them.

indeed ... it'd be nice if we only ran src_unpack() once, but there are 
packages in EAPI0 that modify the source based on ABI/build flags.  the only 
safe way is to always run the src_unpack() multiple times.

once this implementation settles on top of EAPI{0..3}, we can look at EAPI4+ 
to optimize the flow.

> > Since you make each compilation multiple times, you also obtain a fully
> > identical installation of the same package.  How do you deal with that?
> > Do you have /usr/bin{64,32} directories for example too?  If you only
> > keep libs (found by a scanelf scan or something), how do you know what's
> > relevant.  Alternatively, if you build the full application anyway,
> > isn't it a waste to throw away the result?  You could see multilib also
> > as two (unrelated) trees, such as e.g. a Gentoo Prefix installation.
> > A nasty one: how to deal with libs that actually contain hardcoded paths
> > to configuration from e.g. /etc or /var in your implementation?
> 
> Currently i only work and test with amd64-ARCH, so with x86 and amd86 ABI.
>  For those, the lib-part is easy since the crosscompile does install the
>  libs into /usr/lib32 while the 64bit ones go into /usr/lib64. The headers
>  for both ABIs are diffed and different ones are preserved, the rest is
>  isntalled as usual. For binaries, normally only the one for the
>  DEFAULT_ABI, so in this case the 64bit one, will be preserved. But you can
>  tell multilib-portage to preserve the 32bit binaries. In that case, the
>  binaries will be called $binary-$ABI and a symlink $binary to a wrapper
>  created, which calls the real binary depending on the current ABI.

can you guys think of a package where the bindirs differ and/or we care ?
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]