public inbox for gentoo-dev@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-dev] Stable masks on multilib packages
@ 2014-03-31 22:16 Michał Górny
  2014-03-31 23:09 ` Anthony G. Basile
                   ` (3 more replies)
  0 siblings, 4 replies; 25+ messages in thread
From: Michał Górny @ 2014-03-31 22:16 UTC (permalink / raw
  To: multilib; +Cc: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 3424 bytes --]

Hello, all.

The late multilib ppc issues made me re-check our stable masks on
abi_x86_* flags and, honestly, I'm not sure if we're doing things
the right way. First, a quick summary.


Let's consider dev-libs/elfutils that has a dependency
on app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
libbz2. This is enforced through a dep in form of:

  app-arch/bzip2[${MULTILIB_USEDEP}]

that gets expanded into:

  app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]

which means that any of the ABI_* flags that gets enabled on elfutils,
needs to be enabled on bzip2 as well. Of course, some of use.forcing
and masking gets applied here but that doesn't really matter.


Now, since we're pretty much converting a lot of different packages,
some of them are eligible for stabilization earlier than the others.
However, the extra MULTILIB_USEDEPs enforce stabilizing multilib
dependencies before the actual packages which made a few developers
unhappy.

That's why we decided to stable-mask the flags on affected packages.
Since the flags are masked on stable, people can stabilize packages
independently of whether their converted deps are stable or not. We
will be worrying about that consistency once we decide to unmask
the flags.

The extra advantage of that is that we can avoid pushing stable users
into the mess involved with partial conversion of emul-linux. The idea
is pretty simple: we keep emul-linux for stable, and once everything is
ready, we stable-unmask the flags and let stable users grok multilib.


Now, to the problem. Currently we're just stable-masking abi_x86_32
on amd64. This serves the latter purpose well and seems to work for the
former. This is probably because the remaining flag is use.forced
(abi_x86_64 on amd64, and abi_x86_32 on x86) which seems to add it to
implicit IUSE somehow.

floppym has done some research w/ stable elfutils and no stable
converted bzip2. It seems that if abi_x86_32 is stable.use.masked
and abi_x86_64 is use.forced, repoman groks it. However, if we remove
abi_x86_64 from use.force, it properly reports flag mismatch error
(since no stable bzip2 has IUSE=abi_x86_64).

Now, I honestly have no idea if this implicit use.force behavior is
PMS-y or not, and how other PMs handle it. I can't find something like
this in the PMS but that doc is horribly hard to cross-reference, so I
might be missing something. I'd appreciate if someone could help me
with that.


That said, I have an alternate idea inspired by the ppc breakage. I'm
thinking of replacing the amd64 abi_x86_32 mask with a global stable
mask of all abi_*_* flags on the relevant packages.

Differences:

1) old solution: native flag is forced, other flags are masked.
   new solution: all flags are masked.

2) old solution: we need to replicate the masks properly for different
                 arches/profiles.
   new solution: we can keep a single mask for all arches.

3) old solution: MULTILIB_USEDEP magically works (w/ portage at least).
   new solution: since all flags are disabled, MULTILIB_USEDEP
                 is a no-op and old packages match correctly.

4) old solution: forced native flag runs the native build.
   new solution: fallback code runs the native build (since no flags
                 are enabled).


Your thoughts?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-03-31 22:16 [gentoo-dev] Stable masks on multilib packages Michał Górny
@ 2014-03-31 23:09 ` Anthony G. Basile
  2014-04-01  5:54   ` Michał Górny
  2014-04-01  5:13 ` Ben de Groot
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 25+ messages in thread
From: Anthony G. Basile @ 2014-03-31 23:09 UTC (permalink / raw
  To: gentoo-dev

On 03/31/2014 06:16 PM, Michał Górny wrote:
> Hello, all.
>
> The late multilib ppc issues made me re-check our stable masks on
> abi_x86_* flags and, honestly, I'm not sure if we're doing things
> the right way. First, a quick summary.
>
>
> Let's consider dev-libs/elfutils that has a dependency
> on app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit
> libbz2. This is enforced through a dep in form of:
>
>    app-arch/bzip2[${MULTILIB_USEDEP}]
>
> that gets expanded into:
>
>    app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
>
> which means that any of the ABI_* flags that gets enabled on elfutils,
> needs to be enabled on bzip2 as well. Of course, some of use.forcing
> and masking gets applied here but that doesn't really matter.
>
>
> Now, since we're pretty much converting a lot of different packages,
> some of them are eligible for stabilization earlier than the others.
> However, the extra MULTILIB_USEDEPs enforce stabilizing multilib
> dependencies before the actual packages which made a few developers
> unhappy.
>
> That's why we decided to stable-mask the flags on affected packages.
> Since the flags are masked on stable, people can stabilize packages
> independently of whether their converted deps are stable or not. We
> will be worrying about that consistency once we decide to unmask
> the flags.
>
> The extra advantage of that is that we can avoid pushing stable users
> into the mess involved with partial conversion of emul-linux. The idea
> is pretty simple: we keep emul-linux for stable, and once everything is
> ready, we stable-unmask the flags and let stable users grok multilib.
>
>
> Now, to the problem. Currently we're just stable-masking abi_x86_32
> on amd64. This serves the latter purpose well and seems to work for the
> former. This is probably because the remaining flag is use.forced
> (abi_x86_64 on amd64, and abi_x86_32 on x86) which seems to add it to
> implicit IUSE somehow.
>
> floppym has done some research w/ stable elfutils and no stable
> converted bzip2. It seems that if abi_x86_32 is stable.use.masked
> and abi_x86_64 is use.forced, repoman groks it. However, if we remove
> abi_x86_64 from use.force, it properly reports flag mismatch error
> (since no stable bzip2 has IUSE=abi_x86_64).
>
> Now, I honestly have no idea if this implicit use.force behavior is
> PMS-y or not, and how other PMs handle it. I can't find something like
> this in the PMS but that doc is horribly hard to cross-reference, so I
> might be missing something. I'd appreciate if someone could help me
> with that.
>
>
> That said, I have an alternate idea inspired by the ppc breakage. I'm
> thinking of replacing the amd64 abi_x86_32 mask with a global stable
> mask of all abi_*_* flags on the relevant packages.
I'm not sure exactly what you mean here --- where/how does this global 
stable masking happen?  Right now you have a bunch of maskings of 
abi_x86_32 on various packages in arch/amd64/package.use.stable.mask but 
not on any other arches where you just have the use.mask/use.force 
combination.  Are you going to migrate the amd64 file to other multilib 
arches and mask non-native abis?  Like on mips64el/multilib/n32 would 
you be masking abi_mips_o32 and abi_mips_n64 for all those packages?

>
> Differences:
>
> 1) old solution: native flag is forced, other flags are masked.
>     new solution: all flags are masked.
>
> 2) old solution: we need to replicate the masks properly for different
>                   arches/profiles.
>     new solution: we can keep a single mask for all arches.

This sounds like a partial answer to my above question.

>
> 3) old solution: MULTILIB_USEDEP magically works (w/ portage at least).
>     new solution: since all flags are disabled, MULTILIB_USEDEP
>                   is a no-op and old packages match correctly.
>
> 4) old solution: forced native flag runs the native build.
>     new solution: fallback code runs the native build (since no flags
>                   are enabled).
>
>
> Your thoughts?
>

How does this "go live" once these flags are enabled?  Do you just 
remove the global mask all at once?  That sounds a bit scarier than just 
removing the masks one package + deps at a time.  (Maybe a non-question 
because I'm still now sure how this masking works.)

Also, I don't see that it should be an issue, but do you think this 
might affect catalyst runs --- I have to ask because 
repairing/reconfiguring seeds is a lot of work.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-03-31 22:16 [gentoo-dev] Stable masks on multilib packages Michał Górny
  2014-03-31 23:09 ` Anthony G. Basile
@ 2014-04-01  5:13 ` Ben de Groot
  2014-04-01 13:58   ` Alexandre Rostovtsev
  2014-04-01 23:38   ` Patrick Lauer
  2014-04-01 15:12 ` Ian Stakenvicius
  2014-04-01 17:15 ` Michał Górny
  3 siblings, 2 replies; 25+ messages in thread
From: Ben de Groot @ 2014-04-01  5:13 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
> Hello, all.
>
> The late multilib ppc issues made me re-check our stable masks on
> abi_x86_* flags and, honestly, I'm not sure if we're doing things
> the right way.
>
> That said, I have an alternate idea inspired by the ppc breakage.
>
> Your thoughts?

In my opinion your multilib approach introduces an unnecessary degree
of complexity, which --as has been shown here again-- is prone to
breakage.

It would be best for our beloved distro to revert all the multilib
changes, and try a different approach, or leave this prone-to-breakage
implementation to an overlay for the few people who would actually
benefit from it.

-- 
Cheers,

Ben | yngwin
Gentoo developer


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-03-31 23:09 ` Anthony G. Basile
@ 2014-04-01  5:54   ` Michał Górny
  2014-04-01 11:26     ` [gentoo-dev] " Duncan
  2014-04-01 11:40     ` [gentoo-dev] " Anthony G. Basile
  0 siblings, 2 replies; 25+ messages in thread
From: Michał Górny @ 2014-04-01  5:54 UTC (permalink / raw
  To: gentoo-dev; +Cc: blueness

[-- Attachment #1: Type: text/plain, Size: 2938 bytes --]

Dnia 2014-03-31, o godz. 19:09:44
"Anthony G. Basile" <blueness@gentoo.org> napisał(a):

> On 03/31/2014 06:16 PM, Michał Górny wrote:
> > That said, I have an alternate idea inspired by the ppc breakage. I'm
> > thinking of replacing the amd64 abi_x86_32 mask with a global stable
> > mask of all abi_*_* flags on the relevant packages.
> I'm not sure exactly what you mean here --- where/how does this global 
> stable masking happen?  Right now you have a bunch of maskings of 
> abi_x86_32 on various packages in arch/amd64/package.use.stable.mask but 
> not on any other arches where you just have the use.mask/use.force 
> combination.  Are you going to migrate the amd64 file to other multilib 
> arches and mask non-native abis?  Like on mips64el/multilib/n32 would 
> you be masking abi_mips_o32 and abi_mips_n64 for all those packages?

The new solution would be base/package.use.stable.mask that would mask
all of multilib on all stable arches, on this long list of packages.
So, not only the two ABIs but all of them. Which would effectively make
ebuild behave like it had no multilib at all.

> > Your thoughts?
> 
> How does this "go live" once these flags are enabled?  Do you just 
> remove the global mask all at once?  That sounds a bit scarier than just 
> removing the masks one package + deps at a time.  (Maybe a non-question 
> because I'm still now sure how this masking works.)

Something like this, yes. Once all packages are migrated and some time
passes, we unmask all the flags locally and do a repoman run. We find
out what needs to go stable, report bugs, wait and repeat.

Then we do a second stabilization request, this time for testing
the tree (mostly emul-linux replacement) with multilib enabled. Once
arch teams are done testing, they remove the stable masks for
particular ABI.

When all reverse dependencies are fixed to use || () deps instead of
emul-linux (and rev-bumped) we can move away from emul-linux through
the usual procedure of last rites with a proper announcement. This is
likely the most fragile step of all but we will do our best to make it
as simple as possible, and I think our users can handle that.

> Also, I don't see that it should be an issue, but do you think this 
> might affect catalyst runs --- I have to ask because 
> repairing/reconfiguring seeds is a lot of work.

Well, I think this mostly depends on whether you want multiple multilib
ABIs in stages -- or if you assume that the toolchain is enough,
and people will build other ABIs as they need.

Though I think that once toolchain switches to abi_* flags (vapier
seems to show interest in that), we will use.force the necessary ABIs
on it and its dependencies, so there should be no explicit need for
change.

However, I guess the toolchain changes will be sent out for testing
anyway, so releng can check first hand.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* [gentoo-dev] Re: Stable masks on multilib packages
  2014-04-01  5:54   ` Michał Górny
@ 2014-04-01 11:26     ` Duncan
  2014-04-01 15:49       ` Ian Stakenvicius
  2014-04-01 11:40     ` [gentoo-dev] " Anthony G. Basile
  1 sibling, 1 reply; 25+ messages in thread
From: Duncan @ 2014-04-01 11:26 UTC (permalink / raw
  To: gentoo-dev

Michał Górny posted on Tue, 01 Apr 2014 07:54:11 +0200 as excerpted:

> Something like this, yes. Once all packages are migrated and some time
> passes, we unmask all the flags locally and do a repoman run. We find
> out what needs to go stable, report bugs, wait and repeat.

The big question on the user side is how many times we're going to have 
to rebuild the same code, just because people are messing with multilib 
stuff.  Masking all the now unmasked is going to force yet another 
rebuild on stable users, who presumably are stable in part because they 
/don't/ want all those "pointless" rebuilds.

Altho I'm ~arch so stable masks won't directly affect me, but at the same 
time, I'm no-multilib, so all those --newuse and -rX-bump rebuilds really 
ARE useless rebuilds of the /exact/ same thing!  While my system is fast 
enough it's not a big deal here, it's still a pain tracking all those -rX 
bumps to see what justifies the bump (at least I can tell simple USE flag 
changes on sight), and I hurt for those with slower systems!

And wow, if you do go the global masking route, I seriously hope you have 
plenty of announcement before the big unmasking, with at probably 30 days 
warning and coverage on the web site, GMN, and news items, and possibly 
even a second news-item warning 3-5 days before it happens, since the 
global unmask will trigger about as big a universal rebuild as it gets, 
and those always bring quite a few user complaints.  (Tho at least this 
one won't be breaking much of the system until it's done, unlike some of 
the big library upgrades of yore.)

IOW, while that very likely would have been the better way to do it 
originally, I really don't know whether changing things up in mid-stream 
like this is going to result in more or less pain for users.  Not being a 
stable-arch user nor being multilib, I've been spared much of the worry 
since on my systems it should "just work" either way, but honest 
question, how far /are/ we into this, and will the change now result in 
more work for more people (given that there's way more users than devs) 
than just continuing the current path, now that we're in the middle of 
things?

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01  5:54   ` Michał Górny
  2014-04-01 11:26     ` [gentoo-dev] " Duncan
@ 2014-04-01 11:40     ` Anthony G. Basile
  2014-04-01 12:04       ` Anthony G. Basile
  1 sibling, 1 reply; 25+ messages in thread
From: Anthony G. Basile @ 2014-04-01 11:40 UTC (permalink / raw
  To: gentoo-dev

On 04/01/2014 01:54 AM, Michał Górny wrote:
>> >Also, I don't see that it should be an issue, but do you think this
>> >might affect catalyst runs --- I have to ask because
>> >repairing/reconfiguring seeds is a lot of work.
> Well, I think this mostly depends on whether you want multiple multilib
> ABIs in stages -- or if you assume that the toolchain is enough,
> and people will build other ABIs as they need.
>
> Though I think that once toolchain switches to abi_* flags (vapier
> seems to show interest in that), we will use.force the necessary ABIs
> on it and its dependencies, so there should be no explicit need for
> change.
>
> However, I guess the toolchain changes will be sent out for testing
> anyway, so releng can check first hand.
I have taken a non-standard approach to the hardened/linux/uclibc 
profiles where I do no inherit from arch.  There was too much cruft in 
arch, so I just mirror the relevant stuff including the abi flag stuff.  
I understand my approach has pros/cons ... but give out the should when 
the change is being tested and I'll see if I'm about to have any 
breakage.   I currently seed at least 12 different 
arches/isa/abi/hard-soft float combinations which would be a pita to fix 
manually on native (read slow) hardware.

-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 11:40     ` [gentoo-dev] " Anthony G. Basile
@ 2014-04-01 12:04       ` Anthony G. Basile
  0 siblings, 0 replies; 25+ messages in thread
From: Anthony G. Basile @ 2014-04-01 12:04 UTC (permalink / raw
  To: gentoo-dev

On 04/01/2014 07:40 AM, Anthony G. Basile wrote:
> On 04/01/2014 01:54 AM, Michał Górny wrote:
>>> >Also, I don't see that it should be an issue, but do you think this
>>> >might affect catalyst runs --- I have to ask because
>>> >repairing/reconfiguring seeds is a lot of work.
>> Well, I think this mostly depends on whether you want multiple multilib
>> ABIs in stages -- or if you assume that the toolchain is enough,
>> and people will build other ABIs as they need.
>>
>> Though I think that once toolchain switches to abi_* flags (vapier
>> seems to show interest in that), we will use.force the necessary ABIs
>> on it and its dependencies, so there should be no explicit need for
>> change.
>>
>> However, I guess the toolchain changes will be sent out for testing
>> anyway, so releng can check first hand.
> I have taken a non-standard approach to the hardened/linux/uclibc 
> profiles where I do no inherit from arch.  There was too much cruft in 
> arch, so I just mirror the relevant stuff including the abi flag 
> stuff.  I understand my approach has pros/cons ...


> but give out the should when 


Sorry, my fingers and brains are not in sync.  ... give out the SHOUT 
when ...


> the change is being tested and I'll see if I'm about to have any 
> breakage.   I currently seed at least 12 different 
> arches/isa/abi/hard-soft float combinations which would be a pita to 
> fix manually on native (read slow) hardware.
>


-- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail    : blueness@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01  5:13 ` Ben de Groot
@ 2014-04-01 13:58   ` Alexandre Rostovtsev
  2014-04-01 14:43     ` Rich Freeman
  2014-04-02  9:14     ` Ben de Groot
  2014-04-01 23:38   ` Patrick Lauer
  1 sibling, 2 replies; 25+ messages in thread
From: Alexandre Rostovtsev @ 2014-04-01 13:58 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

[-- Attachment #1: Type: text/plain, Size: 1450 bytes --]

On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
> > Hello, all.
> >
> > The late multilib ppc issues made me re-check our stable masks on
> > abi_x86_* flags and, honestly, I'm not sure if we're doing things
> > the right way.
> >
> > That said, I have an alternate idea inspired by the ppc breakage.
> >
> > Your thoughts?
> 
> In my opinion your multilib approach introduces an unnecessary degree
> of complexity, which --as has been shown here again-- is prone to
> breakage.
> 
> It would be best for our beloved distro to revert all the multilib
> changes, and try a different approach, or leave this prone-to-breakage
> implementation to an overlay for the few people who would actually
> benefit from it.

Speaking as a wine maintainer, the emul-linux-x86-* approach has many
times been proven to be an embarrassing failure and the main source of
pain and frustration for wine users. The sooner emul-linux-x86-* can be
removed from the tree, the better for Gentoo.

I am aware of only two solutions to the emul-linux-x86-* problems :
multilib-portage and multilib-build.eclass. The first requires everybody
to switch to a new package manager. The second allows us to keep using
portage, but requires library maintainers to add some simple boilerplate
to their ebuilds for multilib support.

Do you have yet another alternative in mind?

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 13:58   ` Alexandre Rostovtsev
@ 2014-04-01 14:43     ` Rich Freeman
  2014-04-02  0:13       ` Patrick Lauer
  2014-04-02  9:14     ` Ben de Groot
  1 sibling, 1 reply; 25+ messages in thread
From: Rich Freeman @ 2014-04-01 14:43 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

On Tue, Apr 1, 2014 at 9:58 AM, Alexandre Rostovtsev
<tetromino@gentoo.org> wrote:
> On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
>>
>> In my opinion your multilib approach introduces an unnecessary degree
>> of complexity, which --as has been shown here again-- is prone to
>> breakage.
>>
>> It would be best for our beloved distro to revert all the multilib
>> changes, and try a different approach, or leave this prone-to-breakage
>> implementation to an overlay for the few people who would actually
>> benefit from it.
>
> I am aware of only two solutions to the emul-linux-x86-* problems :
> multilib-portage and multilib-build.eclass. The first requires everybody
> to switch to a new package manager. The second allows us to keep using
> portage, but requires library maintainers to add some simple boilerplate
> to their ebuilds for multilib support.
>
> Do you have yet another alternative in mind?

++

I'm all for better solutions.  I'm not in favor of abandoning
solutions that work moderately well in favor of talking about maybe
coming up with something better sometime down the road.

The multilib eclass isn't a perfect solution.  It will have issues in
concept, design, and implementation.  These will come up from time to
time.  I don't think it is productive that anytime any of these pop up
that we end up having a discussion about just reverting it entirely.

By all means work on a competing solution.  Get it working and
supported by portage.  When the day comes that we want to endorse one
solution or the other as the preferred solution we can have that
discussion.

Dirty hack or not the multilib eclass seems to be working fairly well
in practice, especially considering just how big a change it is.  The
emul-linux packages were an even dirtier hack and those worked well
for a decade.

Progress can be incremental.

Rich


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-03-31 22:16 [gentoo-dev] Stable masks on multilib packages Michał Górny
  2014-03-31 23:09 ` Anthony G. Basile
  2014-04-01  5:13 ` Ben de Groot
@ 2014-04-01 15:12 ` Ian Stakenvicius
  2014-04-01 15:32   ` Michał Górny
  2014-04-01 17:15 ` Michał Górny
  3 siblings, 1 reply; 25+ messages in thread
From: Ian Stakenvicius @ 2014-04-01 15:12 UTC (permalink / raw
  To: gentoo-dev

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 31/03/14 06:16 PM, Michał Górny wrote:
> Hello, all.
> 
> The late multilib ppc issues made me re-check our stable masks on 
> abi_x86_* flags and, honestly, I'm not sure if we're doing things 
> the right way. First, a quick summary.
> 
> 
> Let's consider dev-libs/elfutils that has a dependency on
> app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit 
> libbz2. This is enforced through a dep in form of:
> 
> app-arch/bzip2[${MULTILIB_USEDEP}]
> 
> that gets expanded into:
> 
> app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
>
>  which means that any of the ABI_* flags that gets enabled on
> elfutils, needs to be enabled on bzip2 as well. Of course, some of
> use.forcing and masking gets applied here but that doesn't really
> matter.
> 
> 
> Now, since we're pretty much converting a lot of different
> packages, some of them are eligible for stabilization earlier than
> the others. However, the extra MULTILIB_USEDEPs enforce stabilizing
> multilib dependencies before the actual packages which made a few
> developers unhappy.
> 
> That's why we decided to stable-mask the flags on affected
> packages. Since the flags are masked on stable, people can
> stabilize packages independently of whether their converted deps
> are stable or not. We will be worrying about that consistency once
> we decide to unmask the flags.
> 
> The extra advantage of that is that we can avoid pushing stable
> users into the mess involved with partial conversion of emul-linux.
> The idea is pretty simple: we keep emul-linux for stable, and once
> everything is ready, we stable-unmask the flags and let stable
> users grok multilib.
> 
> 
> Now, to the problem. Currently we're just stable-masking
> abi_x86_32 on amd64. This serves the latter purpose well and seems
> to work for the former. This is probably because the remaining flag
> is use.forced (abi_x86_64 on amd64, and abi_x86_32 on x86) which
> seems to add it to implicit IUSE somehow.
> 
> floppym has done some research w/ stable elfutils and no stable 
> converted bzip2. It seems that if abi_x86_32 is stable.use.masked 
> and abi_x86_64 is use.forced, repoman groks it. However, if we
> remove abi_x86_64 from use.force, it properly reports flag mismatch
> error (since no stable bzip2 has IUSE=abi_x86_64).
> 
> Now, I honestly have no idea if this implicit use.force behavior
> is PMS-y or not, and how other PMs handle it. I can't find
> something like this in the PMS but that doc is horribly hard to
> cross-reference, so I might be missing something. I'd appreciate if
> someone could help me with that.
> 
> 
> That said, I have an alternate idea inspired by the ppc breakage.
> I'm thinking of replacing the amd64 abi_x86_32 mask with a global
> stable mask of all abi_*_* flags on the relevant packages.
> 
> Differences:
> 
> 1) old solution: native flag is forced, other flags are masked. new
> solution: all flags are masked.
> 
> 2) old solution: we need to replicate the masks properly for
> different arches/profiles. new solution: we can keep a single mask
> for all arches.
> 
> 3) old solution: MULTILIB_USEDEP magically works (w/ portage at
> least). new solution: since all flags are disabled,
> MULTILIB_USEDEP is a no-op and old packages match correctly.
> 
> 4) old solution: forced native flag runs the native build. new
> solution: fallback code runs the native build (since no flags are
> enabled).
> 
> 
> Your thoughts?
> 


... how messy is this going to be on current systems, if implemented?
 This -is- going to cause use flag changes on a whackload of packages,
which at minimum I expect means a bunch of needless rebuilds.

I think it's probably best to keep with the current path, and just
hurry up the migration on x86.  If we switch to masking all abi_*_*'s,
then not only do we rebuild a whackload of packages now, but we also
end up rebuilding them again when we remove the abi_*_* mask, -even
if- the alternative abi's aren't needed.

A note on #3: if MULTILIB_USEDEP -isnt- working on other PMs, i hope
we have bug reports and we should work to resolve those ASAP or
abandon the project/find a different solution.  We're over a year into
it though (i think?) so i expect it's fine since I don't remember
hearing of any issues yet...


on a side note, to follow up -- when we do remove the masks, what're
the thoughts on perhaps changing the multilib profile on amd64 to
auto-enable abi_x86_32 flags on some packages?  Indeed, if the
toolchain is converted to multilib-build, would this be enough to
differentiate between amd64-multilib and pure64?  (bootstrapping may
of course still be a problem if users are trying to switch, but
probably not any more of a problem than it is right now, right?)



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlM610sACgkQ2ugaI38ACPAPoQD/TpzYLaAwLiRBSVLRII/lI1OK
fN9hOQ8wvMaHoKfuHK4A/RmUtt1G7k8HEsAhsMSWAYFUd0KdlbEd4snhcxaR6W+M
=n7yQ
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 15:12 ` Ian Stakenvicius
@ 2014-04-01 15:32   ` Michał Górny
  0 siblings, 0 replies; 25+ messages in thread
From: Michał Górny @ 2014-04-01 15:32 UTC (permalink / raw
  To: gentoo-dev; +Cc: axs

[-- Attachment #1: Type: text/plain, Size: 5912 bytes --]

Dnia 2014-04-01, o godz. 11:12:11
Ian Stakenvicius <axs@gentoo.org> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 31/03/14 06:16 PM, Michał Górny wrote:
> > Hello, all.
> > 
> > The late multilib ppc issues made me re-check our stable masks on 
> > abi_x86_* flags and, honestly, I'm not sure if we're doing things 
> > the right way. First, a quick summary.
> > 
> > 
> > Let's consider dev-libs/elfutils that has a dependency on
> > app-arch/bzip2. If we're building 32-bit elfutils, it needs 32-bit 
> > libbz2. This is enforced through a dep in form of:
> > 
> > app-arch/bzip2[${MULTILIB_USEDEP}]
> > 
> > that gets expanded into:
> > 
> > app-arch/bzip2[abi_x86_32(-)?,abi_x86_64(-)?,...,abi_mips_o32(-)?,...]
> >
> >  which means that any of the ABI_* flags that gets enabled on
> > elfutils, needs to be enabled on bzip2 as well. Of course, some of
> > use.forcing and masking gets applied here but that doesn't really
> > matter.
> > 
> > 
> > Now, since we're pretty much converting a lot of different
> > packages, some of them are eligible for stabilization earlier than
> > the others. However, the extra MULTILIB_USEDEPs enforce stabilizing
> > multilib dependencies before the actual packages which made a few
> > developers unhappy.
> > 
> > That's why we decided to stable-mask the flags on affected
> > packages. Since the flags are masked on stable, people can
> > stabilize packages independently of whether their converted deps
> > are stable or not. We will be worrying about that consistency once
> > we decide to unmask the flags.
> > 
> > The extra advantage of that is that we can avoid pushing stable
> > users into the mess involved with partial conversion of emul-linux.
> > The idea is pretty simple: we keep emul-linux for stable, and once
> > everything is ready, we stable-unmask the flags and let stable
> > users grok multilib.
> > 
> > 
> > Now, to the problem. Currently we're just stable-masking
> > abi_x86_32 on amd64. This serves the latter purpose well and seems
> > to work for the former. This is probably because the remaining flag
> > is use.forced (abi_x86_64 on amd64, and abi_x86_32 on x86) which
> > seems to add it to implicit IUSE somehow.
> > 
> > floppym has done some research w/ stable elfutils and no stable 
> > converted bzip2. It seems that if abi_x86_32 is stable.use.masked 
> > and abi_x86_64 is use.forced, repoman groks it. However, if we
> > remove abi_x86_64 from use.force, it properly reports flag mismatch
> > error (since no stable bzip2 has IUSE=abi_x86_64).
> > 
> > Now, I honestly have no idea if this implicit use.force behavior
> > is PMS-y or not, and how other PMs handle it. I can't find
> > something like this in the PMS but that doc is horribly hard to
> > cross-reference, so I might be missing something. I'd appreciate if
> > someone could help me with that.
> > 
> > 
> > That said, I have an alternate idea inspired by the ppc breakage.
> > I'm thinking of replacing the amd64 abi_x86_32 mask with a global
> > stable mask of all abi_*_* flags on the relevant packages.
> > 
> > Differences:
> > 
> > 1) old solution: native flag is forced, other flags are masked. new
> > solution: all flags are masked.
> > 
> > 2) old solution: we need to replicate the masks properly for
> > different arches/profiles. new solution: we can keep a single mask
> > for all arches.
> > 
> > 3) old solution: MULTILIB_USEDEP magically works (w/ portage at
> > least). new solution: since all flags are disabled,
> > MULTILIB_USEDEP is a no-op and old packages match correctly.
> > 
> > 4) old solution: forced native flag runs the native build. new
> > solution: fallback code runs the native build (since no flags are
> > enabled).
> > 
> > 
> > Your thoughts?
> > 
> 
> 
> ... how messy is this going to be on current systems, if implemented?
>  This -is- going to cause use flag changes on a whackload of packages,
> which at minimum I expect means a bunch of needless rebuilds.
> 
> I think it's probably best to keep with the current path, and just
> hurry up the migration on x86.  If we switch to masking all abi_*_*'s,
> then not only do we rebuild a whackload of packages now, but we also
> end up rebuilding them again when we remove the abi_*_* mask, -even
> if- the alternative abi's aren't needed.

Note that the same issue is spilling into ppc now that vapier wants to
introduce multilib there. If we keep going the current way, we need to
mask abi_ppc_32 in the ppc profiles, and that means yet another copy of
the same mask to maintain in sync.

> A note on #3: if MULTILIB_USEDEP -isnt- working on other PMs, i hope
> we have bug reports and we should work to resolve those ASAP or
> abandon the project/find a different solution.  We're over a year into
> it though (i think?) so i expect it's fine since I don't remember
> hearing of any issues yet...

I don't know how many people are using Gentoo with alternative PMs
and stable keywords. I think Paludis could be a case but then, I don't
know how Exherbo incorporates our multilib into their package system.

> on a side note, to follow up -- when we do remove the masks, what're
> the thoughts on perhaps changing the multilib profile on amd64 to
> auto-enable abi_x86_32 flags on some packages?  Indeed, if the
> toolchain is converted to multilib-build, would this be enough to
> differentiate between amd64-multilib and pure64?  (bootstrapping may
> of course still be a problem if users are trying to switch, but
> probably not any more of a problem than it is right now, right?)

Could you be more specific on which packages? I used to think about
enabling it globally but now I see that autounmask would do a pretty
good work of unmasking the flags whenever necessary.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Re: Stable masks on multilib packages
  2014-04-01 11:26     ` [gentoo-dev] " Duncan
@ 2014-04-01 15:49       ` Ian Stakenvicius
  2014-04-01 15:55         ` Michał Górny
  0 siblings, 1 reply; 25+ messages in thread
From: Ian Stakenvicius @ 2014-04-01 15:49 UTC (permalink / raw
  To: gentoo-dev

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 01/04/14 07:26 AM, Duncan wrote:
> Altho I'm ~arch so stable masks won't directly affect me, but at
> the same time, I'm no-multilib, so all those --newuse and -rX-bump
> rebuilds really ARE useless rebuilds of the /exact/ same thing!
> While my system is fast enough it's not a big deal here, it's still
> a pain tracking all those -rX bumps to see what justifies the bump
> (at least I can tell simple USE flag changes on sight), and I hurt
> for those with slower systems!

OK, on no-multilib profiles, all of this stuff should be USE-masked
anyhow -- so far as I know, you can't even build the abi_x86_32 side
pf a package if your toolchain wasn't USE=multilib to begin with, can you?


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iF4EAREIAAYFAlM64BEACgkQ2ugaI38ACPACKAD9Ftshw+pl2RPunBcC3ZTDfhqi
y/Ertrl546FHUPzZ1e4A/1/bm7dGP894d+Rc2XfczJk3QoM/y1OcZ6y9ta3z7gGw
=n8me
-----END PGP SIGNATURE-----


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Re: Stable masks on multilib packages
  2014-04-01 15:49       ` Ian Stakenvicius
@ 2014-04-01 15:55         ` Michał Górny
  0 siblings, 0 replies; 25+ messages in thread
From: Michał Górny @ 2014-04-01 15:55 UTC (permalink / raw
  To: gentoo-dev; +Cc: axs

[-- Attachment #1: Type: text/plain, Size: 1091 bytes --]

Dnia 2014-04-01, o godz. 11:49:37
Ian Stakenvicius <axs@gentoo.org> napisał(a):

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
> 
> On 01/04/14 07:26 AM, Duncan wrote:
> > Altho I'm ~arch so stable masks won't directly affect me, but at
> > the same time, I'm no-multilib, so all those --newuse and -rX-bump
> > rebuilds really ARE useless rebuilds of the /exact/ same thing!
> > While my system is fast enough it's not a big deal here, it's still
> > a pain tracking all those -rX bumps to see what justifies the bump
> > (at least I can tell simple USE flag changes on sight), and I hurt
> > for those with slower systems!
> 
> OK, on no-multilib profiles, all of this stuff should be USE-masked
> anyhow -- so far as I know, you can't even build the abi_x86_32 side
> pf a package if your toolchain wasn't USE=multilib to begin with, can you?

No-no-no.

abi_x86_64 designating the native ABI should be there and binary
packages can use it in dependencies. Which pretty much makes me think
we oughtn't mask all the flags.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-03-31 22:16 [gentoo-dev] Stable masks on multilib packages Michał Górny
                   ` (2 preceding siblings ...)
  2014-04-01 15:12 ` Ian Stakenvicius
@ 2014-04-01 17:15 ` Michał Górny
  2014-04-02  8:12   ` [gentoo-dev] " Duncan
  3 siblings, 1 reply; 25+ messages in thread
From: Michał Górny @ 2014-04-01 17:15 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

[-- Attachment #1: Type: text/plain, Size: 2688 bytes --]

Dnia 2014-04-01, o godz. 00:16:17
Michał Górny <mgorny@gentoo.org> napisał(a):

> Your thoughts?

Following the feedback on the mailing list, IRC and bugzilla, and some
of my own thoughts, I'd like to put a few important extra points I've
missed in the original mail:

1. playing with the masks will cause a large unnecessary --changed-use
rebuild for stable users, including the users of no-multilib profiles,

2. prebuilt software ebuilds are allowed to unconditionally use abi_*
flags in USE dependencies. Masking the native flags would break those
dependencies.

For those unaware, 2. here means that a pre-built x86 software can use
a single dependency like:

  dev-libs/libfoo[abi_x86_32]

that would satisfy both x86 (since abi_x86_32 is use.forced) and amd64
(by pulling in multilib).

Considering the two points above, I'd like to withdraw the idea of
adding global masks for abi_* and just leave it amd64-specific.


Additionally, the following important points were raised:

1. ulm believes that the use.force implicit IUSE magic is a bug in
portage and oughtn't be trusted upon,

2. keeping extra package.use.stable.masks for different arches seems
pretty much unmaintainable. It would mean a lot of duplicate package
lists in different profiles that need to be kept in sync.

Instead, I'd like to limit the scope of stable masks necessity and just
focus on stabilizing packages properly.


That is, the new position is:

1. we keep the flags for packages that are in emul-linux stable-masked
on amd64 until we decide to replace emul-linux,

2. we stabilize package dependencies properly so that there are no
dependency tree breakage when new ABIs are added.

Rationale: the multilib conversions are no longer moving in the fast
pace they used to, and the stabilization interest conflicts are less
likely than they were at the time of introducing
package.use.stable.masks.

Especially now that new multilib conversions are made package.masked,
we can give them enough time for people to be ready to stabilize
in some sane way.

We'd like to still keep the emul-linux-replacing packages masked
in order to keep emul-linux as single multilib provider for stable
packages. This is in order to avoid exposing stable users to issues
involved with partial conversions, and to confusing --autounmask
behavior that can possibly be created through mixing emul-linux
and multilib packages.


If nobody minds, I'm doing a repoman run now and in a few hours I
should have a list of packages that need to be stabilized in order to
clean up the dependency trees for multilib packages.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 966 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01  5:13 ` Ben de Groot
  2014-04-01 13:58   ` Alexandre Rostovtsev
@ 2014-04-01 23:38   ` Patrick Lauer
  2014-04-02  9:25     ` Ben de Groot
  1 sibling, 1 reply; 25+ messages in thread
From: Patrick Lauer @ 2014-04-01 23:38 UTC (permalink / raw
  To: gentoo-dev

On 04/01/2014 01:13 PM, Ben de Groot wrote:
> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
>> Hello, all.
>>
>> The late multilib ppc issues made me re-check our stable masks on
>> abi_x86_* flags and, honestly, I'm not sure if we're doing things
>> the right way.
>>
>> That said, I have an alternate idea inspired by the ppc breakage.
>>
>> Your thoughts?
> 
> In my opinion your multilib approach introduces an unnecessary degree
> of complexity, which --as has been shown here again-- is prone to
> breakage.

And it removes any chance of writing ebuilds - I seriously have no idea
how to fix those things now. They are multibuilds, not ebuilds.
> 
> It would be best for our beloved distro to revert all the multilib
> changes, and try a different approach, or leave this prone-to-breakage
> implementation to an overlay for the few people who would actually
> benefit from it.
> 
As a temporary stage they are kinda okish, maybe ... but ...

the whole transition strategy has been very very silly and should have
been staged in an overlay. I'd even build-test them and file bugs - just
don't do this salami tactic of one breakage a day.


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 14:43     ` Rich Freeman
@ 2014-04-02  0:13       ` Patrick Lauer
  2014-04-02  0:58         ` Rich Freeman
  0 siblings, 1 reply; 25+ messages in thread
From: Patrick Lauer @ 2014-04-02  0:13 UTC (permalink / raw
  To: gentoo-dev

On 04/01/2014 10:43 PM, Rich Freeman wrote:
> On Tue, Apr 1, 2014 at 9:58 AM, Alexandre Rostovtsev
> <tetromino@gentoo.org> wrote:
>> On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
>>>
>>> In my opinion your multilib approach introduces an unnecessary degree
>>> of complexity, which --as has been shown here again-- is prone to
>>> breakage.
>>>
>>> It would be best for our beloved distro to revert all the multilib
>>> changes, and try a different approach, or leave this prone-to-breakage
>>> implementation to an overlay for the few people who would actually
>>> benefit from it.
>>
>> I am aware of only two solutions to the emul-linux-x86-* problems :
>> multilib-portage and multilib-build.eclass. The first requires everybody
>> to switch to a new package manager. The second allows us to keep using
>> portage, but requires library maintainers to add some simple boilerplate
>> to their ebuilds for multilib support.
>>
>> Do you have yet another alternative in mind?
> 
> ++
> 
> I'm all for better solutions.  I'm not in favor of abandoning
> solutions that work moderately well in favor of talking about maybe
> coming up with something better sometime down the road.
> 
> The multilib eclass isn't a perfect solution.  It will have issues in
> concept, design, and implementation.  These will come up from time to
> time.  I don't think it is productive that anytime any of these pop up
> that we end up having a discussion about just reverting it entirely.

So why do we have this discussion?

> By all means work on a competing solution.  Get it working and
> supported by portage.  When the day comes that we want to endorse one
> solution or the other as the preferred solution we can have that
> discussion.

That is hilariously funny. Wow.

Now let's just continue to ignore the existing multilib-portage work so
we can claim it's irrelevant, while shifting the conditions for
accepting it whenever it is convenient, while silently adding the
competing method in-tree so it's all decided now anyway ...




^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  0:13       ` Patrick Lauer
@ 2014-04-02  0:58         ` Rich Freeman
  2014-04-02 17:42           ` Ciaran McCreesh
  0 siblings, 1 reply; 25+ messages in thread
From: Rich Freeman @ 2014-04-02  0:58 UTC (permalink / raw
  To: gentoo-dev

On Tue, Apr 1, 2014 at 8:13 PM, Patrick Lauer <patrick@gentoo.org> wrote:
> Now let's just continue to ignore the existing multilib-portage work so
> we can claim it's irrelevant, while shifting the conditions for
> accepting it whenever it is convenient, while silently adding the
> competing method in-tree so it's all decided now anyway ...

Conditions for accepting it?  What are those, and how have they changed?

Rich


^ permalink raw reply	[flat|nested] 25+ messages in thread

* [gentoo-dev] Re: Stable masks on multilib packages
  2014-04-01 17:15 ` Michał Górny
@ 2014-04-02  8:12   ` Duncan
  0 siblings, 0 replies; 25+ messages in thread
From: Duncan @ 2014-04-02  8:12 UTC (permalink / raw
  To: gentoo-dev

Michał Górny posted on Tue, 01 Apr 2014 19:15:26 +0200 as excerpted:

> If nobody minds, I'm doing a repoman run now and in a few hours I should
> have a list of packages that need to be stabilized in order to clean up
> the dependency trees for multilib packages.

All this seems sane by me.  Thanks for all the work you and others are 
putting in on this, even if it's not something I'm likely to use, because 
I know other gentooers do. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman



^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 13:58   ` Alexandre Rostovtsev
  2014-04-01 14:43     ` Rich Freeman
@ 2014-04-02  9:14     ` Ben de Groot
  2014-04-02 13:07       ` Mike Frysinger
  2014-04-02 13:43       ` Alexandre Rostovtsev
  1 sibling, 2 replies; 25+ messages in thread
From: Ben de Groot @ 2014-04-02  9:14 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

On 1 April 2014 21:58, Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
> On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
>> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
>> > Hello, all.
>> >
>> > The late multilib ppc issues made me re-check our stable masks on
>> > abi_x86_* flags and, honestly, I'm not sure if we're doing things
>> > the right way.
>> >
>> > That said, I have an alternate idea inspired by the ppc breakage.
>> >
>> > Your thoughts?
>>
>> In my opinion your multilib approach introduces an unnecessary degree
>> of complexity, which --as has been shown here again-- is prone to
>> breakage.
>>
>> It would be best for our beloved distro to revert all the multilib
>> changes, and try a different approach, or leave this prone-to-breakage
>> implementation to an overlay for the few people who would actually
>> benefit from it.
>
> Speaking as a wine maintainer, the emul-linux-x86-* approach has many
> times been proven to be an embarrassing failure and the main source of
> pain and frustration for wine users. The sooner emul-linux-x86-* can be
> removed from the tree, the better for Gentoo.

I would like to see an honest cost-benefit analysis of the
emul-linux-x86 approach compared to the multilib eclass approach.
Because in my experience the latter introduces more breakage and
higher maintenance costs.

> I am aware of only two solutions to the emul-linux-x86-* problems :
> multilib-portage and multilib-build.eclass. The first requires everybody
> to switch to a new package manager. The second allows us to keep using
> portage, but requires library maintainers to add some simple boilerplate
> to their ebuilds for multilib support.
>
> Do you have yet another alternative in mind?

In my mind the emul-linux-x86 approach is more acceptable. I don't
have experience with multilib-portage, as I don't have a use case for
it. Another option, which seems to me to be more reasonable and which
has greatly lower maintenance costs, is using a chroot.

-- 
Cheers,

Ben | yngwin
Gentoo developer


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-01 23:38   ` Patrick Lauer
@ 2014-04-02  9:25     ` Ben de Groot
  2014-04-02 13:55       ` Alexandre Rostovtsev
  2014-04-02 20:52       ` Matt Turner
  0 siblings, 2 replies; 25+ messages in thread
From: Ben de Groot @ 2014-04-02  9:25 UTC (permalink / raw
  To: gentoo-dev

On 2 April 2014 07:38, Patrick Lauer <patrick@gentoo.org> wrote:
> On 04/01/2014 01:13 PM, Ben de Groot wrote:
>> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
>>> Hello, all.
>>>
>>> The late multilib ppc issues made me re-check our stable masks on
>>> abi_x86_* flags and, honestly, I'm not sure if we're doing things
>>> the right way.
>>>
>>> That said, I have an alternate idea inspired by the ppc breakage.
>>>
>>> Your thoughts?
>>
>> In my opinion your multilib approach introduces an unnecessary degree
>> of complexity, which --as has been shown here again-- is prone to
>> breakage.
>
> And it removes any chance of writing ebuilds - I seriously have no idea
> how to fix those things now. They are multibuilds, not ebuilds.

This is part of the complexity I mentioned. I significantly raises
maintenance costs, and I'm not convinced of the benefit.

>>
>> It would be best for our beloved distro to revert all the multilib
>> changes, and try a different approach, or leave this prone-to-breakage
>> implementation to an overlay for the few people who would actually
>> benefit from it.
>>
> As a temporary stage they are kinda okish, maybe ... but ...
>
> the whole transition strategy has been very very silly and should have
> been staged in an overlay. I'd even build-test them and file bugs - just
> don't do this salami tactic of one breakage a day.
>

I'm strongly considering reverting these changes in the packages I
maintain. I'm tired of having to deal time and again with multilib
breakage.

Either that, or someone else can take over primary maintainership.

-- 
Cheers,

Ben | yngwin
Gentoo developer


^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  9:14     ` Ben de Groot
@ 2014-04-02 13:07       ` Mike Frysinger
  2014-04-02 13:43       ` Alexandre Rostovtsev
  1 sibling, 0 replies; 25+ messages in thread
From: Mike Frysinger @ 2014-04-02 13:07 UTC (permalink / raw
  To: gentoo-dev; +Cc: Ben de Groot, multilib

[-- Attachment #1: Type: text/plain, Size: 2046 bytes --]

On Wed 02 Apr 2014 17:14:02 Ben de Groot wrote:
> On 1 April 2014 21:58, Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
> > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
> >> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
> >> > Hello, all.
> >> > 
> >> > The late multilib ppc issues made me re-check our stable masks on
> >> > abi_x86_* flags and, honestly, I'm not sure if we're doing things
> >> > the right way.
> >> > 
> >> > That said, I have an alternate idea inspired by the ppc breakage.
> >> > 
> >> > Your thoughts?
> >> 
> >> In my opinion your multilib approach introduces an unnecessary degree
> >> of complexity, which --as has been shown here again-- is prone to
> >> breakage.
> >> 
> >> It would be best for our beloved distro to revert all the multilib
> >> changes, and try a different approach, or leave this prone-to-breakage
> >> implementation to an overlay for the few people who would actually
> >> benefit from it.
> > 
> > Speaking as a wine maintainer, the emul-linux-x86-* approach has many
> > times been proven to be an embarrassing failure and the main source of
> > pain and frustration for wine users. The sooner emul-linux-x86-* can be
> > removed from the tree, the better for Gentoo.
> 
> I would like to see an honest cost-benefit analysis of the
> emul-linux-x86 approach compared to the multilib eclass approach.
> Because in my experience the latter introduces more breakage and
> higher maintenance costs.

the emul-linux-* approach is a terrible idea.  it has obviously proven to:
 - not scale (sorry, but x86 is not the only ABI out there people care about)
 - be a huge pita to update/maintain
 - require shipping a lot of precompiled code

the multilib eclasses aren't perfect, but they're a hell of a lot better than 
the emul-linux-* approach.  we've been carrying that crap for over 10 years 
and it's gone literally nowhere.  considering it as a viable alternative to 
the multilib eclasses is preposterous at best.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  9:14     ` Ben de Groot
  2014-04-02 13:07       ` Mike Frysinger
@ 2014-04-02 13:43       ` Alexandre Rostovtsev
  1 sibling, 0 replies; 25+ messages in thread
From: Alexandre Rostovtsev @ 2014-04-02 13:43 UTC (permalink / raw
  To: gentoo-dev; +Cc: multilib

[-- Attachment #1: Type: text/plain, Size: 2613 bytes --]

On Wed, 2014-04-02 at 17:14 +0800, Ben de Groot wrote:
> On 1 April 2014 21:58, Alexandre Rostovtsev <tetromino@gentoo.org> wrote:
> > On Tue, 2014-04-01 at 13:13 +0800, Ben de Groot wrote:
> >> On 1 April 2014 06:16, Michał Górny <mgorny@gentoo.org> wrote:
> >> > Hello, all.
> >> >
> >> > The late multilib ppc issues made me re-check our stable masks on
> >> > abi_x86_* flags and, honestly, I'm not sure if we're doing things
> >> > the right way.
> >> >
> >> > That said, I have an alternate idea inspired by the ppc breakage.
> >> >
> >> > Your thoughts?
> >>
> >> In my opinion your multilib approach introduces an unnecessary degree
> >> of complexity, which --as has been shown here again-- is prone to
> >> breakage.
> >>
> >> It would be best for our beloved distro to revert all the multilib
> >> changes, and try a different approach, or leave this prone-to-breakage
> >> implementation to an overlay for the few people who would actually
> >> benefit from it.
> >
> > Speaking as a wine maintainer, the emul-linux-x86-* approach has many
> > times been proven to be an embarrassing failure and the main source of
> > pain and frustration for wine users. The sooner emul-linux-x86-* can be
> > removed from the tree, the better for Gentoo.
> 
> I would like to see an honest cost-benefit analysis of the
> emul-linux-x86 approach compared to the multilib eclass approach.
> Because in my experience the latter introduces more breakage and
> higher maintenance costs.

libfoo-1.5 becomes available ~arch. Users install it. Users realize that
libfoo-1.5's headers are incompatible with libfoo-1.4 binaries from
emul-linux-x86-bar after they try to build wine and get a failure 30
minutes into the compile. So users complain about gentoo's wine support.

libfoo-1.5 is stabilized. Of course, emul-linux-x86-bar has not been
updated immediately, because *all* parts of emul-linux-x86 need to be
updated at once due to a complex network of library interdependencies,
and then manually checked for sanity, and the folks taking care of
emul-linux-x86 can only really afford to do it 2, max 3 times per year.
So more users install libfoo-1.5, and see wine fail, and complain more
about gentoo's wine support until a new emul-linux-x86 release is rolled
out a couple months later.

But meanwhile, libbaz-5.0 becomes available in ~arch, and its headers
are backwards-incompatible with libbaz-4.2 binaries in
emul-linux-x86-wombatlibs...

And this happens, oh, only every time there are API changes in libpng,
jpeg, llvm, gnutls, libxml2, pulseaudio, etc.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  9:25     ` Ben de Groot
@ 2014-04-02 13:55       ` Alexandre Rostovtsev
  2014-04-02 20:52       ` Matt Turner
  1 sibling, 0 replies; 25+ messages in thread
From: Alexandre Rostovtsev @ 2014-04-02 13:55 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 655 bytes --]

On Wed, 2014-04-02 at 17:25 +0800, Ben de Groot wrote:
> I'm strongly considering reverting these changes in the packages I
> maintain. I'm tired of having to deal time and again with multilib
> breakage.
> 
> Either that, or someone else can take over primary maintainership.

Ben, if you are uncomfortable with multilib changes in cairo and don't
want to see it unmasked ~arch, I am volunteering to co-maintain it.
Assign any multilib issues with it to me. It's worth it to me because I
am a wine maintainer, and wine will benefit enormously once all of its
dependencies have finally been multilibbed and we can unmask the flags
in stable.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  0:58         ` Rich Freeman
@ 2014-04-02 17:42           ` Ciaran McCreesh
  0 siblings, 0 replies; 25+ messages in thread
From: Ciaran McCreesh @ 2014-04-02 17:42 UTC (permalink / raw
  To: gentoo-dev

[-- Attachment #1: Type: text/plain, Size: 648 bytes --]

On Tue, 1 Apr 2014 20:58:30 -0400
Rich Freeman <rich0@gentoo.org> wrote:
> On Tue, Apr 1, 2014 at 8:13 PM, Patrick Lauer <patrick@gentoo.org>
> wrote:
> > Now let's just continue to ignore the existing multilib-portage
> > work so we can claim it's irrelevant, while shifting the conditions
> > for accepting it whenever it is convenient, while silently adding
> > the competing method in-tree so it's all decided now anyway ...
> 
> Conditions for accepting it?  What are those, and how have they
> changed?

The conditions have always been "someone has to be able to explain
exactly what the changes are"...

-- 
Ciaran McCreesh

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

^ permalink raw reply	[flat|nested] 25+ messages in thread

* Re: [gentoo-dev] Stable masks on multilib packages
  2014-04-02  9:25     ` Ben de Groot
  2014-04-02 13:55       ` Alexandre Rostovtsev
@ 2014-04-02 20:52       ` Matt Turner
  1 sibling, 0 replies; 25+ messages in thread
From: Matt Turner @ 2014-04-02 20:52 UTC (permalink / raw
  To: gentoo-dev

On Wed, Apr 2, 2014 at 2:25 AM, Ben de Groot <yngwin@gentoo.org> wrote:
> I'm strongly considering reverting these changes in the packages I
> maintain. I'm tired of having to deal time and again with multilib
> breakage.
>
> Either that, or someone else can take over primary maintainership.

I'd be happy for you to remove yourself as a maintainer of cairo.
You've got one commit (not removing ebuilds) since July 2012,
according to the changelog. At least three X11 team members have more
commits than that, and I don't know how much sense it makes to have a
non-X11-team member being the primary maintainer of a critical X11
package anyway.


^ permalink raw reply	[flat|nested] 25+ messages in thread

end of thread, other threads:[~2014-04-03 12:46 UTC | newest]

Thread overview: 25+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-03-31 22:16 [gentoo-dev] Stable masks on multilib packages Michał Górny
2014-03-31 23:09 ` Anthony G. Basile
2014-04-01  5:54   ` Michał Górny
2014-04-01 11:26     ` [gentoo-dev] " Duncan
2014-04-01 15:49       ` Ian Stakenvicius
2014-04-01 15:55         ` Michał Górny
2014-04-01 11:40     ` [gentoo-dev] " Anthony G. Basile
2014-04-01 12:04       ` Anthony G. Basile
2014-04-01  5:13 ` Ben de Groot
2014-04-01 13:58   ` Alexandre Rostovtsev
2014-04-01 14:43     ` Rich Freeman
2014-04-02  0:13       ` Patrick Lauer
2014-04-02  0:58         ` Rich Freeman
2014-04-02 17:42           ` Ciaran McCreesh
2014-04-02  9:14     ` Ben de Groot
2014-04-02 13:07       ` Mike Frysinger
2014-04-02 13:43       ` Alexandre Rostovtsev
2014-04-01 23:38   ` Patrick Lauer
2014-04-02  9:25     ` Ben de Groot
2014-04-02 13:55       ` Alexandre Rostovtsev
2014-04-02 20:52       ` Matt Turner
2014-04-01 15:12 ` Ian Stakenvicius
2014-04-01 15:32   ` Michał Górny
2014-04-01 17:15 ` Michał Górny
2014-04-02  8:12   ` [gentoo-dev] " Duncan

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox