[gentoo-python] python-r1 <-> python.eclass package dependencies

[gentoo-python] python-r1 <-> python.eclass package dependencies

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2864 bytes --]

Hello,

As some of you are aware already, there is a significant difference
in handling Python implementation choice between python.eclass
and python-r1 / p-d-ng.

The python.eclass packages use USE_PYTHON (undocumented variable)
or the eselected Python2+3, and store that choice implicitly.
Everything works on the assumption that implementations don't change
or user runs python-updater.

The python-r1 / p-d-ng packages use explicit USE flags to choose
implementation. The choice is thus explicitly exported and other
packages can use USE dependencies to enforce matching implementations.

The problems arise whenever python-r1 package depends on package using
the python.eclass, or the other way round. There is no clear way to
enforce the same implementations being used, so everything falls down
to user.

In order to solve the most common problems due to that, I have
published a news item explaining how PYTHON_TARGETS work and what needs
to be done with them.

In the discussion with other Gentoo Python devs, it was assumed that
USE_PYTHON should not be considered public and thus not be described
in the message. Sadly, it seems that this resulted in people actually
trying to disable Python3 using PYTHON_TARGETS and seeing random
failures due to python.eclass enforcing it anyway.


I believe we should thoroughly think what should be done now in order
to solve the issues best for the transition time. Here are a few
solutions I can think of.

1) lard both eclasses with USE_PYTHON <-> PYTHON_TARGETS consistency
checks. This should be pretty easy for python-r1, harder for
python.eclass (anyone volunteering to touch it in thick gloves?).

In other words, we just warn users whenever they need to do something
to ensure things work.


2) make python.eclass respect PYTHON_TARGETS.

Sadly, due to PMS limitations we can't tell if PYTHON_TARGETS was set
by user or has the default value. So we either can:

a) stop using USE_PYTHON and eselect defaults, start using
PYTHON_TARGETS only, pull in correct Python versions, tell users to run
python-updater;

b) prefer PYTHON_TARGETS over USE_PYTHON and the default (but only
if PYTHON_TARGETS != default PYTHON_TARGETS, so we're on thin ice here).


3) make python.eclass respect and export PYTHON_TARGETS in IUSE.

Like the above but instead of running the fragile and broken
python-updater, we simply force rebuild of all packages. People can
start using USE-deps in the meantime, everyone becomes a bit happier
in the further plane. Sadly, this means every Python package is going
to need being rebuilt.


Please also note that 2 & 3 mean mass-changing behavior of stable
packages. Not something I'd do in a short timeframe where the issue
should be best fixed.

Any other ideas? Any +1, -1s?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Ben de Groot]

[-- Attachment #1: Type: text/plain, Size: 3519 bytes --]

On 11 November 2012 00:43, Michał Górny <mgorny@gentoo.org> wrote:

> Hello,
>
> As some of you are aware already, there is a significant difference
> in handling Python implementation choice between python.eclass
> and python-r1 / p-d-ng.
>
> The python.eclass packages use USE_PYTHON (undocumented variable)
> or the eselected Python2+3, and store that choice implicitly.
> Everything works on the assumption that implementations don't change
> or user runs python-updater.
>
> The python-r1 / p-d-ng packages use explicit USE flags to choose
> implementation. The choice is thus explicitly exported and other
> packages can use USE dependencies to enforce matching implementations.
>
> The problems arise whenever python-r1 package depends on package using
> the python.eclass, or the other way round. There is no clear way to
> enforce the same implementations being used, so everything falls down
> to user.
>
> In order to solve the most common problems due to that, I have
> published a news item explaining how PYTHON_TARGETS work and what needs
> to be done with them.
>
> In the discussion with other Gentoo Python devs, it was assumed that
> USE_PYTHON should not be considered public and thus not be described
> in the message. Sadly, it seems that this resulted in people actually
> trying to disable Python3 using PYTHON_TARGETS and seeing random
> failures due to python.eclass enforcing it anyway.
>
>
> I believe we should thoroughly think what should be done now in order
> to solve the issues best for the transition time. Here are a few
> solutions I can think of.
>
> 1) lard both eclasses with USE_PYTHON <-> PYTHON_TARGETS consistency
> checks. This should be pretty easy for python-r1, harder for
> python.eclass (anyone volunteering to touch it in thick gloves?).
>
> In other words, we just warn users whenever they need to do something
> to ensure things work.
>
>
> 2) make python.eclass respect PYTHON_TARGETS.
>
> Sadly, due to PMS limitations we can't tell if PYTHON_TARGETS was set
> by user or has the default value. So we either can:
>
> a) stop using USE_PYTHON and eselect defaults, start using
> PYTHON_TARGETS only, pull in correct Python versions, tell users to run
> python-updater;
>
> b) prefer PYTHON_TARGETS over USE_PYTHON and the default (but only
> if PYTHON_TARGETS != default PYTHON_TARGETS, so we're on thin ice here).
>
>
> 3) make python.eclass respect and export PYTHON_TARGETS in IUSE.
>
> Like the above but instead of running the fragile and broken
> python-updater, we simply force rebuild of all packages. People can
> start using USE-deps in the meantime, everyone becomes a bit happier
> in the further plane. Sadly, this means every Python package is going
> to need being rebuilt.
>
>
> Please also note that 2 & 3 mean mass-changing behavior of stable
> packages. Not something I'd do in a short timeframe where the issue
> should be best fixed.
>
> Any other ideas? Any +1, -1s?
>
> --
> Best regards,
> Michał Górny
>

I'm sorry if I missed something, but I don't see why USE_PYTHON should be
considered not public. That has been the only mechanism until now (in
combination with package.mask) to get a consistent system without redundant
multiple python versions.

I think it is easiest to tell users to set USE_PYTHON as well, until
python.eclass is finally phased out.

-- 
Cheers,

Ben | yngwin
Gentoo developer
Gentoo Qt project lead, Gentoo Wiki admin

[-- Attachment #2: Type: text/html, Size: 4078 bytes --]

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 3800 bytes --]

On Sun, 11 Nov 2012 14:29:20 +0800
Ben de Groot <yngwin@gentoo.org> wrote:

> On 11 November 2012 00:43, Michał Górny <mgorny@gentoo.org> wrote:
> 
> > Hello,
> >
> > As some of you are aware already, there is a significant difference
> > in handling Python implementation choice between python.eclass
> > and python-r1 / p-d-ng.
> >
> > The python.eclass packages use USE_PYTHON (undocumented variable)
> > or the eselected Python2+3, and store that choice implicitly.
> > Everything works on the assumption that implementations don't change
> > or user runs python-updater.
> >
> > The python-r1 / p-d-ng packages use explicit USE flags to choose
> > implementation. The choice is thus explicitly exported and other
> > packages can use USE dependencies to enforce matching implementations.
> >
> > The problems arise whenever python-r1 package depends on package using
> > the python.eclass, or the other way round. There is no clear way to
> > enforce the same implementations being used, so everything falls down
> > to user.
> >
> > In order to solve the most common problems due to that, I have
> > published a news item explaining how PYTHON_TARGETS work and what needs
> > to be done with them.
> >
> > In the discussion with other Gentoo Python devs, it was assumed that
> > USE_PYTHON should not be considered public and thus not be described
> > in the message. Sadly, it seems that this resulted in people actually
> > trying to disable Python3 using PYTHON_TARGETS and seeing random
> > failures due to python.eclass enforcing it anyway.
> >
> >
> > I believe we should thoroughly think what should be done now in order
> > to solve the issues best for the transition time. Here are a few
> > solutions I can think of.
> >
> > 1) lard both eclasses with USE_PYTHON <-> PYTHON_TARGETS consistency
> > checks. This should be pretty easy for python-r1, harder for
> > python.eclass (anyone volunteering to touch it in thick gloves?).
> >
> > In other words, we just warn users whenever they need to do something
> > to ensure things work.
> >
> >
> > 2) make python.eclass respect PYTHON_TARGETS.
> >
> > Sadly, due to PMS limitations we can't tell if PYTHON_TARGETS was set
> > by user or has the default value. So we either can:
> >
> > a) stop using USE_PYTHON and eselect defaults, start using
> > PYTHON_TARGETS only, pull in correct Python versions, tell users to run
> > python-updater;
> >
> > b) prefer PYTHON_TARGETS over USE_PYTHON and the default (but only
> > if PYTHON_TARGETS != default PYTHON_TARGETS, so we're on thin ice here).
> >
> >
> > 3) make python.eclass respect and export PYTHON_TARGETS in IUSE.
> >
> > Like the above but instead of running the fragile and broken
> > python-updater, we simply force rebuild of all packages. People can
> > start using USE-deps in the meantime, everyone becomes a bit happier
> > in the further plane. Sadly, this means every Python package is going
> > to need being rebuilt.
> >
> >
> > Please also note that 2 & 3 mean mass-changing behavior of stable
> > packages. Not something I'd do in a short timeframe where the issue
> > should be best fixed.
> >
> > Any other ideas? Any +1, -1s?
> I'm sorry if I missed something, but I don't see why USE_PYTHON should be
> considered not public. That has been the only mechanism until now (in
> combination with package.mask) to get a consistent system without redundant
> multiple python versions.

Well, the most important reason is that I don't think USE_PYTHON is
pretty much documented *anywhere*.

> I think it is easiest to tell users to set USE_PYTHON as well, until
> python.eclass is finally phased out.

Yes, that's what the warning would say.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Brian Harring]

On Sat, Nov 10, 2012 at 05:43:22PM +0100, Micha?? G??rny wrote:
> Hello,
> 
> As some of you are aware already, there is a significant difference
> in handling Python implementation choice between python.eclass
> and python-r1 / p-d-ng.
> 
> The python.eclass packages use USE_PYTHON (undocumented variable)
> or the eselected Python2+3, and store that choice implicitly.
> Everything works on the assumption that implementations don't change
> or user runs python-updater.
>
> The python-r1 / p-d-ng packages use explicit USE flags to choose
> implementation. The choice is thus explicitly exported and other
> packages can use USE dependencies to enforce matching implementations.
> 
> The problems arise whenever python-r1 package depends on package using
> the python.eclass, or the other way round. There is no clear way to
> enforce the same implementations being used, so everything falls down
> to user.
> 
> In order to solve the most common problems due to that, I have
> published a news item explaining how PYTHON_TARGETS work and what needs
> to be done with them.
> 
> In the discussion with other Gentoo Python devs, it was assumed that
> USE_PYTHON should not be considered public and thus not be described
> in the message. Sadly, it seems that this resulted in people actually
> trying to disable Python3 using PYTHON_TARGETS and seeing random
> failures due to python.eclass enforcing it anyway.
> 
> 
> I believe we should thoroughly think what should be done now in order
> to solve the issues best for the transition time. Here are a few
> solutions I can think of.

Honestly... if we care this much about USE_PYTHON, then we should use 
it, rather than PYTHON_TARGETS.

Mind you, I'm aware someone ignored repeatedly being told "do not 
fucking use periods in the targets- it cannot be allowed as a use flag 
due to compatibility issues in profiles"- meaning we have to dig 
out of a mess that was foresaw, but such is life.

Fixing USE_PYTHON is simpler than trying to get users to migrate to a 
new var, let alone keep those two vars in sync.

To get there, we'd have to basically accept 2.6/2_6 in USE_PYTHON w/in 
python.eclass (a modification I will sort if others won't), have it 
spit warnings, then flip it to a USE_EXPAND target.

Portage, fortunately, is lose in this department- you can have a 
USE_EXPAND target w/ invalid use chars in it.  It's just not usable in 
use dependencies nor IUSE, making it a dead setting if it uses invalid 
chars.

So... thus:

1) make python.eclass know of both forms of USE_PYTHON (with periods, 
without).
2) convert USE_PYTHON to a use expanded target
3) convert python.eclass to use it.

I realize this deprecates/kills PYTHON_TARGETS; my intention here 
isn't to piss on the var you added, it's to choose the less disruptive 
option here- lesser of two evils.  Starting from scratch, 
PYTHON_TARGETS would be fine- unfortunately we need to map existing 
users (and usage) from python.eclass into the replacement, 
constraining our choises a bit.

Counter arguments?  To be clear, this is the path I strongly suggest 
we take- if you can punch holes in the logic/arguments from above, I'd 
definitely back down, but the use of PYTHON_TARGETS here feels like 
we're setting ourselves up for unnecessary pain.  Keep in mind not 
*all* of python.eclass notions/setup has to be chucked- we can 
translate certain parts of it across to ease developer/user pains.


> 1) lard both eclasses with USE_PYTHON <-> PYTHON_TARGETS consistency
> checks. This should be pretty easy for python-r1, harder for
> python.eclass (anyone volunteering to touch it in thick gloves?).
> 
> In other words, we just warn users whenever they need to do something
> to ensure things work.
> 
> 
> 2) make python.eclass respect PYTHON_TARGETS.
> 
> Sadly, due to PMS limitations we can't tell if PYTHON_TARGETS was set
> by user or has the default value. So we either can:
> 
> a) stop using USE_PYTHON and eselect defaults, start using
> PYTHON_TARGETS only, pull in correct Python versions, tell users to run
> python-updater;

Just a note; we /never/ should listen to eselect defaults, only USE 
enforced configuration; I'm appreciative you've not replicated that 
python.eclass idiocy since it craps all over 
deterministic/reproducible builds.

~harring

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Matthew Summers]

> Well, the most important reason is that I don't think USE_PYTHON is
> pretty much documented *anywhere*.
>
>> I think it is easiest to tell users to set USE_PYTHON as well, until
>> python.eclass is finally phased out.
>
> Yes, that's what the warning would say.
>
> --
> Best regards,
> Michał Górny

I wrote this awhile back, it is short and sweet. Feel free to mangle it.
http://dev.gentoo.org/~quantumsummers/use_python_guide.xml

-- 
Matthew W. Summers
Gentoo Foundation Inc.

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Mike Gilbert]

On Sun, Nov 11, 2012 at 2:52 AM, Brian Harring <ferringb@gmail.com> wrote:
> So... thus:
>
> 1) make python.eclass know of both forms of USE_PYTHON (with periods,
> without).
> 2) convert USE_PYTHON to a use expanded target
> 3) convert python.eclass to use it.
>
> I realize this deprecates/kills PYTHON_TARGETS; my intention here
> isn't to piss on the var you added, it's to choose the less disruptive
> option here- lesser of two evils.  Starting from scratch,
> PYTHON_TARGETS would be fine- unfortunately we need to map existing
> users (and usage) from python.eclass into the replacement,
> constraining our choises a bit.
>
> Counter arguments?  To be clear, this is the path I strongly suggest
> we take- if you can punch holes in the logic/arguments from above, I'd
> definitely back down, but the use of PYTHON_TARGETS here feels like
> we're setting ourselves up for unnecessary pain.  Keep in mind not
> *all* of python.eclass notions/setup has to be chucked- we can
> translate certain parts of it across to ease developer/user pains.
>

Following up on this and the irc conversation I eavesdropped on earlier:

More recent versions of portage apparently filter USE_EXPAND
variables, so we can't utilize the old python abi values in USE_PYTHON
if we make that conversion.

Converting USE_PYTHON to a use-expand would entail the following in
python.eclass:

1. Convert between values like "2_7-pypy-1_9" and "2.7-pypy-1.9"
The underscore version would be used for USE and IUSE and in portage
config (make.conf, profiles?).
The dot version should be used by the eclass and ebuilds and in
on-disk file names for backward compatibility.

2. Calculate the python abis supported by the ebuild based on
_PYTHON_GLOBALLY_SUPPORTED_ABIS and RESTRICT_PYTHON_ABIS. Add these
abis to IUSE.
This would affect all ebuilds inheriting python.eclass with
SUPPORT_PYTHON_ABIS set.

This should not be difficult, but you never know with python.eclass.
Personally, I just assume that nobody actually wants to touch
python.eclass if it can be avoided.

Also, we have dug ourselves into a small hole with PYTHON_TARGETS;
there is actually a news item out there that explains its use
(committed about the same time Brian put on the breaks), so we are
going to cause confusion if it is changed back to USE_PYTHON. I
realize this is not a reason to charge blindly forward with
PYTHON_TARGETS, but it needs to be considered.

I think the only benefit to keeping the USE_PYTHON name is so that
existing users don't have to learn another variable name. They would
still need to tweak their config.

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Brian Harring]

On Sun, Nov 11, 2012 at 01:35:20PM -0500, Mike Gilbert wrote:
> On Sun, Nov 11, 2012 at 2:52 AM, Brian Harring <ferringb@gmail.com> wrote:
> > So... thus:
> >
> > 1) make python.eclass know of both forms of USE_PYTHON (with periods,
> > without).
> > 2) convert USE_PYTHON to a use expanded target
> > 3) convert python.eclass to use it.
> >
> > I realize this deprecates/kills PYTHON_TARGETS; my intention here
> > isn't to piss on the var you added, it's to choose the less disruptive
> > option here- lesser of two evils.  Starting from scratch,
> > PYTHON_TARGETS would be fine- unfortunately we need to map existing
> > users (and usage) from python.eclass into the replacement,
> > constraining our choises a bit.
> >
> > Counter arguments?  To be clear, this is the path I strongly suggest
> > we take- if you can punch holes in the logic/arguments from above, I'd
> > definitely back down, but the use of PYTHON_TARGETS here feels like
> > we're setting ourselves up for unnecessary pain.  Keep in mind not
> > *all* of python.eclass notions/setup has to be chucked- we can
> > translate certain parts of it across to ease developer/user pains.
> >
> 
> Following up on this and the irc conversation I eavesdropped on earlier:
> 
> More recent versions of portage apparently filter USE_EXPAND
> variables, so we can't utilize the old python abi values in USE_PYTHON
> if we make that conversion.

Just a note; portage's behaviour here, per the norm, is more stupid 
than that.  EAPIs 0-4 are supposed to /not/ have that filtering, 
meaning portage broke it's own behaviour again.

I suspect we'll retroactively change EAPIs to compensate for this, but 
I don't know for a fact; the discussion for that will occur at 
https://bugs.gentoo.org/442830 .


> Converting USE_PYTHON to a use-expand would entail the following in
> python.eclass:
> 
> 1. Convert between values like "2_7-pypy-1_9" and "2.7-pypy-1.9"
> The underscore version would be used for USE and IUSE and in portage
> config (make.conf, profiles?).
> The dot version should be used by the eclass and ebuilds and in
> on-disk file names for backward compatibility.
> 
> 2. Calculate the python abis supported by the ebuild based on
> _PYTHON_GLOBALLY_SUPPORTED_ABIS and RESTRICT_PYTHON_ABIS. Add these
> abis to IUSE.
> This would affect all ebuilds inheriting python.eclass with
> SUPPORT_PYTHON_ABIS set.
> 
> This should not be difficult, but you never know with python.eclass.
> Personally, I just assume that nobody actually wants to touch
> python.eclass if it can be avoided.

As stated, I'll do the work if it decreases the pains for users and 
eases transition (whether it be USE_PYTHON or PYTHON_TARGETS).


> Also, we have dug ourselves into a small hole with PYTHON_TARGETS;
> there is actually a news item out there that explains its use
> (committed about the same time Brian put on the breaks), so we are
> going to cause confusion if it is changed back to USE_PYTHON. I
> realize this is not a reason to charge blindly forward with
> PYTHON_TARGETS, but it needs to be considered.

This is why I asked for the "basic legwork" that involves a 
roadmap/plan beyond the immediate patch.

The news item has had 5 days of exposure to users.  It's a bit 
amateur-hour to have to pull it, but if the USE_PYTHON fix-it route is 
ultimately less transition pain for users, we should go down that 
path.

I *suspect* that route may not be best, although if PYTHON_TARGETS is 
choosen as the approach, that news item (or another) needs to be cut 
explicitly clarifying that everything is moving away from USE_PYTHON.

Folks may view it as "an internal", but users have been using it.


> I think the only benefit to keeping the USE_PYTHON name is so that
> existing users don't have to learn another variable name. They would
> still need to tweak their config.

Either USE_PYTHON or PYTHON_TARGETS; the proposed notion of both is a 
recipe for pain.  To be clear, any proposal that involves both 
existing long term is an automatic -1 from me.

~harring

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Ben de Groot]

[-- Attachment #1: Type: text/plain, Size: 451 bytes --]

So before we go any further, let's hammer out a plan and bring it to -dev
ML.

We need to have a clear idea where we want to take the python eclasses, and
how we are going to deprecate the old ones. What we need are eclasses and
related variables that are easy to understand for developers who are
writing and maintaining ebuilds, as well as easy for end-users who don't
want redundant multiple python versions on their systems.

Cheers,
Ben | yngwin

[-- Attachment #2: Type: text/html, Size: 486 bytes --]

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2123 bytes --]

On Sun, 11 Nov 2012 15:50:03 -0800
Brian Harring <ferringb@gmail.com> wrote:

> On Sun, Nov 11, 2012 at 01:35:20PM -0500, Mike Gilbert wrote:
> > On Sun, Nov 11, 2012 at 2:52 AM, Brian Harring <ferringb@gmail.com> wrote:
> > > So... thus:
> > >
> > > 1) make python.eclass know of both forms of USE_PYTHON (with periods,
> > > without).
> > > 2) convert USE_PYTHON to a use expanded target
> > > 3) convert python.eclass to use it.
> > >
> > > I realize this deprecates/kills PYTHON_TARGETS; my intention here
> > > isn't to piss on the var you added, it's to choose the less disruptive
> > > option here- lesser of two evils.  Starting from scratch,
> > > PYTHON_TARGETS would be fine- unfortunately we need to map existing
> > > users (and usage) from python.eclass into the replacement,
> > > constraining our choises a bit.
> > >
> > > Counter arguments?  To be clear, this is the path I strongly suggest
> > > we take- if you can punch holes in the logic/arguments from above, I'd
> > > definitely back down, but the use of PYTHON_TARGETS here feels like
> > > we're setting ourselves up for unnecessary pain.  Keep in mind not
> > > *all* of python.eclass notions/setup has to be chucked- we can
> > > translate certain parts of it across to ease developer/user pains.
> > >
> > 
> > Following up on this and the irc conversation I eavesdropped on earlier:
> > 
> > More recent versions of portage apparently filter USE_EXPAND
> > variables, so we can't utilize the old python abi values in USE_PYTHON
> > if we make that conversion.
> 
> Just a note; portage's behaviour here, per the norm, is more stupid 
> than that.  EAPIs 0-4 are supposed to /not/ have that filtering, 
> meaning portage broke it's own behaviour again.
> 
> I suspect we'll retroactively change EAPIs to compensate for this, but 
> I don't know for a fact; the discussion for that will occur at 
> https://bugs.gentoo.org/442830 .

I'm sorry if I miss something but don't EAPIs 0-4 define *no* behavior
for USE_EXPAND variables? (something around don't use them)

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] python-r1 <-> python.eclass package dependencies

[Brian Harring]

On Mon, Nov 12, 2012 at 09:04:35AM +0100, Micha?? G??rny wrote:
> On Sun, 11 Nov 2012 15:50:03 -0800
> Brian Harring <ferringb@gmail.com> wrote:
> 
> > On Sun, Nov 11, 2012 at 01:35:20PM -0500, Mike Gilbert wrote:
> > > On Sun, Nov 11, 2012 at 2:52 AM, Brian Harring <ferringb@gmail.com> wrote:
> > > > So... thus:
> > > >
> > > > 1) make python.eclass know of both forms of USE_PYTHON (with periods,
> > > > without).
> > > > 2) convert USE_PYTHON to a use expanded target
> > > > 3) convert python.eclass to use it.
> > > >
> > > > I realize this deprecates/kills PYTHON_TARGETS; my intention here
> > > > isn't to piss on the var you added, it's to choose the less disruptive
> > > > option here- lesser of two evils.  Starting from scratch,
> > > > PYTHON_TARGETS would be fine- unfortunately we need to map existing
> > > > users (and usage) from python.eclass into the replacement,
> > > > constraining our choises a bit.
> > > >
> > > > Counter arguments?  To be clear, this is the path I strongly suggest
> > > > we take- if you can punch holes in the logic/arguments from above, I'd
> > > > definitely back down, but the use of PYTHON_TARGETS here feels like
> > > > we're setting ourselves up for unnecessary pain.  Keep in mind not
> > > > *all* of python.eclass notions/setup has to be chucked- we can
> > > > translate certain parts of it across to ease developer/user pains.
> > > >
> > > 
> > > Following up on this and the irc conversation I eavesdropped on earlier:
> > > 
> > > More recent versions of portage apparently filter USE_EXPAND
> > > variables, so we can't utilize the old python abi values in USE_PYTHON
> > > if we make that conversion.
> > 
> > Just a note; portage's behaviour here, per the norm, is more stupid 
> > than that.  EAPIs 0-4 are supposed to /not/ have that filtering, 
> > meaning portage broke it's own behaviour again.
> > 
> > I suspect we'll retroactively change EAPIs to compensate for this, but 
> > I don't know for a fact; the discussion for that will occur at 
> > https://bugs.gentoo.org/442830 .
> 
> I'm sorry if I miss something but don't EAPIs 0-4 define *no* behavior
> for USE_EXPAND variables? (something around don't use them)

You're reading that too literally.  The vars weren't supposed to be 
screwed with- ebuilds/eclasses from that era actually relied on access 
of that sort (that's just how it was back then- the problem of 
USE_EXPAND being potentially unbound in values wasn't yet fixed).

Either way, the wording /is/ such that the filtration should only be 
happening in EAPI5.

Bluntly; portage is being buggy here, and unfortunately has been for a 
while.   The spec/original approved behaviour is what I said- and 
portage in quite a few cases does exactly what I said.

However, if there is (package.)use.* that is involved for that 
USE_EXPAND, or the group intersects IUSE, then the filtering kicks in.  
Meaning that if someone right now went and added linguas_ja to the 
base profiles use.mask, minimally 100+ ebuilds would be busted (likely 
more; I did a simple/stupid scan that didn't account for eclasses).  
Remove that use.mask filtering, suddenly the full value is again 
exported into all ebuild envs (eapi0-5).

That's inconsistent behaviour, both for how eapi0-4 is supposed to 
work, and not matching eapi5 requirements since it only does partial 
filtering.

Either way, this idiocy explains the variable results; as for how the 
spec/portage will be rectified, hell if I know.

~harring