[gentoo-python] Package supporting single implementation only -- auxiliary variable solution

[gentoo-python] Package supporting single implementation only -- auxiliary variable solution

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4681 bytes --]

Hello,

I am still thinking what would be the best way of handling packages
which support being built for a single Python implementation only
(like packages embedding Python and not being boost).

A few shortcuts for the scope of this mail:
SIP - package supporting being built for a single implementation only
MIP - package supporting being built for multiple implementations

Now, the problem with SIP is that they can support only one Python
implementation at a time. I'd like to find a solution which:

1) would be the most user- and developer-friendly,

2) would be transparent and explicit (user has to know which
implementation is actually used, and be able to switch it),

3) would not require different dependency atoms for dependencies which
are SIP (i.e. the package just depends on another Python package,
not caring whether it is SIP or MIP).


So far, the solution of using REQUIRED_USE='^^ ( python_targets... )'
fulfilled 2) & 3). Sadly, it is hardly user-friendly because whenever
user has more than one Python implementation in PYTHON_TARGETS, he
needs to explicitly disable other flags for each package.

Additionally, this means that if SIP becomes a MIP, user is unaware
of that and still explicitly disables additional implementations.

I could live with that if not the fact that the default PYTHON_TARGETS
currently list two implementations. Therefore, not only a specific
group but all our users would be forced to switch flags like that.


Hopefully, there's one way we could achieve a bit of user-friendliness
for most of the cases. If we introduced an auxiliary variable, let's
call it PYTHON_SINGLE_TARGET for now, it could work.

The eclass used for SIP:

1) adds PYTHON_SINGLE_TARGET flags (following PYTHON_COMPAT) to IUSE,

2) adds REQUIRED_USE='^^ ( python_single_target... )',

3) adds REQUIRED_USE='python_single_target...? ( python_targets... )',

4) replaces PYTHON_USEDEP with:
       'python_targets...?,python_single_target...?(+)',

5) sets the variables based on PYTHON_SINGLE_TARGET in pkg_setup().

Therefore, users can explicitly use PYTHON_SINGLE_TARGET to choose
the target which would be user for SIP while keeping the complete
PYTHON_TARGETS for MIP. It's much more friendly.


Thanks to EAPI4 USE defaults, the ${PYTHON_USEDEP} works fine both
for MIP and SIP deps.

For a MIP dep, the PYTHON_TARGETS are used as the real check.
PYTHON_SINGLE_TARGET aren't there and they default to enabled,
so the check passes. The REQUIRED_USE ensures that PYTHON_TARGETS
contains PYTHON_SINGLE_TARGET.

For a SIP dep, the PYTHON_SINGLE_TARGET is available and becomes
relevant.

In the corner case when PYTHON_SINGLE_TARGET for package A specifies
an implementation which is not supported by package B,
the PYTHON_SINGLE_TARGET check succeeds (missing flag = enabled)
but the PYTHON_TARGETS check fails due to missing flag.

Of course, a MIP would not be allowed to depend on a SIP.


The profiles would specify a single default implementation
from PYTHON_TARGETS as a default for PYTHON_SINGLE_TARGET. This will
effectively hide the issue from most of our users.

Those who set PYTHON_TARGETS explicitly may need to adjust
the PYTHON_SINGLE_TARGET. Considering that it will be common to all
SIPs, it's much more friendly than switching per-package.


The only real issue unsolved is friendly solving of Py2/3 issues with
SIPs. If user has PYTHON_SINGLE_TARGET=python2_7, SIP package supporting
Python3 only (is there any?) would require explicit flag setting.

I believe that's a corner case. It could be solved if we invented some
kind of 'automatic USE defaults'. In that case, profiles wouldn't
specify a default but instead portage would pick one based on
REQUIRED_USE constraints and flag order.

In other words:
- user has PYTHON_TARGETS='python2_7 python3_2',
- package has PYTHON_COMPAT=( python{2_6,2_7} ),
-> per-package PYTHON_TARGETS becomes 'python2_7',
- REQUIRED_USE='^^ ( pst2_6 pst2_7 ) pst2_6? ( py2_6 ) ...'
-> portage tries pst2_6...
--> py2_6 not set, REQUIRED_USE unsatisfied;
-> portage tries pst2_7
--> py2_7 set, all ok.
---> PYTHON_SINGLE_TARGET=python2_7

But that's not really necessary right now, I think.


To sum it up quickly, the aux variable solution:

Advantages:
- painless for most of our users,
- explicit, transparent,
- simple and matching SIP->SIP and SIP->MIP deps.

Disadvantages:
- a bit redundant,
- does not solve the issue for py3-only SIP packages,
- may require explicit action when changing PYTHON_TARGETS.

What are your thoughts?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-python] Re: Package supporting single implementation only -- auxiliary variable solution

[Dirkjan Ochtman]

On Thu, Nov 15, 2012 at 10:44 PM, Michał Górny <mgorny@gentoo.org> wrote:
> Of course, a MIP would not be allowed to depend on a SIP.

How big of a problem would this be?

> To sum it up quickly, the aux variable solution:
>
> Advantages:
> - painless for most of our users,
> - explicit, transparent,
> - simple and matching SIP->SIP and SIP->MIP deps.
>
> Disadvantages:
> - a bit redundant,

Given the requirement for enabling multiple implementation in
PYTHON_TARGETS, which I feel strongly about, there has to be a choice
which one of the implementations will be target for SINGLE_TARGET
packages. Adding a variable for that, therefore, doesn't seem
redundant.

> - does not solve the issue for py3-only SIP packages,

I guess this might become a problem in the future. Once we have both
py2-only SIP packages and py3-only SIP packages, we have a bit of a
problem. Maybe we should think more about that.

> - may require explicit action when changing PYTHON_TARGETS.

Yeah, I don't see a problem with this. In particular, it seems
unlikely that many people will setup their system to exclude the
default SINGLE_TARGET impl (2.7) from their TARGETS.

Cheers,

Dirkjan

Re: [gentoo-python] Re: Package supporting single implementation only -- auxiliary variable solution

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 2338 bytes --]

On Mon, 19 Nov 2012 10:57:52 +0100
Dirkjan Ochtman <djc@gentoo.org> wrote:

> On Thu, Nov 15, 2012 at 10:44 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > Of course, a MIP would not be allowed to depend on a SIP.
> 
> How big of a problem would this be?

None at all. Just stating the obvious fact, in case anyone wanted
to try doing that or point out that my solution is not well though
out :).

> > To sum it up quickly, the aux variable solution:
> >
> > Advantages:
> > - painless for most of our users,
> > - explicit, transparent,
> > - simple and matching SIP->SIP and SIP->MIP deps.
> >
> > Disadvantages:
> > - a bit redundant,
> 
> Given the requirement for enabling multiple implementation in
> PYTHON_TARGETS, which I feel strongly about, there has to be a choice
> which one of the implementations will be target for SINGLE_TARGET
> packages. Adding a variable for that, therefore, doesn't seem
> redundant.

Well, some people could argue that if they set PYTHON_TARGETS
to a single value (like python3.2), they need to set an auxiliary
variable as well. But that's the lesser evil, I think.

> > - does not solve the issue for py3-only SIP packages,
> 
> I guess this might become a problem in the future. Once we have both
> py2-only SIP packages and py3-only SIP packages, we have a bit of a
> problem. Maybe we should think more about that.

Let's grep the tree then (and a few random overlays I'm using)...

I have used the following command:

$ find -name '*.ebuild' -exec grep -L SUPPORT_PYTHON_ABIS {} + \
  | xargs grep PYTHON_DEPEND

I have counted 1372 ebuilds not supporting multiple Python
implementations, in 648 packages.

Of those ebuilds:
- 4 ebuilds (1 pkg) support both Python 2 & 3:
  - media-gfx/hugin;
- 8 ebuilds (3 pkgs) require Python 3:
  - sys-power/upower with USE=test,
  - media-gfx/blender (3.2+),
  - net-irc/znc with USE=python.

Considering that's ~0.6% of all packages, I'd say we could just be lazy
and use package.use for them.

> > - may require explicit action when changing PYTHON_TARGETS.
> 
> Yeah, I don't see a problem with this. In particular, it seems
> unlikely that many people will setup their system to exclude the
> default SINGLE_TARGET impl (2.7) from their TARGETS.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]