[gentoo-python] USE flag dependencies on Python implementation

[gentoo-python] USE flag dependencies on Python implementation

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 4474 bytes --]

Hello,

The one of the worst problems which we are about to try to solve is how
packages should depend on particular features of Python implementation.
With three different implementations, this seems to become a bit
problematic.

First of all, I'd like to note that different Python implementations
have different USE flags (and sometimes they even change by version).
I've tried to find all the flags and put them in a nicely readable
matrix here:

http://wiki.gentoo.org/wiki/Python/Implementation_USE_flags

Sadly, jython is mostly black magic to me and a few other flags like
'ipv6' and 'threads' which can't be really deduced from deps as well.
I'd really appreciate if someone could fill the table completely.

The idea there is simple. If something is marked as 'flag', then
a particular feature is optional and enabled via the listed flag.
If it's '+', then it's always on. If it's '-', then it's not
implemented at all.

Looking at it now, there isn't yet a single flag which would be
portable to all Python implementations. That rises the question on how
packages should depend on features of the Python implementations.


Secondly, I'd like you to answer the following questions which will
determine which solutions are possible at all:

Let's assume than an ebuild has optional ncurses support via
IUSE=ncurses, and it requires ncurses module in Python. Not all Python
implementations support ncurses. Without the USE flag, the package is
portable to all implementations.

1. Now, the user has enabled all Python implementations,
and USE=ncurses. What should happen?

a) REQUIRED_USE bailing out -- user is explicitly forced to disable
some of the implementations locally or IUSE=ncurses; it is clear to him
what happened;

b) the package is built with implementations supporting ncurses only --
no explicit failure for user but then he is surprised that the package
wasn't actually built for all impls, and he doesn't even know what
happened. We could ewarn him, but that's a bit like reimplementing
REQUIRED_USE.

Feel free to suggest a c).

2. Now, the user has enabled Jython only (the impl. without ncurses)
and USE=ncurses. What should happen?

If you have chosen a) previously, REQUIRED_USE is perfectly fine here
as well.

If you have chosen b) instead, then what now? We are either installing
an empty package or having an inconsistent behavior.


Now, let's get to the actual possible solutions for new python eclasses.

1/ separate USE-dep variables for each implementation

This is the solution suggested by hasufell:

  PYTHON_USEDEP="ncurses?"
  PYPY_USEDEP="ncurses?"
  JYTHON_USEDEP= # err, what exactly here?

The advantage is that... errr... it works? Sadly, I'm a bit afraid it's
like throwing a bunch of screws on the developer and saying 'now figure
out what they may be useful for'.


2/ a single magical variable as in python.eclass

Well, it probably works... somehow. The more features we want, the more
code we have to write. And reinvent the wheel, and remember not to lose
compatibility when adding new features. I'd really like to avoid even
thinking about this.


3/ a single USE-dependency string common to all implementations

Something like in my proposition of python-r1.eclass.

Either:

  PYTHON_USEDEP='ncurses?'
  REQUIRED_USE='ncurses? ( !python_targets.... )'

or:

  PYTHON_USEDEP='ncurses?(-)'

The latter being probably incorrect wrt PMS.


4/ a virtuals for Python features

Instead of setting USE-dependency strings directly, invent a few
virtuals like:

  virtual/python-ncurses:
    python_targets_python2_7? ( dev-lang/python:2.7[ncurses] )
    ...

And then let ebuilds depend on them like:

  RDEPEND="virtual/python-ncurses[$PYTHON_USEDEP]"

This solution introduces magic similar to one in 2/ while keeping it
free of scary code. Not sure if it's really useful considering
the REQUIRED_USE problem.

We can make the virtuals add blockers for python implementations being
enabled without the feature. But I'm not sure if it's semantically
correct, i.e.:

  dev-python/foo[python_targets_jython2_5]:
    dev-java/jython:2.5
    virtual/python-ncurses[python_targets_jython2_5]:
      !dev-java/jython:2.5

This just looks wrong ;).

And it will require user to unnecessarily change flags on virtual
itself.


What are your opinions, ideas?

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

[gentoo-python] Re: USE flag dependencies on Python implementation

[Dirkjan Ochtman]

On Tue, Sep 25, 2012 at 11:59 PM, Michał Górny <mgorny@gentoo.org> wrote:
> What are your opinions, ideas?

Let's not solve it in the eclass, let's solve it in the implementation ebuilds.

I.e. cpython, pypy, jython gets all the same flags. If an
implementation unconditionally includes something that's not selected
(i.e. the system has -threads for PyPy), it gets ignored. If a flag is
required that the implementation doesn't support, the ebuild dies at
some early stage (somewhat similar to check-reqs like things).

I think you're massively overthinking this stuff.

Cheers,

Dirkjan

Re: [gentoo-python] Re: USE flag dependencies on Python implementation

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Wed, 26 Sep 2012 08:42:21 +0200
Dirkjan Ochtman <djc@gentoo.org> wrote:

> On Tue, Sep 25, 2012 at 11:59 PM, Michał Górny <mgorny@gentoo.org> wrote:
> > What are your opinions, ideas?
> 
> Let's not solve it in the eclass, let's solve it in the implementation ebuilds.
> 
> I.e. cpython, pypy, jython gets all the same flags. If an
> implementation unconditionally includes something that's not selected
> (i.e. the system has -threads for PyPy), it gets ignored. If a flag is
> required that the implementation doesn't support, the ebuild dies at
> some early stage (somewhat similar to check-reqs like things).
> 
> I think you're massively overthinking this stuff.

But then you're introducing a bunch of flags which do nothing. However,
if you mask them in the base profile, this may actually get acceptable.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-python] Re: USE flag dependencies on Python implementation

[Dirkjan Ochtman]

On Wed, Sep 26, 2012 at 8:46 AM, Michał Górny <mgorny@gentoo.org> wrote:
> But then you're introducing a bunch of flags which do nothing. However,
> if you mask them in the base profile, this may actually get acceptable.

I prefer having a few few fake flags over the complexity of your proposals.

Cheers,

Dirkjan