[gentoo-dev] The Python problem

[gentoo-dev] The Python problem

[Dirkjan Ochtman]

Hi guys,

I guess by now pretty much everyone knows that the python eclass is
rather complex, and that this poses some problems. This has also been
an important cause for the disagreements between Arfrever and some of
the other developers. Since it appears that Arfrever won't be
committing much code to gentoo-x86 in the near future, I'm trying to
figure out where we should go with the python.eclass. This email is an
attempt at figuring that out, plus eliciting ideas to come up with a
general framework that will also solve this for ruby and other similar
runtimes (while supporting some of the features that the current
python eclass has, but that ruby-ng doesn't have).

So I know a bunch of people have already looked at it, and I'd like to
know: what do you find better about the Ruby approach compared to the
Python approach? Is it just the size of python.eclass, or are there a
number of other issues?

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Pascal J. Bourguignon]

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: [gentoo-dev] The Python problem

[Pascal J. Bourguignon]

Dirkjan Ochtman <djc@gentoo.org> writes:

> I guess by now pretty much everyone knows that the python eclass is
> rather complex, and that this poses some problems. This has also been
> an important cause for the disagreements between Arfrever and some of
> the other developers. Since it appears that Arfrever won't be
> committing much code to gentoo-x86 in the near future, I'm trying to
> figure out where we should go with the python.eclass. This email is an
> attempt at figuring that out, plus eliciting ideas to come up with a
> general framework that will also solve this for ruby and other similar
> runtimes (while supporting some of the features that the current
> python eclass has, but that ruby-ng doesn't have).
>
> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?

Let's skip the Ruby step, and go directly to Common Lisp!

-- 
__Pascal Bourguignon__                     http://www.informatimago.com/
A bad day in () is better than a good day in {}.

Re: [gentoo-dev] The Python problem

[Fabian Groffen]

On 27-06-2011 14:28:34 +0200, Dirkjan Ochtman wrote:
> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?

Part of the eclass problem is IMO coding style.  Change it to use
normally-sized variable names, and to respect 80-columns width, and it
already becomes much more consumable.

The whole comparison of Python vs Ruby vs Java and perhaps the outlook
to Perl is kind of hard.  What the python eclass accomplishes is very
neat.  What probably started as a way to be able to have both python 2.x
and 3.x installed side by side (because the latter is not really
compatible with the former) got IMO out of hand by supporting any python
version to coexist with another.  I guess many people developing with
Python actually love this feature, and in itself, I believe it has use
that cannot be ignored any more now.  The way in which this feature was
implemented -- sometimes it more felt as pushed through the throat -- is
more of the rebelious kind than the smooth path for ebuild developers.

It would be nice when a similar technique could be implemented only
once, in a consistent way.  In a way, multilib-portage can be considered
equal to one of the objectives of the python (and ruby) eclass:
multiple times compiling and installing for different ABIs.

All in all, I don't fancy a rewrite from scratch, since it all works
at the moment, and doing so takes another large bit of work.  Instead,
aligning the work with others to create a similar approach in ebuilds
(for ebuild developers) would be favourable to me.  And perhaps that
should mean that variables which contain versions with some syntax that
specify ranges and more should just go, to be replaced by something
which is much less powerful in expressiveness, but much easier to
understand in the general picture, such as the USE-flags from the ruby
eclass.

Just my €0.02


-- 
Fabian Groffen
Gentoo on a different level

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Mon, Jun 27, 2011 at 15:08, Fabian Groffen <grobian@gentoo.org> wrote:
> On 27-06-2011 14:28:34 +0200, Dirkjan Ochtman wrote:
>> So I know a bunch of people have already looked at it, and I'd like to
>> know: what do you find better about the Ruby approach compared to the
>> Python approach? Is it just the size of python.eclass, or are there a
>> number of other issues?
>
> Part of the eclass problem is IMO coding style.  Change it to use
> normally-sized variable names, and to respect 80-columns width, and it
> already becomes much more consumable.

That seems like the somewhat more straightforward part.

> The whole comparison of Python vs Ruby vs Java and perhaps the outlook
> to Perl is kind of hard.  What the python eclass accomplishes is very
> neat.  What probably started as a way to be able to have both python 2.x
> and 3.x installed side by side (because the latter is not really
> compatible with the former) got IMO out of hand by supporting any python
> version to coexist with another.  I guess many people developing with
> Python actually love this feature, and in itself, I believe it has use
> that cannot be ignored any more now.  The way in which this feature was
> implemented -- sometimes it more felt as pushed through the throat -- is
> more of the rebelious kind than the smooth path for ebuild developers.

Right. I think the ability to have python packages simultaneously
installed in python2.6, python2.7 and pypy1.5, for example, is pretty
cool to have. And going from USE to RESTRICT (Ruby vs. Python, right
now) is probably more sensible in the Python ecosystem (i.e. where
something that runs on 2.6 is likely to also run on 2.7).

> It would be nice when a similar technique could be implemented only
> once, in a consistent way.  In a way, multilib-portage can be considered
> equal to one of the objectives of the python (and ruby) eclass:
> multiple times compiling and installing for different ABIs.

Yeah, but it'd be nice not to have to compile subversion three times
just because we want the python bindings installed in three different
Python environments.

> All in all, I don't fancy a rewrite from scratch, since it all works
> at the moment, and doing so takes another large bit of work.  Instead,
> aligning the work with others to create a similar approach in ebuilds
> (for ebuild developers) would be favourable to me.  And perhaps that
> should mean that variables which contain versions with some syntax that
> specify ranges and more should just go, to be replaced by something
> which is much less powerful in expressiveness, but much easier to
> understand in the general picture, such as the USE-flags from the ruby
> eclass.

That sounds very similar to what I think would be best.

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 991 bytes --]

On Monday, June 27, 2011 09:43:05 Dirkjan Ochtman wrote:
> On Mon, Jun 27, 2011 at 15:08, Fabian Groffen wrote:
> > It would be nice when a similar technique could be implemented only
> > once, in a consistent way.  In a way, multilib-portage can be considered
> > equal to one of the objectives of the python (and ruby) eclass:
> > multiple times compiling and installing for different ABIs.
> 
> Yeah, but it'd be nice not to have to compile subversion three times
> just because we want the python bindings installed in three different
> Python environments.

it'd be nice to not build glibc multiple times for diff multilibs, but it 
simply isnt possible.  either you want multilib support (or in this case 
multi-python version support), or you dont.  if this feature is something like 
(and it sounds like it is), then this is simply something that is accepted.

if you dont want multiple builds on your system, then dont install multiple 
versions of python.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] The Python problem

[Joshua Saddler]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

On Mon, 27 Jun 2011 16:49:23 -0400
Mike Frysinger <vapier@gentoo.org> wrote:
> if you dont want multiple builds on your system, then dont install
> multiple versions of python.
> -mike

This would be nice, but unfortunately the python maintainer forced
3.x on everyone, despite the fact that nothing uses it and no one
really wanted it made the default. So now it's shipped with all the
stage tarballs, in addition to 2.7. You've got it whether you want it
or not.

This is one of the things that needs to be rescinded, along with the
python eclass changes. Get everyone down to just one version
of python installed.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Tue, Jun 28, 2011 at 08:54, Joshua Saddler <nightmorph@gentoo.org> wrote:
> This would be nice, but unfortunately the python maintainer forced
> 3.x on everyone, despite the fact that nothing uses it and no one
> really wanted it made the default. So now it's shipped with all the
> stage tarballs, in addition to 2.7. You've got it whether you want it
> or not.
>
> This is one of the things that needs to be rescinded, along with the
> python eclass changes. Get everyone down to just one version
> of python installed.

It's just slotting like everything else. (There's an open bug for the stages.)

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Jorge Manuel B. S. Vicetto]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 28-06-2011 07:19, Dirkjan Ochtman wrote:
> On Tue, Jun 28, 2011 at 08:54, Joshua Saddler <nightmorph@gentoo.org> wrote:
>> This would be nice, but unfortunately the python maintainer forced
>> 3.x on everyone, despite the fact that nothing uses it and no one
>> really wanted it made the default. So now it's shipped with all the
>> stage tarballs, in addition to 2.7. You've got it whether you want it
>> or not.
>>
>> This is one of the things that needs to be rescinded, along with the
>> python eclass changes. Get everyone down to just one version
>> of python installed.
> 
> It's just slotting like everything else. (There's an open bug for the stages.)

Yes, but with slotting you allow different packages to pull in different
slots of python. Furthermore, when you slot a package and mark more than
one slot stable, you're saying that all the stable slots work and don't
"break" the tree.
About the stages bug (330361), as was discussed to exhaustion on the
bug, mls, IRC and many other locations, python3 isn't added to the
stages by the will or work of releng.

> Cheers,
> 
> Dirkjan
> 

- -- 
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / KDE / Elections / RelEng
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJOCb93AAoJEC8ZTXQF1qEPurcQAKj3i7pL22kd2EpmG2+tDdcg
us3xE2dQofp6hVkiSkz6LVQuyCEH9QV/jujSkwYJLVPDsGGyXzBIkUGA9iLc2iDQ
nJx2r2QqZe11CxOuyd15AomVsLDazDGq8mxkxIBzJl9l9xS5HINqL+D79HxiOsdE
oraOL/cr4tbjDLWgraLZMXM/QPUkhqXkQurgn1uF4U4TAkvjZessJwe2Csi6raUW
xhJ42HAF+Q+VpFuTlBJfJevNsXnyHR4y6qM4xFF5X6sIyPodz3CORfHBMAXrU/pX
MCsMSPRJQRIAc26tEf+V4WJuUsp+yaydejPnEiyZSnDBDmgeS7zgEhQRLAPf+//Y
sMEMqchTAkwS4gxyAyl6AoR3+DFlGZdcTmn2S+AbtQNH318Lt3yamGbwRUMpjUHX
p8YXxsjL6iigyIUaJt594W1Bc2gS6ktkQGUqERLvM0dYUSoQcZJCeWq7y+qO/3vF
uukqOnljgSEvucijeHUM3623ImgpoCw3tzw7UGz74PunqqqL37op7FxX55exyYWw
TjzDqFEPlnaNQNvD3E3dZdU+/KnUlmn9Jtbxv/o04unVfEFspGP9DeuZYUOolWAG
86LFsL9PeSGLhqgxWbFjMR4lmznilnlaaB+4MmEtMV3K3FGdsI/68dhGRNe+oEC7
8JTT1DDZpunk0coni/f3
=aUwd
-----END PGP SIGNATURE-----

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Tue, Jun 28, 2011 at 13:48, Jorge Manuel B. S. Vicetto
<jmbsvicetto@gentoo.org> wrote:
> Yes, but with slotting you allow different packages to pull in different
> slots of python. Furthermore, when you slot a package and mark more than
> one slot stable, you're saying that all the stable slots work and don't
> "break" the tree.

And that's true for Python AFAICT.

> About the stages bug (330361), as was discussed to exhaustion on the
> bug, mls, IRC and many other locations, python3 isn't added to the
> stages by the will or work of releng.

Right; I've also stated my opinion on this exhaustively in that bug.

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Thomas Sachau]

[-- Attachment #1: Type: text/plain, Size: 1953 bytes --]

Dirkjan Ochtman wrote:
> On Mon, Jun 27, 2011 at 15:08, Fabian Groffen <grobian@gentoo.org> wrote:
>> On 27-06-2011 14:28:34 +0200, Dirkjan Ochtman wrote:
>> It would be nice when a similar technique could be implemented only
>> once, in a consistent way.  In a way, multilib-portage can be considered
>> equal to one of the objectives of the python (and ruby) eclass:
>> multiple times compiling and installing for different ABIs.
> 
> Yeah, but it'd be nice not to have to compile subversion three times
> just because we want the python bindings installed in three different
> Python environments.

You wont be able to prevent this with a general solution, only with some specialized solution inside
the build, if you really want and need that.
Currently, if you want python bindings for three different python environments, you will have to
build everything for all three python environments, even if you only need a subset. Moving to a
similar system like ruby, in the long term maybe based on a general PM-implementation, would users
at least allow to select the targets per package and not just global like now.

> On Mon, Jun 27, 2011 at 17:53, Petteri Räty <betelgeuse@gentoo.org> wrote:
>> I like the ruby approach for the reason that it doesn't require users to
>> run update scripts like python-updater.
>
> Sure, but if that means the developers now have to bump every package
> in the tree when a new version of Python comes out, I'm not sure
> that's the best trade-off.

For multilib-portage, packages dont need to define the possible cross-compile targets, they have all
possible options in the USE flag list. Something similar could be done for this case:

Simplified: Define the range of supported python versions in the dependency section and allow the
eclass or PM to sort out the rest (you of course need a bit more, but this should be the only
required dependency/python-version related part).


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 379 bytes --]

Re: [gentoo-dev] The Python problem

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 1801 bytes --]

2011-06-27 22:57:05 Thomas Sachau napisał(a):
> Dirkjan Ochtman wrote:
> > On Mon, Jun 27, 2011 at 15:08, Fabian Groffen <grobian@gentoo.org> wrote:
> >> On 27-06-2011 14:28:34 +0200, Dirkjan Ochtman wrote:
> >> It would be nice when a similar technique could be implemented only
> >> once, in a consistent way.  In a way, multilib-portage can be considered
> >> equal to one of the objectives of the python (and ruby) eclass:
> >> multiple times compiling and installing for different ABIs.
> > 
> > Yeah, but it'd be nice not to have to compile subversion three times
> > just because we want the python bindings installed in three different
> > Python environments.
> 
> You wont be able to prevent this with a general solution, only with some specialized solution inside
> the build, if you really want and need that.
> Currently, if you want python bindings for three different python environments, you will have to
> build everything for all three python environments, even if you only need a subset.

Building everything for all Python ABIs is not required in case of majority of packages, which
optionally install Python bindings. Makefiles of many packages (including dev-vcs/subversion)
provide separate targets for building/installation of core libraries and various bindings.

Example:

src_compile() {
    emake || die

    if use python; then
        python_copy_sources bindings/python
        build_python_bindings() {
            emake \
                PYTHON_INCLUDES="$(python_get_includedir)" \
                PYTHON_SITE_PACKAGES="$(python_get_sitedir)" \
                python_bindings
        }
        python_execute_function -s --source-dir bindings/python build_python_bindings
    fi
}


-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] The Python problem

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 440 bytes --]

On 27.06.2011 15:28, Dirkjan Ochtman wrote:
> 
> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?
> 

I like the ruby approach for the reason that it doesn't require users to
run update scripts like python-updater.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Mon, Jun 27, 2011 at 17:53, Petteri Räty <betelgeuse@gentoo.org> wrote:
> I like the ruby approach for the reason that it doesn't require users to
> run update scripts like python-updater.

Sure, but if that means the developers now have to bump every package
in the tree when a new version of Python comes out, I'm not sure
that's the best trade-off.

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Petteri Räty]

[-- Attachment #1: Type: text/plain, Size: 732 bytes --]

On 27.06.2011 19:00, Dirkjan Ochtman wrote:
> On Mon, Jun 27, 2011 at 17:53, Petteri Räty <betelgeuse@gentoo.org> wrote:
>> I like the ruby approach for the reason that it doesn't require users to
>> run update scripts like python-updater.
> 
> Sure, but if that means the developers now have to bump every package
> in the tree when a new version of Python comes out, I'm not sure
> that's the best trade-off.
> 

And why can't this be handled by the eclass? If the ebuild is able to
specify it supports >=3.0 meaning it's expected that future versions
work then the eclass can make the new values available through IUSE when
new python versions are out and ebuilds don't require any changes.

Regards,
Petteri


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 900 bytes --]

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Mon, Jun 27, 2011 at 22:46, Petteri Räty <betelgeuse@gentoo.org> wrote:
>> Sure, but if that means the developers now have to bump every package
>> in the tree when a new version of Python comes out, I'm not sure
>> that's the best trade-off.
>
> And why can't this be handled by the eclass? If the ebuild is able to
> specify it supports >=3.0 meaning it's expected that future versions
> work then the eclass can make the new values available through IUSE when
> new python versions are out and ebuilds don't require any changes.

I'd be happy to get that to work, it seems like a good solution.

Cheers,

Dirkjan

[gentoo-dev] REMOVE

[Stelios Boulios]

Re: [gentoo-dev] The Python problem

[Mike Frysinger]

[-- Attachment #1: Type: Text/Plain, Size: 688 bytes --]

On Monday, June 27, 2011 12:00:19 Dirkjan Ochtman wrote:
> On Mon, Jun 27, 2011 at 17:53, Petteri Räty <betelgeuse@gentoo.org> wrote:
> > I like the ruby approach for the reason that it doesn't require users to
> > run update scripts like python-updater.
> 
> Sure, but if that means the developers now have to bump every package
> in the tree when a new version of Python comes out, I'm not sure
> that's the best trade-off.

if that's the requirement, i'd lean towards the python route we have today.  
especially because python-updater will hit all necessary packages where-as 
ruby will often not as that requires you to do `emerge -u <all installed 
packages>`.
-mike

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [gentoo-dev] The Python problem

[Hans de Graaff]

[-- Attachment #1: Type: text/plain, Size: 1363 bytes --]

On Mon, 2011-06-27 at 16:52 -0400, Mike Frysinger wrote:
> On Monday, June 27, 2011 12:00:19 Dirkjan Ochtman wrote:
> > On Mon, Jun 27, 2011 at 17:53, Petteri Räty <betelgeuse@gentoo.org> wrote:
> > > I like the ruby approach for the reason that it doesn't require users to
> > > run update scripts like python-updater.
> > 
> > Sure, but if that means the developers now have to bump every package
> > in the tree when a new version of Python comes out, I'm not sure
> > that's the best trade-off.
> 
> if that's the requirement, i'd lean towards the python route we have today.  
> especially because python-updater will hit all necessary packages where-as 
> ruby will often not as that requires you to do `emerge -u <all installed 
> packages>`.

This approach works for ruby because we have multiple implementations
(e.g. jruby, hopefully rubinius soon) rather than just multiple
versions, and because major version updates that break compatibility are
rare. Bumping happens more often than new versions or implementations
come out, and we can't expect implementations to be fully compatible.
Note that minor versions (ruby 1.8.6 vs ruby 1.8.7) are usually backward
compatible, so we can just swap these in within the slot.

I don't really see how a ruby-updater tool would work, so the approach
we've taken works for ruby.

Hans

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 230 bytes --]

Re: [gentoo-dev] The Python problem

[Benedikt Böhm]

On Mon, Jun 27, 2011 at 2:28 PM, Dirkjan Ochtman <djc@gentoo.org> wrote:
> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?

the way python applications are built currently renders all binary
packages useless, since portage does not know which version of python
it was built against. the explicit selection with RUBY_TARGETS or
PHP_TARGETS solves this problem at the small cost of commiting new
values to these variables from time to time.

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Mon, Jun 27, 2011 at 20:23, Benedikt Böhm <hollow@gentoo.org> wrote:
> the way python applications are built currently renders all binary
> packages useless, since portage does not know which version of python
> it was built against. the explicit selection with RUBY_TARGETS or
> PHP_TARGETS solves this problem at the small cost of commiting new
> values to these variables from time to time.

I don't think updating all Python packages every two years is a small
cost, particularly when a large percentage of those packages will just
work on the new version.

(Another problem with the current Ruby approach is that it's
impossible to support unstable interpreters or versions from in stable
package versions, such that Ruby packages now often have multiple
ebuild revisions for stable and unstable interpreters.)

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

On Mon, 27 Jun 2011 14:28:34 +0200
Dirkjan Ochtman <djc@gentoo.org> wrote:

> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?

Working targets. USE_PYTHON is junk. What python.eclass does now with
ABIs is a PITA, and requires manually providing a lot of redudant
information (namely, RESTRICT_PYTHON_ABIS).

Right now, each ebuild has to provide the same information in
PYTHON_DEPEND and RESTRICT_PYTHON_ABIS. Moreover, if an ebuild
supports, say, py3 and its dependency doesn't, the ebuild has to
restrict py3 too.

I'd like to see that fixed somehow. I'd like to set a single supported
Python version information in an ebuild, and let the dependency
resolver handle everything else.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] The Python problem

[Dirkjan Ochtman]

On Mon, Jun 27, 2011 at 21:31, Michał Górny <mgorny@gentoo.org> wrote:
> Working targets. USE_PYTHON is junk. What python.eclass does now with
> ABIs is a PITA, and requires manually providing a lot of redudant
> information (namely, RESTRICT_PYTHON_ABIS).

Please clarify *why* it is a PITA, and what information is redundant.

> Right now, each ebuild has to provide the same information in
> PYTHON_DEPEND and RESTRICT_PYTHON_ABIS. Moreover, if an ebuild
> supports, say, py3 and its dependency doesn't, the ebuild has to
> restrict py3 too.

I'll grant you that PYTHON_DEPEND and RESTRICT_PYTHON_ABIS seem
redundant, we could probably fix that.

> I'd like to see that fixed somehow. I'd like to set a single supported
> Python version information in an ebuild, and let the dependency
> resolver handle everything else.

That would be bad; it's common and useful to have packages installed
in both 2.6 and 2.7 (and even 3.1).

Cheers,

Dirkjan

Re: [gentoo-dev] The Python problem

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1437 bytes --]

On Tue, 28 Jun 2011 10:04:58 +0200
Dirkjan Ochtman <djc@gentoo.org> wrote:

> On Mon, Jun 27, 2011 at 21:31, Michał Górny <mgorny@gentoo.org> wrote:
> > Working targets. USE_PYTHON is junk. What python.eclass does now
> > with ABIs is a PITA, and requires manually providing a lot of
> > redudant information (namely, RESTRICT_PYTHON_ABIS).
> 
> Please clarify *why* it is a PITA, and what information is redundant.

Let's put it like that. I have package foo which support py2 and py3.
It depends on package bar which once supported py3 but the newest
version is known to be broken with py3, and thus breaks building my
package as well.

And right now there's no real way to handle that dependency. With
SLOT-USEdeps I could do 'bar[py3=]'. With current state of Python
eclass I have to restrict ABIs to match newer version and/or restrict
dependency versions which is awfully redundant.

> > I'd like to see that fixed somehow. I'd like to set a single
> > supported Python version information in an ebuild, and let the
> > dependency resolver handle everything else.
> 
> That would be bad; it's common and useful to have packages installed
> in both 2.6 and 2.7 (and even 3.1).

I think you misunderstood me. I mean generating the whole dependency
tree from support Python versions. Like dropping version info from
PYTHON_DEPEND in favor of RESTRICT_PYTHON_ABIS.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] The Python problem

[Jesus Rivero (Neurogeek)]

On Mon, Jun 27, 2011 at 7:58 AM, Dirkjan Ochtman <djc@gentoo.org> wrote:
> Hi guys,
[...]
> So I know a bunch of people have already looked at it, and I'd like to
> know: what do you find better about the Ruby approach compared to the
> Python approach? Is it just the size of python.eclass, or are there a
> number of other issues?
>
> Cheers,
>
> Dirkjan
>
>

Let´s start by agreeing with the complexity of the Python eclass. And
a heavy change right now, is going to be painful to say the least
because a great deal of packages have been adapted to work the way the
eclass works right now.

I agree with Grobian in what he said about the coding style of the
python.eclass, and would love to see that "fixed". I also agree with
the redundancy of some variables and I think just using PYTHON_DEPEND
should be sufficient to let the PM know what should be done with a
specific package.

With python-updater, well, some time ago Ali Polatel implemented some
approaches to get rid of python-updater, by exporting some variable in
ebuilds that needed to be rebuilt when new python versions were
installed. I dont recall what happened with that, but we could think
of some ways to finally get rid of python-updater.

WRT python3, I too believe it is important to have it stable. Thing
is, for better or worse, upstream have it stable, and py2 and py3 are
going to co-exist for a long time. Thing is, we should really figure
out a way to have it stop messing things up. I believe with a proper
dependency handling in the eclass, we could get this to work.

As I said it already, we could start doing things simpler in the
current python.eclass and maybe that would attract some devs to help
out with it, as they find it more comfortable to work with.

Best regards,

-- 
Jesus Rivero (Neurogeek)
Gentoo Developer

Re: [gentoo-dev] The Python problem

[Michał Górny]

[-- Attachment #1: Type: text/plain, Size: 1276 bytes --]

On Tue, 28 Jun 2011 20:03:34 -0430
"Jesus Rivero (Neurogeek)" <neurogeek@gentoo.org> wrote:

> On Mon, Jun 27, 2011 at 7:58 AM, Dirkjan Ochtman <djc@gentoo.org>
> wrote:
> > Hi guys,
> [...]
> > So I know a bunch of people have already looked at it, and I'd like
> > to know: what do you find better about the Ruby approach compared
> > to the Python approach? Is it just the size of python.eclass, or
> > are there a number of other issues?
> 
> Let´s start by agreeing with the complexity of the Python eclass. And
> a heavy change right now, is going to be painful to say the least
> because a great deal of packages have been adapted to work the way the
> eclass works right now.
> 
> [...]
> 
> As I said it already, we could start doing things simpler in the
> current python.eclass and maybe that would attract some devs to help
> out with it, as they find it more comfortable to work with.

I think it would be better to simply start from scratch with
a clean python-2.eclass. Instead of adding new features and another lot
of conditionals for EAPI=4, just make that code a part of new eclass.

And remember, the more complex code is, the more painful it becomes for
things like metadata generation.

-- 
Best regards,
Michał Górny

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 316 bytes --]

Re: [gentoo-dev] The Python problem

[Andreas K. Huettel]

Am Mittwoch, 29. Juni 2011, 06:34:52 schrieb Michał Górny:
>
> > As I said it already, we could start doing things simpler in the
> > current python.eclass and maybe that would attract some devs to help
> > out with it, as they find it more comfortable to work with.
> 
> I think it would be better to simply start from scratch with
> a clean python-2.eclass. Instead of adding new features and another lot
> of conditionals for EAPI=4, just make that code a part of new eclass.
> 

Ack. The cleanest way would definitely be to start from scratch, and provide a 
looooong transition period. Please please make things similar compared to the 
other language eclasses, though...

-- 
Andreas K. Huettel (dilfridge)
Gentoo Linux developer
kde, sci, arm, tex

Re: [gentoo-dev] The Python problem

[Pacho Ramos]

[-- Attachment #1: Type: text/plain, Size: 907 bytes --]

El mié, 29-06-2011 a las 09:18 +0200, Andreas K. Huettel escribió:
> Am Mittwoch, 29. Juni 2011, 06:34:52 schrieb Michał Górny:
> >
> > > As I said it already, we could start doing things simpler in the
> > > current python.eclass and maybe that would attract some devs to help
> > > out with it, as they find it more comfortable to work with.
> > 
> > I think it would be better to simply start from scratch with
> > a clean python-2.eclass. Instead of adding new features and another lot
> > of conditionals for EAPI=4, just make that code a part of new eclass.
> > 
> 
> Ack. The cleanest way would definitely be to start from scratch, and provide a 
> looooong transition period. Please please make things similar compared to the 
> other language eclasses, though...
> 

Arfrever added support for eapi4 in python-overlay, maybe trying to move
it to the tree would be faster :-/

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [gentoo-dev] The Python problem

[Arfrever Frehtes Taifersar Arahesis]

[-- Attachment #1: Type: Text/Plain, Size: 674 bytes --]

2011-06-29 02:33:34 Jesus Rivero (Neurogeek) napisał(a):
> With python-updater, well, some time ago Ali Polatel implemented some
> approaches to get rid of python-updater, by exporting some variable in
> ebuilds that needed to be rebuilt when new python versions were
> installed. I dont recall what happened with that, but we could think
> of some ways to finally get rid of python-updater.

He added python_need_rebuild() function, which sets PYTHON_NEED_REBUILD variable, which is
checked by python-updater, so that python-updater can avoid checking CONTENTS files in VDB
and calling scanelf on all installed files.

-- 
Arfrever Frehtes Taifersar Arahesis

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 836 bytes --]